few issues, mostly ipv6

Once again, probably more of a discussion thread from me
ipv6 issues in domain check
In the meantime I've activated ipv6 on my VPS. Since then , currently, I have following errors on the Domain-> Check DNS page:
TLS
`SMTP connection with STARTTLS to MX hostname "mail.example.com." IP 2a[redacted]: dial tcp [2a[redacted]]:25: i/o timeout

Having learned from our previous discussion, I've checked connectivity using "openssl s_client -connect" , to the ipv6 IP, both to port 465 and to port 25
- port 465: looks all OK to me , I have the 220 smtp response
- port 25: it connects but of course fails
- port 25 w/SMARTTLS: I figured out I need to use following connection string: "openssl s_client -connect [ipaddress]:25 -starttls smtp"
Using either the ipv4 or ipv6 address I get bunch of text related to SSL (looking OK to me), no difference seemingly (I've used 3 variants: domainname, ipv4 address, ipv6 address)
=> so, my question is, why the error message in "Check DNS"
Despite this success, it seems, connection by mail clients doesn't work on ipv 6 - see further below
TLS i/o timeout for both Autoconf and Autodiscover
Still on the Check DNS page, I have this error, for both categories
TLS connection to hostname "autoconfig.example.com", IP "2a01:[redacted, ipv6]": dial tcp [2a01:[redacted]]:443: i/o timeout
The error message refers to DNS records to set up: it's all set up correctly
errors in log connecting ipv6
When I have a look at my log, I see various cases connecting to ipv4 (apparently ok), and also to ipv6 - always timing out.
Example:
mox-mail-mox-1  | l=info m="new connection" pkg=imapserver remote=[2001:[redacted-home ipv6]]:59106 local=[2a01:[redacted, mox-ipv6]]:993 tls=true listener=public cid=18b843f4327 delta="320.433µs"
mox-mail-mox-1  | l=info m="imap command ioerror" err="reading line from remote: read tcp [2a01:[redacted, mox-ipv6]]:993->[2001:[redacted-home ipv6]]:59106: read: connection reset by peer (fatal io error)" pkg=imapserver cmd= duration=1.147467805s cid=18b843f4327 delta=1.373511795s<EMAIL_ADDRESS>mox-mail-mox-1  | l=info m="connection closed" err="reading line from remote: read tcp [2a01:[redacted, mox-ipv6]]:993->[2001:[redacted-home ipv6]]:59106: read: connection reset by peer (fatal io error)" pkg=imapserver cid=18b843f4327 delta="336.22µs"<EMAIL_ADDRESS>
So, it's clear, something is wrong with ipv6. (On the other hand, I'm also not experienced in ipv6, just "dabbling around" in all)
I think, the two issues above are somehow related
long time until closing connection sometimes
I've also noticed that the duration of the transaction shows up in the log:  in some cases the time until closing the connections seems to be extremely long. This affects not only ipv6, but surprisingly ipv4 connections, as well
Example:
mox-mail-mox-1  | l=info m="new connection" pkg=imapserver remote=[redacted, ipv4 home]:50280 local=[redacted, ipv4, mox]:993 tls=true listener=public cid=18b7ed55dc2 delta="252.838µs"
mox-mail-mox-1  | l=info m="connection closed" pkg=imapserver cid=18b7ed55dc2 delta=9m5.19310387s<EMAIL_ADDRESS>
Longest duration of transaction I've seen in the log was about 30m(!). Even though it doesn't disturb now, I'm wondering, is this normal?
Please feel free to let me know if these issues should be split - or if I should choose another way of discussing them.

ipv6 issues in domain check
TLS i/o timeout for both Autoconf and Autodiscover

are you running the openssl command on the VPS, or are you connecting from an external machine?
and is mox running in docker?
a timeout error often indicates there is a firewall completely dropping traffic.
ideally, you would run the nc and openssl commands from the same place (container, vps) that mox is running. if you get a different result from those commands than from mox, we would have to dig deeper in mox. otherwise the most likely cause would be a firewall.

errors in log connecting ipv6

there were no logs snipped out in between those three lines?
the errors may seem bad ("connection reset by peer (fatal io error)"), but the last two log lines are normal for connections that are being closed.
if there was nothing between the first line and last two lines, that would be strange: it means the connection was set up, but then nothing happened. connections where nothing happens after the initial setup sometimes indicates that one side is expecting tls and the other isn't. some mail clients (perhaps even most!) pick the wrong settings by default...

long time until closing connection sometimes

for the IMAP server, this is normal behaviour. the connections are often long-lived: a mail client connects, fetches the latest emails, and runs the imap IDLE command. it blocks until a new message arrives (or other changes happen to the mailbox). mail clients may close and reopen connections periodically. but a connection could also live for a very long time.
for smtpserver long connections would be unexpected. you normally just connect to smtp to send a message, then close the connection. but the logs only show imapserver.

on ipv6 in domain check

are you running the openssl command on the VPS, or are you connecting from an external machine?
and is mox running in docker?

I was running the openssl command from an external machine.
And yes, mox is running in docker (using docker compose, to be exact)

a timeout error often indicates there is a firewall completely dropping traffic.
based on my understanding that would not be the case. I have ufw running, but all necessary ports are allowed (globally).

I also deactivated ufw completely for testing. Based on your hints I tried testing with openssl from within the VPS -> and indeed, in that case it doesn't work (so, would mean, not related to mox?)
I also realized, ipv6 wasn't activated in docker: now I tried to activate it - honestly, didn't help
From what I understand currently, the issue is probably related to the combination of docker and ipv6 - it may be too much (for me), but I'm still looking and trying further
errors in log

there were no logs snipped out in between those three lines?

I believe these were the only lines for the specific transaction (cid): of course much else happening inbetween
What I've realized unfortunately in the meantime: it doesn't only affect ipv6, but also sometimes ipv4 connections
Mostly I use mox from several Android devices, set up various clients (K-9 Mail and Fairemail)
Here few other logs (again, only for the specific cid, but all transactions for that one):
Ex1 - ipv4:
mox-mail-mox-1  | l=info m="new connection" pkg=imapserver remote=157.[home ipv4]:38764 local=45.[mox ipv4]:993 tls=true listener=public cid=18b843f44e4 delta="316.216µs"
mox-mail-mox-1  | l=info m="imap command ioerror" err="reading line from remote: read tcp 45.[mox ipv4]:993->157.[home ipv4]:38764: i/o timeout (fatal io error)" pkg=imapserver cmd= duration=30m0.000402325s cid=18b843f44e4 delta=30m0.936387233s<EMAIL_ADDRESS>mox-mail-mox-1  | l=info m="connection closed" err="reading line from remote: read tcp 45.[mox ipv4]:993->157.[home ipv4]:38764: i/o timeout (fatal io error)" pkg=imapserver cid=18b843f44e4 delta="670.516µs"<EMAIL_ADDRESS>
Ex2 - ipv6:
mox-mail-mox-1  | l=info m="new connection" pkg=imapserver remote=[2001:[home ipv6]]:36228 local=[2a01:[mox ipv6]]:993 tls=true listener=public cid=18b843f450e delta="89.601µs"
mox-mail-mox-1  | l=info m="imap command ioerror" err="reading line from remote: read tcp [2a01:[mox ipv6]]:993->[2001:[home ipv6]]:36228: i/o timeout (fatal io error)" pkg=imapserver cmd= duration=30m0.034121512s cid=18b843f450e delta=31m1.626627362s<EMAIL_ADDRESS>mox-mail-mox-1  | l=info m="connection closed" err="reading line from remote: read tcp [2a01:[mox ipv6]]:993->[2001:[home ipv6]]:36228: i/o timeout (fatal io error)" pkg=imapserver cid=18b843f450e delta=1.827707ms<EMAIL_ADDRESS>
This is a recurring pattern - 3 lines, not much else.

if there was nothing between the first line and last two lines, that would be strange: it means the connection was set up, but then nothing happened. connections where nothing happens after the initial setup sometimes indicates that one side is expecting tls and the other isn't. some mail clients (perhaps even most!) pick the wrong settings by default...

hmm, thanks for the explanation. I have a feeling (but no specific proof) that it may be affecting K-9 Mail more than  Fairemail
also, just now checked on desktop, Thunderbird, and it seems to be affected as well unfortunately
finally

for the IMAP server, this is normal behaviour.

great, that's settled then :)
I looked into it more: I can see long transactions time only for imapserver, not for smtp
I realize, these may all be basic questions, and not always related to mox specifically. I'm only encouraged by the fact that you want more real life experience, from real use of mox.

ipv6 issues
It indeed looks like something with docker and ipv6 was (is) not working: I decided to move away from docker (for the sake of testing) and run mox directly => and it seems to work, also with ipv6
l=info m="new connection" pkg=smtpserver remote=[2607:f8b0:4864:20::a31]:60525 local=[2a01:[mox ipv6]]:25 submission=false tls=false listener=public cid=18b999e61a0 delta="389.068µs"
l=info m="reputation analyzed"<EMAIL_ADDRESS><EMAIL_ADDRESS>pkg=smtpserver conclusive=true isjunk=false method=msgfromfull cid=18b999e61a0 delta=739.213048ms
l=info m="incoming message delivered"<EMAIL_ADDRESS><EMAIL_ADDRESS>pkg=smtpserver reason=msgfromfull<EMAIL_ADDRESS>cid=18b999e61a0 delta=19.832726ms
l=info m="connection closed" pkg=smtpserver cid=18b999e61a0 delta=103.594964ms
l=info m="new connection" pkg=imapserver remote=[2001:home ipv6]:49782 local=[2a01:[mox ipv6]]:993 tls=true listener=public cid=18b999e61a1 delta="307.796µs"
l=info m="connection closed" pkg=imapserver cid=18b999e619f delta=6m3.497069926s<EMAIL_ADDRESS>
So, this shows, both smtpserver (for receiving from gmail) and imapserver (from my local ipv6) work well - without docker.
For now I'm running sudo mox serve , in the foreground (which will not be a good solution long term)... I still need to figure out, how to enable the systemd service
I'll also do some further testing with test levels, than I'll see, what's the situation with the timeouts and similar

Well, once again it turns out it was kind of a "user error"
I found a hint somewhere regarding ipv6 and ensuring that the /etc/sysctl.conf file does not have any values in place that might be disabling IPv6 connectivity. And indeed, there were:
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1

I've changed all values to "0",  lo and behold, ipv6 is working (both in docker, and if running directly). ((Even running on host directly, I discovered, I had problems on the domain check page - and of course, all of this is solved now there as well)). So, even though, I didn't set this setting, it was a basic system setting
So, now I'm able to run mox in 2 ways:

in docker (docker compose) - version 0.0.7
directly on host - very latest version, built from source locally

I have however new , probably easy questions
installing systemd service
I think I might prefer to run mox directly (instead of docker) for latest improvements, like DNSSEC and DANE.
when running it this way, I currently use screen (to run in the background): seems to run fine for now, however I believe the more preferred way for services to run is via systemd services.
On installation mox created a mox.services file, which I somehow just can't activate.
Copied over to /etc/systemd (sudo cp mox.services /etc/systemd) and enabled and started it - however, systemd status mox always show the service is failing / constantly restarting:
mox@mail:~$ sudo systemctl status mox
● mox.service - mox mail server
     Loaded: loaded (/etc/systemd/system/mox.service; enabled; vendor preset: enabled)
     Active: activating (auto-restart) (Result: exit-code) since Sat 2023-11-04 14:39:16 UTC; 4s ago
    Process: 3649 ExecStart=/home/mox/mox serve (code=exited, status=203/EXEC)
   Main PID: 3649 (code=exited, status=203/EXEC)
        CPU: 78ms

Any ideas what I could be doing wrong?
I was checking permissions, it has same permissions as other service files in /etc/systemd/system:
-rw-r--r--  1 root root 1325 Nov  4 06:31 mox.service
Hopefully last question for a while

The error is likely about /home/mox/mox not having execute permissions. I suppose running it as root (with "mox serve") does work? When mox starts through systemd, it also starts as root, then drops privileges.
A quick search leads me to https://unix.stackexchange.com/questions/472950/systemd-status-203-exec-error-when-creating-new-service
One tip stands out: it could be selinux. Whenever something that obviously should work, does not work on linux, and selinux is enabled, that's a prime suspect.
You can probably get a few more log lines with: journalctl -n 100 -u mox.

Once more, thanks for your hints: it helped, and is now solved. It was actually a path issue. (after building it, I moved mox to /usr/local/bin, not to /home/mox for some reason  - however this was of course not changed in mox.service. Anyways, now working well.
I go back to playing with it. My next plans related to mox:

set up properly DNSSEC and DANE (w/authoritative dns running on same server)
connect my mox server by either tailscale (later maybe headscale?) or wireguard to the rest of my network, and bind localhost to it, so I'm able to use webmail and admin pages easier
... and if everything works well, moving over my main domain email to mox

Normally, I don't think these should be too difficult ... however you never know.
Thanks again for your patience so far :)

set up properly DNSSEC and DANE (w/authoritative dns running on same server)

i recently started running dns myself too. you should have at least 2 machines doing dns, for redundancy. perhaps it's even required when updating the name servers at the registrar.
i settled on using bind: it has automatically dnssec signing and key management and is relatively straight-forward, the signed zones can easily be automatically propagated to a second bind instance. for alternatives, i also looked at nsd, but i think it needs separate tools (with cronjobs) to generate signed zones. powerdns should be automatic, but also seems a bit complex and too much for my needs. knot was also an option, but seems younger and less well-known.
when migrating, think about the TTL's of your records, including NS records at your current DNS operator and at the TLD.

Hi Mechiel!
thanks for he hints, I've started running it already for few weeks for this domain: I use knot-dns (authoritative), and it's working well.
There was only DNSSEC and DANE settings remaining, but have been able to solve and set up correctly (actually, the issue was more related to recursive DNS, I've installed unbound, as recommended on Domain Check) => Domain Check shows all green :)
What remains still open for now however: those i/o errors: I'll check if the changes in the meantime have helped + will play more with log levels, to see if there's
Also, one more idea came to me: since now I run mox on host, yet would like to use docker maybe for other things, I will need to dig into mox webserver (e.g. connecting host webserver with docker containers for reverse proxying)
Will update here (if I have progress).


Adding logout mechanism

Superb and really nice framework, I wish I would have find out this earlier, by the way can anyone enlighten me about which is the best way to implement logout mechanism ?

With Instagram I have still the Problem that the user stays logged in through cookies or data in the cache.
For now I have added the code from @pdrlps to fix this.


Latest Holoviews master seems to have a conflict with networkx

the networkx example seems not working any more with latest Holoviews github master:
https://github.com/ioam/holoviews/blob/master/examples/user_guide/Network_Graphs.ipynb
import numpy as np
import pandas as pd
import holoviews as hv
import networkx as nx

hv.extension('bokeh')

%opts Graph [width=400 height=400]

gives:
---------------------------------------------------------------------------
SystemError                               Traceback (most recent call last)
<ipython-input-1-7f1b468611bc> in <module>()
      2 import pandas as pd
      3 import holoviews as hv
----> 4 import networkx as nx
      5
      6 hv.extension('bokeh')

C:\WinPython\basedir36\buildQt5\winpython-64bit-3.6.x.1\python-3.6.2.amd64\lib\site-packages\networkx\__init__.py in <module>()
     72
     73 # Release data
---> 74 from networkx import release
     75
     76 __author__ = '%s <%s>\n%s <%s>\n%s <%s>' % \

C:\WinPython\basedir36\buildQt5\winpython-64bit-3.6.x.1\python-3.6.2.amd64\lib\site-packages\networkx\release.py in <module>()
    223     'Topic :: Scientific/Engineering :: Physics']
    224
--> 225 date, date_info, version, version_info, vcs_info = get_info()
    226
    227 if __name__ == '__main__':

C:\WinPython\basedir36\buildQt5\winpython-64bit-3.6.x.1\python-3.6.2.amd64\lib\site-packages\networkx\release.py in get_info(dynamic)
    155         # sys.path.insert(0, basedir)
    156         try:
--> 157             from version import date, date_info, version, version_info, vcs_info
    158         except ImportError:
    159             import_failed = True

C:\WinPython\basedir36\buildQt5\winpython-64bit-3.6.x.1\python-3.6.2.amd64\lib\site-packages\param\version.py in init param.version (param\version.c:7238)()
     81 __author__ = 'Jean-Luc Stevens'
     82
---> 83 import os, subprocess
     84
     85 def run_cmd(args, cwd=None):

SystemError: Parent module 'version' not loaded, cannot perform relative import

it goes better if I invert  the order of imports, but it's not that good in general case:
import numpy as np
import pandas as pd
import networkx as nx
import holoviews as hv

hv.extension('bokeh')

%opts Graph [width=400 height=400]

@stonebig Thanks for reporting this, I'll try to figure out what's going on.

@jlstevens Any idea what's going on here? from version import ... in networkx seems to be calling into param.version.

I use networkx-2.0.0, 4 days old release.

Any idea what's going on here? from version import ... in networkx seems to be calling into param.version.

That is very odd behavior and I have no idea what is going on. There is no reason for networkx to touch our __version__ variables in any way.

I don't have it on a networkx-2.0 / python-3.5 / bokeh-0.12.9 / day before holoviews, not cython-0.27.0.  so supects could be:

a cython-0.27.0 bug,
a python-3.6 bug,
or something in last 48 hours of holoviews,
or something completly different.

hum, it bugs on idle too...

this "from version import version" may favorise a strange bug ?

I think this suffice to trigger the issue/
import holoviews as hv
from version import version

Param does have a version.py but I don't see anywhere where it is imported to cause such a conflict. HoloViews uses param.Version without ever importing version.

Python 3.6.2 (v3.6.2:5fd33b5, Jul  8 2017, 04:57:36) [MSC v.1900 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> import holoviews as hv
>>> version
Traceback (most recent call last):
  File "<pyshell#1>", line 1, in <module>
    version
NameError: name 'version' is not defined
>>> import version
Traceback (most recent call last):
  File "<pyshell#2>", line 1, in <module>
    import version
  File "param\version.py", line 83, in init param.version (param\version.c:7238)
    import os, subprocess
SystemError: Parent module 'version' not loaded, cannot perform relative import
>>>

or also:
Python 3.6.2 (v3.6.2:5fd33b5, Jul  8 2017, 04:57:36) [MSC v.1900 64 bit (AMD64)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> import version
Traceback (most recent call last):
  File "<pyshell#0>", line 1, in <module>
    import version
ModuleNotFoundError: No module named 'version'
>>> import holoviews as hv
>>> import version
Traceback (most recent call last):
  File "<pyshell#2>", line 1, in <module>
    import version
  File "param\version.py", line 83, in init param.version (param\version.c:7238)
    import os, subprocess
SystemError: Parent module 'version' not loaded, cannot perform relative import
>>>

if you don't reproduce, then it may be the way I created the wheel.

Sounds like an issue with Cython. How did you install param? I thought we hadn't enabled that by default.

param comes from there : http://www.lfd.uci.edu/~gohlke/pythonlibs/#param.
you think I may recompile it ?

i will downgrade to previous cython, to check

Oh wow, I didn't even know that was still in there. Will push a PR removing that regardless. Hopefully that fixes it.

Sounds like an issue with Cython. How did you install param? I thought we hadn't enabled that by default.

I believe that param uses Cython when installed with pip, and since wheels are mentioned above, I assume param is indeed being installed with pip.  I have no idea why that difference between conda and pip installations happened; seems backwards if anything!

 That should have gone away a long time ago!

Not clear this has actually been fixed by that change so I'll leave it reopen until @stonebig can confirm.

It works now as intended. Many thanks to @cgohlke to have made us look in the right direction.

Yes, thanks for looking into this for us @cgohlke!


Update manifests/n/NGWIN/PicPick/6.0.0/NGWIN.PicPick.locale.en-US.yaml

Fixes an issue where this package has multiple different monikers
Microsoft Reviewers: Open in CodeFlow

[Policy] Reset Labels


[BUG] Action appears to be ignoring branches marked by Github as "stale"

Describe the bug
I have a test repo that includes branches that Github has marked as "Stale"

These branches meet the requirements to be considered for deletion by this action but they are not showing up in the output.
To Reproduce
Steps to reproduce the behavior:

Using a Github repo that has "Stale" branches, add the stale-branches action

Expected behavior
I would expect that this action includes all branches, including those marked as "stale" by github.

This was an issue with my branch protection settings. These branches were considered protected.


Tool documentation

For each tool added to https://github.com/magicDGS/ReadTools/wiki/Tools, we require a new entry in the wiki with the description provided in the CMD help and all the parameters.
Related to https://github.com/magicDGS/ReadTools/issues/183

It will be clear if the parameters appear in a table-like format. Although it is not necessary if #182 is implemented at some point.

Done!


Not allowed to load local resource for PNG file

Hi,
I created a react project base on the webview module. But I can't import images. I have this error on the inspector
[Error] Not allowed to load local resource: file:///Users/user/Library/Application%20Support/com.bohemiancoding.sketch3/Plugins/myplugin.sketchplugin/Contents/Resources/_webpack_resources/assets/images/Logo.png
	setValueForProperty (resources_webview.js:25000)
	setInitialDOMProperties (resources_webview.js:29481)
	setInitialProperties (resources_webview.js:29676)
	finalizeInitialChildren (resources_webview.js:31149)
	completeWork (resources_webview.js:42628)
	completeUnitOfWork (resources_webview.js:45839)
	performUnitOfWork (resources_webview.js:45815)
	workLoopSync (resources_webview.js:45780)
	performSyncWorkOnRoot (resources_webview.js:45406)
	scheduleUpdateOnFiber (resources_webview.js:44838)
	updateContainer (resources_webview.js:48023)
	(anonymous function) (resources_webview.js:48408)
	unbatchedUpdates (resources_webview.js:45553)
	legacyRenderSubtreeIntoContainer (resources_webview.js:48407)
	./resources/webview.js (resources_webview.js:54298)
	__webpack_require__ (resources_webview.js:20)
	(anonymous function) (resources_webview.js:84)
	Global Code (resources_webview.js:85)

Everything compile well, the image path is correct, I have all the rights, my webpacks are ok (I tried url-loader, file-loader and image-loader). And if I try an URL instead of a local file, it works fine. Does any one have an idea where this error come from ?
Thank you

I solve my problem by installing a dependency :
npm install<EMAIL_ADDRESS>


Laravel accessors and mutators  does not work as expected when eloquent model is syncable

Currently, I've encountered an issue where the use of accessors and mutators during record synchronization modifies records, leading to unexpected bugs. Is there any way to  to leverage these functionalities (accessors and mutators ) for web actions (modify model from website ) not for syncing.
Model :
class TimeRecord extends Model
{
    use SoftDeletes, HasUuids, Watermelon;

    protected $guarded = [];

    protected array $watermelonAttributes = [
        'time_record_guid',
        'employee_guid',
        'employee_name_last_name_first',
        'work_date',
    ];

    protected function workDate(): Attribute
    {
        return Attribute::make(
            get: fn($value) => formatDateUserTz($value, 'm/d/Y'),
            set: fn($value) => strtotime($value)
        );
    }

}

Controller :
  public function update(Request $request, TimeRecord $timeRecord)
    {

        $timeRecord->work_date = strtotime($request->work_date);

        $timeRecord->save();
    }

I'm not sure what kind of error you are referring to. The Watermelon trait doesn't hook into the regular lifecycle of the model, if you modify a record with your own custom code like in the controller shown the Watermelon trait shouldn't have anything to do with that nor should it interfere. If you can explain in more detail what kind of issues you are having (or ideally, create a brand new Laravel Sail project, require just this package, and then replicate the issue with some steps I can follow) I would be glad to look into this for you.

I am still not sure what is going on with this issue, please file another one with a reproduction and I'd be more than happy to look into it.


Handle live acquisition with a simple GUI

There's a lot of demand to handle live data streams from the detector. We should implement at least a basic graphical user interface (and scripting capability) to hook up a detector to LiberTEM, stream the data to file and allow the user to explore the data while it comes in and accumulates.

Yes, Merlin should be the first candidate because we have already @magnunor 's code and the interface seems benign.

Just a "shower thought", what about using a video stream to show the updating data?

Should compress well -- not much change per frame, probably much more efficient than image fragments for high refresh rate
Should be working natively in modern browsers these days
Optimized for silky-smooth display with low resource use on the client -- uses GPU and works smoothly on any smartphone

What about resource use for encoding? In any case, it shouldn't be worse than sending JPEG snippets. Which codec would work? MP4, WebM llok good: https://developer.mozilla.org/en-US/docs/Web/HTML/Supported_media_formats#Browser_compatibility

Should compress well -- not much change per frame, probably much more efficient than image fragments for high refresh rate
Should be working natively in modern browsers these days
Optimized for silky-smooth display with low resource use on the client -- uses GPU and works smoothly on any smartphone

Agreed!

shouldn't be worse than sending JPEG snippets

Don't know about that. Do you have a comparison handy? Basically comparing CPU usage for encoding h264/h265/... vs ~MJPEG. As far as I know, "real-time" h264 encoding is only possible either because of hardware support or good multi-core usage, where we would rather use these cores for more useful work.
The MJPEG Wikipedia page just hand-waves that it "is not computationally intensive", I'd say citation needed?
A comparison I found:
http://blog.loopbio.com/video-io-2-jpeg-decoding.html
They are comparing single-core ~MJPEG with multi-core (how many?) h264 etc. - so if that benchmark was run on a 6 core machine, I would guess that a trivial parallel implementation of MJPEG could beat h264. Oh well...
Either way, before going into the performance side of video encoding, maybe we should talk about the requirements?

latency?
frame rate?
image quality?
compression ratio / low bandwidth?
resolution / down sampling?
metadata synchronization?
are the requirements different between raw frames vs. analysis result?

And more generally: what workflows are needed?
PS.: https://pypi.org/project/PyTurboJPEG/ and the fork by the loopbio folks at https://github.com/loopbio/PyTurbojpeg looks interesting, and it's quite compact! Also: they have an eye for unnecessary copies, yay! :rocket:

I think we can close this issue, as jupyter notebooks work quite well for simple GUI needs. What do you think?


How to ingest text from binary documents like pdf, docx etc?

Hi,
first of all thank you for this great connector.
We're using kafka-connect-file-pulse to replace a legacy component to ingest documents from various file types like txt, pdf, docx etc. in our project. The text content of each file is handled as one message in Kafka.
Is there a way to extract the document content as text within the connector? We have been using Apache Tika for this in the past. Can this functionality be integrated in the connector? Maybe with a combination of the io.streamthoughts.kafka.connect.filepulse.fs.reader.LocalBytesArrayInputReader and a filter?
Do you have any recommendations?
Thanks in advance,
Christian
Provide details of the setup you're running
File Pulse 2.4.3 in Docker

@fhussonnois thank you very much. I'll try to implement a RecordFilter.

Using a custom RecordFilter works like a charm. Thank you very much


HoloLens image stabilization - Remaining issues

Basing on @anderm's feedback, these are the remaining works:

Update information about the custom WebRTC UWP repo that works with prediction time stamp.
Write unit tests for the BufferCapturer class.
Add documentation about how the image stabilization and prediction time stamp work.

Closing as individual issues were created:
#181 #180 #179


TTS disable join rather than delete/disable

Summary

For TTS sources, it would be great if the migration tool did a disable of downlinks, without deleting or disabling the devices entirely. In other words doing this:
ttn-lw-cli dev set --application-id <app-id> --device-id <device-id> \
    --mac-settings.schedule-downlinks=false

Why do we need this?

Migrating devices between TTS instances, while keeping working uplinks until the device rejoins.
What is already there? What do you see now?

...
What is missing? What do you want to see?

...
Environment

...
How do you propose to implement this?

...
How do you propose to test this?

...
Can you do this yourself and submit a Pull Request?

...

Closing this because omitting --dry-run does exactly this. Ref https://github.com/TheThingsNetwork/lorawan-stack-migrate/blob/26138c0356a8dff20f5d2bd9a3822f614e4581ad/pkg/source/ttnv3/source.go#L77-L84


Verus panics on assert_seqs_equal! after #1289

I have some code that includes an assert_seqs_equal! macro call. Here is a playground link. The example below is minimized from a larger example using the automated minimizer, so the proof is not expected to verify---the problem is the panic, not the proof failure.
use vstd::{prelude::*, *};
verus! {

enum Foo<'a> {
    Foo(&'a [u8]),
}

impl View for Foo<'_> {
    type V = Seq<u8>;

    open spec fn view(&self) -> Self::V;
}

impl<'d> Foo<'d> {
    fn len(&self) -> usize {
        match self {
            Foo::Foo(e) => {
                // NB: This is expected to not verify, since this
                // is a minimized example
                proof {
                    assert_seqs_equal!(e.view(), self.view());
                }
                slice_len(e)
            },
        }
    }
}

fn slice_len(i: &[u8]) -> usize {
    i.len()
}

} // verus!
fn main() {}

When I run Verus I get the panic:
thread '<unnamed>' panicked at rust_verify/src/verifier.rs:780:21:
internal error: generated ill-typed AIR code: error 'in equality, left expression has type vstd!seq.Seq<u8.>. and right expression has different type Poly' in expression '(= s1@ (vstd!view.View.view.? $ (SLICE $ (UINT 8)) (Poly%slice%<u8.>. s@)))'
note: [run with `RUST_BACKTRACE=1` environment variable to display a backtrace](https://play.verus-lang.org/?version=stable&mode=basic&edition=2021#)
thread '<unnamed>' panicked at rust_verify/src/verifier.rs:2110:29:
worker thread panicked

If I comment out proof { assert_seqs_equal!(e.view(), self.view()); }, the panic disappears and Verus handles this code just fine.
Interestingly, if I roll back #1289, the panic also disappears (this time I get a proof failure on the assert_seqs_equal!, as expected). This, combined with the "ill-typed AIR code" message mentioning a type difference between Seq<u8> and Poly, makes me suspect that the poly refactor might have broken this example.

Yes, with Verus version
Verus
  Version: 0.2024.10.11.dba7a12
  Profile: release
  Platform: linux_x86_64
  Toolchain: 1.76.0-x86_64-unknown-linux-gnu

my larger example works. Thanks for fixing!


Population Data - Subnational

Historical data on changes in the population by region over time.

Population pyramid data produced by region for the year 1994. This is the earliest year available with consistent regions. Estimates pulls from the reports found here: https://www.statsethiopia.gov.et/census-1994/

National data (1994): pyramid.csv
Regional Data (1994): pyramid_region.csv

Issue Closed: PR #319


WebUI

Fixes #3123
Short description
Adds Raiden WebUI to CLI :tada:
For compliance with schemas, features, behavior and endpoints expected by WebUI, some adjusments were needed:

on SDK, a new method, getTransfers, allows efficient filtering and pagination when fetching transfers from database
/payments and pending_transfers now use this method to serve properly paginated results to webUI
/config.*.json serves a dynamic JSON to be used by webUI
/rpc proxies ETH JSON RPC calls to SDK's provider
webUI can be downloaded and put inside build folder by yarn build:webui command, to be run after yarn build
a new CLI option --web-ui enables serving the webUI from the /ui endpoint.

Definition of Done

[x] Steps to manually test the change have been documented
[x] Acceptance criteria are met
[x] Change has been manually tested by the reviewer (dApp)

Steps to manually test the change (dApp)

After building the CLI, build the webui with yarn build:webui
Run the CLI with --web-ui parameter
Visit http://localhost:5001/ to see the webUI

Hi, @karlb , thank you very much for the review.

The SDK doesn't handle much PFS's errors, and instead prefer to just pass them through to caller, as they usually contain more details about what happened. The CLI then is responsible for catching this common error and translating it to the user, which it's now doing as a E409, same as python rest api. It seems there's no special handling for this kind of error in the webUI, but the 409 with details added now should look better and more descriptive in the notificationService. It's done now.
This is intended, the LC doesn't sync the same way as PC (i.e. all events of interest in each block[range]), and instead each epic polls independently its contracts for on-chain events, so we can't really tell how many blocks are left to scan, only when it's finished. Hopefully, after first sync, the SDK starts almost instantly, so it shouldn't be noticeable to users often.

I'll consider the points addressed, and merge this PR. If something is still not good, we can improve any point of concern in a followup PR. Thank you very much for going through this, this is a great addition for the LC, and should also extend webUI's life, which is also a great piece of software (good job @manuelwedler).


unable to find local git repo

i have made a git repo on my gitblit server and commited files to it. now i can not find the files on the machine with the repo. where would these files be stored?

I'm not following exactly what you did on your setup but it seems like either you are new to git or that you are using terminology incorrectly.
I am assuming...

made a repo on my gitblit  server == I created a repo using Gitblit
I committed files to it == I cloned the repo, committed files to my clone, and pushed my new commits to Gitblit

Gitblit stores the repos in {baseFolder}/git/{project}/{repo}.git.  The files that you committed & pushed will be named/identified by a SHA-1 hash and will not be found by scanning the filesystem by their filename.  The committed files may be loose (individual files named by SHA1 hash) or they may be packed (single file aggregate of multiple files).
Does that answer your question?

Closed due to inactivity.


Have you a template on Figma of your components?

It will be great to have a template on Figma to design a great interfaces with your components 🚀🚀

Sorry, I don't use Figma when creating components, it's all done in the browser.


Show the crowdsourcing goals

Add a goal (and stretch goals?) to the image total count API, and show in the app (home page, user form, ..) an indicator of how far we are in the total crowdsourcing process to motivate the user to contribute.

Do we have the some designs for this component?

Yes, there's a sketch in the 3rd screen of https://streetwise-app.ch/#/tour

This is continued as part of PR #20

Please see my refactoring of your component in the PR:

Slightly changed the name to reflect functionality
Removed all words, German or otherwise
Added a Vuesax component
Tidied up the code a bit

I wonder how performant that row counter call is. Should we store it in memory, updating every few hours?

I think we should cache this nevertheless. Saving it in app memory works for now.
However if we ever do a multi-dyno deploy, we may end up running into situations where the two app instances have different values cached. But let's solve for that once we cross that bridge.

I will also close #19

I think we can close this now, and add a separate ticket for a performance review.


[OTHER] Font Color

What would you like to share or ask?

In the provided image, there are inconsistencies in the text colors that need to be addressed. Although these inconsistencies may be subtle and minute to observe, I would like to fix them to ensure a cohesive visual experience.
Light Mode:
Header: 111827
Footer: 000000
Dark Mode :
Header : F3F4F6
Footer : E5E7EB
Suggestion: To ensure consistency across the UI, it would be beneficial to create a variable that represents all the colors used.
I have even check your style guide

If I have overlooked or omitted anything, please feel free to correct me.
Checklist

[X] I have checked the existing issues
[X] I have read the Contributing Guidelines
[X] I am willing to work on this issue (optional)
[ ] I am a GSSoC'23 contributor

Great observation @PriyansuMaurya, assigning to you!

Hey, I was just wondering, for which colors should I create variables and how many of them?

@PriyansuMaurya still on this?
need any help?

Are you still working on this @PriyansuMaurya?

I'm on it and Thanks @CBID2, for reminder.

@CBID2 what might be the issue here? can you elaborate more?

@CBID2 what might be the issue here? can you elaborate more?

Hey @hona9. Someone started a PR for this issue already. Right now, I'm trying to find a way to make the colors more accessible and adding them to said PR.

Hey @CBID2 is someone still working on this issue if not could you please assign me this issue to work on?

Hey @CBID2 is someone still working on this issue if not could you please assign me this issue to work on?

@CBID2
Priyanshu is not assigned so maybe we can go forward. If you have the colors, please let him know.

Go right ahead @Anmol-Baranwal

@CBID2
Not me. It's @HimanshuMahto 😅
I've assigned him.

Hey @CBID2 is someone still working on this issue if not could you please assign me this issue to work on?

@HimanshuMahto
I have assigned you the issue. Please proceed with it, and don't hesitate to tag me @Anmol-Baranwal or @CBID2 if you require any assistance.


Reworks yaml-cpp build flags

This includes renaming and localizing usage of them

[ci approve autest]

[approve ci autest]

[approve ci autest]

Something with my patch is breaking autest (loading xdebug.so):
[Jun 21 16:35:17.059] {0x2b9763a70b60} FATAL: unable to load '/home/rmeyer/trafficserver/tests/_sandbox/cache-generation-clear/ts/plugin/xdebug.so': /home/rmeyer/trafficserver/tests/_sandbox/cache-generation-clear/ts/plugin/xdebug.so: undefined symbol: TSHttpTxnReenable

Cherry-picked to 8.0.x


Update splash screen icons.

Ack https://github.com/Groestlcoin/BlueWallet/pull/5/commits/c3ae33e3a4fee54ec5e2bd48b45cc56e93570fb4


add some more polyfrost mods

adds chatting, which adds chat tabs and copying and searching and screenshotting and stuff
replaces sk1er oam with overflowanimations, which has much better compat + more features

Not a fan of replacing a mod.

Not a fan of replacing a mod.

well, its a fork of it and retains all of its past features, so...

if you just want to put it above or below sk1er oam that's fine too, but i just don't see the need

if you just want to put it above or below sk1er oam that's fine too, but i just don't see the need

Sounds better to me, but it was just my opinion, not sure what the other people think about it.

You should leave both next to each other yeah, highlight differences in the 2nd one's description so people can easily tell

if you just want to put it above or below sk1er oam that's fine too, but i just don't see the need

Sounds better to me, but it was just my opinion, not sure what the other people think about it.

Personally I agree

if you just want to put it above or below sk1er oam that's fine too, but i just don't see the need

Sounds better to me, but it was just my opinion, not sure what the other people think about it.

Personally I agree

i too concur


Add a new client / server command to rename CIDR

Closes https://github.com/tonarino/innernet/issues/309

@strohel @mcginty thank you for your review :)
I commited the suggested changes and checked one more time that docker tests succeed for the new test case.


With isDisabled prop, Tab works correctly.

Closes #3797
Description
With isDisabled prop, Tab works correctly.

@Shunii85
Thank you for fixing it 🥹! I’ll be more mindful with my changes to avoid causing regressions.


Allow specifying IdentityIdExtractor at ThrottlingTrollOptions level as well

So that Rules can come from config file, and yet IdentityIdExtractor can also be specified

Done in v4.0.


Generate diagrams, for increased understanding of code

This will look similar to the instrumentation code, and will only be exercised during tests.

I was exploring ways to use go generate for this over break.  Seems low priority at the moment though.

Done in #65 and #66


Fix broken tests

We discovered that some test files had a _test.rb suffix instead of _spec.rb suffix, which meant they weren't being run when we ran rake (i.e. they weren't being run on CI).
In renaming the test files we discovered some failures which we've also fixed.

⚠ This repo is Continuously Deployed: make sure you follow the guidance ⚠
Follow these steps if you are doing a Rails upgrade.

Thanks @catalinailie :+1:


run: Make test result output compact

The new sub test routine causes the test result output cluttered. In
order to avoid that, only print the information of any sub-tests being
skipped or failed.
Before
===> alert-testmyids:
|
 --> Sub test #1: OK

|
 --> Sub test #2: SKIPPED : Suricata v6.0 not found

|
 --> Sub test #3: SKIPPED : Suricata v2.0 not found
OK

After this patch
===> alert-testmyids: Sub test #2: SKIPPED : Suricata v6.0 not found
Sub test #3: SKIPPED : Suricata v2.0 not found
OK

Already merged into master.


Citation section and badge added

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@48cc9a5). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main       #8   +/-   ##
=======================================
  Coverage        ?   91.68%
=======================================
  Files           ?       15
  Lines           ?      445
  Branches        ?        0
=======================================
  Hits            ?      408
  Misses          ?       37
  Partials        ?        0

Flag
Coverage Δ

pytest
91.68% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Also, I did figure out how to do ScrolledWindows rather than Panels. Where it says wx.Panel, I can replace that with wx.ScrolledWindow. And there's likely some panes where that might be preferred.

@Sophist-UK There's not a future spec that says scooped corners are legal. The actual spec is that the value is illegal and therefore the shape throws an error and is not included. The draft spec says the rx value is illegal and therefore ignored, which is less extreme and what I opted to do. That is conforming to the future spec.
I would guess the defacto non-spec would be to take the absolute value. You'd get normal rounded corners equal to the rx value. This is what you do for arcs in general. svgelements is largely to spec. The only pressing issue in it is that it's built completely wrong from the ground up and needs to be rebuilt meerk40t/svgelements#87

Personally I don't see any issue with negative radius if it is unambigous to know what that means (though not sure what happens when rx is negative and ry is positive). But perhaps you are right - that this is a kindness and not a correct interpretation.
My point about the future CSS comment was that the spec does say that CSS specs outside the SVG spec may make it OK to process illegal values rather than ignore them. So there is kind of a 1/2 precedent.

Also, that fix the zoom with the resample size change turned out pretty awesome. Took a little to figure out the math, but totally solid. Finished up the alternative toolbars. So it would be possible to get rid of ribbonbar. Though it's a bit smaller now, so it might be a bit more reasonable. Though I couldn't do a hybrid button like I had for Camera. And I had to put then in panels so they would stop being weird.

Bugs

Rectangle with rounded corners still not displayed in Simulatioin scene
Operation for red paths being classified on load as engrave rather than cut.

I wrote to Oz (lightburn) pointed out how you would properly convert that to svg, gave a half hearted appeal to the svgwg, and corrected my bug (I gave something wrong rather than followed the spec), and added in a non-strict interpretation that would read it as intended. I covered all the bases.

Rectangle with rounded corners still not displayed in Simulatioin scene

That's odd. I must have just broken it because those absolutely should decompose correctly.

Operation for red paths being classified on load as engrave rather than cut.

What are the operations on that? Does it have a red engrave or something?

Rectangle with rounded corners still not displayed in Simulatioin scene

That's odd. I must have just broken it because those absolutely should decompose correctly.

I did report it on a previous beta. Displays correctly in main scene, and cuts correctly. Just an issue in simulation scene.

Operation for red paths being classified on load as engrave rather than cut.

What are the operations on that? Does it have a red engrave or something?

It is an inkscape svg without operations. When imported it creates a single red operation which is an engrave rather than a cut.

I did report it on a previous beta. Displays correctly in main scene, and cuts correctly. Just an issue in simulation scene.

Arcs aren't drawn. The rounded corners are arcs. I'll replace them with some cubic bezier curves.

It is an inkscape svg without operations. When imported it creates a single red operation which is an engrave rather than a cut.

The red-default to cut is a great idea. Here I would like to blame @jaredly. He wrote that code. https://github.com/meerk40t/meerk40t/commit/f5d3ae1003f5c06e10adc2b067a4dfb378f4fa84

Okay, arcs are drawn (replaced with things that do draw). Fully deprecated the ArcCuts (they were weird anyway). And red colors get cuts default classification.

I did report it on a previous beta.

I try not to miss such things, but sometimes very much could or dismiss them wrongly. Thanks for keeping on such things.

Bug
Raster engrave of combined path shows at wrong size in Simulation window.

Duplicate Operation (for cut - haven't tested other types) does not copy paths over.

Ok. Done.

Raster engrave of combined path shows at wrong size in Simulation window.

This sounds bad. Hm. Rasters are really screwy there for even just random images.

Arrows on travel lines in Simulation are still different sizes.

This is by design. They looked really weird when the tiny moves got giant arrows. So they are scaled with distance.

Wait, my rotary was on, rasters might seem fine. I might need some more details on repeating this bug. My stuff lines up.

Raster engrave of combined path shows at wrong size in Simulation window.

This sounds bad. Hm. Rasters are really screwy there for even just random images.

Just so we are clear, these are vectors with fill being rastered not an image. (A 5 x 20ish array of tiny heart shapes.)

Arrows on travel lines in Simulation are still different sizes.

This is by design. They looked really weird when the tiny moves got giant arrows. So they are scaled with distance.

The arrow heads should all be small. They are only needed to give an idea of direction. And don't forget my request for small dots at the ends of the arrows so you can tell where they start and end.

I don't have a rotary either, but I test it here and there.
Small dots on the end? I don't understand how that makes things more able to determine where they start.

Depends on small the dots are. The issue is that if a travel move ends somewhere and the next travel move starts at the same point (because it is a single closed path) and the next travel move is in exactly the same direction as the first, then you have no idea where the end/start point is (travel moves will cross this path at least twice, more if it is a meandering path).

Joe apparently kinda solved that. Should be about 10 mils out. So if your start and end are within 10 steps they likely count as closed. Not that a tiny sliver of material necessarily will stop something from dropping but it seems a reasonable enough guess there. Still working mostly on this silly crash bug for estop. But, optimizations soon. I think I have a solid plan to do them.

Before you do optimisations, if you want my feedback on your intended approach, send me an email with details.

UI Spooler

Estimated time column no longer present
Time boxes at top are not populated - and can be typed into!!

The is "ghost"· movements while burning... check the video

https://youtu.be/QQoSrjL1-5A

@ampostata Please add the SVG file for the ghost movements here so I can take a look,
I also occasionally see ghost movements in my burn and I can usually track them down to oddities in the SVG file.
I think that MK might benefit from a "drop null movements" feature, but we will have to work out what constitutes a "null movement".

The is "ghost"· movements while burning... check the video

I think those are twitches. You get them sometimes when the bot is kicking in or out of particular burns. If all those engraves are at the same speed it likely shouldn't be doing that. If you turn off rapids between jogs it'll likely get rid of them since you often get them jumping from rapid to non-rapid mode. They kick the burns up to speed. And there's more specific research needed to figure out when they will happen.

Mostly it's jogs between different cuts at speeds 25.4 or higher (Accel 1).
They happen with the laser off in places where they wouldn't happen with the laser on, PPI for example on some cuts will trigger them midcut when no-ppi resulted in them only at start and end.
They happen with fast reversals. I'm not sure the exact code for the board but I think maybe switches from R to L or B to T can trigger them.
They are hardware encoded into the board, but there are methods of avoiding them that usually involves remaining in the same speed code.
In some cases like yours they aren't that helpful overall since they don't seem to twitch in an ultimately useful direction.
The direction of the twitches is often set in the preamble of the program mode N??S1E where the ?? define the direction. The cuts pre-check this with the plotplanner and should give the correct directionality of the mode.

Even for this it seems a bit extreme. There's stuff with the common jogging procedure that goes in some direction during the jog. And natively this is cancelled out to zero if the jog is given counter-directions. And for this it would move in the wrong direction, force the move back to the correct direction, then have access to speed mode directions and go to the correct position.
I had @joerlane  run like a 40 minute test of every angle position to make sure this came back to the correct position when setup like this. However there is an open question as to whether the direction of the jog could be used to one's advantage. If I knew what it was going to do in a consistent manner, I could go ahead and let it do a jerk, but know where it ends up, and move from that position.
Also, there's a solid chance this is misoptimized. There's a setting in root that defaults to 127 which is about a 10th of an inch. You can likely move that up a lot higher, in console set -p / opt_jog_minimum 1000 would move it to 1 inch. Which means if your last cuts end and next cut are less than 1 inch apart it will move at speed rather than doing a jog which can in cases introduce jerks like that.
There's a trade off at times and I don't know where the correct value for the optimization jog minimum should be so I made it a variable and didn't put it anywhere in the gui or anything. The distance there likely differs based on the speed of the laser during regular cuts.

Before you do optimisations, if you want my feedback on your intended approach, send me an email with details.

Posted in #429. I outline the current algorithm and the changes the options would require. I think it's perfectly actionable. It also would result in cutcode that could be additionally optimized, which would be helpful.

Estimated time column no longer present

Something useful needs to be done with spooler but likely not this version.

Time boxes at top are not populated - and can be typed into!!

I removed them.

I think that MK might benefit from a "drop null movements" feature, but we will have to work out what constitutes a "null movement".

This already should exist. The Plotplanner really shouldn't care much for null movements. Now admittedly it might be remove some movements that make no difference. Like if I move 10 to the right and then 10 to the left and never fire the laser. It wouldn't be too hard to figure that out, but I don't think anything at a lower level is making such movements. With the possible exception of PPI which might be set to zero, or some odd cases.

@ampostata Please add the SVG file for the ghost movements here so I can take a look,
I also occasionally see ghost movements in my burn and I can usually track them down to oddities in the SVG file.
I think that MK might benefit from a "drop null movements" feature, but we will have to work out what constitutes a "null movement".

Here you have the file:
e.zip

In the requester shown when the job has ended, can we have the text "it has taken 8:34 minutes" or something similar?

MeerK40t crash log. Version: 0.7.0 RC-5 on win32
Traceback (most recent call last):
  File "meerk40t\gui\wxmeerk40t.py", line 2730, in on_click_open
  File "meerk40t\gui\wxmeerk40t.py", line 2437, in load
  File "meerk40t\core\elements.py", line 4455, in load
TypeError: load() got multiple values for argument 'channel'

Caused by somebody trying to load a gcode file. This code still exists but is very old.

MeerK40t crash log. Version: 0.7.0 RC-5 on win32
Traceback (most recent call last):
  File "meerk40t\gui\wxmeerk40t.py", line 2730, in on_click_open
  File "meerk40t\gui\wxmeerk40t.py", line 2437, in load
  File "meerk40t\core\elements.py", line 4455, in load
  File "meerk40t\core\svg_io.py", line 285, in load
  File "meerk40t\core\elements.py", line 419, in add
  File "meerk40t\core\elements.py", line 251, in notify_attached
  File "meerk40t\core\elements.py", line 1136, in notify_attached
  File "meerk40t\gui\wxmeerk40t.py", line 3152, in node_attached
  File "meerk40t\gui\wxmeerk40t.py", line 3313, in node_register
AttributeError: 'Node' object has no attribute 'item'

Caused by registering a node with an unregistered parent.
    def node_register(self, node, pos=None, **kwargs):
        parent = node.parent
        parent_item = parent.item

The is "ghost"· movements while burning... check the video

I think those are twitches.

No - I sometimes get these and I can normally track them down to odd artefacts in the SVG file.

@tatarize Can we start a new issue for the next RC as this one is getting very long?

@ampostata Please add the SVG file for the ghost movements here so I can take a look,
I also occasionally see ghost movements in my burn and I can usually track them down to oddities in the SVG file.
I think that MK might benefit from a "drop null movements" feature, but we will have to work out what constitutes a "null movement".

Here you have the file:
e.zip

This contains e.lbrn which is a lightburn file. MeerK40t does not load lightburn files, so I am confused about how you got this issue.

Wrong file, sorry. I will upload the svg later (anyway, you can load it in lightburn and export as svg, as I will do).

e.zip
Converted from lightburn.

Caused by somebody trying to load a gcode file. This code still exists but is very old.

Fixed. Sadly fixed by fixing a lot of gcode. This might actually open up some issues if people actually try to use this stuff. There's a solid chance the gcode could do operations like move to various places, cut various things and turn on the laser and dwell in place, sorts of things. This lasercode node can be converted into cutcode which works more normally. This also can be turned into paths now. Which would let you see it. But, each step loses data. This might need additional testing. It doesn't just crash now.

@Sophist-UK
Yeah, Okay. Just about there anyhow. I think I got the last known crash bug. So most everything else should be pretty close.

@Sophist-UK @inspectionsbybob @ampostata ... please make sure your issues were properly addressed and if they were lost in the shuffle reraise them in #443.

Wrong file, sorry. I will upload the svg later (anyway, you can load it in lightburn and export as svg, as I will do).

So was your video taken of a burn from lightburn (via MK Ruida emulation?) or from MK? And if MK, why don't you already have the SVG file?

@Sophist-UK Instead of Inkscape, I use Lightburn to design. Then I export as SVG and use MK on a standard K40 (original motherboard).

e.zip
Converted from lightburn.

I will take a look, but I have no idea whether this is the exact same SVG code that was loaded into MK and created the burn issues in the Youtube video. We can only guess that it is - but you might e.g. have a different version of Lightburn or perhaps Lightburn introduces these on a random basis.

Ok - the simulate looks like this:

If you look at the black "petals" you can see that there are two travel moves.

If you look at the animation, you can see it is cut as a single burn starting where the first travel comes in.

The SVG for this node looks like this:
<path d="
M 1653.08042128,2064.60992434
C 1614.28485808,2092.7022078 1605.41489818,2146.60519187 1633.24852386,2185.13734699
C 1653.74120631,2213.70138926 1689.78521818,2226.7356489 1724.09578012,2217.99106011
C 1736.56129666,2261.91630998 1781.39450529,2288.23177229 1826.18729839,2277.91518998
C 1869.7206156,2268.03351856 1898.8261026,2226.88655574 1893.44203699,2182.83778771
C 1891.16256808,2164.34942464 1882.92648906,2147.13600388 1869.96837148,2133.77172611
C 1901.53823289,2101.29897641 1902.73608468,2050.22995542 1872.71280955,2016.73906941
C 1843.30981167,1983.68621505 1793.39543912,1978.37911916 1757.25833905,2004.46295108
C 1741.49834691,2015.76110234 1730.01923481,2032.03130793 1724.70396785,2050.60524289
C 1699.99324818,2044.54191001 1673.78870725,2049.66613343 1653.08042128,2064.60992434
z
" fill="#ff0000" stroke="#000000" stroke-width="6.535410550548559" transform="scale(0.096000)" />

which looks fine.
Without optimisation, you still get a continuous burn:

and there is a single travel in and out.
So it appears that the extra travel (to the node 2 back from the end node) is a minor bug in the optimisation.

2nd report of ghost movements: "Yes, If found that when cutting there are unnecessary movements, but for now it doesn't care.
Will keep testing it in the next few days."


Does exoplayer support high framerate videos (over 100 fps)?

Not a bug, but I was just wondering this.
While analyzing the source code, I found out that ACTIVE_INTERVAL_MS is set to 10 in ExoPlayerImplInternal.
ACTIVE_INTERVAL_MS is used when scheduling next doSomeWork(), which handles the rendering. So I think the possible max framerate per second is 1000/10=100.
Am I getting something wrong? Thanks in advance.

ExoPlayer is able to pass more than one frame to be decoded and shown on the screen in a single iteration of its work loop, so the frame rate is not limited by the 10ms doSomeWork() timing. The number of frames that we can queue to the decoder simultaneously and schedule to be displayed on the screen depends on the device (as well as whether the decoder is fast enough to decode >100fps video in real time).
I'm not aware of a hard limitation within ExoPlayer of frame rate - I would expect any constraint to come from the performance/behaviour of the underlying device.


feat: create a playground for global config

https://splunk.atlassian.net/browse/ADDON-69283
Add playground for storybook to allow global config testing

I thought about some more reach global config editor like https://storybook.js.org/addons/@kickstartds/storybook-addon-jsonschema Do you think it is extra?

Nice, add-on, looks like a much more useful and convenient way of editing config. I will try add it.

I thought about some more reach global config editor like https://storybook.js.org/addons/@kickstartds/storybook-addon-jsonschema Do you think it is extra?

added that schema editor but it requires access to schema.json (to make it useful), right now i just copied it manually but how do you think we should tackle it?


Allow for schema pruning during update check for files to touch

Description

Resolves #1201
Allows for schema pruning in the first part of an update to check for files to touch.
Code snippet I ran:
>>> import pyspark.sql.functions as F
>>> from delta.tables import DeltaTable
>>> table = DeltaTable.forPath(spark, "test")
>>> table.toDF().printSchema()
root
 |-- key: string (nullable = true)
 |-- value: long (nullable = true)

>>> table.update("key = 'c'", set={'value': F.lit(6)})

The execution plan for the find files to update:
before:
(1) Scan parquet
Output [2]: [key#526, value#527L]
Batched: true
Location: TahoeBatchFileIndex [file:.../projects/delta/test]
PushedFilters: [IsNotNull(key), EqualTo(key,c)]
ReadSchema: struct<key:string,value:bigint>

after:
(1) Scan parquet
Output [1]: [key#686]
Batched: true
Location: TahoeBatchFileIndex [file:.../projects/delta/test]
PushedFilters: [IsNotNull(key), EqualTo(key,c)]
ReadSchema: struct<key:string>

Only key is read, not value as well. The line swap should result in the same behavior, but doing the select before the nonDeterminstic UDF allows schema pruning to happen.
How was this patch tested?
Existing UTs plus screenshot of execution plan.

Does this PR introduce any user-facing changes?

Performance improvement for update with data predicate.

I've randomly noticed Update taking an incredibly long time for certain tables. Today it was taking ~10 minute to find the files to update for a 1 GB table, and another 20-30 min to try to rewrite it. This was the first thing I found while trying to look more into that.

I've randomly noticed Update taking an incredibly long time for certain tables. Today it was taking ~10 minute to find the files to update for a 1 GB table, and another 20-30 min to try to rewrite it. This was the first thing I found while trying to look more into that.

@Kimahriman - want to make a separate issue for this? with the full stack trace; so we can help debug.

Yeah it would be during the optimization stage, not sure if that can be captured, I can see

Had to add a physical plan variant of that but got a test working


Port CoreFoundation/Base.subproj/CFFileUtilities.c

This patch makes a part of CoreFoundation/Base.subproj/CFFileUtilities.c compilation work with wasi-libc

@swift-ci test


Update pypi.py

Fixes AssertionError in Python>=V3.9.6 with Pip>=22.0.1 at Line 455

Addresses AssertionError with Pip V>=22.0.1 on Python V>=3.9.6.
Contributor checklist

[x] Provided the tests for the changes.
[x] Assure PR title is short, clear, and good to be included in the user-oriented changelog

Maintainer checklist

[ ] Assure one of these labels is present: backwards incompatible, feature, enhancement, deprecation, bug, dependency, docs or skip-changelog as they determine changelog listing.
[ ] Assign the PR to an existing or new milestone for the target version (following Semantic Versioning).

@fsecada01 Hey, there's a typo on line 453
should handlers and not handles, shouldn't be ? A missing r

It seems that with pip>=22 pip-tools removes dash normalization https://pip.pypa.io/en/stable/news/#id73 even if it was done in pip==21.2 :roll_eyes:

Seems like this pull request resolves the issue more effectively and passes all tests: https://github.com/jazzband/pip-tools/pull/1567
I'll wait until the request is approved and merged so this PR can be closed.


trivial refactor: inlineScan => inlineScanModule

This rename is done because of the several inlineScan functions in inline.d that are different from each other, and so it's easier if they have different names.

Good renaming.

Auto-merge toggled on

@WalterBright is actually maIntainIng the code around inlining. And the renaming will make code more grep-friendly.  I think it's useful to him at least.


Upgrade Groovydoc.classpath to ConfigurableFileCollection

@bot-gradle test RFN

I've triggered the following builds:

Ready for Nightly (Trigger) (Check)

Build Scan

The following builds have passed:

Ready for Nightly (Trigger) (Check)

Build Scan


Running quantized nets other than person_detection_int8

Hello, I'm trying to run quantized mobilenets, other than the default one in the person_detection_int8 example, from https://www.tensorflow.org/lite/guide/hosted_models.
I turn a quantized mobilenet .tflite file I downloaded into a .cc file and replace lite/micro/tools/make/downloads/person_model_int8/person_detect_model_data.cc with it. I also tried increased the tensor arena size and changing model_settings.h appropriately. I compile and run the binary, but I'm getting this error:
tensorflow/lite/micro/kernels/softmax_common.cc:78 input->type != kTfLiteFloat32 (UINT8 != FLOAT32)
Node SOFTMAX (number 30f) failed to prepare with status 1
Type UINT8 (3) not supported.
Node CONV_2D (number 0) failed to invoke with status 1
Invoke failed.
person score:0 no person score 0

What am I doing wrong? Or is there another way to run such models in Tflite micro outside the person_detection example perhaps?

Ensure that your quantization is doing int8 quantization for the entire model (inputs and outputs).
Check this out Post-training quantization.

Type UINT8 (3) not supported.  --> is most most likely because you are passing unit8 to the model/interpreter and its expecting a different type.
input = interpreter->input(0);
printf("kTfLiteInt8 ?:  %d\n", input->type);

In  https://github.com/tensorflow/tflite-micro/blob/main/tensorflow/lite/c/c_api_types.h you will find:
// Types supported by tensor typedef enum {   kTfLiteNoType = 0,   kTfLiteFloat32 = 1,   kTfLiteInt32 = 2,   kTfLiteUInt8 = 3,   kTfLiteInt64 = 4,   kTfLiteString = 5,   kTfLiteBool = 6,   kTfLiteInt16 = 7,   kTfLiteComplex64 = 8,   kTfLiteInt8 = 9,   kTfLiteFloat16 = 10,   kTfLiteFloat64 = 11,   kTfLiteComplex128 = 12,   kTfLiteUInt64 = 13,   kTfLiteResource = 14,   kTfLiteVariant = 15,   kTfLiteUInt32 = 16, } TfLiteType;
so check that  input->type lines up with what you are inputing.

+1 Thanks for your help Stanley.
Please do check out the post training quantization link, especially the integer only quantization section.

Thanks for your replies.
Is it trivial to transform those models to use int8 instead?
If not, shouldn't https://www.tensorflow.org/lite/guide/hosted_models this page host int8 models instead of those current uint8 ones? Since tflite seems to be dropping support for uint8 overall.

Unfortunately converting unit8 models to int8 once they are in .tflite form isn't easy - I agree the hosted models should be converted. Those are primarily designed to run on TFLite not TFLM, which has backwards compatibility with uint8.


Please elaborate how to use Application Insights instead of AzureWebJobsStorage/AzureWebJobsDashboard

As per https://docs.microsoft.com/en-us/azure/azure-functions/functions-monitoring, we see that we can use Application Insights instead of a storage account for Functions Monitoring. Yet, in the ARM Schema's and sample ARM Templates, I cannot find any references how to do this through such a template. It would be great if that information could be added to this page. (My guess is that it would go though microsoft.insights/components...)

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: a1ba8817-3e01-7e84-6c71-b98b131f2927
Version Independent ID: a7703386-9e83-1075-72ef-f9e63f3dc4de
Content: Automate resource deployment for a function app in Azure Functions
Content Source: articles/azure-functions/functions-infrastructure-as-code.md
GitHub Login: @ggailey777
Microsoft Alias: glenga

Thanks for the feedback! We are currently investigating and will update you shortly.

Hey @tinod We have assigned this issue to the author to review and take the right course of action.

@tinod, thanks for the feedback. I have opened an internal issue to track your requested addition of Application Insights info to the ARM deployment topic (1355897). I also opened an issue against our reference templates to have them updated. Per the SLA, we can #please-close this issue.


Ring Two-Factor Authentication

Hi,
I have two-factor authentication (2FA) enabled for my Ring account. I've loaded the app and the drivers into Hubitat, and when I went to authenticate in the app, I was prompted for my credentials. I entered them and received a text from Ring with my verification code, but the app in Hubitat just said I had entered invalid credentials. It does not seem that 2FA is supported by this integration.
I would be willing to temporarily disable 2FA to set this up and then re-enable 2FA, but I don't want to have to disable 2FA permanently for this integration to work.
Thanks...

Is there any plan to support 2FA??

No. I haven't maintained this in a long time. There are much better Hubitat Ring integrations out there and I don't think keeping this one going really helps anything


Added --not-edited option to query | Added corresponding tests for query and export

Added --not-edited option to query (exclusive with --edited) to search for photos that have not been edited.
Added query test scenarios for --edited and --not-edited
Added export test scenarios for --edited and --not-edited

Local test run successfully, although the automated checks fail on the PR.
$ sw_vers
ProductName:	macOS
ProductVersion:	12.6.2
BuildVersion:	21G320

$ python --version
Python 3.9.6

$ python3 -m pytest -vv
=========================================== test session starts ============================================
platform darwin -- Python 3.9.6, pytest-7.2.0, pluggy-1.0.0 -- /Users/MSP/Documents/GitHub/osxphotos/venv/bin/python3
cachedir: .pytest_cache
rootdir: /Users/MSP/Documents/GitHub/osxphotos, configfile: pytest.ini
plugins: mock-3.10.0, cov-4.0.0
collected 1522 items / 1 skipped

tests/test_10_12_6.py::test_init PASSED                                                              [  0%]

(...)

tests/test_ventura_dev_preview_13_0_0.py::test_no_adjustments PASSED                                 [ 99%]
tests/test_ventura_dev_preview_13_0_0.py::test_moment[info0] PASSED                                  [100%]

=============== 1413 passed, 110 skipped in 163.66s (0:02:43) ==============

Local test run successfully, although the automated checks fail on the PR.

Looks like the test seg-faulted. I've noticed that sometimes happens on the Github actions and I think it's due to some config issue on their end (maybe the worker VM runs out of memory or something). I will re-run the tests.

Tests re-ran successfully so must have been an issue with the test runner.

Thanks for this! Bonus points for adding tests!  I've merged and will include in the next release.


JSON schema support for .jscsrc

From @scottaddie on November 22, 2015 4:23
Prior to the 0.10.1 release, the following settings were baked into the editor:
    // Control whether jshint is enabled for JavaScript files or not.
"jshint.enable": true,

// The jshint options object to provide args to the jshint command.
"jshint.options": {},

Now that there's an official JSHint extension, the following schema support should be added by it rather than including it into the editor:
    {
        "fileMatch": [
            "/.jshintrc"
        ],
        "url": "http://json.schemastore.org/jshintrc"
    },

I'd argue the same point regarding the JSCS extension. It should add the following to the settings, rather than baking it into the editor:
    {
        "fileMatch": [
            "/.jscsrc"
        ],
        "url": "http://json.schemastore.org/jscsrc"
    }

Copied from original issue: Microsoft/vscode#433

thanks @scottaddie !


Get workout routes for workout UUID

Issue: https://github.com/kvs-coder/health_kit_reporter/issues/78
Status
READY
Migrations
NO
Description
Getting workout routes with time interval Predicate starting and ending at the time of the workout doesn't explicitly return routes of this workout. We get some (or no) routes without any information in them to which workout they belong to.
Trying to get routes by providng the workout UUID returns the desired routes for each workout.
I've considered extending the current Predicate object and workoutRouteQuery function to take UUID as an additional argument, but it caused a lot of issues:

It would be hard to do it in the way that projects which are currently using the package wouldn't be affected
Passing UUID to the Predicate should mean that start and end date shouldn't be provided and vice versa (needed migration)
I've started with creating a new Predicate for workoutRouteQuery, that would either take start and end time or UUID. I've changed the map to return Map<String, dynamic> and swift code to take [String, Any] arguments map, but trying to parse arguments in workoutRouteQuery failed when any of the parameters was null. Using Map<String, String> and fallbacking to an empty string when any of the parameters was null didn't help, as parsing failed when trying to parse timestamps (and although it could be checked and avoided, I think it's a pretty nasty way of doing it)

Looking from another angle, when I passed the UUID to workoutRouteQuery, I  couldn't get it to work using
HKQuery.predicateForObject(with: uuid)

or any other variations of this method I've found when reserching the topic.
The solution I've seen the most is getting routes by providing a predicate from the workout itself. This is tricky in the current codebase, because I'd need concurrency to get the workout asynchronously, then get its routes and return them. This is available in iOS 13 and forcing projects already using workoutRouteQuery to update the iOS version is a very bad idea.
All of these reasons led me to my final decision: creating a new seperate function for getting routes by providing the workout UUID: workoutRouteForUUIDQuery with new UUIDPredicate.
This way, previous versions won't be affected and we'd get a new feature, available from iOS 13.
Related PRs
Not applicable.
Deploy Notes
Not applicable.
Steps to Test or Reproduce
git pull --prune
git checkout get-workout-routes-for-workout-uuid
Run example app - there is a new text button workoutRouteForUUIDQuery that gets workout routes for a provided UUID in workoutRouteForUUIDQuery method. I've added this to test whether the native connection works. Bear in mind that this package does not provide a way to save workout routes and it's tricky to provide any test data on iOS simulator, so this method will most likely return an empty list.
I've also noticed that although we pass the workout UUID when saving it, it gets its own UUID later on, so setting a valid UUID in saveWorkout method and then trying to get it in workoutRouteForUUIDQuery won't work, as the UUID will be changed.
Impacted Areas in Application

New method workoutRouteForUUIDQuery and new UUIDPredicate
Example app now targets iOS 13

List general components of the application that this PR will affect:

HealthKitReporter (new method)
Example app

hi @karolinaoparczyk
please sorry for my long absence, I have just merged you PR https://github.com/kvs-coder/health_kit_reporter/pull/77

hi @kvs-coder :)
Thanks for merging #77, but it needs https://github.com/kvs-coder/HealthKitReporter/pull/28 merged too to work :)
I've merged this PR with the new changes, have you got a chance to look at this new feature?


Way to output the notification content to a local file

Hello,
I am struggling to filter contents of a "custodian run" output to contain just the resource, policy name and the violation/trigger.
I tried to create a report off of the dryrun and I see that there is a lot of data that cannot be consumed. I can write to sqs and read it via lambda but looking for a simpler solution like read from a local file.
Is there/should there be a way to output the notification content to a local file (in the same structure as -s /tmp/sample/  option?
I can then create a concise report based on this.
Appreciate your input.
Shekar

hm, perhaps adding an additional local file transport type and have that push all the messages into a json file could work

Hmm.. not sure I get it
policies:
- filters:
  - type: metrics
    name: CPUUtilization
    days: 4
    value: 10
    op: less-than
  - type: metrics
    name: NetworkIn
    days: 4
    value: 500
    op: less-than
  actions:
  - type: notify
    transport:
      type: local
      directory: /tmp/notify.json
  description: |
    Yellow: An instance had 10% or less daily average CPU utilization and 5 MB orless network I/O on at least 4 of the previous 14 days.
  name: low-utilization-amazon-ec2-instances
  resource: ec2

Thats my policy which results in an error:
Exception: Failed to validate on policy 'metrics' is not one of ['event']

Failed validating 'enum' in schema[0]['properties']['type']:
    {'enum': ['event']}

On instance['type']:
    'metrics'
 {'filters': [{'type': 'metrics', 'name': 'CPUUtilization', 'days': 4, 'value': 10, 'op': 'less-than'}, {'type': 'metrics', 'name': 'NetworkIn', 'days': 4, 'value': 500, 'op': 'less-than'}], 'actions': [{'type': 'notify', 'transport': {'type': 'local', 'directory': '/tmp/notify.json'}}], 'description': 'Yellow: An instance had 10% or less daily average CPU utilization and 5 MB orless network I/O on at least 4 of the previous 14 days.\n', 'name': 'low-utilization-amazon-ec2-instances', 'resource': 'ec2'} is not valid under any of the given schemas

Failed validating 'anyOf' in schema['properties']['policies']['items']:

Sorry, didn’t mean to imply that local is a transport that’s currently supported, meant to say that might be a pattern we could implement

a couple of things, cli reporting supports adding arbitrary resource fields to its output using the --field, albeit escaping in jmespath and on the shell gets a bit obtuse for some of the annotations prefixes we use for why a given resource matched a filter.
we also support an invoke-lambda action for one off custom integrations on sending the policy name, resource, why the resource matched a given policy to an integration lambda.


chore: Migrate SqlLab API to API v1

SUMMARY
Created api/v1/sqllab/API for future SPA migration of SQL Lab
Migrated superset/sql_json/ to /api/v1/sqllab/execute/sql/
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Required feature flags:
[ ] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

Superseded by https://github.com/apache/superset/pull/22809, closing

I'm trying to use the SqlLab API that has recently been migrated to api/v1/sqllab and am having some difficulties. I can get the api/v1/sqllab/execute endpoint to work fine but don't seem to be able to successfully do an async execute followed by a api/v1/sqllab/results call to get the resulting data rows of the execute. Also I'd like to get the rows in chunks instead of all at once but am unsure if this is supported. Example Python code of an async execute followed by one or more results calls would be very helpful.


Add more descriptive titles instead of the default

It would be great if the favicon includes a countdown of the days left to the next meetup.
I envision the Montevideo Tomster plus the number of days left 😛 . Similar to this


[BUG] searching for multiple indices fails without tenant permissions

Describe the bug
Whenever I try to search for multiple indices, for example, GET /*,-.*/_search (i.e. query containing ,), it will always fail due to the below error, even with unlimited cluster/index permission.
{
  "error": {
    "root_cause": [
      {
        "type": "security_exception",
        "reason": "no permissions for [indices:data/read/search] and User [name=unlimited, backend_roles=[], requestedTenant=null]"
      }
    ],
    "type": "security_exception",
    "reason": "no permissions for [indices:data/read/search] and User [name=unlimited, backend_roles=[], requestedTenant=null]"
  },
  "status": 403
}

The user has unlimited permission for both cluster and index permissions.

If I only search for single index, for example, GET /*/_search, it successes without any problem.
Related component
Plugins
To Reproduce

Create a user with a role with unlimited permission for both cluster and index permissions.

Expected behavior
The query successes.
Additional Details
Plugins
I'm using Amazon OpenSearch Service 2.11 as-is.
Screenshots
If applicable, add screenshots to help explain your problem.
Host/Environment (please complete the following information):

OS: [e.g. iOS]
Version [e.g. 22]

Additional context
After I configured tenant permission for the role as below, the error was gone. It was ok even if the tenant pemission did not have any actions assigned.

I think it's an undocumented behavior. Is it even expected?

[Triage] Hi @tmokmss thanks for filing this issue. Could you please provide some additional information about your scenario to help diagnose this issue?
Specifically, could you share:

The roles definition for the user you are using
Whether this issue occurs without putting the pattern into the Search request HTTP URI
What the expected username for the active user is (it says "unlimited" but it also shows you roles as that)

Thanks

@scrawfor99

The roles definition for the user you are using

The original role definition:
{
  "cluster_permissions": [
    "unlimited"
  ],
  "index_permissions": [{
    "index_patterns": [
      "*"
    ],
    "allowed_actions": [
      "unlimited"
    ]
  }]
}

Then I added tenant_permissions like this:
{
  "cluster_permissions": [
    "unlimited"
  ],
  "index_permissions": [{
    "index_patterns": [
      "*"
    ],
    "allowed_actions": [
      "unlimited"
    ]
  }],
  "tenant_permissions": [{
    "tenant_patterns": [
      "*"
    ],
    "allowed_actions": [
    ]
  }]
}

Whether this issue occurs without putting the pattern into the Search request HTTP URI

I haven't tested it. At least the issue reproduces when using pattern like this: GET /*,-.*/_search? Can you try this?

What the expected username for the active user is (it says "unlimited" but it also shows you roles as that)

I named the user unlimited when I reproduced it, but I guess user name is not relevant to the issue.

@peternied Sorry, I don't understand what you mean. I'm talking about tenant permission, not cluster permission. I suspect it is a functionality bug that an empty tenant permission is required to search indices, while the error message is just saying no permissions for [indices:data/read/search].

@tmokmss Please read the documentation I've referenced and follow up in the forums.  If you have trouble getting traction there please feel free to tag me on this issue.

@peternied
Do you think "cluster_permissions": [ "unlimited" ] causing this issue? Then why adding an empty tenant permission makes the request success? I'm afraid the document does not answer the question, and there's no related post in the forum.

@peternied Hi, I found this doc stating that unlimited on cluster permission grants complete access to action groups. Apparently it contradicts what you said.  Can you confirm which is true?

"cluster_permissions": [ "unlimited" ], this doesn't impart permissions

unlimited | Grants complete access to action groups. Can be used on an cluster- or index- level. Equates to “*”.


Pin zipp

Checklist

[x] Used a personal fork of the feedstock to propose changes
[x] Bumped the build number (if the version is unchanged)
[x] Reset the build number to 0 (if the version changed)
[ ] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

Solves problem with installation on python 3.7 caused by zipp dropping compatibility.

@conda-forge-admin, please rerender


Refactor Consensus

Classes implementing the consensus process on Ripple are cleaned
up in preparation for modularizations and compartmentalization.
Functions and state related to inter-round consensus are moved out
of NetworkOPs and into Consensus, where they are more effectively
isolated.
Some member functions are changed to free functions and some free
functions have their scope reduced to specific translation units.

Track inter-round consensus state using new Consensus object
Devirtualize interfaces
Reduce NetworkOPs, Consensus and LedgerConsensus interfaces
Add comments

The first commit was previously reviewed, but feel free to eyeball it. Reviewers: @vinniefalco, @JoelKatz, @HowardHinnant, @ximinez

Travis failure is valid. Build error on nounity build. https://travis-ci.org/ripple/rippled/jobs/68054407

Oh nounity build... you vex me so! Will fix.

Looks good, other than the broken build, obviously.

:+1:

:+1:
I'm in the middle of removing Serializer from SHAMapItem.  There are some serious merge issues between that work and this.  I'm going to table my work until this gets into develop.

I like that NetworkOPs is smaller. This would be even better if the interfaces were de-virtualized but that can wait for another pull request. I'm assuming that the logic for the large blocks of moved code was not changed. Given that,
:+1:

:+1:


Loading screen?

Do we need a loading screen?  Not even sure how to make one with z88dk!

This is useful info on adding a loading screen: https://www.z88dk.org/forum/viewtopic.php?p=15821&sid=dcde76f0aef28477f2c4e6ef0a3932ca#p15821


Bug: Unable to get bluetooth working in a Docker Instance

Describe the Bug
I've been trying to use BLE instead of OpenAPI for a while, and haven't been able to find the reason why I can't.
The Switchbot error I get is:
[8/28/2023, 1:13:04 PM] [SwitchBot] Curtain: Backyard failed BLERefreshStatus with BLE/OpenAPI Connection, Error Message: "Adapter is not ready: unknown"
[8/28/2023, 1:13:04 PM] [SwitchBot] Curtain: Backyard wasn't able to establish BLE Connection, node-switchbot: [object Object]

Setup

Docker.
network_mode set to host
dbus mounted as readonly from /run/dbus.

I currently have the following to the start-up script, in case that made a difference.  I didn't expect the noble addition to do anything, but bluetooth at least let me verify Bluetooth can be accessed within the container:
  apt update
  apt install -y bluetooth bluez libbluetooth-dev libudev-dev
  npm install @abandonware/noble

What I've tried

I run Switchbot in a child bridge, but I've tried with & without, and get the same result.
Installing the bluetooth packages (see below).  Within the Homebridge terminal, I can scan & see devices (bluetoothctl scan on).  As far as I can tell, Docker isn't the issue here.    bluetoothctl scan on
    ....
    [NEW] Device AA:00:11:22:33:DD AA-00-11-22-33-DD
    ....

Running the Docker container with privileged set to true, didn't make a difference.  But as mentioned above, I believe that's not the issue.
Clearing cached accessories in Homebridge, and turning off caching in the Switchbot device.

To Reproduce
My
services:
  homebridge:
    image: oznu/homebridge
    container_name: homebridge
    restart: always
    network_mode: host
    environment:
      HOMEBRIDGE_CONFIG_UI_PORT: 80
      TZ: Australia/Sydney
    volumes:
      - /home/pi/homebridge:/homebridge
      - /run/dbus:/run/dbus:ro
      - /etc/localtime:/etc/localtime:ro

Expected Behavior
BLE is able to be used, over
Relevant Log Output
[8/28/2023, 1:52:19 PM] [SwitchBot] Total SwitchBot Devices Found: 7
[8/28/2023, 1:52:19 PM] [SwitchBot] Restoring existing accessory from cache: Meter Plus DeviceID: VALID_ID_1
[8/28/2023, 1:52:19 PM] [SwitchBot] Restoring existing accessory from cache: Verandah DeviceID: VALID_ID_2
[8/28/2023, 1:52:19 PM] [SwitchBot] Restoring existing accessory from cache: Backyard DeviceID: VALID_ID_3
[8/28/2023, 1:52:19 PM] [SwitchBot] [DEBUG] Curtain: Backyard Using Device Config Logging: debug
[8/28/2023, 1:52:19 PM] [SwitchBot] [DEBUG] Curtain: Backyard Using Platform Config refreshRate: 120
[8/28/2023, 1:52:19 PM] [SwitchBot] [DEBUG] Curtain: Backyard Using Default Curtain updateRate: 7
[8/28/2023, 1:52:19 PM] [SwitchBot] [DEBUG] Curtain: Backyard Using Default scanDuration: 7
[8/28/2023, 1:52:19 PM] [SwitchBot] [DEBUG] Curtain: Backyard Config: {"set_minStep":10,"hide_lightsensor":true,"connectionType":"BLE/OpenAPI","external":false,"logging":"debug"}
[8/28/2023, 1:52:19 PM] [SwitchBot] [DEBUG] Curtain: Backyard BLERefreshStatus
[8/28/2023, 1:52:19 PM] [SwitchBot] [DEBUG] Curtain: Backyard accessory.context.FirmwareRevision: "V6.3"
[8/28/2023, 1:52:19 PM] [SwitchBot] [DEBUG] Curtain: Backyard device.firmware: undefined
[8/28/2023, 1:52:19 PM] [SwitchBot] [DEBUG] Curtain: Backyard this.platform.version: 2.1.1
[8/28/2023, 1:52:19 PM] [SwitchBot] [DEBUG] Curtain: Backyard setFirmwareRevision: "V6.3"
[8/28/2023, 1:52:19 PM] [SwitchBot] [DEBUG] Curtain: Backyard accessory.context.FirmwareRevision: "V6.3"
[8/28/2023, 1:52:19 PM] [SwitchBot] [DEBUG] Curtain: Backyard device.firmware: undefined
[8/28/2023, 1:52:19 PM] [SwitchBot] [DEBUG] Curtain: Backyard this.platform.version: 2.1.1
[8/28/2023, 1:52:19 PM] [SwitchBot] [DEBUG] Curtain: Backyard setFirmwareRevision: "V6.3"
[8/28/2023, 1:52:19 PM] [SwitchBot] [DEBUG] Curtain: Backyard Removing Light Sensor Service
[8/28/2023, 1:52:19 PM] [SwitchBot] [DEBUG] Curtain: Backyard BLE Address: aa:00:11:22:33:dd
[8/28/2023, 1:52:19 PM] [SwitchBot] [DEBUG] Curtain: Backyard updateCharacteristic CurrentPosition: 100
[8/28/2023, 1:52:19 PM] [SwitchBot] [DEBUG] Curtain: Backyard updateCharacteristic PositionState: 2
[8/28/2023, 1:52:19 PM] [SwitchBot] [DEBUG] Curtain: Backyard updateCharacteristic TargetPosition: 100
[8/28/2023, 1:52:19 PM] [SwitchBot] [DEBUG] Curtain: Backyard updateCharacteristic BatteryLevel: 74
[8/28/2023, 1:52:19 PM] [SwitchBot] [DEBUG] Curtain: Backyard updateCharacteristic StatusLowBattery: 0
[8/28/2023, 1:52:19 PM] [SwitchBot] [DEBUG] Curtain: Backyard updateCharacteristic FirmwareRevision: "V6.3"
[8/28/2023, 1:52:19 PM] [SwitchBot] Curtain: Backyard failed BLERefreshStatus with BLE/OpenAPI Connection, Error Message: "Adapter is not ready: unknown"
[8/28/2023, 1:52:19 PM] [SwitchBot] Curtain: Backyard wasn't able to establish BLE Connection, node-switchbot: [object Object]
[8/28/2023, 1:52:19 PM] [SwitchBot] Curtain: Backyard Using OpenAPI Connection to Refresh Status

Config for homebridge-switchbot
"devices": [
                    {
                        "deviceId": "<MyDeviceId>",
                        "configDeviceName": "Backyard",
                        "hide_device": false,
                        "configDeviceType": "Curtain",
                        "connectionType": "BLE/OpenAPI",
                        "curtain": {
                            "set_minStep": 10,
                            "hide_lightsensor": true
                        },
                        "history": true,
                        "external": false
                    }
]

Screenshots
No response
Device and Model
Switchbot Curtain
Node.js Version
v18.17.1
NPM Version
v9.6.7
Homebridge Version
v1.6.1
Homebridge Switchbot Plugin Version
v2.8.2
Homebridge Config UI X Plugin Version
v4.50.5
Operating System
Docker

Bluetooth issues have nothing to do with the plugin, checkout the Bluetooth Plugins Wiki.

If set to BLE/OpenAPI, BLE will be tried first and if it doesn't work then it will use OpenAPI as a backup.
If set to BLE, then only BLE will be tried.

I found this issue when I searched for the Adapter is not ready: unknown error that I got trying to get the SwitchBot Indoor/Outdoor Thermo-Hygrometer to work. After some research I found the node-switchbot library to cause the error (even when using it in a test script independently of homebridge). I then created a small test script which did indeed work (in the same docker container). So the problem is not one of bluetooth setup but is indeed related to the node-switchbot package.
For anyone who tries to get aforementioned sensor to work might find my  my plugin to be of use :)

@moritzmhmk Why not help update this plugin to not rely on node-SwitchBot?

Adding support for all the different SwitchBot devices with a new code base would require a lot of testing and I do only own the Indoor/Outdoor Thermo-/Hygrometer. I did try to fix node-switchbot for about 2 hours before I decided to write something from scratch - which did happen to be surprisingly simple - so simple indeed that I decided the effort of creating a new plugin was worth it ;)
In general I think that there is space for multiple different plugins. While this one is best suited for someone who operates multiple SwitchBot devices and also has a SwitchBot Hub, my plugin targets those users that just want to get a cheap (but good) BLE Hygrometer running.

but couldn't you just add that support directly into the the iosensor file in this plugin?

you don't have to change it on all of them....

Handling bluetooth differently for the different device types would be a rather unclean solution, since it would still cause the problems for anyone who wants to use a different device type. Imagine setting the plugin up and having all but the iosensor devices fail with the Adapter is not ready: unknown error.

is it really a good idea to mark my comments that hint to a solution as spam?

Now that I had a working solution at hand, I investigated why my fixing of the node-switchbot package failed.
I found that there were three different problems:

BLE support failed because the status in the init function of node-switchbot was not "poweredOn" and no waiting for stateChange was implemented
after the problem was fixed, the device still did not work because the identifier in iosensor.ts was wrong
with the correct identifier it still failed as node-switchbot does not support the sensor

I hope the related PRs are helpful to fix this issue.


Fix typo in doc string

What does this PR do?
Fix typo in doc string (emtpy -> empty)
Checklist

[x] This PR fixes a minor issue (e.g.: typo or small bug) or improves the docs (you can dismiss the other
checks if that's the case).

Codecov Report

Merging #1965 (2ef6dbd) into main (55041e9) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1965   +/-   ##
=======================================
  Coverage   71.26%   71.26%
=======================================
  Files          59       59
  Lines        4980     4980
=======================================
  Hits         3549     3549
  Misses       1431     1431

Impacted Files
Coverage Δ

flax/core/scope.py
89.17% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 55041e9...2ef6dbd. Read the comment docs.