chore: Create PR to Ionic templates

We should try to create a PR to https://github.com/ionic-team/capacitor-solidjs-templates with a new template(s) utilizing our solution.

I might try something this weekend.

I'm creating an app today so I'll prepare the template.
I created templates repo.

Ï pushed the template.
@spion Can you test it?
How to use it is described in the project root README.
PS: There is some issue with Tabs. You cannot see content of other tabs. You can see the first one after page (re)load, but other tabs have some awkward offset that push the content out of the screen. I had no time to investigate.
FYI: @erfanmola

There is also
Error: Cannot find module @rollup/rollup-linux-x64-gnu. npm has a bug related to optional dependencies (https://github.com/npm/cli/issues/4828). Please try `npm i` again after removing both package-lock.json and node_modules directory.


reduce the scope of wl_addr

reported by cppcheck 1.78

after discussing with Alin we decided to leave this as is.
the int is guaranteed to be used


Fix methods for pulse parametrizations

The PulseParametrizations sub-module was defining new methods for evaluate and get_control_deriv without importing them, thus making the definitions ineffective

Codecov Report

Merging #52 (e05807c) into master (116a007) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head e05807c differs from pull request most recent head d5f47a1. Consider uploading reports for the commit d5f47a1 to get more accurate results

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@           Coverage Diff           @@
##           master      #52   +/-   ##
=======================================
  Coverage   76.01%   76.01%
=======================================
  Files          12       12
  Lines         888      888
=======================================
  Hits          675      675
  Misses        213      213

Impacted Files
Coverage Δ

src/pulse_parametrizations.jl
52.72% <ø> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Missing documentation on USING this self service sign up?

How do you initiate the sign-up flow once you've made it?
In B2C it gives you the ability to generate the link to test the flow, but there is nothing like that in the ui, and nothing related to using the flow is documented here?
Just the format of the URL that the app needs to redirect to to initiate the flow would be useful.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 2c05d9cd-275d-4585-f25f-fa5b88f1ec10
Version Independent ID: a3d263e6-20a6-9cea-f60c-a9f772a7f444
Content: Add a self-service sign-up user flow - Microsoft Entra
Content Source: articles/active-directory/external-identities/self-service-sign-up-user-flow.md
Service: active-directory
Sub-service: b2b
GitHub Login: @msmimart
Microsoft Alias: mimart

I sorted this out with chatGPT... it looks like it might be correct, but would love more specific docs here:
https://login.microsoftonline.com//oauth2/v2.0/authorize?p=&client_id=&redirect_uri=&response_type=code&scope=openid&nonce=defaultNonce&prompt=login

@fei0x
Thanks for your feedback! We will investigate and update as appropriate.

@fei0x
Thank you for your suggestion. It would be helpful to provide your ideas in the below link so our product team will work on it.
https://feedback.azure.com/d365community

Hmm? What does the product team have to do with this?
I am suggesting you update the documentation to inform readers of how to use this existing and working feature. Isn't that why this issue forum exists?
When you create a flow. You need to know the URL to initiate it so that your Application can redirect users to it at sign in. This URL is given to you in B2C GUI so it's easy, but in this case it is not so you need to tell us how to formulate it.
B2C has this in their docs... though, this was a pain to find too:
https://learn.microsoft.com/en-us/azure/active-directory-b2c/openid-connect#send-authentication-requests

@fei0x
I'm going to assign this to the document author so they can take a look at it accordingly
@msmimart
Could you Please review this and update as appropriate.


Support reverse geocoding

Nominatim and OCD support reverse geocoding and we should parse this as well

Thanks @karussell. We should support reverse geocoding if the provider supports it.

I just started with the reverse geocoding. Something I just found out is that if you query a reverse geocoding request to Nominatim you get one result (at least in my current test url). GraphHopper returns multiple results.

@karussell could you review my changes in #12 please :)

Should be fixed with #12 and #16


Don't install tests

Please follow the guide below

You will be asked some questions, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your pull request (like that [x])
Use Preview tab to see how your pull request will actually look like

Before submitting a pull request make sure you have:

[x] At least skimmed through adding new extractor tutorial and youtube-dl coding conventions sections
[x] Searched the bugtracker for similar pull requests
[ ] Checked the code with flake8

In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:

[x] I am the original author of this code and I am willing to release it under Unlicense
[ ] I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

[x] Bug fix
[ ] Improvement
[ ] New extractor
[ ] New feature

Description of your pull request and other information
Installing the test suite on the user machine is an useless waste of space and a source of confusion, also forbidden by Gentoo automated QA for Python packages:
Package installs 'test' package which is forbidden and likely a bug in the build system.

I actually had tests included. But when I compared the binaries built with and without tests there was actually no difference in size.

What binaries? The issue is about python packages.


Change the wrong item message on getOSItem

Description:

Changes:

Change the message to The item **itemName** does not exist.

Other checks:

[X] I have tested all my changes thoroughly.

Should only do this in dev.


Inconsistent usage of max_list_len

@yuyiguo @ticoann this is the issue I mentioned yesterday, which is hitting us in production right now.
It seems we have an inconsistent usage of this max_list_len (for lumi_list for instance) between dbs client and server.
In the dbsClient code, we check the length of the list regardless whether it's a plain or a nested list (cannot be bigger than 1000, otherwise we slice it):
https://github.com/dmwm/DBS/blob/d58e4fe7be87a6f84da4f246523641c9988ba1ef/Client/src/python/dbs/apis/dbsClient.py#L1009
on the server side, we first expand this lumi_list in a plain list and then AND only then we check it against
the max length
https://github.com/dmwm/DBS/blob/00f44b9b32cefd79888eb1ded0ab8e359ab47965/Server/Python/src/dbs/web/DBSReaderModel.py#L910
where the following nested list
[[248, 494], [534, 546], [638, 650], [794, 1313], [1327, 1339], [1353, 1482], [1496, 1573], [1652, 1664], [1743, 1755], [1860, 1872]]
has 1053 elements, thus DBSServer returns a Bad Request.
Before putting more effort on it, I'd like to know whether we should fix it on the DBS side
a. checking their length before any expansion
b. if passes previous test but it's bigger, either go ahead or slice on the client side
or fix it in WMCore
a. always expand from range of lumis to a plain lumi list and make POST calls with this plain list.

Hi Alan, it seem this should fixed in DBS side, it sounds like dbs client call slices the parameters assuming which should accept by server but DBS server rejects it due to Oracle constraint. Is that correct?
Since it is hitting production now, so we should have temporary patch in WMAgent side but I think it should be eventually fixed in DBS side. Otherwise, it will have the problem whatever application which uses the same client api. @yuyiguo, Yuyi, what do you think?

Almost :) It does not really reach oracle. In the client side we have a list of lists where its length is smaller than 1000, so no slicing is needed here. However, the first thing done on the server side is expanding this list of lists in a plain list, which goes beyond the 1000 limit. Since the server does not apply any slices, it simply raises an exception in this case:
https://github.com/dmwm/DBS/blob/00f44b9b32cefd79888eb1ded0ab8e359ab47965/Server/Python/src/dbs/web/DBSReaderModel.py#L929
I agree with you, it would be better to fix it in the DBS side. I'll create a temporary agent patch tomorrow.

Actually, there is no temporary fix nor workaround in WMAgent, since this problem involves only central services and GQ fails to acquire work (start policy).
Yuyi, do you think you can fix it by next Testbed? Otherwise I'll add something in WMCore...

Yuyi made the fix in the end of 2015 :-)


How to change the code in Prgram.ps in dotnet?

Problem encountered on https://dotnet.microsoft.com/en-us/learn/dotnet/hello-world-tutorial/edit
Operating System: windows
Provide details about the problem you're experiencing. Include your operating system version, exact error message, code sample, and anything else that is relevant.

You can open the Program.cs in any kind of editor like Notepad, Vs Code, etc.
Does that help?


USB OTG (Mass Storage Gadget)

Hi !!
First thanks a lot for your work on this project, your builds work very nicely.
I use 0.9.14 on Rock64 and I would like to use the USB OTG capability, you marked it working on the compatibility list.
I'm trying to make a USB Mass Storage Gadget like stated in this guide using:
# modprobe g_mass_storage file=/root/data/backing_file

However it returns:
modprobe: FATAL: Module g_mass_storage not found in directory /lib/modules/4.4.190-1233-rockchip-ayufan-gd3f1be0ed310

Apparently the g_mass_storage module is not compiled as part of default kernel configuration.
So I wonder how you use USB OTG in your build? (I will push a guide if I can get it to work)
Thanks!

I think it is, but it is no longer g_mass_storage, but rather: https://github.com/ayufan-rock64/linux-package/blob/master/root/usr/local/sbin/install_gadget#L33.
To use USB OTG, you can checkout those:

https://github.com/ayufan-rock64/linux-package/blob/master/root-rock64/usr/local/sbin/rock64_enable_eth_gadget.sh
https://github.com/ayufan-rock64/linux-package/blob/master/root/usr/local/sbin/install_gadget
Theoretically, it should be enough to change rock64_enable_eth_gadget.sh accordingly

@ayufan many thanks for your help!
You're using ConfigFs indeed.
After messing around for hours trying to understand why it didn't work, it was assigning OTG to the USB3.0 port instead of the dedicated USB2.0 OTG port of the Rock64... 😁
We're almost there. Any idea how we can redirect the UDC to the USB2.0 OTG port ?
(I would like to use USB3.0 for an external drive)

Hmm. Strange. It should work.

We're almost there. Any idea how we can redirect the UDC to the USB2.0 OTG port ?

Can you check which DTS node is updated, maybe this is culprit?

Hi,
Hey @ayufan: can you ever care to respond at least issues like #463 ?
Thanks.

Ok I managed to get it work on port USB 2.0 OTG (it is device usb@ff580000, we can check that running usb-devices).
For USB 2.0 we have to use dwc2 instead of dwc3.
Here is the script:
#!/bin/bash

set -xe

# enable peripheral mode
enable_dtoverlay dwc2_peripheral usb@ff580000 okay 'dr_mode="peripheral"'

# reload dwc2
echo ff580000.usb > /sys/bus/platform/drivers/dwc2/unbind
echo ff580000.usb > /sys/bus/platform/drivers/dwc2/bind

# set the drive to use with mass storage gadget
mkdir -p /sys/kernel/config/usb_gadget/g1/functions/mass_storage.0
echo /dev/sda3 > /sys/kernel/config/usb_gadget/g1/functions/mass_storage.0/lun.0/file

# install and activate mass storage gadget
install_gadget my_usb_drive_name ff580000.usb mass_storage

You need to change /dev/sda3 to the disk partition you want to use as a mass storage.
It can be for example a partition of a USB drive that is plugged into the USB 3.0 port.
:warning: The USB host will not expect to see any changes in the mass storage other than the ones it makes.

Extraneous changes are liable to corrupt the filesystem and may even crash the host. Only one system (normally, the USB host) may write to the backing storage, and if one system is writing that data, no other should be reading it. The only safe way to share the backing storage between the host and the gadget's operating system at the same time is to make it read-only on both sides. linux-usb.org

As a result, while it's technically not impossible, it's hopeless to simultaneously mount/share the drive on a network, and use it via USB.

(uninstall script:)
#!/bin/bash

set -x

# disable peripheral mode
disable_dtoverlay dwc2_peripheral

# reload dwc2
echo ff580000.usb > /sys/bus/platform/drivers/dwc2/unbind
echo ff580000.usb > /sys/bus/platform/drivers/dwc2/bind

# uninstall gadget
uninstall_gadgets


WIP: fixed price sale with admin

Please note that the admin account in the marketplace test is an
account that is not included in the sandbox. This has to be manually
created. If you wish to run this test, please consult me (Emmanuel) for
extra details.

I'm going to be closing this PR. All of these changes (along with some others) will be featured in another branch integrating all of the contracts that both Eli and I have written.


Stack is not created if a dependency within the stack fails

Bug description
I am trying to create a stack with two containers. Container 2 has a service_completed_successfully condition dependency on container 1. Container 1 fails to start. I then end up with the stack not having been created, the browser is still in the "add stack" view which shows me the stack I am currently creating (as if I had posted invalid syntax or another input error). Container 1 is in state 'failed', and Container 2 is in state 'created', both are not connected to any stack. If I want to try creating the stack again I will get an error that the container names are already taken. I need to remove the containers in order to try creating the stack again
Expected behavior
The stack is created, even though Container 1 fails and Container 2 stays in state 'created'.
In my case, the failing of Container 1 is a firewall issue that I am waiting for the network adminsitrators to fix. I would like to "save" the stack in Portainer so I can retry tomorrow, instead of having to enter it again tomorrow when the firewall would allow this.
Portainer Logs
Provide the logs of your Portainer container or Service.
You can see how here
Steps to reproduce the issue:

In an environment, go to Stacks -> Add Stack
Enter a stack where one service is a short-lived CLI tool that prepares the data used by the other service, and that other service has a service_completed_successfully dependency on the preparation service. I have attached an example below.
Click on "Deploy the stack"
Wait for deployment to fail and show an error message.

Technical details:

Portainer version: 2.14.2
Docker version (managed by Portainer):
Platform (windows/linux): linux
Command used to start Portainer (Ansible playbook):

- name: setting up portainer docker container
  docker_container:
          name:  portainer
          image: "portainer/portainer-ce:{{portainer_version}}"
          state: started
          restart_policy: always
          published_ports:
                  - "8000:8000"
          env:
                  http_proxy: "{{proxy_env['http_proxy']}}"
                  https_proxy: "{{proxy_env['https_proxy']}}"
                  no_proxy: "{{proxy_env['no_proxy']}}"
                  TZ: "{{timezone}}"
          volumes:
                  - "/var/run/docker.sock:/var/run/docker.sock"
                  - "portainer_data:/data"
                  - "/etc/localtime:/etc/localtime"
                  - "/etc/timezone:/etc/timezone"
          networks:
                  - name: traefik
          labels:
              com.centurylinklabs.watchtower.enable: "true"
              traefik.enable: "true"
              traefik.http.routers.portainer.rule: "PathPrefix(`/`)"  # Host(`{{ansible_facts['hostname']}}`)"
              traefik.http.routers.portainer.tls: "true"
              traefik.http.routers.portainer_insecure.rule: "PathPrefix(`/`)"  # Host(`{{ansible_facts['hostname']}}`)"
              traefik.http.services.portainer.loadbalancer.server.port: "9000"
          recreate: "{{portainer_upgrade}}"
          pull: "{{portainer_upgrade}}"
  register: docker_container
  when: portainer_upgrade or not portainer_container.exists

Browser: Chrome 105.0.5195.102
Use Case: Using Portainer in a Commercial setup.
Have you reviewed our technical documentation and knowledge base? Yes

Additional context
Here is what I am trying to run:
version: "3.5"

services:
  register:
    restart: 'no'
    image: gitlab/gitlab-runner:latest
    volumes:
      - config:/etc/gitlab-runner
    command:
      - register [..args..]
  runner:
    restart: always
    image: gitlab/gitlab-runner:latest
    depends_on:
      register:
        condition: service_completed_successfully
    volumes:
      - config:/etc/gitlab-runner
      - /var/run/docker.sock:/var/run/docker.sock

volumes:
  config:

The host that I want to run this on cannot reach the gitlab instance, and therefor the register command fails (which is expected behavior).

@a-gerhard
Thank you for the information.  I am going to further investigate and work on reproducing.  I will update you as I learn more.
Thanks!

@fgierlinger
Upon release of 2.17.1 and upgrade, can you please provide feedback?
Thanks!

@a-gerhard
I wanted to follow up on this request.  Have you tested this in later versions of Portainer?  If anything in the Stack fails, Portainer does a clean-up and removes everything in the Stack.
Thanks!


Deterministic chunk mesh ordering

I added parallel chunk updates without thinking very hard about the consequences, and now the glTF export fails nondeterministically, because ordering in the file is determined by ordering of the parallel operations.
Possible solutions:

Have the glTF exporter sort the meshes by their MeshIds.
Give ChunkedSpaceMesh a method to request deterministic updates (which could still be calculated in parallel, but with sorted sequential callbacks).

Fixed by sorting in 590de4d3b4bbedf5552220d519c461f11424c450.


CIV 4712 - Hearing Fee Payment scheduler

JIRA link (if applicable)
https://tools.hmcts.net/jira/browse/CIV-4712
Change description
Added classes that tie in to the scheduler for hearing fee payment.
They'll be filled with logic by another ticket.
Does this PR introduce a breaking change? (check one with "x")
[ ] Yes
[x] No

This ticket is blocked by another ticket, but is still necessary to distinguish between happy and unhappy path in hearing readiness.


Users can add new fields

Some small UI changes to the add field screen and connected it to the API call.

Realized I forgot to pull crops from firebase. One sec

Now it should be fine

@RD211 could you link relevant issue(s) to this PR?

Great work! Nice that you documented relevant widgets and functions 😄


Add report for matched content changes

https://trello.com/c/IAj6XWFo/309-create-a-report-to-show-the-content-changes-that-have-triggered-notifications
This adds a report to output content changes by matching subscriber
list over a given period, calculating the number of emails each match
would have generated using a similar technique to the "active_on" scope
on the Subscription model. We want to use this report to see clusters of
changes that contribute to high email volume.

@kevindew thanks for taking a look. It was a tricky query to write! The extra columns without the select was something I looked into and saw that they turned up anywhere. My assumption was that they came from the includes, and it's surprising to learn they originate from where.
I've added an explicit select for them, which takes effect irrespective of where, and should make it easier for people to understand in future.


Multiomics partitions

[x] Additional function parameters / changed functionality / changed defaults?
[ ] New analysis tool: A simple analysis tool you have been using and are missing in sc.tools?
[ ] New plotting function: A kind of plot you would like to seein sc.pl?
[ ] External tools: Do you know an existing package that should go into sc.external.*?
[ ] Other?

As more and more technologies allow multimodal characterization of single cells it could be useful to  exploit some functionalities of scanpy's toolkit to perform, at least, some rough integrative analysis. Assuming we have to modalities on different layers (say RNA and ATAC), one could create two knn graphs for both layers and use leidenalg.find_partition_multiplex to perform a joint call of partitions handling the two (or more) graphs as a multiplex. I have tested myself this approach, described in leidenalg documentation, it works and it is highly configurable.
We can take care of the implementation of enhancement (as leiden_multiplex() function?), I just want to be sure that it is not already on the development roadmap and that it is ok to have it into scanpy and not as an external tool.

I realize that I'm working on a specific dataset that fits into layer specification (i.e. same obs, same var), while in general this is not true (e.g. scRNA + scATAC). Still, multiplex could be analyzed from 2+ AnnData objects when adata.uns['neighbors'] is present

Hi @dawe ! Just to chip in real quick, I think your suggestion makes a lot of sense and beside the examples you already mention I believe spatial transcriptomics data could also benefit a lot from such approach (since with such data you have both a knn graph from gene expression as well as a graph from spatial coordinates).

I never used spatial data (so far), are they organized as separate AnnData objects? If everything that could be integrated is a single AnnData then the function would be easy, like
def leiden_multiplex(adata: Sequence[AnnData], use_computed: bool = False, weights: None):

    adj_list = [x.uns['neighbors']['connectivities'] for x in adata]
    G_list = [sc._utils.get_igraph_from_adjacency(x) for x in adj_list] #also add the `restrict_to` step

    if use_computed:
        part_list = [get_partitions_from_adata.obs] or [recalculate_partitions_with_neighbors_params]
        # then run the optimizer
    else:
        membership, improv = la.find_partitions_multiplex(**params)

    for a in adata:
        a.obs['multiplex'] = pd.Categorical(membership)

where adata is a list of AnnData objects, use_computed switches between recalculate partitions (False) or optimize partitions already calculated (True). Weights can be specified to give more or less importance to a specific view. Note that, by default, if set to None it is set to a list of ones by leidenalg.
Other options, in addition to the usual copy = False should be the leidenalg type of partitioning (CPMVertexPartition, RBConfigurationVertexPartition...)

Spatial data would also be a single anndata object, similarly to CITE-seq but with images in addition to expression counts.
Anyway, I think this functionality would be very useful and really cool to try out.
Worth mentioning that uns['neighbors'] will be moved to obsp (see theislab/scanpy-tutorials#14) and that probably @ivirshup is working along similar lines (see #1117 ).

 Great, I’m closing this issue, then


cmd/go: 'go build -race' with a cross-compiled GOOS should not suggest setting CGO_ENABLED

$ go version
go version go1.13.7 windows/amd64

Does this issue reproduce with the latest release?
YES
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\8\AppData\Local\go-build
set GOENV=C:\Users\8\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\gopath
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\Go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=C:\Go\src\go.mod
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=d:\TEMP\go-build345576314=/tmp/go-build -gno-record-gcc-switches

What did you do?
i compile this file for test:

package main
import (
"fmt"
)
func main() {
c := make(chan bool)
m := make(map[string]string)
go func() {
m["1"] = "a" // First conflicting access.
c <- true
}()
m["2"] = "b" // Second conflicting access.
<-c
for k, v := range m {
fmt.Println(k, v)
}
}

with
set CGO_ENABLED=1
set GOOS=linux
set GO_ARCH=amd64
go build -race

What did you expect to see?
success compile
What did you see instead?
# runtime/cgo
gcc_linux_amd64.c: In function '_cgo_sys_thread_start':
gcc_linux_amd64.c:66:2: error: implicit declaration of function 'sigfillset' [-Werror=implicit-function-declaration]
   66 |  sigfillset(&ign);
      |  ^~~~~~~~~~
cc1.exe: all warnings being treated as errors

Note, that
set CGO_ENABLED=1
set GOOS=linux
go build

is succecced, but i need to compile with "-race" detector. If CGO_ENABLED=0
set CGO_ENABLED=0
set GOOS=linux
go build -race

then i got
go build: -race requires cgo; enable cgo by setting CGO_ENABLED=1

Cross-compiling programs with C dependencies is generally not supported, and the runtime support for -race mode relies on C dependencies.
If you want to build a race-enabled binary, you need to either compile for GOOS=windows or build the binary on a Linux machine.

Probably the best we can do here is emit a clearer diagnostic for cross-compiled -race builds. (We should not prompt the user to “enable cgo” if we don't have reason to believe that such a build would succeed.)
CC @jayconrod @matloob

just try to cross compile this:
package main

import (
"fmt"
)

func main() {
c := make(chan bool)
m := make(map[string]string)
go func() {
m["1"] = "a" // First conflicting access.
c <- true
}()
m["2"] = "b" // Second conflicting access.
<-c
for k, v := range m {
fmt.Println(k, v)
}
}

from windows\amd64  to linux\amd64  with gox

I think "'go build -race' with a cross-compiled GOOS should not suggest setting CGO_ENABLED" is not what this issue is for, that is another issue\suggestion.
Current issue is unavailability to crosscompile "-race" with gox.

@MasterDimmy, the fact that go test -race has C dependencies is not something we can feasibly fix. (The race detector uses a fairly extensive C library, and the folks on the Go project are not in a position to rewrite it from scratch in pure Go.)
If you want to volunteer to rewrite the race-detector library in pure Go, you're welcome to do so, but as far as “things we can feasibly plan to fix” are concerned, a better error message for this case is about the limit.

I think @bcmills was probably intending that anyone with this problem take the rewrite suggestion as an impractical solution and to instead use one of the work-arounds he suggested, but I was curious and found that https://git.llvm.org/git/compiler-rt.git from https://github.com/golang/go/blob/master/src/runtime/race/README is dead (maybe worth a separate issue if anyone has the energy to write it). For anyone else interested, there are some svn links on this page, though: https://github.com/google/sanitizers/wiki/ThreadSanitizerDevelopment

@bcmills I took a stab at just modifying the error message in https://github.com/golang/go/pull/38670. I'm just trying getting my feet wet and orient myself in the codebase.

Wait, I'm not following the suggestion here.  Cross-compiling cgo code works fine if you have a cross-compiler.
We may want a general fix to do better checking of whether you are using a cross-compiler when doing a cross-build that uses cgo.  But that seems separate from the narrow issue of "go build -race", which should work if a cross-compiler is installed.

I misunderstood Bryan's comment:

Cross-compiling programs with C dependencies is generally not supported

If I understand correctly now, the short term move is just to not mention setting CGO_ENABLED because that isn't sufficient on its' own and it is too verbose to explain that you need to have a cross-compiler installed?


[Bug]: [BETA] [ERR_SOCKET_DGRAM_NOT_RUNNING] Error that makes HB restart every time stream stops

What happened?
Hi, while testing BETA I experience this error [ERR_SOCKET_DGRAM_NOT_RUNNING]. It makes HB restart every time I stop stream within Home app. As I've put Eufy plugin in child bridge, it restart only child bridge, so it helps a bit. As I recall, this issue is present since earliest BETA and was also reported by other users. Can you please have a look? Thx in advance for this amazing plugin!
Device Type
Indoor Camera (Supported)
Plugin Version
beta (Best Effort)
HomeBridge Version
v1.3.x (Supported)
NodeJS Version
v16 (Supported)
Operating System
Raspbian 10+ (Supported)
Relevant log output
[03/01/2022, 18:35:56] [homebridge-eufy-security] Eufy camera Starting video stream: 1280 x 720, 30 fps, 299 kbps (AAC-eld)
[03/01/2022, 18:36:05] [homebridge-eufy-security] Eufy camera Eufy Station stopped the stream. Stopping stream.
[03/01/2022, 18:36:05] [homebridge-eufy-security] Eufy camera Stopped video stream.
[03/01/2022, 18:36:05] [homebridge-eufy-security] Eufy camera Error occurred closing socket: Error [ERR_SOCKET_DGRAM_NOT_RUNNING]: Not running
[03/01/2022, 18:36:05] [homebridge-eufy-security] Eufy camera Stopped video stream.
Error: read ECONNRESET
    at Pipe.onStreamRead (node:internal/stream_base_commons:220:20)
[03/01/2022, 18:36:06] [homebridge-eufy-security] Child bridge process ended
[03/01/2022, 18:36:06] [homebridge-eufy-security] Process Ended. Code: 1, Signal: null
[03/01/2022, 18:36:13] [homebridge-eufy-security] Restarting Process...

can you enable debug ? while doing the test ?
https://github.com/samemory/homebridge-eufy-security/wiki/Basic-Troubleshooting#debug-mode

Done, it's log since stream starts and until I stop it
homebridge.log.txt

can you enable debug ? while doing the test ?
https://github.com/samemory/homebridge-eufy-security/wiki/Basic-Troubleshooting#debug-mode

Done, it's log since stream starts and until I stop it
homebridge.log.txt

I'm experiencing the same. Tried to tinker around with it myself without succeeding.
The refactoring you have made, @lenoxys, is however very nice.

Got another idea. Will test tonight.

the timeout is hardcoded within @bropat lib
https://github.com/bropat/eufy-security-client/blob/master/src/eufysecurity.ts#L38
It could be nice if we can change that.

@lenoxys
That is wrong. This is the default value.
See here

Will gladly help to test if needed

Sorry miss the function !! thanks for your feedback !!

@lenoxys I think I know what our problem is. We need to destroy the readable stream in the UniversalStream.ts.
import net from 'net'
import fs from 'fs'
import os from 'os'
import { Readable } from 'stream';

let counter = 0

export class NamePipeStream {
    private server;
    private sock_id;
    private log;
    public url;
    private stream: Readable;

    constructor(stream, onSocket, log) {
        this.log = log;
        this.stream = stream;
        let path

        this.sock_id = ++counter

        const osType = os.type()
        if (osType === 'Windows_NT') {
            path = '\\\\.\\pipe\\stream' + (++counter)
            this.url = path
        } else {
            path = './' + (++counter) + '.sock'
            this.url = 'unix:' + path
        }

        try {
            fs.statSync(path)
            fs.unlinkSync(path)
        } catch (err) { }
        this.server = net.createServer(onSocket)
        stream.on('finish', () => {
            this.server.close()
        })

         this.server.on('error', (err: Error) => {
             this.log.error('Error in NamePipe', err.message, err.stack);
         });

         this.server.on('close', (err: Error) => {
            this.log.error('Error in NamePipe', err?.message, err?.stack);
        });

        this.server.listen(path)
    }
    close() {
        this.log.info('Trying to close NamePipeStream.')
        if (this.server)
        {
            this.stream.destroy();

            this.server.close();
            this.log.info('Closed NamePipeStream')
        }
    }
}

export function StreamInput(stream, log) {
    return new NamePipeStream(stream, socket => stream.pipe(socket), log)
}

export function StreamOutput(stream, log) {
    return new NamePipeStream(stream, socket => socket.pipe(stream), log)
}

When doing this the audio drags on for some time. I think the rtcp timeout is longer than the video?
But lets me exit ffmpeg without crashing.

Also I think we should close the uaudiostream and uvideostream before we call the stopStream function. Otherwise the Readable is not terminated soon enough.
                uVideoStream.close();
                uAudioStream.close();
                this.stopStream(request.sessionID);

To fix the stream you have to add unpipe the readable stream in UniversalStream.ts.
this.stream.unpipe();

Also we should add a null guard on:
                 this.server.on('error', (err: Error) => {
                     this.log.error('Error in NamePipe', err.message, err.stack);
                 });

to this:
                 this.server.on('error', (err: Error) => {
                     this.log.error('Error in NamePipe', err?.message, err?.stack);
                 });

@lenoxys Do you want me to make the change or would you like to test yourself?

Let me test

yes indeed it's better that way !

Also it seems like the timeout for the audio is a bit long. Should the stream be stopped when the user leaves the stream in homekit? Here:
https://github.com/samemory/homebridge-eufy-security/blob/7584d96b4a9a21baa260f8ec96342d5d7e83d939/src/accessories/streamingDelegate.ts#L498
Because as it is we actually don't end the uVideoStream and uAudioStream there. So the streams continues until the timeout it seems.

We could of course add both uAudioStream and uVideoStream to the activeSession and just call stopStream directly every time. Just can't seem to make it work.

Or we should find another way to gracefully stop the namePibeStream instead. Because we can never ensure that the stream won't end before we close our pipe.
If the station stops the stream this error will always bubble through and kill our process.

We could of course add both uAudioStream and uVideoStream to the activeSession and just call stopStream directly every time. Just can't seem to make it work.

Would be proper approach at least if we rely on other thing

@lenoxys If you change the pipe here:
https://github.com/samemory/homebridge-eufy-security/blob/e38d24a8ffd1e77dcc63fff9712910f7ff3c209f/src/accessories/UniversalStream.ts#L63
to this:
return new NamePipeStream(stream, socket => stream.pipe(socket).on('error', function(e){log.error(e)}), who, log)
We no longer break hard on econnreset. This is probably a good idea since it is not in our control.

Maybe the pipe creation should also be moved into the class instead of through the constructor. And then on error should call close?

I have some changes I will test later tonight regarding the UniversalStream.

Maybe the pipe creation should also be moved into the class instead of through the constructor. And then on error should call close?

agree this need to be rewrite with cleaner manner. I was using this part from @bropat script :
https://github.com/bropat/ioBroker.eufy-security/blob/master/src/lib/video.ts#L14-L65
But we should avoid passing a function into argument of a function.

I have some changes I will test later tonight regarding the UniversalStream.

I let you follow this part I will be off couple of week. thanks for your help on this !

@lenoxys Can you test the changes to the beta streaming before we make a new version with the workflow?

@njuart Would you be able to test beta v1.1.1-beta.14?

@njuart Would you be able to test beta v1.1.1-beta.14?

Sure, with pleasure 👍

@njuart Would you be able to test beta v1.1.1-beta.14?

Just checked latest beta.14, great news! HB doesn't crash and restart anymore after closing stream. Video is a bit laggy, sound is present. I also see this in logs:

[07/01/2022, 10:32:14] [homebridge-eufy-security] Eufy camera Starting video stream: 1280 x 720, 30 fps, 299 kbps (AAC-eld)
[07/01/2022, 10:32:23] [homebridge-eufy-security] Stopping session with id: 0ddec022-cec3-44a9-afc8-36019e907688
[07/01/2022, 10:32:23] [homebridge-eufy-security] Eufy camera Eufy Station stopped the stream. Stopping stream.
[07/01/2022, 10:32:23] [homebridge-eufy-security] Stopping session with id: 131b8a1c-10b5-4887-989d-4f977128f4ab
[07/01/2022, 10:32:23] [homebridge-eufy-security] No session to stop.
[07/01/2022, 10:32:23] [homebridge-eufy-security] Eufy camera Eufy Station stopped the stream. Stopping stream.
[07/01/2022, 10:32:23] [homebridge-eufy-security] Stopping session with id: e06ddeda-0995-4bfc-8dab-b124cd9a067a
[07/01/2022, 10:32:23] [homebridge-eufy-security] No session to stop.
[07/01/2022, 10:32:23] [homebridge-eufy-security] Eufy camera Eufy Station stopped the stream. Stopping stream.
[07/01/2022, 10:32:23] [homebridge-eufy-security] Stopping session with id: d210ffa7-4d67-4781-bbd4-cbfa4b638b45
[07/01/2022, 10:32:23] [homebridge-eufy-security] No session to stop.
[07/01/2022, 10:32:23] [homebridge-eufy-security] Eufy camera Eufy Station stopped the stream. Stopping stream.
[07/01/2022, 10:32:23] [homebridge-eufy-security] Stopping session with id: 0ddec022-cec3-44a9-afc8-36019e907688
[07/01/2022, 10:32:23] [homebridge-eufy-security] Eufy camera Stopped video stream.
[07/01/2022, 10:32:23] [homebridge-eufy-security] Eufy camera Stopped video stream.
[07/01/2022, 10:32:23] [homebridge-eufy-security] Eufy camera Error occurred closing socket: Error [ERR_SOCKET_DGRAM_NOT_RUNNING]: Not running
[07/01/2022, 10:32:23] [homebridge-eufy-security] Connection closed by Eufy station.
[07/01/2022, 10:32:23] [homebridge-eufy-security] ErrorCode ECONNRESET

Perfect. There is obviously room for some logging cleanup :). But I will not prioritize this at the moment.

Perfect. There is obviously room for some logging cleanup :). But I will not prioritize this at the moment.

Thx for the efforts! 🤝 Since we're discussing stream here, is there any way I can increase streaming quality (or play with it to adjust)? Currently It looks pixelated a bit (comparing to regular web stream).

@njuart Difficult for me to say. It might be the way we use ffmpeg. I would think that the stream from the client library is not the issue - since there is no issue on his github regarding the stream quality.

@schliemann Also noticed that stream is live for about 10-15 seconds, then image freezes and after another 10-15 seconds it shows No response.

@schliemann Also noticed that stream is live for about 10-15 seconds, then image freezes and after another 10-15 seconds it shows No response.

Interesting. Mine will run for the maximum periode of 30 seconds and the the station kills it as expected.
https://user-images.githubusercontent.com/10818044/148523296-8dd1af4f-1263-49ca-878f-13aae63e8cfb.MOV

I don't know if it helps (I have the floodlight cam), but I've just swapped to the latest beta and I had to manually change the video codec in streamingDelegate.js to "copy".  After doing this, local streaming works fine for me.

@fullphat Can you try and add a logger line and print the this.videoConfig.vcodec you have when trying to start a stream. I am wondering why you don't have copy already.

Done!
I get "undefined" in the log output.

Ok. I'll add a null check then. But would you mind making an issue with bropat, as well?

Yes, of course.  What do I need to tell him?

Yes, of course. What do I need to tell him?

Wait. It might be our plugin. Let me have a look.

Yes, of course. What do I need to tell him?

Wait. It might be our plugin. Let me have a look.

@fullphat It's our bug. For some reason the floodlight doesn't like codec libx264. I'll put in copy later tonight.

@fullphat Can't get my doorbell to work with 'copy'. Can you share your code-snippet?

Ah. Problem is probably that the codec for the doorbell is not H.264 based.
Set the codec used for encoding video sent to HomeKit, must be H.264-based. You can change to a hardware accelerated video codec with this option, if one is available. (Default: libx264)

@fullphat
Can you try and replace the following in your code and see if streaming works:
       const vcodec  = this.videoConfig.vcodec || 'libx264';

with this:
        let vcodec = '';
        switch(streamData.metadata.videoCodec) {
            case VideoCodec.H264:
                vcodec = 'copy';
                break;
            case VideoCodec.H265:
            case VideoCodec.UNKNOWN:
                vcodec = 'libx264';
                break;
        }

My code snippet is:
            // const vcodec = this.videoConfig.vcodec || 'libx264';
            const vcodec = 'copy';
            this.log.debug('vcodec:', this.videoConfig.vcodec);

But I appreciate this will probably break everything else!  😄
I tried your code, but I get this in the log:
[1/8/2022, 9:16:30 AM] [EufySecurity-1.1.1-beta.14] DEBUG: videoConfig->vcodec: undefined
[1/8/2022, 9:16:30 AM] [EufySecurity-1.1.1-beta.14] DEBUG: streamData->metadata->videoCodec: -1
(node:59117) UnhandledPromiseRejectionWarning: ReferenceError: VideoCodec is not defined
    at StreamingDelegate.startStream (/usr/local/lib/node_modules/homebridge-eufy-security/src/accessories/streamingDelegate.ts:400:20)
    at processTicksAndRejections (internal/process/task_queues.js:95:5)
(Use `node --trace-warnings ...` to show where the warning was created)

Where should the VideoCodec enum be defined?  I don't seem to have that...

@fullphat Ah. Probably forgot an import to give you as well. Will find it.

import { Station, Camera, PropertyName, StreamMetadata, VideoCodec } from 'eufy-security-client';

Where do I paste this line?  I tried putting it at the top of streamingDelegate.js, but I just get a syntax error.

https://github.com/samemory/homebridge-eufy-security/blob/7adf974ec5fd1f7840b6055110edbf2ad5c6586e/src/accessories/streamingDelegate.ts#L30

Odd. It should already be there. Can you confirm you have it?

This is what I have - note I'm using the Beta.14 plugin:
"use strict";
var __importDefault = (this && this.__importDefault) || function (mod) {
    return (mod && mod.__esModule) ? mod : { "default": mod };
};
Object.defineProperty(exports, "__esModule", { value: true });
exports.StreamingDelegate = void 0;
const child_process_1 = require("child_process");
const dgram_1 = require("dgram");
const ffmpeg_for_homebridge_1 = __importDefault(require("ffmpeg-for-homebridge"));
const pick_port_1 = __importDefault(require("pick-port"));
const ffmpeg_1 = require("./ffmpeg");
const eufy_security_client_1 = require("eufy-security-client");
const UniversalStream_1 = require("./UniversalStream");
const fs_1 = require("fs");
const util_1 = require("util");
const readFileAsync = (0, util_1.promisify)(fs_1.readFile), SnapshotUnavailablePath = require.resolve('../../media/Snapshot-Unavailable.png');
class StreamingDelegate {
    constructor(platform, device, cameraConfig, api, hap) {
        // keep track of sessions
        this.pendingSessions = new Map();
        this.ongoingSessions = new Map();
        this.timeouts = new Map();
        this.log = platform.log;
        this.hap = hap;
        this.platform = platform;
        this.device = device;
        this.cameraName = device.getName();
        this.unbridge = false;
        this.videoConfig = cameraConfig.videoConfig;
        this.videoProcessor = ffmpeg_for_homebridge_1.default || 'ffmpeg';
        api.on("shutdown" /* SHUTDOWN */, () => {
            for (const session in this.ongoingSessions) {
                this.stopStream(session);
            }
        });

Well. The codec resides in this require. I'm not an expert on how require importing works.
const eufy_security_client_1 = require("eufy-security-client");

Got it working!
Had to reference eufy_security_client_1 first and also had to change the logic slightly as the T8420 seems to be generating an unknown stream (eufy_security_client_1.VideoCodec.UNKNOWN).  Here's the code that works for me:
this.log.info('videoConfig->vcodec:', this.videoConfig.vcodec);
            this.log.info('streamData->metadata->videoCodec:', streamData.metadata.videoCodec);

            let vcodec = '';
            switch(streamData.metadata.videoCodec) {
              // case -1:
              //   vcodec = 'copy';
              //   break;

              case eufy_security_client_1.VideoCodec.H264:
              case eufy_security_client_1.VideoCodec.UNKNOWN:
                  vcodec = 'copy';
                  break;
              case eufy_security_client_1.VideoCodec.H265:
                  vcodec = 'libx264';
                  break;
            }

            this.log.info('vcodec:', vcodec);

And here's the resulting output in the log:
[1/8/2022, 10:02:25 AM] [EufySecurity-1.1.1-beta.14] DEBUG: ReqEufy: {"videoCodec":-1,"videoFPS":15,"videoHeight":1080,"videoWidth":1920,"audioCodec":1}
[1/8/2022, 10:02:25 AM] [EufySecurity-1.1.1-beta.14] INFO:  videoConfig->vcodec: undefined
[1/8/2022, 10:02:25 AM] [EufySecurity-1.1.1-beta.14] INFO:  streamData->metadata->videoCodec: -1
[1/8/2022, 10:02:25 AM] [EufySecurity-1.1.1-beta.14] INFO:  vcodec: copy
[1/8/2022, 10:02:25 AM] [EufySecurity-1.1.1-beta.14] DEBUG: Floodlight Cam Video stream requested: 1280 x 720, 30 fps, 299 kbps true
[1/8/2022, 10:02:25 AM] [EufySecurity-1.1.1-beta.14] INFO:  Floodlight Cam Starting video stream: native x native, native fps, ??? kbps (AAC-eld)
[1/8/2022, 10:02:25 AM] [EufySecurity-1.1.1-beta.14] DEBUG: Floodlight Cam_video Stream command: /usr/local/lib/node_modules/homebridge-eufy-security/node_modules/ffmpeg-for-homebridge/ffmpeg -use_wallclock_as_timestamps 1 -r 15 -i unix:./6.sock -an -sn -dn -codec:v copy -pix_fmt yuv420p -color_range mpeg -payload_type 99 -ssrc 10499619 -f rtp -srtp_out_suite AES_CM_128_HMAC_SHA1_80 -srtp_out_params wQbyHGSWG7gWNwXZTiKNrjuGHvOy7c7DTE+41av6 srtp://<IP_ADDRESS>:49633?rtcpport=49633&pkt_size=1316 -loglevel level+verbose -progress pipe:1

Hm. What happens with your stream when you have changed nothing and it uses libx264.
Basically the idea of my code change was to change everything not libx264 to 264. Since this is the only format homekit accepts.

In theory a copy on unknown could cause an error if the incomming encoding is actually not libx264. Can we ask bropat if something is wrong with the client library?

Hm. What happens with your stream when you have changed nothing and it uses libx264.
Basically the idea of my code change was to change everything not libx264 to 264. Since this is the only format homekit accepts.

So if I don't use copy as the codec, I don't get any video in the Home app.  In the HomeBridge log, it looks like video is coming through as I can see the frames, but the Home app just sits spinning.
As soon as a I make ffmpeg use copy the stream springs to life immediately (it does sometimes crash after a few seconds but restarting it works).

In theory a copy on unknown could cause an error if the incoming encoding is actually not libx264. Can we ask bropat if something is wrong with the client library?

Done!
https://github.com/bropat/eufy-security-client/issues/97

Got snapshot working. Still slow and the colors look weird however.
https://user-images.githubusercontent.com/10818044/148776194-e4eaa6f8-79b8-4495-b826-255e326008ae.MOV

Is this available to test? 😊

Not yet. And not sure this implementation is a good solution. The snapshot still takes far too long. Will try and determine the bottleck. But the problem is that while generating the snapshot it blocks the rest of the plugin.

Ok - good that it's do-able though 👍🏼

@fullphat Not yet. And not sure this implementation is a good solution. The snapshot still takes far too long. Will try and determine the bottleck. But the problem is that while generating the snapshot it blocks the rest of the plugin.

Looks like apple iOS issue
https://piunikaweb.com/2022/01/03/apple-homekit-app-camera-thumbnails-or-snapshots-not-updating/

@fullphat Not yet. And not sure this implementation is a good solution. The snapshot still takes far too long. Will try and determine the bottleck. But the problem is that while generating the snapshot it blocks the rest of the plugin.

Looks like apple iOS issue
https://piunikaweb.com/2022/01/03/apple-homekit-app-camera-thumbnails-or-snapshots-not-updating/

I'm not sure - I have seen this a few times, but since upgrading to 15.2 everything has been generally ok.  I run one of my Eufy P&T 2K cameras through the homebridge-ffmpeg plugin and that updates it's thumbnail fine too.

My thumbnail was stuck yesterday. When clearing cache it worked again.

But what I was referring to is that generating the snapshot from the readable stream takes 5-6 seconds.

Ok. So. Got some progress made today. Now the code can generate snapshots a bit faster. Also - I finally figured out why night mode wasn't working on the doorbell. Turns out that the first frame of the stream is not with night vision. So added some frame skipping.
Uploading IMG_4929.MOV…

@fullphat Would you be willing to try and test the new snapshot at some point before it is added to the beta?

@fullphat Would you be willing to try and test the new snapshot at some point before it is added to the beta?

Yes absolutely! 😊

I'll clean up the code a bit first and then i'll send it in this thread.

@fullphat Do you want it in ts or js?

JS if possible as I can then just drop it straight in. 👍🏼

streamingDelegate.log

@fullphat Couldn't upload js files. Please rename extention to .js. Then you should be set.

Working a treat!  👍  Not seeing any lag, and getting a new snapshot every 10 seconds in the Home app.

Great. I'll clean up the code in the weekend then.

Maybe we can also clean up the way we local stream video. It might solve the issue with not being able to save files to current directory. I'll take a look at that next.

Sounds good.  I have spotted an issue - the camera has gone "no response" and checking the logs after restarting both the specific bridge and all of HB, I get this:

Sorry it's an image - using my phone.  Seems the plugin is crashing when trying to open a socket?

@fullphat Was it caused by livestream or snapshot?

I'm not sure - I opened the Home app and the camera was showing as no response.

Try running sudo lsof -i :41089
This should show you what is using this address.

Hi @schliemann I've also checked flow with updated streamingDelegate you've provided. Works flawlessly with my Indoor 2k . Snapshots are being updated every 10 seconds, also starting / ending stream works fine. Meanwhile in logs I see this, every time snapshot is updated, but I suggest it's expected currently

[28/01/2022, 12:53:50] [homebridge-eufy-security] handleSnapshotRequest
[28/01/2022, 12:53:50] [homebridge-eufy-security] snapshotPromise: false
[28/01/2022, 12:53:50] [homebridge-eufy-security] Calling fetchSnapshot
[28/01/2022, 12:53:50] [homebridge-eufy-security] isLiveStreaming: false
[28/01/2022, 12:53:51] [homebridge-eufy-security] ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers Eufy camera] [Snapshot
[28/01/2022, 12:53:51] [homebridge-eufy-security]   built with gcc 6.3.0 (Raspbian 6.3.0-18+rpi1+deb9u1) 20170516 Eufy camera] [Snapshot
[28/01/2022, 12:53:51] [homebridge-eufy-security]   configuration: --pkgconfigdir=/build/workspace/lib/pkgconfig --prefix=/build/workspace --pkg-config-flags=--static --extra-cflags=-I/build/workspace/include --extra-ldflags=-L/build/workspace/lib --extra-libs='-lpthread -lm' --enable-static --disable-debug --disable-shared --disable-ffplay --disable-doc --enable-openssl --enable-gpl --enable-version3 --enable-nonfree --enable-pthreads --enable-libvpx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libx264 --enable-runtime-cpudetect --enable-libfdk-aac --enable-avfilter --enable-libopencore_amrwb --enable-libopencore_amrnb --enable-filters --enable-decoder=h264 --enable-network --enable-protocol=tcp --enable-libspeex --enable-demuxer=rtsp --enable-omx-rpi --enable-mmal --enable-indev=alsa --enable-outdev=alsa Eufy camera] [Snapshot
[28/01/2022, 12:53:51] [homebridge-eufy-security]   libavutil      56. 51.100 / 56. 51.100 Eufy camera] [Snapshot
[28/01/2022, 12:53:51] [homebridge-eufy-security]   libavcodec     58. 91.100 / 58. 91.100 Eufy camera] [Snapshot
[28/01/2022, 12:53:51] [homebridge-eufy-security]   libavformat    58. 45.100 / 58. 45.100 Eufy camera] [Snapshot
[28/01/2022, 12:53:51] [homebridge-eufy-security]   libavdevice    58. 10.100 / 58. 10.100 Eufy camera] [Snapshot
[28/01/2022, 12:53:51] [homebridge-eufy-security]   libavfilter     7. 85.100 /  7. 85.100 Eufy camera] [Snapshot
[28/01/2022, 12:53:51] [homebridge-eufy-security]   libswscale      5.  7.100 /  5.  7.100 Eufy camera] [Snapshot
[28/01/2022, 12:53:51] [homebridge-eufy-security]   libswresample   3.  7.100 /  3.  7.100 Eufy camera] [Snapshot
[28/01/2022, 12:53:51] [homebridge-eufy-security]   libpostproc    55.  7.100 / 55.  7.100 Eufy camera] [Snapshot
[28/01/2022, 12:53:51] [homebridge-eufy-security] [h264 @ 0x43cf980] Reinit context to 2304x1296, pix_fmt: yuv420p Eufy camera] [Snapshot
[28/01/2022, 12:53:51] [homebridge-eufy-security] [h264 @ 0x43ce450] Stream #0: not enough frames to estimate rate; consider increasing probesize Eufy camera] [Snapshot
[28/01/2022, 12:53:51] [homebridge-eufy-security] [h264 @ 0x43ce450] decoding for stream 0 failed Eufy camera] [Snapshot
[28/01/2022, 12:53:51] [homebridge-eufy-security] pipe:: could not seek to position 0.500 Eufy camera] [Snapshot
[28/01/2022, 12:53:51] [homebridge-eufy-security] Input #0, h264, from 'pipe:': Eufy camera] [Snapshot
[28/01/2022, 12:53:51] [homebridge-eufy-security]   Duration: N/A, bitrate: N/A Eufy camera] [Snapshot
[28/01/2022, 12:53:51] [homebridge-eufy-security]     Stream #0:0: Video: h264 (High), 1 reference frame, yuv420p(tv, bt709, progressive, left), 2304x1296, 15 tbr, 1200k tbn, 30 tbc Eufy camera] [Snapshot
[28/01/2022, 12:53:51] [homebridge-eufy-security] Output #0, image2, to 'pipe:': Eufy camera] [Snapshot
[28/01/2022, 12:53:51] [homebridge-eufy-security]   Metadata: Eufy camera] [Snapshot
[28/01/2022, 12:53:51] [homebridge-eufy-security]     encoder         : Lavf58.45.100 Eufy camera] [Snapshot
[28/01/2022, 12:53:51] [homebridge-eufy-security]     Stream #0:0: Video: h264 (High), 1 reference frame, yuv420p(tv, bt709, progressive, left), 2304x1296 (0x0), q=2-31, 15 tbr, 15 tbn, 15 tbc Eufy camera] [Snapshot
[28/01/2022, 12:53:51] [homebridge-eufy-security] Stream mapping: Eufy camera] [Snapshot
[28/01/2022, 12:53:51] [homebridge-eufy-security]   Stream #0:0 -> #0:0 (copy) Eufy camera] [Snapshot
Eufy camera] [Snapshot[homebridge-eufy-security] frame=    0 fps=0.0 q=-1.0 size=N/A time=00:00:00.00 bitrate=N/A speed=   0x
Eufy camera] [Snapshot[homebridge-eufy-security] frame=    0 fps=0.0 q=-1.0 size=N/A time=00:00:00.00 bitrate=N/A speed=   0x
Eufy camera] [Snapshot[homebridge-eufy-security] frame=    0 fps=0.0 q=-1.0 size=N/A time=00:00:00.00 bitrate=N/A speed=   0x
[28/01/2022, 12:53:53] [homebridge-eufy-security] [AVIOContext @ 0x43de390] Statistics: 0 seeks, 3 writeouts Eufy camera] [Snapshot
[28/01/2022, 12:53:53] [homebridge-eufy-security] No more output streams to write to, finishing. Eufy camera] [Snapshot
[28/01/2022, 12:53:53] [homebridge-eufy-security] frame=    1 fps=0.6 q=-1.0 Lsize=N/A time=00:00:01.93 bitrate=N/A speed=1.07x     Eufy camera] [Snapshot
[28/01/2022, 12:53:53] [homebridge-eufy-security] video:88kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown Eufy camera] [Snapshot
[28/01/2022, 12:53:53] [homebridge-eufy-security] Input file #0 (pipe:): Eufy camera] [Snapshot
[28/01/2022, 12:53:53] [homebridge-eufy-security]   Input stream #0:0 (video): 29 packets read (149937 bytes);  Eufy camera] [Snapshot
[28/01/2022, 12:53:53] [homebridge-eufy-security]   Total: 29 packets (149937 bytes) demuxed Eufy camera] [Snapshot
[28/01/2022, 12:53:53] [homebridge-eufy-security] Output file #0 (pipe:): Eufy camera] [Snapshot
[28/01/2022, 12:53:53] [homebridge-eufy-security]   Output stream #0:0 (video): 1 packets muxed (89885 bytes);  Eufy camera] [Snapshot
[28/01/2022, 12:53:53] [homebridge-eufy-security]   Total: 1 packets (89885 bytes) muxed Eufy camera] [Snapshot
[28/01/2022, 12:53:53] [homebridge-eufy-security] [AVIOContext @ 0x43d6fe0] Statistics: 250215 bytes read, 0 seeks Eufy camera] [Snapshot
[28/01/2022, 12:53:53] [homebridge-eufy-security] Eufy camera Eufy Station stopped the stream. Stopping stream.
[28/01/2022, 12:53:53] [homebridge-eufy-security] Stopping session with id: 0a69ac32-dbda-4a55-a285-0a7935098ac6
[28/01/2022, 12:53:53] [homebridge-eufy-security] No session to stop.
[28/01/2022, 12:53:53] [homebridge-eufy-security] Eufy camera Eufy Station stopped the stream. Stopping stream.
[28/01/2022, 12:53:53] [homebridge-eufy-security] Stopping session with id: b7f9fa56-ae8d-4f3a-9a83-70570742b995
[28/01/2022, 12:53:53] [homebridge-eufy-security] No session to stop.
[28/01/2022, 12:53:53] [homebridge-eufy-security] snapshot byte lenght: 89885
[28/01/2022, 12:53:56] [homebridge-eufy-security] Setting snapshotPromise to undefined.
[28/01/2022, 12:53:56] [homebridge-eufy-security] isLiveStreaming: false

@njuart
Yeah. I added a bunch of logging. Will change them to debug and only enabledetailedlogging of 2 will display verbose messages from ffmpeg.

Try running sudo lsof -i :41089
This should show you what is using this address.

Gives me two PIDs which come back to "hb-service" and "homebridge: homebridge-eufy-security"

Just FYI, installed yesterday 15.4 PB1 and it seems like interval of updating snapshots in Home app now is 1 minute instead of 10 seconds.

Just FYI, installed yesterday 15.4 PB1 and it seems like interval of updating snapshots in Home app now is 1 minute instead of 10 seconds.

My iPad is on iPadOS 15.4 Dev Beta 1 and it's still refreshing every 10 seconds

Just FYI, installed yesterday 15.4 PB1 and it seems like interval of updating snapshots in Home app now is 1 minute instead of 10 seconds.

My iPad is on iPadOS 15.4 Dev Beta 1 and it's still refreshing every 10 seconds

Weird, on iPhone I clearly see it started to update every minute. I also got this crash just right now

@njuart Are you on the local network? When I'm not home the interval is also 1 minute. Pretty sure the interval is managed by IOS.

@njuart Are you on the local network? When I'm not home the interval is also 1 minute. Pretty sure the interval is managed by IOS.

Yup, the weird part is that I'm on local network. Before installing 15.4 interval was 10 sec. Yeah, I'm pretty sure it's managed by iOS, just pointing out that on new beta I have this behavior. Other than that, and a couple of crashes, snapshots work amazing. Thx 👍

@njuart I can confirm. iOS 15.3 changed the snapshot interval.

@njuart I can confirm. iOS 15.3 changed the snapshot interval.

Hope it's just beta tuning.

I'm not on beta. Just regular 15.3.

@njuart, @fullphat, @lenoxys
1.1.1-beta.15 is ready. Just includes snapshots and some changed settings for ffmpeg logging. "+verbose" is only set if enableDetailedLogging is 1 or above.

@njuart, @fullphat, @lenoxys
1.1.1-beta.15 is ready. Just includes local live snapshots and some changed settings for ffmpeg logging. "+verbose" is only set if enableDetailedLogging is 1 or above.

Seems to be even more stable now, and log is a way much cleaner =) Thx, will keep you posted how it goes through the day

@njuart, @fullphat, @lenoxys
1.1.1-beta.15 is ready. Just includes local live snapshots and some changed settings for ffmpeg logging. "+verbose" is only set if enableDetailedLogging is 1 or above.

Seems to be even more stable now, and log is a way much cleaner =) Thx, will keep you posted how it goes through the day

Installed as well.  All good here once I'd fixed the video codec issue.  Will continue to test as well. 👍🏼

Installed the new version. Snapshots working on Wireless Doorbell now - hope my battery will survive … But video stream still fails. Should I do something extra?

I've noticed that on beta 15 stream goes live for 3-5 seconds and then it stops with No Response.

@njuart Odd. Works here. Can you share your log? See what the exception is?

Installed the new version. Snapshots working on Wireless Doorbell now - hope my battery will survive … But video stream still fails. Should I do something extra?

Please share your log.

Installed the new version. Snapshots working on Wireless Doorbell now - hope my battery will survive … But video stream still fails. Should I do something extra?

Please share your log.

[1/30/2022, 10:49:38 PM] [EufySecurity-1.1.1-beta.15] DEBUG: Received local livestream.
[1/30/2022, 10:49:38 PM] [EufySecurity-1.1.1-beta.15] DEBUG: Doorbell Snapshot command: ffmpeg -probesize 3000 -analyzeduration 0 -ss 00:00:00.500 -i pipe: -frames:v 1 -c:v copy -f image2 - -hide_banner -loglevel +verbose true
[1/30/2022, 10:49:38 PM] [EufySecurity-1.1.1-beta.15] ERROR: ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers Doorbell] [Snapshot
[1/30/2022, 10:49:38 PM] [EufySecurity-1.1.1-beta.15] ERROR:   built with gcc 9.3.0 (Alpine 9.3.0) Doorbell] [Snapshot
[1/30/2022, 10:49:38 PM] [EufySecurity-1.1.1-beta.15] ERROR:   configuration: --pkgconfigdir=/build/workspace/lib/pkgconfig --prefix=/build/workspace --pkg-config-flags=--static --extra-cflags=-I/build/workspace/include --extra-ldflags=-L/build/workspace/lib --extra-libs='-lpthread -lm' --enable-static --disable-debug --disable-shared --disable-ffplay --disable-doc --enable-openssl --enable-gpl --enable-version3 --enable-nonfree --enable-pthreads --enable-libvpx --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libx264 --enable-runtime-cpudetect --enable-libfdk-aac --enable-avfilter --enable-libopencore_amrwb --enable-libopencore_amrnb --enable-filters --enable-decoder=h264 --enable-network --enable-protocol=tcp --enable-libspeex --enable-demuxer=rtsp --enable-indev=alsa --enable-outdev=alsa Doorbell] [Snapshot
[1/30/2022, 10:49:38 PM] [EufySecurity-1.1.1-beta.15] ERROR:   libavutil      56. 51.100 / 56. 51.100 Doorbell] [Snapshot
[1/30/2022, 10:49:38 PM] [EufySecurity-1.1.1-beta.15] ERROR:   libavcodec     58. 91.100 / 58. 91.100 Doorbell] [Snapshot
[1/30/2022, 10:49:38 PM] [EufySecurity-1.1.1-beta.15] ERROR:   libavformat    58. 45.100 / 58. 45.100 Doorbell] [Snapshot
[1/30/2022, 10:49:38 PM] [EufySecurity-1.1.1-beta.15] ERROR:   libavdevice    58. 10.100 / 58. 10.100 Doorbell] [Snapshot
[1/30/2022, 10:49:38 PM] [EufySecurity-1.1.1-beta.15] ERROR:   libavfilter     7. 85.100 /  7. 85.100 Doorbell] [Snapshot
[1/30/2022, 10:49:38 PM] [EufySecurity-1.1.1-beta.15] ERROR:   libswscale      5.  7.100 /  5.  7.100 Doorbell] [Snapshot
[1/30/2022, 10:49:38 PM] [EufySecurity-1.1.1-beta.15] ERROR:   libswresample   3.  7.100 /  3.  7.100 Doorbell] [Snapshot
[1/30/2022, 10:49:38 PM] [EufySecurity-1.1.1-beta.15] ERROR:   libpostproc    55.  7.100 / 55.  7.100 Doorbell] [Snapshot
[1/30/2022, 10:49:38 PM] [EufySecurity-1.1.1-beta.15] ERROR: [h264 @ 0x56321db4b100] Reinit context to 1600x1200, pix_fmt: yuv420p Doorbell] [Snapshot
[1/30/2022, 10:49:38 PM] [EufySecurity-1.1.1-beta.15] ERROR: [h264 @ 0x7f49e08c9840] Stream #0: not enough frames to estimate rate; consider increasing probesize Doorbell] [Snapshot
[1/30/2022, 10:49:38 PM] [EufySecurity-1.1.1-beta.15] ERROR: [h264 @ 0x7f49e08c9840] decoding for stream 0 failed Doorbell] [Snapshot

@onkelfarmor Your log seems to indicate the snapshot is the issue? Try and increase probesize if you have access to the js file. The probesize and analyse duration are tricky since we want the fastest snapshot possible but of course without exception.

@njuart Odd. Works here. Can you share your log? See what the exception is?
https://user-images.githubusercontent.com/10818044/151719123-cbbb04f3-fb9a-4acf-9c8c-56841b9d5f4e.MOV

I can confirm for sure, double checked couple of times, with Enable Debug flag it's not reproduced, as soon as I disable Debug, stream cuts off after 3 seconds 😵💫

@njuart Hehe. Does the log show anything without enabledetailedlogging of 0?

@onkelfarmor Your log seems to indicate the snapshot is the issue? Try and increase probesize if you have access to the js file. The probesize and analyse duration are tricky since we want the fastest snapshot possible but of course without exception.

Where do I find the js file? And what is the name?
I get the snapshots, the errors start, when I try stream
[Doorbell] The image snapshot handler for the given accessory is slow to respond! See https://git.io/JtMGR for more info.
[1/30/2022, 11:03:45 PM] [EufySecurity] Doorbell Starting video stream: 1280 x 720, 30 fps, 299 kbps (AAC-eld)
[1/30/2022, 11:03:45 PM] [EufySecurity] Doorbell_video Error in NamePipe listen EACCES: permission denied ./2.sock Error: listen EACCES: permission denied ./2.sock
at Server.setupListenHandle [as _listen2] (node:net:1317:21)
at listenInCluster (node:net:1382:12)
at Server.listen (node:net:1480:5)
at new NamePipeStream (/homebridge/node_modules/homebridge-eufy-security/src/accessories/UniversalStream.ts:54:21)
at StreamInput (/homebridge/node_modules/homebridge-eufy-security/src/accessories/UniversalStream.ts:83:12)
at StreamingDelegate.startStream (/homebridge/node_modules/homebridge-eufy-security/src/accessories/streamingDelegate.ts:428:41)
at runMicrotasks ()
at processTicksAndRejections (node:internal/process/task_queues:96:5)
[1/30/2022, 11:03:45 PM] [EufySecurity] Doorbell_video FFmpeg exited with code: 1 and signal: null (Error)
[1/30/2022, 11:03:45 PM] [EufySecurity] Stopping session with id: 8f8923d2-adbc-4e18-9398-a6adc88327fb
[1/30/2022, 11:03:45 PM] [EufySecurity] No session to stop.
[1/30/2022, 11:03:51 PM] [EufySecurity] Stopping session with id: 8f8923d2-adbc-4e18-9398-a6adc88327fb
[1/30/2022, 11:03:51 PM] [EufySecurity] Doorbell Eufy Station stopped the stream. Stopping stream.
[1/30/2022, 11:03:51 PM] [EufySecurity] Stopping session with id: 8f8923d2-adbc-4e18-9398-a6adc88327fb
[1/30/2022, 11:03:51 PM] [EufySecurity] Doorbell Stopped video stream.
[1/30/2022, 11:03:51 PM] [EufySecurity] Doorbell Stopped video stream.
[1/30/2022, 11:03:51 PM] [EufySecurity] Doorbell Error occurred closing socket: Error [ERR_SOCKET_DGRAM_NOT_RUNNING]: Not running
[1/30/2022, 11:03:51 PM] [EufySecurity] Doorbell_audio Error in NamePipe listen EACCES: permission denied ./4.sock Error: listen EACCES: permission denied ./4.sock
at Server.setupListenHandle [as _listen2] (node:net:1317:21)
at listenInCluster (node:net:1382:12)
at Server.listen (node:net:1480:5)
at new NamePipeStream (/homebridge/node_modules/homebridge-eufy-security/src/accessories/UniversalStream.ts:54:21)
at StreamInput (/homebridge/node_modules/homebridge-eufy-security/src/accessories/UniversalStream.ts:83:12)
at StreamingDelegate.startStream (/homebridge/node_modules/homebridge-eufy-security/src/accessories/streamingDelegate.ts:528:45)
[1/30/2022, 11:03:51 PM] [EufySecurity] Doorbell_audio FFmpeg exited with code: 1 and signal: null (Error)
[1/30/2022, 11:03:51 PM] [EufySecurity] Stopping session with id: 8f8923d2-adbc-4e18-9398-a6adc88327fb
[1/30/2022, 11:03:51 PM] [EufySecurity] No session to stop.

@onkelfarmor
Your streaming error is known. And described in this issue I believe.
https://github.com/samemory/homebridge-eufy-security/issues/308
I think we will rewrite the local streaming to not write to a local temporary file.

@njuart Hehe. Does the log show anything without enabledetailedlogging of 0?

This

@onkelfarmor
Your streaming error is known. And described in this issue I believe.
#308
I think we will rewrite the local streaming to not write to a local temporary file.

Sorry, didn’t get that - will try on a spare RPI instead.

@njuart Might be some timing issue then. Never run without enableDetailedLogging :)

@njuart Might be some timing issue then. Never run without enableDetailedLogging :)

I see, let me know if there's anything else I can help with regarding this.

@njuart Hehe. Does the log show anything without enabledetailedlogging of 0?

This

Odd. Looks like the station stops the stream. Since the first line of logging originates from here:
https://github.com/samemory/homebridge-eufy-security/blob/e82d4ad866cecc09382946f8e6b7690f3557076f/src/accessories/streamingDelegate.ts#L593
Could you try the following.

Open home app.
Let snapshot generate.
Wait a few seconds and start the stream?

https://github.com/samemory/homebridge-eufy-security/blob/e82d4ad866cecc09382946f8e6b7690f3557076f/src/accessories/streamingDelegate.ts#L593
Could you try the following.

Open home app.

Let snapshot generate.

Wait a few seconds and start the stream?

Nope, still the same, stream stops after 3 seconds. So until it stops it's fine, I mean I see real stream like usual, then it stops with No Response. With Debug enabled works fine (decided to check one more time).

@njuart I got home and can reproduce your error. Will take a look tonight.

@onkelfarmor
Your streaming error is known. And described in this issue I believe.
#308
I think we will rewrite the local streaming to not write to a local temporary file.

Sorry, didn’t get that - will try on a spare RPI instead.

Works perfect in a non Docker install.

@onkelfarmor
Your streaming error is known. And described in this issue I believe.
#308
I think we will rewrite the local streaming to not write to a local temporary file.

Sorry, didn’t get that - will try on a spare RPI instead.

Works perfect in a non Docker install.

Great. If possible this will be refactored to not use these local files.

@lenoxys What is this awaited promise used for exactly? If I remove it the stream works with enableDetailedLogging of 0.
https://github.com/samemory/homebridge-eufy-security/blob/436419a121cb9fce3c61fc354f390747e049df14/src/accessories/streamingDelegate.ts#L526

it's because we receive the sound too early both stream (video and audio) have their own timeout.

@lenoxys  How did you come up with 6 seconds?

@lenoxys  How did you come up with 6 seconds?

Hum by tweaking until audio and video match correctly :*

Any luck with the issue by chance?

Just a note to say that my T8420 now streams without any modification so I assume Bropat has fixed the client library. 👍🏼
It also looks that the HomeKit alarm accessory works properly too, but I've not tested it fully yet.

I have snapshots and stream working but only once debug is enabled without I see same errors of no response on camera and snapshots don't seem to generate
I am however also seeing lot of messages about the snapshot taking too long to fetch and that I should run in unbridge mode?

Yeah it's a good idea to bridge the plugin

Just a note to say that my T8420 now streams without any modification so I assume Bropat has fixed the client library. 👍🏼
It also looks that the HomeKit alarm accessory works properly too, but I've not tested it fully yet.

I had to manually change Beta 19 to once again force the vcodec to be "copy" so I'm not sure if this has been fixed in the client library or not now.  FWIW it'd be handy to have an option in this plug-in's settings to manually override the vcodec setting.  A check box would be fine which, if checked, would use "copy" otherwise it would use the stream-provided codec.

Yeah it's a good idea to bridge the plugin. at least it won't slow down only Eufy accessories. I'm thinking about providing specific bridge for each camera.

I am running it inside a child bridge which did gain some performance but still receive messages looks as though the snapshot takes while to retrieve could be down to my older pi 3b rather than 4 causing the issue


Support for different storage impl

Hi there,
Nice job! Are you planning to support multiple storage implementations like jpa, or custom ones? thank you :)

Planning to support JPA/Hibernate, though the code I'd ultimately want should be readily adaptable in case we transition in the future.  We're using Spring too.

How can I help to make it happen faster?

Persistence mechanisms were requested in the past and make a lot of sense. PR's are appreciated!
@georgerb Maybe you can team up with bphilipnyc to make it happen faster if he is interested.

Oh sorry @georgerb - please disregard my note.  Misread your message thinking that it was originally coming from @Nick-Triller.  But we can surely collaborate if you're interested!

@bphilipnyc of course I am, just pass me your coding guidelines, javadocs, and where should I start looking at to implement the jpa storage.

@bphilipnyc of course I am, just pass me your coding guidelines, javadocs, and where should I start looking at to implement the jpa storage.

Was there any progress on this issue? Thanks.

Oh wow. what happened? I am looking at the code to see if it can be extended to support apache derby storage


Web: allow to run extension tests

Currently seems to not set the extension tests path properly.

There are two issues:

the environment attribute 'extensionTestsPath' needs to be passed properly.
when running VS Code in the web UI or in a remote setup, the value of the 'extensionTestsPath' attribute must be a valid path that exists in the filesystem. This is a problem because for historical reasons the path generated by yeoman is ${workspaceFolder}/out/test/suite/index which does not exist in the filesystem (because it lacks the file extension .js). When running tests locally, the non-existing path is not an issue because mocha will find the tests anyway.

The first problem has been fixed in VS Code.
The second problem requires a change in "node-debug2".

Verification: open a hello world extension in the web and run its unit tests.

The tab where the VS Code instance runs the tests is not closed automatically at the end.

I hadn't published node2 with the fix. Doing that now, we can reverify this tomorrow.

Verification is trivial:

open a hello world extension in the web and run its unit tests.

Verified using https://github.com/microsoft/vscode-extension-samples/tree/master/helloworld-test-sample and running the tests.


New schedule component for day wise sessions

Clone of https://reactnexus.com/#schedule with proper credits

Closed by #37 by @Mithun-750


not an issue more of a question

will you add support for ctgp7 to be streamable and other homebrew apps to support arctic base?

Hello, it's not possible to stream Ctgp7. About homebrew it's listed in the readme


Set custom parser in non-hacky way (fixes #12)

This should not be merged until a new version of eslint (> 1.3.1) is released.
This will likely happen by 9/11 or 9/12.

Failing because the list of repos in the clone test is out of date. Made an issue to work on improving this situation: https://github.com/feross/standard/issues/253

Nice! :+1:

This was merged in https://github.com/Flet/standard-engine/commit/0f606e146f7048c1d86c46b1c2f119f2786af475


Style/weekview

Added weekview
Changed dayview html page to enable weekview
added get event's div attrributs function so i could use it in weekview
added dayview fixture
added color arg to create_event function

Codecov Report

Merging #217 (195dfb7) into develop (b5f0545) will decrease coverage by 0.11%.
The diff coverage is 95.91%.

@@             Coverage Diff             @@
##           develop     #217      +/-   ##
===========================================
- Coverage    99.38%   99.26%   -0.12%
===========================================
  Files           37       38       +1
  Lines         1456     1496      +40
===========================================
+ Hits          1447     1485      +38
- Misses           9       11       +2

Impacted Files
Coverage Δ

app/main.py
96.55% <ø> (ø)

app/routers/dayview.py
97.87% <88.88%> (-2.13%)
:arrow_down:

app/routers/event.py
96.77% <100.00%> (ø)

app/routers/weekview.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b5f0545...195dfb7. Read the comment docs.


nested message payload not correctly serialized

method used: chat.postMessage
When I try to post a message with attachments, the attachments are not 'seen' by the slack service. I tried encoding them to json prior to sending them to slack via a guzzle event listener like this:
$guzzle->getEmitter()->on('before', function (BeforeEvent $before) {
    $body = $before->getRequest()->getBody();
    $attachments = $body->getField('attachments');
    $body->setField('attachments', json_encode($attachments));
});

And got what I expected (a chat message with attachments). However, this leads me to think there is something wrong with the serialization process of nested payload entries.
In short, my findings:
//does not work
[
    'attachments' => [
        ['title' => 'hello world']
    ]
]

//does work
[
    'attachments' => "[{'title': 'hello world'}]"
]

Wating approve #14

@eelkevdbos Sorry to keep you waiting so long, the PR has been merged. I will be making a new release soon; you can use the master version in the meantime.


Teach Page Rebasing

Fixes #[Add issue number here]
Changes:
upstream: main --> origin: teach
Screenshots of the change:

Hi @Gracia-zhang, I have seen your past PRs.
Am I correct here that you want to match the teach branch to the latest main?
What you are doing is correct, but in the end, you need to resolve conflicts between branches.

You can see what exactly is in conflict here.

Conflict resolution is simply the process of deciding whether to prioritize current or past commitments.
As far as I can ascertain, the conflicts this time are for 2020-2021 commits (8b55a3278e31b40b510f21c09866460235c0e718, 899b132100194ebbc255a54d66818871777e8572, etc.) and it is better to ask @yinhwa for help.
Also, regarding the PR outline, it would be great if you could include a description of why you want to do it and how the PR will solve the problem. This will help us in the present, but also for someone in the future😸.

Hi @3ru thank you so much for the kindly instruction for Gracia :) I really appreciate it. We'll discuss on the conflicts soon and get the rebasing done!

Thanks @3ru for your detailed guide! I'll discuss with @yinhwa and I think we'll figure it out soon!

Hi @yinhwa , I reverted the Spanish version of es.yml file on line 1707-1715 with GitHub editor. I think we are good to merge!

Think you can reopen the post updates PR now @Gracia-zhang  yay~💙💙


provides generator to set const insecure for services

Codecov Report

Merging #13 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #13   +/-   ##
=======================================
  Coverage   62.29%   62.29%
=======================================
  Files          18       18
  Lines         602      602
=======================================
  Hits          375      375
  Misses        172      172
  Partials       55       55

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f53533d...8933b17. Read the comment docs.


Whitelist passed process.env variables

It creates a security hole if you pass init variables to your application through environment. If any security tokens are used, raven passes them through, which is an undesired behavior.

I'd +1 for just removing this functionality entirely out of raven-node. I just checked, and our python implementation doesn't capture environment variables at all by default.

https://github.com/getsentry/raven-python/blob/ed91eef804eb1453ca625270fe1b67e2b687b89b/raven/utils/wsgi.py#L27
looks like python whitelists as well

anyway, I will reopen PR from different branch. Will close this one. Sorry about that.

Those are different. In python, specifically for web servers, these are related to the HTTP implementation, specifically a part of the WSGI spec. These aren't environment variables on the machine, or environment variables global for the process as in process.env here.

I see your point. I guess I will rip out process.env from here and create a new PR for this. For now, we just added this whitelist functionality to make sure we do not pass anything secured.


implement CommandParser

Implemented A Begin CommandParser which parses until the next \end. Doesn't support nested environments.
Should I execlude the other files (fonts and images)?

At least, I think I know the source of changes in the .ttf and .png files. This was a misconfiguration of .gitattributes. I have fixed it in #327, but I guess I really need to cherry-pick that onto master.
Don't worry about that, I'll fix it in your branch myself.

Now, having the binary files' issue resolved, I am ready to take a look at the code for real. Sorry, it takes some time, because I am usually looking at the PRs at night or on weekends.

No problem, thanks for looking at it! 👍

Ok, your solution works (but generates exceptions sometimes, I'll investigate), but I think a more complete fix is possible. I'll work on your branch a bit.

Sure, if you want me to do anything let me know


CustomTab example could be enhanced

Lacks a Control example or a comment indicating that Control is necessary inside the CustomTab

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 9a1c62c2-4463-cfe3-2ecb-f4565204ab89
Version Independent ID: a8df560c-8653-f5f9-f2aa-63af7e26eb30
Content: CustomTab element in the manifest file - Office Add-ins
Content Source: docs/reference/manifest/customtab.md
Product: non-product-specific
Technology: add-ins
GitHub Login: @o365devx
Microsoft Alias: o365devx

@f-santos thanks for bringing this to our attention. Control is not actually a valid child element of the CustomTab element -- the fact that it is listed as such is a mistake in the docs. I've submitted a pull request (#936) to address this issue, and you'll see this change reflected in the live docs later this week.  (This doc update will be in the queue for the next publication of doc updates, which should happen tomorrow.) Thanks again!


Add Roslyn analyzers to help with common Autofac usage issues

xunit ships with a set of Roslyn analyzers that help with common testing errors. This might be a good thing for common Autofac usage issues, too. Things we might be able to include:

Only register IStartable and AutoActivate() things as singletons to avoid multiple instantiations or start executions.
Missing .As<T>() on delegate registrations.

Unclear if this may be overkill, but it could turn into something pretty handy if we can help address some of the challenges seen in StackOverflow.

Repo's created, AppVeyor project is in place, and it looks like proposed initial analyzers are being tracked in #2. I'm going to close this one.


word Alignment

Been working on incorporating the library into nRF52840 (Nordic) test project

Flash device: MX25R6435F (on the DevKit)
Connected via QSPI

Been getting a lot of error on word alignment, but the most problematic one is inside spiffs_page_move the code requests via hal_write_f to write a page of 1 byte
As the common QSPI flash are word aligned (32bit) AFAIK, is this 1 byte write by design?
Is there a flag somewhere to control and make sure everything is word aligned in read/write operations?

Is this project in active development?
will a demo project for Nordic DevKit would help?

No pretty dormant. One byte writes are per design see wiki.
Den mån 6 apr. 2020 08:35Shahar Hadas<EMAIL_ADDRESS>skrev:

Is this project in active development?
will a demo project for Nordic DevKit would help?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/pellepl/spiffs/issues/264#issuecomment-609593283, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ABG42V326TJ5A3WRA3YMZRTRLFZ4NANCNFSM4LFVYZLQ
.

Maybe worthing mentioning it on the main README.md - alignment is a big thing on embedded devices which uses QSPI flash ...

Hi, I'm curious what's the actual problem here. QSPI flashs don't have word/dword alignment. At least I've never seen one. You can write a single byte to any given address.
How do you come to this conclusion? Even your device MX25R6435F doesn't mention anything like this in the datasheet.

The nRF528xx chip has a QSPI interface which is 4 byte aligned (pretty standard).
SPI Flash is different (meaning you can have 1 byte read and write ... and requires software to manage all the communication) from QSPI as it does most of the heavy lifting in the background.
From the Nordic InfoCenter QSPI
Look at WRITE.CNT and READ.CNT for example:

Thanks for the explanation. Now I understand it. So it's actually an MCU dependent issue. And yes, I assume that most 32 bit systems with a QSPI interface will have this alignment. You could implement a read-modify-write operation for unaligned accesses. But this would lead to bad performance.
In my projects I used STM32 which is an ARM too. But my specific model doesn't have a QSPI interface and I'm only using manually implemented SPI operations.
I might try littlefs sameday too. My main reason would be to have directories. The point that it is under active development is a good reason too.

Thanks for the explanation. Now I understand it. So it's actually an MCU dependent issue. And yes, I assume that most 32 bit systems with a QSPI interface will have this alignment. You could implement a read-modify-write operation for unaligned accesses. But this would lead to bad performance.
Correct ... which makes the whole QSPI interface optimization of 4IO (read/write). Further more, most MCU which has QSPI interface, can even use some of the QSPI flash as XIP (execute in place) which makes the useable flash size for code much bigger than the on-board flash.

In my projects I used STM32 which is an ARM too. But my specific model doesn't have a QSPI interface and I'm only using manually implemented SPI operations.
Some STM do have QSPI interface - it is becoming more and more popular today.

I might try littlefs sameday too. My main reason would be to have directories. The point that it is under active development is a good reason too.
Worth mentioning that Espressif has their own for SPIFFS in their SDK Espressif spiffs - but I don't think they are compatible and it's designed with FreeRTOS in mind and their ESP32 line of MCUs

You could implement a read-modify-write operation for unaligned accesses. But this would lead to bad performance.

I thought that as well, and was on the verge of implementing something that did that, until a colleague pointed out this is not necessary.  At least with any flash chip which allows "writing" bytes that are currently erased to a value of 0xFF and thereby leaving them still erased (which I believe is just about any flash chip anyone looking at this would be using, but feel free to correct me if I'm wrong).
If you want to change a single byte within a word (the third one, say) and leave the others alone, don't do this:

read current value -> e.g. [0x12, 0xfe, 0xff, 0xff]  // note: last two bytes are erased in this example
modify buffer -> [0x12, 0xfe, 0x01, xff]
write buffer back to flash, leaving bytes 0, 1, and 3 as they were and changing 2 from 0xff to 0x01.

Instead, just make a buffer with all bytes "erased", then change and write it:

make buffer: [0xff, 0xff, 0xff, 0xff]  // no read involved
modify your target byte in the buffer: [0xff, 0xff, 0x01, 0xff]
write it out...  this will leave bytes 0, 1, and 3 as whatever they were before, since you tried setting their bits all to 0xff, but since you can't change a 0 to a 1, they'll just stay as they were.

Hope that's sufficiently clear...  The flash contents before and after are the same for the two cases, but one of them didn't need to read the contents at all.

Worth mentioning that Espressif have their own version of SPIFFS in their SDK Espressif spiffs - but I don't think they are compatible and it's designed with FreeRTOS in mind and their ESP32 line of MCUs

Not really. Espressif SDK uses upstream SPIFFS without any modifications. The files in the directory you have linked are just the porting layer, the adaptation layer to the virtual filesystem framework of the SDK, and the tests.


File format tooltip copy

tooltip documentation for the info icon next to "File Format" that describes the file format options

Dannielle to decide if additional supporting documentation explaining file format is needed (outside of the copy in the tooltip)

Link to design: https://www.figma.com/file/gwX2PjfGPfMrbEWMMJq6rW/H1-2024---Annotations-Table-Update?node-id=527%3A139630&mode=dev

Copy is here. Approved by @uermel


session_0/kavyabhat

Please review my code for fizz_buzz and item.

Feel free to ask @addhyanmalhotra if you need hints for the word processing problem

Feel free to ask @addhyanmalhotra if you need hints for the word processing problem

Thanks, but I haven't gotten around to solving the word_processing problem yet, I'm still working on session_0.
Also, thanks for the suggestions, I'll implement them in the code.

Thanks, but I haven't gotten around to solving the word_processing problem yet, I'm still working on session_0.
Also, thanks for the suggestions, I'll implement them in the code.

Closed due to a more recent PR: #42

Closed due to a more recent PR: #42


Improve function to add an individual to an existing tree sequence

The current add_individual_to_tree_sequence function adds one sample path at a time and associates each with a separate individual. So, two sample paths from a diploid individual are treated as separate sample paths from two separate individuals. Obviously, this is wrong, but it did the job when I needed something quick. It needs to be implemented and tested properly.
See #96 and #97

It is much faster to use append_columns to add a bunch of new edges at once.

There are more TODOs in the comments, but good for now.


cliques are not detected by grouping

yes, this is the example

Please specify an example for a non detected clique

solved by #461


Sneaking While Mining Single Slabs Causes Duplication

If you sneak while mining a single slab, it drops 2 slabs instead of one. Seems to happen for all types of slabs(wooden, stone, etc.), but I haven't tested with modded slabs.
Minecraft Version: 1.20.1
AutoSlabs Version: 1.0.1
Video:
https://github.com/Andrew6rant/Auto-Slabs/assets/89557012/2d8c4484-e679-4243-81de-a8297c0a4a92
Other mods present in the video were Mod Menu and Fabric API.
Currently working on a PR to fix if you haven't fixed it locally already.

Whenever I tried removing that if statement, it fixed the duplication when sneaking, but then caused duplication for mining a single slab on a 2 slab block when not sneaking. That's what I've kept the if statement. Should the pull request get merged, you would still be good to mine vertical slabs, sneaking or not.

Yeah that sneaking check was to try and fix the dupe bug (meant for KleeSlabs functionality). Didn't realize it caused another dupe, haha


Why pl.Trainer can not handle multi-gpu case?

I can run the original tutorial_train.py with single 3090Ti GPU (24G) with batch_size 3.
However, when upgrade to 2 or more gpus, it keep warning OOM.
trainer = pl.Trainer(gpus=2 precision=32, callbacks=[logger])

I am curious why? Why single GPU can handle batch 3 while multi-GPU can only handle 1?? The GPUS hold batches on their own parallelly, am I right?

because one gpu need to compute "gradient = (gradient_from_gpu_1 + gradient_from_gpu_2) / 2"
This computation will take many vram.

gradient_from_gpu_1 + gradient_from_gpu_2

Thanks @lllyasviel !
So basically bottleneck is the one holding gradient averaging, while remaining should work fine, e.g., GPU0 require 24G+24G; GPU1 require 24G; GPU2 require 24G; GPU3 require 24G.
THUS, we should ensure GPU0 some space, e.g., work with GPU0 12G+12G, GPU1 12G, GPU2 12G, GPU3 12G.

Sry I got question again.
I travel from recognition community. In recognition, normally the multi-GPU training won't result in significant different RAMs among GPUs. Does this "1-big-gpu" thing only happen in stable diffusion/control net?

use fsdp or deepspeed training strategy

HuggingFace Diffusers ControlNet training script https://huggingface.co/docs/diffusers/training/controlnet has different optimizations builtin

all duplicates concerning "Multi GPU"
https://github.com/lllyasviel/ControlNet/issues/148
https://github.com/lllyasviel/ControlNet/issues/314
https://github.com/lllyasviel/ControlNet/issues/319
https://github.com/lllyasviel/ControlNet/issues/507


Check scalafmt directly with sbt

It is better to use sbt directly to check for format correctness. The action I used earlier is not actively maintained
This version uses sbt and maybe a bit slower but will support upgrades better

LGTM


[global] Windows support?

Following instructions gives errors on windows 10. This is probably not a surprise. Too bad.

Unfortunately we do not have the resources to test on windows. People had good luck with running the project for the web and react native parts on windows in the past. It is possible that later changes, or newer versions of windows, have broken that capability. Windows versions of scripts are located in urb-tools-windows. Will happily merge PR enabling/fixing/extending windows support.

Thanks. That helps.


Switched to Algolia for searching

Closes #3

@Haroenv Good news! I (finally) got around to doing daily indexing of definitions in Algolia. I have another project up on Heroku that fetches and indexes the definitions daily. The latest version of search-flow-typed is now using it to search!

Nice!


Consistency between type enum swh and value swh-identifier

The type is swh, but the definition is swh-identifier:
https://github.com/citation-file-format/citation-file-format/blob/52647a247e9b1a5b04154934f39615b5ee8c4d65/schema.json#L436
https://github.com/citation-file-format/citation-file-format/blob/52647a247e9b1a5b04154934f39615b5ee8c4d65/schema.json#L441
https://github.com/citation-file-format/citation-file-format/blob/52647a247e9b1a5b04154934f39615b5ee8c4d65/schema.json#L1693
Could these be made uniform like doi?
Thanks in advance.

Hey @abelsiqueira, thanks for this issue.
swh is the user-facing enum value, and saves people some typing. I don't think it's strictly necessary for it to be the same string, and I like that it's a three-letter value, just as doi and url. But can I ask if this is more than cosmetic, and does actually hinder some use case, such as the implementation of a solution? In this case I'd be happy to discuss changing it in future versions.

Hinder is a strong word :)
I was working on cffinit in the validation of the identifiers and I needed to check for the schemaPath #/definitions/XXX/pattern (XXX being doi, url or swh-identifier) given the type (doi, url, swh).
A dictionary solved it. Namely:
schemaPath: `#/definitions/${linkInfo[props.type].type}/pattern`

And we already had a dictionary in place so it was not a hassle, but it made me realize the issue.
Either way, it is not urgent because we have schema 1.2.0 fixed.

Okay, thanks for clarifying this. I'll mark this as discussion and add it to the next milestone, but will close for now. We can pick this up when we discuss the issues going into the next version.


Feature request: Different light and dark mode highlight "theme" color

Feature: Use different highlight "theme" color for light and dark mode.
Currently the same customizable color is used as highlight "theme" color for both light and dark mode. This is not ideal, since there are very few and limited color options that work well in both dark and light mode as a highlight "theme" color.
It would be useful if it was possible to specify separate custom colors for light and dark mode highlight color.

Closing as this is supported now

Awesome 👍 Going to make use of it soon. Thanks 🙏💙

Does not seem to be working for me:
Config;
https://github.com/rydmike/flex_color_scheme_docs/blob/master/docs.json#L18-L24
Site result, still gets docs defaults:
https://rydmike.docs.page/flex_color_scheme_docs/all_themes
New issue? Or maybe I'm doing something wrong?

Hmm that config seems correct - re-opening this issue
cc @Ehesp

@rydmike hey so this is because the config is in a sort of limbo where it's using new and old features, but the parser then assumes its the old style config. I've pushed a PR to fix this for you: https://github.com/rydmike/flex_color_scheme_docs/pull/3


Build docker images for jindofuse bug fix

Ⅰ. Describe what this PR does
Ⅱ. Does this pull request fix one issue?

fixes #XXXX
Ⅲ. List the added test cases (unit test/integration test) if any, please explain if no tests are needed.
Ⅳ. Describe how to verify it
Ⅴ. Special notes for reviews

/approve
/lgtm