Add support for buzzer sounds in actions

hello and thank you ..... congratulations ...
I have some requests !!!

the possibility of having the bypass sensor by default ... the security door is always open to exit ... so every time you have to press the 3 dots and select bypas sensor ..... it would be very convenient to put a function that by default all open windows are bypassed by default .... but when the window / door closes the sensor is activated (in traditional burglar alarms it works like this)

emit a beep every second when entering the house until the trigger time is reached .... at least it reminds you that the alarm is on ...

a beep beep after 30 seconds of exit ... as confirmation that the alarm is armed.

thank you

im working on creating automations for exit delay and entry delay audible alerts. however something always breaks. ill post when i find out more, i agree that this would be most needed.
exit delay audible beep whilst alarm is arming.
entry delay audible beep whilst alarm is pending.
i can create these automations but its very frustrating, something always breaks....

request for

Exit delay audible notification (buzzer) until alarmo armed.
Entry delay audible notification (buzzer) until alarmo disarmed.
Open sensor disarming and buzzer silence.
Ive managed to create these using some scripts and also alarmo's actions but its quite frustrating to create. I have to created 2 scripts
script one is exit delay buzzer. toggles my siren on and off every 200ms until alarmo is armed.
script two is entry delay buzzer, this toggles my siren on and off every 100ms until alarmo is disarmed.
i then use alarmo action to call these when alarmo is set to arming (exit delay script) or pending (entry delay script)
these work great as long as you dont exceed 30 seconds entry delay, going anywhere over this figure results in the sire triggering constantly once entry delay has completed and alarmed should be armed.
open sensor disarm is so that if i set my alarm and say my front door is open and stays open whilst exit delay completes, alarm should count down exit delay and then not arm and siren should not sound. as it is, on completion of exit delay , alarm remains disarmed but siren triggers and i am unable to disable it by entering code or running any script. i have to restart home assistant and restart my alarm interface hardware.
id like to thank you for your integration, its great and is much appreciated.
im just trying to create similar behavior to a real alarm.
audible notifications i think are needed on exit and entry, especially entry so you know the alarm is set. home assistant is brilliant but ive spent days and days trying to get progress on this, but there are a million ways to do the same thing and inevitably something breaks at some point or another. many thanks

@formatta2003
Regarding

the possibility of having the bypass sensor by default ... the security door is always open to exit ... so every time you have to press the 3 dots and select bypas sensor ..... it would be very convenient to put a function that by default all open windows are bypassed by default .... but when the window / door closes the sensor is activated (in traditional burglar alarms it works like this)

This is very user dependent.
I don't want to make this choice for all users by setting this as default, I rather have the safe/conservative setting as default.
You will have to set this yourself for the doors/windows which should be bypassable.
@formatta2003  @thecheekymonkey
Which device are you using for making beeps, how can it be controlled (which service) from HA?
It is hard for me to include support for actions which are very depending on the device used.
Some users want the beeps via an Alexa media player, some have a siren-like device which makes beeps when turned on, etc.
I'd like to collect the different scenarios so I can see what I can do..

hi there thank you, i am using some konnected hardware interfaces and attaching a small piezo buzzer to the siren output.
i do understand that people want to use different hardware and its difficult to implement with many different hardware scenarios and configurations but i do think that audible alarm for exit and entry is important in an alarm system.
so i propose that maybe if it is possible could it not be implemented by class? i.e. have an option to set a tone sound or user selected media out to a class device?  so if you choose siren you could select a siren that you have installed, or media you could do likewise?
set exit or entry delay
check box for output?
i apologise, my understanding of homeassistant is ongoing, and im very grateful for the work you have done and appreciate the responses.
thank you

@nielsfaber
hi there thank you, i am using some konnected hardware interfaces and attaching a small piezo buzzer to the siren output.
i do understand that people want to use different hardware and its difficult to implement with many different hardware scenarios and configurations but i do think that audible alarm for exit and entry is important in an alarm system.
so i propose that maybe if it is possible could it not be implemented by class? i.e. have an option to set a tone sound or user selected media out to a class device? so if you choose siren you could select a siren that you have installed, or media you could do likewise?
set exit or entry delay
check box for output?
i apologise, my understanding of homeassistant is ongoing, and im very grateful for the work you have done and appreciate the responses.
thank you

@thecheekymonkey Which service / entity are you using to control the buzzer?

@nielsfaber
i am using alarm_control_panel.alarmo
so i have created two scripts to create the buzzer sounds under HA , this is basically a repeated toggle for my siren (on/off) with a delay of 200ms for exit delay and 100ms for entry delay.
i then create an alarmo action to run for instance my "EXIT DELAY" script which does this "Repeat an action until Alarmo is armed away is true"
this makes the buzzer beep for however long i have set exit delay for in alarmo until it is armed. this works fine.
i then create an alarmo action to run for instance my "ENTRY DELAY" script which does this " Repeat an action while Confirm Alarmo is pending is true"
this makes the buzzer beep for however long i have set entry delay (bug see notes) for in alarmo whilst alarm is PENDING. when pending complete, i have another alarmo action for when alarmo state is triggered to just turn the siren on constant.
this is all working great so for with a few caveats.
if i set the ENTRY delay for any longer than 30 seconds, for instance say 40 seconds then when arming alarmo after the 30 second exit beep the siren just beeps constant and basically crashes and i have to restart HA and my hardware instances. if i set ENTRY delay back to 30 seconds or below everything works as it should.
hope this helps, i can post configs or screenshots if you like or need and even a video if needed of the behaviours, so please let me know.
thanks again for the great work

Its a old topic, but im also in a similar situation
While status is pending it should play a sound file on my fire tablet. But it doesnt. When i enter the code an press disarm then the sound will be played.
Any ideas?
`alias: 0_Alarm_pending
description: ""
trigger:

platform: state
entity_id:

alarm_control_panel.alarmo
from: armed_away
to: pending
condition: []
action:

service: media_player.volume_set
metadata: {}
data:
volume_level: 0.2
target:
entity_id: media_player.fire_tablet
repeat:
sequence:
- service: media_player.play_media
target:
entity_id: media_player.fire_tablet
data:
media_content_id: media-source://media_source/local/bedside-clock-alarm-95792.mp3
media_content_type: audio/mpeg
metadata:
title: bedside-clock-alarm-95792.mp3
thumbnail: null
media_class: music
children_media_class: null
navigateIds:
- {}
- media_content_type: app
media_content_id: media-source://media_source
while:
- condition: state
entity_id: alarm_control_panel.alarmo
state: pending
mode: single
`


Improve the batch_count behaviour

Experiment with batching to fetch enough to fill the queue, anymore would be a waste.

Can we increase the batch_size to 100 by default. From a recent user debugging, I saw a big jump in throughput and reduction in CPU utilization when changed from 1 to 500

I am sure we can and also improve the LUA script. See this comment

https://www.elastic.co/guide/en/logstash/current/plugins-inputs-redis.html
documentation not updated


Draenei quest to repair generator does not complete above 5000 hp

The hp of generator increased with capital/gate hp (?) so fully repairing it does not fulfil the requirement to "repair to 5000 hp". Damaging it to below the threshold of 5000 (e.g 4900/8100 hp) and repairing to 5000 completes the quest.

I can't replicate this issue. Here's what I tried:

Cheat -complete "a new home"
Cheat -hp 4998 on the Dimensional Generator
Cheat -skipturns 1 to kick off the HP increase
Observation: quest completed

TurnBasedHitpointsManager actually heals the capital so I'm not sure when it wouldn't trigger the objective completion.
Moving to "needs reproduction steps" column.

Didn't save the replay where it occured, not sure what condition meant it didn't complete and haven't been able to replicate in testing thus far.


Button links missing.php

Clicking tabs or buttons gives correct but incomplete urls
like
http://<IP_ADDRESS>/getstakinginfo
instead of
http://<IP_ADDRESS>/getstakinginfo.php
typing in the .php works and the page with correct info is shown

Manually editing my local copy of header.php so the href links for the tabs have .php works.
but manually changing every <a href="link" to "link.php" seems wrong. digging into php and css to see if there is default we can update to do this.

this is actually handled by htaccess and all you need to do is configure for mod_rewrite in your apache vhost file. This is working on my Pi and will be part of the released image when working. If you want to update your vhost file, I will paste the snippet. PS. Be careful using this version of code, it is not fully tested yet.

/etc/apache2/sites-available/000-default.conf
        # The ServerName directive sets the request scheme, hostname and port that
        # the server uses to identify itself. This is used when creating
        # redirection URLs. In the context of virtual hosts, the ServerName
        # specifies what hostname must appear in the request's Host: header to
        # match this virtual host. For the default virtual host (this file) this
        # value is not decisive as it is used as a last resort host regardless.
        # However, you must set it for any further virtual host explicitly.
        #ServerName www.example.com

        ServerAdmin webmaster@localhost
        DocumentRoot /home/stakebox/UI

        # Available loglevels: trace8, ..., trace1, debug, info, notice, warn,
        # error, crit, alert, emerg.
        # It is also possible to configure the loglevel for particular
        # modules, e.g.
        #LogLevel info ssl:warn

        <Directory /home/stakebox/UI/>
                AllowOverride All
                Order allow,deny
                allow from all
        </Directory>

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

        # For most configuration files from conf-available/, which are
        # enabled or disabled at a global level, it is possible to
        # include a line for only one particular virtual host. For example the
        # following line enables the CGI configuration for this host only
        # after it has been globally disabled with "a2disconf".
        #Include conf-available/serve-cgi-bin.conf
</VirtualHost>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

and make sure you have mod_rewrite enabled

Figured there had to be something to do that.


Print extension errors to the console with a stacktrace.

There's a developer usability problem in vscode. If exception happens during callback handling, the error message isn't shown, and the stacktrace isn't printed to the output. Instead the message that unknown error happened is printed with an instruction to take a look at the log. This is not an optimal developer experience. The error might have happened for vscode's internal reasons. The log is in the running vscode, in the output, and there's no stacktrace navigation there.
This PR adds printing of all such errors in to the console. This makes it obvious what error happened, and allows navigation to the stack trace. It reports only user errors, and not internal vscode errors, thus providing only relevant logging.

@jrieken So, how can you recommend me approach it? I want to print all extension error to the console for easier understanding of want went wrong. Should I just create an onUnexpectedExternalError handler somewhere in the extension host?

Ok. I see what happens in that handler. I plan to add an external error handler infra there. So, closing this PR. Will create a new one.


"Copy link" doesn't work

Hi
how exactly should work Copy link function? When I'm logged, copy link of a file and open it in anonymous window I see only text "Not Found", there is nothing in the app.log.
When I copy link in the window where I'm logged in, it only refresh page in folder where I was (not in the folder with the linked file).
thx

This was broken, let try it out on the next release.

Hello,
also can be the random string a tiny bit longer? or it is proof against brute force attacks?
also have the download links some timeout or they are permanent?
thank you

Random string in the link is just a base64-encoded file path. Link is permanent, unless you move or rename the file itself.
https://github.com/filegator/filegator/blob/fd2d3d2f92871aaad7b254dcd5fbfef741a7bcea/frontend/mixins/shared.js#L124


Added new app settings page

Changes proposed in this pull request:

Added new app settings page based on this google doc.
Added the new doc under 'Wanikani' category.

The credentials to view the Netlify deploy is the following:

username: crabigator
password: googlebegone

Deploy preview for wanikani-knowledgebase processing.
Building with commit 5d657c90b19f740609a346791b042a93ac42ade0
https://app.netlify.com/sites/wanikani-knowledgebase/deploys/5e5ea901497f0b0007d185fe


Update chart

depends on #807

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

LGTM. can we merge it ?


Layout updates to double and integer parameters.

Hi Thomas,
I'd suggest to use HBox instead of FlowPane, because FlowPane will break into 2 (or more) rows if you resize the component down, while HBox keeps all component in a single row.

Done. Shouldn't we then also change the range editors as they are currently using FlowPane?

Good idea.
On Friday, December 4, 2015, Thomas F. Dyrlund<EMAIL_ADDRESS>wrote:

Done. Shouldn't we then also change the range editors as they are
currently using FlowPane?
—
Reply to this email directly or view it on GitHub
https://github.com/mzmine/mzmine3/pull/41#issuecomment-161980154.


gRPC tunneling

Issue for cross-language tunneling coordination, especially reverse tunneling (where the client can become a server).
https://github.com/grpc/grpc-java/pull/3987 (functional Java POC)
https://github.com/grpc/grpc/pull/14100

The internal push for this fell away. We're looking for user interest. I know there is user interest, but it's hard to say how much there is. If you're interested, vote up the initial comment of this issue and make a comment describing the sort of use-case you're looking at (like Android or firewall-punching). Also mention whether the tunnel component would be generic or app-specific (as app-specific is easier to workaround today with a bidi stream).

Well, I will need this (as I definitely do not want to work-around with bidi streams as it's just bad interface for server which needs to connect to client)
So I may as well do pull request for C++ implementation in near-future.

I think I have a use case for this which is an app which allows an Android phone's file system to be explored / read / manipulated via a PC application. Essentially the phone is a file server or hosts a remote file system of sorts.
I've read a comment (I think by yourself @ejona86) whereby you say that a listening server on the phone can be flaky depending on screen off / rom configuration and therefore it's possibly appealing to have the connection set up in the reverse direction (PC listens, Android connects) and then reversing the relationship if I'm understanding this feature correctly.
Still looking into GRPC and not entirely sure this is a sensible use case for it, but it seems right now it'd rely on Android netty server working (not sure on the status of this), or doing some kind of bidi stream workaround (not sure how much of a mess this would be when the phone is very much the server in this relationship).
Not exactly a concrete need from me here, but just a 2 cents on a possible use case that might be useful to me.

The use case is a remote operations management platform (kiosks/iot). We'd like the ability to register a client kiosk to a server, and, on certain events, have the server invert into a client that issues requests to the kiosk. This can currently be done by multiplexing over a bidirectional stream, but would be much better with language support.

I am interested in this feature for a use case which service A submit a job through a scheduler and we have no idea the job ends up running at, and we need to send events to the job from service A.
But I am concern about how we can scale up service A after having a tunnel established to one instance of service A. May result in a lookaside table or sticky session

I have a use-case for this motivated by routability - my client can dial a server today but the reverse is not true. The server wants to make side-channel requests of the client, but these "reverse" service definition is largely orthogonal to the "forward" service definition, so intermingling them using bidi streaming and flipping the reverse service are unattractive options.
(this is for a python & C++ app in non-mobile environments, where the server and client are in independent network contexts)

I have basically the same use case as @fischman, right down to the language requirements. Something like Crossbar's router is perfect for our needs, but we've already gotten pretty invested in gRPC, so I'd rather not just drop it all and switch to WAMP.

More users interested in this enhancement https://github.com/grpc/grpc-go/issues/484

@ejona86 I'd ask on the (grpc.io) discussion group - the sample size will be larger :)

The use case is clients on remote locations behind heavily NATted or downright unreliable connections, like clients which can only connect seldom through mobile wifi hotspots.

My use case:
Servers in cloud and clients behind firewalls and NATs(for example, terminals in shops).
Client firewalls configured only for outcoimng connection on server port.
So when client starts, it try to initiate long live connection to server.
After connection establiashed, server wants to do RPC request to client.
Remark:
I didn't use gRPC before. I only doing first look on this technology and trying to understand can i use it in my project or no. And now i can't understand, how i can do rpc requests from server to clients using gRPC.
I searched on stackowerflow and did't find any solution. This issue looks like that's what I need.
And if you know, how i can implement server to client RPC using existing gRPC mechanics, pls tell me.

I have the same use cases mentioned by fischman and Frank591 above

Chiming in with an IOT use case where the client IP address is not stable and the client is in an unknown and probably NAT'd network.
Also of interest to those here might be one of the reverse http standards-attemtps that are out there. None have become standardized, but several have been implemented by multiple people.

Github traffic stats shows ~1 unique visitor per day on the code in https://github.com/dustin-decker/grpc-firewall-bypass which is a firewall-punching hack that came out of https://github.com/grpc/grpc-go/issues/484
There's definitely some demand among the go users for this functionality.

Yes, having the native ability to reverse tunnel for C++ / C# would be very useful.
Use case(s) -

remote controlling photo / video kiosks for sport events.
remote video judging for sport events.

In both cases target servers must be are located on event premises, behind NAT/firewall. And calls coming from outside (from cloud).
The topology where on-premise app (protected by firewall) initiates an outgoing connection to the cloud, and becomes an rpc server accepting calls on that connection would solve most of the problems.

Would be very useful for remote IOT devices. I need it in my project and not having this feature makes me hesitate regarding gRPC.

As WCF is deprecated, we want to move to gRPC.
As far as I understand it, this feature request would enable the equivalent to client callbacks.

Use Case:
We have a WebApi (C# asp.net core) in the cloud and a backend (c# winforms) hosted on-premise at our customers.

The backend connects to it from behind the corp firewall as a WCF client and establishes a TCP channel.
If a HTTP request comes in, the webapi calls back to the backend through the client callballback.

This architecture has the really big advantage that we don't need any incoming ports opened for the internal customer network.

@0x53A, to be clear: it is possible to do reverse tunneling today with a bidi stream. However, it can be annoying to tunnel many different things over the same stream and requires you to manually wire things up. This feature is to make type of setup you described easy.

we're looking for node to node communication which would enable something like reverse ssh tunneling. so we could have ad hoc or on demand communication with client node, doing remote upgrade, troubleshooting or proactive deployments of sorts. client nodes could be behind NAT.

I have a use case where a host pc can request the Android device for validating some test cases. Android device is always on. Can we run the server on Android? What would be the ip address?

I'm looking for a solution to allow local gRPC "backend" (behind local network) to register at cloud server and let gRPC clients connecting to that cloud server reach the registered backend (by its registered id, so a certain level of inspection into forwarded traffic would be desirable for routing to different registered backends).
A requirement here is to let encryption / TLS pass through the cloud server uninterrupted (cloud server doesn't know certificates used by backend and client to communicate with each other).
The solution coming to my mind is an ssh tunnel from backend host to cloud server (exposing backend's serving port at cloud server) and some reverse proxy at cloud server that will route client connections to local ports and tunnel them to backends.
Another approach: let backend establish grpc stream connection to cloud server as a "transport" layer, receiving binary traffic as payload in messages and feeding it locally to server hosted in the same app (worst case through loopback network socket, preferably directly to gRPC server implementation). Client would need to do similar trick of sending its traffic to itself so it can be transported by stream connection to cloud server. The advantage here is that client/backend connection to cloud server can use dedicated server's certificate and routing is easy to implement for such transport connection.

Firewall punching for remote management. Having an out of box way would be great, in meantime any example working code either in java or c# to do this manually?

Since I managed to implement a byte-level tunnel in my project, I can share some of the core components here https://github.com/kskalski/snippets/tree/master/GrpcTunnel/GrpcBytesTunnel
It uses the technique I mentioned before, intercepting connection from grpc client by means of local socket server, sending bytes through opaque messages to cloud and on the other side doing the reverse - creating socket client connections and transporting bytes from tunnel to them.
This works, though obviously it would be nicer to feed bytes directly to gRPC library instead of going through local (loopback) socket connections.  I'm not sure if C# lib provides enough abstraction for replacing the transport layer. I will also need a Java counterpart, possibly there it will be easier, since I read that it has replaceable transports.

I am looking for a solution for C++/C#
A solution was found on this thread https://github.com/grpc/grpc-go/issues/484 without modifying grpc code base but it uses go and cannot be implemented in C++/C# since the APIs are different.
I had a hard time understanding the solution in your PR as I am not familiar with the code base. Is it possible to explain what needs to be changed to achieve reverse tunneling? (Server becomes client and vice versa)

The code is still a bit incomplete, but in general it doesn't matter much which direction is the tunneling going to be. You can think of it as party A and party B both connecting to server S and then either A or B can choose to become server while the other side will be a client.
I guess it makes sense to distinguish those use-cases (being a server or being a client) by using different gRPC method for connecting to S (in fact I do have them separate in my code, but I will rename them accordingly as OpenChannelAsServer/OpenChannelAsClient). This way A and B will simply call S using desired method and when S sees the two matching parties it can join them in a connection.

@kskalski Thanks for your reply! I had a bit of time to dig through the code you shared. I still need to dig into it more to understand it but I think it may work for my requirements. Very interesting solution. There is still a couple of things I am confused about currently.
So from my understanding, party A and party B behave as grpc clients that connect to server S and server S forwards calls back and forth using the tunnel.
Questions:
1- Do party A and B, always use the ElectorTunner service when sending messages?  Or is that service used only once to open just open up the tunnel channels after which a custom service can be used to communicate with the other party?
2- If party A uses a custom proto grpc service, shouldn't there be a grpc server registered with the service on party B side? How is the desired method automatically called without a grpc server on party B?
I am having a hard time imagining how the code would look like in both parties.

So from my understanding, party A and party B behave as grpc clients that connect to server S and server S forwards calls back and forth using the tunnel.

Exactly

1- Do party A and B, always use the ElectorTunner service when sending messages? Or is that service used only once to open just open up the tunnel channels after which a custom service can be used to communicate with the other party?

You open up a channel to ElectorTunnel and then you use your own grpc service / client. In fact the current code is a byte-level transport that mimics something like SSH tunnel and it could be even used for tunneling any TCP/HTTP connections, not necessarily gRPC. It's a TPC-over-gRPC implementation.

2- If party A uses a custom proto grpc service, shouldn't there be a grpc server registered with the service on party B side? How is the desired method automatically called without a grpc server on party B?

S is just passing chunks of bytes, it doesn't have any idea about gRPC and the protocol or services that A and B are using. In my use-case A and B use private encryption certificates that are not known to S, so S is simply passing encrypted bytes without knowing what A and B are doing.
The schema is as follows:
A -> A's grpc client -> localhost socket -> raw bytes -> gRPC tunnel -> S -> gRPC tunnel -> raw bytes -> B's socket client -> B's grpc server -> B
Ideally I would prefer to replace the "localhost socket / client" with deeper integration into gRPC - it would limit the power of above schema (no longer being a plain TCP tunnel), but should reduce performance overheads of "loopback data copying" on server and client.

Currently I'm using a remote ssh tunnel to achieve this :nauseated_face: . I want to be able to do it natively on gRPC Python.

@BuBuMan FYI, I updated code and added working project with simple tunnel server S and A/B example client/server ready to check and try. The C# (forwarded) client code is a bit limited, for this part I use Java version, which has more features, but doesn't have a full working example yet.

@kskalski Thanks for the notification. Unfortunately, I decided to use a custom RPC solution since gRPC has quite a few missing features for my use case. I figured adding those missing features to gRPC would take more time and work than just writing a custom solution that meets my exact needs.

@kskalski Where can i find the code you describe. Thanks in advance.

@ptsneves
https://github.com/kskalski/snippets/tree/master/GrpcTunnel/

@kskalski  This unfortunately will not work because underneath there would to be a server grpc class in an hypothetical device. This makes it a no go given the limitation of Android grpc of making a server. The concept of tunnel also feels a bit contrived.
I thought a bit about the topic and i think the tunnel is a wrong approach. I think the correct approach is to decouple the transport server/client concept from the RPC server/client concept. I think this is the core of the problem. People do not want to change the concept of rpc from server to client, they want to change the transport from server to client. Please do not get me wrong, as I have no idea if the gRPC code base has such rpc/transport decoupling in it's design, although i will probably study it.
My naive approach would be like the following in python - grpc server side/transport client side:
grpc_server = grpc.server(futures.ThreadPoolExecutor(max_workers=10)) product_pb2_grpc.add_ProductServicer_to_server(ProductRPC(product), server) server.add_remote_transport('myservice:50051')
In the snipet above as we decouple the grpc from the transport we can then add a remote transport of the form we want. Theoretically we could have our grpc server be available locally with a transport server or as a transport client, just we may have multiple ports.
grpc_server = grpc.server(futures.ThreadPoolExecutor(max_workers=10)) product_pb2_grpc.add_ProductServicer_to_server(ProductRPC(product), server) server.add_remote_transport('myservice:50051') server.add_insecure_port('[::]:50051')

@ptsneves
I think the concepts of "server" and "client" are embedded fairly deep into gRPC's abstractions and approach. However they don't seem to be very limiting and can basically be considered more as "initiator" and "receiver" of communication channel, since after channel is initiated you can exchange messages freely and in any direction (using steaming rpc). One could imagine an extension of gRPC which defines "bi-direction" agents, such that if both are available (on local network? on tunnel service?), then both can initiate communication, but it just seems redundant with the capabilities of current schema.
My implementation has an obvious flaw of "working-around" difficulty of replacing transport in gRPC and it does it by creating regular server and client and then piping them locally to a transport. This is far from good and as you mention might even break on some platforms (fortunately in my case Android runs the client, not a server). The improvement of that implementation would do something similar to what you suggest - create a dedicated transport that would plug server/client directly into communicating with my tunnel service. It might or might not be possible (and maybe on some, but not all platforms) to implement using current grpc libraries, I looked at it and current abstraction of replacing transport is way too high level for the needs of tunnel service (it operates on events like rpc call, start, end, etc., while in my case I need to replace transport below encryption layer).
The tunnel service is probably the only part that is fully flexible and closest to target design, since it is the remote part of your proposed "transport" and just proxies the bytes between interested parties.

@kskalski Thanks for the thorough answer, my apologies for keeping dragging the subject. Comments inline:

I think the concepts of "server" and "client" are embedded fairly deep into gRPC's abstractions and approach. However they don't seem to be very limiting and can basically be considered more as "initiator" and "receiver" of communication channel, since after channel is initiated you can exchange messages freely and in any direction (using steaming rpc).

Given my android server limitation I am starting to think this may be the only way forward for me. I see one problem though: If i invert the semantics with the bidirectional abstraction, will i not need to initialize all the possible calls that my server might be interested in? If so, what about having connections open with no traffic for a long time, with possibly no traffic ever going through. This could very fast lead to resource exhaustion, with threads and sockets waiting and allocated in RPCs x connections.
(I need to leave now so I will resume my answer soon)
Pozdrawiam

@ptsneves you didn't describe your use-case, but from what you say I suppose you just need a way for cloud server to initiate messages to your android app (e.g. notifications?). This is precisely what bidi streams solve and it's actually quite simple to do it with them - you just open streaming connection and then server will send messages on its stream, client will respond to them. I wouldn't worry too much with resource exhaustion, I believe TCP/gRPC connections can be opened and kept alive (with heartbeats built into gRPC) at quite large scale, e.g. single server can host 1000s of them without problem.
My tunnel code and this issue is handling a rather different use-case: two agents that are not publicly visible or accessible on internet want to "meet" and talk to each other, in this case both of them need to first connect to the 3rd server (on the internet) and communicate through it.

@kskalski  The use case may be a bit complex but here it goes.
I work in embedded devices. Currently I have website where users are able to remotely control an embedded device through this website. The device has it's functionalities available through a grpc server. Be aware that the grpc server is not the device itself. So the website connects to this grpc server and is able to request actions or data streams.
This is nice if I have a static infrastructure where a computer is always available and provides this discoverable device server. What I want now is to make this infrastructure more flexible.
The grpc server is still providing me RPC calls for my website to use, but I want to make the connection be initiated by an Android application to my website. This allows for the flexible infrastructure that i was speaking about. It also requires that the rpc client/server architecture remains, but the transport be inverted.
The bidirectional case looked simple at first but now I am slightly stuck, because i kind of need connection state management. I need to initiate the connection on the android app(transport client) to the website(transport server) on every defined RPCs and then on the website block until my user asks for an action(for example a physical movement). One of the things i am concerned is about timeouts in this dormant connections. Does grpc allow for an infinite deadline? And if so does it auto manage the keep alive of each bidirectional RPC?

Your case can be easily handled by streaming call from android app to the website, this issue is probably not the place to discuss details, but in general I wouldn't worry too much, gRPC is well suited for keeping long-running streaming connections open.

I'm very interested using gRPC for peer-to-peer networking & clustering.
I'm interested in bidirectional RPC for these cases.
For such a case, streaming is a very, very hacky solution.  If I'm going to use streams for two-way RPC requests, I'm basically implementing my own protocol on top of gRPC, (which may even be a second layer of gRPC).
If I have to implement my own protocol anyways, might as well cut out this gRPC layer, do away with all the extra complexity, and implement on TCP directly.
If, on the other hand, bidirectional requests were a first-class citizen, gRPC would be the obvious choice for p2p & clustering applications.
As an aside: In applications I've seen, it's common to want a server to make occasional unsolicited calls to a client.  This is usually implemented through some event system.  Bidrectional streams would greatly simplify this case.

We (sila-standard.org) have a large use-case, where e.g. lab devices / IoT devices should be controlled by a client in the cloud, but institution firewalls prevent connecting to the servers. We would be interested in a standardized (non-hacky solution) :)

Hi @markdoerr @Kent-H , we are facing exactly the same situation (IoT scenario, we want bi-directional RPC) which gRPC right now. We are using streams to listen for commands from behind the NAT, but it's a very hacky solution indeed.
I'd be curious to learn what you did to resolve those problems. Any hints?

Hi @pehlert , SiLA went (to my impression) a rather ugly way - basically: opening a bidirectional channel, client initiated and then sending back RPC commands to a receiving server through this bidirectional channel, please see  this SiLA specification document for a clearer explaination: https://docs.google.com/document/d/1nGGEwbx45ZpKeKYH18VnNysREbr1EXH6FqlCo03yASM/edit
I think the solution is not nice - and should be happening on the gRPC layer transparently.
I would be happy, if this community could add such a feature soon :)

I am having similar requirement where I have few set of client(s) behind firewall, they should initiate the single TCP connection with a server ( publicly exposed). There is a 3rd party client who wants to call these clients. This 3rd party client will connect to publicly exposed server, and server will pass the request to client(now act as server) over same tcp connection established earlier.
I tested the same set up for poc, keeping nginx as publicly exposed server. There is grpc client ( will act as server later).  I ran above setup on different dockers.
a) grpc client ( server) works as bi-di server. It initiates the call to nginx and nginx forwards ( grpc_pass) the request to same client.
- But in this step I observed that there are 2 connections : 1) from grpc-client to nginx 2) from nginx to grpc-client.
- Why is it not using same connection established by grpc client. Why there are 2 connections ?
b) 3rd party clients calls nginx and nginx forwards the request to any of connected servers ( earlier client). Now I can see it is forwarding the request over same connection.

I have a use case for this as well - server needs to be able to act as a client and call RPCs on the client's internal gRPC server. Client is behind NAT/firewall.
Currently it's implemented using bidi streams. Both client and server run "internal" gRPC servers. Client connects to the server's "external" gRPC API endpoint to establish a bidi connection. When the server needs to perform an RPC, it makes a request to its internal gRPC server as if it's talking to a suitable client directly. What "suitable client" means is passed in the metadata of the request. The handler on that internal server looks up the list of connected clients in Redis to locate the "suitable" ones and to which instances of the server they have an open connection. Each client may establish N connections to M servers so a server may need to perform an RPC to a client that is connected to another server. After the set of suitable connections is found, handler routes the request to the correct server, which then routes the request to the correct connection of the desired client. The client receives the request over that bidi stream it established and then turns it into an RPC to it's own internal server. The response is proxied back.
That whole routing layer is generic and it's trivial to add new endpoints on the server. It's implemented in Go and is available here:

Most of it: https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/tree/master/internal/module/reverse_tunnel
With some pieces that glue everything together: https://gitlab.com/gitlab-org/cluster-integration/gitlab-agent/-/tree/master/cmd/kas/kasapp (see router*.go files)

Thanks @ash2k  for reply. Suppose we have 10 servers ( horizontally scaled app servers) and 3 clients ( behind firewall).
As per my understanding of your project, each of these 3 client will establish bi-di connection(s) to 10 servers directly without any proxy server in between. And when any server needs to talk to any suitable client, it will call the client over the earlier established connection and again there is no proxy server in between.
Can you please correct if I got it wrong.
If we go this way, then we might need to expose our app servers publicly, which doesn't seem a very good idea ?

A client establishes N gRPC connections (over 1 or more underlying TCP connections) to an address. This address is a load balancer with M app servers behind it. Each app server tracks connected clients in Redis+its own address.
When a server (let's call it "routing server") needs to make a request to a certain client, it:

Finds records about all suitable clients in Redis (not all clients may be able to answer a particular request)
Tries to make a request to another server ("gateway server") that has a connection from a certain client
If it fails, another gateway server is tried
If it succeeds, gateway server routes the request to the connected client.

It's described here but there is a lot of unrelated details.

Well, I will need this (as I definitely do not want to work-around with bidi streams as it's just bad interface for server which needs to connect to client)
So I might as well do pull request for C++ implementation in near-future.

Is there any pull request for the c++ implementation for Bidi

Has anyone tried to build a python client similar to what @ejona86 has done in TunnelClient.java ?

I'm expressing interest for this as well for the purpose of firewall-punching.  I know many teams that use VPNs to get around this that would greatly appreciate this as a far simpler alternative.

I am wondering is there any other RPC framework that provide such feature if gRPC is not viable now?

I am wondering is there any other RPC framework that provide such feature if gRPC is not viable now?

Maybe json-rpc? I'm currently using this in my project (Golang), and all it needs is an established connection (net.Conn)

I am wondering is there any other RPC framework that provide such feature if gRPC is not viable now?

You can take a look at RSocket. I'm using this in some projects (Kotlin, Go, JS). Here is a quick overview from my experience:

There is client/server connection-wise (who connects to whom), but both can request and respond equally (just register callbacks on connect/listen)
Transport via TCP or WebSockets
Different interaction models for calls (streaming inputs and/or outputs)
Messages have metadata and data payloads, those are just bytes and can be encoded however you want (e.g. Protobuf, JSON)
For metadata you probably want to use the Composite Metadata Extension, so you can put several things into a single message's metadata (e.g. Routing, Auth)
Supports resumption with missed messages playback (depends on the implementation)
Can be a bit rough around the edges in some places

I've recently built a framework for Go that does exactly this (you can call RPCs on both the client and the server, with an arbitrary transport layer & even closure support: https://github.com/pojntfx/dudirekta)

FYI I submitted a "client channels" impl to grpc-java that allows calling services on the client:
https://github.com/grpc/grpc-java/pull/11023
Here is an example:
https://gist.github.com/bsideup/bd62eb19e7d7cbc590fad07b140c9201
I was also able to successfully connect a Go-based client to it, exposing a service:
https://gist.github.com/bsideup/49a031c3360a68e4542e5597669fe51d
Unlike Gitlab's solution, it uses raw streams to exchange the bytes (HTTP2 over raw gRPC streams without encoding/decoding) and pretty efficient (should close to raw HTTP2 over TCP performance)

Hey everyone! Quick update:
The above is now developed here: https://github.com/bsideup/grpc-bidi
Currently, both Java (in prod for ~6 months) and Go (sample code provided and it is solid, just haven't had a chance to put it in prod yet) implementations exist (see provided samples for Client & Server).
Note that they can connect to each other, with the exception of a Java client connecting to a Go server (grpc-java's unfortunate limitation of not being able to set a sub content type, and Go's server impl not supporting forcing a codec per service/method: https://github.com/grpc/grpc-java/issues/10959)


WIP: [Artwork view] Artwork commercial flow

Adds inquiry and BNMO flow to Artwork view
WIP: Still needs Eigen routes for /inquiry/:id and /order/:id
WIP: Needs tests for new content

Tickets

https://artsyproduct.atlassian.net/browse/PURCHASE-1338
https://artsyproduct.atlassian.net/browse/PURCHASE-1310
https://artsyproduct.atlassian.net/browse/PURCHASE-1311
https://artsyproduct.atlassian.net/browse/PURCHASE-1312

BNMO

Inquiry

Fails

:no_entry_sign:
No CHANGELOG added.

:no_entry_sign:
Missing Test Files:

src/lib/Scenes/Artwork/Components/__tests__/CommercialButtons-tests.tsx

If these files are supposed to not exist, please update your PR body to include "#skip_new_tests".

  Generated by :no_entry_sign: dangerJS

Fails

:no_entry_sign:

🃏 FAIL in CommercialInformation-tests.tsx

renders all information when the data is present

Expected substring: "Sold"Received string: "Contact For PriceAt I'm a GalleryShipping, tax, and service quoted by sellerWant to sell a work by this artist? Consign with Artsy."

Full message

Error: expect(received).toContain(expected) // indexOf
Expected substring: "Sold"
Received string:    "Contact For PriceAt I'm a GalleryShipping, tax, and service quoted by sellerWant to sell a work by this artist? Consign with Artsy."
at Object.toContain (/home/travis/build/artsy/emission/src/lib/Scenes/Artwork/Components/tests/CommercialInformation-tests.tsx:17:30)
at Object.asyncJestTest (/home/travis/build/artsy/emission/node_modules/jest-jasmine2/build/jasmineAsyncInstall.js:102:37)
at resolve (/home/travis/build/artsy/emission/node_modules/jest-jasmine2/build/queueRunner.js:43:12)
at new Promise ()
at mapper (/home/travis/build/artsy/emission/node_modules/jest-jasmine2/build/queueRunner.js:26:19)
at promise.then (/home/travis/build/artsy/emission/node_modules/jest-jasmine2/build/queueRunner.js:73:41)

hides seller info for works from closed auctions

Expected substring: "Sold"Received string: "Contact For PriceWant to sell a work by this artist? Consign with Artsy."

Full message

Error: expect(received).toContain(expected) // indexOf
Expected substring: "Sold"
Received string:    "Contact For PriceWant to sell a work by this artist? Consign with Artsy."
at Object.toContain (/home/travis/build/artsy/emission/src/lib/Scenes/Artwork/Components/tests/CommercialInformation-tests.tsx:35:30)
at Object.asyncJestTest (/home/travis/build/artsy/emission/node_modules/jest-jasmine2/build/jasmineAsyncInstall.js:102:37)
at resolve (/home/travis/build/artsy/emission/node_modules/jest-jasmine2/build/queueRunner.js:43:12)
at new Promise ()
at mapper (/home/travis/build/artsy/emission/node_modules/jest-jasmine2/build/queueRunner.js:26:19)
at promise.then (/home/travis/build/artsy/emission/node_modules/jest-jasmine2/build/queueRunner.js:73:41)

renders seller info correctly for commercial works

Snapshot name: `CommercialInformation renders seller info correctly for commercial works 1`- Snapshot+ Received- "Sold"+ "Contact For Price"

Full message

Error: expect(received).toMatchInlineSnapshot(snapshot)
Snapshot name: CommercialInformation renders seller info correctly for commercial works 1

Snapshot

Received

"Sold"

"Contact For Price"
at Object.toMatchInlineSnapshot (/home/travis/build/artsy/emission/src/lib/Scenes/Artwork/Components/tests/CommercialInformation-tests.tsx:90:7)
at Object.asyncJestTest (/home/travis/build/artsy/emission/node_modules/jest-jasmine2/build/jasmineAsyncInstall.js:102:37)
at resolve (/home/travis/build/artsy/emission/node_modules/jest-jasmine2/build/queueRunner.js:43:12)
at new Promise ()
at mapper (/home/travis/build/artsy/emission/node_modules/jest-jasmine2/build/queueRunner.js:26:19)
at promise.then (/home/travis/build/artsy/emission/node_modules/jest-jasmine2/build/queueRunner.js:73:41)

renders seller info correctly for non-commercial works

Snapshot name: `CommercialInformation renders seller info correctly for non-commercial works 1`- Snapshot+ Received- "Sold"+ "Contact For Price"

Full message

Error: expect(received).toMatchInlineSnapshot(snapshot)
Snapshot name: CommercialInformation renders seller info correctly for non-commercial works 1

Snapshot

Received

"Sold"

"Contact For Price"
at Object.toMatchInlineSnapshot (/home/travis/build/artsy/emission/src/lib/Scenes/Artwork/Components/tests/CommercialInformation-tests.tsx:104:7)
at Object.asyncJestTest (/home/travis/build/artsy/emission/node_modules/jest-jasmine2/build/jasmineAsyncInstall.js:102:37)
at resolve (/home/travis/build/artsy/emission/node_modules/jest-jasmine2/build/queueRunner.js:43:12)
at new Promise ()
at mapper (/home/travis/build/artsy/emission/node_modules/jest-jasmine2/build/queueRunner.js:26:19)
at promise.then (/home/travis/build/artsy/emission/node_modules/jest-jasmine2/build/queueRunner.js:73:41)

renders artwork availability correctly

Snapshot name: `CommercialInformation renders artwork availability correctly 1`- Snapshot+ Received- "Sold"+ "Contact For Price"

Full message

Error: expect(received).toMatchInlineSnapshot(snapshot)
Snapshot name: CommercialInformation renders artwork availability correctly 1

Snapshot

Received

"Sold"

"Contact For Price"
at Object.toMatchInlineSnapshot (/home/travis/build/artsy/emission/src/lib/Scenes/Artwork/Components/tests/CommercialInformation-tests.tsx:119:7)
at Object.asyncJestTest (/home/travis/build/artsy/emission/node_modules/jest-jasmine2/build/jasmineAsyncInstall.js:102:37)
at resolve (/home/travis/build/artsy/emission/node_modules/jest-jasmine2/build/queueRunner.js:43:12)
at new Promise ()
at mapper (/home/travis/build/artsy/emission/node_modules/jest-jasmine2/build/queueRunner.js:26:19)
at promise.then (/home/travis/build/artsy/emission/node_modules/jest-jasmine2/build/queueRunner.js:73:41)

  Generated by :no_entry_sign: dangerJS

Warnings

:warning:
Missing Test Files:

src/lib/Scenes/Artwork/Components/__tests__/CommercialInformationSizeInfo-tests.tsx
src/lib/Scenes/Artwork/Components/__tests__/CommercialPartnerInformation-tests.tsx

If these files are supposed to not exist, please update your PR body to include "#skip_new_tests".

  Generated by :no_entry_sign: dangerJS


i was looking into the markdown, found a typo in the keyword "cross-site"

i was looking into the markdown, found a typo in the keyword "cross-site"
first word under the  introduction!  please have look
https://cdn.discordapp.com/attachments/377895389992321064/444124455145308160/crosssite.JPG

thank you

fixed


Added popup interface support

This pull requests adds support for popup menus in game. It includes:

Adding XRToolsPopup script/scene for managing popup interfaces.
Adding XRToolsFunctionPointer static find helper methods.
Adding class_name to XRToolsViewport2Din3D.
Modifying XRToolsViewport2Din3D to only enable when visible.
Adding layer 24 - Popup UI
Adding popup menu to pointer-demo scene

lgtm, like the idea of this but yeah i heard some concerns about its implementation, someone suggested doing a raycast i think
anyways this works good

Discussed with Malcolm, we are going to rethink this approach, especially with the new composition layers being a better fit for menus. Closing this and will raise something new in the future.


BigQuery insertAll always retries operation

Hi. First time reporter here.
I wanted to report a small but perhaps inconvenient bug in BigQuery, specifically regarding streaming inserts using an InsertAllRequest. It seems for some reason unknown to me that inserts are retried using the client's RetrySettings whenever row ids are supplied for deduplication.
When the current implementation always attempts a retry (which I discovered during these odd retry errors) due to the fact that Lists.transform() is strictly lazy.
Anyhow the fix is minor and I corrected it on my branch but I'm unsure how to write the necessary tests in order to validate my claim. My commit is here: https://github.com/neuromantik33/google-cloud-java/commit/984b5bb41cfb1ec0ef7cea27a0bc38d8adde5051, and after proper review I can submit a pull request.
Thanks in advance for any feedback.

Hi @neuromantik33,
Thanks for catching this issue! Would you mind turning your commit into a pull request? For testing, one option could be to add a unit test to BigQueryImplTest.java, set a spy on the BigQueryOptions options (see line 274), and assert that getRetrySettings() is called on that spy object when BigQueryImpl.insertAll is run or not run.


Deep learning crop type notebook tested in colab

Going to pull this in. There's probably minor changes to come with data loading paths and polishing but that can be a new PR.


Fix a typo in docs/README.md

Also congrats on the first contribution! I'm checking workflows now to ensure they get run

Thank you for the great project!


can not select file

There is no way for selecting the uploaded file. It is visible in the filebrowser but is not selectable. Clicking the name opens the file itself.

Might have something in common with https://github.com/wardi/django-filebrowser-no-grappelli/issues/56
A possible fix is to copy the uploaded file URL, then clip the MEDIA_ROOT part and paste into the slide filename input field.

Oops, i saw this bug but forgot to fix it, i'm working on it

Well, i tested it again and remember how i resolved it, with disabling AdBlock Plus for my server hostname :)
Don't know really why ABP is catching the version selector stuff, eventually a false positive.
All in all, this is not related to emencia-django-slideshows, you probably have enabled ABP / some Javascript blocker.


Slow Down and 100% CPU Usage

One thing I've noticed is that Reactotron gets slow over time.  Eventually it pins the CPU at 100%.
I think this is websockets related (or how I handle them).
It seems to show up sooner the more I reload & hot reload my React Native apps.
:(

It's running significantly faster on the Electron version.  Still on my radar for the CLI.

Closing this out... much much better now in the next branch.

Is it possible that there are to many entries in the event timeline after a while?

Possibly.
I do have (what I believe to be) a 100 message cap though.
https://github.com/reactotron/reactotron/blob/master/packages/reactotron-core-server/src/commands.js#L49


feat: add create service data

Create service data key model
Create Service Data Key Creation API

@FemiNoviaLina some lint errors there, can you please run go mod tidy first?


docs(website): Add Logrocker documentation to showcase

Motivation
The motivation is to create something helpful, especially for ReactJS community. Important is to share knowledge and also create some open source projects, invite collaborators and let developers know what libraries they can use and support the others.
Have you read the Contributing Guidelines on pull requests?
Yes
Test Plan
none
Related PRs
none

Hi, thanks for taking interest in this.

Your source does not point to your site's source. Is you site itself actually open-source?
The documentation size is too small to be sound. Would it be expanded in the future?
Your site is not using versioning. Please re-choose tags.

please add a bit more doc, it feels empty atm
FYI I'm not very convinced by your project.
The goal of NOT having such abstraction is to avoid the runtime cost of log formatting and also being able to strip useless logs in the production build. Your abstraction removes these possibilities.

Off-topic: React built another approach to dev/prod distinction exactly because reading from process.env is slower than people think. (Dan Abramov talked about that somewhere)

Sorry @patrikmasiar while we rarely shut down showcase PRs, we do try to keep quality in our showcase. I believe your single-page documentation, while being "production-ready" and not "work-in-progress", doesn't really stand out and serve as inspiration for other users.
I will close it for now, but we are happy to re-open and reconsider this if the site becomes more polished and meaningful over time. (e.g. your library deserves more API documentation than a single usage example) Thanks for choosing Docusaurus still.


Out of Control Memory Management?

Describe the bug
While attempting to digest a ~450GB Photos database, it gets to around 20-35% before it hangs and subsequently fails. At this time, the Python interpreter is sitting around 95GB of memory
To Reproduce
Steps to reproduce the behavior:

Command: osxphotos --library /Volumes/METAETHER/Master_Merged.photoslibrary export /Volumes/METAETHER/Exported_Master_Merged --person-keyword --album-keyword --keyword-template --update --exiftool --directory "{created.year}/{created.month}/{folder_album,Unnamed Event}"
No error output, please refer to attached screenshot.

Expected behavior
I expect for the script to run 'till completion. This appears to be either memory leak, or it is literally loading every single image into memory and not clearing the memory of the already processed images. Given the size of the memory/virtual memory, I believe the aforementioned to be the case. If I can't get this working, I'll have to resort to using elodie (which I would prefer not to use, but as a backup plan is running effortlessly (900GB of photos; on a docker container on my NAS with only 2GB of memory) for a failed PowerPhotos merge (didn't generate a database file, hence Apple Photos cannot repair it.) I have tried running it several times. First time it made it to around 25% before it failed, then second time it made it to 34% before it failed. I had to clear up ~100GB on my internal drive to make room for more virtual memory. This doesn't seem to be the expected behavior. I know there is a RAMdisk option, but I'm not using that feature.
Screenshots

Desktop (please complete the following information):

OS: Ventura 13.3
osxphotos version (osxphotos --version) ? That command doesn't seem to work. It generates a trace callback.

Additional context
Since PowerPhotos was unable to handle the full merge, I was excited to come across osxphotos, ideally to move away from Photos permanently, or temporarily move away so I can easily separate them into different libraries by year (we're talking ~18 years worth of photos). Barring I can't get osxphotos working, I would only be able to fall back on using elodie to process the 'failed merge' from PowerPhotos; although off-topic (for clarification sake): it completed successfully, but Photos app wasn't able tor repair the library (as far as size is concerned, appeared to have worked, hence some possible utility with running elodie, but would prefer using osxphotos -- if possible since it seems more capable).
Thanks for any feedback/help and/or taking this bug report.

osxphotos version (osxphotos --version) ? That command doesn't seem to work. It generates a trace callback.

Something is definitely wrong with your osxphotos installation then as both osxphotos version and osxphotos --version should work (they generate slightly different output).  How did you install osxphotos?  If via pipx (which is what the install instructions recommend) try this:

pipx uninstall osxphotos
pipx install osxphotos

Then try running the version command again.

I was able to replicate this on a large library -- osxphotos definitely appears to have a memory leak in the latest version.  I'll investigate. In the mean time, try downgrading:
pipx uninstall osxphotos
pipx install osxphotos==0.58.2

@all-contributors please add @wernerzj  for bug

I believe this should be fixed in v0.59.3

I've done a full export of my 36k image, 125GB library and while memory does slowly grow during the export, it completed in ~3 hours using 2GB of memory.  I'll continue to investigate memory growth but I believe the run-away memory issue is resolved.

Thanks so much for all of your efforts with this, I truly appreciate it!


Update python tests suite

Some functions in the SWIG Python interface need to be updated for use in Python. I've marked these tests to be skipped for now.

Do you have a list of functions which are not working? and have you tried anything in the *.i files to get them working?

helicsPublicationGetKey and helicsSubscriptionGetKey were also having an issue. It needs to return a string instead of taking a str as an argument.


Add parameterization for signals, in the load test

Add parameter signalToOpRatio, to specify the configuration for signals
Log and enable Submitted and Received Signals metrics
Add signalToOpRatio parameter to the ILoadTestConfig interface
Add and Implement async function getRuntime() through the ILoadTest interface, which exposes the runtime needed to access signal events
If undefined, signalsPerOp is set to a default value of 0

Opened a merge request into the wrong branch, sorry.


Improve ergonomics of Query

There's a few things that could make Query more usable for programmatic manipulations.
My particular desired use-cases are:

Be able to nicely display a query and its terms. Having downcast::Any implemented on Query would help with this. fmt::Debug is almost enough, except that Terms display raw byte slices and Field numbers, neither of which is particularly readable.
Be able to debug why queries match / do not match a given document. So for a complicated query I want to see which of its clauses matched a given document, and evaluate this recursively.

The changes to Query necessary to do this are:

Add downcast::Any for checking if we have a particular type. Necessary for recursive pattern matching without turning everything into enums.
Add a fn box_clone(&self) -> Box<Query> for turning a &Query into a Box<Query>, in order to deconstruct queries and put them back together into other queries.
Edit the common default built-in Query impls to support introspection. This means being able to see the clauses of a BooleanQuery, the Term of a TermQuery, and the Field + Terms of a PhraseQuery.

Will first add the capabilities to Query necessary to make this happen, and then after API discussion add the other two things. I imagine println! is not the right target :smile:.

This is closed isn't it? @jason-wolfe


WindowCovering doesn't work

When I use WindowCovering I get an error from the iOS Home app "Home couldn't connect to this accessory.". I don't see any error in the debug console.

Duplicate of #56

@beowulfe let's close this as a duplicate of #56; I can definitely reproduce this bug and it is caused by the inclusion of HoldPositionCharacteristic

No update since 10 month. Why is the issue not closed?


Move Configurate Output Out of /madminer directory

Problem
The configurate step writes the output into the same directory where the Madminer code lives. This is bad practice and causes security faults.
There may be outputs from other steps that do the same thing
Approach
Create a scratch volume where these files are written

After a brief discussion over Zoom, we clarified why this goal is important (basically to allow files persistency across Yadage / REANA steps), and we agreed I will proposed a PR to address it.

PR addressing this issue: https://github.com/scailfin/workflow-madminer/pull/27


Support AbortController for aborting fetch operations

Is your feature request related to a problem? Please describe.
AbortController is part of the DOM Living standard. It allows to abort an in-progress fetch  operation in a standard way via the signal option.  Specifically it allows to apply a timeout to a fetch operation in a standard-compliant portable way.
Describe the solution you'd like
Export AbortController, AbortError and optionally a small helper timeoutSignal for specifying a timeout signal to support the following code snippets:
const { fetch, AbortController, AbortError } = require('@adobe/helix-fetch`);

const controller = new AbortController();
setTimeout(() => controller.abort(), 1000);

try {
  await fetch('https://httpbin.org/delay/2', { signal: controller.signal });
} catch (err) {
  if (err instanceof AbortError) {
    console.log('fetch timed out');
  }
}

or shorter:
const { fetch, timeoutSignal, AbortError } = require('@adobe/helix-fetch`);

try {
  await fetch('https://httpbin.org/delay/2', { signal: timeoutSignal(1000) });
} catch (err) {
  if (err instanceof AbortError) {
    console.log('fetch timed out');
  }
}

The existing non-standard timeout option and TimeoutError should be deprecated.
Additional context
Abortable fetch
fetch timeout discussion

implemented in bb06b3e


Add 'ignore-clusterless' annotations to resources that are irrelevant to clusterless actuation.

Add 'ignore-clusterless' annotations to resources that are irrelevant to clusterless actuation mode (mainly workload identity bindings are irrelevant when a GKE cluster is not involved)

/ok-to-test

/test all


Versioninfo

Tidies up VersionInfo and uses Scala 2.10 string interpolation.

:+1:  Maybe we could squish the commits to one?

I'll squish the second commit into another PR where Im actually changing the file.


City View - right hand resource list display order

Salt peter should come before oil in the list .

Fixed!


Collapsible Collection

A collection field has the "collapsible" option that collapses the entire collection.
There should be an option to collapse each childe of a collection individually.

Fixed https://github.com/ems-project/EMSCoreBundle/pull/45


vlan support

first pass merged in

@amaan-fmad  need to print the vlan in the json output
https://github.com/fmadio/pcap_microburst/blob/master/main.c#L264

Pushed to master

awsome thanks


NTR: proteaphagy

def: The autophagic clearance of 26S proteasomes; homeostatic mechanism within the ubiquitin system that modulates proteolytic capacity and eliminates damaged particles.
PMID: 27477278 | PMID: 26670610
parentage: is_a child of GO:0006914 autophagy

note to self: CUE5, HSP42

Hi Stacia,
It seems like this is a type of macroautophagy in which the target is engulfed by an autophagosome. Do you agree?
-D

Hi Stacia,
I went ahead and made it as follows. I think it makes sense. I made the logical def use the most generic proteasome complex as input. If you see any issues with this, let me know and I will edit the term.

[Term]
+id: GO:0061816
+name: proteaphagy
+namespace: biological_process
+def: "The macroautophagy process that clears 26S proteasomes; homeostatic mechanism within the ubiquitin system that modulates proteolytic capacity and eliminates damaged particles." [GOC:dph, GOC:se, PMID:26670610, PMID:27477278]
+is_a: GO:0016236 ! macroautophagy
+intersection_of: GO:0016236 ! macroautophagy
+intersection_of: has_input GO:0000502 ! proteasome complex
+created_by: dph
+creation_date: 2016-12-01T20:21:28Z

-D

thanks David.  sorry i just needed more time to get to this.  happy with how you did it.

No problem Stacia. It got automatically classified as a type of aggrephagy, 'selective degradation of protein aggregates by macroautophagy'. The logical definition of aggrephagy is a type of autophagy that has an input of a protein complex. Since the proteasome is a protein complex, it gets slotted here. If you think this is incorrect, I can remove the logical definition of aggrephagy, but it seems ok to me based on the refs that you provided.
-D


Added implementation for assigning student to classroom

Resolves https://github.com/CS2113-AY1819S1-T16-1/main/issues/105

Changed prefix of classname to be 'cn/'
Classroom now keeps track of list of students assigned to the classroom
Duplication check for assignment of students to the same classroom

Coverage decreased (-1.0%) to 66.816% when pulling 64db4f41abe1d96e1caaf50e43049ab8ee275f3a on zhicaizack:ClassStudent into 0755ceff670cf05363f6a4e8d251a7600deb3c61 on CS2113-AY1819S1-T16-1:master.


TkAdmin.java:54-57: Admin page should be accessible...

Puzzle 213-408dfdc6 in src/main/java/com/nerodesk/takes/TkAdmin.java:54-57 has to be resolved: Admin page should be accessible for predefined and hardcoded urns. Now it is accessible for any registered user. Please, specify several urns and check if logged user is authorized to see admin page. If not - redirect to root (?)... The puzzle was created by longtimeago on 22-Apr-2015.
If you have any technical questions, don't ask me, submit new tickets instead

@yegor256 I set milestone here to 0.7, let me know if it is wrong


[Outlook] (OWA) Update distinction between new and classic

Docs Build status updates of commit 6cd0c2c:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/reference/objectmodel/preview-requirement-set/outlook-requirement-set-preview.md
:white_check_mark:Succeeded

docs/reference/requirement-sets/outlook-api-requirement-sets.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


GPUImage not working with Xcode 9 GM (iOS 11)?

GPUImage stopped working in my app at Xcode 9 beta 6. Now that GM was released I tested it again and it's still not working.
Tried the FilterShowcase demo at it also doesn't work (on the simulator, didn't test on device).
It doesn't crash or say anything, it just doesn't filter anything. Does anybody please has any idea what's going on?

None of the examples that rely on the camera being present will work in the Simulator. Frankly, I only test on device and don't even bother with the Simulator. I have heard that the Simulator is a little off in the latest Xcode 9 betas.

Thank you, Brad. Yes, I tested on device and it's alright, it's just the simulator that isn't working for me.


Make it clearer that rustup will also install cargo

Make it clearer that cargo is automatically installed when installing Rust using rustup.

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @alexcrichton (or someone else) soon.
If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.
Please see the contribution instructions for more information.

Seems like a reasonable addition to me!

@bors: r+

:pushpin: Commit 0fb6f81469f885e67dec76ee2f387809cce9460c has been approved by alexcrichton

:hourglass: Testing commit 0fb6f81469f885e67dec76ee2f387809cce9460c with merge c2829b3e8b69d7f2159e73fe6d41a09ecb1acdd3...

:sunny: Test successful - status-appveyor, status-travis
Approved by: alexcrichton
Pushing c2829b3e8b69d7f2159e73fe6d41a09ecb1acdd3 to master...


docs: add Windows image release notes for 2024-04B with updated cache

What type of PR is this?
/kind documentation

What this PR does / why we need it:
add Windows image release notes for 2024-04B with updated cache
Which issue(s) this PR fixes:

Fixes #
Requirements:

[x] uses conventional commit messages

[x] includes documentation
[ ] adds unit tests
[ ] tested upgrade from previous version

Special notes for your reviewer:
Release note:
none

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 79.051%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
2415

Relevant Lines:
3055

💛  - Coveralls


Simplify Required assert

This commit updates the Required assert to check only if the property is defined.
Refs https://github.com/guillaumepotier/validator.js/issues/29.

pinging @guillaumepotier

Yes sorry, this is going too fast for me these days --'
You're right in #29, these unnecessary checks had been made for Parsley.js initially when they where tightly coupled.
I think this change is a good thing, do not need to mix too many concerns in the Required assert. Ok for that, but maybe we could write it in the changelog and make a more important version bump since it is BC break.
2.1.0 should be ok IMO. Need to update it in package.json, Readme, build dist version, etc.. I'd like to just have to tag and npm publish (or only npm publish if you have the tagging rights)
Best


Feature request: CSS Preprocessor Support

It might be nice for topdoc to output the preprocessed code snippets (e.g., LESS, Sass) instead of just CSS.

I believe that this should not be done.
You can always compile your less/sass/stylus and keep comments in the CSS and then run topdoc before minification.

@zeropaper I think that should be the plan moving forward. The current 1.0.0-beta is using postcss to parse out the comments. It could be possible to use a different parser with postcss like the postcss-scss parser. If we did go with a solution like that this would be the way to go.


error: no such subcommand: wasm

Hello I'm running on Windows 10. I have been successfully installed cargo wasm

But when I execute:
cargo wasm => error: no such subcommand: wasm`
Any ideas how to fix it?

Hi @AndonMitev
This is the repository for the rust cross tool for cross compiling executables. Did you mean to post this issue here?
For cross, it would not be able to run cargo wasm as it only enables cross functionality on certain subcommands.
as for your issue, I think this is due to the executable not being installed properly.
error: no such subcommand comes from cargo.
Custom cargo subcommands require a executable named cargo-* present in path. You do not have an executable named cargo-wasm in this directory you've shown. I presume its not anywhere else in your $env:PATH either.
if you meant to install cargo-wasm, try installing cargo-wasm again with cargo install --force cargo-wasm
I'm closing this issue as answered.
Either you've hit a limitation with cross, i.e it doesn't work for all subcommands, or you've hit an issue with your installation of the executable.


Incorrect syntax highlight of a paren inside a quoted string

I couldn't reproduce this

Closing this as cannot reproduce. I think it is a theming issue (works fine in default theme)


Multipart does not accept json objects along with photos.

Multipart does not accept json objects along with photos.

Can you please provide your code here?

https://github.com/lexiforest/curl_cffi/issues/404

I can only fix the bug when I can reproduce it. Code, please.


RFC - Bash App definition change

We currently use a magic variable "cmd_line" in bash apps to specify the command-line string to be executed. For eg :
@App('python', dfk)
def hello ( ):
   cmd_line = "echo 'Hello world' "

This is bad for a few reasons:

This is surprising behavior in python
There's ambiguity in behavior if the user were to reassign to cmd_line
Any values returned are certainly lost (again unusual behavior for python)
Implementation wise, we trace the value assigned to the magic variable and this is computationally expensive.

I recommend that we switch to having bash apps return a string, and that string be treated as the command line executable and if we decide that this change is acceptable, we should merge for 0.3.0.
For eg:
@App('python', dfk)
def hello ( ):
   return "echo 'Hello World' "

I can see how this solves 1 and 2.  What about 3?

Also, how does this compare with subprocess's way of doing things, which is what I guess most people would use in a roughly similar situation in "normal" python?

3: The return value is not lost, it is used by Parsl.  Multiple return statements are processed with expected Python behavior

@danielskatz The normal way of doing it with subprocess would look like this :
import subprocess
proc = subprocess.Popen(['echo', 'Hello World'])

# You handle waiting or asynchronous behavior explicitly
proc.wait()

PyDFlow does something like this :
from PyDFlow.app import *
from PyDFlow.types import *

@app((localfile), (Multiple(localfile)))
def sort(*infiles):
    return App("sort", "-o", outfiles[0], *infiles)

ok, looks fine to me

I dont fully understand all the issues here yet. We're just talking about
changing how the decorator gets the command line string from the app
definition "pseudo-function", right?
The return approach sound fine to me for that.  I assume that argument
insertion will be done in a similar manner as with "cmd_line"?
I dont understand the returns issue.  I would have thought those are
handled by the decorator, and are independent of whether youre using the
"return" or "cmd_line" approach.
Pardon me if Ive mis-understood the whole issue here, though.
On Wed, Nov 1, 2017 at 7:01 PM, Daniel S. Katz<EMAIL_ADDRESS>wrote:

ok, looks fine to me
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/Parsl/parsl/issues/37#issuecomment-341279166, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABqb2FD5vF2ULznRGuJsALC5zW3bxhK2ks5syQZxgaJpZM4QOq4U
.

--
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division, Argonne National Laboratory
m:<PHONE_NUMBER>

Yes, the proposed change is about how the decorator gets the command line string from the decorated function. Moving to capturing from returns instead of intercepting mid-run makes the implementation simpler, and leaves little room for surprises and confusion for the user (I hope).
If we put " return cmd_line " at the end of every bash app we have now, we get the target behavior, but without it existing code will break.
Here's what I meant by the returns issues :
@App('python', dfk)
def hello ( ):
   cmd_line = "echo 'Hello world' "
   return 10

# The return value is lost since the app call returns an app future
x = hello ()

Ah, ok , I get that - makes sense.  Thanks for clarifying.  I can see how
that makes the "return" approach much better.
On Wed, Nov 1, 2017 at 11:41 PM, Yadu Nand B<EMAIL_ADDRESS>wrote:

Yes, the proposed change is about how the decorator gets the command line
string from the decorated function. Moving to capturing from returns
instead of intercepting mid-run makes the implementation simpler, and
leaves little room for surprises and confusion for the user (I hope).
If we put " return cmd_line " at the end of every bash app we have now, we
get the target behavior, but without it existing code will break.
Here's what I meant by the returns issues :
@App('python', dfk)def hello ( ):
cmd_line = "echo 'Hello world' "
return 10
The return value is lost since the app call returns an app future
x = hello ()
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/Parsl/parsl/issues/37#issuecomment-341316103, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABqb2AFwBGv9u_38v7ffZ-LSnDxhcKEoks5syUfygaJpZM4QOq4U
.

--
Michael Wilde
Computation Institute, University of Chicago
Mathematics and Computer Science Division, Argonne National Laboratory
m:<PHONE_NUMBER>

This behavior is in master now.


[fix] Agent cookies are not correctly attached to right domains/paths on redirect

Describe the bug
Node.js version: 20.11.0
OS version: MacOS Sonoma 14.5
Description:
We are using superagent agent to perform an OpenID authorization code flow before interface tests, crossing domain boundaries on multiple redirects. After upgrading from superagent 8.0.8 to 9.0.2, this fails because of missing cookies in the latter redirects.
After adding some console.logs to the agent lib, it looks like in 8.0.8 the _saveCookies() method is not storing the right url.hostname/path but null/null, which means that all cookies are attached later irrespective of the domain, while in 9.0.2 it's passing wrong domains and paths into this.jar.setCookies(), so that they are not attached later.
Actual behavior
Example:
agent starts in domain www.service.org which sets a session Cookie, then redirects to oidc.service.org for authentication, which itself sets cookies.
Afterwards the same agent is used to post login credentials on oidc.service.org, which causes a redirect back to www.service.org. Now the session cookie is not passed to www.service.org, as it has been saved in the cookie jar with the redirection host/path for oidc.service.org.
Expected behavior
Cookies set in the response of any redirecting request that have no domain part by themselves should be saved with the domain of that exact request, not with that of the followup request.
Checklist

[X] I have searched through GitHub issues for similar issues.
[X] I have completely read through the README and documentation.
[X] I have tested my code with the latest version of Node.js and this package and confirmed it is still not working.

btw we cannot workaround setting a domain as our service is multi-tenant and the express-session middleware does not support this. Created https://github.com/expressjs/session/issues/988 in addition.

I created a failing test here:
https://github.com/ladjs/superagent/commit/22f6632d8df6ef08de23ceb19d4a21a5099a1d64

I tried a fix here: https://github.com/ladjs/superagent/compare/master...robaca:superagent:master
For some reason the multipart tests are failing for me locally (maybe because of Node version), otherwise looks promising. I didn't check the PR rules beforehands, so it's most probably not yet ready for a PR.

btw. I get the same error in supertest. If I use it with my patched version and change one line in supertest/lib/agent.js to req.on('pre-redirect', this._saveCookies.bind(this)), it's working for me.


Rc datadump

I made a json file of content.json with filler data so the screen isn't blank when working on your own file. To use just  run 'python manage.py loaddata content.json'

@jinpark can you check this out?

lgtm


Convert to noarch Python

Makes the package a noarch: python package.

Checklist

[ ] Used a fork of the feedstock to propose changes
[ ] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[ ] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[ ] Ensured the license file is being packaged.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

Please let me know if you have any thoughts on this. :)

I think its a great idea! Is there something to review or should we just merge?

You tell me ;)

Merging then. Thanks for doing the work!

Thanks @danielfrg!


Does not now support interactive use in (Windows) Powershell?

I used to use an old version of this interactively in Powershell.  It is extremely useful for interactive exploration of remote devices.
However, using 4.0.30:
> [InTheHand.Net.Sockets.BluetoothClient]::new()
Exception calling ".ctor" with "0" argument(s): "This functionality is not implemented in the portable version of this
assembly. You should reference the NuGet package from your main application project in order to reference the
platform-specific implementation."
At line:1 char:1

Which seems to have come from a base implementation at
https://github.com/inthehand/32feet/blob/73a3fc215f23e633d7c0f8b9d697233f18cb2b22/InTheHand.Net.Bluetooth/Platforms/Standard/BluetoothClient.standard.cs#L18
It would be wonderful if I could use the assembly in Powershell.

The big change from v3 to v4 was instead of a single .NET Framework dll the NuGet package contains runtime versions for multiple platforms. The .NET 4.x runtime dll should support Powershell in the same way. How are you extracting the dll from the NuGet package to use in Powershell?

Thanks @peterfoot .  Your question actually just prompted my to try the net461 dll in the nupkg, and that works.  The problem I mentioned was when I had imported the netstandard2.0 dll.
In each case I just extract the dll(s) (one in the case of this package) to some location on disk, the import-module it (your package wonderfully has no deps to worry about).  In this case it works for net461 but not netstandard2.0.  I have successfully used this approach before with other packages, and usually go for the netstandard versions just to assist any forward migration I might be doing.

The netstandard2.0 dll contains no implementation and just allows you to program against the API with the correct platform version being dropped in at build time. I'll see if there is an easier way of facilitating this in the future.


Add Python client

Add a preliminary Python client implementation.
Description
I've based my implementation on #68 and our OpenAPI definition.
How Has This Been Tested?
There are simple tests included. I still have to add instructions to run, a CI job to run them
automatically, and docstrings.
To test, first make sure you have at least Python 3.10 installed, if you don't:

Setup pyenv
pyenv install 3.10.11

Now, you can test by doing:

cd clients/python
pyenv env set $(which python3)
pyenv run pytest -v

TODO:

[ ] Add pre-commit hooks (?)
[ ] Document nox, etc on the README
[ ] Build Docs
[ ] Release the first version
[ ] Add CI jobs for building docs and running tests

Merge criteria:

[x] The commits and have meaningful messages
[ ] The author will squash them after approval or will ask to merge with squash.
[x] Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
[x] The developer has manually tested the changes and verified that the changes work

I don't know if it makes sense to squash this, we can discuss that.

@isinyaaa I think one of the suggestions from @rimolive from DSP team was that we could re-use mlmd base classes and extend our types from them.
Would that make it possible to inherit common properties and perhaps make it easier to implement the API somehow?
I am not sure about all the implementation details. @rimolive can you provide more information about what you were thinking?

v2 updates:

Moved the code into a src folder as per guidelines.
Removed map submodule: let's keep things simple with a single interface and helper methods.
Moved model registry client into submodule
Created a new interface for prefixable types
Simplified artifact state mapping (removed bidict dependency)
Added simple docstrings
Created a noxfile to simplify running tasks
Better separation of core functionality from MLMD integration/dependency
Dealt with a few TODOs
Added pytest coverage plugin
Improved test coverage (now at 87%) and logic

lgtm
Just to clarify, is the primary purpose of the store wrapper type for parity with the Golang client's service layer API?

I'm focusing on parity on the base type definitions and on the registry client. The store wrapper should simply provide a useful interface to gRPC focusing on our use case (upserting and querying single protos).

Changes in v3:

Added sphinx to build docs
Added basic usage on README
Added attrs
Made exceptions a common submodule
Brought back client as single file module as we don't need much more right now
Fixed license string
Added CI to build docs and test
Added release commit
Fixed (most) get_* calls to automatically populate references to other proto objects. (Artifacts are still pending)
Fixed ListOptions
Fixed ModelRegistry constructor to take in SSL connection args
Renamed *Error -> *Exception following python conventions.
Some more improvements to tests

Changes in v4:

Applied suggestions
Used structural pattern matching where possible

Changes in v5:

Dropped structural pattern matching to keep compatibility with 3.9.


Replace Shift-click shortcut for crafting

Maybe by right-clicking? Or by holding the left-click (the longer you click, the more you craft)

Un-abandoned, it would be cool to at least allow to customize the Shift+clicking mappings.


[TOPI] add conv2d_transpose_nchw

conv2d_transpose is almost the same as conv2d, except that conv2d_transpose has a dilation stage, and the filter is rotated by 180 degrees. It's reusing conv2d's cuda schedule.

According to the nnvm doc http://nnvm.tvmlang.org/top.html#nnvm.symbol.conv2d_transpose, there seems a padding stage after the convolution. This is not supported for now, becasue I am not clear about the purpose of this padding operation and tensorflow doesn't have it in tf.nn.conv2d_transpose(). Also, bias and groups are not supported for now.

bias is not necessary as it can simply be composed with the current conv2d

I confirmed test_topi_conv2d_transpose_nchw.py passed with ROCm backend.


Broken in Atom 1.0.9

sync-settings has broken after my Atom was autoupdated from 1.0.7 to 1.0.9. It's not sync'ing my setting changes to Gist anymore.

Can you provide some clear steps how to reproduce.
Do you see any errors in the console?
Are you running the latest version?

First things first... does the package automatically sync any change to the settings back to the Gist file?

No it does not, you have to manually back the settings up via sync-settings:backup command.

Ooookay, oops.


[Machine Learning] An Overview of Jina AI

Proposed title of article
Your title should be descriptive of the article/tutorial. Be Specific.
An Overview of Jina AI
Proposed article introduction
We're looking for the first 2-3 paragraphs of your article that appropriately summarize what your article will be about.
We will use this writing snippet to help us assess the overall quality before approval so ensure it has been proofread, spellchecked and grammarchecked.
Jina AI is a cloud-native neural search framework for building state-of-the-art and scalable deep learning search applications in minutes for any kind of unstructured data (videos, images, source code, long text etc.). In this tutorial, I will walk through our readers on how to use this framework to get their ML tasks done.
Key takeaways
What are the 3-5 most important things the reader should understand or be able to do after reading this article?
Use this area to get your ideas down on the bulk of your article or tutorial.
By the end of this tutorial, a reader should be able to understand:

What is Jina AI?
Components of Jina AI.
How to install Jina AI.
Implement an example to demonstrate how one can use the framework.

Article quality
How is your article unique? Tell us what makes your approach different from similar articles that have published on the same topic?
Is yours more in-depth? Does it cover additional topics? Do you provide handy tips or anecdotal advice?
Jina AI is a fairly new framework released recently. Many many tutorials are available on it's use. This tutorial will give our readers a high-level overview of the framework and implement an example on how one can use the framework.
References
Please list links to any published content/research that you intend to use to support/guide this article.

Jina AI

Sounds like a helpful topic - let's please be sure it adds value beyond what is in any official docs and/or what is covered in other blog sites. (the articles should go beyond a basic explanation - and it is always best to reference any EngEd article and build upon it). @wilkistermumbi
Please be attentive to grammar/readability and make sure that you put your article through a thorough editing review prior to submitting it for final approval. (There are some great free tools that we reference in EngEd resources.)
ANY ARTICLE SUBMITTED WITH GLARING ERRORS WILL BE IMMEDIATELY CLOSED.
Please be sure to double-check that it does not overlap with any existing EngEd articles, articles on other blog sites, or any incoming EngEd topic suggestions (if you haven't already) to avoid any potential article closure, please reference any relevant EngEd articles in yours. - Approved


comfig.cfg update

It is unnecessary to have the network_optimizations module at this point.
cl_interp_all and cl_SetupAllBones are defaulted to 0.

Yes, but the user should be able to select the "off" option in order to get better hit registration possibly, at the cost of performance.

Yes, but the user should be able to select the "off" option in order to get better hit registration possibly, at the cost of performance.

So what do you think of it being in the "off" option by default and only "on" by default on the very low preset?

It's significantly lower performance for a very slight increase in accuracy. I don't think it should be the default.

It's significantly lower performance for a very slight increase in accuracy. I don't think it should be the default.

If it is significantly lower performance, then I do not agree to have the option to choose "off" or "on". They always was defaulted to 0 on both values, and in most Source games this is also the default. I think we shouldn't mess with this.

It used to be part of the experimental addon, but now is being moved into the base config.

It used to be part of the experimental addon, but now is being moved into the base config.

I understand, but I suggest to only document this on the main comfig.cfg file, with the appropriate commands.
Enforcing they to its defaults it is useless at this point.
As you writed in the comfig.cfg: "Not intended to be documented at the moment.".

Sometimes, modules set things to their default values, but only because another module level sets them to a different value. This is one of those cases. It's not documented at the moment because it needs further testing on my end, until the release.

Sometimes, modules set things to their default values, but only because another module level sets them to a different value. This is one of those cases. It's not documented at the moment because it needs further testing on my end, until the release.

Alright. Document it when you find the time is appropriated.
And about this pull request, will you merge it?

I can take some of the changes. Will merge it soon.


[MISC] Fixes for PITR backup test instabilities

This PR addresses some of the sources of instability in CI, mainly those introduced with the new PITR feature.
Unit testing will be implemented once you are OK with the changes.
See review comments for specific context.

Great findings and improvements! Actually, this PR will fix #622


Client: take project as URL parameter, and ask for it if not supplied

Currently needs <mqtt-project .... >
It should get this from

[x]  the URL as a parameter e.g.    localhost:8080?project=Lotus Ponds
[x] and build mqtt-project node appropriately
[x] If project is not supplied it should put up a UI element to ask for it
[x] At the same time as doing this, ask for the MQTT server, but can default this to naturalinnovation.org

Closed - its asking the HTTP server for where the broker is, since we want them both talking to the same one!


Change encoder for org admin emails csv

BACKEND PULL REQUEST
Related Issue

Continuation of #7318 and #7462

Changes Proposed

While debugging on dev3 noticed CSVs not being sent for admin emails with a "+" symbol in the email
Realized I should be using Base64.getEncoder() instead of Base64.getMimeEncoder() because mime encoder does the following:

Each line of the output is no longer than 76 characters. Also, it ends with a carriage return followed by a linefeed (\r\n):
Source

SendGrid throws a 400 when the attachment has those characters
Additional Information

Using type: "patients" and state: "MN"

Before fix: Failed sendgrid POST request on original branch due to email with a "+" symbol

After fix: Successful sendgrid POST request

Testing

a modified branch elisa/dev3-fix-admin-email-csv-try2 has been deployed on dev3
use the following mutation to send an email with a CSV of org admin emails

mutation ($state: String!, $type: String!, $email: String!) {
  sendOrgAdminEmailCSV(
    type: $type,
    state: $state,
    email: $email,
  )
}

⚠ SENDGRID IS ENABLED ON DEV3 SO PLEASE USE YOUR EMAIL ADDRESS ⚠

I think I messed something up by closing and reopening this PR with some very different code changes 😓 Going to reopen this as a new PR

@bobbywells52 @fzhao99 opened a new PR here https://github.com/CDCgov/prime-simplereport/pull/7515 due to issues with the checks passing


Better Buildpack Caching

I've just added
https://github.com/pivotal-cf-experimental/diego-dev-notes/blob/master/proposals/better-buildpack-caching.md
It's a WIP proposal for improving Diego's staging performance.  I'm (very) open to suggestions.  We aren't likely to schedule this immediately but wanted to get some thoughts together.

closing this in favor of #30


Unable to run Text-Classification task because of error related to 'experiment_name'

I am loading this pre-trained model,
model = SetFitModel.from_pretrained("all-MiniLM-L6-v2")
and tuning with new data
train_ds = my_dataset_dict["train"]
val_ds = my_dataset_dict["val"]
eval_ds= my_dataset_dict["eval"]
args = TrainingArguments(
batch_size=64,
num_epochs=2, # The number of epochs to use for constrastive learning
num_iterations=70, # The number of text pairs to generate for contrastive learning
loss=CosineSimilarityLoss,
evaluation_strategy="epoch",
save_strategy="epoch",
load_best_model_at_end=True,
#exp_name = 'exp_1'
)
trainer = Trainer(
model=model,
args=args,
train_dataset=train_ds,
eval_dataset=val_ds,
metric="accuracy",
column_mapping={"text": "text", "label": "label"}  # Map dataset columns to text/label expected by trainer
)
when I run trainer.train() i am getting error "TypeError: init() missing 1 required positional argument: 'experiment_name'"
I am not sure where i need to define this, i tried setting this param in trainer class and TrainerArguments class, but seems like they do not take a param like this. Can you please help?

Hello!
Could you please share your transformers version?

Tom Aarsen