TimeoutError: timed out

Good afternoon.
I tried your solution. Apparently it is a little outdated, since it returns an error. I took the code from you - https://mcstatus.readthedocs.io/en/stable/examples/player_list_from_query_with_fallback_on_status/
Traceback (most recent call last):
  File "/APP01/main3.py", line 4, in <module>
    query = server.query()
  File "/APP01/env/lib/python3.10/site-packages/mcstatus/server.py", line 168, in query
    return self._retry_query(Address(ip, self.address.port))
  File "/APP01/env/lib/python3.10/site-packages/mcstatus/utils.py", line 67, in sync_wrapper
    raise last_exc  # type: ignore # (This won't actually be unbound)
  File "/APP01/env/lib/python3.10/site-packages/mcstatus/utils.py", line 63, in sync_wrapper
    return func(*args, **kwargs)
  File "/APP01/env/lib/python3.10/site-packages/mcstatus/server.py", line 174, in _retry_query
    querier.handshake()
  File "/APP01/env/lib/python3.10/site-packages/mcstatus/querier.py", line 55, in handshake
    packet = self._read_packet()
  File "/APP01/env/lib/python3.10/site-packages/mcstatus/querier.py", line 48, in _read_packet
    packet.receive(self.connection.read(self.connection.remaining()))
  File "/APP01/env/lib/python3.10/site-packages/mcstatus/protocol/connection.py", line 584, in read
    result.extend(self.socket.recvfrom(self.remaining())[0])
TimeoutError: timed out

This can happen when query isn't enabled on the server. Servers can only return query if they have it enabled in server.properties (the server owner must do this).
That said, considering the example in the docs is supposed to provide a "fallback" to query, that uses status, I agree that it should probably be updated.

I stopped the server, changed the enable-query value in the server management server.properties to true and started the server again, but the error did not go away.
Thank you very much for your example.
There are actually 2 typos in it:
def print_server_players - print_server_address
except TImeoutError: - TimeoutError

Ah, thanks for pointing those out, I just quickly wrote this directly from the github UI, so I didn't really check for typos. But I'm glad it worked for you.

I stopped the server, changed the enable-query value in the server management server.properties to true and started the server again, but the error did not go away.

That is a bit tricker to solve. There's 2 general reasons why this might be failing:
Firewall
Query uses UDP network traffic (unlike status which uses TCP). Many firewalls block UDP connections, so if you're running this in some server rather than on your personal machine, you might be running into that.
The other possibility is that the minecraft hosting your server is running on isn't allowing UDP, in which case it's their firewall that's blocking this traffic.  This is actually fairly common with a bunch of these hostings. If this is the case, the only real way to get around it would then be to switch to another hosting provider.
Wrong port
Another consideration to make is that query might be running on a different port than the server itself, JavaServer.lookup will find the TCP port for that server, this is the port you would use to connect to this server from a minecraft launcher, and through it you can get status info.
In server.properties, there's query-port setting (or something like that, don't quote me on the exact name of the setting), which controls the UDP port used for query. If this port isn't the same as the regular port, query won't be able to run with this example.
If this is the case, and you don't want to set the port to the same one as the TCP port, you can work around it, but it is annoying. To do so, you will need to manually specify the port in lookup, and you'll need to do lookup twice, once for query, and second time for status, as the ports are different.
query_server = JavaServer.lookup("<IP_ADDRESS>:25568")  # assuming your query port is 25568

...  # code to try out query

# if query failed (above the try: status = ... block)
status_server = JavaServer.lookup("<IP_ADDRESS>:25565")  # assuming your TCP port is 25565

... # rest of the code for getting status

Thank you very much for the clarification. I have a personal server and now I wanted to write a TelegramBot in which it will be possible to monitor the status of the server.
On all servers I have a standard port of 25565 and all traffic goes through a reverse proxy (https://github.com/lhridder/infrared). I took all the examples from your documentation page and only query = server.query() did not work and gave an error.
@logger.catch
def print_names():
    query = server.query()
    print(f"The server has the following players online: {', '.join(query.players.names)}")

Can you also try to get query without your reverse-proxy? It might not support UDP

Yes I can. After all, the servers are located on the local network, despite the fact that they are deployed on Proxmox.
But with your latest example on exception handling and enabling enable-query in the server settings, players began to appear, albeit with a slight delay.
Do you just want to see the result and compare it?

The players you're seeing are almost certainly coming from status, not from query, query is a better choice, because status is not that precise, it only gives you a small number of players (like say 10 players, even if the server has 100 players online). This is why it's called a "sample" (status.players.sample).
Status can also be modified by plugins to put advertisements in there, or to just make it empty regardless of who's online. Since you're the server owner, you can of course just avoid installing these plugins, so this isn't necessarily an issue for you, but it's yet another reason to generally prefer query if you want reliable information about players. (Most servers don't have query enabled though).
The reason for the delay is then because you're first making a query, and since mcstatus can't seem to be able to access query, it waits until a 3 second timeout, and then it tries again, up to 3 times. With status, you can reduce the retry count by passing server.status(tries=1), this doesn't currently work with query though, that's a bug, it will be fixed in a future version.
If you really don't want query, you can tremendously reduce the time it takes to get the data by always preferring status, getting status should work every time, for every supported minecraft server implementation:
server = JavaServer.lookup("...")
status = server.status()
print("Players online:", ", ".join([player.name for player in status.players.sample]))

But again, ideally you'd really want to use query here instead of status.

Maybe it's worth making additional methods for getting players, mods and plugins?
I could be wrong, but local queries and parameter changes began to be executed very, very quickly.
server = JavaServer.lookup("<IP_ADDRESS>:25565", timeout=1)
...
# 'query' has to be enabled in a server's server.properties file!
# It may give more information than a ping, such as a full player list or mod information.
server.status(tries=1)
query = server.query()
print(f"The server has the following players online: {', '.join(query.players.names)}")

Error: time exceeded is no longer displayed.

Maybe it's worth making additional methods for getting players, mods and plugins?

Mcstatus obtains the data you see here through various interactions with the minecraft protocol. We can't just invent a new way to obtain the players, and make it easier to get them reliably, because servers simply don't give us this information.
Java minecraft servers primarily expose their information through status. This is used by the minecraft launcher to show you some details about the server in the multiplayer screen. This is why we can get something like the MOTD, amount of slots and online players.
The player sample in status is actually something that you can see in the launcher when you hover over the slot count. This is why many servers put advertisements there, instead of actually giving you the list of players. It's also why this list of players is limited to only a few players, instead of giving you all of them. If a server has 1000 online players, the launcher wouldn't be able to create a window big enough to show you all of those anyway. So to save some bandwidth, only a small sample of the connected players is actually sent by the server.
 This response contains various things, which mcstatus parses and exposes to you in a nice way, one of the things that this response contains is indeed the player sample.
On the other hand, query is something that servers can optionally enable, and is used precisely for the purpose of being able to get some reliable information from the server, which can be used by various external tools. It's disabled by default though, since it can expose this information to everyone, and many server owners don't necessarily want to do that. (Query doesn't really contain anything that sensitive to the point where it would be a security risk to enable it, it just has info on the minecraft version and server implementation you're using, motd and the player list. It's more about just preventing some scripts from constantly checking who is online - privacy reasons).
This is why you should prefer query if you can, it's just that you usually can't. In this case, you own the server, so you should be able to enable query and use it. If it doesn't work, it's almost certainly due to a misconfigured network that doesn't let UDP traffic through. The query function should work reliably for pretty much all server implementations, unless you're using something really esoteric.

As for the other suggestions, of adding methods to get mods and plugins:

It's not really possible to obtain plugins from a server, they simply aren't exposed by neither query nor status  (and for a good reason, you wouldn't want people to know which anti-cheat you're using for example).
With mods, there is already a PR that adds support for getting data from forge servers (#578).

Getting plugins might be possible by going through a full login sequence, and actually having a user join the server for a while. this bot user could then execute a command to get plugins and we could read the chat response back. However this command is often only available for op users, so it generally wouldn't work anyway.
Also, for online servers (that is, non-warez servers, that only allows people with purchased minecraft accounts to join), it can be really annoying to configure everything up, as there's a lot of things you need to do for it. For that reason, mcstatus will almost certainly never implement logic to do something like this.
There is another library called mcproto under the py-mine github organization (disclaimer: I maintain this library). With it, you can actually make it past login, and do some more interesting things, but it's not trivial, even with the library, and mcproto is still WIP. You can read a bit about how to set up authentication with mcproto if you're really interested, but it is annoying (and there's not much mcproto can even do about this, as is explained): in the mcproto docs. Here's some example code that performs login with a server using mcproto: paste.gg

Wonderful. It’s great that there is a project that allows you to authorize and connect to the server. But I meant something a little different. After executing status or query, you can not go through the received data array yourself, but simply look at the counter and display a list if necessary.
print(f"The server has the following players online: {', '.join(query.players.names)}")

As an example:
server = JavaServer.lookup("address")
status = server.status()
if status.players.online > 0:
  query = server.query()
  players_list = query.players.names
  for player in players_list:
    print(f"{player.name} - {player.id}")

A dictionary would be handy to save in a player's database.

I asked about mods and plugins because I saw it here - https://api.mcstatus.io/v2/status/java/gtnh.jdi.su
Here you can see the documentation - https://mcstatus.io/docs
To be honest, at first I thought it was your project. But when I went to their github I realized that this is a separate project and it was written not in python, but in GO.

It's not really possible to obtain plugins from a server, they simply aren't exposed by neither query nor status (and for a good reason, you wouldn't want people to know which anti-cheat you're using for example).

bukkit.yml or spigot.yml (don't really remember) has an query-plugins option, that exposes the list of plugins into query. Also, you should just close your query port for outside world and host the telegram bot on the same machine with the server, so you don't need to care about privacy/security so much.

To be honest, at first I thought it was your project. But when I went to their github I realized that this is a separate project and it was written not in python, but in GO.

Yes, it is hard to devise a unique name for this theme. You can also check out my similar project, which is written using mcstatus - StatusMC (as I said, naming is hard here).

P.S. I didn't implement mods support in StatusMC because I am waiting for #578. Although, you can extract the mods from additional field.
But you really should just use mcstatus, because such websites add additional unnecessary network communication, and it would be useful only if you use non-Python language, can't use mcstatus directly or there is no library for Minecraft in your language. You can also use those if you really like some additional features, like modular system (you can get status and query response in one HTTP request), but this is not really your case, as you do not just ping some random servers, and I would rather close query for outside world.

After executing status or query, you can not go through the received data array yourself, but simply look at the counter and display a list if necessary.

So, what you're suggesting is a method that internally uses query and status, and you can just call it immediately? I don't think that's a good idea, since we'd be expected to provide functions like that for every little thing, i.e. get_minecraft_version, get_online_players, get_player_slots_amount, etc. Each one of these methods would call query or status internally, obtain all of the data at once, and then only give you access to one specific thing.
This would very likely lead people to call a bunch of these methods if they wanted various things, for example someone might want both version, slot amount and the player list. Calling all 3 methods would result in making the same request 3 times, each time we get all of the data, but we only give you one specific thing, and throw away the rest.
I think it's a good thing to allow people to see that by enabling query, you get all of these things at once, from a single request. How you use those things is then up to you, you can ignore the rest and only use the one thing you care about, but if you do care about a whole bunch of info on the server, you can get all of it at once.
This is both faster, and it impacts the server less. If someone made a program requesting 10 different things from a server, it would spam the server with a bunch of status request packets, all of which giving you the same data over and over. If you over do this, it might even yield to you getting banned by the server for suspected DoS (denial of service attack). You're simply making needless requests over and over again, and the server you're requesting this from will have to handle that.
I also don't think that it's particularly difficult to learn how mcstatus works, as we have a relatively simple to use public API for the library, and the response objects are all documented and type-hinted.

In no case did I suggest doing query or status every time. I just suggested creating a method that would pull out a dictionary from the existing data.
I apologize for my bad English.

I don't understand what you mean by "pulling out a dictionary from the existing data". Like represent the data as a dictionary, rather than have it in the response classes? You can access status.raw which will give you a dictionary of the received data, exactly as it was received from the server.
However this data is unparsed, so for example the motd may be hard to work with in comparison to using status.motd, which gives you back an instance of the Motd class, that contains some neat logic allowing you to get various representations of the motd (such as pure text, ignoring any colors and formatting), minecraft formatting with the section/paragraph signs (like §a for green), and some other things.

Also, this discussion went pretty off topic from the original issue by now, github issues aren't really a great place for bikeshedding, they're here to report a bug, propose a feature, or ask questions about a single thing. Not to discuss the project as a whole. If you're interested in those kinds of discussions, consider visiting our discord instead: https://discord.gg/C2wX7zduxC
As for the original question/issue, as I've mentioned query timing out just means mcstatus wasn't able to make a connection to the server. This is likely due to a misconfiguration in your network, which ends up blocking UDP traffic. Try making the request directly from the same server where the minecraft server is running, and use a local IP in lookup. If this also doesn't work, make absolutely certain that you've saved the server.properties file with query enabled, and restarted the server afterwards. Also make sure that query-port is the same as the server port you're using for all TCP traffic.
If none of these suggestions worked for you, we'll need a minimal reproducable example. Since you're running a minecraft full server here, please provide us with the exact steps to recreate it, so that we can test this ourselves and figure out what's wrong.
If you're using docker, this can just mean giving us the docker image, and all of the plugins that you're using. Otherwise, we'll need to know the operating system you're using, and have the exact instructions you've used from after installing this operating system to set up the minecraft server (and again a list of all plugins you've used).
I would suggest to first try and remove all plugins and see if it works then, if so, just keep adding plugins until it stops working, so you can figure out which plugin is causing the issue. If it doesn't it's the server distribution itself, which is odd, as it likely means it doesn't conform to the established minecraft protocol standard. In this case, we'll need the details I mentioned above.

Thank you very much. As I already wrote above, everything works, your advice with limiting the timings helped.


Change all reports from minutes -> hours

Some reports show time in minutes, it should display hours.

Completed


HackOlympians Registration Flow and Challenge

Added screen to select between HackOlympians and General Attendee Tracks. This page is accessible at /registration-type:

Changed all internal files/variables with HackOlympian in name to use "Pro" in naming

Added screens for HackOlympians challenge:

Connected the Pro Track Challenge endpoint to the HackOlympians Challenge. Behavior:

On /registration-type (HackOlympian or General Track selection page): If the user finished the challenge, redirect to /register. Otherwise, stay on the page.
On /hack-olympians-challenge (Wrapper page for the entire challenge): If the user finished the challenge, redirect to /register. Otherwise, stay on the page.
When the user clicks "Finished Challenge", if they passed the challenge, they're redirected to the challenge successfully completed page. If they did not pass the challenge, they're redirected to the challenge failure page. Otherwise, text below appears saying that they have not finished the challenge yet.

Update: The "Sign Up As" page is now accessible at /register/sign-up-as instead of /registration-type


Valid URLs failing validation - query and fragment parts

Hi!
I have encountered a few weird urls that are not passing validation even though they are functional.
The issue is with the validation of the query and fragment parts of the URL.
Furthermore, the query and fragment validations in url.py do not conform to their rather loose definitions in RFC3986.
Is that intended or not? If not, I'll open a PR to fix it.
Examples:

https://vydrica.com/ponuka-bytov/#3d-navigator/

https://www.karpatium.sk/ponuka-byvania#/budova/C2/podlazie/3/byt/96

https://moldaupark.cz/img/reality/262/ddca52d8e0680aa3efe02416daf30eab/vamax800/25.jpg?-62169987208

https://t.rmcl.cz/fhi.cz/estate/i0/81/1024x1024-ke5f0-48bdb-prodej-rodinneho-domu-145m2-pardubice-hostovice-b1-hostovice-int0007-resize-978a19b97a-2226429600.jpg?-JJ3I5Aj2jBX0fDNYvETA9DwXTq_wsCIEBHV-DdFyxT6vVsPh6Ges8F0UgAUKmabnAwcYfA2rSt0SdrwaNFR1lb8vA8Fx_AFNi-B3YlnwsWD_CkjZ19OfbGYlMTWEE_lyTPNHAwESPFFBD1ccfylVagVLIqU1t1N7TmiIV0TGg0Xyd9dKipP7p_4kmWDbY2jJMjDEcOiGs1HiVPe8lytnqCetIQ

Hello,
i just was in the mid of creating a bug report as well (thanks for raising this :+1:) for previously valid URLs which are now reported as invalid after the update from 0.20.0 to 0.21.2 (still seen also after the update to 0.22.0 to include the fix from #289 for #288).
These are the cases we're seeing as reported to be invalid while they are actually (to the best of my knowledge, haven't checked all of them) valid:
https://launchpad.support.sap.com/#/notes/2718993
https://forums.livezilla.net/index.php?/topic/10983-fg-vd-19-086-livezilla-server-is-vulnerable-to-sql-injection-ii/
http://www.brocade.com/en/backend-content/pdf-page.html?/content/dam/common/documents/content-types/security-bulletin/brocade-security-advisory-2016-168.pdf
https://groups.google.com/forum/?fromgroups#!topic/rubyonrails-security/8SA-M3as7A8
https://forum.bitdefender.com/index.php?/topic/75470-doubleagent/
https://www.smartftp.com/forums/index.php?/topic/16425-smartftp-client-40-change-log/
https://exchange.xforce.ibmcloud.com/#/vulnerabilities/100912
https://support.microsoft.com/en-us/lifecycle#gp/Microsoft-Internet-Explorer
https://www.watchguard.com/support/release-notes/fireware/12/en-US/EN_ReleaseNotes_Fireware_12_5_9/index.html#Fireware/en-US/resolved_issues.html
https://code.wireshark.org/review/#/c/25660/
https://groups.google.com/forum/#!topic/kubernetes-announce/yBrFf5nmvfI
https://issues.sonatype.org/plugins/servlet/mobile#issue/NEXUS-16870
http://forums.livezilla.net/index.php?/topic/163-livezilla-changelog/
https://www.watchguard.com/support/release-notes/fireware/11/en-US/EN_ReleaseNotes_Fireware_11_12_1/index.html#Fireware/en-US/resolved_issues.html%3FTocPath%3D_____13
https://review.typo3.org/#/c/37013
https://forums.malwarebytes.org/index.php?/topic/158251-malwarebytes-anti-exploit-hall-of-fame/
http://speedtouch.sourceforge.io/index.php?/news.en.html
https://support.k7computing.com/index.php?/Knowledgebase/Article/View/173/41/advisory-issued-on-6th-november-2017
http://support.novell.com/cgi-bin/search/searchtid.cgi?/10077872.htm

Here's the patch:
�diff --git a/src/validators/url.py b/src/validators/url.py
index 16698b1..16259e7 100644
--- a/src/validators/url.py
+++ b/src/validators/url.py
@@ -3,7 +3,7 @@
 # standard
 from functools import lru_cache
 import re
-from urllib.parse import unquote, urlsplit
+from urllib.parse import parse_qs, unquote, urlsplit

 # local
 from .hostname import hostname
@@ -34,11 +34,6 @@ def _path_regex():
     )

-@lru_cache
-def _query_regex():
-    return re.compile(r"&?(\w+=?[^\s&]*)", re.IGNORECASE)
-
-
 def _validate_scheme(value: str):
     """Validate scheme."""
     # More schemes will be considered later.
@@ -108,16 +103,23 @@ def _validate_netloc(
     ) and _validate_auth_segment(basic_auth)

-def _validate_optionals(path: str, query: str, fragment: str):
+def _validate_optionals(
+    path: str,
+    query: str,
+    fragment: str,
+    strict_query: bool = False
+):
     """Validate path query and fragments."""
     optional_segments = True
     if path:
         optional_segments &= bool(_path_regex().match(path))
-    if query:
-        optional_segments &= bool(_query_regex().match(query))
+    if query and parse_qs(query, strict_parsing=strict_query):
+        optional_segments &= True
     if fragment:
         fragment = fragment.lstrip("/") if fragment.startswith("/") else fragment
-        optional_segments &= all(char_to_avoid not in fragment for char_to_avoid in ("/", "?"))
+        optional_segments &= all(
+            char_to_avoid not in fragment for char_to_avoid in ("?",)
+        )
     return optional_segments

@@ -130,6 +132,7 @@ def url(
     skip_ipv4_addr: bool = False,
     may_have_port: bool = True,
     simple_host: bool = False,
+    strict_query: bool = True,
     rfc_1034: bool = False,
     rfc_2782: bool = False,
 ):
@@ -167,6 +170,8 @@ def url(
             URL string may contain port number.
         simple_host:
             URL string maybe only hyphens and alpha-numerals.
+        strict_query:
+            Fail validation on query string parsing error.
         rfc_1034:
             Allow trailing dot in domain/host name.
             Ref: [RFC 1034](https://www.rfc-editor.org/rfc/rfc1034).
@@ -214,5 +219,5 @@ def url(
             rfc_1034,
             rfc_2782,
         )
-        and _validate_optionals(path, query, fragment)
+        and _validate_optionals(path, query, fragment, strict_query)
     )

After applying you'll have to pass strict_query = False to get those URLs validated.
PR is welcome.


Support ability to pass in placeholder values when deploying

I need the ability to dynamically pass in placeholder values from XLR to XLD. It is possible by simply modifying the deployment XML post but at present the only way to do so is hacky (#31). I can't just use dictionaries since they change with every deployment. Using dictionaries would require one per each application (or else concurrent deployments become a problem) which would mean 1000's of them which isn't maintainable.

Hi Mark,
Can you have a look if v2.7.0 fits your need. I've added the overrideDeployedProps.

@px-djozis what do you think? XPath seems necessary since placeholders store the key in the attributes section.
For usability would it make sense to use findall so that the XPath expression could modify all matches at once? It would make things easier. If someone wanted to just modify one thing then they can just use the exact match for the XPath expression.
Is it named properly? It was intended for deployed properties but it looks like it can be used to modify the entire deployment XML (https://docs.xebialabs.com/generated/xl-deploy/5.1.x/rest-api/com.xebialabs.deployit.engine.api.dto.Deployment.html).

I don't personally find xpath an intuitive way to set placeholders. I think having a way to specify a list of deployable id matches supporting */ style wildcarding and their placeholder substitutions to make would be both flexible and intuitive, since you asked for my input.

David and I talked about it. We don't actually need the power of xpath - I was simply trying to use it to achieve what I wanted in #31. We think a more user friendly impl would be this:
A dictionary of id to a dictionary of key, value pairs. The property would be "overridePlaceholders" or something like that.

{ "id1" : { "key": "value" }, { "key2" : "value2" }, "id2": ... }

"id" would correspond to the id in XLD. It should be able to use the standard "ant style" path wildcarding:

** matches zero or more directories

matches zero or more characters

? matches one character

Hi Mark,
That would be limiting this feature into placeholders only. Which I don't like, as for example you might also want to change non placeholder deployed properties. So for the time being I'll stick with xpath.
Maybe we should go back to the beginning, and ask why you want to change placeholders/deployed properties during deployment (as that is some anti pattern already).
Regards,
Joris

@jdewinne could you at least change the find to a findall and loop over all xpath matches? This would make things much easier.
The reason we are using placeholder replacement because the placeholders represent the state of an application in a blue/green deployment. Storing the state in dictionaries becomes complicated because of concurrency issues and annoying application restrictions that would have to be set.

This would do it:
def override_deployed_properties(deployment_xml, deployed_properties):
    root = ET.fromstring(deployment_xml)
    if deployed_properties:
        deployeds_properties_dict = dict(ast.literal_eval(deployed_properties))
        for key in deployeds_properties_dict:
            for pkey_xml in root.findall(key):
                pkey_xml.text = deployeds_properties_dict[key]
    return ET.tostring(root)

Hence I can do something like this:
<EMAIL_ADDRESS>'blue'),
<EMAIL_ADDRESS>'green')]'''
Instead of manually listing out all id's. The id's approach is also problematic because it requires knowledge of that whose will be and what deployables will actually end up creating deployeds.

Released v2.8.0 with findall in it.

A coworker working on this has come across an issue with the documentation that also demonstrates a limitation of the plugin:

overrideDeployedProps (Map contain xpath expression to override deployed properties.) e.g.: {'deployeds/openshift.ResourceModule[<EMAIL_ADDRESS>'value'}

The xpath @id expression must be an exact match. It will not match a subet and appears it does not work with wild carding. xpath does have a contains() function but the python ET implementation does not support it.
As a result either

all matching deployeds must have placeholders set at the same time or
each deployed must be explicitly configured.

The problem with the first is it may not be flexible enough. The problem with the second is you won't know the deployed paths until the deployment is created (you could make a guess based on the application but without replicating tag processing an restrictions you wouldn't be 100% sure). The deployment is created by this plugin but there is no way to get that information in-between the initialization of the deployment and the filling in of the deployed properties. Hence deployeds cannot be modified based on information that is dependent on the deployed first being initialized.


https://mathstutor.life/ doesn't actually work in a proxy.

Title says it all, probably remove the clarification.

Thanks for letting me know


Ctrl+C behaviour

Maybe I simply remember it wrongly. If so apologies.
In the past we discussed about how to stop a measurement, which I believe generated the multiple Ctrl+C behaviour currently implemented.
We still get problems with scripts, motor motions (also via HKL calcs), detector counting (sometimes keeping counting) when pressing (one or more) time the key combination. For example:

|       127 | 11:31:01.5 |    72.0000 |             72.0000 |           1365106 |         109934284 |          77012195 |          39928449 |    302.772 |    302.755 |       33.3 |       28.8 |      90.20 |
|       128 | 11:31:17.5 |    71.5000 |             71.5000 |           1464403 |         115451570 |          82354251 |          42525927 |    302.773 |    302.748 |       33.4 |       28.9 |      88.08 |
|       129 | 11:31:33.6 |    71.0000 |             71.0000 |           1553105 |         122522550 |          88431463 |          45705911 |    302.809 |    302.746 |       33.4 |       28.9 |      85.50 |
|       130 | 11:31:49.5 |    70.5000 |             70.5000 |           -580641 |          -8838139 |         -31941176 |         -13613515 |    302.792 |    302.747 |       33.0 |       28.8 |      82.85 |
^CA 'deferred pause' has been requested. The RunEngine will pause at the next checkpoint. To pause immediately, hit Ctrl+C again in the next 10 seconds.
Deferred pause acknowledged.  The RunEngine will pause at the next checkpoint. To pause immediately, hit Ctrl+C again in the next 10 seconds.
Deferred pause acknowledged. Continuing to checkpoint.
^C^CAborting....
^Cgenerator ct ['12774f'] (scan num: 66659)

Starting procedure to acquire camera gains at 0.5Hz
^C---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
/home/xf23id1/Beamline/Commissioning/2016_07/2016_07_09_morning.py in <module>()
     54 #TempEquil(templimit=0.1)
     55 #sleep(600)
---> 56 one_T()
     57
     58 # T = 280K

/home/xf23id1/Beamline/Commissioning/2016_07/2016_07_09_morning.py in one_T()
      7     ## on the main edge
      8     one_meas(776.5)
----> 9     service_meas()
     10     ## on the pre-edge
     11     one_meas(773.5)

/home/xf23id1/Beamline/Commissioning/2016_07/2016_07_09_morning.py in service_meas()
     35     # darks
     36     olog('All darks #{}+0:2 @ {:.1f}Hz'.format((db[-1].get('start').get('scan_id'))+1, 1./fccd.acquire_time.get()))
---> 37     ct_all_dark()
     38
     39

/home/xf23id1/Beamline/Commissioning/2016_07/2016_07_09_morning.py in ct_all_dark()
     93     print('\nStarting procedure to acquire camera gains at {}Hz'.format(1/fccd.acquire_time.get()))
     94     caput('XF:23ID1-OP{FBck}Sts:FB-Sel',0) #maintain finepitch
---> 95     sleep(2)
     96     # Save current settings
     97     shutter_open = shchk()

KeyboardInterrupt:

In [314]: Traceback (most recent call last):
  File "_ctypes/callbacks.c", line 234, in 'calling callback function'
  File "/home/xf23id1/conda_envs/collection/lib/python3.4/site-packages/IPython/lib/inputhookqt4.py", line 90, in inputhook_qt4
    try:
KeyboardInterrupt

If you suspect this is an IPython bug, please report it at:
    https://github.com/ipython/ipython/issues
or send an email to the mailing list at<EMAIL_ADDRESS>
You can print a more detailed traceback right now with "%tb", or use "%debug"
to interactively debug it.

Extra-detailed tracebacks for bug-reporting purposes can be enabled via:
    %config Application.verbose_crash=True

KeyboardInterrupt

In [314]:

I had to Crtl+C a number of times trying to avoid other commands to be executed (without success...) and finally getting the prompt back.
Unfortunately, this might result in dramatic perturbations of the measurement status.
Is the equivalent of a simple, robust and effective "STOP everything NOW and listen to me" command available?

This is the reason that we keep telling you to write plans (that get handed to the RE on one shot) rather than writing functions that call the RE.
If you insist on making multiple calls to the RE in a function, do so with raise_if_interrupted=True as a kwarg.

@cmazzoli When I get back from scipy, let's sit down and convert some of your scripts to proper plans. There are others who are also interested in this.

A workshop covering plans is tentatively planned for a week from this coming Wednesday. Closing this issue.


Config file verbosity

It seems that the verbosity set in the config file no longer works, can you update the example?
The error outputs expected struct Verbosity.

I think the changes made to log verbosity #956 broken verbosity setting through configuration file.
In the past a test has been added to validate the configuration file parsing but it does not seem to be executed in the pipeline which might have helped introducing this issue.
PS: I don't know known Rust so I can't really help much fixing the issue.

Can you try again with the latest master? (You need to compile from source for that.)
The recent changes in #956 should actually have fixed the setting and the example file  should be up to date again. We support multiple verbosity levels now, so you need to set the verbosity like this:
verbose = "warn"

You can set it to info or error as well.
It's true that the tests don't get executed right now. I'm working on fixing that.

I was expecting to fail but vscode setup the devcontainers which made it easy :) Nice job and nice to discover this feature :+1:
Yes, adjusting the verbose level to strings ("warn", etc.) it works fine.
Since are using the GitHub action I think we can just wait for the next release.

Nice. Thanks for the feedback. I'll go ahead and close this. The release is scheduled to happen very soon, so stay tuned for that.


Compatibility with HP/HC not available ?

Hello @Aohzan and thank for you work!
I have an EDF contract with HP/HC so in the Teleinfo frame the field “T1_base” is always set at 0.
My real consumption is in the fields “T1_HCHC” and “T1_HCHP”.
It is possible to catch those fields in HA ?
Thanks in advance.

Hello,
you have them in the attributes of main sensor : https://github.com/Aohzan/ecodevices/blob/master/custom_components/ecodevices/sensor.py#L227

Hello,
Thanks for the quick answer  !


got the error "Check failed: target_blobs.size() == source_layer.blobs_size() (2 vs. 0) "

When i run python mnist_vae.py, got "Check failed: target_blobs.size() == source_layer.blobs_size() (2 vs. 0) Incompatible number of blobs for layer decode4". This error come from 'net=caffe.Net('mnist_vae.prototxt','snapshots/mnist_vae_iter_60000.caffemodel', caffe.TEST'.
I have checked the code,but found nothing wrong. hope for your help! :sob:

Thank you, I had changed some layers before.When I revert the commit,it runs without error! :+1:


Feat/upgradable kakarot

Pull request type

Please check the type of change your PR introduces:

[ ] Bugfix
[X] Feature
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] Documentation content changes
[ ] Other (please describe):

What is the current behavior?

Kakarot is deployed as is. Every change requires an new deployment resulting in a new contract address.
Resolves #284
What is the new behavior?

Kakarot_proxy.cairo A proxy contract that implements kakarot
A deploy-script for deploying all contracts on testnet.
A basic admin setup for ownership over the Proxy contract. It consists of two Roles:

Implementation Role: Can upgrade the proxy implementation
Admin Role: Can assign Implementation role

Other information

I know there is an old deployment script using CLI commands.
However, that deployment script does not work anymore and as I am not that familiar with shell scripting I create a python deployment script.

Closing PR due to lack of activity from my end.


clarifications

Hi,
I'm working on getting this deployed and having some issues, so just wanted to clarify a few things from the Deployment section of the wiki:

the wiki says "You deploy all this to a Continuous WebJob as you would normally for a regular WebJobs SDK job." I'm following Amit Apple's instructions for where to store jobs. That said, would these jobs go in the continuous folder even though they are triggered (queue for example)?

is run.cmd supposed to live in wwwroot/? and if so, do the only contents need to be .\bin\WebJobs.Script.Host.exe?

I copied the simple QueueTrigger example, but it keeps saying the status is Pending Restart, hence my questions above. Thinking something may be wrongly configured.
Thanks in advance!
James

I'm going to update/fix the deployment steps this morning based on recent changes. I'll point you to then when I'm done :)

@mathewc thank you! much appreciated!

I've made a first stab at updating the deployment steps. See: https://github.com/Azure/azure-webjobs-sdk-script/wiki#deployment.

Hi again, @mathewc. I'm still not 100% clear on the deployment instructions, so again just wanted to clarify to make sure what I'm doing is correct:

Add connection strings to the web app
Add host.json and any web jobs to the D:\HOME\SITE\WWWROOT\App_Data/jobs/continuous/ folder
Add the WebJobs.Script.Host files to the D:\HOME\SITE\WWWROOT\bin directory
Create a run.cmd file in D:\HOME\SITE\WWWROOT that executes the script host

Thanks and sorry to be a pain!
James

Nothing gets put in App_Data/jobs/continuous - put everything in the exact paths I indicated in the wiki :) To clarify, the Kudu job runner is not running these script functions - this is an entirely new model. Give it a try and let me know if you have any issues.

 I've corrected them, and added some screen shots. Let me know it that works for you :)

@mathewc haven't tried the new instructions out yet, but they are definitely super-clear now. thanks for the updates!


Add support for nested fields arguments

As eagle258 pointed out on reddit, it is possible to have nested field variables:
query {
  viewer {
    name
     repositories(last: 5) {
       nodes {
         name
       }
     }
   }
}

Although the qlient supports nested field selection, a nested variables selection is yet not possible.

Implemented with #27

@dfseifert
I can't figure out how to set nested field arguments with qlient.
Looking at the source code, it seems that this feature was removed with the v1.0.0 release.
Is this true, or did I miss something?


Cython support

add support for Cython
add benchmark test script
modified image pixel access method for performance purpose

Hi, have you review PR may be? I wish used it in my production project and to switch from own forked version to basic PyPI.


"Invalid char ':'" error when building with GO111MODULE=on

Go v1.12.4 on macOS High Sierra.
When trying to build caddy v1.0.0-beta1 or master with GO111MODULE=on, go fails to extract this plugin due to colons in filenames:
go: extracting github.com/pyed/ipfilter v1.1.1
-> unzip<EMAIL_ADDRESS>malformed file path "testdata/blacklist/1234/abcd/1234:abcd::1": invalid char ':'
build github.com/mholt/caddy/caddy: cannot load github.com/pyed/ipfilter: unzip<EMAIL_ADDRESS>malformed file path "testdata/blacklist/1234/abcd/1234:abcd::1": invalid char ':'
2019/04/16 18:26:12 exit status 1
exit status 1

It seems the Go team do not plan to allow colons in filenames, as it's a reserved character on Windows.

I can appreciate why a colon is a reserved character in MS Windows. It also has special meaning on other non-UNIX operating systems (many of which are of historical interest only). But looking at https://docs.microsoft.com/en-us/windows/desktop/fileio/naming-a-file#naming-conventions I'm left shaking my head in disbelief. They also disallow <, >, ?, and * and other characters in file names. WTF? Does Go also disallow those chars in module file names?
Regardless, we should probably rename those unit test support files and modify this module to support both colons and some other separator char for IPv6 addresses.

Does anyone have an opinion on the optimal alternative separator char for IPv6 addresses? Underscore? Hyphen? Dollar-sign? Hash(sharp)-sign? Something else? I'd prefer not using periods due to ambiguity with IPv4 addresses.

Does anyone have an opinion on the optimal alternative separator char for IPv6 addresses?

🍺

We need to go with an alternative character, hyphens sounds okay .. I guess, any thoughts ?

and .. isn't going to work at all in a filename

I can't speak for how MS Windows file APIs work but dots, even two consecutive dots, are valid in a UNIX file name. A double dot only has special significance when that is the entire file name.

is confusion with IPv4 addresses likely?

There isn't any ambiguity in practice but using periods does make it marginally harder for a human to tell at a glance whether it's an IPv4 or IPv6 address.

hyphens sounds okay .. I guess, any thoughts ?

It should be a char that doesn't have special-meaning when used in a shell bare word. So that rules out dollar-sign, hash-mark, etc. I'm leaning towards = for the symmetry with : but - is fine too.

2001=db8=a0b=12f0==1 OR 2001-dbz-a0b-12f0--1, I guess = looks more like : and - makes it look like a Mac address, so = it is.

@pyed Do you want to take care of this or would you prefer I do so since I wrote the code in question?

@krader1961 I'd really appreciate it if you write the fix for this as you know your way around this part of the code better than I do.

@krader1961 Any estimate on when you'll open a PR to fix this issue? I can't build Caddy from source because of this problem.

@whalehub I will try fix it this weekend; i.e., the next 48 hours. But I'll need to send a P.R. for review and that could take a couple more days before it is merged.

@krader1961 Thank you!

@krader1961 Thank you!
@pyed Can you please push a v1.1.2 release which includes this commit?

I can't speak for how MS Windows file APIs work but dots, even two consecutive dots, are valid in a UNIX file name

Windows doesn't like filename ending in a dot. Yup, it's technically valid in UNIX filenames, but code to mitigate filesystem traversal vulns doesn't always know that and may reject any path with .. in it.

@pyed Thank you!

@pyed Thank you for the v1.1.2 release!


De-duplicate DefaultTagHelperDescriptor instances

Heap dump analysis for https://devdiv.visualstudio.com/DevDiv/_workitems/edit/1775286 revealed 853,941 instances of DefaultTagHelperDescriptor retaining 3.2GB memory, including a subset of 3,613,367 instances of DefaultBoundAttributeDescriptor consuming 2.5GB. Verifying immutability and de-duplicating these instances is likely to produce a significant improvement to working set.

@dotnet/razor-compiler. I'll take a look at this. It's something that I'm actively working on. Also, I don't believe the Compiler is really at fault here.

This was fixed with #9214


No 'version' key in the manifest file for custom integration 'tapo_p100_control'

Hi,
Just seen this in my logs in HA, I cloned the latest version just now and the error is still in the log, not sure if you were aware or not.
Logger: homeassistant.loader
Source: loader.py:794
First occurred: 17:21:50 (1 occurrences)
Last logged: 17:21:50
No 'version' key in the manifest file for custom integration 'tapo_p100_control'. As of Home Assistant 2021.6, this integration will no longer be loaded. Please report this to the maintainer of 'tapo_p100_control'

That's weird, could it be:
"version" : "0.0.0" causing the issue?

I wrote that patch, and it works fine for me. I can only assume your issue is a local one.


Improved schema and converters for bibliographic data

I've tried to consolidate a number of proposals by @zuphilip (#487, #490, #491, #481)
Tasks:

[ ] verify updated Dar spec
[ ] Normalise properties of EntityDatabase to use same props as CSL-JSON (not reinvent the wheel), e.g. some of the source properties can be called containerTitle (we just add a note to which JATS elements those properties refer to)
[ ] Update EntityDatabase schema to conform to latest spec
[ ] converters (JATS <-> EntityDb, and import from CSL-JSON

@substance/texture-contributors please comment on the latest Dar spec! It's important that we now settle on the schema so our implementation will hold for a long time.
@zuphilip want to take over the implementation part? As a time-frame end of next week would be ideal... (does not need to be final, just important to have the implementation in sync with the spec again).

Okay, I can start with the implementation. I can spend some time today, but don't know how much time I can spent the other days...

Think we can close this.


User Dashboard

- [ ] Section to display current access rights or user permissions (optional (not sure, whether it is needed or not))
https://realpython.com/django-user-management/#create-a-dashboard-view

I am working on this and an issue came to mind:
Normally we would not allow to change the email or the password without entering the current password. This is to prevent the account getting hijacked if somebody can access the users session (e.g. because the user left the computer unattended for a moment or with a virus or in-browser attack with javascript). Normally this is solved by letting the user enter the current password before submitting a new password or change the email address.
So how should the edit-modals look like? Email is on the same "security-level" as password because it can be used for the password reset. Should we move email to the right and put it under username? Then I would show a field to enter the current password, in the dialog for the right column.

@frederikbugglin Could you take a look at this at the DEMO instance? I made a few changes:

Move the email field to the account information
Allow password changes only via reset-email

Also, I added the dev-mode feature in here, is it acceptable like this?


DAX for date range returning data from date slicer

Hi All,
i need your help,  i am trying to return the max(date) from the main table "Test" according to the Date slicer filter.
Please find the attached photo which shows three different dates, i created measure to return (min_Date and max_Date) based on filter Date, i created summarize table from "Test" table as i need to have the possibility to display values from Test table based on selected date slicer [Date], but unfortunately i am returning the max date value of Test table not from  max_date measure.
DAx function:

min_Date = min(test[Date])
max_Date = max(test[Date])
Selected max Slicer Date= return SUMMARIZE(test,"Modified",[max_Date])

Thanks in advance

Thanks for your comment, @mibrahim1976 . This looks like a support request rather than documentation feedback. You might find a solution in the Power BI Community, found at https://community.powerbi.com. If not, you can get support by going to https://support.powerbi.com

Thanks @davidiseminger , i just open a ticket https://community.powerbi.com/t5/Desktop/Calculate-max-date-value-from-date-range/td-p/2321185

add tag


Fixed example image in cloudflare.md

example.png wasn't pointing toward the right place, updated path to fix. :)

That image path is used on cloudflare site: https://www.cloudflare.com/apps/abetterbrowser/


What has changed in v0.90.6?

Hey guys,
great work with this db.
Where can easily see, what's new in each new version?

The v0.90.6 changelog hasn't been pushed to master yet:
https://github.com/realm/realm-cocoa/blob/v0.90.6/CHANGELOG.md

Usually the changelog on master has the changes for all previous releases.


Federico: Watch Cybershieldnotch

Federico requests the watch of the watch_keyword Cybershieldnotch. See the MS search here and the Stack Exchange search here.

Cybershieldnotch has been seen in 2 true positives, 0 false positives, and 0 NAAs.

!!/approve

Coverage remained the same at 66.34% when pulling ac5f9041ac042e6951961cb6fc19aa63ed8ab0d6 on auto-blacklist-1533902756 into d5b181495ca1fcca6d27070793831283e2dd059a on master.


Null safety support

Can't compile sdk for Android:
Error: Cannot run with sound null safety, because the following dependencies
don't support null safety:

package:flutter_flurry_sdk

The library 'package:flutter_flurry_sdk/flurry.dart' is legacy, and should not be imported into a null safe library.

Oh, seems to be the duplicate of the existing issue https://github.com/flurry/flutter-flurry-sdk/issues/2
Closing this issue then

@Talgat777
Null Safety is now supported https://pub.dev/packages/flutter_flurry_sdk
Please use $ flutter pub add flutter_flurry_sdk to install in your projects.
Thanks for your supports!


Load reimbursements text

fix #222

Missing steps:

[x] Add a field to Reimbursement model
[x] Add a command similar to the one that load suspicions
[ ] Add this field (and set is as searchable) in the dashboard

Also, this new command has a lot of testing similar to suspicions, some testing refactor is required ;)


Thread processing of v2 public API within cache crawler

Description
This PR includes the following:

Processes the v2 public API endpoints within a seperate thread when crawling the public API endpoints. This is so that we don't wait for the v1 API to be processed completely first before moving onto the v2 API.
Due to the nature of the index, the first few paths will be the slowest (before being cached) so its important to cache them first as quick as possible

Fixes #CDD-2165

Type of change
Please select the options that are relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Tech debt item (this is focused solely on addressing any relevant technical debt)

Checklist:

[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added tests at the right levels to prove my change is effective
[ ] I have added screenshots or screen grabs where appropriate
[ ] I have added docstrings in the correct style (google)


Sqlite chrono::DateTime timezone fix

Fixes: #1617

@abonander When can you merge this? Is this going to be in 0.6?

I'm currently working on a 0.5.11 release so I can't say when 0.6.0 is coming out.
Since this changes the format that timestamps serialize to in an observable way, I think this is technically a breaking change and would have to be part of 0.6.0.


fixes 41446 ocp 4.6 support

changes

fixed ocp 4.6 permission issue
switched operator to 3.2.3

tests

passed local test

/lgtm


ranger: 1.9.3-unstable-2023-08-23 -> 1.9.4

Package Information

Package name: ranger
Latest released version: 1.9.4
Current version on the unstable channel: 1.9.3-unstable-2023-08-23
Current version on the stable/release channel: 1.9.3-unstable-2023-08-23

Maintainer Ping:
@toonn @magnetophon

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

nixpkgs-reviewd workflow started
Status and logs: https://github.com/nixosbrasil/nixpkgs-reviewd/actions/runs/12468504402

nixpkgs-reviewd workflow finished
Status and logs: https://github.com/nixosbrasil/nixpkgs-reviewd/actions/runs/12468504402
nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review pr 367654

x86_64-darwin

:white_check_mark: 3 packages built:

ranger
ranger.dist
vimPlugins.ranger-nvim

nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review pr 367654

aarch64-darwin

:white_check_mark: 3 packages built:

ranger
ranger.dist
vimPlugins.ranger-nvim

nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review pr 367654

x86_64-linux

:white_check_mark: 3 packages built:

ranger
ranger.dist
vimPlugins.ranger-nvim

This PR doesn't update the src argument. Weird.


Testing Integration

I have implemented a basic test suite using Tape.

Coverage Report is generated in coverage/lcov-report/
Refactored Server to allow for easier testing

Moved Dotenv init into Server.start() to simplify starting the server when testing. Would it make sense to maintain a separate env file for testing?

Issues

With the current data persistence setup it's difficult to create seed data, but that should change with the typeorm integration.

Thank you very much for your contribution! :smile:


Update lighthouse dependency to official release version

In package.json it's currently pinned to a specific commit because we wanted to use a feature that wasn't published yet. We should update to the next release whenever it's available.

There is a new release: https://github.com/GoogleChrome/lighthouse/releases/tag/v6.4.0


Cannot reuse a MaterialPageRoute after disposing it.

I/flutter (13371): #12     PointerRouter._dispatch (package:flutter/src/gestures/pointer_router.dart:73:12)
I/flutter (13371): #13     PointerRouter.route (package:flutter/src/gestures/pointer_router.dart:101:11)
I/flutter (13371): #14     _WidgetsFlutterBinding&BindingBase&GestureBinding.handleEvent (package:flutter/src/gestures/binding.dart:143:19)
I/flutter (13371): #15     _WidgetsFlutterBinding&BindingBase&GestureBinding.dispatchEvent (package:flutter/src/gestures/binding.dart:121:22)
I/flutter (13371): #16     _WidgetsFlutterBinding&BindingBase&GestureBinding._handlePointerEvent (package:flutter/src/gestures/binding.dart:101:7)
I/flutter (13371): #17     _WidgetsFlutterBinding&BindingBase&GestureBinding._flushPointerEventQueue (package:flutter/src/gestures/binding.dart:64:7)
I/flutter (13371): #18     _WidgetsFlutterBinding&BindingBase&GestureBinding._handlePointerDataPacket (package:flutter/src/gestures/binding.dart:48:7)
I/flutter (13371): #19     _invoke1 (dart:ui/hooks.dart:134:13)
I/flutter (13371): #20     _dispatchPointerDataPacket (dart:ui/hooks.dart:91:5)
I/flutter (13371): (elided 2 frames from class _AssertionError)
I/flutter (13371):
I/flutter (13371): Handler: onTap
I/flutter (13371): Recognizer:
I/flutter (13371):   TapGestureRecognizer#7377b(debugOwner: GestureDetector, state: possible, won arena, finalPosition:
I/flutter (13371):   Offset(324.0, 549.1), sent tap down)
I/flutter (13371): ════════════════════════════════════════════════════════════════════════════════════════════════════
Application finished.

Occurred after a FAB pressure.

Please add the output of flutter doctor.
Can you please also add the code that allows to reproduce and the full error output?

The problem was that I was using Navigator.pop(context) before navigating to the new route, and when I clicked in the appBar's go back button, the error was generated. I can't get the original environment now, but let me know if you still need and I will post by the evening. Thx!

Unfortunately the full output from vscode debug is on the issue description, that's all I got. :(
About the code I can provide some in the next days.

@Hixie

Sounds like you're somehow interacting with a route that's already been removed?

@odahcam Do you still have some example code that caused the issue? Can you post it?

Sorry, I don't even remember what project it was. 😞


fix: LocalizationProvider in a Suspense boundary

Description
RE: #64
@frandiox discovered we aren't wrapping <LocalizationProvider> in a Suspense boundary. That means that when the query runs uncached, it throws to the React runtime and perhaps causes the Cannot use <Switch>... error.
Additional context

Before submitting the PR, please make sure you do the following:

[x] Add your change under the Unreleased heading in the package's CHANGELOG.md
[x] Read the Contributing Guidelines
[x] Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. fixes #123)
[x] Update docs in this repository for your change, if needed

Excited to see if this gets rid of the cannot use <Switch> error! 🎉


WebSocket Support

Will WebSocket be implemented in the project or should it be handled in middleware ?

+1


Release option for item should allow options dictated by collection release setting.

Release option on the item is not displaying the option selected on the collection.
The collection is js114xn4901/83.
The item is yp355vz4656/4349.
This deposit is set to release on Jan. 14, 2022. The author requested an extension. The collection was originally set for 6 months max embargo, with the depositor choosing release timing. I updated the collection setting earlier today to allow for one year, and then planned to update the item.
However, when I went to look at the item in H2, it said the item was released immediately. Argo and PURL indicate the embargo is still active as it should be. Initially after talking to Andrew, we thought the embargo date was not migrated properly from Hydrus. However, I'm now wondering if this is a problem with the application not reflecting the proper release setting for the item since when I open the item to edit it, I still says it will be released immediately and will not give me the option to choose.

A few additional details:
In Hydrus, the item ~shows~ showed the embargo release setting for January 14, 2022 - which is what can be seen in Argo https://argo.stanford.edu/view/yp355vz4656.
In H2, the item says it was released immediately upon deposit, even though the item still has not been released.

The item was registered in Hydrus in May 2021 and then published in mid-July 2021, not long before the launch of H2 and possibly after the soft launch of H2. I still do wonder if the embargo metadata was never migrated to H2. If not, I wonder if that could still be done and if that would then allow us to extend the embargo within H2 itself.

Embargo has been updated to July 14, 2022 via Argo, but the above issues still need to be investigated.

I concur with Andrew. In the data export the item "yp355vz4656" did not have a embargo_release_date property, but other items did.

We determined this was a bad migration due to Fedora having stored the embargo date in two places.  Some of the records (with cc 4.0 licenses) were corrected to only have one location, but the export from Hydrus used the old location. We remediated with:
data =[
%w[druid:ct913hh2270 2023-11-28T00:00:00+00:00],
%w[druid:gf072tq5537 2022-05-05T00:00:00+00:00],
%w[druid:gz610dt4642 2022-02-01T00:00:00+00:00],
%w[druid:jg418vz8621 2024-01-01T00:00:00+00:00],
%w[druid:kd878gf5841 2022-06-02T00:00:00+00:00],
%w[druid:kq632vn7933 2024-11-05T00:00:00+00:00],
%w[druid:kv313yy7422 2024-05-13T00:00:00+00:00],
%w[druid:mx814rh9820 2022-03-22T00:00:00+00:00],
%w[druid:rk392dj6710 2022-06-03T00:00:00+00:00],
%w[druid:sr420hn6830 2024-03-02T00:00:00+00:00],
%w[druid:tq095bx1940 2023-01-01T00:00:00+00:00],
%w[druid:xq853fr8935 2022-03-16T00:00:00+00:00],
%w[druid:xw875vf5293 2022-06-29T00:00:00+00:00],
%w[druid:xy954tt5426 2022-06-03T00:00:00+00:00],
%w[druid:yp355vz4656 2022-07-14T00:00:00+00:00]
]

data.map do |row|
  work = Work.find_by(druid: row.first)
  work.head.update(embargo_date: DateTime.parse(row.last))
end

@amyehodge Is this closeable now?

@jcoyne @andrewjbtw I just checked all of these. All the embargo dates are reflected accurately in H2. However, for two of them, the release information is wrong. They indicate release only to Stanford but should say world. gf072tq5537 is one of these, but it’s a test object of mine, so I’m not concerned about it. However, jg418vz8621 is a real object. The release info on this one should be updated to world.


Progress bar when using evaluator

Currently, when using evaluator.compute, there is no indication of the process and no good way to wrap tqdm around it.
I suggest give an option in compute so that the progress bar can be shown, i.e.
task_evaluator = evaluator("image-classification")
eval_results = task_evaluator.compute(
model_or_pipeline=pipe,
data=dataset,
metric=metric,
label_mapping= lm,
device=0,
progress_bar = True
)
Please let me know if there are any potential concerns related!

After doing some digging, I believe that this is basically dependent on the pipeline component of the transformer library. It's easy to forward a progress_bar: bool = False param into the pipeline's __call__ kwargs (here  and here).
But from that point on, it's a matter of what you're trying to do and if the dataset+pipeline can support progress bars. For example, on the call pipeline function, we can see that the actual pipeline could be many things, including but not limited to a GeneratorType (which does not advertise a __len__, a Dataset or a list (which typically have __len__), so the worse-case progress bar you can get would be a tqdm "X iterations / s" dialogue.
I believe it's totally possible to do that, so I'm going to open an issue referencing this in the transformers repo.


Fixes/improvements for the H.264 parser

A set of small changes that restrict the data types and valid ranges of some SPS parameters to make sure we don't get runtime panics due to overflows when processing invalid data.
Some of these problems have been revealed by fuzzing, some others are just drive-by improvements, notably those that move pre-computed values into methods to remove the possibility of inconsistencies and better align with the spec.

@bgrzesik I am mostly interested in your feedback for the first CL since it touches the encoder, but please feel free to look at the rest as this is the global direction I would like to move all the parsers towards (i.e. limit the range of read data to prevent runtime panics even with random or forged input).
For the encoder side, I wonder whether we should not make some of the builder methods fallible if their input would result in a potentially panicking SPS. Actually I think we should do this, but would like your thoughts before proceeding.


Add extension json2ts-multi-converter

Description
Convert JSON data to Typescript code

Screencast
https://user-images.githubusercontent.com/18333716/202422087-9605ee90-eb1a-42a5-b6a1-03450da84db6.mp4

Checklist

[x] I read the extension guidelines
[x] I read the documentation about publishing
[x] I ran npm run build and tested this distribution build in Raycast
[x] I checked that files in the assets folder are used by the extension itself
[x] I checked that assets used by the README are placed outside of the metadata folder

Hi @kyuuseiryuu 👋
Thanks for your contribution 🔥
It might look like we already have something like that in the store, could you check this out

Hi @kyuuseiryuu 👋
Thanks for your contribution 🔥
It might look like we already have something like that in the store, could you check this out
Thank you for reply.
Yes, I knew it, but it's not easy to use. So I want re-publish a new one.

Instead of making another one with the same purpose, it could be an idea to hear if @gon250 is up for a revamp 😊
Added some JSON data to my clipboard but I was not able to get it to work, do you have some example data?

Yeah, it's the same purpose right. But they are entirely different project.
if extension support install from Github Repo, I think it's will really cool.

example JSON data
{
  "code": 0,
  "result": {
    "name": "string",
    "continent": "string",
    "oldContinent": "string",
    "openTime": "2022-11-16T05:46:01.778Z",
    "officialId": "string",
    "isMerge": true,
    "districtMerges": [
      {
        "mergeDistrictId": 0,
        "mergeDistrictName": "string",
        "beDistrictId": "string",
        "beDistrictName": "string",
        "districtMergeDate": "2022-11-16T05:46:01.778Z"
      }
    ],
    "districtMergeDate": "2022-11-16T05:46:01.778Z",
    "nowDistrictId": "string",
    "nowDistrictName": "string",
    "isMergeStr": "string",
    "timeToLive": 0,
    "mergeQuantity": 0
  },
  "msg": "string"
}

Thank you for review and feedback. "preferences" is awesome features. Thanks for your teams hard work.

I'm open to review what you are implementing and try to update the extension we already have to add more features and improve usability with your help and ideas 🤩

@kyuuseiryuu Let me know if you want :), also I dont see a recording at the description to see what have this extension to make it even more simple. thanks!
cc @pernielsentikaer

@gon250  Hi, in your extension, user operating flow is:

copy JSON data
open extension
paste data
set TS code to clipboard

so, if user is copy the data to the clipboard already, why we should not read it automatically? so we can skip the step 3.
Mabe you hope user can re-edit the data in extensions, so you make a textarea field. But, someone who will use the extension he must be a developer right ? They must have better editor to do that. so I think we can remove the textarea field.
If we use form field, we have to use mouse or Tab key to move the cursor, it's can not over the operating flow quickly.
In this extension, user just copy data -> open extension -> press Enter key done without any mouse or Tab key operation.
Finally, some library may be has unfixed bug, so I think we should support multiple librarise to make users can chose.

@gon250 I updated the summary, will you merge this PR ?


Add window size setting?

Hello, is it possible to add the zstd window size / buffer setting to this API?
I can try a pull request but not familiar with the underlying zstd . I believe it is this: ZSTD_c_windowLog ?
Cheers,
--Joseph

Hi!
I just exposed this window_log parameter (as well as any zstd_safe::CParameter) on the encoders.
I also added the same for decoders.

Should be published with zstd 0.8.2.


Import openapi spec from url

currently, when a user wants to publish a new api, user has to paste the openapi spec  yaml/json to the form.
we can add way to import the openapi spec from url
** current form **

great idea! - we discussed this initially but were worried that there could be CORS issues when trying to request an openapi definition from an arbitrary URL - but that might be a corner case!?


Revert "build: Reduce patch needed to build chromium URL"

Reverts envoyproxy/envoy#14853

/assign @yanavlasov
Hi Yan, I'm currently working on https://github.com/envoyproxy/envoy/pull/17782 which needs to use the GURL class.
The optimization you did on this dependency removed GURL and I think reverting it might be the easiest solution.

Sorry I realized the revert still won't bring back GURL. I'll look for another solution. :<


is it possible to set and don`t let user change begin value?

I need to make user able to choose range of date only from a specific date is it possible ? :)

yes. My datepicker is based on Material datepicker, and have 100% support of min/max and other parameters. See official documentation about that.

yes, but i need to make, for example, from server i got date 20th of june and i need to made user to choose date 20th of june - ...

I don't see a problem

🤔 no I don't think it's possible to lock the 1st selected date without changing the lib

example: https://stackblitz.com/edit/angular-3lkegw?file=src/app/hello.component.ts
API: https://material.angular.io/components/datepicker/api

yes, but, for example, from server i got date 20th of june and i need to made user to choose date 20th of june - ... and user shouldnt be able to change start date

@DennyReM Did you found any solution for this?


Anything special to do for ssh?

I'm unable to get in via ssh after creating workstations. (Specifically trying centos gfx after doing a single-connector deployment). I understand to ssh I need to connect to CAC as bastion, but when I do I'm consistently rejected.
made key with ssh-keygen -t rsa @ ~/.ssh/id_rsa
var in terraform: ~/.ssh/id_rsa.pub
attempting connection with CAC via ssh -i -A -v ~/.ssh/id_rsa<EMAIL_ADDRESS>No dice. Any thoughts? I need to get these workstations wrapped into ansible and I'm currently unable to. Thank you for any thoughts or assistance.

Hello! Thanks for reaching out. Are you using the AWS or GCP single-connector deployment?

Very sorry, the AWS one.

Please try using the user ubuntu for the AWS deployments on the CAC. For more information on the different login users for each VM, please refer to our Troubleshooting page.

Im tremendously sorry for overlooking that table. Thank you for the prompt help!  Excellent build quality by the way!
Cheers


ReCaptcha Enterprise high TCP sage

Hi all,
We are integrating reCaptcha enterprise solution on our proyect and the service is crashing with the following when there is a high usage of the reCaptcha client:
Status(StatusCode="Unavailable", Detail="failed to connect to all addresses", DebugException="Grpc.Core.Internal.CoreErrorDetailException: {"created":"@1613648642.332000000","description":"Failed to pick subchannel","file":"T:\src\github\grpc\workspace_csharp_ext_windows_x86\src\core\ext\filters\client_channel\client_channel.cc","file_line":4090,"referenced_errors":[{"created":"@1613648642.332000000","description":"failed to connect to all addresses","file":"T:\src\github\grpc\workspace_csharp_ext_windows_x86\src\core\ext\filters\client_channel\lb_policy\pick_first\pick_first.cc","file_line":394,"grpc_status":14}]}")

And the following image is the tcp usage duing our performance tests on a 3xS2 webapp (Azure), you can see the difference between normal execution vs recatpcha gets enabled/working

This is our implementation of the code (following the recomendation of the documentation to use the same client, we have tried different ones but we still getting the same error)

public class ReCaptchaValidator : IReCaptchaValidator
    {
        private readonly ILogger<ReCaptchaValidator> _logger;
        private readonly ReCaptchaOptions _reCaptchaOptions;
        private readonly string _host;
        private readonly RecaptchaEnterpriseServiceClient _client;

        public ReCaptchaValidator(IOptionsMonitor<ReCaptchaOptions> optionsMonitor, ILogger<ReCaptchaValidator> logger, IConfiguration configuration)
        {
            _logger = logger;
            _reCaptchaOptions = optionsMonitor.CurrentValue;
            _host = configuration.GetValue<string>("Host");
            var builder = new RecaptchaEnterpriseServiceClientBuilder { JsonCredentials = _reCaptchaOptions.CredentialsJson };
            _client = builder.Build();
        }

        public async Task<bool> IsValid(string token, string action, CancellationToken cancellationToken = default)
        {
            if (!_reCaptchaOptions.Enabled)
            {
                return true;
            }

            // Initialize request argument(s)
            var createAssessmentRequest = new CreateAssessmentRequest
            {
                ParentAsProjectName = ProjectName.FromProject(_reCaptchaOptions.ProjectId),
                Assessment = new Assessment()
                {
                    Event = new Google.Cloud.RecaptchaEnterprise.V1.Event()
                    {
                        SiteKey = _reCaptchaOptions.SiteKey,
                        Token = token
                    }
                },
            };

            try
            {
                // Make the request
                var response = await _client.CreateAssessmentAsync(createAssessmentRequest, cancellationToken);

                _logger.LogInformation($"ReCaptcha response {JsonConvert.SerializeObject(response)}");

                return response != null
                    && response.TokenProperties.Hostname.Equals(_host.Replace("https://", string.Empty))
                    && response.TokenProperties.Valid
                    && response.TokenProperties.Action.Equals(action)
                    && response.RiskAnalysis?.Score >= _reCaptchaOptions.Threshold;
            }
            catch (Exception e)
            {
                _logger.LogError(e, "ReCaptcha: Error getting response");
                return true;
            }
        }
    }

Note for future issues: it's a lot easier to read and work with code if it's copy/pasted as text rather than provided as a screenshot.
The code looks fine to me, and I'd expect this to just work.
A few questions:

I can't read the graph, in terms of either the key or the values... are you able to provide a less blurry version?
Do you experience any other network issues around the time that these calls fail?
Does the service recover after a while using the same client objects, or is it permanently broken once it's started failing?

I strongly suspect I'll need to get some of our gRPC folks involved (as the gRPC layer is one that I don't have detailed knowledge of) but the answers to those questions would be useful for them, I suspect.

Note for future issues: it's a lot easier to read and work with code if it's copy/pasted as text rather than provided as a screenshot.
The code looks fine to me, and I'd expect this to just work.
A few questions:

I can't read the graph, in terms of either the key or the values... are you able to provide a less blurry version?
Do you experience any other network issues around the time that these calls fail?
Does the service recover after a while using the same client objects, or is it permanently broken once it's started failing?

I strongly suspect I'll need to get some of our gRPC folks involved (as the gRPC layer is one that I don't have detailed knowledge of) but the answers to those questions would be useful for them, I suspect.

Hi @jskeet
sorry for the blurry images, I will update the screenshot with a clear one and put the code as it is rather than screenshot.

No, no other network issues were found during these tests, we were testing this functionality after getting the error on production, because as you said we expected the code to just work. So lets say its a isolated env to test this.
It goes up until no tcp is available and causes the app to crash

Lots more questions I'm afraid - please don't read any hostility into the level of questions here; there's just a lot I don't understand at the moment :)
Okay, now I can read the graphs, I'm afraid I still don't fully understand what they're saying. You said "you can see the difference between normal execution vs recatpcha gets enabled/working" - do you mean different instances are configured differently? Or that you enabled it at a different time?
Are these just graphs of established TCP connections (both directions? incoming? outgoing?)? Do you have graphs of error rates etc? The more we can understand what's going on, the better.
When you say no other network issues were found during these tests - is anything else trying to talk to any GCP services? (For example, if you try to make an API call to another service once a minute just as a sort of heartbeat, does that work?) Have you conducted the tests multiple times and seen similar results? If it works for a while and then starts failing, is that timing consistent?
I'm out of the office right now, but will contact a gRPC-based colleague later today to ask for assistance.

I will add another question, just to make sure.
When you call the ReCaptchaValidator.IsValid method, are you awaiting the call (await validator.IsValid(...)) or blocking (validator.IsValid(...)Result). If you are blocking, that could very well explain increased latency and possibly resource depletion.

Hi! fast answer @amanda-tarafa , we are not blocking it:
if (request.ReCaptchaEnabled && !await _reCaptchaValidator.IsValid(request.ReCaptchaToken, ReCaptchaActions.LOGIN, cancellationToken))

I will get more info @jskeet and answer you ;) (lot of work today on another areas! ahhaha)

Lots more questions I'm afraid - please don't read any hostility into the level of questions here; there's just a lot I don't understand at the moment :)
Okay, now I can read the graphs, I'm afraid I still don't fully understand what they're saying. You said "you can see the difference between normal execution vs recatpcha gets enabled/working" - do you mean different instances are configured differently? Or that you enabled it at a different time?
Are these just graphs of established TCP connections (both directions? incoming? outgoing?)? Do you have graphs of error rates etc? The more we can understand what's going on, the better.
When you say no other network issues were found during these tests - is anything else trying to talk to any GCP services? (For example, if you try to make an API call to another service once a minute just as a sort of heartbeat, does that work?) Have you conducted the tests multiple times and seen similar results? If it works for a while and then starts failing, is that timing consistent?
I'm out of the office right now, but will contact a gRPC-based colleague later today to ask for assistance.

No, I mean that the TCP connections go to high (at least for my point of view) when the code starts connecting to the google services
Let me ask my collegues for more info about the graphs as im on a very slow network and its imposible to send images.
No there is no other talk to any GCP (we are only using ReCaptcha, for now). Yes those tests were run multiple times with the sames results, we didnt check the timing.

Just coming back to this again - could you add logging for your ReCaptchaValidator constructor? I wonder whether some DI configuration issue means it's being created on every request rather than acting as the singleton I'd expect it to be. If you're creating a new RecaptchaEnterpriseServiceClient on every request (and with a specific JSON credential rather than the default credential, which allows channel reuse) then that could explain things.

hi @jskeet
find attached a screenshot with a sample text setted on the constructor (on a 250k calls, more or less, only 1 log was done):

@Rassell: Okay, that's good to know.
I'm currently lost as to why you'd end up with so many TCP connections then. I wonder whether the gRPC library is initializing more connections in order to meet demand... do you know what request rate you're achieving, roughly?
Also, you've talked about this happening on Azure - are you able to try reproducing it using local machines, potentially just issuing recaptcha requests? (I wonder whether Azure is rate limiting the outward bound connections somehow?) I'm not quite sure where to go next in diagnosing this, although I'll refer this question to a gRPC team member...

Hi @jskeet
we are on 478 more or less per second during our tests but with number of threads (users) 50 , using jmeter for our tests
we didnt test anything on local at the moment, we will take a look to double check

Hi all,
We are doing some tests and we have found that using "normal" httpclients the cpu usage and tcp are "stables":
using Google.Apis.Auth.OAuth2;
using Google.Apis.Util.Store;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;
using Newtonsoft.Json;
using Newtonsoft.Json.Serialization;
using System;
using System.IO;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Security.Cryptography.X509Certificates;
using System.Text;
using System.Threading;
using System.Threading.Tasks;

namespace Lidl.Sso.ReCaptcha
{

    public class ReCaptchaValidator : IReCaptchaValidator
    {
        private readonly ILogger<ReCaptchaValidator> _logger;
        private readonly ReCaptchaOptions _reCaptchaOptions;
        private readonly string _host;
        private readonly HttpClient _httpClient;

        public ReCaptchaValidator(IOptionsMonitor<ReCaptchaOptions> optionsMonitor, ILogger<ReCaptchaValidator> logger, IConfiguration configuration, HttpClient client)
        {
            _logger = logger;
            _reCaptchaOptions = optionsMonitor.CurrentValue;
            _host = configuration.GetValue<string>("Host");
            _httpClient = client;
        }

        public async Task<bool> IsValid(string token, string action, CancellationToken cancellationToken = default)
        {
            if (!_reCaptchaOptions.Enabled)
            {
                return true;
            }

            // Initialize request argument(s)
            var createAssessmentRequest = new Request
            {
                Event = new Event
                {
                    SiteKey = _reCaptchaOptions.SiteKey,
                    Token = token
                }
            };

            try
            {
                var stringContent = new StringContent(JsonConvert.SerializeObject(createAssessmentRequest), Encoding.UTF8, "application/json");

                // Make the request
                _httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", "");
                var response = await _httpClient.PostAsync($"https://recaptchaenterprise.googleapis.com/v1/projects/{_reCaptchaOptions.ProjectId}/assessments", stringContent, cancellationToken);

                if (!response.IsSuccessStatusCode)
                {
                    return false;
                }

                var json = await response.Content.ReadAsStringAsync();
                var result = JsonConvert.DeserializeObject<Response>(json);

                _logger.LogInformation($"ReCaptcha response {JsonConvert.SerializeObject(response)}");

                return result != null
                    && result.TokenProperties.Hostname.Equals(_host.Replace("https://", string.Empty))
                    && result.TokenProperties.Valid
                    && result.TokenProperties.Action.Equals(action)
                    && result.RiskAnalysis?.Score >= _reCaptchaOptions.Threshold;
            }
            catch (Exception e)
            {
                _logger.LogError(e, "ReCaptcha: Error getting response");
                return true;
            }
        }
    }

    class Event
    {
        [JsonProperty("token")]
        public string Token { get; set; }

        [JsonProperty("siteKey")]
        public string SiteKey { get; set; }
    }

    class Request
    {
        [JsonProperty("event")]
        public Event Event { get; set; }
    }

    class Response
    {
        public string Name { get; set; }

        public Event Event { get; set; }

        public TokenProperties TokenProperties { get; set; }

        public Google.Cloud.RecaptchaEnterprise.V1.RiskAnalysis RiskAnalysis { get; set; }
    }

    public class TokenProperties
    {
        public bool Valid { get; set; }
        public string InvalidReason { get; set; }
        public string Hostname { get; set; }
        public string Action { get; set; }
    }
}

services.AddHttpClient<IReCaptchaValidator, ReCaptchaValidator>();
services.AddTransient<IReCaptchaValidator, ReCaptchaValidator>();

the only issue with this is the auth_token that we are not able to generate this, only with powershell with the gcloud console, is there any way to generate this token?

You can certainly use GoogleCredential to generate auth tokens (and refresh them automatically). You'll need a dependency on Google.Apis.Auth (which you'll already have transitively if you've still got Google.Cloud.RecaptchaEnterprise.V1 installed.
// In the constructor (assigning to a field)
ICredential credential = GoogleCredential.FromJson(_reCaptchaOptions.CredentialsJson);

// On each request:
string accessToken = await credential.GetAccessTokenForRequestAsync();

However, doing the serialization yourself like this has significant risks - the generated protobuf classes can sometimes have property names which may not deserialize directly using Json.NET. It would be somewhat better to just use the protobuf classes directly, e.g.:
var request = new Assessment
{
    Event = new Event { ... }
};
var stringContent = new StringContent(request.ToString(),  Encoding.UTF8, "application/json");
...
var result = Assessment.Parser.ParsJson(json);

(You can then get rid of your TokenProperties, Event, Request and Response classes.)
That still assumes the JSON used by the service exactly matches the proto3 representation. That's probably true - and it's a better approach than using Json.NET with the protobuf classes - but using gRPC would definitely be better. I'm hoping we can resolve why there's such a high usage level here.
Have you had any luck reproducing this outside Azure, ideally with just a console app firing requests?

Hi @jskeet
Thanks for the fast answers, about the
// In the constructor (assigning to a field)
ICredential credential = GoogleCredential.FromJson(_reCaptchaOptions.CredentialsJson);

// On each request:
string accessToken = await credential.GetAccessTokenForRequestAsync();

we are getting:
Error [string]:"invalid_scope"
ErrorDescription [string]:"Invalid OAuth scope or ID token audience provided."

I have also tried to put "cloud-platform" on the GetAccessTokenForRequestAsyncbut but got a 401
Also thanks for pointing the issue with the json.
I will try to reproduce this on local asap

Ah, okay. Fortunately that's easy to fix. I'd hoped it would just work with a self-signed JWT, but I'm hopeful that's easy to fix:
ICredential credential = GoogleCredential
    .FromJson(_reCaptchaOptions.CredentialsJson)
    .CreateScoped(RecaptchaEnterpriseServiceClient.DefaultScopes);

Ah, okay. Fortunately that's easy to fix. I'd hoped it would just work with a self-signed JWT, but I'm hopeful that's easy to fix:
ICredential credential = GoogleCredential
    .FromJson(_reCaptchaOptions.CredentialsJson)
    .CreateScoped(RecaptchaEnterpriseServiceClient.DefaultScopes);

working! thanks!

Great! I'll keep this issue open though, as I still want to understand what's leading to the high TCP count. Hopefully this has at least unblocked you for now though.

@jskeet hi! we tested "localy" (only library, one endpoint on a rest api calling on the reCaptcha nuget) and see that the problem reproduces, with the same machine etc.

Okay, thanks. That's good to know. Is that still with your full web app, or have you reproduced it in a minimal way I'd be able to test here? Not to worry if not - just thought I'd ask. Still hoping to get the help of the gRPC folks here.

Hi @Rassell: are you able to retest this? I believe the problem has been resolved, probably due to a network reconfiguration somewhere. It would be good to hear that you're seeing the same change though.

Closing now as I believe it's fine, and we haven't heard any more information for 12 days.


Support using V1 tasks within V2 WorkflowTask

(ref https://github.com/fractal-analytics-platform/fractal-server/issues/1328)

This is not yet fully tested in the backend internal runner component, but the API should be mostly stable. Here is a first plan on how this could look in fractal-web.
In the V2 Workflow page, we should offer more options in the "add a task" modal.

By default, the user can pick a task from the V2 list (the current feature). V2 tasks can be compound, parallel or non-parallel (which then determines which parts of the arguments form should be available.
This choice, as we already had in V1, is split into common and users tasks.
By actively clicking on a switch, the modal should use the V1 task list (the one coming from GET /api/v1/task/) rather than the V2.
The split into common and users remains.
When adding a V1 task to a V2 workflow, the POST request body should include "is_legacy_task": true, see endpoint docs below.
When a V1 task is included in a V2 workflow, it is mostly treated as if a V2 task was included (note: it will always have WorkflowTask.task_type="parallel"). A couple of relevant differences:

The excluded arguments are the V1 ones (ref https://github.com/fractal-analytics-platform/fractal-web/issues/429).
It'd be useful to display somewhere in the task list that this task is legacy. I don't have any specific ideas.
As a first version, I think we should not offer version updates for V1 tasks which are run from a V2 workflow. I haven't checked yet whether this would be easy or complex.

(side note: https://fractal-analytics-platform.github.io/fractal-server/openapi is now up-to-date)

Great list. One addition:

the modal should use the V1 task list (the one coming from GET /api/v1/task/) rather than the V2.

We have a flag that is set by admins on whether a V1 task is V2 compatible, right? We should only show the ones that have this flag set to True.

We have a flag that is set by admins on whether a V1 task is V2 compatible, right? We should only show the ones that have this flag set to True.

That's correct, thanks. We should filter the V1 task list and only include tasks with is_v2_compatible=true.
The V1 Task objects have a boolean property is_v2_compatible. This is always false, by default, and the only way to set it to true is that an admin should make a PATCH request to /admin/v2/task-v1/{task_id}/ (this will be available in the V2 admin area - ref #438).

Note that the relevant backend API was partly redefined in fractal-server 2.0.0a5, see

https://github.com/fractal-analytics-platform/fractal-server/issues/1381
https://github.com/fractal-analytics-platform/fractal-server/pull/1379/files

Now the relevant endpoint to get the list of V2-compatible V1 tasks is
GET /api/v2/task-legacy/?only_v2_compatible=true&args_schema=false


Sprint board proposals for sprint 4/9 - 4/20

Please comment on this issue with proposals for our next two week sprint on:
Your capacity for this sprint, which begins Monday, 4/9 and goes through Friday, 4/20.
Your proposals for what work items (links to issue numbers) you'd like to work on for this sprint.
Please have your comments in by the end of your day today so we can review in our sprint review meeting. Our sprint review/planning meeting is at 8am PDT/4pm CET on Monday, 4/9.

I am out of the office on vacation 4/12 - 4/16 and will miss the weekly call on 4/16.
I have been working on curation topics and plan to continue down that path the next couple weeks. @jeffmcaffer and I have had some productive conversations with potential partners as well and may be talking more detailed implementation with them, we'll see.


[FR] Encode QR data more efficiently

Please verify that this feature request has NOT been suggested before.

[X] I checked and didn't find a similar feature request

Problem statement
Current QR codes encode lots of unnecessary data, and even in binary format (which requires much more bits per char) which causes higher version QR codes more modules => larger grid => cannot be printed that small. This especially happens with stock location QR codes because they encode the most of unnecessary data, and even an id with only one char already requires a v2 QR code

{"part": 9} (11 chars) => v1
{"part": 99999} (15 chars) => v2
{"stocklocation": 9} (20 chars) => v2
{"stocklocation": 99999999} (27 chars) => v3

QR version overview

Source
Comparison

Part
Location
Stockitem
Buildline

QR code data (base coast/extra bytes)
{"part": ID} (10)
{"stocklocation": ID} (19)
{"stockitem": ID} (15)
{"build": ID} (11)

V1 21x21(max: 14)
4
-
-
3

V2 25x25(max: 26)
16
7
11
15

V3 29x29(max: 42)
32
23
27
31

The cells show the max digit count of which an ID can consist to work with that version of QR code
Suggested solution
I suggest to reduce the size of unnecessary data and either add the ability to increase the ECL level from M (15%) to something like Q (25%) or even H (30%) while still staying with a V1 QR code for all types currently available. And to still keep backwards compatibility I suggest we add a toggle to the settings to decide which type to use. And for scanning systems (web UI, app, ...), we could easily support to read both types.
For encoding the data a bit more sparingly I have two ideas:

Still use the RAW (binary) QR type and just prepend one char. E.g. P9999 (part)
Since our ids are only numbers, we could easily use the numeric encoding and prepend one or even two digits (to be save for the future) to indicate the QR type

Idea 1
Idea 2

Data
P9999
019999

v1 L (7%)
16
39

v1 M (15%)
13
32

v1 Q (25%)
10
25

v1 H (30%)
6
15

The cells show the max digit count of which an ID can consist to work with that version of QR code
Describe alternatives you've considered
/
Examples of other systems
I'm not sure how other systems efficiently encode their data in QR codes, but I'm sure we're not the first ones. Maybe there is even a efficient standard we could use?
Do you want to develop this?

[ ] I want to develop this.

@wolflu05 I'm on board, it would be a good idea to improve our QR system for better printing.
Preserving backwards compatibility is obviously necessary, but mostly on the "scanning" side.

I suggest we add a toggle to the settings to decide which type to use.

Good idea, we could default to the new style, and allow users to preserve old style, potentially
I would suggest that we make the new codes a bit more unique, though. Perhaps an INV- prefix, or something? Having a simpler alphanumeric code as you suggest increases the risk of collision with other third-party barcodes

I seem to remember that @matmair has tried to standardize things across inventory software. Maybe you know something/have some ideas in this area too, on how qrcodes can be standardized across inventory software while trying to fit the required data into a v1 QR code with abilities to increase the error correction level.

The standard is to use EAN or G1 codes but that will take more space than your suggestion. This could be combined with #5208 / a generic reference model to make this useable for plugins too

But aren't ean/gtin numbers just 13 digits so they would fit in a v1 qr?

Sorry I was unclear; typeIds would need v2, they are at least 28 chars

They even would need v3 then, because 15% error correction would be minimum in my opinion (max 26chars) . Higher ECL would be even better. So when we do it, we should make typeids optional as a third encoding type next to the current behavior and something short for small qrs (this issue).

I still need to find a better solution for this. It's definitely on my roadmap.

I'd like to push this forward now. I understand that only numbers are not the right choice here as they could get easily mixed with normal GTIN Codes or other codes used by manufactures. Actually I like the idea from Oliver to just prefix it with INV-.
So I suggest the following scheme: INV-??x

INV- is a constant prefix
?? is a two character alphanumeric (0-9A-Z  $%*+-./: (45 chars)) prefix, individual to each model. That means in theory we could support 45**2=2025 different models. (part => PA, location => LO, stock item => SI, build => BL)
x the actual pk of the model. This means that we could support the following amount of model instances with the following modes:

v1 M ECL (15%) => 10**14-1 items (create ~1.902.587 items per minute for 100 years, until we need V2)
v1 Q ECL (25%) => 10**14-1 items (create ~190 items per minute for 100 years, until we need V2)

I'd like to implement this behind a server setting, which still defaults to the old format, but can be changed to the new. What is your opinion on this format?

Another interesting idea would be to also integrate qrcode generation to the plugin interface with the barcode mixin, so that users could provide their own format. But I think that would be a different issue.

ok, I found out, there is something called mixed mode for QR codes. Using mixed mode, data can be encoded using multiple formats in one QR code. That means that the first part can be encoded using the alphanumeric scheme and the second part using the numeric scheme. That results in the following updated stats, which are quite acceptable:

v1 M ECL (15%) => 10**20 (old: 14) items (create ~3.170.979.198 items per second for 1000 years, until we need V2)
v1 Q ECL (25%) => 10**13 (old: 10) items (create ~317 items per second for 1000 years, until we need V2)
v1 H ECL (30%) => 10**3 (old: 4 got even worse) items (create ~100 items per day for 10 days, until we need V2)