More efficiently reading the socket from python

While debugging some performance problems (at around 4000 notifications per second, my rpi could not keep up and the socket buffer would fill up causing latency), I made some changes to the python code to read more efficiently from the socket. The idea is to call recv to read more than one packet at a time, which should reduce the number of syscalls (not verified) and thus be more efficient. In practice, this didn't work well enough to solve my problem completely, but I wanted to share the change anyway.
I'm probably switching to a different setup, so I won't be needing this code anymore, but I wanted to share it anyway in case it's useful (but I don't want to spend the time on a proper pullrequest, so here's the patch inline):
diff --git a/pigpio.py b/pigpio.py
index df01c0d..371115f 100644
--- a/pigpio.py
+++ b/pigpio.py
@@ -1110,17 +1110,19 @@ class _callback_thread(threading.Thread):

       lastLevel = self.lastLevel

+      RECV_SIZ = 4096
       MSG_SIZ = 12

+      buf = bytes()
       while self.go:

-         buf = self.sl.s.recv(MSG_SIZ)
+         buf += self.sl.s.recv(RECV_SIZ)
+         offset = 0

-         while self.go and len(buf) < MSG_SIZ:
-            buf += self.sl.s.recv(MSG_SIZ-len(buf))
-
-         if self.go:
-            seq, flags, tick, level = (struct.unpack('HHII', buf))
+         while self.go and len(buf) - offset > MSG_SIZ:
+            msgbuf = buf[offset:offset + MSG_SIZ]
+            offset += MSG_SIZ
+            seq, flags, tick, level = (struct.unpack('HHII', msgbuf))

             if flags == 0:
                changed = level ^ lastLevel
@@ -1143,6 +1145,7 @@ class _callback_thread(threading.Thread):
                   for cb in self.events:
                      if cb.event == event:
                         cb.func(event, tick)
+         buf = buf[offset:]

       self.sl.s.close()

Thanks for that.
I batch reads in piscope, pigpiod_if.c, and pigpiod_if2.c but may have assumed it wouldn't make a significant difference in Python.
Did you get a measure of the performance improvement?

Did you get a measure of the performance improvement?

It seemed a little better, but I haven't done thorough measurements (with our without this patch, python was too slow to handle 4k events per second, I haven't tried throttling the events to find the turnover point). Since I'll be using a different setup, I won't be able to invest more time in investigating this, so feel free to measure yourself, or ignore my patch if that's better :-)

I have some tests on a original PiB and a Pi3.
There does seem to be a good performance increase for Python running on the Pi but no benefit for Python running over the network (i.e. script running on a remote PC talking to a Pi).
The following figures are the events per second handled (an event in this case is a callback call).
              unpatched patched improvement
Pi B (local)  10k       15k     50%
Pi 3 (local)  67k       80k     19%

Pi B (remote) 89k       89k     0%
Pi 3 (remote) 81k       81k     0%

I'm currently working on C#/.Net bindings for pigpiod and while integrating this batch-reading optimization, I discovered a bug in the diff above.
+  while self.go and len(buf) - offset > MSG_SIZ:
                                      ^^^^

len(buf) should be greater than or equal to MSG_SIZ otherwise the last/only message in the buffer is ignored.

Good point, that should indeed be >=.

This has been added to V65.


busybox: add /etc/services

Otherwise, resolving addresses that include named ports (e.g.
code.i3wm.org:git instead of code.i3wm.org:9418) won’t work.

Hmm, something seems wrong with the circle ci config.
Otherwise :+1:

I will take a look at the error.

Ok the problem is a that circleci is checking out this PR as branch name pull/3. When building the docker container for a non-master branch, the branch name is included in the tag where / are not allowed.

Problem fixed, @stapelberg can you rebase against master ?

Done, and thanks for the quick fix!

ok LGTM, thanks.


Bad advancement

[16Nov2021 21:11:45.502] [Worker-Main-14/ERROR] [net.minecraft.advancements.AdvancementList/]: Couldn't load advancement storage4computercraft:me_peripheral: Task Advancement{parentId=appliedenergistics2:main/interface, display=net.minecraft.advancements.DisplayInfo@52420221, rewards=AdvancementRewards{experience=0, loot=[], recipes=[], function=net.minecraft.command.FunctionObject$CacheableFunction@dffbd18}, criteria={peripheral_in_inventory=net.minecraft.advancements.Criterion@294407d9}, requirements=[[peripheral_in_inventory]]}

AE not installed on server, only RS. Using https://www.curseforge.com/minecraft/modpacks/hexworks/relations/dependencies as the pack.

You're getting this error precisely because AE is not installed.  It should be safe to ignore, though.
I'll investigate further but I'm not sure it's possible to "hide" this advancement when the mod is not installed.


Permit adding arbitrary (virtual) interfaces (created by third-party software) to be added to the configuration

Important notices
Before you add a new report, we ask you kindly to acknowledge the following:

[*] I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
[*] I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/core/issues?q=is%3Aissue

Is your feature request related to a problem? Please describe.
I would like adding a generic tunneling solution to OPNSense routers and have their (synthetic) interfaces show up like any other "supported" interface for assignment.
Describe the solution you like
Add a sub-menu below "Interfaces -> other types" where it is possible to add random interface names that are outside OPNSense's control to be brought into the configuration so they can be used for assignment.
Describe alternatives you considered
Just renaming the interface to something known (like gre or such) is obviously insufficient.
Creating an OpenVPN server that is not started and abuse its interface name. Would require changes to the code, too.
Putting it by hand into /conf/config.xml but this will probably break sooner or later and should not be considered a very bright idea.
Additional context
I would like to add third-party VPN mesh software to the firewall itself instead of having a collection of single-purpose Raspberries to maintain in addition to the designated router itself. It just does not make sense. At the same time I don't believe in asking the project to add the latest development in VPN technology. Nevertheless adding Nebula to our clients' setups is definitely the better choice over anything included in OPNSense right now.
Adding Nebula (at least as a mesh node) to the OPNSense supported VPN list would be the best idea but that will probably bind resources needed somewhere else.

Start the VPN or service which creates the device, go to assignments and link the interface. Don’t forget to lock the interface. I don’t expect anything else is needed.

It might have issues with the device name presented, but a custom plugin file will allow the devices to be assignable.

I guess I will have to file a different bug or feature request now...
I already didn't understand the requirement for VLAN names to contain a digit.
It did not work for a simple reason: I called the interface "nebula". After renaming it to "nebula3.14159" it showed up in assignments. Isn't it time to drop the requirement for interface names ending in digits?

After renaming it to "nebula3.14159" it showed up in assignments. Isn't it time to drop the requirement for interface names ending in digits?

probably not, a unique name isn’t very scalable, usually the same type can be created multiple times.

What is the recommended minimum for integration? An rc.d for starting and stopping or adding it to configd?

best read the development documentation at https://docs.opnsense.org first and checkout similar plugins in our repo.

There is a number if VPN plugins to see what is necessary for implementation. About interface names I’m not 100% sure, but in general only known ones are missing (including tap/tun) but all unknown ones are passed through to the GUI for assignment.

After renaming it to "nebula3.14159" it showed up in assignments. Isn't it time to drop the requirement for interface names ending in digits?

probably not, a unique name isn’t very scalable, usually the same type can be created multiple times.

If you are auto-generating the names from OPNSense scripts that might be the case. If third party software is creating interfaces it should be considered "other peoples' problem"...
In the case of overlay meshes you usually have only one of them (there are a few special cases where you want to isolate software that is doing a lot of broadcasting/multicasting in its own mesh or where you want to interconnect meshes under separate management) but usually you get only
Nebula is a bit worse in that regard: You put a fixed(!) interface name into the configuration file; if it is colliding with an existing interface all hell breaks loose. So you will add a separate name for every single config file you create and keep track of them anyway. Things are getting worse if other programs are creating interface names the same way.

There is a number if VPN plugins to see what is necessary for implementation.

That's the special case where you are generating interfaces by software.

About interface names I’m not 100% sure, but in general only known ones are missing (including tap/tun) but all unknown ones are passed through to the GUI for assignment.

I remember defining VLANs a few weeks ago and having to name them with trailing numbers even though "office" "servers" and "printers" would have done just as easily. There is a reason why FreeBSD added the ability to rename interfaces after the optional description was used  to simulate this feature (see OPNSense itself...).
The interface numbering thing is a thing of early history and has been kept since BSD3 (you could configure global driver features that way as front end to obscure ioctl()s). Linux showed us that calling everything net(number) did not improve things either -- just see the interface renaming taking place these days (and I don't know what's worse -- having the MAC address that might change later as part of the interface name or the path in the driver tree).

Most of this is off-topic. If you want to use arbitrary content for device names fine, but 15 characters limit and identifiable names that are unique at the same time are rather difficult to pull off if you think about it.

Succeeding opened so many more questions that I will move over to the forum with it.

Request doesn't make sense like this.


Don't show game join menu item when game mode is ladder

Kindof hard to test on production since it's a very rare edge case. Hopefully the tests I provided are sufficient.
Fixes #1761
Fixes #936

need to test it still

It’s actually not too hard to test on the test server. I did it a few times. Need 3 clients though, 2 to search ladder and one to have added the other two as a friend.

So u tested it?

Yes

Do we need to exclude tutorials as well?

No, tutorials aren't even registered on the server, they aren't real games.

tested

So can we merge?

Yes sure just forgot it


Add Awesome Git list to Development Environment section

Development Environment section has lots of git-related stuff, but I don't see awesome-git on there:
https://github.com/dictcp/awesome-git

The contribution guidelines are clearly documented.


Codegen (Block) Nested Loop Joins

This PR adds block nest-loop joins to the codegen engine. This brings us one step closer to achieving feature parity with the interpreted engine (and one step closer to phasing it out).
Initially this PR had sort-merge joins and union/intersect/extract too, but with all the required infrastructure changes the PR was too complex. I'll submit those once this goes through.
In this PR:

The main BNLJ logic.
Ability to define auxiliary producer functions aside from the "main" plan function. This will be needed for outer joins later on.
Separate a function's declaration from definition so that the function can be used before it is defined (i.e., implemented).
Ability to reset parameter cache so that it can be reused across multiple functions.
Optimize parameter cache to only generate null checks if parameter is nullable.
Optimize zone map scanning logic.
Removed all local (i.e., on-stack) from runtime state. Now local variables can be safely defined anywhere during code generation. This makes it much easier for beginners since they don't have to worry about whether allocation happens deeply in a loop.
Enabled NLJ plans to participate in parameterization and caching.
Random formatting.

Tests for all of the above are included.

Coverage increased (+0.2%) to 75.938% when pulling cc2d503029d8d287fb5c395727c652b1662b51af on pmenon:bnlj into<PHONE_NUMBER>0cf539838c20bfd48952d677e8cf12 on cmu-db:master.

Coverage increased (+0.2%) to 75.938% when pulling cc2d503029d8d287fb5c395727c652b1662b51af on pmenon:bnlj into<PHONE_NUMBER>0cf539838c20bfd48952d677e8cf12 on cmu-db:master.

@pmenon How easy will it be to modify this to use indexes once #946 is merged?

@apavlo I think the index scan, if implemented correctly, should be orthogonal to this.

Coverage increased (+0.2%) to 75.938% when pulling f53662f7fc5934d6296de1047c2e1f0b4d06ad61 on pmenon:bnlj into<PHONE_NUMBER>0cf539838c20bfd48952d677e8cf12 on cmu-db:master.

@apavlo The short answer is that modifying BNLJ to support indexes is easy, but we probably shouldn't do it this way. This BNLJ materializes tuples that we can avoid when doing index nested-loop joins.

Coverage increased (+0.2%) to 75.928% when pulling 18dd2e76bc7e42663e79709154d85a3c2754bf33 on pmenon:bnlj into<PHONE_NUMBER>0cf539838c20bfd48952d677e8cf12 on cmu-db:master.

Coverage increased (+0.2%) to 75.921% when pulling 18dd2e76bc7e42663e79709154d85a3c2754bf33 on pmenon:bnlj into<PHONE_NUMBER>0cf539838c20bfd48952d677e8cf12 on cmu-db:master.

I encountered the same error when doing #1016. This is a failure that happens randomly.
The reason is that you allocate a buffer type::Value[N] on the stack without initializing it. Therefore, when calling functions in https://github.com/cmu-db/peloton/blob/master/src/codegen/values_runtime.cpp, you are effectively destructing objects that don't exist.
For example, if you do vals[idx] = type::ValueFactory::GetTinyIntValue(val);, then you are calling operator= on vals[idx], which will first free the resources in vals[idx]. But vals[idx] is garbage memory before assignment.
The solution is to directly call the constructor on the buffer, like this:
new (&vals[idx]) type::Value(type::ValueFactory::GetTinyIntValue(val));

@phisiart Thanks for the catch.
@apavlo I think there is another more subtle bug that this PR exposes. On our dev machines, master fails a few tests with LLVM 3.9, but passes with 3.7. I spent a little time and tracked it down to how we're doing vectorized scans. @seojungmin Can you take a look at this?

Coverage increased (+0.2%) to 75.854% when pulling 36c1c372314d624216cb55fda68453515ddc95f5 on pmenon:bnlj into 0e7f6460582d0a40dc9d34f8d0d2fe7b8a6b2373 on cmu-db:master.

@pmenon I just read this. I was busy clearing the snow on my door way haha :-) I hope you don't have to do this there.  I think this is not related to this PR.  The current branch also crashes at  ScanWithSubtractColumnsPredicate.  I will keep it posted when I find more.

@seojungmin It isn't related to this PR. Master seems to be broken on dev4/5 for the last few weeks. If I remember correctly, it fails predictably only on ScanWithSubtractColumnsPredicate and ScanWithAddColumnsPredicate and only only LLVM 3.9 (4.0 works).


[LAB] Week 1_Problem 2_채이한

Week
Week 1_Problem 2
Source Code
import torch
from random import random
from typing import Callable

##                        Problem 2                           ##
##                                                            ##
##           Arbitrary quartic function will be given.        ##
## Return the optimal point(global minimum) of given function ##
##          Condition: highest order term is positive         ##
##                  Made by @jangyoujin0917                   ##
##                                                            ##
def grad(func,point):
    h=0.0001
    return (func(point+h)-func(point-h))/(2*h)

def solution(func: Callable, start_point: float): # DO NOT MODIFY FUNCTION NAME
    v=0
    point=start_point
    beta=0.7
    alpha=0.001
    #print(type(point),type(v))
    for i in range(100):
          temp_v=v
          v=temp_v*beta-grad(func,point)*alpha
          #print(point,v,grad(func,point))
          point+=v
    return point

if __name__ == '__main__':
    def test_func(x): # function for testing;function for evaluation will be different.
        return x**4+3*x**3-2*x
    t = 10*random()-5
    print(solution(test_func, t))

Description
.
Output (Optional)
No response

RuntimeError: a leaf Variable that requires grad is being used in an in-place operation.

On re-scoring, this code do not get error, but it gets 0/100.
How about adjusting alpha, beta, and # of iteration?

I adjusted tolerance, this code got 52/100.
I think lots of testcase stuck in local minima.

This issue is closed now because of the lack of progress.


The replay tool initialises libstdc++ in single thread mode on Linux

The Vulkan loader or any ICD loaded using dlopen() won't have a working std::thread as a result.

This is a simple example demonstrating the issue: threads.tar.gz
It dlopen()s a multi-threaded library from a single-threaded process. An attempt to start a thread causes a segfault in std::thread constructor.


istioctl produces unhelpful error message when using a incorrectly formatted file

E.g.,
istioctl create -f guestbook-service-retry.yaml
Error: the server rejected our request for an unknown reason

However, when you turn on verbose mode, you can see the server side error:
...
I0619 17:33:00.227081   92174 request.go:991] Response Body: cannot parse proto message: unknown field "uri" in istio_proxy_v1_config.MatchCondition

move to 0.3 as #1140 is there.

Too old, no activity.


[terra-button] Create selectable button

Summary

What was changed:
Add private prop isSelectable to allow terra-button to function as a toggleable button.
Why it was changed:
These changes were requested by Mpages team to have a sub-variant of terra-button which could be toggleable.
Testing

This change was tested using:

[ ] WDIO
[ ] Jest
[x] Visual testing (please attach a screenshot or recording)
[ ] Other (please describe below)
[x] No tests are needed

Reviews
In addition to engineering reviews, this PR needs:

[x] UX review
[x] Accessibility review
[x] Functional review

Additional Details

This PR resolves:
UXPLATFORM-9749

Thank you for contributing to Terra.
@cerner/terra
Unselected

Selected

Fails
🚫	Please include a CHANGELOG entry for each changed package this PR. Looks like a CHANGELOG entry is missing for:

terra-core-docs

Generated by 🚫 dangerJS against 54c4c36

Have retained test example for final validation. Will be removed after approval

Unlisted props seem to cause lint err in ComponenetDidMount as well


Azure problem

Azure is slow.. or is it?
Runnning this code
from tts_wrapper import MicrosoftTTS, MicrosoftClient
import time
import os
from load_credentials import load_credentials
import logging

# Load credentials
load_credentials("credentials.json")
client = MicrosoftClient(
    credentials=(os.getenv("MICROSOFT_TOKEN"), os.getenv("MICROSOFT_REGION"))
)
tts = MicrosoftTTS(client)

# Set up logging
logging.basicConfig(level=logging.DEBUG)

# Global variable to capture first word time
first_word_time = None

# Define a callback for when speech starts
def on_start():
    global first_word_time
    first_word_time = time.time()
    logging.info(f"First word spoken after {first_word_time - start_time:.2f} seconds")

# Connect the on_start event to the speak_streamed method
tts.connect("onStart", on_start)

# Set TTS properties
tts.set_property("volume", "100")
tts.ssml.clear_ssml()
tts.set_property("rate", "medium")

# Prepare SSML text
text_read = f"Hello, this is a streamed test"
text_with_prosody = tts.construct_prosody_tag(text_read)
ssml_text = tts.ssml.add(text_with_prosody)
print("ssml_test: ", ssml_text)

# Dictionary to store method timings
method_timings = {}

# Measure speak_streamed time
start_time = time.time()
tts.speak_streamed(ssml_text)
end_time = time.time()
speakstream_time = end_time - start_time
logging.info(f"speak_streamed method took {speakstream_time:.2f} seconds")
method_timings["speak_streamed"] = speakstream_time

# Calculate first word spoken time
if first_word_time is not None:
    first_word_spoken_time = first_word_time - start_time
    method_timings["first_word_spoken"] = first_word_spoken_time
    logging.info(f"First word spoken after {first_word_spoken_time:.2f} seconds")

# Measure synth_to_file time
start_time = time.time()
tts.synth_to_file(ssml_text, "test-microsoft.mp3", "mp3")
end_time = time.time()
synthfile_time = end_time - start_time
logging.info(f"synth_to_file method took {synthfile_time:.2f} seconds")
method_timings["synth_to_file"] = synthfile_time

# Measure synth_to_bytestream time
start_time = time.time()
bytestream = tts.synth_to_bytestream(ssml_text)
with open("output_testazure.mp3", "wb") as f:
    f.write(bytestream.read())
end_time = time.time()
bytestream_time = end_time - start_time
logging.info(f"synth_to_bytestream method took {bytestream_time:.2f} seconds")
method_timings["synth_to_bytestream"] = bytestream_time

# Measure speak time
start_time = time.time()
tts.speak(ssml_text)
end_time = time.time()
speak_time = end_time - start_time
logging.info(f"speak method took {speak_time:.2f} seconds")
method_timings["speak"] = speak_time

# Find the fastest method
fastest_method = min(method_timings, key=method_timings.get)
fastest_time = method_timings[fastest_method]

# Pretty-print the results
print("\nMethod Timing Results:")
for method, timing in method_timings.items():
    print(f"{method}: {timing:.2f} seconds")

print(
    f"\nThe fastest method was '{fastest_method}' with a time of {fastest_time:.2f} seconds."
)

if "first_word_spoken" in method_timings:
    print(
        f"\nThe first word was spoken in the speak_streamed method after {first_word_spoken_time:.2f} seconds."
    )

Run it with
poetry run python -m pyinstrument basic_azure_test.py
MicrosoftClient initialized with region: uksouth
ssml_test:  <speak version="1.0" xml:lang="en-US" xmlns="https://www.w3.org/2001/10/synthesis" xmlns:mstts="https://www.w3.org/2001/mstts"><voice name="en-US-JennyNeural"><prosody rate="medium" volume="100">Hello, this is a streamed test</prosody></voice></speak>
DEBUG:root:Word: Hello, Start: 0.050s, Duration: 0.525s
DEBUG:root:Word: ,, Start: 0.675s, Duration: 0.100s
DEBUG:root:Word: this, Start: 0.775s, Duration: 0.188s
DEBUG:root:Word: is, Start: 0.975s, Duration: 0.100s
DEBUG:root:Word: a, Start: 1.087s, Duration: 0.062s
DEBUG:root:Word: streamed, Start: 1.163s, Duration: 0.362s
DEBUG:root:Word: test, Start: 1.538s, Duration: 0.588s
INFO:root:Captured 7 word timings
INFO:root:First word spoken after 0.57 seconds
INFO:root:speak_streamed method took 4.11 seconds
INFO:root:First word spoken after 0.57 seconds
Info: on_underlying_io_bytes_received: Close frame received
Info: on_underlying_io_bytes_received: closing underlying io.
Info: on_underlying_io_close_complete: uws_state: 6.

Info: on_underlying_io_close_complete: uws_state: 6.

The first word was spoken in the speak_streamed method after 0.57 seconds.

  _     ._
Info: on_underlying_io_close_complete: uws_state: 6.

NB: So whats longer? I dont think there is much in this?

What is interesting is that the bytestream method calls synth_to_bytes (ie. not streaming). And its faster..

if we change the text to a longer stream we get
text_read = """In the age of digital transformation, technologies like artificial intelligence and machine learning
are reshaping industries at a rapid pace. The automation of processes, once considered science fiction, is now a
reality that businesses across sectors are embracing. From healthcare to finance, the impact of AI is profound,
offering solutions that improve efficiency and decision-making. However, as with any powerful technology, AI also
comes with its own set of challenges, particularly in areas of ethics, privacy, and employment. It is crucial that
we navigate these challenges thoughtfully to ensure a balanced and fair future."""

speak_streamed: 50.06 seconds
first_word_spoken: 1.00 seconds
synth_to_file: 0.92 seconds
synth_to_bytestream: 0.91 seconds
speak: 0.63 seconds

The fastest method was 'speak' with a time of 0.63 seconds.

Ah! Its the overhead in setting up the first call!
So if we run this the other way round
``python
from tts_wrapper import MicrosoftTTS, MicrosoftClient
import time
import os
from load_credentials import load_credentials
import logging
Load credentials
load_credentials("credentials.json")
client = MicrosoftClient(
credentials=(os.getenv("MICROSOFT_TOKEN"), os.getenv("MICROSOFT_REGION"))
)
tts = MicrosoftTTS(client)
Set up logging
logging.basicConfig(level=logging.DEBUG)
Global variable to capture first word time
first_word_time = None
Define a callback for when speech starts
def on_start():
global first_word_time
first_word_time = time.time()
logging.info(f"First word spoken after {first_word_time - start_time:.2f} seconds")
Connect the on_start event to the speak_streamed method
tts.connect("onStart", on_start)
Set TTS properties
tts.set_property("volume", "100")
tts.ssml.clear_ssml()
tts.set_property("rate", "medium")
Prepare SSML text
text_read = f"Hello, this is a streamed test"
text_read = """In the age of digital transformation, technologies like artificial intelligence and machine learning
are reshaping industries at a rapid pace. The automation of processes, once considered science fiction, is now a
reality that businesses across sectors are embracing. From healthcare to finance, the impact of AI is profound,
offering solutions that improve efficiency and decision-making. However, as with any powerful technology, AI also
comes with its own set of challenges, particularly in areas of ethics, privacy, and employment. It is crucial that
we navigate these challenges thoughtfully to ensure a balanced and fair future."""
text_with_prosody = tts.construct_prosody_tag(text_read)
ssml_text = tts.ssml.add(text_with_prosody)
print("ssml_test: ", ssml_text)
Dictionary to store method timings
method_timings = {}
Measure synth_to_file time
start_time = time.time()
tts.synth_to_file(ssml_text, "test-microsoft.mp3", "mp3")
end_time = time.time()
synthfile_time = end_time - start_time
logging.info(f"synth_to_file method took {synthfile_time:.2f} seconds")
method_timings["synth_to_file"] = synthfile_time
Measure synth_to_bytestream time
start_time = time.time()
bytestream = tts.synth_to_bytestream(ssml_text)
with open("output_testazure.mp3", "wb") as f:
f.write(bytestream.read())
end_time = time.time()
bytestream_time = end_time - start_time
logging.info(f"synth_to_bytestream method took {bytestream_time:.2f} seconds")
method_timings["synth_to_bytestream"] = bytestream_time
Measure speak time
start_time = time.time()
tts.speak(ssml_text)
end_time = time.time()
speak_time = end_time - start_time
logging.info(f"speak method took {speak_time:.2f} seconds")
method_timings["speak"] = speak_time
start_time = time.time()
tts.speak_streamed(ssml_text)
end_time = time.time()
speakstream_time = end_time - start_time
logging.info(f"speak_streamed method took {speakstream_time:.2f} seconds")
method_timings["speak_streamed"] = speakstream_time
Measure speak_streamed time
Calculate first word spoken time
if first_word_time is not None:
first_word_spoken_time = first_word_time - start_time
method_timings["first_word_spoken"] = first_word_spoken_time
logging.info(f"First word spoken after {first_word_spoken_time:.2f} seconds")
Find the fastest method
fastest_method = min(method_timings, key=method_timings.get)
fastest_time = method_timings[fastest_method]
Pretty-print the results
print("\nMethod Timing Results:")
for method, timing in method_timings.items():
print(f"{method}: {timing:.2f} seconds")
print(
f"\nThe fastest method was '{fastest_method}' with a time of {fastest_time:.2f} seconds."
)
if "first_word_spoken" in method_timings:
print(
f"\nThe first word was spoken in the speak_streamed method after {first_word_spoken_time:.2f} seconds."
)

we get

```bash
Method Timing Results:
synth_to_file: 1.03 seconds
synth_to_bytestream: 0.77 seconds
speak: 0.60 seconds
speak_streamed: 49.97 seconds
first_word_spoken: 0.70 seconds

The fastest method was 'speak' with a time of 0.60 seconds.

The first word was spoken in the speak_streamed method after 0.70 seconds.

  _     ._


Insert new line on enter press in a TextField

Hi.
How do I create a TextArea? That means a TextField where when you press enter on your keyboard it adds a new line, something like notepad, or really any text editor. When I press enter in a TextField it just toggles the keyboard.
How can I achieve this in flutter? Thank you.

@gspencergoog was looking at this recently, may have fixed?

TextField has a contructor param - maxLines, by setting maxLines greater than 1, TextField accepts multi-line text input but currently there are issues with cursor placement (When I tested, after pressing enter, cursor didn't move to the new line, even though a new line was created), see #12120 for more details.
Was looking for similar feature and sharing what I had found.

@selvan i ran into the same issue -  (When I tested, after pressing enter, cursor didn't move to the new line, even though a new line was created).
but i didn't see any related information for the tracking of this issue, except

There are still some issues with cursor placement in multi-line input, but those may go away when we switch to a non-Blink text renderer.

do you mean this one??
excuse me, @gspencergoog. is there any workaround for it??
thanks.

libtxt landed in master as far as I know and this could be fixed

Tested in Flutter v0.4.4 with libtxt landed and still has the described behavior.

Take a look on this workaround #15572 from @oshima819
It worked for me!

I close this as dup of #15572 because it has a bir more information.


Support compression in CPHD read / write

When trying to read or write compressed CPHD, I was receiving an error message "Gout unhandled signal array format". This update modifies CPHDReader and CPHDWriter objects to allow reading/writing raw byte data to the signal block when SignalCompressionID is set in the Data block.

I've retargeted to a new feature branch so that unittests and associated fixes can be added on in a subsequent PR. Thanks for the pull request; this is a useful capability.


#361: fixed nightly build pipeline

Closes #361

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 59.627%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
4670

Relevant Lines:
7545

💛  - Coveralls


Feat/release 1.13 docs

Applicable issues

fixes #

Description of changes

Additional information

Testing

Checklist

[x] Chart version bumped in Chart.yaml according to semver
[x] appVersion bumped in Chart.yaml if it's dial chart and any application version changed
[x] Variables are documented in the values.yaml and added to the README.md using helm-docs
[x] Title of the pull request follows Conventional Commits specification

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Hey there and thank you for opening this pull request! 👋🏼
We require pull request titles to follow the Conventional Commits specification and it looks like your proposed title needs to be adjusted.
Details:
No release type found in pull request title "Feat/release 1.13 docs". Add a prefix to indicate what kind of release this pull request corresponds to. For reference, see https://www.conventionalcommits.org/

Available types:
 - fix
 - hotfix
 - feat
 - feature
 - chore


Missing validation for start & end dates on vouchers, sales, gift cards

What I'm trying to achieve
…
Steps to reproduce the problem

Create a voucher or a sale through the dashboard, or directly through the API
Set the end date BEFORE the start date

What I expected to happen
Should have prevented me from creating a voucher with incorrect dates on 3 levels:

Client side React of the dashboard
GraphQL API
Django models

Screenshots

System information
Saleor 2.10 (saleor-platform) run through Docker
Operating system:
Windows 10
Question
This could've been partially addressed by having database-side validation.
Is there a reason why saleor does not use Postgres built-in Range types? I know graphene support for those is patchy, but other than that?
Also, I believe DateTimeRangeField implements this type of validation out of the box.

Hi, can I work on this issue?

Feel free to pick it up @akash1997!


vavr 0.9.3 workshops

Hi, I've created vavr 0.9.3 workshops: https://github.com/mtumilowicz/java11-vavr093-workshops:

option-workshop
try-workshop
partial-function-lifting-workshop
either-workshop
pattern-matching-workshop
validation-workshop

Popularity is quite high as many people message me about it, so maybe it would be reasonable to pin it here somewhere?

Thx, that's awesome!
We will find some place...

Part of the web site, therefore closing it here...


namespaces - provided namespaces should always show in namespace list

Signed-off-by: Wayne Witzel III<EMAIL_ADDRESS>What this PR does / why we need it:
Which issue(s) this PR fixes

Fixes #2259

Special notes for your reviewer:
Release note:
release-note

This can be squashed if we want to remove the changelog commit


Remove the background from the movie and makes it transparent

Is it possible?
I tried with:
out = remover.process(img) # same as image, except for 'rgba' which is not for video.
I removed the 'type' argument because in the image it doesn't have one, but it still doesn't work.

Hi @Hemilibeatriz , could you be more specific? I cannot understand the issue. The type argument has 6 different options.

An example:
import cv2
import numpy as np

from PIL import Image
from transparent_background import Remover

# Load model
remover = Remover() # default setting
remover = Remover(mode='fast', jit=True, device='cuda:0', ckpt='/root/.transparent-background/ckpt_base.pth') # custom setting
remover = Remover(mode='base-nightly') # nightly release checkpoint

# Usage for video
cap = cv2.VideoCapture('/content/Background.mp4') # video reader for input
fps = cap.get(cv2.CAP_PROP_FPS)

writer = None

while cap.isOpened():
    ret, frame = cap.read() # read video

    if ret is False:
        break

    frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
    img = Image.fromarray(frame).convert('RGB')

    if writer is None:
        writer = cv2.VideoWriter('output.mp4', cv2.VideoWriter_fourcc(*'mp4v'), fps, img.size) # video writer for output

    out = remover.process(img, type='*png') # same as image, except for 'rgba' which is not for video.
    writer.write(cv2.cvtColor(np.array(out), cv2.COLOR_BGR2RGB))

cap.release()
writer.release()

This code creates a movie without a transparent background. The background is white. What´s wrong? Can you help me?

for *.png type, you need to specify a background image path such as /home/user/background_sample.png. Valid types for video output are map, green, blur, overlay, *.png. rgba won't work since there are no video format for rgba.

Where do I need to specify this? Can you tell me?

In Line 30
out = remover.process(img, type='*png')

Change this line to below.
out = remover.process(img, type='green')

You'll have green screen applied result.


Helm CLI 3.7 or greater required

The chart requires Helm version 3.7 or greater to work for a few reasons:
Helm version <3.6 does not support the multi-line { fail (print ...  command in _helpers.tpl
Helm version <3.7 does not support the .Subcharts variable.
We should look into ways to improve this and support earlier versions of Helm without loosing functionality.

Solving the multiline fail might be as simple as not multi-line'ing it.
e.g.: "grafana-agent:\n  agent:\n    controller: ..."
Solving the .Subcharts might be done by changing the config file to pick up the ConfigMap dynamically, rather than volume mounted. Then we can control the configmap name without using the grafana-agent subchart fullname command.


Add feature flag magicCommandsAsComments (#6408)

For #6408

[x] Pull request represents a single change (i.e. not fixing disparate/unrelated things in a single PR)
[x] Title summarizes what is changing
[x] Has a news entry file (remember to thank yourself!)
[x] Appropriate comments and documentation strings in the code
[ ] Has sufficient logging.
[ ] Has telemetry for enhancements.
[ ] Unit tests & system/integration tests are added/updated
[ ] Test plan is updated as appropriate
[x] ~package-lock.json has been regenerated by running npm install (if dependencies have changed)~
[ ] The wiki is updated with any design decisions/details.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: janosh sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

Can you add a 6408.md file into the news\Enhancements folder? It should describe what the change is for. Thanks

Thanks @janosh . Looks good to me. If @IanMatthewHuff 	is okay with it, we can submit once the PR validation passes.

Thanks @janosh

Thanks for the lightning-fast reviews! Can you say when this will land?

Just tried it but doesn't appear to contain the enhancement yet.

Ah that's because the master branch is currently failing the CI tests. Not because of this PR though. A VEnv failure. Not sure if you can see these or not:
https://dev.azure.com/ms/vscode-python/_build/results?buildId=24069

Ah, thanks for letting me know. (The build status you linked is indeed public.)


F7 Lightning Plus board SMD units system

I am browsing parts list for F7 Lightning Plus and found no information about SMD units system used. Which one is used for the SMD? The imperial or metric?

0603 imperial
On Sun, 18 Apr 2021, 18:43 Jujan, @.***> wrote:

I am browsing parts list for F7 Lightning Plus and found no information
about SMD units system used. Which one is used for the SMD? The imperial or
metric?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/keirf/Greaseweazle/issues/99, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AATU2PDABNNSA77X42L3WQDTJMK5DANCNFSM43EOGFTA
.

We wouldn't be popular forcing people to solder 0603 metric!

Thank you. Closing now.


Hook prefix error for 'the_content' filter

Plugin gives error for not prefixing 'the_content' filter hook (if it is invoked by a theme). There are many valid reasons for theme to use 'the_content' filter in themes and it obviously cannot be prefixed. Even twenty seventeen uses this filter.
This error is only visible if you have added theme prefix.

Hi @vedathemes thanks for the testing.
If you could provide some more details that would be great.

The standards that you've selected when this error happened
Example of the code, or the theme you tested
Exact error message

The prefix check is coming from WordPress Coding Standards, it would be great if you could open an issue there with a code example as well so this can be fixed in the proper place 🙂

I have selected "WPThemeReview" standard.
I checked it on my own theme https://wordpress.org/themes/bayleaf/
Error msg: Hook names invoked by a theme/plugin should start with the theme/plugin prefix. Found: "the_content".

I will check and open an issue as suggested by you. Thanks

I can confirm this. I think we should exclude core hooks from this prefixing check.

There is a
protected $whitelisted_core_hooks = array(
	'widget_title'   => true,
	'add_meta_boxes' => true,
);

In the sniff checking this rule. My guess is this should be extended to include the_content as well. Do you have any more examples of core hooks which should be in this list?

I am afraid I dont have the list exactly. I guess there would be dozens of filters.

Interesting comment: https://github.com/WordPress-Coding-Standards/WordPress-Coding-Standards/issues/1139

This is about apply_filters, right? There shouldn't be dozens, but there might be a few.
When a theme has a nav_walker, it should still apply core filters. Same for image_size_names_choose.
the_content is a tricky one. If a theme is applying that filter, it's probably manipulating the content when it shouldn't. I don't think it should be white listed.

I think, there are a few valid cases where a theme can use 'the_content' filter.
One possible reason could be to take benefit of various functions that are hooked to this filter for processing raw data. Like, run_shortcode, autoembed, do_shortcode, do_blocks etc. (With future proof correct hook priority order). get_the_content() is not providing processed content and the_content() just echoing processed data. So, there is not way other than manually passing get_the_content() through this filter without echo.
one possible use case for this processed data (there could be many more) is to fetch video, audio or gallery from the content (If you want to display post video or audio on index pages).

This is about apply_filters, right? There shouldn't be dozens, but there might be a few.
When a theme has a nav_walker, it should still apply core filters. Same for image_size_names_choose.
the_content is a tricky one. If a theme is applying that filter, it's probably manipulating the content when it shouldn't. I don't think it should be white listed.

I think it is add_filter where content is manipulated and not apply_filters.

This is about apply_filters, right? There shouldn't be dozens, but there might be a few.
When a theme has a nav_walker, it should still apply core filters. Same for image_size_names_choose.

If there are examples of more hooks that a theme can use, could you write them down? It would be good to add them to the PR I've made to the WPCS 🙂

I think it is add_filter where content is manipulated and not apply_filters.

Correct

I think it is add_filter where content is manipulated and not apply_filters.

What I meant was that a filter is the correct way for a theme to manipulate content, done with add_filter, but this is checking for apply_filter which is only used when the content is built or retrieved outside the normal flow of the WordPress query. In other words, manipulating the content outside of a filter, which it shouldn't necessarily be doing (referring here to the_content) and then calling apply_filter on it so the rest of the normal filters are run. So I still feel that the_content should not be on the whitelist.

could you write them down?

Look in the core nav_walker and see which filters are there. Those should be in other nav_walkers.

I think it is add_filter where content is manipulated and not apply_filters.

What I meant was that a filter is the correct way for a theme to manipulate content, done with add_filter, but this is checking for apply_filter which is only used when the content is built or retrieved outside the normal flow of the WordPress query. In other words, manipulating the content outside of a filter, which it shouldn't necessarily be doing (referring here to the_content) and then calling apply_filter on it so the rest of the normal filters are run. So I still feel that the_content should not be on the whitelist.

I agree that for normally displaying post content, we should always use the_content() function rather than generating it manually. However, there are cases when we don't want to display the content but use it to fetch some data from it (as explained in my previous comment). 'the_content' filter is the most cleanest and future proof method to do this. I suggest you to check twenty seventeen theme for more on how a theme can use this.

Can anybody explain why this type of code is not good in theme? Just curious.
$content = apply_filters( 'the_content', $some_content );

In my view, applying the_content filter should not be a problem. It will only pass $some_content to hooked functions. However, there are few things that should be taken care of,

If some theme author unknowingly recreate a core filter as a theme specific filter ( That's where theme prefixing is required ). Here, sniffer should create a Warning (not ERROR) that "a core filter is used and it should be checked that author is really intending to use the core filter".

If your $some_content is not get_the_content() and there is a hooked function which is updating some data in database or cache from that content (It should be possible, but I don't know if there is any function that is really doing that). It might pose a security or compatibility problem. Therefore, use of this filter should be restricted to get_the_content() only.

I think Sniffer should show a warning message that the_content core filter has been used and reviewer should manually check for above two things. But, in anyway it should not be an ERROR message (because there are many valid reasons to manually apply the_content filter on get_the_content()).
On a side note, I think this filter should be shifted to get_the_content() function.

@vedathemes You could open an issue in the WPCS repo outlining the possible issue? As the prefixAllGlobalsSniff is pulled from there. Also any code examples will help, alongside any other allowed theme filters.
@ernilambar I think that it's considered 'not good' because it's a core filter. If this was
$content = apply_filters( 'plugin_name_content_filter', $content );

that would basically allow some plugin to hook to your theme filter, that would be ok.
Setting it as a core filter means that any plugin that hooks to the_contentfilter may impact your output (and you may not want that).
At least that's the way I see it 🤷♂

I see it as plugin functionality or content creation. The theme should not be gathering content to apply that filter to. It should only be affecting the content by adding a filter to the_content.

Here's a full description of why apply_filters( 'the_content', $content ) is often a problem and breaks plugins: https://themehybrid.com/weblog/how-to-apply-content-filters
Basically, it should only ever be called in The Loop context where a global $post object is available.  Under that scenario is the only time a theme author should do it.

Here's a full description of why apply_filters( 'the_content', $content ) is often a problem and breaks plugins: https://themehybrid.com/weblog/how-to-apply-content-filters
Basically, it should only ever be called in The Loop context where a global $post object is available. Under that scenario is the only time a theme author should do it.
I can't even count the number of times I've had to contact plugin/theme authors to fix this because they were breaking a plugin of mine.

Thank you for this insight. It's a learning for me.

I have closed the upstream PR in the WPCS about this, and it seems like themes shouldn't mess with this filter, right? So I'll close this.
Feel free to ping me if this needs to be opened 🙂


Expose offsets in reader and writer

Some serialization/deserialization libraries consume the writer/reader, and in this case the amount of bytes involved is lost. Surfacing the offsets of the stable storage reader and writer as a public API, would allow stable storage to be used more effectively (for example, key, value storage of serialized blobs).

Here's the change: https://github.com/dscvr-one/cdk-rs/commit/c85d8258287f091a404abd590812d8f94e3d20a8. I can post a PR if this proposal sounds good.

I believe I handled this in my icfs crate (which is really providing Read, Write, and Seek implementations)
https://github.com/codebase-labs/icfs/tree/943fa65ec89436d3438e11f620dfcca983e37811/crates/icfs
@ncpenke it's well tested so you may prefer to use that.
I created https://github.com/codebase-labs/icfs/issues/25 for potentially upstreaming the changes to cdk-rs.

For what it's worth, my motivation for providing a different implementation was this issue which required a single struct to implement the relevant traits for everything to work together properly.

Thanks @paulyoung, this is a much more smaller/isolated change and we've done some testing on top of it already for our use case. As part of this issue I was also hoping to get some feedback from the Dfinity team if there were any downsides to exposing the offset in this way.
As an aside, are looking into evaluating rust-fatfs so will definitely give your icfs crate a try as part of that. Really appreciate the work you've done to unify the structs for compatibility. If you can make your crate backwards compatible by type aliasing with StableWriter and StableReader it may be an easier merge path.

@ncpenke Thanks for the suggestion.
Feel free to send a PR.

As an aside, are looking into evaluating rust-fatfs so will definitely give your icfs crate a try as part of that.

@ncpenke, in case you didn’t already see, there’s an example of how to do that here: https://github.com/codebase-labs/icfs/blob/943fa65ec89436d3438e11f620dfcca983e37811/examples/fatfs
It demonstrates a posix-style API but I have an issue for potentially creating something more idiomatic for Rust: https://github.com/codebase-labs/icfs/issues/23

@lwshang, sorry for the delayed response. I opened a PR. Please let me know if you have any feedback.
@paulyoung, thanks for the info, will cycle back with you on this.


build subnetworks

A new utility to build subnetworks of common order from larger networks.

suggested improvement
Currently, subnetwork creation is constrained by a user-appointed maximum subnetwork depth. Subnetworks are created via a breadth-first-search. The search begins from a source node and stops once the maximum depth is reached OR once there are no more nodes to reach.
Should we change slightly such that the search continues until either 1) a threshold number of segments are in the subnetwork or 2) there are no more nodes to reach? The issue with the code as it stands now, is that the depth of the subnetwork is not immediately related to the size (number of segments) in the subnetwork.

Replaced by #175

As noted, closing to use #175 which contains this code rebased to the current master + additional capability for using the grouped/clustered subnetworks in the actual routing function.


How to configure HTML, can automatically complete the head?

Please, provide a more detailed description.

Input HTML press the tab key will automatically add the HTML's head, a lot of the editor is not it?

You can a snippet that contains any content you want.
Check out File > Preferences> User Snippets.


Csdp git source UI

[CI] Staging environment is at http://cap-staging-docs.codefresh.io.s3-website.us-east-1.amazonaws.com/csdp-git-source-ui/docs/

[CI] Staging environment is at http://cap-staging-docs.codefresh.io.s3-website.us-east-1.amazonaws.com/csdp-git-source-ui/docs/

[CI] Staging environment is at http://cap-staging-docs.codefresh.io.s3-website.us-east-1.amazonaws.com/csdp-git-source-ui/docs/

[CI] Staging environment is at http://cap-staging-docs.codefresh.io.s3-website.us-east-1.amazonaws.com/csdp-git-source-ui/docs/

[CI] Staging environment is at http://cap-staging-docs.codefresh.io.s3-website.us-east-1.amazonaws.com/csdp-git-source-ui/docs/

[CI] Staging environment is at http://cap-staging-docs.codefresh.io.s3-website.us-east-1.amazonaws.com/csdp-git-source-ui/docs/


How to create a release.keystore in godot

Godot version:

OS/device including version:

Issue description:

Steps to reproduce:
Minimal reproduction project:

Hello, I am using Godot 3.0.5 and I made a game. I did almost everything, I put AdMob and I closed debugging mode but there was one last thing. When I was uploading my apk file to google play it gave me an error that was saying that the apk file wasn't signed. I looked for a way to do this, however, it was saying to create a release.keystore file but I have no idea how to create it. Please help me!! Also, I use windows 10 and Godot 3.0.5.


list index out of range error when trying to save registered slides

Hey Chandler,
I just made it work on Databricks! However, I am when I try to save registered slides with
registrar.warp_and_save_slides(registered_slide_dst_dir, crop="overlap")
I get this error:
Saving images:   0%|                                                                    | 0/2 [00:14<?, ?image/s] Traceback (most recent call last):   File "<stdin>", line 1, in <module>   File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/valis/valtils.py", line 30, in wrapper     return f(*args, **kwargs)   File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/valis/registration.py", line 4518, in warp_and_save_slides     slide_obj.warp_and_save_slide(dst_f=dst_f, level=level,   File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/valis/valtils.py", line 30, in wrapper     return f(*args, **kwargs)   File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/valis/registration.py", line 1004, in warp_and_save_slide     ome_xml_obj = slide_io.update_xml_for_new_img(current_ome_xml_str=slide_meta.original_xml,   File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/valis/valtils.py", line 30, in wrapper     return f(*args, **kwargs)   File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.10/site-packages/valis/slide_io.py", line 2674, in update_xml_for_new_img     img = og_ome.images[series] IndexError: list index out of range
Could you advice what could be going on here? Thanks!

Hi @ciaobladoo,
Congrats on getting valis to work on Databricks! I'll have to check it out. Regarding this error, it looks like there is a mismatch between the series that was read, and the series that is being used to access the image metadata. I've made some updates that should prevent this from happening, so hopefully it should be fixed in the next update (1.0.3), which I think should be ready sometime next week. However, if you're able to share the images that are causing the issue, I could test before pushing the update to make sure that this won't be an issue in the future. If you'd prefer to share them privately, you can email me at<EMAIL_ADDRESS>Best,
-Chandler

Hi Chandler,
Do you mean this error is caused by the particular images I am registering? My colleague have previously made it work on different set of images on a local machine. I will double check with my manager if I could send you the images. If so I will send to your email.
Thank you so much!
Best,
Chao

Hey @cdgatenbee, just wanted to give you a quick update - it seems like the error may not be because of the image metadata. I tried to register images my colleague Daniel (who you may know) successfully registered on a local machine and am getting this same error. We will find time to discuss and see what is going on and get back to you.

The same error.
Previously it worked perfectly, I guess something changes in the dependencies...
JVM has been initialized. Be sure to call registration.kill_jvm() or slide_io.kill_jvm() at the end of your script.

==== Converting images

Converting images: 100%|██████████| 3/3 [00:05<00:00,  1.96s/image]

==== Processing images

Processing images : 100%|██████████| 3/3 [00:06<00:00,  2.15s/image]
Normalizing images: 100%|██████████| 3/3 [00:00<00:00,  8.90image/s]
Denoising images  : 100%|██████████| 3/3 [00:00<00:00,  3.05image/s]

==== Rigid registration

Detecting features   : 100%|██████████| 3/3 [00:18<00:00,  6.25s/image]
Matching images      : 100%|██████████| 3/3 [00:01<00:00,  1.52image/s]
Finding transforms   : 100%|██████████| 2/2 [00:00<00:00, 1370.69image/s]
Finalizing           : 100%|██████████| 3/3 [00:00<00:00, 2830.80image/s]

======== Rigid registration complete in 24.707 seconds

==== Non-rigid registration

Preparing images for non-rigid registration: 100%|██████████| 3/3 [00:03<00:00,  1.08s/image]
Finding non-rigid transforms: 100%|██████████| 2/2 [00:08<00:00,  4.14s/image]

======== Non-rigid registration complete in 10.528 seconds

/usr/local/lib/python3.10/dist-packages/interpolation/splines/eval_splines.py:318: NumbaPendingDeprecationWarning: Code using Numba extension API maybe depending on 'old_style' error-capturing, which is deprecated and will be replaced by 'new_style' in a future release. See details at https://numba.readthedocs.io/en/latest/reference/deprecation.html#deprecation-of-old-style-numba-captured-errors
Exception origin:
  File "/usr/local/lib/python3.10/dist-packages/interpolation/splines/eval_splines.py", line 116, in __eval_spline
    kk = (order).literal_value

  return _eval_cubic(*args)

==== Measuring error

Measuring error: 100%|██████████| 3/3 [00:04<00:00,  1.51s/image]
Saving images:   0%|          | 0/3 [00:05<?, ?image/s]
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
[<ipython-input-5-0175eabb0ab3>](https://localhost:8080/#) in <cell line: 6>()
      4
      5 # Save all registered slides as ome.tiff
----> 6 registrar.warp_and_save_slides(registered_slide_dst_dir, crop="overlap")

5 frames
[/usr/local/lib/python3.10/dist-packages/valis/valtils.py](https://localhost:8080/#) in wrapper(*args, **kwargs)
     28         def wrapper(*args, **kwargs):
     29             rename_kwargs(f.__name__, kwargs, aliases)
---> 30             return f(*args, **kwargs)
     31
     32         return wrapper

[/usr/local/lib/python3.10/dist-packages/valis/registration.py](https://localhost:8080/#) in warp_and_save_slides(self, dst_dir, level, non_rigid, crop, colormap, interp_method, tile_wh, compression, Q)
   4516             dst_f = os.path.join(dst_dir, slide_obj.name + ".ome.tiff")
   4517
-> 4518             slide_obj.warp_and_save_slide(dst_f=dst_f, level=level,
   4519                                           non_rigid=non_rigid,
   4520                                           crop=crop,

[/usr/local/lib/python3.10/dist-packages/valis/valtils.py](https://localhost:8080/#) in wrapper(*args, **kwargs)
     28         def wrapper(*args, **kwargs):
     29             rename_kwargs(f.__name__, kwargs, aliases)
---> 30             return f(*args, **kwargs)
     31
     32         return wrapper

[/usr/local/lib/python3.10/dist-packages/valis/registration.py](https://localhost:8080/#) in warp_and_save_slide(self, dst_f, level, non_rigid, crop, src_f, channel_names, colormap, interp_method, tile_wh, compression, Q)
   1002         bf_dtype = slide_io.vips2bf_dtype(warped_slide.format)
   1003         out_xyczt = slide_io.get_shape_xyzct((warped_slide.width, warped_slide.height), warped_slide.bands)
-> 1004         ome_xml_obj = slide_io.update_xml_for_new_img(current_ome_xml_str=slide_meta.original_xml,
   1005                                                       new_xyzct=out_xyczt,
   1006                                                       bf_dtype=bf_dtype,

[/usr/local/lib/python3.10/dist-packages/valis/valtils.py](https://localhost:8080/#) in wrapper(*args, **kwargs)
     28         def wrapper(*args, **kwargs):
     29             rename_kwargs(f.__name__, kwargs, aliases)
---> 30             return f(*args, **kwargs)
     31
     32         return wrapper

[/usr/local/lib/python3.10/dist-packages/valis/slide_io.py](https://localhost:8080/#) in update_xml_for_new_img(current_ome_xml_str, new_xyzct, bf_dtype, is_rgb, series, pixel_physical_size_xyu, channel_names, colormap)
   2672             if colormap is None:
   2673                 # Get original channel colors
-> 2674                 img = og_ome.images[series]
   2675                 colormap = {c.name: c.color.as_rgb_tuple() for c in img.pixels.channels}
   2676                 all_rgb = set(list(colormap.values()))

IndexError: list index out of range

Thanks for the updates @ciaobladoo and @unnir! I think I have the fix for this in the next update (1.0.3), but before pushing I'll test using updated dependencies to make sure everything works as expected.
Best,
-Chandler

great! Thank you!

Hi, @cdgatenbee. I talked to Daniel and the only reason I can think of that for the same pair of images he can register on a local machine and I cannot on Databricks is that Docker does something that prevents this error (I cannot use Docker on Databricks). Hope your fix can help. I sent you a pair of images last week via email. Feel free to use them for testing. Let me know if you need anything else. Thank you so much for your diligent help!

Hi @ciaobladoo,
Could you try updating valis to 1.0.3 (pushed today) to see if that fixes the issue?
Best,
-Chandler

Will do, Chandler! Thank you for the hard work!

Great! Glad this fixed the issue :)


master (pre-receive hook declined)

Hi.
I have a problem with our tsuru because after the last upgrade I can't make git push any reject the pre-recive hook
root@tsurudev:~/teststatic# git push tsusu master
Counting objects: 3, done.
Writing objects: 100% (3/3), 227 bytes | 0 bytes/s, done.
Total 3 (delta 0), reused 0 (delta 0)
remote: Traceback (most recent call last):
remote:   File "<stdin>", line 2, in <module>
remote:   File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
remote:     return _default_decoder.decode(s)
remote:   File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
remote:     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
remote:   File "/usr/lib/python2.7/json/decoder.py", line 384, in raw_decode
remote:     raise ValueError("No JSON object could be decoded")
remote: ValueError: No JSON object could be decoded
To git@tsurudev.qdqmedia.com:teststatic.git
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'git@tsurudev.qdqmedia.com:teststatic.git'

I don't know if is gandalf-server or archive-server causing the error.
The gandalf server create the repository on git@tsurudev.qdqmedia.com:teststatic.git but it is empty if i try to clone it
root@tsurudev:/tmp# git clone git@tsurudev.qdqmedia.com:teststatic.git
Cloning into 'teststatic'...
warning: You appear to have cloned an empty repository.
Checking connectivity... done.

Is there any way to debug this problem ?

enabling -x on pre-receive hook I can see the reason of the error, archive-server return missing archive file to the curl request
remote: + export TSURU_HOST=http://<IP_ADDRESS>:8080
remote: + TSURU_HOST=http://<IP_ADDRESS>:8080
remote: + export TSURU_TOKEN=bf1181bb84ef7fe24c8f035cf3941b3843450580
remote: + TSURU_TOKEN=bf1181bb84ef7fe24c8f035cf3941b3843450580
remote: + export ARCHIVE_SERVER_READ=http://<IP_ADDRESS>:3232 ARCHIVE_SERVER_WRITE=http://<IP_ADDRESS>:3131
remote: + ARCHIVE_SERVER_READ=http://<IP_ADDRESS>:3232
remote: + ARCHIVE_SERVER_WRITE=http://<IP_ADDRESS>:3131
remote: + read oldrev newrev refname
remote: + set +e
remote: + echo refs/heads/master
remote: + grep -v 'tags/master$'
remote: + grep -q '/master$'
remote: + status=0
remote: + set -e
remote: + '[' 0 = 0 ']'
remote: + COMMIT=1d4cbcf5f55b4720091384241ce163155f3d1607
remote: + read oldrev newrev refname
remote: + '[' -z 1d4cbcf5f55b4720091384241ce163155f3d1607 ']'
remote: + APP_DIR=tests3.git
remote: + APP_NAME=tests3
remote: ++ generate_archive /var/lib/gandalf/repositories/tests3.git 1d4cbcf5f55b4720091384241ce163155f3d1607
remote: ++ url=http://<IP_ADDRESS>:3131/
remote: +++ curl -sNd 'path=/var/lib/gandalf/repositories/tests3.git&refid=1d4cbcf5f55b4720091384241ce163155f3d1607' http://<IP_ADDRESS>:3131/
remote: ++ result='missing archive file'

@vmalaga looks like it's indeed archive-server. Did you upgrade it as well? If so, could you please upgrade the pre-receive hook? There was a change in the archive-server (I expected it to be backward compatible, but looks like it wasn't).
I'm at the moment fixing archive-server so it's backward compatible, sorry for about that :-(

Launchpad is finishing the build of the deb packages, can you try upgrading archive-server afterwards and confirm that it works as expected?

@fsouza I just upgraded archive-server and now it works like expected.  Thanks for the speed, so leave the platform working for tomorrow.
I am going to finish review all the stack to give the upgrade to 0.13 closed

@vmalaga awesome, please let us know if you find any other issue.


Error when typing

I just wrote a simple markdown file and when I was (vertically) near the end of the screen/browser window these error popped up two times. Every
Error: Got both html and text fields on Linter Response, expecting only one
  at D:\Users\%username%\.atom\packages\linter\lib\validate.coffee:33:19
  at Array.forEach (native)
  at Object.module.exports.Validate.messages (D:\Users\%username%\.atom\packages\linter\lib\validate.coffee:26:14)
  at MessageRegistry.set (D:\Users\%username%\.atom\packages\linter\lib\message-registry.js:56:18)
  at D:\Users\%username%\.atom\packages\linter\lib\linter.coffee:27:17
  at Emitter.module.exports.Emitter.emit (C:\Users\%username%\AppData\Local\atom\app-1.1.0\resources\app.asar\node_modules\event-kit\lib\emitter.js:86:11)
  at D:\Users\%username%\.atom\packages\linter\lib\linter-registry.js:98:29

Installed linter packages:

linter 1.10.0
linter-alex 1.5.0
linter-htmlhint 0.2.1
linter-php 1.1.0

Can you tell us which type of file were you editing when you got this error? It'll help us pinpoint the provider.

Markdown file like I said.
But I already got it out by disabling the linter packages one by one.
So moving this issue: https://github.com/wooorm/atom-linter-alex/issues/11


Add a shortcut to open the last closed file

Fixes #2415

Shortcut is Ctrl+Shift+N.
There is an option in the file menu, but the keyboard shortcut seems a better choice.
The code wasn't changed much.
Submitted via a branch

Thanks for this @thewhitetulip, will review and let you know of any comments to the code.
Cheers

Ok, final review is ready. @thewhitetulip, after you do my suggested changes this is ready for merging.
Sorry for the delay in coming back to you, we're very busy these days with other things.

@ccordoba12 May I ask status about this PR? I had made the necessary changes a long time ago. Also i have no idea why, the Ctrl+Shift+N shortcut works but Ctrl+shift+T doesn't work

@ccordoba12 ?

@ccordoba12 is this something we can merge?

@goanpeca, nop, according to @thewhitetulip this is not working for him. If you want to take care of his PR and make it work, that'd be great! It should be an easy fix :-)

ok

Yes, I believe it should be a small enough fix for you guys :+1:
The behaviour was actually strange, because if I added cmd+shift+T shortcut, then it mysteriously stopped working :D

Closing in favor of #3620.


Frame Jacobian Time Variation

Fix a few deprecated functions
add framejacobiantimevariation
*add unittest for above

updated and rebased commit: adds fd for framejacobiantimederivative local frame.


Not an issue

I created a Swift 3.x implementation based off your python/rust/C++ code and it is available as a cocoapod for people who want to use it in developing iOS applications. If you'd rather that I just give you the code (since it's 99% yours), I'd be more than happy to.
The cocoapod is here

Oh that's great! I'll create a /swift directory and link to your cocoapod from the README if that's ok.

It's perfectly fine with me... I'll be adding more to the framework shortly. As of now all it does is decode and encode. It doesn't do any of the "find nearest" stuff yet.


Add user scope for NetworkAdapterSummary v<IP_ADDRESS>

[ ] Have you signed the Contributor License Agreement?
[ ] Have you checked that there aren't other open pull requests for the same manifest update/change?
[ ] Have you validated your manifest locally with winget validate --manifest <path>?
[ ] Have you tested your manifest locally with winget install --manifest <path>?
[ ] Does your manifest conform to the 1.0 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

Microsoft Reviewers: Open in CodeFlow

Original PR -

https://github.com/microsoft/winget-pkgs/pull/32751


Added multi-node statistic capabilities to geopmsession

Fixes to #3542

A new python class (based on the Session class) reads signals provided by the end user in a file. When the job is launched, mpi is used to read signals per-node and accumulate moments in C++. At the end of the run, statistics are exported in a yaml file.
$ printf 'TIME board 0\nCPU_POWER board 0\nCPU_FREQUENCY_STATUS board 0\n' | geopmsession -t 2 -p 0.01 --print-header --report-out=report.yaml --trace-out=trace.csv --mpi-report
$ cat report.yaml
Hosts:
  saruman:
    TIME:
      count: 201
      first: 0.189681
      last: 2.04973
      min: 0.189681
      max: 2.04973
      mean: 1.05533
      std: 0.572718
    CPU_POWER:
      count: 201
      first: 1.02625e+09
      last: 73.4791
      min: 63.4933
      max: 1.02625e+09
      mean: 5.10579e+06
      std: 7.23861e+07
    CPU_FREQUENCY_STATUS:
      count: 200
      first: 1.055e+09
      last: 1e+09
      min: 1e+09
      max: 1.055e+09
      mean: 1.00399e+09
      std: 1.39072e+07

Other opens:

A new python class (based on the Session class) reads signals provided by the end user in a file.

Does this accept a file?  An example showing as much should be included in the docs.

    CPU_POWER:
      count: 201
      first: 1.02625e+09
      last: 73.4791
      min: 63.4933
      max: 1.02625e+09
      mean: 5.10579e+06
      std: 7.23861e+07

These numbers look odd.  Why is the min and last 6 to 9 orders of magnitude off compared to the other stats?

These numbers look odd. Why are the min and last 6 to 9 orders of magnitude off compared to the other stats?

I'm guessing we still have the issue where you get a random first reading on a signal based on a difference (POWER). This would mess up first/max/mean/std but leave count/min/last unaltered.

A suggested future feature of the report:  Enable selecting on a parameter like hint or hash.  The current report looks like:
$ printf 'TIME board 0\nCPU_POWER board 0\nCPU_FREQUENCY_STATUS board 0\n' |  geopmsession -t 2 -p 0.01 --print-header --report-out=report.yaml -d\| --trace-out=trace.csv
$ cat report.yaml
hosts:
  saruman:
    time-begin: "Thu Aug 01 11:50:42 2024"
    time-end: "Thu Aug 01 11:50:43 2024"
    metrics:
      TIME:
        count: 201
        first: 0.165067
        last: 2.04619
        min: 0.165067
        max: 2.04619
        mean: 1.05035
        std: 0.575056
      CPU_POWER:
        count: 200
        first: 109.982
        last: 73.0664
        min: 67.2973
        max: 109.982
        mean: 74.319
        std: 6.90355
      CPU_FREQUENCY_STATUS:
        count: 201
        first: 1.02625e+09
        last: 1e+09
        min: 1e+09
        max: 1.0525e+09
        mean: 1.00371e+09
        std: 1.32582e+07

in the future we may consider adding a "--report-select" feature where the output from --report-select=REGION_HINT might look like (made up data):
$ printf 'TIME board 0\nCPU_POWER board 0\nCPU_FREQUENCY_STATUS board 0\nREGION_HINT board 0\n' |  geopmsession -t 2 -p 0.01 --print-header --report-out=report.yaml -d\| --trace-out=trace.csv --report-select=REGION_HINT-board-0
$ cat report.yaml
hosts:
  saruman:
    time-begin: "Thu Aug 01 11:50:42 2024"
    time-end: "Thu Aug 01 11:50:43 2024"
    metrics:
      TIME:
        count: 201
        first: 0.165067
        last: 2.04619
        min: 0.165067
        max: 2.04619
        mean: 1.05035
        std: 0.575056
      CPU_POWER:
        count: 200
        first: 109.982
        last: 73.0664
        min: 67.2973
        max: 109.982
        mean: 74.319
        std: 6.90355
      CPU_FREQUENCY_STATUS:
        count: 201
        first: 1.02625e+09
        last: 1e+09
        min: 1e+09
        max: 1.0525e+09
        mean: 1.00371e+09
        std: 1.32582e+07
      REGION_HINT:
        count: 201
        first: 1
        last: 1
        min: 0
        max: 4
        mean: 3.01
    select:
      REGION_HINT:
        value: 1
          metrics:
            TIME:
              count: 100
              first: 0.165067
              last: 1.04619
              min: 0.165067
              max: 1.04619
              mean: 0.55035
              std: 0.275056
            CPU_POWER:
              count: 100
              first: 109.982
              last: 73.0664
              min: 67.2973
              max: 109.982
              mean: 74.319
              std: 6.90355
            CPU_FREQUENCY_STATUS:
              count: 100
              first: 1.02625e+09
              last: 1e+09
              min: 1e+09
              max: 1.0525e+09
              mean: 1.00371e+09
              std: 1.32582e+07
        value: 4
          metrics:
            TIME:
              count: 101
              first: 0.165067
              last: 1.04619
              min: 0.165067
              max: 1.04619
              mean: 0.55035
              std: 0.275056
            CPU_POWER:
              count: 100
              first: 109.982
              last: 73.0664
              min: 67.2973
              max: 109.982
              mean: 74.319
              std: 6.90355
            CPU_FREQUENCY_STATUS:
              count: 101
              first: 1.02625e+09
              last: 1e+09
              min: 1e+09
              max: 1.0525e+09
              mean: 1.00371e+09
              std: 1.32582e+07

How does this react to sigint?
E.g.
geopmsession -t 9e9 --report-out=report.yaml --trace-out=/dev/null
...
*ctrl-C*

Will the user still receive a report of whatever happened up until the interrupt?

We don't have any signal handling in geopmsession, so you would get no output if you sent a sig cancel

printf 'TIME board 0\nCPU_POWER board 0\nCPU_FREQUENCY_STATUS board 0\n' | mpiexec -np 2 geopmsession -t 10 -p 0.005  --report-out=report.yaml --trace-out=/dev/null --enable-mpi


Roblox Crashes Every time I open it

Every time I open Roblox with Bloxstrap it says Roblox has Crashed

What do I do?
I'm on Version 2.3.0

Are you sure this is a Bloxstrap problem? Revert back to Roblox's original bootstrapper for a second, and see if it happens witih that too. https://github.com/pizzaboxer/bloxstrap/wiki/Switching-between-Roblox-and-Bloxstrap