Speed and pitch control for large documents

Looking for a way to control the speed and pitch of  the  Polly TTS conversion using aws-tts. When I use ssml tags in the large documents the process of chunking up the ssml document causes errors. I want to use the   tags at the beginning and end of a large document.
Thanks,
Dave

@sonodave Could you provide an example SSML document that I can use for testing? It doesn't have to be large, but it would help me fix & test this faster with a real example.

Hi,
I have attached an example SSML document that is failing with a picture of
the errors I receive.  I would like to slow the narration down for several
books that I want to convert. They are in text format but I want to insert
the speak and prosody  tags into the text at the beginning and ending of
the text to have the entire document read at the slow pace.
Thanks
On Sat, Sep 16, 2017 at 10:02 PM, Eric Heikes<EMAIL_ADDRESS>wrote:

@sonodave https://github.com/sonodave Could you provide an example SSML
document that I can use for testing? It doesn't have to be large, but it
would help me fix & test this faster with a real example.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/eheikes/aws-tts/issues/17#issuecomment-330007159, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AedCMG5hVKYEQkiYkoNwcv_9GWnGjk2Tks5sjH3DgaJpZM4PWccW
.

@sonodave Can you try using v1.3.1 and see if it fixes your problem?
npm i<EMAIL_ADDRESS>-g

I installed v1.3.1 and it does work with the ssml documents.  It seems to have changed how the Spanish language reads the text from the previous version for some reason.  There is a difference now for the narration on the Polly console with text input and on the narrration I get using aws-tts. The pronunciations are off. I tried rolling back to v1.3.0 and it is still doing it. I have attached 2 sample narration files. This is with text and ssml documents. Did your changes affect how the document writes accents? Or default to an English specified encoding?

I installed v1.3.1 and it does work with the ssml documents. It seems to
have changed how the Spanish language reads the text from the previous
version for some reason. There is a difference now for the narration on the
Polly console with text input and on the narrration I get using aws-tts.
The pronunciations are off. I tried rolling back to v1.3.0 and it is still
doing it. I have attached 2 sample narration files. This is with text and
ssml documents. Did your changes affect how the document writes accents? Or
default to an English specified encoding?
I have attached two mp3 files and the text that was used so you can hear
the difference in pronunciation using the same text from Polly console to
aws-tts. The mispronunciation also is with the other Spanish voices.
.
On Thu, Sep 21, 2017 at 11:29 AM, Eric Heikes<EMAIL_ADDRESS>wrote:

@sonodave https://github.com/sonodave Can you try using v1.3.1 and see
if it fixes your problem?
npm i<EMAIL_ADDRESS>-g
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/eheikes/aws-tts/issues/17#issuecomment-331192403, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AedCMMRFJIUtVfdy1BhGm1YWHm4xrDDPks5skoBfgaJpZM4PWccW
.

Libro de Isaías.
Visión de Isaías hijo de Amoz, la cual vió sobre Judá y Jerusalem, en días de Uzzías, Jotham, Achâz y Ezechîas, reyes de Judá.
Oid, cielos, y escucha tú, tierra; porque habla Jehová: Crié hijos, y engrandecílos, y ellos se rebelaron contra mí.

When I restarted the machine under 1.3.0 it is translating/converting correctly again so I suspect that something you changed is telling the API that the document is english.

Did your changes affect how the document writes accents? Or default to an English specified encoding?

I don't think anything I changed would have changed the pronunciation. It's possible that AWS interprets SSML text different from plain text. Have you tried using the xml:lang attribute?

When I restarted the machine under 1.3.0 it is translating/converting correctly again so I suspect that something you changed is telling the API that the document is english.

This seems to point to something on your machine, since it changes with a restart. I'm not sure what it could be though.

I have attached two mp3 files and the text that was used so you can hear the difference in pronunciation using the same text from Polly console to aws-tts.

The files that you are attaching (I'm assuming through email) are not being included in the Github issue here, so can you paste your example files into a service like gist or pastebin and include the link here? Thanks.

@sonodave Please see my reply ^

Tied up for a few days but I will get the files linked up soon. I will also test the bug fix version on another machine  Dave

The text file used for test 1.3.1 with 2 different mp3 files showing different pronunciations in spanish for Console vs CLI with aws-tts
https://drive.google.com/drive/folders/0Bx5zBmY4ZLenQkx1VE5zVjBEbGc?usp=sharing

@sonodave What is the full aws-tts command you are running to get the converted with aws-tts.mp3 output?

I was able to test some more and the problem with the erratic conversion to Spanish was on my setup.
Some of my text documents were ansi and not UTF-8 and this caused the problem

I was wondering if the character encoding had something to do with it. Glad it worked out!


Support screen orientation change

hi
On screen orientation change activity stock in a loop of refreshing view. I think this is a bug in videoview, I every tutorials and libraries I found this problem, is there any solution?

Hi @HamidrezaAmz
What kind of refreshing loop? Are you implementing onConfigurationChanged method? Are you calling videoview.resize()?
Would you mind passing me more information about how can I achieve this issue? So I can see how can I help you.
Thanks.

Hi @rtoshiro
Every thing is OK until I force the screen to be in landscape mode, I test all suggested solutions on stack-overflow about handling orientation change bud nothing achieved!  My activity stuck in a loop of calling onDestroy and onCreate Continuously. Do you have any idea!
Thanks.

Hi @rtoshiro
I found the solution, I was overriding configuration method in my main application class. the new config was the problem.
have fun, thanks.


Tweak deployment

I currently cannot make the check dataflow workflow reusable. Let me merge this and check if that was an issue with running from a PR branch.


Firebase build issue

anyone, can i be assisted please
Severity	Code	Description	Project	File	Line	Suppression State
Error		C:\New folder\BlankCordovaApp4\BlankCordovaApp4\platforms\android\src\org\apache\cordova\firebase\FirebasePluginMessagingService.java:3: error: cannot find symbol	BlankCordovaApp4		1
Severity	Code	Description	Project	File	Line	Suppression State
Error		 if (android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.O) 	BlankCordovaApp4		1

Managed to solve it..

@TysonMot How did you solve the issue? I am also facing the same issue.


Injection attack potential

Because ng-select just dumps the content of the text attribute into [innerHtml] that makes it vulnerable to javascript injection attacks and it is certainly not obvious from the documentation that the developer needs to escape everything inside of text in order to make it secure.
Instead, I recommend allowing a template for the item to be passed in from another attribute on SelectItem or (a more general purpose solution) to provide an [itemTemplate] @Input on the component itself.  In order for this to work you will need to construct the list items as separate components with a dynamic template. here is an example of how to create a child component whose template is specified dynamically by a parent component.
Another general-purpose solution would be to allow a component factory to be passed to ng-select that displays the items.  You would need to publish an API of which @Inputs and events the component would need to handle.
The general purpose solutions would probably require a major re-work of the component, though, but would also add a the possibility of very flexible behaviour for the selection list, such as trees.

@cherrydev the attack would only be on your own system, any server or point where this would matter should have it's own injection protection. There isn't anything to stop me from adding my own  elements, even my own javascript via developer tools.
I'm no expert but if I can only change my own browser code I dont understand the security risk.. Can you explain?

Just imagine any multi-user system where User A can craft malicious content that will be displayed, and therefore execute on User B's browser, and therefore with any credentials of User B.

Yes, but when User A inputs the malicious code it only exists on their own system. When you submit it to the server to be stored in the database it gets sanitized so before user B see’s the results it’s processed. Stripping HTML, JS code, etc. is pretty standard in that regard…  I get that injection happens and can be a problem but I don’t think it matters as much on the front end for a single user.. Maybe I’m wrong?

On Jun 15, 2016, at 3:43 PM, cherrydev<EMAIL_ADDRESS>wrote:
Just imagine any multi-user system where User A can craft malicious content that will be displayed, and therefore execute on User B's browser, and therefore with any credentials of User B.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/valor-software/ng2-select/issues/242#issuecomment-226314052, or mute the thread https://github.com/notifications/unsubscribe/ABVRPOAPdvdGH_yirrxNohv-CuSgvVCuks5qMGP_gaJpZM4Iyh_P.

Attack possibilities may not be obvious but it looks like you anticipated what needed to be done. With the latest version of Angular2, each item in the selection throws now a warning in the console WARNING: sanitizing HTML stripped some content (see http://g.co/ng/security#xss).
I think the HTML to build the selection shall be sanitized like mentioned here https://angular.io/docs/ts/latest/guide/security.html#!#xss

Another thing I'd point out is that using [innerHtml] at all in Angular2 is pretty much giving up on using the framework in the way it was designed. I'm pretty much re-writing ng2-select for my own specific usage right now.  One of the non-specific changes I made was fixing the above to choose a component selector (<select-option-item>) that the ng2-select component will pass the current item to in order to render.  The component that satisfies that selector, though, is not provided by the ng2-select component itself, but instead is provided by a parent component. This effectively allows it to be customizable.


Masks in BSD X-ray date are not representative for certain time periods

Due to the configuration of the SumDMask parameter the CFL and BK detector data was not being recorded to the archive buffer so even if data is requested it will contain zeros. For normalisation purposes it would be better to adjust the detector mask on ground to reflect this. Date for this are late November 2020 to mid March 2021 @grazwegian

[ ]  Get exact date for the problem - possibly query tc for tm or parameter change?

[ ] Add case to override the detector masks

[ ] Add test

I think this is the TC the set the incorrect mask https://pub023.cs.technik.fhnw.ch/view/packet/id/977968
and this is the one the resets the correct value https://pub023.cs.technik.fhnw.ch/view/packet/id/1367499
so dates would be 2020-11-22T00:02:10.875000Z to 2021-03-12T11:26:53.725585Z


GUI: Inconsistent order of field value change and control pressing / item selection signals

Godot version
v4.1.1.stable.official [bd6af8e0e]
System information
Kubuntu 23.04
Issue description
There is an inconsistency in the order of losing focus / applying a field value (LineEdit, TextEdit, SpinBox) and receiving focus / pressing a control (Button, Tree, ItemList).
In the case of Button, the order of signals is expected: first the field value is changed, then the button is pressed.
In the case of Tree and ItemList, the signal order is reversed, which is inconvenient and leads to bugs in applying changes to the current item instead of the previous one (see below). You can use deferred signals as a workaround, but in my opinion this is a hack that could cause other bugs.
_on_spin_box_value_changed 10
_on_button_pressed

_on_tree_cell_selected
_on_spin_box_value_changed 20
_on_tree_cell_selected_deferred

_on_item_list_item_selected 1
_on_spin_box_value_changed 30
_on_item_list_item_selected_deferred 1

See also:

#26979
#28729
#79445
#79692

Steps to reproduce
Change the value of the SpinBox field (without pressing Enter) and try clicking on the Button, on the Tree item and on the ItemList item.

Minimal reproduction project
focus_order_bug.zip

Looks like this is a SpinBox issue (in the case of the Button it works correctly, since default action mode is on releasing the button). Are there any reasons why the SpinBox uses the deferred signal?
_on_spin_box_line_edit_focus_exited
_on_button_focus_entered
_on_spin_box_value_changed 10
_on_button_pressed

_on_spin_box_line_edit_focus_exited
_on_tree_focus_entered
_on_tree_cell_selected
_on_spin_box_value_changed 20
_on_tree_cell_selected_deferred

_on_spin_box_line_edit_focus_exited
_on_item_list_focus_entered
_on_item_list_item_selected 1
_on_spin_box_value_changed 30
_on_item_list_item_selected_deferred 1

https://github.com/godotengine/godot/blob/0c2144da908a8223e188d27ed1d31d8248056c78/scene/gui/spin_box.cpp#L387-L390

I quite agree.

If it just delays the timing of showing/refreshing the UI, it has no side effects.
If this delay affects the timing of data changes, it will cause data synchronization problems.

If both the data and the UI are updated in the connection method, we can just defer the UI update call.

I just ran into this.  A SpinBox that's part of a Dialog will process losing focus after its ancestor Dialog's 'canceled' signal if you close that Dialog with the X button (in my case, even if that 'canceled' signal handler was itself deferred).  So trying to reset the SpinBox's value in the 'canceled' handler won't work as it will get overwritten by its text.  A dirty workaround I've found is to do something weird like spinbox->get_line_edit()->set_text("null"); spinbox->apply(); to get _update_text to run early and have the value change stick, but I don't like it.


add glider QA/QC to the ideas-list.md

@mwengren and @MathewBiddle I know you are very busy but can you please review this one and check #55 ?
@callumrollo is this something you want to participate? Maybe help shape the idea and/or help mentor someone?

LGTM Thanks!!

Great project idea! I don't think I have the QC/QA expertise to help mentor this, but will certainly follow its development. There's been quite some interest in the UG2 community for improving glider QC


lax.igamma and igammac fail with internal error on float16 and bfloat16

The following code:
    a = np.ones((20, 20), dtype=dtypes.bfloat16)
    lax.igammac(a, a)

fails with
RuntimeError: Internal: Seen floating point types of different precisions in %compare.199 = pred[20,20]{1,0} compare(f32[20,20]{1,0} %divide.196, bf16[20,20]{1,0} %broadcast.198), direction=GT, but mixed precision is disallowed.

This problem looks fixable by upcasting to F32. But XLA is trying to do just that. Filed b/177754567 to get the fix in XLA, and then we don't need to workaround in JAX.

This problem looks fixable by upcasting to F32. But XLA is trying to do just that. Filed b/177754567 to get the fix in XLA, and then we don't need to workaround in JAX.

This issue is fixed at head.


Remove Facter Dependencies

Need to leverage updated scripts found in this issue/commit https://github.com/mrlesmithjr/packer-templates/issues/38

#72 Resolves this issue. Closing.


New version: astral-sh.uv version 0.5.5

Checklist for Pull Requests

[x] Have you signed the Contributor License Agreement?
[x] Resolve #194715

Manifests

[x] Have you checked that there aren't other open pull requests for the same manifest update/change?
[x] This PR only modifies one (1) manifest
[x] Have you validated your manifest locally with winget validate --manifest <path>?

[x] Does your manifest conform to the 1.6 schema?

Note: <path> is the directory's name containing the manifest you're submitting.
Manual validation

Microsoft Reviewers: Open in CodeFlow

The behavior of validation pipeline is weird 😕
Can't identify the reason of https://github.com/microsoft/winget-pkgs/labels/Validation-Installation-Error.

@wingetbot run


Programmatically Create Images in Assets

I'm not so sure that doing this would actually be a benefit to overall package size. Plus, it's just so much easier to copy an image :)


Function to compute endpoints

Return a vector of index values for last observation in ts for each period given by on, ending with the last values of index(ts). The values are picked up every k instance of period. The last observation of each unique group (based on on) is returned.
Supported interfaces
endpoints(timestamps::AbstractVector{T}, on::V) where {T<:Union{Date, DateTime, Time},
                                                       V<:Union{
                                                           Year,
                                                           Quarter,
                                                           Month,
                                                           Week,
                                                           Day,
                                                           Hour,
                                                           Minute,
                                                           Second,
                                                           Millisecond,
                                                           Microsecond,
                                                           Nanosecond
                                                       }}
endpoints(ts::TS, on::T) where {T<:Dates.Period}
endpoints(ts::TS, on::Symbol, k::Int=1)
endpoints(ts::TS, on::String, k::Int=1)
endpoints(ts::TS, on::Function, k::Int=1)
endpoints(values::AbstractVector, on::Function, k::Int=1)

The primary interface works on all time type vectors, namely, Date, DateTime, and Time and on all sub-types of Dates.Period. Rest of the methods except the ones accepting ::Function type for on call the primary method with the relevant arguments.
endpoints(values::AbstractVector, on::Function, k::Int=1) allows one to pass a function which creates unique groups which will then be used for getting the last observation from.
Examples
julia> ts = TS(random(length(dates)), dates)
(731 x 1) TS with Date Index

 Index       x1
 Date        Float64
───────────────────────
 2017-01-01  0.768448
 2017-01-02  0.940515
 2017-01-03  0.673959
 2017-01-04  0.395453
 2017-01-05  0.313244
 2017-01-06  0.662555
 2017-01-07  0.586022
 2017-01-08  0.0521332
 2017-01-09  0.26864
 2017-01-10  0.108871
     ⋮           ⋮
 2018-12-24  0.812797
 2018-12-25  0.158056
 2018-12-26  0.269285
 2018-12-27  0.15065
 2018-12-28  0.916177
 2018-12-29  0.278016
 2018-12-30  0.617211
 2018-12-31  0.67549
 2019-01-01  0.910285
       712 rows omitted

julia> ts[endpoints(ts, Month(1))]
(25 x 1) TS with Date Index

 Index       x1
 Date        Float64
───────────────────────
 2017-01-31  0.48
 2017-02-28  0.458476
 2017-03-31  0.274441
 2017-04-30  0.413966
 2017-05-31  0.734931
 2017-06-30  0.257159
 2017-07-31  0.415851
 2017-08-31  0.0377973
 2017-09-30  0.934059
 2017-10-31  0.413175
 2017-11-30  0.557009
 2017-12-31  0.346659
 2018-01-31  0.174777
 2018-02-28  0.432223
 2018-03-31  0.835142
 2018-04-30  0.945539
 2018-05-31  0.0635483
 2018-06-30  0.589922
 2018-07-31  0.285088
 2018-08-31  0.912558
 2018-09-30  0.238931
 2018-10-31  0.49775
 2018-11-30  0.830232
 2018-12-31  0.67549
 2019-01-01  0.910285

julia> ts[endpoints(ts, Month(2))]
(12 x 1) TS with Date Index

 Index       x1
 Date        Float64
───────────────────────
 2017-02-28  0.458476
 2017-04-30  0.413966
 2017-06-30  0.257159
 2017-08-31  0.0377973
 2017-10-31  0.413175
 2017-12-31  0.346659
 2018-02-28  0.432223
 2018-04-30  0.945539
 2018-06-30  0.589922
 2018-08-31  0.912558
 2018-10-31  0.49775
 2018-12-31  0.67549
 2019-01-01  0.910285

Refs #51.

Codecov Report

Merging #60 (f86b3d0) into main (545d0b9) will increase coverage by 9.21%.
The diff coverage is 84.12%.

@@            Coverage Diff             @@
##             main      #60      +/-   ##
==========================================
+ Coverage   39.04%   48.25%   +9.21%
==========================================
  Files          17       19       +2
  Lines         251      315      +64
==========================================
+ Hits           98      152      +54
- Misses        153      163      +10

Impacted Files
Coverage Δ

src/TSx.jl
100.00% <ø> (ø)

src/endpoints.jl
84.12% <84.12%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Done with this PR from my end. Ready for review. @ValentinKaisermayer would you please?

Merging this now so that rest of the development can proceed. Any further issues can be resolved later.


extends escape function for postgre database

Thanks for your pull request! I know these 5 lines hide way more hours of research and reverse engineering than it seems 🙂


[test] Adjust regex patterns for CP2K 8.1

The pull request addresses minor changes in the output of CP2K 8.1 with respect to previous releases:

the step keyword STEP NUMBER now becomes MD| Step number
the energy units (a.u.) are now enclosed in square branckets [a.u.]

Hello @lucamar, Thank you for submitting the Pull Request!

In the file cscs-checks/apps/cp2k/cp2k_check.py:

Line 18:80: E501 line too long (82 > 79 characters)

Do see the ReFrame Coding Style Guide

I don't know how to shorten line 82, as reported by pep8speaks: could you please add a suggestion directly?

I have indented all the energy block to reduce the length of line 18, in order to comply with the ReFrame Coding Style Guide as reported by pep8speaks.


Coreupdater doesn't work with Windows

Coreupdater doesn't work with Windows. You get an error like
ERROR: Failed to download files with error: fatal: path not found: \admin1/ajax-tab.php
The problem is in the way the variable $adminDir is constructed in /modules/coreupdater/classes/GitUpdate.php:
$adminDir = str_replace(_PS_ROOT_DIR_, '', _PS_ADMIN_DIR_);  $adminDir = trim($adminDir, '/').'/';
Both PS_ROOT_DIR and PS_ADMIN_DIR are dynamically constructed with php functions like realpath() and as such they contain backslashes under Windows. So under Windows the first line will put "\admin1" in the variable $adminDir and the trim() command on the next line won't have any effect.
I propose inserting
if($adminDir[0] == "\\") $adminDir = substr($adminDir,1);
between the two lines at line 454 and 874. It will delete a leadingbackslash when present.
See also: https://forum.thirtybees.com/topic/2628-core-updater-windows-xampp-problem/

@musicpanda seems to be already fixed a9abf97

I think this should do the trick. for windows dir \ escaping the $adminDir variable before parsing it to preg_replace .
I did this when upgrading, after this upgrade when without issues, and i were able to continue the development before releasing the shop update.
in (classes/GitUpdate.php#L881)[https://github.com/thirtybees/coreupdater/blob/master/classes/GitUpdate.php#L881]
$path = preg_replace('#^'.$adminDir.'#', 'admin/', $path);
to
$path = preg_replace('#^'.preg_quote($adminDir).'#', 'admin/', $path);

$path = preg_replace('#^'.preg_quote($adminDir).'#', 'admin/', $path);

Good point. Picked to master, but applied out of the loop: caebdb94838657c0dfa35cd048c3828568d0ef62
As the other thing was picked as well, this issue should be solved.


Not Working For Parsec Anymore

It worked yesterday, but when I hosted parsec again.. it didn't host.

I just tested it here and it is working.
Maybe your session became invalid and you are trying to join from the link. if that is the case logging out and re-inserting the auth code should solve it.


Is there a way to fetch own recipe name as a variable?

I would like to know if, is there a way to ask for the running recipe name?
That means for example
hello-myself:
echo "Hello from {{RECIPE_NAME}}"

Nope, there currently isn't. What is the use-case?

Nope, there currently isn't. What is the use-case?

We are currently working with just, and loving it. But, we are getting a lot of commands that they should be documented/show help better than just one line when executing "just --list"; the idea is to insert kind of piece of code that is capable to open a man page everytime you execute
just foo help or just foo --help
That was the idea we were thinking about. And I was thinking to have a very generic code that pass the name of the recipe to pass it to the man command or whatever... Does it make sense?

So, you want a command that shows custom documentation for your justfile?

Why not just help foo?

Theres is nothing better than talking with people and having other point of view, thanks probably that would help

So, you want a command that shows custom documentation for your justfile?

Yes, we are thinking in a README with pandoc, so the readme can be read in github but also from the just command. once again these are just ideas, thank you for your help!


Apply Jonathan Gallimore's JMX Patch to 7.0.5

Backport this to 7.0.x
https://github.com/apache/tomee/pull/160/files

Nuts, this PR breaks hawt.io FYI

Interesting - you got any more details? I'm happy to take a look.
On Wed, Sep 19, 2018 at 5:09 PM Jonathan S. Fisher<EMAIL_ADDRESS>wrote:

Nuts, this PR breaks hawt.io FYI
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/apache/tomee/pull/166#issuecomment-422862585, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAhfENtKGbJJpanHtaaltNy6qh9pyQ6Cks5ucmxOgaJpZM4WqaCr
.

Yes, to reproduce:

Build this branch from current master
Unzip the plume tar.gz
Download https://oss.sonatype.org/content/repositories/public/io/hawt/hawtio-default/2.1.0/hawtio-default-2.1.0.war to webapps/hawtio.war [the filename hawtio.war is important, must be exactly that]
start the server with catalina.sh run
navigate to localhost:8080/hawtio and see jmx is working
stop the server
deploy the tomee.xml I emailed to you
restart the server
nagivate to localhost:8080/hawtio and see the jmx section is missing

It seems to revolve around the setting of a remote broker rather than using a local broker. The tomee.xml I sent you has a remote broker and breaks hawtio, but if you use a vm://localhost it works fine. Strange

Hey I think I know what's causing the problem...it looks like the patch set we applied causes a NullPointerException when clicking on the AMQ RA.
Hawtio or jolokia might not be able to handle that.

It looks like on the ActiveMQ resource adapter, mBeanInfo.getAttributes() is returning null, breaking hawtio. They should obviously handle this case, but also guessing we're probably not supposed to be doing that either:

@exabrial - fixed here: https://github.com/apache/tomee/commit/e1853b10c52f72c7e9dd43c049098d7f69a43a49 thanks for the report!

Thanks for the fix! I'm surprised the jvm allowed the nulls.


Messages are getting stuck in mpool

Describe the bug
When running a Forest node on calibnet, the message pool (mpool) get slowly but steadily crowded with messages.
Forest's mpool is supposed to have a maximum size of 5000 messages to avoid DoS attacks, where nodes are spammed and run out of memory.
So if the node can get its whole mpool filled with those stuck messages this could lead Forest to stop processing messages quite simply.
However it turns out that we don't use this max_tx_pool_size or any other protection, exposing us to the aforementioned attack vector.
    /// This is max number of messages in the pool.
    pub max_tx_pool_size: i64,  // <- never used!

We need to address both issues.
To Reproduce
Steps to reproduce the behavior:

Init a new Forest node with calibnet snapshot
Wait for some time (one or two hours) and inspect mpool
See console output

Console output
$ ./target/release/forest-cli mpool pending --cids && ./target/release/forest-cli mpool stat
bafy2bzaceb763nepfmpi5tdf5mejdk5eucoa3s7bbtpmqklshxbc5obfurhoq
bafy2bzacedjmjcotou2r45h4uhz53tqksi5rxyhlv3cc63sgq4rdoj5z5pqbw
bafy2bzaceaehvo2kmd4q3dn7n2l4eou4es2ywwxrp7b3wnw3knxaxwbpw6zlg
bafy2bzaceaxo6cvlmyigjqpscu7yutvkeydfawb7s7jpw7qdxdatf5nsxrvhm
bafy2bzaceaiqabqrfyy7vgbqyfe64iqvbgd7d42jao765zvxetdzouffii2pm
bafy2bzaceareywffz76km3c4be4mg7tev7kmbsujwoqclwzg2iqwudsoi4zi6
bafy2bzacecycefhboujsoigndcsrmoxvtx5kuiayoltwpmtsvqq3fslabvblm
bafy2bzacedoh6uux6iuc466nkfohtfbizmeoxmn26y54yyfstd3t4d2yfvad4
bafy2bzacedwpstszwjm2kbbvtcvjfky7vns6stpsjxabruuoqyz5fbbdd7f46
bafy2bzacecyz7yw6ewngd5bjhfnhywehtwcfa6o7rxs6loazgct6pxgrb2q3a
bafy2bzacecftla6tnomq4ktbe2333y6wavp2dao5sz3kb7hqa5nig5u36x5v4
bafy2bzaceask3ercyoun7abbxybibkceaavpgcrwey35kko4mw6jgyapzoacw
bafy2bzaceaaszuxovhjqip7ybuqwbybmsonx5ob6b6fbdjwn33yjyu52dg57w
bafy2bzacebaah7y7gq5rvrfyubyfcltw4pza6yup2so35wzh3imxwkfy2ri76
bafy2bzacecw4hcwwfolshqfl5nfov4n6uoaf23il5i5vs2gwais6xo6hz6eua
bafy2bzaceakdsshbrifsldjlo3po6o7iepyrsyqwqo6nz3en7i2iawiqpeiji
bafy2bzacecvuq5jcipz7j5d7tuwery2dcacwhhrfjzf33g4vto7qegx5sqa7a
bafy2bzaceazzd3fbwvkvlkn4opjq6reeut33fapiulgtjubwfjbifev2bjtia
bafy2bzacebl6hr4gvobstfdxhkk2jgtghryyi5v4eehnmv27wqq3vbmouuvcc
bafy2bzacealobnaccrr55jtgkk43yubkj7ix2tf4bvoft3xnn252vrzw5aa34
bafy2bzaced3x23qepryljfwt2phvszf77zzccheqvdlc3co6j46aunt2uvn7w
bafy2bzacedrghsadirxkmlppnslzljdkc6nb5rxx6rt4k2zkchl5y72xrv7yc
bafy2bzacea73h4yrh2jyapaob3tuc6nxakhzdn5asjuljpfy2ldob2x5ydzbq
bafy2bzacechqklmz76vrpinrijlvk5txbiwzrmp2amdmb3otjdgsbpvfhmvmc
bafy2bzacedtsokwtd4go4qsqt2wwt7hyw4nkh3bjlm4645cuc27n6bu7jqbcs
bafy2bzacedbv5ev7zr2res5bou7n5gegqgzpdtulry2nau4vc7r3buv5ixvwg
bafy2bzacebfnzytabebqzozje2wwa5zuqwl52i2ougup7swim7vy32ftmmxje
bafy2bzacebz3wqdajfygd6zhfucffigdviv5ybwsbzb33ukat6ccltwobj2yw
bafy2bzacedsmtfwzxhr2xdigewphlcqzip2iaprwvbrf2o7vdkukd4kfpq4gc
bafy2bzacecmu3wdtzsigzkuy6shbyzz6liugo7gbfdy6os4hhm6qygy2v3zeu
bafy2bzaceby46bct45yoyxpixd6iz5wdzyowkzunnh3pzgyh5tndfo42migya
bafy2bzacecr4g2s3b6es5k56j322t3n3sng6iee5bco6vjc7xsasxu7m63hgc
bafy2bzacec544d5j2w7dklcjdy3mfh5t4yu3ndm53qg675fk2lj5caikaih36
bafy2bzacebalr4b54vh5wgezh4tmcnftwbrxxz2yy5qzsgqbaa3xr3evypncw
t1u2htzdz77nhhrial6cfdlekcou5cbhtmtbrzoaq: Nonce past: 3, cur: 0, future: 0; FeeCap cur: 0, min-60: 0, gasLimit: 175728384
t1uiyoi7rdnubtxtnt5swn632fdb3zw36j5xuviwq: Nonce past: 1, cur: 0, future: 0; FeeCap cur: 0, min-60: 0, gasLimit: 930953
t3qeydz3j3ayvjvognxoatc3vahaquz2mph4donx4kdu2mfkgsb2ebq6mslrsabkkqwub2rgtcomc56hri4aga: Nonce past: 1, cur: 0, future: 0; FeeCap cur: 0, min-60: 0, gasLimit: 22589408
t3qn7ask2us7xhhi5yoclb3pmpjofrv7lynnzia6dgdvj5bjdztnoyzobrk3rfmcpinpi55hdxvyiany4knw3q: Nonce past: 10, cur: 0, future: 0; FeeCap cur: 0, min-60: 0, gasLimit: 836985198
t3qupitigb6rownwycx64jge6sa34mjou2rv3xihs2gv2dwp7l4rpmhb2dsxibpabuu2qrlnoobyvmmkysekcq: Nonce past: 4, cur: 0, future: 0; FeeCap cur: 0, min-60: 0, gasLimit: 337096276
t3qxetyfmkkumqfdfxugexhsdzmdnmhrj4yo26sg2633yxqsqlsmddkz5q2urhq7crpa33nqmg6axtwxm2mnvq: Nonce past: 1, cur: 0, future: 0; FeeCap cur: 0, min-60: 0, gasLimit: 25074707
t3s6t7fnk6cjiuyyqeexruzixsca2zbdrd237aewt322ehblslq3cdtwrsb5p3hykewgf3rhv3etola2dtmjwa: Nonce past: 2, cur: 0, future: 0; FeeCap cur: 0, min-60: 0, gasLimit: 79621365
t3tc7bgy4fpmlrqnb62rfoe27b6hxmzkxddvd4igjh56txxculjxkbfds3r46of7oo5qd4rhbxg2fdgb6za4ea: Nonce past: 0, cur: 0, future: 2; FeeCap cur: 0, min-60: 0, gasLimit: 44593869
t3u3ekmkxmnp7vwgc4eifoxztrv6olerla7tsjcpd6376ug2f2pl3hukjkxc754gzsnvuilmtzo3kzxb4mmmua: Nonce past: 1, cur: 0, future: 0; FeeCap cur: 0, min-60: 0, gasLimit: 25241553
t3umcpde3adqvf6edjuxvqw77zibtqn3kpl6u2ewyxvgbq6elcxwaz6u23hzrvsrrg423aep6ho5bje3drt26q: Nonce past: 3, cur: 0, future: 0; FeeCap cur: 0, min-60: 0, gasLimit: 77099350
t3vm6qoze7ajlin3e6x2smphspvfmtrnxi4qp6hq45ljmrdeojn7dini6o6rd2y2yluycsevtx2tm57xhmnafq: Nonce past: 4, cur: 0, future: 0; FeeCap cur: 0, min-60: 0, gasLimit: 284333025
t3wt6c4wla5egncjsgq67lsu4wzu4xtnbeskgupty7udysbiqkr4sw6inqli2nazks2ypwwnmlahtkzd4ghjja: Nonce past: 2, cur: 0, future: 0; FeeCap cur: 0, min-60: 0, gasLimit: 50654328
-----
total: Nonce past: 32, cur: 0, future: 2; FeeCap cur: 0, min-60: 0, gasLimit:<PHONE_NUMBER>

A bit later:
...
total: Nonce past: 43, cur: 4, future: 3; FeeCap cur: 0, min-60: 0, gasLimit:<PHONE_NUMBER>

Expected behaviour

Messages should not get stuck
We must implement spam protection

Environment

OS: macOS
Rust version: 1.70.0-nightly
Branch/commit: 6050cc6399cf1e50472ec7856cb0003e95b326ad

Other information and links
Lotus never had its mpool exceeding ~10 messages while Forest's one was constantly growing.
MessagePool
TODO comment

Result of my initial investigation:
Consider following log of a Forest node on calibnet.
2023-05-04T11:32:30.067295Z  INFO forest_chain::store::chain_store: New heaviest tipset! [bafy2bzacedtn7fcy2fwy2oibegccu5bq6try5gzbtjjgwe6wrg4ir7mrk5mxi] (EPOCH = 529119)
2023-05-04T11:32:30.067707Z  INFO forest_chain::store::chain_store: New heaviest tipset! [bafy2bzacedus57iegr2xpf7xbsirougy5ar7vu2e4b3jpa7mw4nhero5c3ypu, bafy2bzacedtn7fcy2fwy2oibegccu5bq6try5gzbtjjgwe6wrg4ir7mrk5mxi] (EPOCH = 529119)
2023-05-04T11:32:30.067804Z  INFO forest_chain_sync::tipset_syncer: Successfully synced tipset range: [529118, 529119]
2023-05-04T11:33:00.048036Z  INFO forest_blocks::tipset: Weight tie left unbroken, default to [bafy2bzacea5ras4kslg5hconbaonujcm6yoyi4m3e6l3egu6ocny4cbub7uya]
2023-05-04T11:33:00.048620Z  WARN forest_chain_sync::tipset_syncer: Validating block [CID = bafy2bzaceaez7hf4lz4h7dkk2egdbt4pum6dxaf4xtgv5a6pjydtcbxbmxfik] in EPOCH = 529120 failed: Loading tipset parent from the store failed: Key for header not found
2023-05-04T11:33:00.048646Z  WARN forest_chain_sync::tipset_syncer: Sync messages check state failed for single tipset
2023-05-04T11:33:00.048659Z ERROR forest_chain_sync::tipset_syncer: Syncing tipset range [529119, 529120] failed: Loading tipset parent from the store failed: Key for header not found
2023-05-04T11:33:01.507198Z  INFO forest_blocks::tipset: Weight tie broken in favour of [bafy2bzacecfrul5ri2ggwbgmrcmsfbfnc65nomcakileyok7hqbpxo6vyczmi]
2023-05-04T11:33:01.507670Z  INFO forest_chain_sync::tipset_syncer: Validating tipset: EPOCH = 529120, N blocks = 1
2023-05-04T11:33:01.507714Z  WARN forest_chain_sync::tipset_syncer: Validating block [CID = bafy2bzaced732r3eipj2fesqt7vt4iksthm2qujqqqd6lzl5hjt2mnaqaw43o] in EPOCH = 529120 failed: Loading tipset parent from the store failed: Key for header not found
2023-05-04T11:33:01.507732Z  WARN forest_chain_sync::tipset_syncer: Sync messages check state failed for single tipset
2023-05-04T11:33:01.507741Z ERROR forest_chain_sync::tipset_syncer: Syncing tipset range [529119, 529120] failed: Loading tipset parent from the store failed: Key for header not found
2023-05-04T11:33:30.701583Z  WARN forest_chain_sync::tipset_syncer: Validating block [CID = bafy2bzacea3zaonwshrt5jbmqctyufamht3mqj3ndgjx2vsghr2t2p4ta4pvc] in EPOCH = 529121 failed: Loading tipset parent from the store failed: Key for header not found
2023-05-04T11:33:30.701603Z  WARN forest_chain_sync::tipset_syncer: Sync messages check state failed for single tipset
2023-05-04T11:33:30.701613Z ERROR forest_chain_sync::tipset_syncer: Syncing tipset range [529119, 529121] failed: Loading tipset parent from the store failed: Key for header not found
2023-05-04T11:33:31.527452Z  INFO forest_blocks::tipset: Weight tie left unbroken, default to [bafy2bzaceckxdkiitpszrn6voswbcjq3ncqmz3ux4s5jhlbuiosr7rvcqsuqq]
2023-05-04T11:33:31.527463Z  INFO forest_blocks::tipset: Weight tie broken in favour of [bafy2bzaceckxdkiitpszrn6voswbcjq3ncqmz3ux4s5jhlbuiosr7rvcqsuqq]
2023-05-04T11:33:31.528125Z  INFO forest_chain_sync::tipset_syncer: Validating tipset: EPOCH = 529121, N blocks = 1
2023-05-04T11:33:31.528142Z  INFO forest_chain_sync::tipset_syncer: Validating tipset: EPOCH = 529121, N blocks = 1
2023-05-04T11:33:31.528159Z  WARN forest_chain_sync::tipset_syncer: Validating block [CID = bafy2bzacebzeufi7dkpsle6ys2hojn2cogxm7tajqeoqzgbkz3wcdu3j3gzi2] in EPOCH = 529121 failed: Loading tipset parent from the store failed: Key for header not found
2023-05-04T11:33:31.528179Z  WARN forest_chain_sync::tipset_syncer: Sync messages check state failed for single tipset
2023-05-04T11:33:31.528188Z ERROR forest_chain_sync::tipset_syncer: Syncing tipset range [529119, 529121] failed: Loading tipset parent from the store failed: Key for header not found
2023-05-04T11:34:00.907991Z  INFO forest_chain_sync::tipset_syncer: Successfully synced tipset range: [529119, 529122]

The tipset found for the epoch 529119 is [bafy2bzacedus57iegr2xpf7xbsirougy5ar7vu2e4b3jpa7mw4nhero5c3ypu, bafy2bzacedtn7fcy2fwy2oibegccu5bq6try5gzbtjjgwe6wrg4ir7mrk5mxi]
A head change event is triggered with associated tipset:
https://github.com/ChainSafe/forest/blob/main/blockchain/message_pool/src/msgpool/msg_pool.rs#L239
Forest message pool can react to this event and remove all mined messages inside the two blocks of this tipset:
https://github.com/ChainSafe/forest/blob/main/blockchain/message_pool/src/msgpool/mod.rs#L244
(via calls to remove_from_selected_msgs)
Then later, Forest tries to validate tipset at epoch 529120:
Validating tipset: EPOCH = 529120, N blocks = 1
This fails (it's a legit error):
2023-05-04T11:33:01.507741Z ERROR forest_chain_sync::tipset_syncer: Syncing tipset range [529119, 529120] failed: Loading tipset parent from the store failed: Key for header not found
2023-05-04T11:33:00.048646Z  WARN forest_chain_sync::tipset_syncer: Sync messages check state failed for single tipset

But this also means the head change event for this tipset won't be triggered.
The next time it will, will be for tipset at epoch 529122 (last line of the log).
As a result, messages mined in blocks at epoch 529120 and 529121 and that have reached this Forest node will never be erased and pile up in the associated MsgSet hash map until forest daemon is restarted.
I will investigate next how Lotus mpool implementation works and can deal will messages removal.

Lotus doesn't have this kind of "single tipset" failure.
I think we should understand and fix this issue in the first place.


Conditional proposal

This provides the proposal for the conditional exports resolution.
I hope this clarifies the intent, further feedback welcome.

One more question, why package.json#default over an export keyed on . with no forward slash?

The main justification is a package that wants a browser and node main would be written:
{
  "exports": {
    ".": {
      "node": "./index.js",
      "browser": "./index-browser.js"
    }
  }
}

or it would be written:
{
  "exports": [{
    "node": "./index.js",
    "browser": "./index-browser.js"
  }]
}

So in terms of usability you would likely start out with "exports" as a string, then turn it into one of the above to support a browser condition. I just prefer not having two identical ways to do the same thing that are both aesthetically non-ideal, when we could just have a dedicated field. This is very much up for debate though and we may well end up without the "default".

I've updated this proposal to separate out the "default" field and just make it a conditional exports proposal.
I still believe a "default" condition would be nicer than the dot main for these use cases, but don't want to cloud the arguments here with these concerns.
I've also made the following updates:

Included a "main" condition which always matches
Gone back to object matching order (author defined over resolver defined)
Allowed nested conditions for composition
Supported conditions as acting like fallbacks themselves in that if a condition target is invalid it will fall through to the next condition.

Happy to reconsider arguments on any of the above too.

Thanks! Do we want to land as-is and iterate? I think the guts of the proposal have agreement, not sure if it's worth it to keep it open until it has been discussed more widely.

I've made those updates and also added a section on having a custom --environment flag for Node.js to include additional custom conditions. @jkrems if you have a moment to review this extra section that would be great, then certainly let's aim to merge soon.

a custom --environment flag for Node.js to include additional custom conditions.

I like the addition but I assume it would be a potential follow-up, not necessarily a day one feature?

@jkrems yes it doesn't need to be in the first implementation but I think it is an important feature for release to allow extension and not to be restricted to the core conditions. I've tried to explain this a bit better as well.


Middle Click doesn't Pick Block

When middle-clicking salmonberries in creative, they didn't show up in my hotbar.
On newest 1.19.2 Forge

Fixed in v3.1.3


Electrify on Windows Fails on Meteor Boot unless NPM Install Manually

Adding arboleya:electrify to Meteor projects causes them to crash on boot.  This happens in both Windows 8.1 and Windows 10 (separate computers).  This happens with pre-existing projects as well as the leaderboard example:

=> Started proxy.
=> Started MongoDB.
I20150914-18:49:10.001(-4)? electrify:  installing electrified dependencies
I20150914-18:49:10.297(-4)? electrify:  launching electron
=> Started your app.

=> App running at: http://localhost:3000/
   Type Control-C twice to stop.

W20150914-18:49:10.328(-4)? (STDERR)
W20150914-18:49:10.328(-4)? (STDERR) module.js:340
W20150914-18:49:10.328(-4)? (STDERR)     throw err;
W20150914-18:49:10.328(-4)? (STDERR)           ^
W20150914-18:49:10.328(-4)? (STDERR) Error: Cannot find module 'C:\Users\Mark Solters\Documents\GitHub\leaderboard\.elec
trify\node_modules\electron-prebuilt\cli.js'
W20150914-18:49:10.328(-4)? (STDERR)     at Function.Module._resolveFilename (module.js:338:15)
W20150914-18:49:10.328(-4)? (STDERR)     at Function.Module._load (module.js:280:25)
W20150914-18:49:10.328(-4)? (STDERR)     at Function.Module.runMain (module.js:497:10)
W20150914-18:49:10.328(-4)? (STDERR)     at startup (node.js:119:16)
W20150914-18:49:10.328(-4)? (STDERR)     at node.js:929:3

But, if I cd .\.electrify and then manually install NPM dependencies with npm install, everything works.  This took me a while to figure out by trial and error.  I'm not sure why this is not actually being done when it says "Installing electrified dependencies."

It should be related to my last patch. Weird enough, for some reason it worked fine here.
Which version of Windows are you using?

Btw, the patch in question solved in issue #8 apparently have worked and should not raise the error you've mentioned.

Which version of Windows are you using?
Did you test it with a plan leaderboard example? Just so I can replicate it here.

Both Windows 8.1 and 10.
Yes, I created the leaderboard example from the meteor cli, cd'd in and immediately added the electrify package. I did not alter any code.

I don't have any windows 8.1 or 10 machine, but I went through the whole test again in Win7 (32bits & 64bits), and it's working fine -- output below.
Curious that Electrify just enters the .electrified folder and executes a simple npm install. I was thinking that your problem may be some kinda of network permissions issue, like if the Electrify couldn't install things that needs to be downloaded, as the npm packages.
Makes any sense?
win7-64@WIN7-64-PC C:\Users\win7-64\Desktop\tmp
> meteor create --example leaderboard && cd leaderboard
leaderboard: created.

To run your new app:
  cd leaderboard
  meteor

win7-64@WIN7-64-PC C:\Users\win7-64\Desktop\tmp\leaderboard
> meteor add arboleya:electrify && meteor

Changes to your project's package version selections:
arboleya:electrify  added, version 1.2.2

arboleya:electrify: Package your Meteor apps with Electron, and butter.
[[[[[ C:\Users\win7-64\Desktop\tmp\leaderboard ]]]]]

=> Started proxy.
=> Started MongoDB.
I20150915-09:49:31.266(-7)? electrify:  setting up folders
I20150915-09:49:47.134(-7)? electrify:  copying template files
I20150915-09:49:47.134(-7)? electrify:  installing electrified dependencies
I20150915-09:49:47.134(-7)? electrify:  launching electron
=> Started your app.

=> App running at: http://localhost:3000/
   Type Control-C twice to stop.

[756:0915/094941:INFO:renderer_main.cc(200)] Renderer process started

I get the same errors if I run these commands (i.e. npm install and/or meteor) from a terminal that I've opened as administrator.  Not sure what other permission elevation one can do in a windows environment?

hi Mark @msolters
I had this same problem on a brand new installation of Windows 7 under VirtualBox.
I got the same error you did.
Maybe Anderson @arboleya had already installed node and npm using the installer from the official nodejs.org site? (If you still can't install it manually and get it to run)
So I went there and downloaded the *.msi file, and installed it.
I still had my Meteor project open in a command window, and tried to run Meteor again and see if I could get past the error messages. No luck.
However, after closing the command prompt window and re-opening it, I see that my path now included the npm folder.
I re-ran my Meteor project, and all went according to plan! Success!
So for anyone else looking for a resolution, just install the relevant package (architecture - I was prompted to download the 64-bit version) from nodejs.org and you should be good to go.
Thanks to Anderson @arboleya for all your work on this - amazing effort!  I can now produce apps for Web, iOS, Android, and with your Electrify package, apps for Mac Desktop, *nix Desktop and now, Win Desktop.
Meteor + Electrify FTW.

@bradzo Are you saying all you did was to install NodeJS & NPM?
All machines that this problem occurs on -- and that now includes at least one instance of Yosemite -- have both node and npm as valid commands in their path when Meteor is run.  I've tried both v4.0 (from the NodeJS) and v0.10.36 (the version that Meteor bundles with its own installations).
The presence of these commands is not the problem I am seeing.  The problem is that electron-prebuilt is not being properly installed by the electrify package.

Hi Mark @msolters
Essentially, yes. That's all I did. Just downloaded the .msi installer from the nodejs.org site and ran it.
Also, on Yosemite for me, the same Meteor app ran fine with Electrify - I didn't do anything extra, so I really can't help you there either... :(

Hello everyone, the question is that Electrify itself doesn't use the user installed version no NodeJS or NPM. The way it install the dependencies is through the node/npm binaries that are distributed along Meteor itself.
// https://github.com/arboleya/electrify/blob/master/index.meteor.js#L273-L275
  log('installing electrified dependencies');
  var cmd = ['cd', _ELECTRIFIED, '&&', _METEOR_NODE, _METEOR_NPM, 'install'];
  exec(cmd.join(' '), _SILENT_EXECS);

However the current error-reporting system is crap, the _SILENT_EXECS silent any error when it's not in dev mode. And enabling DEV mode requires lots of other settings to work properly.
Good news is I'm working on a new version which will be more stable and specially more easy about chasing cross-platform bugs. Stay tuned, I'll update here when it's available.

I totally understand, Meteor is still hanging back with some older exec() support.  I wish they'd just move up at least enough to use child_process.execSync!  In general, assuming Node modules will just run fine is never a safe supposition, so I appreciate the struggle.
Still, this explains why manually running npm install inside .electrify works -- even though it grabs the dependencies via whatever the user's system Node and NPM versions are, it does grab the missing files that constitute electron-prebuilt.

Hello @msolters @bradzo
I've just released a complete revamp version of the project, now it has tests and I'm confident it's way more stable than before.
Can you please update and report back if you still are facing the problem you've mentioned?
Some notes on upgrading:
https://github.com/arboleya/electrify#upgrading
Thanks in advance.

Awesome!
I tried it out on Windows 10.  Still no luck.  Now when I start my app up:
I20151003-03:14:25.851(-4)? INFO  electrify:app:  packaging app
I20151003-03:14:30.507(-4)? INFO  electrify:scaffold:  ensuring basic structure
I20151003-03:14:30.507(-4)? INFO  electrify:electron:  ensuring electron dependencies
I20151003-03:14:30.507(-4)? INFO  electrify:electron:  electron-prebuilt installation seems ok, moving on
I20151003-03:14:30.507(-4)? INFO  electrify:electron:  electron-packager installation seems ok, moving on
I20151003-03:14:30.507(-4)? INFO  electrify:plugins:  acquire
=> Exited with code: 8
W20151003-03:14:30.502(-4)? (STDERR)
W20151003-03:14:30.502(-4)? (STDERR) C:\Users\Mark Solters\AppData\Local\.meteor\packages\arboleya_electrify\1.3.4\npm\n
ode_modules\electrify\lib\app.js:103
W20151003-03:14:30.502(-4)? (STDERR)   var pkg_version = require(pkg_path).dependencies.electrify;
W20151003-03:14:30.503(-4)? (STDERR)                                                   ^
W20151003-03:14:30.503(-4)? (STDERR) TypeError: Cannot read property 'electrify' of undefined
W20151003-03:14:30.503(-4)? (STDERR)     at App.ensure_deps (C:\Users\Mark Solters\AppData\Local\.meteor\packages\arbole
ya_electrify\1.3.4\npm\node_modules\electrify\lib\app.js:103:51)
W20151003-03:14:30.503(-4)? (STDERR)     at Electrify.waterfal (C:\Users\Mark Solters\AppData\Local\.meteor\packages\arb
oleya_electrify\1.3.4\npm\node_modules\electrify\lib\index.js:45:13)
W20151003-03:14:30.503(-4)? (STDERR)     at C:\Users\Mark Solters\AppData\Local\.meteor\packages\arboleya_electrify\1.3.
4\npm\node_modules\electrify\lib\index.js:47:12
W20151003-03:14:30.503(-4)? (STDERR)     at C:\Users\Mark Solters\AppData\Local\.meteor\packages\arboleya_electrify\1.3.
4\npm\node_modules\electrify\node_modules\lodash\index.js:7305:23
W20151003-03:14:30.503(-4)? (STDERR)     at NodeJS.acquire (C:\Users\Mark Solters\AppData\Local\.meteor\packages\arboley
a_electrify\1.3.4\npm\node_modules\electrify\lib\plugins\nodejs.js:31:3)
W20151003-03:14:30.504(-4)? (STDERR)     at C:\Users\Mark Solters\AppData\Local\.meteor\packages\arboleya_electrify\1.3.
4\npm\node_modules\electrify\lib\plugins.js:28:12
W20151003-03:14:30.504(-4)? (STDERR)     at arrayEach (C:\Users\Mark Solters\AppData\Local\.meteor\packages\arboleya_ele
ctrify\1.3.4\npm\node_modules\electrify\node_modules\lodash\index.js:1289:13)
W20151003-03:14:30.504(-4)? (STDERR)     at Function.<anonymous> (C:\Users\Mark Solters\AppData\Local\.meteor\packages\a
rboleya_electrify\1.3.4\npm\node_modules\electrify\node_modules\lodash\index.js:3345:13)
W20151003-03:14:30.504(-4)? (STDERR)     at Plugins.acquire (C:\Users\Mark Solters\AppData\Local\.meteor\packages\arbole
ya_electrify\1.3.4\npm\node_modules\electrify\lib\plugins.js:27:5)
W20151003-03:14:30.504(-4)? (STDERR)     at Electrify.waterfal (C:\Users\Mark Solters\AppData\Local\.meteor\packages\arb
oleya_electrify\1.3.4\npm\node_modules\electrify\lib\index.js:45:13)

The server will recover from these errors, and eventually server the Meteor app.  But the Electron app will never start up.  In addition, I get the exact same STDERR output as above when I issue Electrify.app.package() from meteor shell.

Ah!  I have found the solution.  I just deleted all of my old .electrify, and then removed and re-added the arboleya:electrify.
Restarting the Meteor server with these fresh conditions, the new package proceeds to work as expected on Windows 10!

Glad to know :)


Add test for SR-4378

This snippet type checks in Swift 4 mode, but not in Swift 3.
It used to work in Swift 3.0, so something broke when we
implemented SE-0110 and redid the Swift 3 mode emulation.

@swift-ci Please smoke test

🙏


Are there any PictureOutput and MovieOutput yet?

I am using GPUimage well. Thank you. ^^
I am using GPUimage3.
GPUimage3, Can I still import filters and applied images and images?

Both of these are now present in the framework.


Fix 1039

BACKWARDS INCOMPATIBLE CHANGES!!! Softroots only cut 'downwards' now,
not 'upwards'. According to @evildmp that's the expected behavior.
@c4urself who reported #1039 should confirm that this is actually what
is expected.
The fix itself was kinda simple because the 'downward' cuts were clearly
marked in the code.
I also added more extensive and better documented tests.

seems sensible to me

excellent this is exactly what i expected

It seems to work as expected, thanks.


python310Packages.caldav: 1.0.1 -> 1.1.1

Description of changes
Diff: https://github.com/python-caldav/caldav/compare/refs/tags/v1.0.1...v1.1.1
Changelog: https://github.com/python-caldav/caldav/blob/master/changelog-1.1.md

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 216993 run on x86_64-linux 1

5 packages built:

calendar-cli
gtg
home-assistant-component-tests.caldav
python310Packages.caldav
python311Packages.caldav


Replace PERL/regex translator with python/clang, for generating pyopenvr module

Because this will make it easier for others to reason about the code translator: you will only need to be skilled in two programming languages: python and C++, instead of three.
My perl skill, at one time expert, is fading; and maintaining the old translate.pl is becoming more irritating.
Using clang makes the initial parsing more elegant. I had actually hoped that this would mean that there would be less translation code to write. But alas, the new translator is already more lines of code than the old translate.pl. Even so, the new parser is better compartmentalized.

Branch is here: https://github.com/cmbruns/pyopenvr/tree/cmbruns/clang_translate

Implemented for release 1.4.1801


Update docs to better describe support for floating point aggregation and NaNs

Adds partial support notes for aggregation expressions that have issues with NaNs and require spark.rapids.sql.hasNans to be false before floats and doubles can be supported.

build

build


[DOC] Remove usage instructions & update development environment section in README

Usage instructions removed from README to reduce repetition w/ official documentation

Checklist
This section is for the PR reviewer

[ ] PR has an interpretable title with a prefix ([ENH], [FIX], [REF], [TST], [CI], [MNT], [INF], [MODEL], [DOC]) (see our Contributing Guidelines for more info)
[ ] PR has a label for the release changelog or skip-release (to be applied by maintainers only)
[ ] PR links to GitHub issue with mention Closes #XXXX
[ ] Tests pass
[ ] Checks pass

For new features:

[ ] Tests have been added

For bug fixes:

[ ] There is at least one test that would fail under the original bug conditions.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>9
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 98.372%

Totals

Change from base Build<PHONE_NUMBER>2:
0.0%

Covered Lines:
967

Relevant Lines:
983

💛  - Coveralls


[stable/gcloud-sqlproxy] Allow specifying more than one instance.

Useful when you want the proxy to serve multiple instances on different ports.
I'm bumping chart version to 0.3.0 as this is a breaking change.

/assign @seanknox
cc: @rimusz

/ok-to-test

I have rebased to master and the tests pass. Please let me know if there's anything blocking this PR from getting merged.

cc @rimusz

/assign

/ok-to-test

/lgtm


QR (String > 251  characters)

Everything is fine until I exceed 251 characters in the string for Qr.
Any ideas?
thanks,
Esteban

sorry, this topic has already been closed here https://github.com/NielsLeenheer/EscPosEncoder/issues/6


Spurious "unclosed transport" and "Exception ignored in" message when connecting Unix pipes

If connect_read_pipe() or connect_write_pipe() is called with an object which is not a pipe, socket, or character device, a ValueError is raised indicating that other file types are not allowed. However, the partially constructed transport object triggers an incorrect "unclosed transport" error when del is called on it, and another exception is raised during that call due to the "_closing" member variable having not yet been set due to the constructor never finishing. Here's a test program which reproduces this:
import asyncio

class Writer:
    pass

async def run(loop):
    f = open('/tmp/x', 'wb')

    try:
        transport, protocol = await loop.connect_write_pipe(Writer, f)
    except ValueError as exc:
        print('Exception:', exc)

loop = asyncio.get_event_loop()
loop.run_until_complete(run(loop))
loop.close()

Here's the output I get when running this with Python 3.5.2:
Exception: Pipe transport is only for pipes, sockets and character devices
Exception ignored in: <object repr() failed>
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/unix_events.py", line 576, in __del__
    warnings.warn("unclosed transport %r" % self, ResourceWarning)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/asyncio/unix_events.py", line 456, in __repr__
    elif self._closing:
AttributeError: '_UnixWritePipeTransport' object has no attribute '_closing'

The first line of output is expected, but the "Exception ignored in" message is the problem.
Before Python 3.5.2, this would show up as an "Exception ignored in " message with no traceback, due to another exception raised when calling repr() on this object. This is also related to _closing not having been set yet.
It would be good to set always set _closing before any exception is raised, and probably not set _pipe until after this, so the "unclosed transport" message won't be triggered in del().

Do you feel up to making a PR?
--Guido (mobile)

Sure - I'll try and put something together for this later this week.

Closing this one since the relevant PR has been merged.


Beautify show detail page

Add two sections in the show detail page to represent similar, casts

Dear @u3frajaeian,
Thanks for your contribution. would you please update your branch with dev?

And we do do not need play button or navigating to player from shows detail page. You can remove it. (Episodes should have play button not Show details)

Dear @u3frajaeian, Thanks for your contribution. would you please update your branch with dev?

Hi it's already updated

And we do do not need play button or navigating to player from shows detail page. You can remove it. (Episodes should have play button not Show details)

Hi, I think we must have an intro for the series, and this button should play something like a trailer or intro.

Dear @u3frajaeian, Thanks for your contribution. would you please update your branch with dev?

Hi it's already updated

But the conflicts are not resolved yet.

Hi, I think we must have an intro for the series, and this button should play something like a trailer or intro.

Yes, we could consider having a "Trailers" button, But Using a play icon without accompanying text for different functionalities can be confusing. The button's label should clearly indicate its purpose so that users can easily understand its function. Also that is outside the scope of this issue. and if we want to add that button, it should navigate to another screen since Shows and movies usually have more than one trailers.

Dear @u3frajaeian, Thanks for your contribution. would you please update your branch with dev?

Hi it's already updated

But the conflicts are not resolved yet.

Hi, I think we must have an intro for the series, and this button should play something like a trailer or intro.

Yes, we could consider having a "Trailers" button, But Using a play icon without accompanying text for different functionalities can be confusing. The button's label should clearly indicate its purpose so that users can easily understand its function. Also that is outside the scope of this issue. and if we want to add that button, it should navigate to another screen since Shows and movies usually have more than one trailers.

Dear Reza, It appears that everything is already updated. After fetching the updated branches and attempting to merge remote/dev with my branch, I received a message stating: "Already up to date."

Dear @u3frajaeian, Thanks for your contribution. would you please update your branch with dev?

Hi it's already updated

But the conflicts are not resolved yet.

Hi, I think we must have an intro for the series, and this button should play something like a trailer or intro.

Yes, we could consider having a "Trailers" button, But Using a play icon without accompanying text for different functionalities can be confusing. The button's label should clearly indicate its purpose so that users can easily understand its function. Also that is outside the scope of this issue. and if we want to add that button, it should navigate to another screen since Shows and movies usually have more than one trailers.

Yes, you are right. I completely misunderstood the function of this button. I assumed the play button was responsible for showing trailers because Filmtime isn't a streaming application. However, based on your description, I will remove it.

Dear Reza, It appears that everything is already updated. After fetching the updated branches and attempting to merge remote/dev with my branch, I received a message stating: "Already up to date."

Hi @u3frajaeian, The message indicates that the branch is updated because it is being compared with your dev branch on your repository. When you create a PR against the original repository you forked from, you must sync your branch with the latest changes before creating the PR. Currently, there are many conflicts, as shown in the screenshot I sent you earlier.
To have a better overview of what I am telling, take a look at this screenshot:

Solution
I suggest you follow one of the options from this document and sync your repo.


Use stats poller service for vpp-agent forwarder metrics

Signed-off-by: Denis Tingajkin<EMAIL_ADDRESS>
Description

Motivation and Context

How Has This Been Tested?

[ ] Covered by existing integration testing
[ ] Added integration testing to cover
[ ] Tested locally
[ ] Have not tested

Types of changes

[X] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.

Should we mention this is vppagent related?

@nickolaev thanks, title updated

This PR is blocked.
Reason: I faced with unexpected behavior. I've created an issue in ligato repo: https://github.com/ligato/vpp-agent/issues/1590

The problem is solved on vpp-agent side. We need to wait new release.


HitTest utilities (for click-events)

Just the first underlying utility -- given a Shape -- did a mouse X,Y intersect it (with tolerance)

working on performance testing now ...


saved model size exceeds 800MB using TF-hub Universal Sentence encoder.

Hi,
Here is the code.
everything works fine model got exported also as a saved model.
But the saved model size increased to 870MB.
Is there any way to reduce the model size?
Implements the Keras model for Text Classification.
`
from tensorflow.python.lib.io import file_io
from tensorflow.keras.layers import Conv1D, Dropout, Dense, Flatten, MaxPooling1D
from tensorflow.keras.models import Model, Sequential
import pandas as pd
import numpy as np
from tensorflow.keras.layers import Conv1D, Dropout, Dense, Flatten, LSTM, MaxPooling1D, Bidirectional
from tensorflow.keras.callbacks import TensorBoard
import tensorflow_hub as hub
from tensorflow.keras.models import load_model
from trainer.utils.fs_resolver import resolve_file_path
Path(estimator.eval_dir()).mkdir(parents=True, exist_ok=True)
import os
import tensorflow as tf
import functools
`
def get_callbacks(config):
"""
A single callback to save the model h5 & tensor-board training logs during training.
"""
h5_model_path = os.path.join(resolve_file_path(
    config["ml_cloud_config"]["job_dir"]), config["_id"] + ".h5")
tb_log_path = os.path.join(resolve_file_path(
    config["ml_cloud_config"]["job_dir"]), 'tensor-board-logs')

tb_log = TensorBoard(
    log_dir=tb_log_path,
    histogram_freq=0,
    write_graph=True
)

checkpoint = tf.keras.callbacks.ModelCheckpoint(h5_model_path,
                                                save_best_only=True,
                                                save_weights_only=False,
                                                verbose=1)
callbacks = [checkpoint, tb_log]
return callbacks

def model_fn(training_config):
"""Create a Keras Sequential model with layers.
Args:
training_config: (int) model_definition object.
Returns:
A Keras model.
"""
embedding_size = 512
model = Sequential()
model.add(Conv1D(32, kernel_size=3, activation='relu', padding='same',
                 input_shape=(embedding_size, 1), name="input"))
model.add(MaxPooling1D(pool_size=3))
model.add(Flatten())
model.add(Dense(256, activation='relu'))
model.add(Dropout(0.25))
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.25))
model.add(Dense(6, activation='softmax'))

model.compile(loss='categorical_crossentropy', optimizer='adam',
              metrics=['accuracy'])
model.summary()
return model

def dataset_fn(filename_pattern, config):
def read_csv_file(filename):
    with file_io.FileIO(filename, 'r') as f:
        df = pd.read_csv(f, header=0, names=['category', 'free_text'])
        return df

filenames = tf.gfile.Glob(filename_pattern)
dataframes = [read_csv_file(filename) for filename in filenames]
df = pd.concat(dataframes)
text_data = df['free_text'].tolist()[:10]
label = np.asarray(pd.get_dummies(df["category"]), dtype=np.int8)

embed_size = embed.get_output_info_dict()['default'].get_shape()[1].value
embed = get_embed()
with tf.Session() as session:
    session.run([tf.global_variables_initializer(),
                 tf.tables_initializer()])
    text_data = session.run(embed(text_data))

text_data = np.array(text_data)
text_data = np.array([np.reshape(data, (len(data), 1))
                      for data in text_data])

print(label.shape, text_data.shape)
return text_data, label[:10]

def get_embed():
print("Loading TF hub module. It might take a while.")
module_url = "https://tfhub.dev/google/universal-sentence-encoder-large/3"
return hub.Module(module_url)

def serving_input_receiver_fn():
input_ph = tf.placeholder(tf.string, shape=[None])
embed = get_embed()
embeddings = embed(input_ph)
embed_tensor = tf.convert_to_tensor(embeddings)
sentence_embedding = tf.reshape(embed_tensor, [1, 512, 1])
return tf.estimator.export.ServingInputReceiver({"input_input": sentence_embedding}, {'free_text': input_ph})

def train(config):
print(config)
batch_size = 500
no_epochs = 5
model = model_fn(config["training_config"])
model_input_name = model.input_names[0]
train_inpf = functools.partial(dataset_fn, resolve_file_path(config["featureset"]["test_train"]["train_path"]),
                               config)
eval_inpf = functools.partial(dataset_fn, resolve_file_path(config["featureset"]["test_train"]["test_path"]),
                              config)

train_spec = tf.estimator.TrainSpec(input_fn=train_inpf, max_steps=2000)
eval_spec = tf.estimator.EvalSpec(input_fn=eval_inpf)

estimator = tf.keras.estimator.model_to_estimator(
    keras_model=model,
    model_dir=os.path.join(resolve_file_path(config["ml_cloud_config"]["job_dir"]), 'tensor-board-logs')
)
tf.estimator.train_and_evaluate(estimator, train_spec, eval_spec)

model_export_path = resolve_file_path(config["model_export_path"])
estimator.export_savedmodel(
    model_export_path, serving_input_receiver_fn=serving_input_receiver_fn)
print("model exported @" + str(model_export_path))

You can try a different embedding w/ a smaller dimensionality for example: nnlm-en-dim50-with-normalization vs nnlm-en-dim128-with-normalization

You can try a different embedding w/ a smaller dimensionality for example: nnlm-en-dim50-with-normalization vs nnlm-en-dim128-with-normalization

But using these embeddings, model performance goes down ie. model is that accurate.
I want to host above model to google cloud ml engine.
But they have this limit of model size should not exceed 250 MB.
What are different options to reduce the size of the model ?

It’s resolved✌

@Aashish-1008,
Can you please share your resolution so that it will be useful to the community. Thanks in advance.

@Aashish-1008, can you mention how you solved this? I'm also struggling with this issue.

@rakeshmothukuru1 @jwsmithers  Any update on this issue? I'm also facing the same issue. Thanks in advance :)

@AjithPanja Have you heard anything - I'm also curious about this.


Feature/specialist resource provision

I tested it manually and there are a few issues with swapping between the school types

On check your answers if you select the alternative provisions options, then immediately edit class type directly, you cannot select alternative provisions again
If you switch between the school types in the school types task, it doesn't repopulate the alternative provision options and gives a validation error when trying to submit, not sure how we fix this with the current design though

Cpyress tests will break when this is submitted


IncludedPolyfills only includes the first element

Version of the Meziantou.Polyfill NuGet package
1.0.28
Description
I tried the whitelist option to only include a few polyfills.
For some reason only the first element is included.
<MeziantouPolyfill_IncludedPolyfills>T:System.Runtime.CompilerServices;T:System.Diagnostics.CodeAnalysis;T:System.Runtime.InteropServices;T:System.Memory</MeziantouPolyfill_IncludedPolyfills>

Reproduction Steps
Create a project and add the source generator to it.
Add
<MeziantouPolyfill_IncludedPolyfills>T:System.Runtime.CompilerServices;T:System.Diagnostics.CodeAnalysis;T:System.Runtime.InteropServices;T:System.Memory</MeziantouPolyfill_IncludedPolyfills>

In a property group
Other information
I am using Rider, maybe thats an unsupported IDE?

I'm not sure to understand what's happening in this case. It's not a bug in the parsing of the string, so I guess it's a limitation of MSBuild and source generators.
I've added the support of the | separator in 1.0.28. You can now replace the property with
<MeziantouPolyfill_IncludedPolyfills>T:System.Runtime.CompilerServices|T:System.Diagnostics.CodeAnalysis|T:System.Runtime.InteropServices|T:System.Memory</MeziantouPolyfill_IncludedPolyfills>

Let me know if this fix your issue

Note: the package will be available in a few minutes. nuget.org needs to index it first.

@meziantou so the latest version of the package is still 1.0.28 and even after a nuget cache clear I get the version without support for | could you maybe bump the version to 1.0.29 not sure you can actually update existing packages on nuget.

My bad, I forgot to update the package version in the commit. The new package will be available soon

The package is now on nuget. It needs to be indexed first, and it should be available in a few minutes

Works like a charm thank you <3

Seems like there are more issues apperantly "M: ..."  work just fine any suggestions?
I tried "M:System.Linq" should I maybe open another issue for this one?


Update Error Message When Starting MonikAI. (Also typos and capitalization errors)

Not quite sure what the problem could be, but just in case it can be hotfixed, here's what I've been getting since yesterday whenever MonikAI starts up. Everything functions as normally, however she just can't update on startup.
Also, the Antarctica dialogue has some typos, as well as one of the 'full screen' dialogues.
First one, "You know what ironic...Ants don't live in Antartica." should be corrected to, "You know what's ironic? Ants don't live in Antarctica."
The 'full screen' dialogue just has some capitalization errors with some I's being lowercase (i) instead of capitalized.

The update thing is weird, are you maybe running on a shared IP? GitHub (what MonikAI uses for downloads) might be rate limiting you.
For typos, visit the Spreadsheet.


Reminders not working on my own (private) repo

I have added the yml file with these contents to my .github/workflows/main.yml file in the private repo
name: Issue Management Workflow

 on:
   schedule:
     - cron: '0 9 * * *' # This line sets a schedule for the action to run. This example would run every day at 9:00 AM.

 jobs:
   reminder:
     runs-on: ubuntu-latest
     steps:
       - name: Issue Reminder
         uses<EMAIL_ADDRESS>
I then added a comment to an issue /remind me of this tomorrow which did not work. It did not work.
I tried a similar one on the very repo here (#9) and it worked.
I then copied the issues comment yml and scheduled yml files from this very repo to my own, tried the /remind me of this in 5 seconds there... did not work.
What am I doing wrong?

Can you confirm that the actions are running in your private repo? Maybe that's something extra that you have to pay for?

Hummm, I got to the actions settings and saw that it had restricted access, changed that and I got a comment in the issue such as "user has set reminder to..."
Now waiting for the actual reminder to kick in (as it is far down in the future)
Thanks...


Q: How do you generate the rotation (*_rot.txt) files?

The code does not say how the rotation txt files e.g. plant/plant_0163_rot.txt are generated.
How is this done please? Thank you.

The rotation files are generated at flare_estimation.cpp but with *_lrf.txt suffix. So, changing _rot.txt to _lrf.txt in dataset_preparation.ipynb should solve the issue. You can also find the change in https://github.com/tolgabirdal/qecnetworks/pull/10.

The following segment will also need to be updated (first 3 values must be ignored):
vecs[0] = [float(lrf[3]), float(lrf[4]), float(lrf[5])] # x
vecs[1] = [float(lrf[6]), float(lrf[7]), float(lrf[8])] # y
vecs[2] = [float(lrf[9]), float(lrf[10]), float(lrf[11][:-1])] # z - don't take last char since it is \n

Thank you for sharing this, it looks correct. I included this change in https://github.com/tolgabirdal/qecnetworks/pull/10.


The same parameter happens in multiple runs in a Sweep

Current Behavior
The config.yaml is as follows:
program: hpo.py
project: random_name
name: hqo
method: grid
parameters:
seed:
values: [0,1,2,3,4,5,6,7]
I ran 8 agents on 8 gpu manually, it gives me 8 runs. WHILE only 4 seeds are chosen (should be 8) as follows, it wastes half of my gpu!!!

Expected Behavior
It should gives 8 runs, each run has a unique seed in [0, 7]
Steps To Reproduce
No response
Screenshots
No response
Environment
OS:
Browsers:
Version:
Additional Context
No response

Hi @talenz thank you for reporting this! Can you please share a screenshot of your Sweep Overview page? also would it be possible to provide a code snippet of your agent and wandb.init calls and the part where you specify your GPU according to the seed number?

Usage of wandb.config in my hpo.py:
_wandb.init()
random.seed(wandb.config.seed)
torch.manual_seed(wandb.config.seed)_
I didn't specify the GPU according to the seed, while the GPU is specified in ENV CUDA_VISIBLE_DEVICES. I ran 8 times in 8 terminal use the same following command:
wandb agent user/project_name/59atlow4

Thanks @talenz for the additional information, can you also try running the agents by specifying the GPU as follows:
CUDA_VISIBLE_DEVICES=0 wandb agent 59atlow4
  CUDA_VISIBLE_DEVICES=1 wandb agent 59atlow4
  ...
  CUDA_VISIBLE_DEVICES=7 wandb agent 59atlow4

Would this work for you?

Thanks @talenz for the additional information, can you also try running the agents by specifying the GPU as follows:
CUDA_VISIBLE_DEVICES=0 wandb agent 59atlow4
  CUDA_VISIBLE_DEVICES=1 wandb agent 59atlow4
  ...
  CUDA_VISIBLE_DEVICES=7 wandb agent 59atlow4

Would this work for you?

That's exactly what I did !

Hi @talenz can you please send a screenshot of your Sweep overview page (as the one below) to check if seed gets all the values there? may I also ask where are you using the random.seed() and torch.manual_seed() values? thanks

Hi @talenz can you please send a screenshot of your Sweep overview page (as the one below) to check if seed gets all the values there? may I also ask where are you using the random.seed() and torch.manual_seed() values? thanks

I set seeds in yaml as follows:
seed:
    values: [0,1,2,3,4,5,6,7]

And the seeds is used in .py file:

random.seed(wandb.config.seed)
torch.manual_seed(wandb.config.seed)
cudnn.deterministic = True
cudnn.benchmark = False

Hi @talenz thank you for the additional information. The amount of estimated runs (8 GPUs * 8 parameters) points out that the issue is that each GPU tries out all the possible permutations. Have you deleted some experiments and then run the agents using the same sweep_id? that would explain why the run count is more than estimated runs.
Would it be possible to run the same code in single GPU and see if that would use all seeds? Also, could you please provide a bit more context on how you're doing the distributed training (frameworks, logging in one or multiple processes)?

Hi @talenz just checking in with you to see if there are any updates about this issue, and if you could provide any additional information regarding the above? thanks

Hi @talenz since we haven't heard back from you, I will go ahead and close the ticket for now. Please let us know if you still experience this issue, and we can re-open this and keep investigating.

same thing happened to me!!!


JBDS-4248 Uninstaller exits only on enter not any key pressed

Current coverage is 77.47% (diff: 100%)

No coverage report found for 1.2.x at c1af72d.

Powered by Codecov. Last update c1af72d...0784564


did you forget to call torch.distributed.all_reduce function in train_gpt2.py

did you forget to call torch.distributed.all_reduce function in train_gpt2.py ?

If you wrap in DDP it's automatic with the backward pass

Oh I see what you mean, regarding the loss logging. You're right.

Yea, to log exact loss, you may need that call.


fix(js_formatter): fix invalid formatting of long arrow function for AsNeeded arrow parens

Summary
Fixes: #1905
Test Plan

Cool

I updated this pr, PTAL for free~ @ematipico

it seems the ci block? i think it has nothing to do with this PR


Code coverage

There should be a maven plugin somewhere to measure this, e.g. with cobertura

https://codecov.io/?src=pr


[RFC] New Augmentation techniques in Torchvison

🚀 Feature
Inclusion of new Augmentation techniques in torchvision.transforms.
Motivation
Transforms are important for data augmentation :sweat_smile:
Proposals

[x] RandAugment Citations 306 #4348
[x] Cutout Citations 964 Superseded by RandomErasing
[x] MixUp Citations 1624 Code (Note it is CC-BY-NC-license) #4379
[x] CutMix Citations 437 Code  ICCV 2019 #4379
[x] TrivialAugment ICCV 2021 #4221
[x] AugMix Citations 157
[x] Scale-Jitter Citations 11 - code - benchmarks
[x] Simple CopyPaste #5825
[ ] Port SSD & SSDlite Augmentations and Mixup/Cutmix from references to vision
[ ] Detection Transforms #1406 #2213 #3980
[ ] AutoAugment Detection 231 Citations code
[ ] Greedy Search Policy
[ ] FastAutoAugment Citations 135 NeurIPS 2019 Code (1k+ GitHub stars)
[ ] ReMixMatch Citations 165
[ ] FixMatch Citations 258
[ ] Transforms for color spaces augmentations E.g. rgb2hsv, rgb2bgr, rgb2lab, etc #4029

Additional context
To visitors
Kindly give a  :+1:  if you think any of these would help in your work.
Also if you have any transform in mind please provide few details here!
Linked to #3221
cc @vfdev-5 @fmassa

Would be nice to have these Augmentations in torchvision.transforms. 😄
Also found this official code implementation for Cutout.

@oke-aditya Do we need the cutout given we have Random Erasing which can be configured to have the same more or less effect?

I think the same. I compared both the implementations. RandomErasing is newer than Cutout, also both the augmentations produce almost similar results.
Also, As per docs RandomErasing does not work for for PIL Images. It works only for torch.Tensor. I am not sure if that is intentional or needs some work.

Not a transform idea but what about adding an optional 'target_transforms' argument in transforms.RandomApply? That way random imgs transforms and their targets equivalents could be applied at the same time.
The current way of doing so (from what I know) is by writing your own class and using functional transforms. For example :
class AugmentExample:
    def __init__(self, p=0.1):
        self.p = p

    def __call__(self, img, box):
        if self.p < torch.rand(1):
            img = img.flip(-1)
            box = box_hflip(box)
        return img, box

Also having a lot more keypoints/bbox transforms would be really great (ideally any image transform that involves a transformation of the targets should be accompanied by one?).
(Sorry if my english isn't right, I speak baguette.)

@datumbox is closing this issue intended?
As I understand there is dataset and transforms rework. Which would be a major refactor
Do we plan to migrate all transforms to new ones in near future?
(I had a minor look at the proposal which looks fantastic)

Not at all intended; Github just closed it when I merged the PR. We are certainly not done here :D

@lezwon Thanks a lot for offering to help!
ReMixMatch focuses on learning augmentations and on using unlabelled data. One challenge with that is that the majority of the changes will have to land on references which are outside of TorchVision. Currently the reference scripts are in need of some rework to reduce the amount of duplicate code and improve the overall quality. It's on the top of our todos and until that's done, ideally we would like to avoid introducing significantly complex techniques like ReMixMatch.
I wonder if you would be interested in implementing the AutoAugment Detection algorithm listed above. @vfdev-5 has already added most of the necessary low-level kernels for doing transforms on the BBoxes in torchvision.prototype, so what's needed is to implement the AutoAugment technique itself. Of course since it touches prototype APIs it can be tricky too. Let me know your thoughts and perhaps Victor can also pitch in to see if it makes sense to work together and test the new API. Alternatively we can discuss for another contribution that you find interesting.
BTW I'm currently working on the SimpleCopyPaste contribution trying to if we can train more accurate models using it. I'll let you know when I have the full results. :)

@datumbox AutoAugment sounds good. I'll start looking into it. :) Also, I noticed your comment on SimpleCopyPaste PR. Lemme know if I can help in any way :)

@lezwon Fantastic! Just note that I'm talking about this version designed for detection: AutoAugment Detection. This is different from the already supported algorithm for classification. :)

Without talking about transforms implementation in the prototype (how input type dispatch would happen etc), the only thing I think we could miss to implement AA Detection is bbox_only_* color aug. I do not think that it is something complicated to implement.

May I ask what's the current plan for the Fast AutoAugment?
I have the implementation of another paper called Faster Autoaugment:
https://github.com/ain-soph/trojanzoo/tree/main/trojanvision/utils/autoaugment.
It's the reimplementation based on autoalbument.
Are maintainers interested in embedding this technique as well? If interested, what's the expected api to use it?
If there is a similar plan for the Fast AutoAugment as the template, I'm glad to follow.


Support for other cricket matches and websocket information request

This looks awesome, and I'd like to use it to watch test matches, but I have to ask. What is on the other end of this web scoket? wss://criclive-72dee-43d33.firebaseio.com/.ws?ns=criclive-72dee-43d33&v=5
It looks like it's hosted by google. Can we know more about it? Can we adapt it for test matches around the world?

This code completely works for all cricket match's which are on live. This code includes all T20,ODI,TEST and even T10 match's.


Check if the rebase would add any workflows and/or actions and fail

I was thinking about the security of github workflows if you automatically merge any changes from a remote repository into yours. Maybe a workflow input would be helpfull to enable a functionality that checks if any files in .github/workflowsor .github/actions are going to be changed and if so fail the workflow. If the user of this action actually wants to have these changes as well it would be best to do that manually or only on workflow_dispatch (or disable the check).
If you think this is a good idea I'll be happy to provide a PR.

Do you mean the situation that the upstream have some workflows, and fork must enable Actions to use this repo, so it also runs workflows from upstream?
If so, I prefer placing the check in normal run phase, and provede it as an example in Readme, rather than integrated into this action.

Thats what I mean. I'm not sure how you could put that in the normal run phase as this would require the upstream repository to be checked out already.

I was thinking about something like
git diff upstream/master --name-status | grep .github && exit 1

Oh, you are right :joy:
Ok, you are welcomed to make the PR.

Apparently this is not necessary. A similar case just happened and github just denied the push with
 ! [remote rejected]   master -> master (refusing to allow a GitHub App to create or update workflow `.github/workflows/ansible-lint.yml` without `workflows` permission)

I didn't test, but I think Actions don't have that restriction by default.
Now I see GitHub has some ways to control the permission of github.token

https://docs.github.com/en/actions/security-guides/automatic-token-authentication
https://github.blog/changelog/2021-04-20-github-actions-control-permissions-for-github_token/

People can look into them if they are concerned about security.


Remove synchronous IO for metrics

Summary
This change removes the need for synchronous IO when collecting metrics.

Release Notes Entry

/backport to release/7.x


Issue with new bot update - can't instaskip and no audio?

Please tick all applicable boxes.

[x] I am using Python 3.5 or higher (run python --version on the command line)
[x] I have followed the official guides to install the bot for my system
[x] I have updated my dependencies to the latest version using the appropriate update_dependencies script

Which version are you using?

[x] The latest master version (1.9.7)
[ ] The latest review version

What type of issue are you creating?

[x] Bug
[ ] Feature request
[ ] Question

Description of issue
Hi there! I updated to the latest version of the bot today and after re-creating the options.ini and permissions.ini with the new options, I'm experiencing a few issues...
Firstly, any role within the MusicMaster role cannot instaskip any longer, and the bot won't output audio to Discord. (Region EUW and EUC tried).
Even the owner (me) can't instaskip.
Log attached.
I'm running on a Ubuntu instance on a free tier AWS Instance.
Steps to reproduce

Update
Create new permissions.ini and options.ini based on example files.
Run bot and get MusicMaster member to attempt to skip.

Log file
Please attach your MusicBot log file (located at logs/musicbot.log) to this issue. You can do so by dragging and dropping the file here. If you do not include your log file, you WILL be asked to provide one.
musicbot.log

Your instaskip problem isn’t an issue, the behaviour has changed. See https://github.com/Just-Some-Bots/MusicBot/releases/tag/1.9.7
As for your audio issue, try running update.py to update your dependencies.

The one time I skip reading a change log... Thanks a lot. I'll give it a
whirl.
On Thu, 22 Mar 2018, 23:26 Jayden Bailey<EMAIL_ADDRESS>wrote:

Your instaskip problem isn’t an issue, the behaviour has changed. See
https://github.com/Just-Some-Bots/MusicBot/releases/tag/1.9.7
As for your audio issue, try running update.py to update your dependencies.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Just-Some-Bots/MusicBot/issues/1550#issuecomment-375490341,
or mute the thread
https://github.com/notifications/unsubscribe-auth/APSf5Dpwf6M9A3Ibz9W5ZG1MamFXWsb2ks5thDMegaJpZM4S37Ht
.


Support "clearing" the Disruptor

Have a single method call that can stop all of the consumers, reset their sequence number to the cursor value of the ring buffer and restart them all.  This would only be available to users of the Disruptor class, which has a view of all of the consumers.  Should not impact performance of current fast paths and should race free.

How far with this?has any anyone taken this up?

Not as yet.  The user discussion that triggered this moved in a direction where I suggested a number of other options (https://groups.google.com/d/msg/lmax-disruptor/UFBGf7gps1Q/a1oQjpWfISIJ).  I didn't get any further response so I've not pursued this feature.

OK thanks

I am not going to implement this.  As preventing publication during this process becomes difficult without introducing additional performance cost.


⚠ BeyondHD has degraded performance

In db9904e, BeyondHD ($BHD) experienced degraded performance:

HTTP code: 200
Response time: 2653 ms

Resolved: BeyondHD performance has improved in 1e1d6b9 after 9 minutes.


Refactor LAPACK/BLAS signatures.

This PR merely aims to neaten the call sites where a LAPACK or
BLAS function is required, no external functional changes are
made. The LAPACK/BLAS type signatures and bindings are used in
multiple places in the np.linalg module and so it makes sense
for their definitions to be done upfront and only once, this also
reduces the SLOC required for each function making a call to one
of these libraries.

Why not make class _blas and _lapack into modules?  Things in __init__ can be module-level calls (thus import-time).

I think perhaps we leave making these into modules until there is a pressing need (most likely to occur if/when LAPACK is needed outside of np.linalg) and then think carefully about how to make importing efficient and functionally correct with respect to the environment? In the meantime I've adjusted the class names against PEP8 in e092173.

Current coverage is 86.97% (diff: 96.92%)

Merging #2008 into master will decrease coverage by 0.01%

@@             master      #2008   diff @@
==========================================
  Files           303        303
  Lines         54379      54495   +116
  Methods           0          0
  Messages          0          0
  Branches       5649       5652     +3
==========================================
+ Hits          47306      47398    +92
- Misses         6153       6169    +16
- Partials        920        928     +8

Powered by Codecov. Last update 7fd9d6e...e092173


Powerglitch license?

Hi, thanks for the cool app.
How is it licensed? There is no license specification in git repo!
Tnx! :)

Thank you for this issue, the license should have been added there already (Apache License 2.0) 😊


single datasource multiple indexers, native distributed indexing question

any timeline on native distributed indexing supported in single data source? what kind of native indexing are we talking here? APIs, files?

I am not sure what you mean. We support distributed indexing in single data source:

kafka
kinesis
ingest v2

Ingest v2 is the new version of our push API. It is available in main and in v0.8... but it is not the default.

@fulmicoton so do you mean single instance already support multiple indexer with ingest v2 in v0.8?


Automated Package Upgrade

:tada: This PR is included in version 2.2.207 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


New Environment - ccms-performance-tests

Environment details
The intention of this environment is to contain performance test scripts required by CCMS to evaluate performance in Pre-production.
Application Name
ccms-performance-tests
Description of application
The application consists of performance / load testing scripts written using Grafana k6. The performance tests will provide assurance that the Oracle SOA interface used by CCMS can scale to handle additional users as more providers are onboarded.
The test script will be run on an EC2 instance.
SSO Group Name
laa-ccms-devs
GitHub code owner team slug
laa-ccms-devs
GitHub actions reviewer team slug
laa-ccms-devs
Environments

[ ] Development
[ ] test
[x] Preproduction
[ ] Production

Environment access level Development
No response
Environment access level Test
No response
Environment access level Preproduction
developer
Environment access level Production
No response
application
Civil applications and billing
business-unit
LAA
infrastructure-support
<EMAIL_ADDRESS>owner
ministryofjustice
Subnet sets

[x] General
[ ] Isolated

How do users connect to the application
With a MoJ Official device
Additional features
Please check any additional features required

[ ] Additional VPC Endpoints
[ ] Extended DNS Zones
[x] Other - please specify

The performance test scripts will require access to the SOA service running at http://ccms-soa-managed.stg.legalservices.gov.uk:7777.
Other information
No response
Definition of Done
Definition of Done

[ ] account created
[ ] customer informed

Marked as blocked for the moment. The CCMS team is investigating the possibility of containerising this application and deploying to Cloud Platform instead.


Not loading and downloading does not work why

man, you saved me, I really wanted to delete the database

@9ofe man, you saved me, I really wanted to delete the database

@9ofe q If you want, I had such a problem I had to delete the database, I didn't write posts about it, but now I did and thank you very much 🔥😀😀


[#13477] Git commit message template and validation

Closes #13477

Yeah, I believe I've fixed it

Does the Jenkins release script need to be updated? The release commit message will not match the validation. (I think)

Yes, we need to add --no-verify where appropriate


Implement limusbook to ensure that the target is scheduled with its corresponding statefulset  application instance

In case of Statefulset applications using OpenEBS volumes, target and replica related resources need to have affinity with application. This means, for a given OpenEBS volume, target and replica related resources should be scheduled on the same node where application resides.

Include the label openebs.io/sts-target-affinity: <.metadata.name of STS> in the application manifest and deploy the application through litmusbook.

Litmusbook should validate if the storage target, replica and application instance are scheduled together on same node.

closed #624


Fix typo in README.md

--shut-up option is written incorrectly in README.md

--shut_up is the way to specify that option, the README is correct. Whether we should make the flag shut-up or shut_up is a different matter...


Improve docs of RelayPoolNotification::Shutdown

Describe the enhancement

Will Shutdown appear in the notification channel once RelayPool::shutdown is called or once all connections are closed? It would be nice if the docs answered that question.
Use case

Not having to check the source code to answer question above. Clients rely on it for proper app termination.
Additional context

Updated both RelayPoolNotification::Shutdown variant and RelayPool::shutdown method docs.


Remove big binaries and help files from git

It is not recommended to store large binaries into git.
Because git is architectured to store every version of every file, everytime you push a new large binary, you simply increase the size of the repository for the same amount.
Currently, this repository is already larger than 200MB compressed.
It is recommended to always push source files, and let every developer being able to recreate binaries from his disk, or during a specific build process.
My suggestion here is to remove (and add to .gitignore) all help related files.

[ ] remove and ignore the .chm file (66MB)
[ ] remove the output/ directory and add to .gitignore

I need your confirmation and advice about what could be removed and added to .gitignore. Perhaps some documentation files are outdated and legacy, perhaps some others are just out of date...
After that, I will try to find a way to also remove these binaries from the git history, so the project size could be reduced.

Ooops, wrong repo.


ci: don't skip test under macOS

Currently, there is one test being skipped under macOS. Since that does not seem to be necessary anymore, this PR re-enables the test for all platforms.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 82.63%. Comparing base (643eb35) to head (6548cf7).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #105      +/-   ##
==========================================
- Coverage   82.76%   82.63%   -0.14%
==========================================
  Files          11       11
  Lines         737      737
==========================================
- Hits          610      609       -1
- Misses        127      128       +1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Adjustments required as part of ECMWF ML PP WP1 Task 1.3

These changes are required in order to prepare anemoi for ICON data.

I made a new push with the setup that we are currently using to do inference with our unstructured ICON grid. At this stage the PR is rather a preview than an actual PR.