Enable constant and non-constant parameters for Spark SQL regex_replace function

PR enables the Spark SQL regex_replace function found here: https://spark.apache.org/docs/latest/api/sql/#regexp_replace
Spark implementation found here: https://github.com/apache/spark/blob/v3.2.2/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala#L521
Spark UT found here: https://github.com/apache/spark/blob/v3.2.2/sql/core/src/test/scala/org/apache/spark/sql/StringFunctionsSuite.scala#L131
Key note is this unit test works on dataframes row-by-row. Typical Velox implementations of regex expressions do not support non-constant parameters, a workaround needed to be introduced to match the unit test.
Function matches Spark implementation. Parameters match, position parameter is 1-indexed. Positions below 1 yield an error, positions greater than str.length() yield the same str.
There are 30 unit tests that can fall into one of two categories: constant and non-constant. Constant unit tests operate on a single "row". Non-constant unit tests operate on a collection of rows.
Unit tests are as follows:

Constant Basic Function Registration
Constant Empty String
Constant Bad Java Patterns (RE2 library does not support all Java regexes)
Constant Replace with Valid Position
Constant Replace with Negative Position
Constant Replace with Zero Position
Constant Replace with Too Large Position
Constant Replace with Special Characters
Constant Replace with Multiple Matches
Constant Replace Whole Word
Constant Replace with Pattern Before Position
Non-Constant Replace Registration
Non Constant Replace with Multiple Matches
Non-Constant Replace with Special Characters
Non-Constant Replace with Different Sizes
Non-Constant Replace with No Matches
Non-Constant Replace with Position Too Large
Non-Constant Replace with Empty String and Pattern
Non-Constant Replace with Different Character Types
Non-Constant Replace on Final Character
Non-Constant Replace Spark SQL UT
Non-Constant Replace Spark SQL UT with No Position
Non-Constant Replace Spark SQL UT with Massive Dataframe (300,000 rows)

The PR structure is overall in good shape now. But seems these 4 UTs failed in the CI check, can you resolve them first?
261: [  FAILED  ] 4 tests, listed below:
261: [  FAILED  ] RegexFunctionsTest.RegexReplaceRegistration
261: [  FAILED  ] RegexFunctionsTest.RegexReplacePosition
261: [  FAILED  ] RegexFunctionsTest.RegexReplaceMultipleMatches
261: [  FAILED  ] RegexFunctionsTest.RegexReplaceWord

I am wondering whether presto/spark have some different behaviors in this regex_replace function because if so, we will need move this implementation to spark functions.

Where are you seeing this? In the most recent run I am seeing all the tests pass.

@assignUser I am getting fails on tests that are not changed by my PR. Specifically in velox_cache_test. Is there anything I can do to resolve this?

https://github.com/facebookincubator/velox/pull/6681 was just merged and should fix the flaky test. So a rebase should fix your issue :)

All tests passed in current run after rebase. Error is now found in spark fuzzer, which is from code I have not changed
Error Source: USER
Error Code: UNSUPPORTED
Reason: Type not supported yet: VARBINARY
Retriable: False
Function: appendSqlLiteral
File: ../../velox/expression/ConstantExpr.cpp
Line: 213
Also seeing errors in uploading results

All tests passed in current run after rebase. Error is now found in spark fuzzer, which is from code I have not changed
Error Source: USER Error Code: UNSUPPORTED Reason: Type not supported yet: VARBINARY Retriable: False Function: appendSqlLiteral File: ../../velox/expression/ConstantExpr.cpp Line: 213
Also seeing errors in uploading results

@assignUser - Is this something you're aware of? We have not added fuzzer tests, and are not sure if this failure is due to any missing fuzzer testing we should do.

Yeah the fuzzer runs do fail on unreleated things, that's kind of in the nature of a fuzzer so no wrires. The upload fail seems familiar but I can't recall @kgpai wasn't it related to cci quotas or something?

Yeah the fuzzer runs do fail on unreleated things, that's kind of in the nature of a fuzzer so no worries. The upload fail seems familiar but I can't recall @kgpai wasn't it related to cci quotas or something?

Yeah the fuzzer runs do fail on unreleated things, that's kind of in the nature of a fuzzer so no worries. The upload fail seems familiar but I can't recall @kgpai wasn't it related to cci quotas or something?

Would also appreciate your reviews on this change, thanks. Please help include anyone else who is familiar with this area.

@codyschierbeck Thank you for iterating. Would you squash all commits. GitHub becomes slow when there are too many commits.

Made a mistake with rebase, please ignore

@mbasmanova Hi, I am having problems with squashing these commits. Whenever I rebase and squash into one commit, GitHub no longer recognizes the commits that I merged into this branch that fixed problems with velox_memory_test, and believes that I now have 380 new commits. I am not sure how to resolve this problem.

@mbasmanova Managed to resolve the issues I was having with squashing the commits. Apologies for the delay.

Is there any particular reason to implement this function as vector function? Can it be simple function instead?

@codyschierbeck Thank you for iterating. Still wondering about the above.
CC: @laithsakka

@assignUser Sorry bit out of the context - what upload failures are we talking about - is it fuzzer artifacts ? Those are best effort and if fuzzer crashes then there wont be artifacts to upload ; Like @assignUser said fuzzer can find bugs unrelated to the PR but it might still bear some investigation.

@mbasmanova Hello, in my latest commit I implemented the changes you requested, leveraged StringPiece to reduce the number of copies when dealing with position parameter string slicing, and added a regex pattern cache which reduced the RegexFunctionsTest.RegexReplaceMassiveVectors test runtime from ~3322ms to ~870ms on average.
This test case creates a RowVector that matches the 3-row Spark Unit test but concatenates it 100,000 times. The cache reduces 300,000 regex compilations to 2 (there is a repeated regex in the test case originally). Unfortunately, this cache does not resolve the problems/concerns you raised in #6804, especially if there aren't repeated regular expressions, but can be helpful in some corner cases.
If we do come up with a solution for many unrepeated regexes, I would be glad to implement them into this function.

@codyschierbeck I haven't looked at the latest code yet, but would you clarify whether there is a cap on number of regex compilations per function instance. If there is no such cap, let's introduce it.

@mbasmanova @laithsakka I implemented most of the changes requested. Kept a single change as it resolved a bug I was running into, explained in a comment below @laithsakka question about it

@mbasmanova Apologies, I was out for the past week. I have implemented the changes you requested as far as I can tell. Please let me know if I missed anything.

@mbasmanova Is this pull request ready for sign-off or are there other changes you suggest?

@codyschierbeck I'm running behind on PRs. Will try to catch up tomorrow, but, please, ping me on Thu in case you don't hear from me.
CI is red. Please, take a look.

@codyschierbeck I'm running behind on PRs. Will try to catch up tomorrow, but, please, ping me on Thu in case you don't hear from me.
CI is red. Please, take a look.

@mbasmanova Hi, I fixed the formatting issues that were causing the CI failures. The only remaining ones are the fuzzer (discussed previously) and uploading results. I did notice that I missed some review comments so I force-pushed those changes, including the additional non-ascii test.

@codyschierbeck Looks like fuzzer failure is related to this PR. Please, take a look.
E20231026 18:35:24.263033 198765 Exceptions.h:69] Line: ../../velox/functions/sparksql/RegexFunctions.cpp:145, Function:call, Expression: -1 < position - 1 (-1 vs. -5432941019078731721), Source: RUNTIME, ErrorCode: INVALID_STATE
E20231026 18:35:24.264618 198765 ExpressionVerifier.cpp:176] Common eval: Exceptions other than VeloxUserError are not allowed.

hi @mbasmanova @laithsakka, went through and added changes that I missed, and rebased on recent changes to fix the fuzzer issues I was running into. Please take a look at let me know if you think this PR is ready.

@mbasmanova @laithsakka I made some changes to the testing functions to make the code less verbose. I also implemented the requested changes. Please take a look and let me know what you think.

@rui-mo

@codyschierbeck CI is red. Would you take a look?

@codyschierbeck CI is red. Would you take a look?

Hi @mbasmanova, I took a look at the fuzzer in linux-build and I don't understand why it is failing. The fuzzer terminates after RegexFunctions.cpp line 153 which is the VELOX_USER_CHECK_GT(charLength, 0, "regex_replace encountered invalid UTF-8 character"); line. This error is expected to be thrown.
I tried to replicate this locally but the fuzzer uses ${RANDOM} for seed. I looked through the rest of the fuzzer and it seemed fine but I still don't understand why it terminated after encountering an expected error.
In linux-pr-fuzzer-run I checked each VeloxUserError occurance and none of them referred to RegexFunctions.cpp, so I do not know why it failed there. It may have occured below line 103121 but that's as far as the big view would let me see.
I talked with @assignUser on Slack and let him know the issues I was running into.

@codyschierbeck Have you tried reproducing the failures using seed? See https://facebookincubator.github.io/velox/develop/testing/fuzzer.html#how-to-reproduce-failures

The error is:
I20231108 19:23:11.234970 224813 ExpressionFuzzer.cpp:1313] ==============================> Started iteration 150 (seed: 964558807)
I20231108 19:23:11.236704 224813 ExpressionVerifier.cpp:86] Executing expression 0 : upper(left((sha1("c0"),;~,"c1",doy(539141144)),2100894719))
I20231108 19:23:11.239480 224813 FuzzerToolkit.cpp:103] Exceptions match.
I20231108 19:23:11.239521 224813 ExpressionFuzzer.cpp:1353] Both paths failed with compatible exceptions. Retrying expression using try().
I20231108 19:23:11.239557 224813 ExpressionVerifier.cpp:86] Executing expression 0 : try(upper(left((sha1("c0"),;~,"c1",doy(539141144)),2100894719)))
I20231108 19:23:11.241263 224813 ExpressionVerifier.cpp:74] All results match.
I20231108 19:23:11.242923 224813 ExpressionVerifier.cpp:74] All results match.
I20231108 19:23:11.243070 224813 ExpressionVerifier.cpp:74] All results match.
I20231108 19:23:11.243163 224813 ExpressionFuzzer.cpp:1274] Retrying original expression on 78 rows without errors
I20231108 19:23:11.243212 224813 ExpressionVerifier.cpp:86] Executing expression 0 : upper(left((sha1("c0"),;~,"c1",doy(539141144)),2100894719))
I20231108 19:23:11.244624 224813 ExpressionVerifier.cpp:74] All results match.
E20231108 19:23:11.245645 224813 FuzzerToolkit.cpp:95] Only simplified path threw exception:
I20231108 19:23:11.246937 224813 ExpressionVerifier.cpp:346] Persisted input: --fuzzer_repro_path /tmp/spark_fuzzer_repro/velox_expressionVerifier_MMxTym --input_path /tmp/spark_fuzzer_repro/velox_expressionVerifier_MMxTym/input_vector --result_path /tmp/spark_fuzzer_repro/velox_expressionVerifier_MMxTym/result_vector --sql_path /tmp/spark_fuzzer_repro/velox_expressionVerifier_MMxTym/sql
terminate called after throwing an instance of 'facebook::velox::VeloxUserError'
  what():  Exception: VeloxUserError
Error Source: USER
Error Code: INVALID_ARGUMENT
Reason: (-1 vs. 0) regex_replace encountered invalid UTF-8 character
Retriable: False
Expression: charLength > 0
Function: call
File: ../../velox/functions/sparksql/RegexFunctions.cpp
Line: 153

The error is:
I20231108 19:23:11.234970 224813 ExpressionFuzzer.cpp:1313] ==============================> Started iteration 150 (seed: 964558807)
I20231108 19:23:11.236704 224813 ExpressionVerifier.cpp:86] Executing expression 0 : upper(left((sha1("c0"),;~,"c1",doy(539141144)),2100894719))
I20231108 19:23:11.239480 224813 FuzzerToolkit.cpp:103] Exceptions match.
I20231108 19:23:11.239521 224813 ExpressionFuzzer.cpp:1353] Both paths failed with compatible exceptions. Retrying expression using try().
I20231108 19:23:11.239557 224813 ExpressionVerifier.cpp:86] Executing expression 0 : try(upper(left((sha1("c0"),;~,"c1",doy(539141144)),2100894719)))
I20231108 19:23:11.241263 224813 ExpressionVerifier.cpp:74] All results match.
I20231108 19:23:11.242923 224813 ExpressionVerifier.cpp:74] All results match.
I20231108 19:23:11.243070 224813 ExpressionVerifier.cpp:74] All results match.
I20231108 19:23:11.243163 224813 ExpressionFuzzer.cpp:1274] Retrying original expression on 78 rows without errors
I20231108 19:23:11.243212 224813 ExpressionVerifier.cpp:86] Executing expression 0 : upper(left((sha1("c0"),;~,"c1",doy(539141144)),2100894719))
I20231108 19:23:11.244624 224813 ExpressionVerifier.cpp:74] All results match.
E20231108 19:23:11.245645 224813 FuzzerToolkit.cpp:95] Only simplified path threw exception:
I20231108 19:23:11.246937 224813 ExpressionVerifier.cpp:346] Persisted input: --fuzzer_repro_path /tmp/spark_fuzzer_repro/velox_expressionVerifier_MMxTym --input_path /tmp/spark_fuzzer_repro/velox_expressionVerifier_MMxTym/input_vector --result_path /tmp/spark_fuzzer_repro/velox_expressionVerifier_MMxTym/result_vector --sql_path /tmp/spark_fuzzer_repro/velox_expressionVerifier_MMxTym/sql
terminate called after throwing an instance of 'facebook::velox::VeloxUserError'
  what():  Exception: VeloxUserError
Error Source: USER
Error Code: INVALID_ARGUMENT
Reason: (-1 vs. 0) regex_replace encountered invalid UTF-8 character
Retriable: False
Expression: charLength > 0
Function: call
File: ../../velox/functions/sparksql/RegexFunctions.cpp
Line: 153

@mbasmanova  I used that seed and was able to reproduce both with the original VELOX_USER_FAIL line and the VELOX_USER_CHECK_GT.
I changed the VELOX_USER_CHECK_GT to VELOX_CHECK_GT and got more information from the stack, specifically the context and top-level context
terminate called after throwing an instance of 'facebook::velox::VeloxRuntimeError'
what():  Exception: VeloxRuntimeError
Error Source: RUNTIME
Error Code: INVALID_STATE
Reason: (-1 vs. 0) regex_replace encountered invalid UTF-8 character
Retriable: False
Expression: charLength > 0
Context: (sha1(c0), ;~:VARCHAR, c1, 116:BIGINT)
Top-Level Context: upper(left((sha1(c0), ;~:VARCHAR, c1, 116:BIGINT),<PHONE_NUMBER>:INTEGER))
Function: call
File: /home/sdp/pinterest/velox/velox/functions/sparksql/RegexFunctions.cpp
Line: 153
Neither of the contexts appear to be calling regex_replace.
I found SparkExpressionFuzzerTest.cpp in the stack trace: (_build/debug/velox/expression/tests/spark_expression_fuzzer_test(_start+0x24)[0x55e20cfbaec4])
In that file there appears to be list of skipFunctions to not run the fuzzer on. I added regex_replace to the list of skipFunctions in SparkExpressionFuzzerTest.cpp and re-ran the fuzzer with the given seed, and it still failed.
I will take another look tomorrow morning.

@assignUser Do you know why the fuzzer is calling regex_replace is in upper(left((sha1(c0), ;:VARCHAR, c1, 116:BIGINT),<PHONE_NUMBER>:INTEGER))? My best guess as to why the fuzzer is failing is because it expects an output from the above expression but at some point (sha1(c0), ;:VARCHAR, c1, 116:BIGINT) calls regex_replace with an invalid input causing regex_replace to throw an exception that the fuzzer does not expect.
I looked at the sha1 spark implementation and I can't find any reason why it would call regex_replace

Hi @mbasmanova, could you help merge this PR? All validation is passing now.
For the issue you mentioned about empty function aliases, I have a draft PR located here #7569

@mbasmanova What are the next steps to merge this?

@codyschierbeck Last time I looked there were Fuzzer failures. Are these fixed now? If so, please, rebase and I'll try to merge.

@mbasmanova Were those the fuzzer failures that came from registering a function without a name? I resolved that issue with your recommended solution and then handled some merge conflicts we were seeing with 8e86d7e. The fuzzer doesn't appear to be failing right now, but I may be missing something.

Were those the fuzzer failures that came from registering a function without a name?

Yes. I was referring to these. If there are fixed now, then, please, rebase and I'll try to merge.

@mbasmanova Seeing a Facebook Internal-Linter issue. It says there is 1 advice and 1 warning. but I can't see details besides that.

@codyschierbeck A few linter warnings:

@mbasmanova Hi, is there a way for me to run this Linter on my end? I saw it failed again so I tested each import and found more unused. I don't want you to have to reimport and grab screenshots for each time it fails.
I have those changes ready, but I don't want to push in case there's something in this Linter run I missed.

@codyschierbeck

is there a way for me to run this Linter on my end?

This is a Meta's internal linter. I don't think there is a way to run it outside of Meta's infra.

@mbasmanova Okay, I will try pushing these changes and see if there is still a Linter error afterwords.


Struct generation issue

So sometimes when I'm running the generation I get this error,
expected Address to be a named struct, instead found string
and the interface will generate all the fields, but sometimes the generate runs successfully with no errors and its using the type I put in types.json.
I have been trying to figure out why this is happening, any clues?

If I change the name of the struct from Address to something else like AddressHolder this seems to go away.  Now I'm guessing that I have something else called address somewhere.

Ya that does sound like something that could be affecting me.  I do have other structs which have an address field that is a string.


feat: add the global constant "UNI_PLATFORM"

UNI_PLATFORM can help developers quickly determine the current platform enviroment

感谢，merged ~


Add GPS precision field

It would be nice to record the GPS precision, as shown on the map, with locations.

as far as "accurate within 5 meters" or "low", "medium" , "high"?

Stale issue

Stale issue


\Sammyjo20\Saloon\Http\SaloonRequest::getFullRequestUrl() does not contain query params

I assumed that the \Sammyjo20\Saloon\Http\SaloonRequest::getFullRequestUrl method returns, well, the full request URL. Fr me this includes the query params as well.
How can I retrieve the full URL used for the request from the response? 🤔
Using $request->getQuery() doesn't 100% feel right/reliable as theoretically the return value could change between running the request and asking for the used query string.

Hey @Gummibeer
Unfortunately, there is no way to get the final query parameters, except the "getQuery". You could grab them from within a request handler but I know that's messy.
What's the use case?

My response mock fixture file generator. As the query params are important for TMDB API as they control the response content.
But also useful for exceptions, logging and stuff like that to log the real full URL.

This although is a little misleading is currently going to remain unchanged. I have changed the name of the method in the v2 branch, as well as the PendingSaloonRequest that you will get in your fixtures will have all of the request properties like query parameters which you can trust.


[Enhancement]: There is no "HOME" button in CSS_SELECT GAME.

Do you want to have the enhancement of existing game ? 😀 Describe yourself..
Game enhancement
There should be an home button which navigates to home page
Describe the solution you'd like
Solution steps
i will add a home button which will navigate to home page
Select program in which you are contributing
GSSoC24
Code of Conduct

[X] I follow CONTRIBUTING GUIDELINE of this project.

Hey @TusharChamlikar !
Thank you for raising an issue 💗
You can self assign the issue by commenting /assign in comment 😀
Make sure you follow CODE OF CONDUCT and CONTRIBUTING GUIDELINES 🚀
Don’t Forget to ⭐ our GameZone🎮
Make sure you join our Discord🕹

/assign

Hey @TusharChamlikar ! Thank you so much for your raising the issue💗
It’s all yours, you can come anytime again and make some contributions! 🚀
Alone, we can do little, but together we can do so much! 😇


How to compute the same scores shown by Lighthouse for LCP and CLS?

Is there a way to produce the same metrics seen in Lighthouse?

Rather than trying to simulate lab data in the field, most people try to simulate field data in the lab, so I'm very curious to learn more about your use case. The lab-based Lighthouse data is just a simulation and may not reflect the way users actually experience the page.

Is there a way to produce the same metrics seen in Lighthouse?

Can you be more specific? Do you mean metrics like Time to Interactive, Total Blocking Time, and Speed Index? If so then the answer is no, those are lab-specific metrics and we don't recommend sites try to measure those in the field (e.g. measuring them is either too difficult or performance intensive).

I was just a bit surprised that numbers I collected from my workstation using the library and using Lighthouse devtools UI didn't match. To start with, I was not sure when to call getLCP so I figured I should use reportAllChanges = true, which produces a list of results. The documentation says (https://web.dev/lcp/#when-is-largest-contentful-paint-reported):

For analysis purposes, you should only report the most recently dispatched PerformanceEntry to your analytics service.

One of my website page has this late popup. The getLCP is sensed by this popup but Lighthouse is not because it stops measuring before the popup shows up. It is not clear to me when Lighthouse stops collecting data.
The reason why I'd like to get closer number is simple: if I'm using the same computer and same browser, shouldn't it produce similar results for both getLCP and Lighthouse?

@igorgatis Yes, Lighthouse waits until load: https://github.com/GoogleChrome/lighthouse/blob/master/lighthouse-core/gather/gather-runner.js#L75

I was just a bit surprised that numbers I collected from my workstation using the library and using Lighthouse devtools UI didn't match.

The numbers are not expected to match. Every page load consists of a unique network request and unique CPU conditions, so the expectation is that every load (even from the same machine) will be slightly different.  See the Lighthouse documentation on variability for more details.

To start with, I was not sure when to call getLCP so I figured I should use reportAllChanges = true, which produces a list of results.

It doesn't matter when you call getLCP(), it can be before LCP happens or afterward, the reported value will be the same for any given page load. You do not need to set reportAllChanges to true to get the correct LCP value, using reportAllChanges is only important if you really need to know what the progression of LCP candidates was.

One of my website page has this late popup. The getLCP is sensed by this popup but Lighthouse is not because it stops measuring before the popup shows up. It is not clear to me when Lighthouse stops collecting data.

To Rick's point above, this is one of the reason to favor real-user data in the field over lab data collected by tools like Lighthouse. And it's why Google tools that provide incentives based on Core Web Vitals score (e.g. Google Search) always base those scores on field data and not lab data.
Lab data can be used as a guide when debugging (or before releasing new code to production), but it is not the ground truth. The real performance of your site is what is actually experienced by your real users.

The reason why I'd like to get closer number is simple: if I'm using the same computer and same browser, shouldn't it produce similar results for both getLCP and Lighthouse?

For the same page load, yes the results should be the same. But running Lighthouse typically initiates another page load, and that is expected to produce a different value.

@philipwalton I was expecting some variance, but in my case it felt completely far apart because I was considering results after load event. Thanks for pointing that documentation. The cut off based on load event works perfectly for my use case.
Regarding getLCP, image that the page is composed of small parts the largest one triggers, say, at 2.0s. Around 10s later, a major popup shows up. AFAIU, if I call getLCP around time=5s, it LCP will be 2.0s and if I call it again around time=20s, LCP will be 12.0s. Is that correct?

In this case it seems like you'd want to measure beyond load so that you more realistically capture what users are experiencing and how it's reported in tools like CrUX. It's not that the field data is wrong; Lighthouse isn't seeing the bigger picture.

Regarding getLCP, image that the page is composed of small parts the largest one triggers, say, at 2.0s. Around 10s later, a major popup shows up. AFAIU, if I call getLCP around time=5s, it LCP will be 2.0s and if I call it again around time=20s, LCP will be 12.0s. Is that correct?

This is not correct. If the "real" LCP candidate for the page loads at 12 seconds, then that will be the element that is reported regardless of when you call getLCP().
Note that getLCP() doesn't return a value, instead it takes a callback that will be run once LCP has been determined. So it doesn't matter when you call getLCP(), what matters is when LCP actually was.
You can verify this yourself by setting up a test page with this exact scenario and trying it out.


Add React-Scroll AI Botin Readme

Gen AI chatbot for react-scroll. Will help Developers understand this repository better. You can ask for code examples, installation guide, debugging help and much more.

Thank you!


'Ingore' button in 'Bind parameter(s)' dialog not working as expected

dbeaver 6.2.3
the ignore tool tip says ignore parameters and execute script as is
but it doesn't work as expected: it always replaces the key with the value

Thanks for the bug report.
This function was broken in 6.2.3.

Verified


Not compatible with nvidia-tacotron?

I trained Multiband-Melgan model and intergrate with Nvidia-tacotron2 model, I also use this comment to make it work. But the results voice is bad with discontinuous pitch. The melspechtrogram below show the difference melspectrogram of output wave files of tacotron2+waveglow and tacotron2+MB_melgan (tacotron2+waveglow have great audio output). I try to replace the preprocess of this repos by nvidia-tacotron2 repos, but the results is same.
Tacotron2+waveglow:

Tacotron2+Mb_Melgan:

I also attached results audio.
results.zip
@kan-bayashi  Can you have any idea to fix this problem?

I want to clarify what is the problem.
When you synthesize the audio with natural features, how is the quality?
If the quality is still bad, we need to tune the hyperparameters of MB-MelGAN training.

When you synthesize the audio with natural features, how

The quality of audio which generated from natural features is very good.

Could you share the sample of MB-MelGAN with natural features?
If the audio sounds good, I think there are something mismatched between the models.
Please describe the feature extraction setting.

@kan-bayashi  Oke, I will report it tomorrow. Hope you help me.

@kan-bayashi
Here is the sample of MB-MelGAN with natural features (the audio sound is very good):
sample.zip
Melspectrogram of sample:

I also compare the audio and melspec output of nvidia-tacotron2 in tranning phase, and the input of MB-MelGAN in trainning phase, both audio and melspec is the same between nvidia-tacotron2 and MB-MelGan if I replace MB-MelGAN preprocessing stage.

The Feature extraction setting of MB-MelGAN:
###########################################################
FEATURE EXTRACTION SETTING
###########################################################
sampling_rate: 22050     # Sampling rate.
fft_size: 1024           # FFT size.
hop_size: 256            # Hop size.
win_length: 1024         # Window length.
# If set to null, it will be the same as fft_size.
window: "hann"           # Window function.
num_mels: 80             # Number of mel basis.
fmin: 80                 # Minimum freq in mel basis calculation.
fmax: 7600               # Maximum frequency in mel basis calculation.
global_gain_scale: 1.0   # Will be multiplied to all of waveform.
trim_silence: true       # Whether to trim the start and end of silence.
trim_threshold_in_db: 60 # Need to tune carefully if the recording is not good.
trim_frame_size: 2048    # Frame size in trimming.
trim_hop_size: 512       # Hop size in trimming.
format: "hdf5"           # Feature file format. "npy" or "hdf5" is supported.

I also compare the audio and melspec output of nvidia-tacotron2 in tranning phase, and the input of MB-MelGAN in trainning phase, both audio and melspec is the same between nvidia-tacotron2 and MB-MelGan if I replace MB-MelGAN preprocessing stage.

OK. How did you perform normalization?
Did you use nomralized features for both text2mel and vocoder model?

OK. How did you perform normalization?

In the case of use this comment I keep normalized features in trainning phase of MB-MelGan and use original preprocess of nvidia-tacotron2. In the inference phase, I generate melspec from tacotron2 and then convert it by using that code to compatible with Melgan, finally I generate audio from converted melspec.
In the case of replace preprocess of Mb-MelGan by nvidia-tacotron2 preprocess, I remove normalize procedure of MB-melgan both tranning and infererence stage.
In addition, I generate audio from one generated melspec output of nvidia-tacotron2 with Waveglow and MB-Melgan, and I see that the pulse of MB-Melgan output audio is not continuous:

In the case of replace preprocess of Mb-MelGan by nvidia-tacotron2 preprocess, I remove normalize procedure of MB-melgan both training and inference stage.

OK. Then, did you use the same files to train the vocoder and the model?
If you just replace the function, please try to generate audio using the mel-spectrogram file which exactly used for the training of tacotron2.

In addition, I generate audio from one generated melspec output of nvidia-tacotron2 with Waveglow and MB-Melgan, and I see that the pulse of MB-Melgan output audio is not continuous:

What is the difference compared to the sample you shared?
When I heard your sample, the audio quality is clearly different between GT and generated features.
So I doubt that there is a bug in your code.
But if the quality degradation is reasonable, that may be the problem of MB-MelGAN.

If you just replace the function, please try to generate audio using the mel-spectrogram file which exactly used for the training of tacotron2.

I did this way, and get the bad audio quality (same that quality result of tacotron2+MB-Melgan), so I will debug this point and report results here. Please, wait my response.

When I heard your sample, the audio quality is clearly different between GT and generated features.

the quality of GT audio and generated audio by MB-MelGan from natural features is same.

I did this way, and get the bad audio quality (same that quality result of tacotron2+MB-Melgan), so I will debug this point and report results here. Please, wait my response.

Then, there is a bug in your code.
Please carefully check the difference (e.g., log_e vs log_10).

The problem is resolved, I realized that the cause of the problem was because I kept the same fmax and fmin as the default configuration of this repos while fmax and fmin of nividia-tacotron2 is different.


semantic web

added semantic web conference type as well as eswc and iswc conferences

I adjusted data mining to data mining / web science. Do you think that's okay? Or do you have any other suggestions?


Feature Request: Allow a puzzle number in the future / after today

My friends and I like to play "Night Wordle" where we do a second word in the evening, and we always set the day to 1000 days in the future from today.  So if today's puzzle 262, "Night Wordle" is 1262.
Would you be willing to (in essence) eliminate like 44 of App.js?  This line checks if the number is in "the future" or not, and sets it to today if so.  I'd love to be able to set any day I want, including in the future.

Hi! I want the archive to be an archive in the truest sense and only allow past words, so this sadly won't be possible. Closing this for now.


Bingo: Add an exporter tool to make All Players Win prize-giving easier

This is a pretty HACKY way of accomplishing our goals, but it'll work! I need more backend experience...

@zfox23 trying to test this PR, i am not seeing any instructions how? can you ping me with some instructions of what i should run for the exporter tool? :)

Really cool stuff, added some possible SQL changes from what we talked about yesterday

closing to revamp the tool


How to run own Proxy in Heroku?

Hello,
this works great! I'll probably go back to Roam due to this 🗡
Any instructions on running the proxy on Heroku?
I thought the "Deploy" button would start with some kind of template, but that doesn't seem to be the case.
Thanks for this!

Tried again and now it Just Worked. Sorry for the noise.


Hotfix/ra units

Reorganizes some aspects of how the Model class interacts with the parent limepy models and clears up some plotting labels, all in the effort to make better sense of the ra parameter.
Firstly, it was not being noted that the ra parameter is actually logged.
Secondly, the scaling done by limepy wasn't entirely being accounted for correctly.
Nothing here changes the actual fit outcomes or models, but does change how you interact with the Model class.

We might want to continue to tweak, in the future, what exactly is displayed (in terms of scaled radii and etc.) but for now the important stuff is done here.


Right click menu in Total Commander

Hi is there some possibility to hook Shell into right menu in Total Commander ?

Im using win 11, shell is not working in another programs like everything, total commander etc.
It seems like the exclude setting is ignored.


Add regression test for issue#9007

Resolves #9007

Not needed. Found the disabled test. Will just enable it instead.


Compatibility with LangChain and Other Ideas

So I have been following the AI boom that's started the past 1-2 months.
I am personally very interested in decentralizing AI models and access. My project (https://lumeweb.com) is in the crypto/web3 space and I am building with https://hypercore-protocol.org. My project is combining a number of Lego blocks as well.
What I am interested in doing, which is something in the distant future ATM based on my current project roadmap, is enabling models to be downloaded from web3 content networks like IPFS, Sia, and Arweave, and then using HyperDB that was recently released to read training data.
This data would use hypercores which are basically append-only logs, shared BitTorrent style, aka P2P blockchains.
The result would be the ability to have fully decentralized AI access and distribution over web3. All you would need is the CID of a model and a hash/ID of a stream that would be a vector DB of information.
I am first interested in any thoughts or feedback on this idea, and second, as I can assume LangChain is definitely going to be getting on onboard with vector databases, and how this can integrate with the ecosystem they are building.

Kudos!

This is a great idea, we have updated the webllm to become a standalone npm package, so it can be composed with other projects


Allow slot-scope to be set on the defining component

What problem does this feature solve?
slot-scope can be set on a component that's a child of the component that defines the scoped slot.
<vue-parent>
  <vue-child slot-scope=props>
    {{ props.value }}
  </vue-child>
</vue-parent>

But it fails when it's set on the defining component itself. If there's only a default scoped slot then setting it on the parent should be allowed, because it would remove an unnecessary pair of tags in some cases and make the templates cleaner.
But mainly it's confusing and inconsistent to get an error that props isn't defined in the code below but not in the code above.
What does the proposed API look like?
<vue-parent slot-scope=props>
  <vue-child>
    {{ props.value }}
  </vue-child>
</vue-parent>

This is not possible if the vue-parent component is used inside of another vue-parent component. Which one scope are we referring to? The first one or the second one?
<vue-parent>
<vue-parent slot-scope=props>
  <vue-child>
    {{ props.value }}
  </vue-child>
</vue-parent>
</vue-parent>

I feel the need for this, but the ambiguity pointed out by @posva is indeed a problem.
We probably need some form of special syntax to get rid of the ambiguity:
<foo slot-scope.self="{ msg }">
  {{ msg }}
</foo>

Or
<foo $slot-scope="{ msg }">
  {{ msg }}
</foo>

It's confusing slot-scope not working  but scope can ？
@yyx990803 大大你好，我使用文档介绍的slot-scope=“props”，会报props not defined，而偶然只使用scope="props"却可以正常得到想要的结果。<EMAIL_ADDRESS>

I'm not convinced about .self because slot-scope is not a directive. Is there something wrong with a different name like self-slot-scope (apart from looking different from slot-scope while doing the same thing)?
$slot-scope is nice because it's short and relates clearly to slot-scope but I personally don't like the $ too much at the beginning because it has no semantical meaning 😆 thus, it doesn't read like most stuff does in Vue

@anselwong slot-scope requires 2.5

@posva that's true. Primary reason is that using self-slot-scope means it becomes a new reserved prop (i.e. user components can no longer use a prop with that name).

+1 for self-slot-scope over $slot-scope.

Hi! 👋 Is this in the v3 roadmap? I'm working on a collection of renderless components, and a self-slot-scope would really clean up my templates. I find myself adding more and more useless divs just to extract props from the scoped slot.
Thanks!!

Since we've shipped a new v-slot syntax, this issue can be closed. See https://github.com/vuejs/rfcs/blob/master/active-rfcs/0001-new-slot-syntax.md.

@yyx990803 您好，关于slot发现最新版本的Vue已经将$scopedSlots的值修改而没有在文档中提及。在代码中依赖this.$scopedSlots.default做判断的语法就会失效，理由是之前非作用域插槽的default并不会出现在$scopedSlots里面。
另外,关于低版本的插槽jsx语法如下：
slot
<tag>
  {this.$slot.default}
</tag>

slot-scope
// parent
<tag>
  {
    this.$scopedSlots.default({data})
  }
</tag>

// children
const scopedSlots = {
  scopedSlots:{
    default:scope=>(<span>{scope.data}</span>)
  }
}

<Parent {...scopedSlots}></Parent>

在Vue@3.x.x的时候,你们删除旧的插槽语法，上述的写法应该不再支持了吧？


Also wanted for this:

Fix sync ID
Allow large pulls (the core, OP's issue)
Allow log size customization through optional --log= (default to 20, much like git merge --log)

The git merge options --squash --log=2, as used in this commit 0c2879c, don't seem to work together. The --log=2 works when not combined with --squash. Examining the .git/SQUASH_MSG file shows that the entire remote log is taken again as in the original report by
wotwot. Tested with git version <IP_ADDRESS> and <IP_ADDRESS>. In what versions is this fix confirmed to work?


Is there going to be any movement on this?

Hi there,
Just wondering if there's any movement on this?
Would be nice to have an 'off the shelf' install for a headless version of the CMS and this was looking really promising.
Thanks,
Jack

Hi @jackkemmish,
CMS itself has just released version 5 and when installed via the CMS recipe it comes with GraphQL 4 module that can provide you with the content over an API, so the CMS itself can act in the headless capacity as a content provider already.
What's your use case, what do you want to build, what limitations around the current setup have you encountered? Or have you not try going with the existing version for your build yet?

Hi @michalkleiner thanks for the reply!
Currently downloading SilverStripe 5 as we speak 😃
In terms of my use case, I am just exploring how we can use a front-end framework such as Next.js with SilverStripe as I have quite a few sites built using the CMS, and would be nice to switch up my offering.
I originally installed it and there were some errors so I held off continuing, but if version 5 has has a GraphQL module installed I can take a look at the docs too.
Just thought it would be nice to have some off the shelf examples with SilverStripe best practises etc, not a full project, but a working example similar to how the likes of Prismic.io and Sanity.io do.
Thanks,
Jack

Fair enough. There might still be room for a module like this, it probably solves some headless-specific issues like returning correct error responses, image URLs and whatnot, haven't look at the code in detail, but standard CMS already comes with GraphQL, so I would give that a go first and then see what needs to be added/tweaked/ported from here.

Check out the GraphQL docs
https://docs.silverstripe.org/en/4/developer_guides/graphql/

Thanks for the messages @michalkleiner.
I will give SS5 and the GraphQL docs a go for now then thanks!


Added NoThunks instance for Identity

Added NoThunks instance for Identity

@bezirg could you resolve the conflict.

Closing in favour of #39.


Arch Linux: "Could not find avalid instrument. Sound is disabled" error

It looks like this repository has been abandoned. Perhaps look for forks or fork it yourself?


Use the fitpredict function in Interpolate and InterpolateNeighbors

Codecov Report

Merging #14 (2b62922) into main (d6cc688) will decrease coverage by 0.09%.
Report is 6 commits behind head on main.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main      #14      +/-   ##
==========================================
- Coverage   92.75%   92.66%   -0.09%
==========================================
  Files          14       14
  Lines         745      695      -50
==========================================
- Hits          691      644      -47
+ Misses         54       51       -3

Files
Coverage Δ

src/GeoStatsTransforms.jl
100.00% <ø> (ø)

src/interpneighbors.jl
100.00% <100.00%> (+5.66%)
:arrow_up:

src/interpolate.jl
100.00% <100.00%> (ø)

src/potrace.jl
98.65% <100.00%> (ø)

src/rasterize.jl
98.33% <100.00%> (ø)

src/uniquecoords.jl
75.00% <100.00%> (ø)

src/utils.jl
100.00% <100.00%> (ø)


Unprivileged user destroys pidfile with kong service script

Running 'kong stop' as a non-privileged user destroys the pidfile at /usr/local/kong/kong.pid, which means that the privileged user can no longer stop the process.
The gist file below shows the root user starting and stopping kong, followed by the unprivileged user destroying the pidfile and stopping root from managing the process.
https://gist.github.com/8d72e2ceb26fadf8f0dc.git
Looks like the perms on /usr/local/kong (where the pidfile lives) are too open (777 instead of 755)

@vacri thanks for the heads up. This problem will be fixed starting from the next version.


Please support mouse select, middle click copy and paste

Subject pretty much says it all.
Please support the Linux style select with mouse then middle click to paste operations.

Do you mean everything you select should be added to the clipboard history?

I believe other clipboards handle the X11 style copy and paste by adding it when it's pasted.
Not 100% sure how that works, maybe it's not easy.
Thanks for the reply.
On February 26, 2022 10:19:53 a.m. PST, Alex Saveau @.***> wrote:

Do you mean everything you select should be added to the clipboard
history?
--
Reply to this email directly or view it on GitHub:
https://github.com/SUPERCILEX/gnome-clipboard-history/issues/38#issuecomment-1052413358
You are receiving this because you authored the thread.
Message ID:
@.***>

Gnome doesn't have an API to know when you paste something AFAIK, but I can support putting everything you select in the history. Note that this has unintended consequences: if you drag select for example, every time a new character is selected that counts as a new selection (so you'll end up with a, ab, abc, abcd, etc in your history). That could maybe be fixed by adding an N ms delay before registering the selection, but it's still hacky.

Right, it sounds like it's a bit tricky.  Thank you for considering it.
 It would be great if it works but I understand the challenge.
Maybe it's best just to leave it as is.  I have such muscle memory for
select and middle click it's hard to remember to do the other.
Mostly useful in command line work in a terminal.
Cheers
George
On Sat, 2022-02-26 at 11:06 -0800, Alex Saveau wrote:

Gnome doesn't have an API to know when you paste something AFAIK, but
I can support putting everything you select in the history. Note that
this has unintended consequences: if you drag select for example,
every time a new character is selected that counts as a new selection
(so you'll end up with a, ab, abc, abcd, etc in your history). That
could maybe be fixed by adding an N ms delay before registering the
selection, but it's still hacky.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you authored the thread.Message ID:
@.***>

I mean, if you're ok with the shortcomings I mentioned then I can implement it fairly easily.

Sure why not, lets try it, maybe it could be an option switch so as not
to upset everyone:-)
On Sat, 2022-02-26 at 12:09 -0800, Alex Saveau wrote:

I mean, if you're ok with the shortcomings I mentioned then I can
implement it fairly easily.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you authored the thread.Message ID:
@.***>

BTW, is the primary clip supposed to be automatically pastable from Ctrl + V or just saved to the history. Right now I'm doing the latter.

There is Ctrl-C and Ctrl-V for cut and paste.  There is also highlight
with mouse, which automatically puts the selection in a buffer, then
middle click of the mouse pastes.  Super handy copying and pasting
commands into the terminal etc.
On Sun, 2022-03-06 at 21:51 -0800, Alex Saveau wrote:

BTW, is the primary clip supposed to be automatically pastable from
Ctrl + V or just saved to the history. Right now I'm doing the
latter.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you authored the thread.Message ID:
@.***>

And so when you highlight with your mouse, do you expect to be able to paste with Ctrl + V or only middle click?

Oh sorry, yes only middle click.
On Sun, 2022-03-06 at 22:32 -0800, Alex Saveau wrote:

And so when you highlight with your mouse, do you expect to be able
to paste with Ctrl + V or only middle click?
—
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.
You are receiving this because you authored the thread.Message ID:
@.***>

Sweet, that's how I implemented it!


change directories

i got sptarkov 3.7.5 and i got real tarkov in the new 1.40 patch or whichever. ive tried to run the installer in every place in the sptarkov folder but keeps redirecting it to the real tarkov. any fix?

same situation

sailro, could you make a separate dll file for the BepInEx plugins?

Do you have any console output? Did you place the installer in the sptarkov folder?

And what about using Installer "path/to/eft-sptaki" to install the trainer in a specific path ?.

i mean i ran the installer in every single sptarkov folder and place, but it just automatically made it connect to the real tarkov. didnt even let me change directories nothing. just auto tarkov

yo nvm thx i figured it out 🗡


feat(issue): stop processing an issue already stale

PR Checklist
Please check if your PR fulfils the following requirements:

[x] The commit message follows our guidelines
[ ] Tests for the changes have been added/updated (for bugfix/feature)
[ ] Docs have been added/updated (for bugfix/feature)
[ ] Contributing implementation have been added/updated (for bugfix/feature)

PR Type
What kind of change does this PR introduce?

[ ] Feature (a new feature)
[ ] Bugfix (a bug fix)
[ ] Style (changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc))
[ ] Refactor (a code change that neither fixes a bug nor adds a feature)
[ ] Perf (a code change that improves performance)
[ ] Test (adding missing tests or correcting existing tests)
[ ] Build (changes that affect the build system, CI configuration or external dependencies)
[ ] Docs (changes that affect the documentation)
[ ] Chore (anything else), please describe:

What is the current behaviour?

What is the new behaviour?
Does this PR introduce a breaking change?

[ ] Yes
[x] No

Other information

Closes #77

Codecov Report

Merging #83 (d5d1c8d) into develop (372dc29) will decrease coverage by 13.35%.
The diff coverage is 77.50%.

@@             Coverage Diff              @@
##           develop      #83       +/-   ##
============================================
- Coverage    99.33%   85.98%   -13.36%
============================================
  Files           25       25
  Lines          454      478       +24
  Branches        45       47        +2
============================================
- Hits           451      411       -40
- Misses           3       67       +64

Flag
Coverage Δ

integration
85.98% <77.50%> (-0.27%)
:arrow_down:

unit
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

src/core/issues/issue-ignore-processor.ts
97.29% <ø> (-2.71%)
:arrow_down:

src/core/issues/issue-processor.ts
84.09% <45.45%> (-15.91%)
:arrow_down:

src/core/issues/issue-is-stale-processor.ts
87.50% <87.50%> (ø)

src/core/issues/issue-stale-processor.ts
95.12% <92.30%> (-4.88%)
:arrow_down:

src/core/stale.service.ts
62.50% <0.00%> (-37.50%)
:arrow_down:

src/github/api/labels/github-api-labels.service.ts
63.63% <0.00%> (-36.37%)
:arrow_down:

src/utils/loggers/logger-format.service.ts
66.66% <0.00%> (-33.34%)
:arrow_down:

src/core/issues/issue-logger.ts
73.33% <0.00%> (-26.67%)
:arrow_down:

src/utils/loggers/logger.service.ts
77.14% <0.00%> (-22.86%)
:arrow_down:

src/github/api/issues/github-api-issues.service.ts
88.46% <0.00%> (-11.54%)
:arrow_down:

... and 6 more


Add readme.md

Should add a readme file to your github with basic information.
Example: https://github.com/BuiltBrokenModding/ICBM/blob/development/README.md

Probably also worth adding a licence.
I suggest MIT, but find one that suits you.
Example: https://github.com/Lemonszz/Totem-Expansion/blob/master/LICENSE.md

Totally forgot about that, fixed it.


Refresh openwhisk

How can I clear all previous state of openwhisk and run it fresh all over again? I run the Openwhisk locally and when the connection is terminated and I open it again, all the environmental variables I set, are set back to default and I when I set then again they don't have any effect.

Can you clarify which values do you want to reset? You can delete ~/.openwhisk/standalone/ which is where some metadata gets stored.


misinterpreting IVM roundtrip binary to text

/home/travis/build/amzn/ion-js/ion-tests/iontestdata/good/symbols.ion
Error: Only ion version 1.0 is supported.

Resolved by #274, closing


Refactoring client/

Only cosmetic changes included, no functionality has changed in this PR. It's broken off the fetch tagged one to keep it shorter, changes:

Refactor session.pools into it's own struct
Break up host queue tests into files per RPC endpoint
Extract writeConsistencyAchieved and readConsistencyAchieved methods from session

@richardartoul yep, no changes:
~/code/gocode/src/github.com/m3db/m3db prateek/client/session-refactor*
❯ go test -list '.*' ./client |grep -v ok  > m3db-client-new-branch

~/code/gocode/src/github.com/m3db/m3db master* ⇣
❯ go test -list '.*' ./client |grep -v ok > m3db-client-master

~/code/gocode/src/github.com/m3db/m3db master* ⇣
❯ diff <(cat m3db-client-master| sort) <(cat m3db-client-new-branch | sort)
# no output


🔨 fixed link to keyrun scss

Hi, I love your wrapper for keyrune!
Recently, my project started failing builds and I discovered that a recent update to keyrune broke react-keyrune. So I renamed the variable. If any other problems come up related to this one, I'll attempt to pull similar branches.
Take care!

Codecov Report

Merging #18 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##           master    #18   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files           1      1
  Lines          11     11
=====================================
  Hits           11     11

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0e79822...41f77e6. Read the comment docs.

Thanks for the fix! I haven't been working on anything using this library for a bit, so I probably wouldn't have caught it anytime soon. Good to know others have found it useful!


added card structure with styles & animation for mission details

Fixes #9
Added Card structure with details inside separate blocks using CSS styles and animation

This is amazing work @ShwetKhatri2001 , thank you for your contribution 🥇


ModuleNotFoundError: No module named '_learn'

I installed all the dependencies and I have no errors while running make but when I tried to run main.py it shows:

Traceback (most recent call last):
File "main.py", line 6, in
from tribeflow import dataio
File "/home/h_anh/tribeflow/tribeflow/init.py", line 10, in
from tribeflow.learn import fit
File "/home/h_anh/tribeflow/tribeflow/learn.py", line 4, in
from _learn import em
ModuleNotFoundError: No module named '_learn'

Did you compile using make? Also, are you using python2?

Since there was no response im closing this.


Better Prompt

Hello, Is there a way to make text-davinci-003 act more like ChatGPT? By providing it with more detailed and thorough prompts?

Maybe. Current prompt is really a random guess which worked fine, and I think it’s possible to make up some better one


Enable github actions for pull requests

Closes #57

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@396efe4). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #58   +/-   ##
=========================================
  Coverage          ?   76.41%
=========================================
  Files             ?        8
  Lines             ?      424
  Branches          ?        0
=========================================
  Hits              ?      324
  Misses            ?       66
  Partials          ?       34

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 396efe4...cf51fc4. Read the comment docs.


DocsArchive-<IP_ADDRESS>Auto archive content

Please don't merge this PR until target archive repo PR https://github.com/OPS-E2E-PPE/docs-archive-test-target_7109c1f4-ee68-456e-bba2-b8c70169d575/pull/1 is merged into live branch.
Auto archive content to https://github.com/OPS-E2E-PPE/docs-archive-test-target_7109c1f4-ee68-456e-bba2-b8c70169d575.git

Docs Build status updates of commit 5393edf:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error]

Cannot sync git repo to specified commit because branch Archive_master_2019-09-30-03-00-57 has been deleted or has been force pushed: fatal: Couldn't find remote ref Archive_master_2019-09-30-03-00-57

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Build: Fix warnings in DataGrid examples and tests

Description

Tidy up DataGridTests and example page.
Remove tests that were obviously playground type components due to large, commented sections.
Fix BL0005 in test by using pager directly rather than modifying parameters outside of component.
How Has This Been Tested?

Build to see warnings gone
Types of changes

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

[x] The PR is submitted to the correct branch (dev).
[x] My code follows the code style of this project.
[ ] I've added relevant tests.

I wonder, is this some old tests and do they have analogue for new datagrid?
I just see that some could be converted to use PropertyColumn and we also do support IDictionary(tho I hate the support of it) and I'm not sure we have it covered.

I wonder, is this some old tests and do they have analogue for new datagrid? I just see that some could be converted to use PropertyColumn and we also do support IDictionary (tho I hate that we support it in the PropertyColumn) and I'm not sure we have it covered.

The thing is large bodies of the test are commented out (github doesnt colourise this well).  So they really werent testing the datagrid at all .

To be clear there were no unit tests referring to these components in the test viewer either

@tjscience We'd like to remove these components from the TestComponentsViewer.  It seems they were only part of a preliminary exploration and mostly commented out. They are not involved in any unit tests.

@henon Given @tjscience comment I would say this is good to go


LuaRocks

Hi. Have you deployed this to LuaRocks so it can be added easily to OpenResty?

@seddonm1 why do not choose OpenResty Package Manager opm

@seddonm1 I'm try to add LuaRocks on latest commit, and you can try:)


Tables not directly mapped to a class don't get dumped.

If, for example, your database contains an association table for a many-to-many mapping as described here: http://docs.sqlalchemy.org/en/latest/orm/basic_relationships.html#many-to-many, the many-to-many association table doesn't get dumped by alchemy dumps. This seems to be because the base declarative class's subclasses are the only things enumerated when creating dumps, which will result in missing other data. Some other examples of usages that alchemydumps doesn't support can be found here: http://docs.sqlalchemy.org/en/latest/orm/nonstandard_mappings.html
Would it work to enumerate db.Model.metadata.tables and dump the contents that way instead?

This seems to be because the base declarative class's subclasses are the only things enumerated when creating dumps, which will result in missing other data.

That's completely true, @baumatron. You've found a huge bug.
Unfortunately I cannot take a look on these issues this soon because things are kind of hectic here the following weeks.
However, fell free to send e PR if you want ti give it a try.
Usually I follow a TDD (test-driven development) approach:

Add the missing models at tests/app.py (e.g. many-to-many relationships this time)
Write tests for the new models at tests/test_commands/py
Fix the package ; )
Add the novelties to theREADME.md

Thanks for your reply and feedback! I unfortunately am also pretty busy at the moment so I won't be able to look at this either for the time being.
For anyone who might be trying to a create many-to-many association table and need to save it with achemy dumps, I came up with a workaround for that specific case. I created a mapped class that would create a table with the same shape that I needed. That way alchemy dumps will pick it up since it will be enumerated as a subclass of db.Model.
For example:
user_to_project_table = db.Table(     'user_to_project',     Base.metadata,     db.Column('user_id', db.ForeignKey('user.id'), primary_key=True),     db.Column('project_id', db.ForeignKey('project.id'), primary_key=True))
Becomes something like:
class UserToProjectTable(db.Model):     __tablename__ = 'user_to_project'     user_id = db.Column('user_id', db.ForeignKey('user.id'), primary_key=True)     project_id = db.Column('project_id', db.ForeignKey('project.id'), primary_key=True)
Then in your classes where you declare the relationship, the relationship's 'secondary' property can be assigned the string 'user_to_project' instead of user_to_project_table.

Many thanks again @baumatron!
I edited your last comment just to add Python color to the code bits, ok?

Perfect, thanks for making that more readable.


How to pause with CountdownTimerController?

Hello,
I've noticed that there are 2 controllers and from #14 you use CountdownController to pause. I can't see a method to pause using (what I'm speculating is the new one) CountdownTimerController.

You can use controller.disposeTimer() to pause it. But since endTime is a timestamp, it will skip the paused time when it starts again. See the example code for details.
  if (controller.isRunning) {
            controller.disposeTimer();
          } else {
            controller.start();
          }

Is this what you need?There is also a plugin Countdown in the repository, which runs according to the given timing constants. You can see if this is the effect you are looking for.
import 'package:flutter/material.dart';
import 'package:flutter_countdown_timer/index.dart';

class CountdownPage extends StatefulWidget {
  @override
  _CountdownPageState createState() => _CountdownPageState();
}

class _CountdownPageState extends State<CountdownPage> {
  CountdownController countdownController =
      CountdownController(duration: Duration(hours: 1));
  bool isRunning = false;
  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: Center(
        child: Countdown(
            countdownController: countdownController,
            builder: (_, Duration time) {
              var currentRemainingTime = countdownController.currentRemainingTime;
              return Text(
                  'days: [ ${currentRemainingTime.days} ], hours: [ ${currentRemainingTime.hours} ], min: [ ${currentRemainingTime.min} ], sec: [ ${currentRemainingTime.sec} ]');
            }),
      ),
      floatingActionButton: FloatingActionButton(
        child:
            Icon(isRunning ? Icons.stop : Icons.play_arrow),
        onPressed: () {
          if (!countdownController.isRunning) {
            countdownController.start();
            setState(() {
              isRunning = true;
            });
          } else {
            countdownController.stop();
            setState(() {
              isRunning = false;
            });
          }
        },
      ),
    );
  }
}

I'm trying this out right now and it seems to do the job great but only for pausing and playing. Calling dispose() gives me the first log and I don't know the proper way to dispose of it. I also can't find an appropriate way to call an onEnd() method, unlike CountdownTimerController. It causes an issue if I call it within the builder() method as I use setState in my onEnd() method.

Code
dispose() exception:
══╡ EXCEPTION CAUGHT BY WIDGETS LIBRARY ╞═══════════════════════════════════════════════════════════
The following assertion was thrown while finalizing the widget tree:
A CountdownController was used after being disposed.
Once you have called dispose() on a CountdownController, it can no longer be used.

This PreparationPage widget cannot be marked as needing to build because the framework is already in
the process of building widgets.  A widget can be marked as needing to be built during the build
phase only if one of its ancestors is currently building. This exception is allowed because the
framework builds parent widgets before children, which means a dirty descendant will always be
built. Otherwise, the framework might not visit this widget during this build phase.
The widget on which setState() or markNeedsBuild() was called was:
  PreparationPage
The widget which was currently being built when the offending call was made was:
  Countdown

The relevant error-causing widget was:
  Countdown
  file:///home/jrmallorca/Projects/individualProject/application/habitualise/lib/pages/exercise/preparation_page.dart:282:12

When the exception was thrown, this was the stack:
#0      Element.markNeedsBuild.<anonymous closure> (package:flutter/src/widgets/framework.dart:4138:11)
#1      Element.markNeedsBuild (package:flutter/src/widgets/framework.dart:4153:6)
#2      State.setState (package:flutter/src/widgets/framework.dart:1287:15)
#3      _PreparationPageState._onFinishCountdown (package:habitualise/pages/exercise/preparation_page.dart:143:9)
#4      _PreparationPageState._buildCountdownTimer.<anonymous closure> (package:habitualise/pages/exercise/preparation_page.dart:285:36)
#5      _CountdownState.build (package:flutter_countdown_timer/countdown.dart:46:20)
#6      StatefulElement.build (package:flutter/src/widgets/framework.dart:4612:27)
#7      ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:4495:15)
#8      StatefulElement.performRebuild (package:flutter/src/widgets/framework.dart:4667:11)
#9      Element.rebuild (package:flutter/src/widgets/framework.dart:4189:5)
#10     BuildOwner.buildScope (package:flutter/src/widgets/framework.dart:2694:33)
#11     WidgetsBinding.drawFrame (package:flutter/src/widgets/binding.dart:873:21)
#12     RendererBinding._handlePersistentFrameCallback (package:flutter/src/rendering/binding.dart:319:5)
#13     SchedulerBinding._invokeFrameCallback (package:flutter/src/scheduler/binding.dart:1144:15)
#14     SchedulerBinding.handleDrawFrame (package:flutter/src/scheduler/binding.dart:1082:9)
#15     SchedulerBinding._handleDrawFrame (package:flutter/src/scheduler/binding.dart:998:5)
#19     _invoke (dart:ui/hooks.dart:161:10)
#20     PlatformDispatcher._drawFrame (dart:ui/platform_dispatcher.dart:253:5)
#21     _drawFrame (dart:ui/hooks.dart:120:31)
(elided 3 frames from dart:async)

════════════════════════════════════════════════════════════════════════════════════════════════════

I will update a version to try to fix these issues.

I have released 4.1.0 version, you can see if it can solve your problem.
example

Thank you for implementing an onEnd() method! It's working now but I'm still confused as to how to dispose() properly as it keeps saying that Another exception was thrown: A CountdownController was used after being disposed. just like my above comment with the logs.
This happens when I call Navigator.pop(context) to exit from where Countdown is displayed. I've already overriden my dispose() such that it's like this:
@override
void dispose() {
  _controller.dispose();
  super.dispose();
}

You don't need to dispose it.I have already executed its dispose method in the 'countdown' class


Fix naming consistency and remove intCache.

intCache does not make sense for non refs, nim will copy it anyways.

Thanks for your interest in my little experiment and for your improvements! With my high_v/low_v I probably wanted to avoid possible confusions with Nim's builtin high/low procedures, but your PR shows me, that no problems arise using the nicer names.
Unfortunately this thing is terribly slow and therefore probably unusable...

Well slow or not slow, JS does not support int64s and nim does, so some thing needs to be done about it. I would take slowness plane wrong. I was thinking instead of doing 32bit and 32bit integers doing a 53bit int (max js int) and a 11bit int. It might be fast to check if the 11bit float is zero and not involve it in multiplications and stuff.


Fix connectivity issues

See https://github.com/home-assistant/core/issues/43670 and elsewhere.
I was wrong about race conditions.  However, if _get_receiver_sources fails it was mutating the self._receiver_type which caused lots of other requests to fail, timeout 403 etc.
Still not sure why receivers were actually "locking up" - could not recreate even sending batches of 2000 updates with 50 in parallel.  However, during big requests I was able to observe the device info endpoint request time rising from 8ms to 3.3s
We may want to revisit ensure_configuration since it still does have a lot of blocking IO that HA doesn't like.

@JPHutchins this change as probably only little effect, because _get_receiver_sources is only called when the current input function is not found in _input_func_list or when it is not existing at all.
I don't understand your point regarding ensure_configuration. It is just performing HTTP requests when there are missing configuration parameter. This is usually only the case when an instance is created.
Additionally it is only called from update method which is exactly the place HA is expecting blocking calls.
Anyhow your change sounds reasonable 😄


DeeplabV3 MobileNetV2 Quantization for USB Coral TPU with custom input size ((100, 100) rather than (513, 513))

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): MacOS
Mobile device (e.g., Pixel 4, Samsung Galaxy 10) if the issue happens on mobile device: Coral TPU
TensorFlow installed from (source or binary): pip install tensorflow==1.15.0
TensorFlow version (use command below):1.15.0
Python version:3.7
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version: N/A
GPU model and memory: N/A

Please provide the entire URL of the model you are using?

https://github.com/tensorflow/models/blob/394baa9f21424d3522ccfbdcee8acd3840891ff6/research/deeplab/g3doc/quantize.md
Describe the current behavior
Good Evening,
I have been following the steps in https://github.com/tensorflow/models/blob/394baa9f21424d3522ccfbdcee8acd3840891ff6/research/deeplab/g3doc/quantize.md to quantize a DeeplabV3 MobileNetV2 model for the USB Coral TPU but I cannot get it to work.
Even when I start from the models provided at the bottom (not the 8_bits, the raw models), I cannot get it to run. And I even get different problem for both models.
For the deeplabv3_mnv2_pascal_train_aug:

The deeplab/train.py script for quantizing runs successfully.
The deeplab/export_model.py scripts step works.
The tflite_convert works too.
EdgeTPU compiler gets cryptic error message: Internal compiler error. Aborting!

For the deeplabv3_mnv2_dm05_pascal_trainaug:

The deeplab/train.py script for quantizing runs gets:
ValueError: Total size of new array must be unchanged for MobilenetV2/Conv/weights lh_shape: [(3, 3, 3, 16)], rh_shape: [(3, 3, 3, 32)]
I tried this solution unsuccessfully.

Using the .pb file directly, the deeplab/train.py with quantization aware training fails with python crashing.

Doing a post-quantization in python directly works.

edgetpu_compiler works.

But once ran on the TPU, we get 0.87 TPS, most likely because the edgetpu-converter cannot convert all to edgtpu.

Any guidance to get it working would be great. I am also open to solutions not following the guide from tensorflow.
Describe the expected behavior
It should not crash.
Code to reproduce the issue

Note: This might be related to https://github.com/google-coral/edgetpu/issues/72
Take the 2 models at the bottom of the https://github.com/tensorflow/models/blob/394baa9f21424d3522ccfbdcee8acd3840891ff6/research/deeplab/g3doc/quantize.md page (not the 8_bits one) and try to quantize them following the guide and run them on an USB Coral Edge TPU.
Other info / logs

See above

Hello @saturnin13
Could you attach both models so I can take a look?
Since you reached out to us via our private channel, you can also send it to us there.

The models that I have tried unsuccessfully to get working are the ones directly on the https://github.com/tensorflow/models/blob/394baa9f21424d3522ccfbdcee8acd3840891ff6/research/deeplab/g3doc/quantize.md page. The one at the bottom, namely mobilenetv2_dm05_coco_voc_trainaug and mobilenetv2_coco_voc_trainaug. I have also sent you our personal models via the private mean for you to be able to have a look.

@saturnin13
Thanks I'll response to you via our private channel!

Any news about this topic?

@sramirez My colleague shared his procedure for exporting and converting the model from thewith deeplabv3_mnv2_dm05_pascal_trainaug_8bit checkpoints.


specs with ns-qualified keywords

This is WIP.
First step done

split cimi common spec ns into common, core, acl, common-operation ns-es. @loomis Please comment on this.  Thanks.

Connected to #1473

@loomis

collapsed :cimi.common.link/href into ::href (instead of ::link-href)
added cimi-operation ns to define ::href and ::rel (previous :cimi.common.operation/href and :cimi.common.operation/rel)

PRs for Connectors
https://github.com/slipstream/SlipStreamConnectors/pull/218
https://github.com/SixSq/SlipStreamConnectors/pull/160


update weaviate version

update weaviate version to 1.18.3
update python client version to 3.15.3

To avoid any confusion in the future about your contribution to Weaviate, we work with a Contributor License Agreement. If you agree, you can simply add a comment to this PR that you agree with the CLA so that we can merge.beep boop - the SeMI bot 👋🤖PS:Are you already a member of the Weaviate Slack channel?

I agree to the CLA.


in mxnet0.9.4,I also find a problem, the model load is so so so so so slow, the train is also so so so slow. the mxnet0.9.3 is faster.

This speed is so slow in mxnet0.9.4, and it has not any errors. but finally it successfully load the module.

This issue is closed due to lack of activity in the last 90 days. Feel free to ping me to reopen if this is still an active issue. Thanks!