[Fix] Fix spark/flink starter script error on windows

This PR is to fix seatunnel issue 6386
Does this PR introduce any user-facing change?
No
How was this patch tested?
This pach can be tested via  general windows specific flink/spark starter script.
Flink:   start-seatunnel-flink-13(15)-connector-v2.cmd
Spark: start-seatunnel-spark-2(3)-connector-v2.cmd
For conf file, just using the example conf file from "seatunnel_home/config" dir.
v2.batch.config.template, v2.streaming.template
Check list

Please add test in e2e.

Hi, please tell me the full detailed steps for "add test in e2e" since I'm fresh on this.
Bases on my understanding, it seems that it's unnecessary to add new test cases because current test case is enoguh to cover the bug fix.  As  I mentioned before, we can verify the fix like below:
"
The fix can be tested via general windows specific flink/spark starter script.
Flink: start-seatunnel-flink-13(15)-connector-v2.cmd
Spark: start-seatunnel-spark-2(3)-connector-v2.cmd
For conf file, just using the example conf file from "seatunnel_home/config" dir.
v2.batch.config.template, v2.streaming.template
"

Please add test in e2e.

Hi, please tell me the full detailed steps for "add test in e2e" since I'm fresh on this.
Bases on my understanding, it seems that it's unnecessary to add new test cases because current test case is enoguh to cover the bug fix. As I mentioned before, we can verify the fix like below: " The fix can be tested via general windows specific flink/spark starter script. Flink: start-seatunnel-flink-13(15)-connector-v2.cmd Spark: start-seatunnel-spark-2(3)-connector-v2.cmd
For conf file, just using the example conf file from "seatunnel_home/config" dir. v2.batch.config.template, v2.streaming.template "

Oh I see. How about add some UT to verify whether the output command is right. Our UT will exeucted both on windows and linux.

Please add test in e2e.

Hi, please tell me the full detailed steps for "add test in e2e" since I'm fresh on this.
Bases on my understanding, it seems that it's unnecessary to add new test cases because current test case is enoguh to cover the bug fix. As I mentioned before, we can verify the fix like below: " The fix can be tested via general windows specific flink/spark starter script. Flink: start-seatunnel-flink-13(15)-connector-v2.cmd Spark: start-seatunnel-spark-2(3)-connector-v2.cmd
For conf file, just using the example conf file from "seatunnel_home/config" dir. v2.batch.config.template, v2.streaming.template "

Oh I see. How about add some UT to verify whether the output command is right. Our UT will exeucted both on windows and linux.

What does UT mean ?  user testcase ?

Please add test in e2e.

Hi, please tell me the full detailed steps for "add test in e2e" since I'm fresh on this.
Bases on my understanding, it seems that it's unnecessary to add new test cases because current test case is enoguh to cover the bug fix. As I mentioned before, we can verify the fix like below: " The fix can be tested via general windows specific flink/spark starter script. Flink: start-seatunnel-flink-13(15)-connector-v2.cmd Spark: start-seatunnel-spark-2(3)-connector-v2.cmd
For conf file, just using the example conf file from "seatunnel_home/config" dir. v2.batch.config.template, v2.streaming.template "

Oh I see. How about add some UT to verify whether the output command is right. Our UT will exeucted both on windows and linux.

What does UT mean ? user testcase ?

unit test. you can refer https://github.com/apache/seatunnel/blob/ec533ecd9a7d94d1c3e6d6ad7cc0440315d897e5/seatunnel-core/seatunnel-flink-starter/seatunnel-flink-starter-common/src/test/java/org/apache/seatunnel/core/starter/flink/TestFlinkParameter.java#L37 so on

Please add test in e2e.

Hi, please tell me the full detailed steps for "add test in e2e" since I'm fresh on this.
Bases on my understanding, it seems that it's unnecessary to add new test cases because current test case is enoguh to cover the bug fix. As I mentioned before, we can verify the fix like below: " The fix can be tested via general windows specific flink/spark starter script. Flink: start-seatunnel-flink-13(15)-connector-v2.cmd Spark: start-seatunnel-spark-2(3)-connector-v2.cmd
For conf file, just using the example conf file from "seatunnel_home/config" dir. v2.batch.config.template, v2.streaming.template "

Oh I see. How about add some UT to verify whether the output command is right. Our UT will exeucted both on windows and linux.

What does UT mean ? user testcase ?

unit test. you can refer
https://github.com/apache/seatunnel/blob/ec533ecd9a7d94d1c3e6d6ad7cc0440315d897e5/seatunnel-core/seatunnel-flink-starter/seatunnel-flink-starter-common/src/test/java/org/apache/seatunnel/core/starter/flink/TestFlinkParameter.java#L37
so on

I checked the source code of "TestFlinkParameter.java" and feel a bit confused.
It imports two new class:
import org.apache.seatunnel.core.starter.flink.args.FlinkCommandArgs;
import org.apache.seatunnel.core.starter.flink.utils.EnvironmentUtil;
I can't find these classes in dev branch of  base repo(apache/seatunnel)'s code line,
Has these two java classes added recently ?
About this PR,  Is it also necessary to introduce  some new java classes to finish a simple unit test case ?
Do we have a guideline or standard rule to submit  unit test case ?

I can't find these classes in dev branch of base repo(apache/seatunnel)'s code line,

You can search it

About this PR, Is it also necessary to introduce some new java classes to finish a simple unit test case ?

Yes, just create new java file with new test case. The TestFlinkParameter.java just a demo of unit test in seatunnel

please merge dev branch update
6442

Sorry, I made an error operation and I'm trying to recover it

I've finished java unit test cases and tested successfully.
BTW,  I also fixed windows specific starter script errors(e.g. start-seatunnel-flink-15-connector-v2.cmd, start-seatunnel-spark-3-connector-v2.cmd) because in 2.3.4 release version  running these scripts will trigger "JAVA CLASSNOTFOUNDEXCEPTION".

please merge dev branch update 6442

please fix it

If the merge conflict cannot be resolved, you can resubmit the PR

I closed this PR and submitted a new PR：
https://github.com/apache/seatunnel/pull/6511


Upgrade tensorflow dependency to 1.13.1

Looks like tensorflow 1.13.1 has finally been released. We probably want to update to this version so that we can take full advantage of int64 matrix multiplications and no longer have to support tensorflow 1.12.0 custom builds. This should be fairly straight forward just need to update the tensorflow dependency version in the setup.py, requirements.txt and Makefile.

Closed by https://github.com/mortendahl/tf-encrypted/pull/409


Go tests should cover the features with apis scaffolding using v1 version

Feature Request
Is your feature request related to a problem? Please describe.
Ensure that all features work fine for GO when CRD's versions are v1.
Describe the solution you'd like
After we merge the PR https://github.com/operator-framework/operator-sdk/pull/3499 we add another test to check OLM and Scorecard with a project that was scaffolded with the Makefile updated as:
By("Updating Makefile to use crd:crdVersions=v1")
replaceCRDOptions := "crd:trivialVersions=true"
testutils.ReplaceInFile(filepath.Join(tc.Dir, "Makefile"), replaceCRDOptions, "crd:crdVersions=v1")

This has been done for awhile now.

This has been done for awhile now.


chore: add Swahili translation of "Base is for everyone" on homepage

What changed? Why?
Notes to reviewers
How has it been tested?

🟡 Heimdall Review Status

Requirement
Status
More Info

Reviews

🟡
0/2

Denominator calculation

Show calculation

1 if user is bot
0

1 if user is external
0

From .codeflow.yml

1

Additional review requirements

Show calculation

Max
0

0

From CODEOWNERS

0

Global minimum
0

Max

1

1

1 if commit is unverified
1

Sum
2


Path(writable=True) not check subdirectory exist

Thanks for your work on Click.
Maybe I didn't read correctly documentation but on my understanding click.Path(writable=True) should check if subfolder in path is exist.
How to reproduce:
Create file test.py with:
import click

@click.command()
@click.option("--output-path", help="File where result is write", type=click.Path(writable=True))
def main(output_path: str) -> None:
    out = open(output_path, "w")
    print("Hello Word", file=out)

if __name__ == "__main__":
    main()

Run:
python test.py --output-path test

No failed and test file contains Hello Word
Run:
python tmp.py --output-path test/test # I assume you didn't have test folder in run directory

Produce this error:
Traceback (most recent call last):
  File "$HOME/tmp.py", line 11, in <module>
    main()
  File "/usr/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "$HOME/tmp.py", line 6, in main
    out = open(output_path, "w")
          ^^^^^^^^^^^^^^^^^^^^^^
NotADirectoryError: [Errno 20] Not a directory: 'test/test'

Expected behavior:
Usage: test.py [OPTIONS]
Try 'test.py --help' for help.

Error: Invalid value for '--output-path': Path 'test/test'  is not writable.

I tried to find an equivalent issue but I failed, I apologize if I create a duplicated issue.
Environment:

Python version: 3.11.2
Click version: 8.1.3

@natir I've been having a look at the docs and the "writable" check only happens if the file you are writing to already exists.

So if I understand correctly click does not allow to check that you can write in a file if this file does not exist.
So if I want to check that I can write in a path that does not exist yet I should not use the type click.Path?
Is there a method that would allow me to verify that I can actually write to a file indicated by a path provided by the user directly in click?
In any case, my issue is not really about the right problem.

If you need to do other checks, you're welcome to write a custom type, use a parameter callback, or use the command's function.


How long will the session be kept?

Would like to add auto-run script to run the software so don’t need to log again?

I don't think it can be achieved. It's based on the web version. You still need to scan the QR-code to log in. Like some other functions (auto forwards and add to the favor) can't be achieved.


1.9.3 fixes

1.9.3 seems like it should actually still be using the rb_thread_blocking_region() call, as ...call_without_gvl() isn't defined in any header (although it is available).
Consuming that implicit declaration was causing crashes in my production, although not, interestingly, in travis.

I support supporting 1.9.3. >_>


Add support for date-based restart frequency

Add support for date-based restart pruning in Payu - should close #358.
As suggested by @aidanheerdegen here, an option is to support a subset of pandas date offset frequency aliases:

Unit
Description

MS
Month start frequency

YS
Year start frequency

M
Month frequency

Y
Year frequency

W
Weekly frequency

D
Day frequency

H
Hour frequency

T
Minute frequency

S
Second freqeuncy

So for example, if in config.yaml,
restart_freq: 5YS

Then the earliest restart of the year, every 5 years from the first restart will be retained.
During archive, the function Experiment.prune_restarts() would inspect each restart directory and return a list of restart directories that can removed using integer or date-based restart frequency. For date-based frequency, individual restart directory paths are passed to the model specific driver that can parse the restart files and return a cftime (or standard) datetime object.
In this branch, access-om2 has the model.get_restart_datetime() function implemented, which calls the mom's get_restart_datetime() which parses the ocean_solo.res file for the final datetime of the restart. I used this file as it is what is used in COSIMA's tidy_restarts script and it also contains information on what calendar it is using.
Currently the month (M) and year (Y) frequencies are defined similar to adding relativedelta's month and years. So for example if a datetime was 15/01/2000, after adding an offset of '6M' it would be 15/07/2000. I have only just noticed that 'M' in pandas documentation is actually the end of the month.. Would the end of the month/year be more useful? Also, with the different cftime calendars M and Y could give unexpected results i.e with different day months. For example, I've got some logic for cftime's360_day, noleap and all_leap calendars but nothing yet for the julian calendar. Also would the start of month/year (MS/YS) frequencies be sufficient- Is M and Y frequencies even needed?
Week, day, hour, minute frequencies may not be super useful or even necessary but they were easy to add as adding timedeltas is supported with both cftime and standard datetime calendars.
The first restart datetime is used as a point of reference for adding the first frequency interval.  Then the next restart with a datetime at or after this checkpoint is kept and then becomes the reference when adding the next time interval. I think using the most recent "kept" datetime as a reference for the next checkpoint could be better than strict regular intervals from the earliest datetime. As the first restart could eventually be lost due to scratch timeouts and using "YS"/"MS" would still keep the earliest restarts in each year/month.
As one of the motivations for date-based restart frequencies was to make syncing restarts to a remote archive easier as payu would know what restarts could be stored permanently. A way to check if a restart would eventually be removed would be to see if the restart directory was in the list returned by prune_restarts(from_n_restart=restart_history - 1, to_n_restart:self.counter) where restart_history is the config.yaml or default value of how many latest restarts are kept.
Any feedback, questions or suggestions is much appreciated!

coverage: 42.619% (+0.6%) from 42.016% when pulling e3b228166811e02b661d52a82650c63c68a8e478 on jo-basevi:358-date-based-frequency into 2d16f18ea24d764b07a41cbcb934cecb7646eaa9 on payu-org:master.

Hello @jo-basevi! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file payu/calendar.py:

Line 98:1: W293 blank line contains whitespace
Line 104:80: E501 line too long (88 > 79 characters)
Line 106:80: E501 line too long (94 > 79 characters)
Line 107:1: W293 blank line contains whitespace
Line 109:80: E501 line too long (103 > 79 characters)
Line 113:80: E501 line too long (90 > 79 characters)
Line 115:80: E501 line too long (86 > 79 characters)
Line 116:1: W293 blank line contains whitespace
Line 120:1: W293 blank line contains whitespace
Line 121:80: E501 line too long (92 > 79 characters)
Line 128:1: W293 blank line contains whitespace
Line 133:1: W293 blank line contains whitespace
Line 134:80: E501 line too long (114 > 79 characters)
Line 136:80: E501 line too long (106 > 79 characters)
Line 137:1: W293 blank line contains whitespace
Line 139:37: E261 at least two spaces before inline comment
Line 140:80: E501 line too long (106 > 79 characters)
Line 141:1: W293 blank line contains whitespace
Line 143:1: W293 blank line contains whitespace
Line 146:80: E501 line too long (83 > 79 characters)
Line 152:1: W293 blank line contains whitespace
Line 154:80: E501 line too long (107 > 79 characters)
Line 155:80: E501 line too long (100 > 79 characters)
Line 167:80: E501 line too long (102 > 79 characters)
Line 173:5: E303 too many blank lines (2)
Line 175:80: E501 line too long (91 > 79 characters)
Line 177:80: W291 trailing whitespace
Line 180:80: E501 line too long (97 > 79 characters)
Line 181:41: E128 continuation line under-indented for visual indent
Line 181:80: E501 line too long (94 > 79 characters)
Line 183:80: E501 line too long (121 > 79 characters)
Line 184:1: W293 blank line contains whitespace
Line 185:80: E501 line too long (102 > 79 characters)
Line 186:80: E501 line too long (135 > 79 characters)
Line 188:80: E501 line too long (83 > 79 characters)
Line 192:80: E501 line too long (80 > 79 characters)
Line 194:80: E501 line too long (91 > 79 characters)
Line 197:55: W292 no newline at end of file

In the file payu/experiment.py:

Line 750:80: E501 line too long (82 > 79 characters)
Line 983:1: W293 blank line contains whitespace
Line 987:80: E501 line too long (85 > 79 characters)
Line 991:33: E127 continuation line over-indented for visual indent
Line 995:80: E501 line too long (110 > 79 characters)
Line 997:46: W291 trailing whitespace
Line 999:57: W291 trailing whitespace
Line 1003:80: E501 line too long (105 > 79 characters)
Line 1004:80: E501 line too long (98 > 79 characters)
Line 1006:56: W291 trailing whitespace
Line 1020:1: W293 blank line contains whitespace
Line 1022:80: E501 line too long (84 > 79 characters)
Line 1022:85: W291 trailing whitespace
Line 1023:1: W293 blank line contains whitespace
Line 1024:80: E501 line too long (86 > 79 characters)
Line 1027:80: E501 line too long (136 > 79 characters)
Line 1028:80: E501 line too long (85 > 79 characters)
Line 1030:80: E501 line too long (82 > 79 characters)
Line 1031:1: W293 blank line contains whitespace

In the file payu/models/accessom2.py:

Line 95:1: W293 blank line contains whitespace
Line 96:80: E501 line too long (134 > 79 characters)

In the file payu/models/model.py:

Line 461:1: W293 blank line contains whitespace
Line 463:80: E501 line too long (119 > 79 characters)
Line 463:120: W291 trailing whitespace
Line 465:34: W292 no newline at end of file

In the file payu/models/mom.py:

Line 13:14: W291 trailing whitespace
Line 231:5: E303 too many blank lines (2)
Line 232:80: E501 line too long (121 > 79 characters)
Line 236:1: W293 blank line contains whitespace
Line 245:1: W293 blank line contains whitespace
Line 247:80: E501 line too long (85 > 79 characters)
Line 248:80: E501 line too long (104 > 79 characters)
Line 248:105: W291 trailing whitespace
Line 249:33: E127 continuation line over-indented for visual indent
Line 250:1: W293 blank line contains whitespace
Line 250:1: W391 blank line at end of file

In the file test/test_calendar.py:

Line 8:1: E302 expected 2 blank lines, found 1
Line 11:80: E501 line too long (95 > 79 characters)
Line 16:1: W293 blank line contains whitespace
Line 22:80: E501 line too long (90 > 79 characters)
Line 26:80: E501 line too long (87 > 79 characters)
Line 27:34: E127 continuation line over-indented for visual indent
Line 28:80: E501 line too long (85 > 79 characters)
Line 29:34: E127 continuation line over-indented for visual indent
Line 30:80: E501 line too long (86 > 79 characters)
Line 32:80: E501 line too long (85 > 79 characters)
Line 34:80: E501 line too long (86 > 79 characters)
Line 42:33: W291 trailing whitespace
Line 43:80: E501 line too long (90 > 79 characters)
Line 47:80: E501 line too long (87 > 79 characters)
Line 48:34: E127 continuation line over-indented for visual indent
Line 49:80: E501 line too long (85 > 79 characters)
Line 50:34: E127 continuation line over-indented for visual indent
Line 51:80: E501 line too long (86 > 79 characters)
Line 53:80: E501 line too long (85 > 79 characters)
Line 55:80: E501 line too long (86 > 79 characters)
Line 63:32: W291 trailing whitespace
Line 64:80: E501 line too long (90 > 79 characters)
Line 68:80: E501 line too long (87 > 79 characters)
Line 69:34: E127 continuation line over-indented for visual indent
Line 71:80: E501 line too long (86 > 79 characters)
Line 78:1: E302 expected 2 blank lines, found 1
Line 79:53: W291 trailing whitespace
Line 81:80: E501 line too long (81 > 79 characters)
Line 82:34: E127 continuation line over-indented for visual indent
Line 83:80: E501 line too long (83 > 79 characters)
Line 87:80: E501 line too long (90 > 79 characters)
Line 89:80: E501 line too long (89 > 79 characters)
Line 99:80: E501 line too long (90 > 79 characters)
Line 101:1: W293 blank line contains whitespace
Line 106:1: W293 blank line contains whitespace
Line 107:80: E501 line too long (105 > 79 characters)
Line 112:1: W293 blank line contains whitespace
Line 116:1: W293 blank line contains whitespace
Line 117:80: E501 line too long (121 > 79 characters)
Line 119:1: W293 blank line contains whitespace
Line 122:80: E501 line too long (92 > 79 characters)
Line 125:80: E501 line too long (93 > 79 characters)
Line 126:80: E501 line too long (90 > 79 characters)
Line 127:80: E501 line too long (90 > 79 characters)
Line 128:80: E501 line too long (90 > 79 characters)
Line 129:80: E501 line too long (88 > 79 characters)
Line 132:50: W292 no newline at end of file

Main changes were removing M and Y offset options which simplified the code quite a bit
as all cftime calendars are supported with YS - Year start and MS - Month start (i.e don't have to worry about the end of month logic..).
Other changes were rewriting the calendar tests and added integration-like tests for parsing MOM restart files and the prune_restarts() method in the Experiment class.
Other Notes
I removed this from the documentation:
" Intermediate restarts are not deleted until a permanently archived restart
has been produced. For example, if we have just completed run 11, then
we keep restart004, restart009, restart010, and restart011.
Restarts 10 through 13 are not deleted until restart014 has been saved."
Because the logic previously was to not delete the last restart_history number of restarts. Intermediate restarts were not deleted until a permanently saved restart was only automatically the case when restart_freq was 5 or less as the default restart_history was also 5. Or if restart_history was also configured to be greater than or equal to restart_freq. I could change it so restart_history is set to be minimum of restart_freq to guarantee immediate restarts aren't pruned too early for integer restart frequency and add it back into the documentation?
Other things I got caught out on was the difference between 12MS and 1YS, as the logic uses the runtime of earliest restart as a reference (not the model start time). Say for example, there were runs with runtime of 1 month, with a mom model start time of 1900/01/01:
(run_date, restart_directory)
(1900/02/01, restart000)
(1900/03/01, restart001)
(1900/04/01, restart002)
...
If using frequency 12MS, the restarts that would be saved are:
(1900/02/01, restart000)
(1901/02/01, restart012)
(1901/02/01, restart024)
...
While 1YS would keep
(1900/02/01, restart000)
(1901/01/01, restart011)
(1901/01/01, restart023)
...
The other thing to keep in mind is that the code currently uses the next kept restart as a reference for the next time interval. This means that restarts saved may not be uniformly a fixed frequency from the first restart if there's irregular time steps.
For restart_freq: 6MS, theres a minimum 6 months between each saved restart. So with using the restart's list from above, the restarts saved would be:
(1900/02/01, restart000)
(1900/09/01, restart002) (earliest restart after 6 months)
(1901/03/01, restart004) (6 months after previous kept restart)
If instead there was a fixed frequency from the first restart, the restarts kept would be:
(1900/02/01, restart000)
(1900/09/01, restart002)
(1901/02/01, restart003)
If restart000 was lost somehow then the logic would break as restart002 would be the new reference and restart003 would be purged in a subsequent run through..

Other Notes I removed this from the documentation: " Intermediate restarts are not deleted until a permanently archived restart has been produced. For example, if we have just completed run 11, then we keep restart004, restart009, restart010, and restart011. Restarts 10 through 13 are not deleted until restart014 has been saved."
Because the logic previously was to not delete the last restart_history number of restarts. Intermediate restarts were not deleted until a permanently saved restart was only automatically the case when restart_freq was 5 or less as the default restart_history was also 5. Or if restart_history was also configured to be greater than or equal to restart_freq. I could change it so restart_history is set to be minimum of restart_freq to guarantee immediate restarts aren't pruned too early for integer restart frequency and add it back into the documentation?

Good catch. Yes i think restart_history used should be min(restart_history, restart_freq). It doesn't make sense to keep five previous restarts if the restart_freq is 2. So that would mean reinstating that bit in the docs, though it does need editing to ay it uses restart_history and keeps min(restart_history, restart_freq) of the most recent history files.
How does this affect date based pruning though? We can keep restart_history as an integer (just keep the n most recent restarts) but then what does min(restart_history, restart_freq)mean? Would you translaterestart_freq` into an integer number of runs? Not sure how they could be determined.

Other things I got caught out on was the difference between 12MS and 1YS

So this is just something that needs to be kept in mind by the user?

The other thing to keep in mind is that the code currently uses the next kept restart as a reference for the next time interval. This means that restarts saved may not be uniformly a fixed frequency from the first restart if there's irregular time steps.
For restart_freq: 6MS, theres a minimum 6 months between each saved restart. So with using the restart's list from above, the restarts saved would be: (1900/02/01, restart000) (1900/09/01, restart002) (earliest restart after 6 months) (1901/03/01, restart004) (6 months after previous kept restart)

The restarts from above were monthly, so I'm confused by (1900/02/01, restart000) (1900/09/01, restart002) shouldn't it be (1900/02/01, restart000) (1900/09/01, restart007)?

If instead there was a fixed frequency from the first restart, the restarts kept would be: (1900/02/01, restart000) (1900/09/01, restart002) (1901/02/01, restart003)

Which is the same as above .. no?

If restart000 was lost somehow then the logic would break as restart002 would be the new reference and restart003 would be purged in a subsequent run through..

I'm still not sure those restart numbers are correct, or if they are I understand why, so it's a bit hard to make an informed comment on this bit.
Just some general questions:

If a user would like to always save restarts at the beginning of a year (or as close as possible) is that possible?
Do we need to save some state about previous inferred restart frequencies to be able to warn users if they make a change that could be deleterious?

I'm still not sure those restart numbers are correct, or if they are I understand why, so it's a bit hard to make an informed comment on this bit.

Sorry that's my bad!! I deleted the list of restarts I was talking about. So here is the list of restarts I was referring to:
(1900/02/01, restart000)
(1900/07/01, restart001)
(1900/09/01, restart002)
(1901/02/01, restart003)
(1901/03/01, restart004)
...

If a user would like to always save restarts at the beginning of a year (or as close as possible) is that possible?

Yes, using 1YS would save the restart where current model run time is the earliest of each year.

Do we need to save some state about previous inferred restart frequencies to be able to warn users if they make a change that could be deleterious?

Would that require saving the previous restart frequency as an enviroment variable? Changing to a longer
date-based period or a shorter integer frequency would not result in deletions. Though it would be tricky comparing an integer frequency with a datebased frequency..
As archive() runs after each experiment run, if there is greater than 1 restart to prune, that will indicate that either:

restart_frequency is set to a shorter date-based or  higher integer value
or that restart_history is changed to a lower value.

Instead of saving previous state, during setup and if archive is enabled, I could inspect what restarts it would prune (but not actually delete them..). If restart_freq and restart_history are the same, it should be an empty list at that stage. If not, emit a warning that the config has changed, and more restarts will be deleted at the next time archive is run.
How does that sound? Running that check in setup would also have the benefit for any warnings regarding invalid restart_freq values.

Good catch. Yes i think restart_history used should be min(restart_history, restart_freq). It doesn't make sense to keep five previous restarts if the restart_freq is 2. So that would mean reinstating that bit in the docs, though it does need editing to ay it uses restart_history and keeps min(restart_history, restart_freq) of the most recent history files.
How does this affect date based pruning though? We can keep restart_history as an integer (just keep the n most recent restarts) but then what does min(restart_history, restart_freq)mean? Would you translaterestart_freq` into an integer number of runs? Not sure how they could be determined.

Ok, an idea for an simple algorithm pattern to keep intermediate restarts is:
restarts_to_prune = []
intermediate_restarts = []
for restart in restarts:
    if restart_should_be_pruned():
        # keep track of intermediate restarts
        intermediate_restarts.append(intermediate_restarts)

    else:  # restart should be saved
        # add prior intermediate restarts to the list of restarts_to_prune
        restarts_to_prune.extend(intermediate_restarts)
        intermediate_restarts = []

This will work for both integer and date_based frequency and wouldn't require a restart_history config variable at all..
If restart_history is useful to keep, it could act as an override option to ensure that the last n restarts are always kept?

How does that sound? Running that check in setup would also have the benefit for any warnings regarding invalid restart_freq values.

That sounds like a great idea, thanks. I definitely think there is potential for Bad ThingsTM to happen, and checks and emitting information to the user is a good idea.
I definitely can see a use-case for someone to decide to change their restart_frequency when their experiment progresses. This could be to reduce disk usage or just because they realise they don't need so many.
But I think we want to make this sort of mass-deletion an interactive only option. So warn that lots of restarts would be the result of the next archive, so make them use a -f/--force flag to make it happen, otherwise no restarts are pruned?
Specifying flags isn't something you can do as part of an automated run, so by default it has to be done interactively.

New Changes
Support for both mom5 and mom6 models

Move mom driver restart parsing method to fms driver so it can work with mom6 and mom models (as both have the ocean_solo.res file). The other fms model subclass is GOLD. I’m not too sure whether it has similar file or not.. So the method checks first that the file exists and raises a descriptive error if not.

Intermediate restarts and restart_hisory

Intermediate restarts can be kept with both integer and date-based restart pruning
However, if restart_history is defined, it works as it did previously and only keeps restart_history number of restarts. This would be useful for only keeping more or less restarts than intermediate restarts.

Force restart pruning

At the end of setup, a scan of what restarts would be pruned is run. This will error out if there's any invalid configuration (i.e. unsupported values, or if restart files can't be parsed). It will also warn if more restarts than expected would be pruned, and whether --force-prune-restarts needs to be added to next run/archive. This would be the case if restart_history or restart_freq has been changed to values which would prune more restarts, or if auto archiving is suddenly enabled.
Added --force-prune-restarts flag to allow greater deletions of restarts, otherwise no restarts will be deleted. This can be run as payu run --force-prune-restarts or payu archive --force-prune-restarts - if archive is being run manually.

Move mom driver restart parsing method to fms driver so it can work with mom6 and mom models (as both have the ocean_solo.res file). The other fms model subclass is GOLD. I’m not too sure whether it has similar file or not.. So the method checks first that the file exists and raises a descriptive error if not.

I was wrong when I said this was FMS based, I mean, they're all FMS based, and so end up using the same convention, but the naming of the file is in the model driver code, not FMS. Regardless I think it is the right decision to put the code in the FMS driver for the moment.
I guess longer term the best option might be to put it in a mixin class and inherit that from the models that use ocean_solo.res.

Thanks for the review @aidanheerdegen! I've added in the suggested changes and the small fix to counter > 999. Should close #372

I've replied where appropriate and resolved all the conversations, so good to merge.


TS-4976: Regularize plugins - redirect_1

This actually needs a serious review. This hasn't been built for a long time, it used a stat API so old I had a hard time finding any reference to it, so I'm not confident I've extracted the relevant features. In addition there is the question if any of this should be used in the statistics documentation section.

RAT check successful! https://ci.trafficserver.apache.org/job/RAT-github/211/

clang format failed! https://ci.trafficserver.apache.org/job/clang-format-github/197/

AU check failed! https://ci.trafficserver.apache.org/job/autest-github/193/

FreeBSD11 build successful! https://ci.trafficserver.apache.org/job/freebsd-github/1892/

Intel CC build successful! https://ci.trafficserver.apache.org/job/icc-github/322/

Linux build successful! https://ci.trafficserver.apache.org/job/linux-github/1783/

clang-analyzer build successful! https://ci.trafficserver.apache.org/job/clang-analyzer-github/455/

@maskit - Should the plugin stats have a prefix? I think that has been discussed but I don't recall off hand what the conclusion was. I think it was that the stats should be in the "plugin." prefix to avoid collisions.

clang format successful! https://ci.trafficserver.apache.org/job/clang-format-github/212/

RAT check successful! https://ci.trafficserver.apache.org/job/RAT-github/226/

FreeBSD11 build successful! https://ci.trafficserver.apache.org/job/freebsd-github/1907/

AU check successful! https://ci.trafficserver.apache.org/job/autest-github/208/

Intel CC build successful! https://ci.trafficserver.apache.org/job/icc-github/337/

Linux build successful! https://ci.trafficserver.apache.org/job/linux-github/1798/

clang-analyzer build successful! https://ci.trafficserver.apache.org/job/clang-analyzer-github/470/

@SolidWallOfCode Yes, I think they should, I don't remember the discussion though. I think having a prefix is good idea, and the plugin should be a role model since it is an example.

clang format failed! https://ci.trafficserver.apache.org/job/clang-format-github/266/

RAT check successful! https://ci.trafficserver.apache.org/job/RAT-github/280/

AU check successful! https://ci.trafficserver.apache.org/job/autest-github/262/

FreeBSD11 build successful! https://ci.trafficserver.apache.org/job/freebsd-github/1961/

Intel CC build successful! https://ci.trafficserver.apache.org/job/icc-github/391/

Linux build successful! https://ci.trafficserver.apache.org/job/linux-github/1853/

clang-analyzer build failed! https://ci.trafficserver.apache.org/job/clang-analyzer-github/524/

I was leaving them unsquashed while I was still working on them.

clang format failed! https://ci.trafficserver.apache.org/job/clang-format-github/280/

RAT check successful! https://ci.trafficserver.apache.org/job/RAT-github/294/

AU check failed! https://ci.trafficserver.apache.org/job/autest-github/276/

FreeBSD11 build successful! https://ci.trafficserver.apache.org/job/freebsd-github/1975/

Linux build successful! https://ci.trafficserver.apache.org/job/linux-github/1867/

clang-analyzer build failed! https://ci.trafficserver.apache.org/job/clang-analyzer-github/539/


Fix typo and highlight the correct keyword

Fix small typing mistake and highlight the correct keyword

This seems like a small (but important) contribution, so no Contribution License Agreement is required at this point. We will now review your pull request.
Thanks,
Microsoft Pull Request Bot

@prashantkhandelwal : Thanks for your contribution to the Azure documentation! The author, @saveenr, has been notified to review your proposed change.

#sign-off

@prashantkhandelwal Thanks for this contribution - it appears the scope of change requires you to sign our CLA. I'm closing this now - if you are able to sign the CLA, you can reopen, sign, and then this can go through.


How to get D400 series camera status that is it ON or OFF or Disconnected

I am unable to find the object key which will give the status of the camera.

@itachi134999 , can you elaborate what is the actual use-case?
My guess that you refer to the librealsense's context object, which provides (among others):

Device factory
System  Bookkeeper - handle OS-level device connect/disconnect event.

In order to be notified when device is attached/disconnect one needs to register a callback.
Getting the list of currently attached devices on demand is achieved with auto dev_list = context.query_devices().
With regards to the camera's on/off status - I'm not sure whether these terms can be translated directly. Please review the SDK's core capabilities, especially with regards to multi-cam/multi-app setups that are listed here .

@ambikesh13499 With ev-mp's previous reply, did you get it through now? Looking forward to your update. Thanks!

Yeah this issue is solved. Thanks


Argument to run arbitrary tests on prediction/instance?

We (AbanteAI) are using swe-bench quite a lot in developing and benchmarking our coding agent, and are adding this docker image to our workflow - thanks for building it!
We give our agents the ability to select and run arbitary tests from the repo as part of its development process (before swe-bench evaluation). It'd be convenient to use this Docker environment, since it's already setup for the specific codebase/instance.
Would you consider supporting this? If you like I could put together a PR. Thanks.

I've been thinking of the same use case. So it would be greatly appreciated.


Create BlowfishAlgo.py

Added Python code for Blowfish Encryption Algorithm in python folder

Hi @ananya2407 , Could you add the Description of the file in the respective python folder readme


LocalFileSystem Not sorted by file name， As a result, the data lines queried in multiple files are out of order.

Describe the bug
the data lines queried in multiple files are out of order.

Partitions are processed in parallel on multiple threads so DataFusion cannot provide any guarantee of ordering of results unless the query contains an ORDER BY clause.


Report RPC errors in a gauge, refactor error reporting

@jevonearth Here's a suggestion for how to do per-RPC error reporting using a gauge, while also factoring out the common error reporting code a bit.

Output of this looks like this:
# HELP tezos_rpc_failed A gauge that is set to 1 when a metrics collection RPC failed during the current scrape, 0 otherwise.
# TYPE tezos_rpc_failed gauge
tezos_rpc_failed{rpc="/monitor/bootstrapped"} 0
tezos_rpc_failed{rpc="/network/connections"} 0
tezos_rpc_failed{rpc="/network/stat"} 0

Got preliminary 👍 in chat, merging.


ModuleNotFoundError: No module named 'pypandoc'

Rasa Core version:
Trying to install latest rasa_core (0.7.9) in a fresh virtualenv
Python version:
Python 3.6.1
Operating system (windows, osx, ...):
Windows 10
Issue:
Trying to install latest rasa_core in a fresh virtualenv. Get this error when running pip install rasa_core:
(env) C:\Users\user\Desktop\projects\rasa_project>pip install rasa_core
Collecting rasa_core
  Using cached rasa_core-0.7.9.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "C:\Users\user\AppData\Local\Temp\pip-build-b55h3weg\rasa-core\setup.py", line 9, in <module>
        import pypandoc
    ModuleNotFoundError: No module named 'pypandoc'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\user\AppData\Local\Temp\pip-build-b55h3weg\rasa-core\setup.py", line 12, in <module>
        readme = open('README.md').read()
      File "c:\users\user\desktop\projects\rasa_project\env\lib\encodings\cp1252.py", line 23, in decode
        return codecs.charmap_decode(input,self.errors,decoding_table)[0]
    UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 882: character maps to <undefined>

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\user\AppData\Local\Temp\pip-build-b55h3weg\rasa-core\

Can you please try this locally:
Replace the Line 12 in rasa_core/setup.py with:
with io.open('README.md', encoding='utf-8') as f:
    readme = f.read()

It seems, windows is assuming the wrong encoding.

I cloned the repo and first run pip install -r requirements.txt. Now I get the following:
Could not find a version that satisfies the requirement tensorflow==1.1.0 (from -r requirements.txt (line 14)) (from versions: 1.2.0rc2, 1.2.0, 1.2.1, 1.3.0rc0, 1.3.0rc1, 1.3.0rc2, 1.3.0, 1.4.0rc0, 1.4.0rc1, 1.4.0)
No matching distribution found for tensorflow==1.1.0 (from -r requirements.txt (line 14))

Didn't have any issues when installing rasa-core (0.7.6) on the same machine in virtualenv.

Any update or workaround on this? I am facing the same issue. Operating System : Windows 7,64 bit.

@PreranaPolekar I am simply using an older version from a month ago for now. Hopefully this will be fixed in a future version. Try pip install rasa_core==0.7.6.

I used tensorflow cpu only version, downgraded to Python 3.5 to get all pieces work together.

I think there is just no tensorflow version for the combination of windows+python3.5+tensorflow==1.1.0
I'd suggest updating to python 3.6

Let me know if there are still issues with this.

could not find a version that satisfies the requirement tensorflow (from rasa_core) (from versions: )no matching distribution found for tensorflow (from rasa_core)
please help me!!

pip3 install rasa_core


feat: [M3-8682] - Add 'Credit Card Expired' banner

Description 📝
Add a Credit Card Expired banner for accounts with expired credit cards
Changes  🔄

Implemented the banner

Preview 📷

How to test 🧪
Verification steps

Turn on MSW
Verify that the banner is visible

As an Author I have considered 🤔
Check all that apply

[ ] 👀 Doing a self review
[ ] ❔ Our contribution guidelines
[ ] 🤏 Splitting feature into small PRs
[ ] ➕ Adding a changeset
[ ] 🧪 Providing/Improving test coverage
[ ] 🔐 Removing all sensitive information from the code and PR description
[ ] 🚩 Using a feature flag to protect the release
[ ] 👣 Providing comprehensive reproduction steps
[ ] 📑 Providing or updating our documentation
[ ] 🕛 Scheduling a pair reviewing session
[ ] 📱 Providing mobile support
[ ] ♿  Providing accessibility support

also, can you please confirm we don't need to add a display for a card that's expiring soon?

@abailly-akamai the reporter for this ticket mentioned that expiring soon is not the scope of this ticket.


Epub error

MathML should either have an 'alttext' attribute or 'annotation-xml' child element error how to solved

@prashant1688 @rdeltour
Hm.  My reading of the specs
https://www.w3.org/TR/MathML3/chapter5.html
https://www.iso.org/schema/isosts/v1.1/doc/n-tre0.html
suggests that including either the alttext attribute or annotation-xml elements is a MAY and hence omitting them is not an error.  In which case, we might want to emit a warning.  But I could be misinterpreting the spec.
But adding all that additional content, while perhaps desirable would be additional work and might end up dissuading people from using MathML at all.

It comes from the 3.0.1 content documents spec:

The alttext attribute on the m:math element should be used for this purpose primarily when shorter alternative text runs are sufficient. When more extensive alternative text is required, XHTML Content Document fragments should be used. (Note that Reading Systems query these two alternative text locations using a defined preference order.)

3.1 did away with these are normative requirements. Authors are only "encouraged" to provide a fallback image.

Also see discussion in #678.
Hi @prashant1688, unfortunately this is the wrong place to ask questions about solving EpubCheck errors. This place is meant for reporting bugs in EpubCheck. This is not a bug.
As the error messages says, you have to add analttext attribute to the mathml element.
Best,
Tobias


Translate portuguese-written challenges to english

Almost all challenges are written in portuguese and it makes it hard for people like me who doesn't understand it.
It would be cool if someone could take some time to translate them to english.

Well, i guess it would be nice

Hi @javierojeda94, @wesmelo, how are you?
As I am Brazilian, this project ended up reaching more people in Brazil and consequently most of the tests here are in Portuguese.
Do you have suggestions for more open tests in English or other languages?

Consider the next statement:

Simplemente necesito poder entender los problemas para intentar resolverlos pero no puedo entenderlos debido a que están en un idioma distinto al inglés o al español.

Was it easy to understand? Probably. The thing here is that the problems seem to be very cool actually but I don't understand the requirements so I can't solve them :( The request is simple, is there someone able to translate them? I myself may try to translate a few of them but someone that speaks the language would do it pretty quickly I think and make this repo more accesible and interesting to other people

@javierojeda94 I understand your question, but this repo is just an aggregator. We can ask to owners of challenges translate it to provide acessibility to non-brazilian people.
What can we do to improve this repository is increase de quantity of challenges in english.
Do you know more challenges that could stay here?

This would be awesome

We can ask to owners of challenges translate it to provide acessibility to non-brazilian people.

I have some more challenges to share, how can I add them?

You can open PR with challenges or send me here the link of the repositories that I add

Hello guys, I'm brazilian also, and I can translate the challenges on my free time. I think it would be nice to reach out as many people as we can. @LFeh, how can I contribute? maybe sending a PR with the translations and to which challenge they refer to?

I can translate the challenges on my free time too. How can I help?
On Wed, Dec 19, 2018 at 8:35 AM Augusto Oliveira Santana <
<EMAIL_ADDRESS>wrote:

Hello guys, I'm brazilian also, and I can translate the challenges on my
free time. I think it would be nice to reach out as many people as we can.
@LFeh https://github.com/LFeh, how can I contribute? maybe sending a PR
with the translations and to which challenge they refer to?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/LFeh/frontend-challenges/issues/53#issuecomment-448547961,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AYN1kk9Lyw81Q__bvmd7S_V-baB64SQJks5u6hZYgaJpZM4ZXQxj
.

I think that if we add a disclaimer at the beginning of the translation is more than enough, isn't it?
Probably I'm saying bs is just an idea. At the end the original problems are also open source and some of them are MIT licensed, what do you guys think?

If you need help with the translation from PT-BR to ENG, i can help

The best thing to do is to create a fork of the challenge and translate it. Then submit here (in a new category maybe?) with a disclaimer saying that it's not the official challenge repo and probably the company is not hiring foreign people.

Perfect @LucasLeandro1204, after translate you can link it here.

hey, how can i help with the translations? I would like to help!!!

@Gabrielasalvo You can to open a issue (or PR) on tests repository with your translation


duplicate svg files

Hello
I am having a problem using. When I connect svg as components and there are more than 1 of them - instead of loading all svg into my html - the output of my svg code is simply duplicated several times. Please explain what the problem is - the Internet has not found a solution.
thanks for the help.
http://joxi.ru/V2VldakCdRKdKr
http://joxi.ru/krD3j8esKqJKdA
http://joxi.ru/D2Pq34RSqykqBA

I also encountered the same problem.
In my case, same id attribute value was exists in imported svg.

Yes, look a the id or the class inside your svg. It must be unique to not use the same style.


Remove redundant cmake==3.28.3 installs

The CI image has this cmake version installed.

I thought this was left over from that PR as well :). Closing this in favor of https://github.com/facebookincubator/velox/pull/9072


SemanticKitti: How does training with n_frames > 1 work?

Hi,
I am just trying to test your network for the SemanticKitti dataset with 26 classes. I changed n_frames to 3 and the network is using all 26 classes. But now I am getting an error in datasets/SemanticKitti.py:
File "/workspace/Experiments/KPConv-PyTorch/datasets/SemanticKitti.py", line 330, in __getitem__                                                                                new_coords = np.hstack((new_coords, points[:, 3:]))
ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 0, the array at index 0 has size 242 and the array at index 1 has size 126798

How can I solve this? I changed
n_frames = 3
batch_num = 1
val_batch_num = 1

Thanks a lot in advance!
Greetings,
Peer

I think I know where the error comes from. I just pushed a correction. Can you try it with the new version of SemanticKitti.py?

Thanks that solves it! Additionally I found a problem in datasets/SemanticKitti.py line 772:
while class_indices.shape < class_n:  does not work, while while class_indices.shape[0] < class_n:  does.

Thanks! I made the correction


Ensure Cargo.lock is up to date

Pull request
Description
Ensure that the Cargo.lock is always up to date in a pull request
Checklist

[x] The RFC, if required, has been submitted and approved
[x] Any user-facing impact of the changes is reflected in docs.tremor.rs
[x] The code is tested
[x] Use of unsafe code is reasoned about in a comment
[x] Update CHANGELOG.md appropriately, recording any changes, bug fixes or other observable changes in behavior

What if we need to change the lockfile? That is when we add a dependency etc. This might break CI for dependabot too, no?
I do love the clippy part though ❤

Ah no, got it. The build and test run should not change the Cargo.lock from what is checked into git. all good.

Coverage remained the same at 81.35% when pulling 823707a8817d8ad46248f138aeb136379621403f on lock-vsn-test into 86dd84c8d2631d0b1f97c1486d0af3ae6f076df4 on main.

I tested with an outdated Cargo.lock and it failed as expected, to keep the history clean I squashed that before the merge.


anki-remote-decks compatible which anki versions(s)

Hello
I presume this addon isn't compatible with the latest Anki Version 2.1.54?
On the Anki web addon page  I read ->  2021-09-10 | 2.1.29~
Or do I interpret something wrong?
My apologies if is this already stated somewhere, could't find it,
ps/ I used it with an older versions doesn't seem to work with the latest version, it's really a superbe addon many thx!!
best regards
Riccardo

Hi,
In the meantime I could fix it, meaning I installed a lower version (2.1.35) and now it's working fine (again).
Conclusion is that this addon is not yet compatible with the latest version.
Is this correct? Will it be in the future?
best regards
Riccardo

it is ( kind of ) working in the newer versions. but you just have to manualy deleta an import statement in one file. i described it in a pull request for c-ocelly / org-to-anki


Cherrypick #31555 onto release branch

Cherrypick #31555 onto release branch to fix bug
R: @kennknowles
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

I'll let it go green then merge.

looking green ! @kennknowles thank you


Use the latest event as the last event in Exercise 4 of Chapter 6

See #27.

Modulo minor remark, looks great! Thanks!

Thank you for your comment !
Indeed, since the initial element of Observable.scan(initialValue)(accumulator) is the initialValue,
I fixed so.

LGTM!

I will send other PRs of chapter 6 :)
Thanks.

Sounds great - thank you!


fix: change luis editor prebuilt entity error to warn

fixes #8475
Change luis authoring a prebuilt entity name reserved error to warning.

.

Coverage decreased (-0.0003%) to 55.946% when pulling ae1d1cf1b8f4ae99e711bb5b36714ce6425cfd5e on shuwan/changeDiagnosLevelOfPreBuiltEntity into b44768d19a987c363c41adb97cc0ae44784a5082 on main.

Thank you @cosmicshuai


Checkbox column overwrites summary field

I'm submitting a ...
[ ] bug report

What is the current behavior?
Instead of my summary field a checkbox is displayed which is the behavior of the whole column (the data above the last row).

If the current behavior is a bug, please provide the steps to reproduce and if possible a screenshots.
Set one of the column fields to { editable: { type: 'checkbox' }}. Add a summary record. The summary record has a checkbox in place of the provided value (last column in the attached screenshot below).

What is the expected behavior?
A summary value should be printed.

What is the motivation / use case for changing the behavior?
It's a bug.

Link to jsFiddle that demonstrates the problem:
http://jsfiddle.net/e7bqdtp2/2/

Other information

Hello to Bratislava bmakam
I hope that's what you are looking for...
http://jsfiddle.net/jirisejhar/h81rek6d/6/

Screenshot from the fiddle you posted.

Are you seeing something else? I checked it on Chromium as well.
I used these browsers:
Firefox 60.0.2
Chromium 65.0.3325.181

You must update w2ui table
Touch refresh table icon to view result.
I send you pm to g plus...

OK, sorry, I was completely ignoring the render function.
Still, in my opinion, the summary record should take precedence over the rule created for the column. When we want to put some kind of statistics there, whatever we put there will, almost certainly, never be a simple true/false.

I'd like to add that this solution pidlo provided comes with additional price. Once you change the value of a checkbox and then change it back to what it was originally, it will still appear as changed (red mark in the corner).


Add HighLatency Flag to PingStats

Additional plumbing needed for https://github.com/jacktrip/jacktrip/pull/1144

Also, what is the process for integrating this change into vs-server? Does the commits need to get merged into main for vs-server to be able to pull in changes from this repo?


To support and facilitate the process of translating the interfaces

Please get the latest version, fix conflicts, and resubmit.

First of all, I apologize for my late reply..
I didn't get a notification from Github for your comment for some reason..
In general, I will resolve the conflicts at the weekend, thank you

I've agreed with @Sukikiroi to fix the conflict and prepare the PR in 7 days for $100


[stable/grafana] Enable extra init containers and extra emptyDir mounts

What this PR does / why we need it:
This PR introduces two new parameters for the Grafana chart, extraInitContainers and extraEmptyDirMounts.
Since Grafana v6.0.0, it is possible to provision notification channels. These channels might have some sensitive parts (i.e. tokens) and therefore I would like to be able to have an init container which initializes somehow the necessary yaml files. The values can then be shared with the actual Grafana container via emptyDir volumes.
cc @zanhsieh @rtluckie @maorfr
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] DCO signed
[x] Chart Version bumped
[x] Variables are documented in the README.md

@rtluckie @maorfr @jokogr
PR initial impression looks good. Just curious one thing: how high chance would it be if Grafana pod gets auto-injected by other services, e.g. Jaeger? Do I worry too much? Should it out-weight the simplicity of the original chart? What other option we have here?

Please take a look at these 2 PRs:
#12184
#12351
is this PR still needed?

Please take a look at these 2 PRs:
#12184
#12351
is this PR still needed?

I think @jokogr probably needs to mount some empty volume.

@zanhsieh thanks for reviewing this. I also agree that this PR adds complexity, but the Grafana chart is already complex: there are already other init containers and a fine-grained selection of volumes and their respective mounts (e.g. from ConfigMaps, secrets).
I am not aware of Jaeger and I do not have any other services auto-injecting Grafana pods. I came up with this PR because it suits to my case as explained above (and below). If you have a better way, just let me know.
@maorfr thanks for providing the PRs, I actually missed #12184, which does the first half of this PR. However, this PR is still needed to define and mount the emptyDir volume, as @zanhsieh pointed out.
Let me explain once again the reasoning behind this PR:

Grafana introduced support for provisioning notifiers. This is well addressed and implemented in #1235.

However, I want to be able to use a secret value in them (I consider the token part for the notifiers a secret) and currently Grafana does not support parsing environment variables in YAML files (see grafana/grafana#12896). I do not want to store the secret values directly to a ConfigMap.

In the current chart I could mount a big secret with all the necessary configuration settings, but this grows to be inflexible for the deployment strategy I use. I prefer storing only secret values in Kubernetes secrets.

So ideally I have an init container with some secrets mounted or as environment variables, which creates this notifiers file. The file is created on an emptyDir volume which is also mounted on the Grafana pod, so that data is shared.

I am going to rebase this PR and update the Chart version number.

Rebase is done, chart version number got updated.
If you think so, I can skip the first commit (which introduces the extra init containers) and wait for #12184 to be merged. Just let me know here.

/assign
/hold
hey @jokogr,
thanks for getting involved!
as I was stating in #12184, I am not fond of such general fields such as extraInitContainers and extraEmptyDirMounts, as they are very general, and I prefer that we provide our users with a simpler approach (as you said, this chart is already complex as it is). If there is a possibility to implement this feature while exposing a minimal configuration in values.yaml, I would prefer it. Take a look at some other init containers as references. The heavy lifting should be done in the templates and not in values.yaml.
What do you think?

Hey @maorfr,
I share your concerns over over-generalizing the charts, but I have yet to find a more elegant way to avoid doing so.
From one hand, I would like to introduce some custom stuff that are possibly not interesting to anyone outside my company. On the other hand, I would like to use the official public charts, so that my deployments are up-to-date with minimal effort.
As such, my current approach is to use official, public charts stated in the requirements of a custom chart (so to have them as subcharts). If I want to make any expansions, e.g. write my own config maps, then I just add a new template in the custom chart I maintain. If I have to make a change in a pre-existing template, then eventually I have to rely on the subchart's (aka the official chart's) values to do so.
What do you think? Is there a more easy / elegant way to achieve what I am describing?

@jokogr,
as i understand, and please correct me if i am wrong -
the problem you are describing is that you need to to have files for notifiers, but they need to include sensitive data (such as tokens) which you do not want to be included in a ConfigMap.
my proposed solution would be -
same way to can mount a ConfigMap, you can also mount a Secret. This can be a pre-existing secret with all the sensitive data (in this case, files to be used with the notifiers feature).
in this is indeed the case, you can use the extraSecretMounts field which already exists in the chart!
if this is not sufficient, this can be easily exposed using a single field in values.yaml, maybe under the notifiers section?
let me know what you think! :)

@maorfr what you propose is indeed a valid option, i.e. to create the complete notifiers files as a Kubernetes secret and then mount it via the extraSecretsMount. Thanks for taking the time to think about it!
Unfortunately I am currently against it, as I stated before:

In the current chart I could mount a big secret with all the necessary configuration settings, but this grows to be inflexible for the deployment strategy I use. I prefer storing only secret values in Kubernetes secrets.

The reason for this is that Kubernetes secrets on my production environment are handled manually / outside of the Helm scope. Maintaining a big notifiers file as a secret would be a bigger burden for me.
Anyway, I am adding one last point to backup this PR: Several official charts from stable repo share the same practice:

ark (#11615)
elasticsearch (#8476 - moved from incubation already with extra init containers)
filebeat (#7401)
gocd (#10397)
keycloak (#5887)
kibana (#11199)
mariadb (#10423)
mattermost-team-edition (#9458)
mongodb (#11949)
mysql (#7187)
nginx-ingress (#5801)
neo4j (#3047)
sonatype-nexus (#8338)

If you still find this PR unwanted, please feel free to close it; no hard feelings. After all you are one of the maintainers, so I understand your burden and thank you for your time evaluating my PR :)

/hold cancel
/ok-to-test
important to say - i am not so much against it, but i do feel that anyone who brings up a problem is likely representing more users who are having it as well. my approach is to simplify as much as possible.
it is obvious that you have given this much more thought then i have, and as a maintainer, one of the things that is in my top priorities is that this chart will be used!
lets go ahead and merge your change :)

/approve
/lgtm

Hi @jokogr , as now this change is live, do you have by any chance example how to use this, to create notifications file as you described?


Set a default and validate Certificate#attained_level

Certificate#attained_level is always assumed to have a sensible value elsewhere in the code, but is often null in the database. For example, used to look up the correct translation for display on the certificate view.

Ok, made that change.


chore: add docs for billing subscriptions

Pull Request Test Coverage Report for Build<PHONE_NUMBER>6
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 30.423%

Totals

Change from base Build<PHONE_NUMBER>5:
0.0%

Covered Lines:
7667

Relevant Lines:
25201

💛  - Coveralls


A couple of issues...

Two projects are unavailable because the don't exist, FastDeepCloner & Generic.LighDataTable, neither of which appear to be needed

Nuget restore fails every time

Yes i know. for working with the library download the nuget version. for development you you dont need  Generic.LighDataTable amd fastdeepcloner is library in the nuger. you can also download :)

That's fine, I can use the nuget version, but do you plan to fix this in the future?

Yes.
This project is under development and i'm working parallel with other project.
So when i have spare time will fix it,
Thx for mentioning the issues and please feel free to report any other.

No problem, I understand working in parallel, thanks for taking the time to respond!


feat: 16 idim service

Thanks for the PR!
Any successful deployments (not always required) will be available below.
Backend available
Once merged, code will be promoted and handed off to following workflow run.
Main Merge Workflow

Disabled the unit test pipeline for now, because the tests for idim web service needs vpn or run in openshift.


Rerender and bump

R 3.5.1

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

Blocked by https://github.com/conda-forge/staged-recipes/pull/6839


Add different domain

Most websites block the main domain, instead add one of the ones listed:
[   "1secmail.com",   "1secmail.org",   "1secmail.net",   "wwjmp.com",   "esiix.com",   "xojxe.com",   "yoggm.com" ]

Hey! Apologies for the delay, I rarely check this repository for updates. I'll have this reviewed and merged today I hope.
Thank you so much for the contribution!

Closing as obselete due to #6 .


Voice chat buttons

Also, I wanna thank you for keeping up with Discord updates, albeit a little slower as of now.
The update rate is very high and I believe it takes quite some time to fix the theme every week or two.


INDArray.sum(INDArray, int...) - no null check

The following code should (IMO) throw an exception - passing a null result array like this into the public API is unlikely to be what the user intended.
INDArray arr = Nd4j.create(10, 10);
arr.sum(null, 1);

Fixed. Will be merged later.


User restrictions based on karma

❗ NOTE:   this PR is made on top of  #94  (review and merge #94 first)
Implemented:

Slow mode for posts and comments (wait time between posting two posts or comments)
Commenting restrictions to the post with specific ID
Restrictions are calculated on the server and validated both on the server and the client
Voting restrictions

Current thresholds:
when effective karma < -10:

slow mode for posts 12 hours
slow mode for comments 10 minutes
disabled ability to vote for anything

when effective karma <= -1000 (lowest possible value currently):

ability create one last post if there aren't any currently
commenting restricted to the last own post
comment slow mode 1 hour
disabled ability to vote for anything

Demo:
(don't pay attention to timer values, see above for actual numbers)

User with karma -1000 and no posts can create one last post and comment only there:

Commenting slow mode demo:

Slow mode demo2:

Post slow mode demo:

Testing:

manually verified all comment/post scenarios for users:

with positive effective karma
with karma < -10
with karma == -1000

Disabled voting when effective karma <-10 (only on the server for simplicity):

Awesome) Meesa likadica)


What should we call this tool?

swift-public-api-generator whilst being verbose, is clearly un-imaginative. Any suggestions for names?
e.g.
swiftapi
facace

facade
guise (Doesn't matter what's behind the mask)
interface

I like Guise, let's go for that! Thanks @Sherlouk


Refactor history packages

What changed?
Moving structs in the service/history package to their own packages
The handler is moved to it's own package with the interface (like in frontend)
The handler wrappers (grpc, and thrift) are moved to their own packages under wrapped (like in frontend)
The engine implementation is moved to it's own package as there was a circular dependency
history (service) -> handler -> history (engine)
and the engine implementation cannot be in the same package as the engine interface, as there is a dependency
engineimpl -> shard -> engine (Interface)
Moved all the error values in handler to a constants package as at least one of them (ErrRunIDNotValid) is referenced from two different packages (handler and engineimpl tests)

Why?
We should not have this much in the same package
Also when service and handler are in the same package we cannot introduce wrappers around handler in any other package than history as that will create a circular dependency:
history (service) -> wrapper -> history (handler)

How did you test it?
Ran unit tests

Potential risks
It just moves things around, so no real code changes

Release notes

Documentation Changes

Pull Request Test Coverage Report for Build 018dc18a-181a-45be-8939-ecb15fdc8a26
Details

-86 of 91 (5.49%)  changed or added relevant lines in 3 files are covered.
272 unchanged lines in 26 files lost coverage.
Overall coverage decreased (-0.04%) to 62.702%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

service/history/handler/handler.go
1
87
1.15%

Files with Coverage Reduction
New Missed Lines
%

service/matching/matcher.go
1
91.07%

common/persistence/sql/common.go
2
72.73%

common/persistence/sql/sqlplugin/mysql/db.go
2
83.33%

common/persistence/sql/sqlplugin/mysql/task.go
2
73.68%

common/task/fifo_task_scheduler.go
2
85.57%

common/util.go
2
91.69%

service/history/task/fetcher.go
2
85.05%

service/history/task/transfer_standby_task_executor.go
2
87.01%

service/matching/db.go
2
73.23%

common/log/tag/tags.go
3
50.18%

Totals

Change from base Build 018db582-cf6a-4726-8ce9-adba3d0a8053:
-0.04%

Covered Lines:
92543

Relevant Lines:
147592

💛  - Coveralls


Rewrite dependency injection to use Dagger

Takeaways from this effort:

Dagger is ridiculously complex without a human to guide you around.
Koin's service locator pattern makes it far too easy to write bad code because you can inject anything anywhere.
Dagger can feel like a lot of boilerplate but some clever techniques can mitigate that.
Writing inject methods for every single class can feel like a drag because it is.
Injecting into Kotlin objects appears to be a no-go. I opted to refactor out the staticity where possible, pass injected dependencies to the function or fall back to 'dirty' patterns as needed. Do what you feel like.
I still do not love Dagger.


Power BI Content Pack Deprecation

The Power BI content pack for Azure AD has been deprecated as of October 1st.  This section should be removed from the article to reduce confusion.
https://docs.microsoft.com/en-us/azure/active-directory/fundamentals/whats-new#september-2019

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 53a138ee-fab3-0542-f8f7-084c7ee7514c
Version Independent ID: 0ef5a7e6-8e8b-f005-2be9-de0e5ea45efe
Content: Self-service password reset reports - Azure Active Directory
Content Source: articles/active-directory/authentication/howto-sspr-reporting.md
Service: active-directory
Sub-service: authentication
GitHub Login: @MicrosoftGuyJFlo
Microsoft Alias: joflore

@DanielBirmingham
Thanks for your feedback! We will investigate and update as appropriate.

I think that's a good call out. I made a pull request to remove this. https://github.com/MicrosoftDocs/azure-docs-pr/pull/92488


[Unity] Remove non-deterministic behavior from graph pattern matching

Graph pattern matching uses std::map and std::set keyed on pointers, which leads to non-deterministic matching result. This PR removed all non-determinism.
For example, consider matching against QKV projection before attention. Given a sequence of bindings,
with R.dataflow():
    lv0 = R.matmul(x, w0)
    lv1 = R.matmul(x, w1)
    lv2 = R.matmul(x, w2)
    ...

and patterns,
with PatternContext() as ctx:
    inp_pat = wildcard()
    Q_weight_pat = wildcard()
    K_weight_pat = wildcard()
    V_weight_pat = wildcard()

    matmul1 = is_op("relax.matmul")(inp_pat, Q_weight_pat)
    matmul2 = is_op("relax.matmul")(inp_pat, K_weight_pat)
    matmul3 = is_op("relax.matmul")(inp_pat, V_weight_pat)

intuitively I expect the following matching result:
Q_weight_pat - w0
K_weight_pat - w1
V_weight_pat - w2

But since three matmul patterns are independent of each other, each pattern can match any of three matmuls in the bindings. And since graph pattern matching processes patterns and variables in the bindings in a non-deterministic order, matching result becomes non deterministic as well.
To make graph pattern matching deterministic, I introduced an implicit ordering constraint on the patterns: Patterns are matched in the order that they are declared in the pattern context. So in the example above, we make sure to start pattern matching from the first matmul pattern. Likewise, the variables in the bindings, which are the candidates for matching, are also considered for matching in the order they appear in the bindings. So lv0 is matched first, followed by lv1 etc.
@ganler @sunggg

@ganler Is start_hint stuff necessary? I want to remove it, since matching is now deterministic. Some tests depend on the use of start_hint.

@masahi Thanks. Start hint means the start point of graph matching. It is useful when you have multiple matchable patterns and want to explicitly match one of them which are close to / subsuming certain node.

(Traveling) if you can wait I can file a review after 10 hours but definitely within 24 hours. :). Hope it won't get you blocked.

Just hit an interesting bug in the graph matcher.
The following fake QKV projection mod and pattern should not match. Without this PR I get segfault. With this PR, it incorrectly matches with the result
{*: x2, *: w1, Op(relax.matmul)(*, *): lv1, *: w2, Op(relax.matmul)(*, *): lv2, Op(relax.matmul)(*, *): lv0, *: w0}
@R.function
def main(...) -> R.Tensor:
    with R.dataflow():
        lv0 = R.matmul(x1, w0)
        lv1 = R.matmul(x2, w1)
        lv2 = R.matmul(x2, w2)
        ...

with PatternContext() as ctx:
    inp_pat = wildcard()
    ...

    matmul1 = is_op("relax.matmul")(inp_pat, Q_weight_pat)
    matmul2 = is_op("relax.matmul")(inp_pat, K_weight_pat)
    matmul3 = is_op("relax.matmul")(inp_pat, V_weight_pat)

I'll work on a fix in a separate PR.


clouddrift.wavelet + xarray

I am sharing a snippet of code showing how the wavelet transform function from clouddrift can be used with xarray dataarrays. It doesn't work directly because of the np.moveaxis function to start with. The proposed solution is to use xr.apply_ufunc, which seems to work with dask arrays as well. A remaining issue that I have not solved is that I would like to use morse_wavelet then wavelet_transform in order to calculate the wavelet only once in this case of time series of common length, instead of using morse_wavelet_transform which calculates the wavelet for each iteration, so not necessarily efficient.
https://gist.github.com/selipot/93c061f7715b95ce3ec3bcf169c1165a
I'd like to discuss!

I can reproduce your behavior and don't yet understand what's causing the error with the last one. Will need to study it more.