Add DejaVue - the Vue podcast.

Add DejaVue - the Vue podcast to remember- from Alexander Lichter (@manniL) and Michael Thiessen (@tesla3327)
It's now 18 Episodes in - with each one being about vue - or some part of vue.
General

✏ Mark the necessary items without changing the structure of the PR template.

[x] Pull request template structure not broken

Type

ℹ  What types of changes does your code introduce?

👉 Put an x in the boxes that apply

[ ] Fix
[x] Feature

Checklist

ℹ  Check all checkboxes - this will indicate that you have done everything in accordance with the rules in CONTRIBUTING.

👉  Put an x in the boxes that apply.

[x] Title as described
[x] Make sure you put things in the right category!
[x] Always add your items to the end of a list

Open Source

[ ] Link description does not contain a link to an author / third-party resource
[ ] The documentation (README) contains a description of the project, illustration of the project with a demo or screenshots and a CONTRIBUTING section
[ ] The documentation is in English.
[ ] The project is active and maintained.
[ ] The project accepts contributions.
[ ] Not a commercial product

Apps/Websites

[x] The website is available without errors or ssl certificate problems, and load in a reasonable amount of time.
[x] The website is using vuejs intensively. It should detect vue with vue-devtools.

If you cannot detect vue with vue-devtools due to work at non public pages (e.g. for enterprise website), you can send Pull Request with screenshot that detected it.

[ ] The website is original and not too simple. For that reason, blogs and simple landing pages are rejected.
[ ] A commercial product using Vue, provided that guests could reasonably check out how Vue was used (i.e. A headless CMS which uses Vue for the Admin/editor Area and offers a free tier).

Thank you for the PR 🙏


Can I turn on the spark speculation on rapids ? [QST]

spark will improve performan about long task when set the spark.speculation=true , i don't know if I set this flag, it will geting better or getting worse.
I attemp to set the flag, and it can work corretly.

according spark expericence ，my speculation conf is :
spark.speculation.quantile 0.9
spark.speculation true
spark.speculation.multiplier 3
spark.speculation.interval 1s

Speculative execution in Spark is fine with the RAPIDS accelerator plugin.
As with all speculative execution, it may or may not be beneficial depending on the nature of why a particular task is slow and how easily speculation is triggered.  You should monitor your Spark jobs to see how often task speculation occurs and how often the speculating task (i.e.: the one launched later) finishes before the slow task that triggered speculation.  If the speculating task often finishes first then that's good, it is working as intended.  If many tasks are speculating but the original task always finishes first then this is a pure loss, the speculation is adding load to the Spark cluster with no benefit.

got it, thanks


"import rapidfuzz" causes "illegal hardware instruction"

While I had used rapidfuzz previously without issues in my Python scripts, now every script that uses it crashes with an error such as illegal hardware instruction (Console) or Terminated due to signal: ILLEGAL INSTRUCTION (4) (Development environment)
The simplest script to demonstrate this just consists of
import rapidfuzz
Versions:
rapidfuzz 3.9.0 (installed with pip install rapidfuzz)
Python 3.11.9
After I reverted to rapidfuzz 3.8.1, the issue did not occur anymore
Thanks!

strange nothing really changed between these two versions that should affect the C++ version. Can you provide some context:

what is the CPU you are using?
what is the OS?

Machine information:
macOS 10.13.6
Intel(R) Core(TM) i7 CPU 870 @ 2.93GHz

I think I have an idea what this could be. I did build the wheels using the macos-latest image. Apparently github moved this from macOS 12 x86 to macOS 14 arm in between the builds. Possibly this raced e.g. the min macOS version.
I will make a test build using macOS 13 x86 and link you the wheel here for a test install once it's finished for you to test.

Can you test this wheel?
artifact-build_wheels_macos-3.zip

I installed the wheel with pip install ~/rapidfuzz-3.9.0-cp311-cp311-macosx_10_9_x86_64.whl
After that, running the test script still causes illegal hardware instructionand Python crashes.
The first lines of the crash report follow. Maybe this can give you a clue...
Process:               Python [3585]
Path:                  /opt/local/Library/Frameworks/Python.framework/Versions/3.11/Resources/Python.app/Contents/MacOS/Python
Identifier:            Python
Version:               3.11.9 (3.11.9)
Code Type:             X86-64 (Native)
Parent Process:        zsh [750]
Responsible:           Python [3585]
User ID:               501

Date/Time:             2024-05-18 11:52:12.219 +0200
OS Version:            Mac OS X 10.13.6 (17G14042)
Report Version:        12
Anonymous UUID:        A41421B4-F1A6-23F0-F507-8C741C7D2643

Sleep/Wake UUID:       269CD014-9694-4B86-8B91-B0410B57CBA8

Time Awake Since Boot: 18000 seconds
Time Since Wake:       3700 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_INSTRUCTION (SIGILL)
Exception Codes:       0x0000000000000001, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Illegal instruction: 4
Termination Reason:    Namespace SIGNAL, Code 0x4
Terminating Process:   exc handler [0]

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   metrics_cpp_avx2.cpython-311-darwin.so	0x000000010abc1a52 __pyx_pymod_exec_metrics_cpp_avx2(_object*) + 8242
1   org.python.python             	0x000000010a064de2 PyModule_ExecDef + 114
2   org.python.python             	0x000000010a16873e _imp_exec_dynamic + 12
3   org.python.python             	0x000000010a062bf8 cfunction_vectorcall_O + 104
4   org.python.python             	0x000000010a11b35e _PyEval_EvalFrameDefault + 62106
5   org.python.python             	0x000000010a10adce PyEval_EvalCode + 281
6   org.python.python             	0x000000010a1058b5 builtin_exec + 491
7   org.python.python             	0x000000010a062a53 cfunction_vectorcall_FASTCALL_KEYWORDS + 95
8   org.python.python             	0x000000010a11b35e _PyEval_EvalFrameDefault + 62106
9   org.python.python             	0x000000010a11dd6e _PyEval_Vector + 96
10  org.python.python             	0x000000010a00f9bf object_vacall + 296
11  org.python.python             	0x000000010a00f808 PyObject_CallMethodObjArgs + 227
12  org.python.python             	0x000000010a163150 PyImport_ImportModuleLevelObject + 2957
13  org.python.python             	0x000000010a103044 builtin___import__ + 199
14  org.python.python             	0x000000010a062a53 cfunction_vectorcall_FASTCALL_KEYWORDS + 95
15  org.python.python             	0x000000010a11b35e _PyEval_EvalFrameDefault + 62106
16  org.python.python             	0x000000010a11dd6e _PyEval_Vector + 96
17  org.python.python             	0x000000010a00f9bf object_vacall + 296
18  org.python.python             	0x000000010a00f808 PyObject_CallMethodObjArgs + 227
19  org.python.python             	0x000000010a162878 PyImport_ImportModuleLevelObject + 693
20  org.python.python             	0x000000010a114a89 _PyEval_EvalFrameDefault + 35269
21  org.python.python             	0x000000010a10adce PyEval_EvalCode + 281
22  org.python.python             	0x000000010a1058b5 builtin_exec + 491
23  org.python.python             	0x000000010a062a53 cfunction_vectorcall_FASTCALL_KEYWORDS + 95
24  org.python.python             	0x000000010a11b35e _PyEval_EvalFrameDefault + 62106
25  org.python.python             	0x000000010a11dd6e _PyEval_Vector + 96
26  org.python.python             	0x000000010a00f9bf object_vacall + 296
27  org.python.python             	0x000000010a00f808 PyObject_CallMethodObjArgs + 227
28  org.python.python             	0x000000010a163150 PyImport_ImportModuleLevelObject + 2957
29  org.python.python             	0x000000010a103044 builtin___import__ + 199
30  org.python.python             	0x000000010a062a53 cfunction_vectorcall_FASTCALL_KEYWORDS + 95
31  org.python.python             	0x000000010a11b35e _PyEval_EvalFrameDefault + 62106
32  org.python.python             	0x000000010a11dd6e _PyEval_Vector + 96
33  org.python.python             	0x000000010a00f9bf object_vacall + 296
34  org.python.python             	0x000000010a00f808 PyObject_CallMethodObjArgs + 227
35  org.python.python             	0x000000010a162878 PyImport_ImportModuleLevelObject + 693
36  org.python.python             	0x000000010a114a89 _PyEval_EvalFrameDefault + 35269
37  org.python.python             	0x000000010a10adce PyEval_EvalCode + 281
38  org.python.python             	0x000000010a1058b5 builtin_exec + 491
39  org.python.python             	0x000000010a062a53 cfunction_vectorcall_FASTCALL_KEYWORDS + 95
40  org.python.python             	0x000000010a11b35e _PyEval_EvalFrameDefault + 62106
41  org.python.python             	0x000000010a11dd6e _PyEval_Vector + 96
42  org.python.python             	0x000000010a00f9bf object_vacall + 296
43  org.python.python             	0x000000010a00f808 PyObject_CallMethodObjArgs + 227
44  org.python.python             	0x000000010a163150 PyImport_ImportModuleLevelObject + 2957
45  org.python.python             	0x000000010a114a89 _PyEval_EvalFrameDefault + 35269
46  org.python.python             	0x000000010a10adce PyEval_EvalCode + 281
47  org.python.python             	0x000000010a18c9e1 run_eval_code_obj + 78
48  org.python.python             	0x000000010a18ce31 run_mod + 96
49  org.python.python             	0x000000010a18cdaa pyrun_file + 133
50  org.python.python             	0x000000010a18c3dd _PyRun_SimpleFileObject + 558
51  org.python.python             	0x000000010a18b8b6 _PyRun_AnyFileObject + 136
52  org.python.python             	0x000000010a1b2e48 Py_RunMain + 2222
53  org.python.python             	0x000000010a1b3af0 pymain_main + 482
54  org.python.python             	0x000000010a1b3c74 Py_BytesMain + 42
55  libdyld.dylib                 	0x00007fff71a44015 start + 1

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x000000010ae3571f  rbx: 0x000000010ae444eb  rcx: 0x000000010ae4b1a8  rdx: 0x000000010ae4ae68
  rdi: 0x000000010ab1a6d0  rsi: 0x000000010ab50170  rbp: 0x00007ffee5c7c220  rsp: 0x00007ffee5c7b000
   r8: 0x3891b6418558226a   r9: 0x10b1d3cd09b79304  r10: 0x000000010ab501b8  r11: 0x000000010ab501a0
  r12: 0x000000010ab1a480  r13: 0x000000010aba53e8  r14: 0x000000010ae35420  r15: 0x0000000000000000
  rip: 0x000000010abc1a52  rfl: 0x0000000000010206  cr2: 0x000000010abc1000

Logical CPU:     6
Error Code:      0x00000000
Trap Number:     6

Thanks. Can you test the following package as well?
artifact-build_wheels_macos-0.zip
This is version 3.8.1 built with the current toolchains since I suspect this is a build issue and not an issue with the library itself.

This also causes the error message illegal hardware instruction in the console, followed by a cash of Python:
Process:               Python [61921]
Path:                  /opt/local/Library/Frameworks/Python.framework/Versions/3.11/Resources/Python.app/Contents/MacOS/Python
Identifier:            Python
Version:               3.11.9 (3.11.9)
Code Type:             X86-64 (Native)
Parent Process:        zsh [779]
Responsible:           Python [61921]
User ID:               501

Date/Time:             2024-05-19 10:40:48.508 +0200
OS Version:            Mac OS X 10.13.6 (17G14042)
Report Version:        12
Anonymous UUID:        A41421B4-F1A6-23F0-F507-8C741C7D2643

Time Awake Since Boot: 16000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_INSTRUCTION (SIGILL)
Exception Codes:       0x0000000000000001, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Signal:    Illegal instruction: 4
Termination Reason:    Namespace SIGNAL, Code 0x4
Terminating Process:   exc handler [0]

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   metrics_cpp_avx2.cpython-311-darwin.so	0x0000000103d76543 __pyx_pymod_exec_metrics_cpp_avx2(_object*) + 8179
1   org.python.python             	0x0000000103219de2 PyModule_ExecDef + 114
2   org.python.python             	0x000000010331d73e _imp_exec_dynamic + 12
3   org.python.python             	0x0000000103217bf8 cfunction_vectorcall_O + 104
4   org.python.python             	0x00000001032d035e _PyEval_EvalFrameDefault + 62106
5   org.python.python             	0x00000001032bfdce PyEval_EvalCode + 281
6   org.python.python             	0x00000001032ba8b5 builtin_exec + 491
7   org.python.python             	0x0000000103217a53 cfunction_vectorcall_FASTCALL_KEYWORDS + 95
8   org.python.python             	0x00000001032d035e _PyEval_EvalFrameDefault + 62106
9   org.python.python             	0x00000001032d2d6e _PyEval_Vector + 96
10  org.python.python             	0x00000001031c49bf object_vacall + 296
11  org.python.python             	0x00000001031c4808 PyObject_CallMethodObjArgs + 227
12  org.python.python             	0x0000000103318150 PyImport_ImportModuleLevelObject + 2957
13  org.python.python             	0x00000001032b8044 builtin___import__ + 199
14  org.python.python             	0x0000000103217a53 cfunction_vectorcall_FASTCALL_KEYWORDS + 95
15  org.python.python             	0x00000001032d035e _PyEval_EvalFrameDefault + 62106
16  org.python.python             	0x00000001032d2d6e _PyEval_Vector + 96
17  org.python.python             	0x00000001031c49bf object_vacall + 296
18  org.python.python             	0x00000001031c4808 PyObject_CallMethodObjArgs + 227
19  org.python.python             	0x0000000103317878 PyImport_ImportModuleLevelObject + 693
20  org.python.python             	0x00000001032c9a89 _PyEval_EvalFrameDefault + 35269
21  org.python.python             	0x00000001032bfdce PyEval_EvalCode + 281
22  org.python.python             	0x00000001032ba8b5 builtin_exec + 491
23  org.python.python             	0x0000000103217a53 cfunction_vectorcall_FASTCALL_KEYWORDS + 95
24  org.python.python             	0x00000001032d035e _PyEval_EvalFrameDefault + 62106
25  org.python.python             	0x00000001032d2d6e _PyEval_Vector + 96
26  org.python.python             	0x00000001031c49bf object_vacall + 296
27  org.python.python             	0x00000001031c4808 PyObject_CallMethodObjArgs + 227
28  org.python.python             	0x0000000103318150 PyImport_ImportModuleLevelObject + 2957
29  org.python.python             	0x00000001032b8044 builtin___import__ + 199
30  org.python.python             	0x0000000103217a53 cfunction_vectorcall_FASTCALL_KEYWORDS + 95
31  org.python.python             	0x00000001032d035e _PyEval_EvalFrameDefault + 62106
32  org.python.python             	0x00000001032d2d6e _PyEval_Vector + 96
33  org.python.python             	0x00000001031c49bf object_vacall + 296
34  org.python.python             	0x00000001031c4808 PyObject_CallMethodObjArgs + 227
35  org.python.python             	0x0000000103317878 PyImport_ImportModuleLevelObject + 693
36  org.python.python             	0x00000001032c9a89 _PyEval_EvalFrameDefault + 35269
37  org.python.python             	0x00000001032bfdce PyEval_EvalCode + 281
38  org.python.python             	0x00000001032ba8b5 builtin_exec + 491
39  org.python.python             	0x0000000103217a53 cfunction_vectorcall_FASTCALL_KEYWORDS + 95
40  org.python.python             	0x00000001032d035e _PyEval_EvalFrameDefault + 62106
41  org.python.python             	0x00000001032d2d6e _PyEval_Vector + 96
42  org.python.python             	0x00000001031c49bf object_vacall + 296
43  org.python.python             	0x00000001031c4808 PyObject_CallMethodObjArgs + 227
44  org.python.python             	0x0000000103318150 PyImport_ImportModuleLevelObject + 2957
45  org.python.python             	0x00000001032c9a89 _PyEval_EvalFrameDefault + 35269
46  org.python.python             	0x00000001032bfdce PyEval_EvalCode + 281
47  org.python.python             	0x00000001033419e1 run_eval_code_obj + 78
48  org.python.python             	0x0000000103341e31 run_mod + 96
49  org.python.python             	0x0000000103341daa pyrun_file + 133
50  org.python.python             	0x00000001033413dd _PyRun_SimpleFileObject + 558
51  org.python.python             	0x00000001033408b6 _PyRun_AnyFileObject + 136
52  org.python.python             	0x0000000103367e48 Py_RunMain + 2222
53  org.python.python             	0x0000000103368af0 pymain_main + 482
54  org.python.python             	0x0000000103368c74 Py_BytesMain + 42
55  libdyld.dylib                 	0x00007fff557b1015 start + 1

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000103fea72f  rbx: 0x0000000103ff94eb  rcx: 0x00000001040001a0  rdx: 0x0000000103fffe68
  rdi: 0x0000000103cce860  rsi: 0x0000000103d00cb0  rbp: 0x00007ffeecac8220  rsp: 0x00007ffeecac7020
   r8: 0xc0b04e9999526c62   r9: 0x01c27fe65c0f7904  r10: 0x0000000103d00cf8  r11: 0x0000000103d00ce0
  r12: 0x0000000103cce610  r13: 0x0000000103d5a3e8  r14: 0x0000000103fea440  r15: 0x0000000000000000
  rip: 0x0000000103d76543  rfl: 0x0000000000010206  cr2: 0x000000010bd2a000

Logical CPU:     5
Error Code:      0x00000000
Trap Number:     6

@henryiii can you have a quick look at this? You are usually more familiar with ecosystem changes in Python build tools.
To give a short recap of what is happening here:

mac os build from 07.04.2024 is working as expected https://github.com/rapidfuzz/RapidFuzz/actions/runs/8591457629/job/23540277660
mac os build of the exact same source code from 18.05.2024 is causing illegal instruction error on old versions of mac os https://github.com/rapidfuzz/RapidFuzz/actions/runs/9140181829/job/25133302598

So this sounds very much like something broke due to implicit updates of build tooling. Looking at the build log I noticed a couple of differences:

macos-latest changed from macOs 12 x86 to macOs 14 arm. I already tried macOs 13 x86 which leads to the same error. I couldn't get macOs 12 to work anymore. When building with it cibuildwheels says that I didn't select any targets https://github.com/rapidfuzz/RapidFuzz/actions/runs/9140101349/job/25133150472
apple clang 14 vs 15
both builds set CMAKE_OSX_DEPLOYMENT_TARGET:STRING=10.9 so they are still supposed to work for macOS 10.13.6

I don't see any other differences between the two builds, but maybe I am missing something. Do you have any idea what could go wrong here?

Can you give the following build a try:
artifact-build_wheels_macos-0.zip
This is the mac os version without the avx2 version.

As a first step,  I'd recommend using macos-X and macos-14, where X < 14, building only native packages on each. (I have no idea why 12 isn't working, that was the default for a long time and cibuildwheel should be fine on it, certainly shouldn't have an issue!) If you want a universal wheel too (pip won't ever download it with the native packages present), then you can fuse the native packages into a universal2 package. (I think it's not too hard, but don't remember where I saw it done).
The only similar issue I've seen so far was that either uv or ruff builds with an old macOS because there was an issue using the newer Xcode when targeting older macOS. I haven't checked yet to see exactly what the issue is, they are building on macos-11 for now. (maturin-action and Rust, not cibuildwheel)
I technically do have access to a macOS <11 machine (10.12 IIRC - it's an iMac with an NVIDEA graphics card, it's that old!), but it's in my old office and I won't be going there probably at least for a week. (Currently at PyCon, so def. don't have access to it now!)

Can you give the following build a try: artifact-build_wheels_macos-0.zip
This is a build without the avx2 versions

No crash from the test script 👍

On a branch macOS-12 worked. But sounds like this is the correct fix. Does avx make it noticeably faster? You could make a higher minimum version wheel too with avx if it was important.

On a branch macOS-12 worked.

I don't see anything directly obvious on what I might have done wrong. I don't see anything glaringly wrong in https://github.com/rapidfuzz/RapidFuzz/commit/177663a371577e4e56359221ad0d1bea51a92287 that would lead to no build types being selected

But sounds like this is the correct fix.

Which fix do you mean? Disabling avx2 using macOs-12 or something else?

Does avx make it noticeably faster?

When performing many x many comparisons it actually yields close to a 2x performance improvement in some cases, since it allows me to compare twice as many strings in parallel. So I would prefer to keep it. Currently I include the binary twice and just load the correct one based on CPU feature flags.

You could make a higher minimum version wheel too with avx if it was important.

Do you mean publishing both an 10.9+ version using only sse2 and a version targeting a later version which guarantees AVX2? If so which one does actually guarantee this? I mean the cpu in question here (i7 870) already supports AVX2.

I don't see anything directly obvious

I don't either, but #384 passes.

Disabling avx2 using macOs-12 or something else?

I was thinking avx2, but it was previously working, so to me this sounds like a bug. What's odd though the 13+ (probably Xcode is the important part, the previous macOS always updates Xcode once to next macOS's Xcode) does seem to work except on older macOS. It would be interesting to list the binary symbols, I think.
I think since macos-12 does work (in #384), the simplest thing might be to use that until GHA drops the image. I think the macos-11 image is still usable but deprecated, so it's probably ~1 year unless there are hardware reasons to drop it sooner.

Do you mean publishing both an 10.9+ version using only sse2 and a version targeting a later version which guarantees AVX2?

Yes, but I don't know what version this would be, somewhere between 10.13 and 13; since it only happens on the actual macOS versions, it would be really hard to pin down.
I rather expect this is a bug and it might get ironed out in later releases, if we could pin it down a bit more possibly could even report it.

I don't see anything directly obvious on what I might have done wrong in 177663a that would lead to no build targets being selected

You forgot a quote.

I don't see anything directly obvious on what I might have done wrong in 177663a that would lead to no build targets being selected

You forgot a quote.

Oh true

@workflowsguy I published a version with avx2 yesterday to take a bit of the urgency out of this.

I rather expect this is a bug and it might get ironed out in later releases, if we could pin it down a bit more possibly could even report it.

Yes I think so too. I mean it already fails in Cythons init functions for the module because it comes across an unsupported instruction. Since I only enable AVX2 which is supported on the CPU in question, this really shouldn't occur. So maybe an empty Cython module would be a good first test.

@workflowsguy I published a version with avx2 yesterday to take a bit of the urgency out of this.

Thanks!

I rather expect this is a bug and it might get ironed out in later releases, if we could pin it down a bit more possibly could even report it.

I can't really test this myself since I don't have access to a machine that leads to the crash to try and create a minimal reproducer. However this appears to already crash pretty early on while initializing the module. So we might be able to reduce the sample by quite a bit.


Time between scrapes?

Hi, I came across your visor-exporter to see if it will work with my Vigor 2860n, it seems it pulls lots of data from it, however I am wondering if I can reduce the time between scrapes/reads?
I am assuming your code does this every 1 minute?
 time.Sleep(60 * time.Second)
Is there any issue with me adjusting the above to say every 5 mins?  I
Thanks
Mike

This should really be controlled by the Prometheus scrape interval, rather than the exporter itself.


radiation can stop do afters

Description

taking radiation damage stops do afters
Reproduction

try to build a bananium statue - you can't because bananium is radioactive
Screenshots

Additional context

flashbacks to singuloose on core when pros are trying to pry open a firelock with lord singuloth behind them stopping the doafters :trollface:


refactor signatures

Closes #782

Nice work @alexcos20 !  Approved.
But FYI, it looks like the build (pull_request) check failed because the Moonbeam RPC endpoint was not responding for a short time. I just re-ran it to see if it's fixed now.


Gem install with RAILS_ENV=production is missing something

Doing a bundle install with IGNORE_ASSETS="no"  and RAILS_ENV=production causes several things to not work.
Logout link is broken/404 along with trying to make a namespace public and team creation.

This is weird (and I can't reproduce it). Are you sure that the needed gems have been installed ? Have you changed the Gemfile.lock somehow ? Which version of Portus are you using ?

I used this docker file to build and run it.
ENV PORTUS_VER 2.0.2

RUN git clone https://github.com/SUSE/Portus.git /portus && \
    cd /portus && \
	git checkout $PORTUS_VER

WORKDIR /portus

ENV IGNORE_ASSETS="no"
ENV RAILS_ENV=production
RUN bundle install --retry=3

RUN bundle exec rake assets:precompile
ENV IGNORE_ASSETS="yes"

ADD puma.rb /portus/config/

EXPOSE 3000

CMD ["/usr/local/bundle/bin/puma", "-C", "/portus/config/puma.rb"]

Removing these lines fixes things:
ENV IGNORE_ASSETS="no"
ENV RAILS_ENV=production

RUN bundle exec rake assets:precompile
ENV IGNORE_ASSETS="yes"

puma.rb
#!/usr/bin/env puma

# Workers and connections.
threads 1, 4
workers 3
bind "tcp://<IP_ADDRESS>:3000"

# #Daemon config. It will save the pid to tmp/pids/puma.pid. All the output
# #from both stdout and stderr will be redirected to logs/puma.log.
# log_file = File.join(Dir.pwd, "log/puma.log")
# stdout_redirect log_file, log_file, true
# pidfile File.join(Dir.pwd, "tmp/pids/puma.pid")
# daemonize

# Copy on write.
preload_app!

on_worker_boot do
  ActiveSupport.on_load(:active_record) do
    ActiveRecord::Base.establish_connection
  end
end

I tested this on what I think is the latest version of the rails container. This container ships with ruby 2.2.4p230 (2015-12-16 revision 53155) [x86_64-linux]
I see a note about only being compatible with ruby 2.1. Is this still the case?
http://port.us.org/docs/setups/1_rpm_packages.html#common-pitfalls

So officially we only support 2.1. That being said, future ruby versions should be ok, as we also test them. Could you use the same rails version that we are using, so ruby 2.1 is picked, and test if you can still reproduce it then ?

I do not currently have time to reproduce this error. If someone else runs into it they can look at simply removing the RAILS_ENV from their dockerfile.


Pre-initialize output xarray time coordinates and variables

Preinitializing an array at the start of a computation is well known to improve performance, because updating values in the array is much more CPU and memory efficient than resizing or appending to an array.
This issue separates a sub-thread started in this issue:

https://github.com/EcohydrologyTeam/ClearWater-modules/issues/57#issuecomment-1857094494

In that comment, @xaviernogueira describes the results of a simple test in the compute_engine_testing.ipynb notebook (currently in the 57-improve-timestep-performance-further branch) that showed it improved runtime by 4x-5x. See the link above for a discussion of these results.
NOTE that we expect all users to know exactly how many time steps they want to run before starting a simulation, so we do not need to preserve that current flexibility in the modeling system.

@aufdenkampe, I started working on this late last week by creating a profiling script and notebook as a benchmark. My plan for this week was to pre-init and run the same suite of tests.

Notebook: performance_profiling_tsm.ipynb
Profiling Script: performance_profiling_tsm.py

These could also serve as a point of comparison for any testing you do with xarray-simlab. I can work on making the profiling script more generalized/user friendly.

@sjordan29, having a common set of profiling scripts that we use for our various test branches sounds very useful! Thank you!
I just created a new Milestone for tracking these collective efforts: Performance enhancements for NSM demo, and I want to create a number of specific issues for us to each test and characterize with our different approaches.

Pasting https://github.com/EcohydrologyTeam/ClearWater-modules/issues/57#issuecomment-1857094494 here, to keep discussion on this specific issue in one place:

Strategy of writing to a pre-initialized set of xarray time slices

This was proposed by @aufdenkampe earlier as a potential avenue for performance increases.
While I have not had time to implement a fully version into the existing code for true apples to apples comparison (more on that later), I did make simplified xarray workflows in the compute_engine_testing.ipynb notebook to gage if the approach is worth developing.
The TLDR: Computation becomes 4-5x faster, however, implementation may involve some tradeoffs, especially when the user hasn't pre-decided how many timesteps will be executed.

Performance Comparison

I used the tutorial air_temperature Dataset from xarray, which is a (lat: 25, lon: 53, time: 50) grid.
The test was adding 50 extra time steps where a mock water_temp_c (renamed air temperature) is updated to +10 more than the last timestep.

Test 1:
I implement the current algo that is in clearwater_modules: copy the last timestep (makes a Dataset with len(time=1)), update it's values, the concatenate it at the end of our existing Dataset.

Time: 364ms
Test 2:
Here we make another (lat: 25, lon: 53, time: 50) dataset, where time ranges from 1+ the last time of the existing dataset, to 50+. All values are initialized as np.NaN. Then concat to our existing dataset, and iterate over the new indices writing to each time slice.
Creating the new Dataset and concatenating it takes about 9ms (O(0) time since it runs once). The test itself

Considerations:

An array the size of your output needs to be initialized upon startup, which can potentially hog memory for a very long model run, or make workflows where the data is saved to a file at regular intervals more difficult to implement.
Not a massive issue, but if the user doesn't know how many timesteps they will run a-priori, the code will have to make a "guess", and add those time slices with np.NaN. This can make the dataset look odd in between timesteps and can make debugging more difficult. Additionally, if one runs more timesteps than the guess, there needs to be logic to add in another "buffer" of empty time slices to write too.
One option around the above issue is to simply require the user to set a model iteration time, but is this desired?
That said, 4-5x faster is hard to argue against. Implementation shouldn't be too bad.

If the desired behavior is well constrained, I am happy to implement it, potentially past my office time in one shot. This can be discussed in our meeting this Friday (12/15/2023).

@aufdenkampe, some findings (also outlined in  performance_profiling_tsm.ipynb)
I worked to pre-initialize the model xarray and compare against the baseline model. When we track dynamic variables, we see some interesting behavior. On the graphs, each panel has the time/iteration on the y axis, the grid size on the x-axis, and each panel represents a different number of iterations (e.g., the top panel is for 1 iteration, the second panel is for 10 iterations, etc.).
For a low number of iterations and 1-1,000 cells, we see the baseline is faster, but when we start to get to a large number of cells and timesteps, the baseline computation time takes off while the updated computation time remains steady. I think it is slower than the baseline up front because of the additional overhead memory required to store an empty array over both space and time for all dynamic variables.

When we stop tracking dynamic variables and compare the two models, we see better outcomes for our updated pre-initialized model. For single-cell models, it is slightly slower (by <0.002 seconds), but once we get up to more than one cell, the pre-initialized model is faster. Once we get up to high numbers of cells and timesteps, the pre-initialized model is more than 10x faster than the baseline model

I imagine we can get the 1-cell model faster by:

Reducing dimensionality of state / dynamic variables if they aren't variable over the model mesh
Addressing #62
Additional memory management via #70

I'd also propose we set the default to not track dynamic variables so that new users don't have this computational expense by default (currently default is to track dynamic variables).

@sjordan29, those are really interesting results! Thanks for that impressive analysis. Your suggestions for next steps all sound spot-on.
The performance advantages are quite impressive for >10,000 time steps and grids >1,000 cells, which I think is where we need it the most.
It might be interesting to try a 10-cell or 100-cell model, just to get an idea of some intermediate differences.
For the slowdowns with larger number of time steps, I'm thinking that the answer might be:

#69

Great -- I'll run those additional tests to fill in the gaps. Then I think I can clean this branch up and merge to main, and we can pivot to some of the memory management issues.

Ran the tests for 10 and 100 timesteps as well. Start seeing the benefits at 100 iterations. Seems like the 1 and 10 timestep are more impacted by some natural variability since we're not averaging out over a large number of timesteps.

@aufdenkampe, a couple questions/notes while I get the tests updated.

I've made time_steps a required input. We could, optionally, default to a certain number of timesteps to run but this seems like a waste of memory and could promote inefficient use of the model.
I'll need to adjust how we handle a hotstart dataset. My sense is that we should grab the last timestep from a hotstart dataset and drop the rest of it to save on memory. Users could merge the two datasets on their own if desired.
Any thoughts/preferences?

I like both of you suggestions.

Require number of time_steps (or a time series / coordinate) as input.
For a hotstart dataset, grab the last tilmestep by default. There's no need to hold previous timesteps in memory while computing.

Sounds good. I am leaving time_steps as an integer in the original implementation to prevent this branch from getting too big, but opened #75 to address this in the future. Hotstart datasets have been addressed as described above and now all tests are passing.
I will probably also need to update example notebooks and documentation now that I am thinking of it.
Do you want me to work on a PR to main, or should we keep these speed/memory enhancements on a dev branch for now?


FEATURE - App name

Before we go public: Let's finally find a good name, that doesn't make it too complicated. It should be super easy to google it.
If we keep the "Bricks" thing, my favoured suggestions:

MIDI Brikks
MIDI Briqqs
...or just stay with MIDI Bricks

Of course open for very different suggestions! :)

..or just stay with MIDI Bricks


Create seeds as in Eloquent (Laravel)

https://laravel.com/docs/6.x/seeding

I was already working on something similar but then decided to drop it as it looked like unnecessary complex thing to write. Will look at how laravel does it, what I checked so far was how knex and typeorm seeding is working, but I did not really like the approach there, I saw too many things that could break easily (like the order of multiple seed files being decided by the file names). Also if one want the way knex is working already, then they can use knex directly (as it is direct dependency of MikroORM v3)...
Currently for seeding purposes, there is a CLI command to execute SQL dump (mikro-orm database:import <file>).

Looked at the link in more detail, and I kinda like it. Will try to incorporate something similar in one of 3.x releases.

👍 +1

@B4nan If that would help, we could expose something similar to CustomMigrationSource for seeds if you would like to reuse the rest of knex seeding functionality, or possibility to provide custom seed ordering implementation, to reduce amount of things you'd have to reimplement from a scratch.

@kibertoad thanks, but I will probably do it from scratch here, as I want to support it in mongo driver too. The implementation should be quite similar to migrations, so hopefully it won't be that time consuming. i might be able to reuse umzug here, or maybe get rid of it and replace it with custom shared implementation for both migrations and seeds.

You guys may want to look at , this is what I've been using. I like this approach better as it just take existing entity classes and generate random values/relations on the fly.

I have worked with https://github.com/w3tecch/typeorm-seeding before. It was not bad, but definitely could use improvement.
Currently I have written a very basic seeder for my project, but see a lot of potential in writing something with a similar api to Laravel's seeder and factory. I'd like to work in my own time on such a thing if you'd agree @B4nan. Then we could include that as a package or as command to the cli?

Seeding would be a really nice feature. I've also worked with typorm-seeding before. It is basically a must have for setting up and reinitialize the database for e2e testing purposes.

I am currently switching from TypeORM to Mikro ORM.
I have successfully used seeds with TypeORM migrations. I had a different set of configuration options for seeds with a separate migrations table and directory for seed "migrations". Inside those migrations, I've used different techniques - from pure insert queries to converting CSV files to SQL.


refresh infinite loop issue

I installed the package and I tested it in my react app. I am not sure for some reason the react app keeps refreshing again and again. How can this issue be resolved?

I'm facing with the same issue

I'm also facing the same issue when I enable the reloadOnDowngrade attribute. after the page refresh it keeps showing the version from meta.json as the new version found and keeps reloading the page.
Another issue is that it takes an extra reload (manual) after opening the website in order to see the new changes.
Has anyone found any solution for these issues?

Check package json and change number of version ,
"version": "22.0.0",
increamental, i was facing the same issue, but when i changed it to higher number then infinite loop stopped

I generally don't really bother with bumping the version in the package.json.
So, I use this postbuild script that automatically generates the meta.json file by appending the git commit hash to the end of the app version stated in the package.json.
// post-build.cjs
const fs = require("fs");
const cp = require("child_process");

const packageJson = require("./package.json");

const commitHash = cp
  .execSync("git rev-parse --short HEAD")
  .toString()
  .replace("\n", "");

const meta = {
  version: packageJson.version + "-" + commitHash,
};
const data = JSON.stringify(meta);

if (fs.existsSync("dist")) {
  fs.writeFileSync("dist/meta.json", data, { encoding: "utf8" });
  console.log('postbuild: Wrote application metadata to "dist/meta.json"');
  console.log("postbuild:", data);
}

// package.json
{
  "scripts": {
    "postbuild": "node ./postbuild.cjs"
  }
}

This glitch may be due to the browser cache not updating/erasing on refresh and that it keeps caching the old version/file of the react app. So, the script keeps asking for a new version on refresh and it creates a loop because the new version never comes.


Update scalafmt to 2.0.1

Updates org.scalameta:scalafmt from 2.0.0 to 2.0.1.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention @scala-steward in the comments below.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [{ groupId = "org.scalameta", artifactId = "scalafmt" }]

labels: semver-patch

Codecov Report

Merging #42 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #42   +/-   ##
=======================================
  Coverage   45.45%   45.45%
=======================================
  Files           1        1
  Lines          44       44
  Branches        1        1
=======================================
  Hits           20       20
  Misses         24       24

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5bc0278...98ae900. Read the comment docs.


Transmission efficiency between federal states is 70%, isn't that too low?

Will be updated in ID19

Transmission efficiency was updated in ID19


Grid And Phrase

Aim
Want to add a solution of "Grid And Phrase"
Kindly assign me this problem. I would like to add my solution to this problem in C++ easily and efficiently using recursion.
Link to the problem: https://www.hackerearth.com/practice/data-structures/arrays/multi-dimensional/practice-problems/algorithm/jadvaliioo-62280ff6/
Details
You are given an n*m grid which contains lower case English letters. How many times does the phrase "saba" appear horizontally, vertically, and diagonally in the grid?
Sample Input            Sample Output
55                               2
safer
amjad
babol
aaron
songs
Programming language

[ ] C
[ ] C++
[ ] Java
[ ] Python

Please assign me this issue under LGM-SOC'21

Please assign this issue under LGM-SOC'21 @ravikr126

i want to work on this issue under java language
please assign me @ravikr126


tests: switch mixer to v1alpha3

Prow has been using v1alpha1, which was a false positive.

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: kyessenov
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

tests/OWNERS [kyessenov]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@hklai if e2e-mixer passes on mixer, can you please make it required?

Do we need all the other args as in https://github.com/istio/istio/blob/master/tests/istio.mk#L217?
Once e2e-mixer-noauth-v1alpha3-v2 passes, we should make it required.

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: hklai, kyessenov
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

tests/OWNERS [hklai,kyessenov]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

The test is flaking in rate limit and ingress check cache. It did pass before.
Filed https://circleci.com/gh/istio/istio/115139?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link

@kyessenov: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e-bookInfoTests.sh
b790015a81e9c3cef8274a749ea9f4d8208b0160
link
/test e2e-bookInfo

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@kyessenov: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e-bookInfoTests.sh
b790015a81e9c3cef8274a749ea9f4d8208b0160
link
/test e2e-bookInfo

prow/e2e-simpleTests.sh
b790015a81e9c3cef8274a749ea9f4d8208b0160
link
/test e2e-simple

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@kyessenov: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e-bookInfoTests.sh
b790015a81e9c3cef8274a749ea9f4d8208b0160
link
/test e2e-bookInfo

prow/e2e-simpleTests.sh
b790015a81e9c3cef8274a749ea9f4d8208b0160
link
/test e2e-simple

prow/istio-pilot-e2e.sh
b790015a81e9c3cef8274a749ea9f4d8208b0160
link
/test istio-pilot-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@kyessenov: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/e2e-bookInfoTests.sh
b790015a81e9c3cef8274a749ea9f4d8208b0160
link
/test e2e-bookInfo

prow/e2e-simpleTests.sh
b790015a81e9c3cef8274a749ea9f4d8208b0160
link
/test e2e-simple

prow/istio-pilot-e2e.sh
b790015a81e9c3cef8274a749ea9f4d8208b0160
link
/test istio-pilot-e2e

prow/e2e-bookInfoTests-v1alpha3.sh
b790015a81e9c3cef8274a749ea9f4d8208b0160
link
/test e2e-bookInfo-envoyv2-v1alpha3

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Force merge please? circle unit tests flaked, too.

TestServices - kube flakes and it shows up in prow too.
https://k8s-gubernator.appspot.com/build/istio-prow/istio-postsubmit/2165/
I just kicked off a rerun in circle.


ci: changed travis ci to github actions

https://github.com/atassis/abort-controller-x/runs/2654511114?check_suite_focus=true

Thanks!


Impact of enabling a Service Endpoint for access to Blob Storage

If an existing Azure Storage Blob is used by devices on the public internet (over HTTPS using access keys), and we deploy a VM in the same region and enable Service Endpoints on that VNET (so that the VM can access the blob over your private network to avoid flowing over the public internet resulting in network egress fees), will this DISABLE access via the public internet that our existing resources were using, resulting in an outage to those resources?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 1eb0407d-f4a5-6059-488d-fd082915c675
Version Independent ID: 7c2fb429-743c-aea0-f1d5-de6dc2e62e8e
Content: Azure virtual network service endpoints
Content Source: articles/virtual-network/virtual-network-service-endpoints-overview.md
Service: virtual-network
GitHub Login: @sumeetmittal
Microsoft Alias: sumi

@bultaf14 VNET Service Endpoints creates a separate path for traffic in your Virtual Network to route to an Azure Storage account in the same region, and it should have no effect on your storage account.
Azure Storage Firewall is what controls access from the storage account side. As long as the firewall allows access from the public internet, existing resources will not be affected.

@TravisCragg-MSFT, I appreciate your response but need a definitive answer. "Should have no effect" is not definitive. The diagram in the middle of this page illustrates that internet traffic is blocked to the service (see the red X), however, there is no description on this page that explains what causes it to be blocked. There is plenty of detail explaining how a VM within an Azure VNET is impacted when Service Endpoints are enabled (this causes the VMs on the impacted VNET to start communicating with the service via it's private IP instead of it's public IP, which will therefore impact firewall rules since the VM's source IP is now different). However, it doesn't explain how OTHER (non-Azure) traffic (such as devices on the internet) are impacted to the service (in our case, Storage Blob).

@bultaf14 Allow me to make the statement more clear.
Virtual Network Service Endpoints for Microsoft.Storage will only affect the outbound storage traffic coming from the Subnet that it is enabled on, and will have no effect on public access of a storage account by itself.
Access to storage accounts is controlled via the Azure Storage Firewall, and both combined can be used to allow a storage account to be accessed only from a particular VNET and not via the Public Internet, as is shown in the diagram.
Please let me know if you need any additional clarification.

@TravisCragg-MSFT thanks for clarifying. It's good to know that these are handled independently and have no impact on one another.

@bultaf14 I don't think you got the answer you need. Read this:
https://docs.microsoft.com/en-us/azure/storage/common/storage-network-security?toc=%2Fazure%2Fvirtual-network%2Ftoc.json#change-the-default-network-access-rule
I believe you're just asking if turning on service endpoints to storage will start denying all other traffic, and the answer is YES.
Travis is sort of "technically correct" in that the SERVICE ENDPOINT doesn't have anything to do with the traffic coming to your Storage account from the Internet, but he's missing the point that by enabling the Storage virtual network rule (allowing traffic from "Selected networks only") you switch your Storage account to deny all other traffic.

@dcbrown16 Thanks for the additional clarification. It is important to note that Service Endpoints and Storage Firewalls are different things, but typically are used together to only allow traffic from a specific VNET to a storage account.
@bultaf14 We will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.


Use public domain turn.domain for UDP iceServers

Describe the bug
Now server return iceServers which use NodeIp for TURN/UDP, but domain for TURN/TCP:
{ iceServers: [turn:<IP_ADDRESS>:3478?transport=udp, turns:livekit-turn-dev.host.com:443?transport=tcp], iceTransportPolicy: all, bundlePolicy: balanced, rtcpMuxPolicy: require, iceCandidatePoolSize: 0 }

The line of code in which this logic is implemented:
https://github.com/livekit/livekit/blob/d8629172494f046a9827edbab68e5a7a62833772/pkg/service/roommanager.go#L928C77-L928C83
Server

Version: [1.8.0]
Environment: Kubernetes

To Reproduce
livekit:
  prometheus:
    port: 6789
  log_level: info
  rtc:
    use_external_ip: false
    tcp_port: 7881
    udp_port: 7882
  turn:
      enabled: true
      domain: livekit-turn-dev.host.com
      tls_port: 443
      udp_port: 3478
      external_tls: true

Expected behavior
Is it possible to use a domain for UDP also?
{ iceServers: [turn:livekit-turn-dev.host.com:3478?transport=udp, turns:livekit-turn-dev.host.com:443?transport=tcp], iceTransportPolicy: all, bundlePolicy: balanced, rtcpMuxPolicy: require, iceCandidatePoolSize: 0 }

You can disable embedded TURN here and set up your own TURN servers here.
That will be used by code here.


Remove unnecessary type annotation leading to build errors

I also incremented the version to allow a new release

Hi @xmakro, thanks for your contributions. I updated tiktoken-rs to include code from tiktoken at v0.8.0 addressing the compile issues that were reporting.
Is this PR still needed?


fix(docker-swarm) let the chmod command fail so docker-swarm is happy

A possible resolution to https://github.com/Kong/docker-kong/issues/224

No longer needed


Nutch-2510

Now host db is not created automatically in the crawl script, you can use hostdb for generate plut the description is added.

Now we check existence of hostdb directroy before we process
echo "Generating a new segment"
if [[ "$HOSTDBGENERATE" = true  && -d "$CRAWL_PATH"/hostdb ]];
One question left,  how should we modify(or shouldn't) this part?
if [[ "$SITEMAPS_FROM_HOSTDB_FREQUENCY" = "always" || ("$SITEMAPS_FROM_HOSTDB_FREQUENCY" = "once" && $a -eq 1) ]]; then
# create / update HostDB on first run
[[ $a -eq 1 ]] && __update_hostdb
# sitemap processing based on HostDB
if [[ -d "$CRAWL_PATH"/hostdb ]]; then
  echo "Processing sitemaps based on hosts in HostDB"
  __bin_nutch sitemap "$CRAWL_PATH"/crawldb -hostdb "$CRAWL_PATH"/hostdb -threads $NUM_THREADS
fi

fi


fix(editor): ensure wait before serialising

In cases when the editor is rendered and very quickly set the quads property, it may happen that the serialising starts before code mirror is ready and the serialised output get lost somehow
This fixes this problem by waiting for this.ready to resolved before starting to parse

Codecov Report

Merging #56 (bd04d6a) into master (0707006) will increase coverage by 0.01%.
The diff coverage is 100.00%.

:exclamation: Current head bd04d6a differs from pull request most recent head 66305ae. Consider uploading reports for the commit 66305ae to get more accurate results

@@            Coverage Diff             @@
##           master      #56      +/-   ##
==========================================
+ Coverage   93.22%   93.23%   +0.01%
==========================================
  Files          15       15
  Lines        1180     1182       +2
==========================================
+ Hits         1100     1102       +2
  Misses         80       80

Impacted Files
Coverage Δ

packages/rdf-editor/src/RdfEditor.js
95.02% <100.00%> (+0.04%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0707006...66305ae. Read the comment docs.


[Bug]: The pipelines are duplicated

Apache Hop version?
2.8.0
Java version?
11
Operating system
Docker
What happened?
I'm deploying a pipeline to a remote server, however the pipeline is duplicated n times and doesn't run

Issue Priority
Priority: 3
Issue Component
Component: Hop Server

The problem concerned an incorrect configuration of the "Run Configuration" parameter which instead of being set as local was set as remote, creating a loop

thanks @Matt87s for the reply.


The random seed doesn't work

Even if I set the same random seed, the result is different, and you can test it on ddpg. I think tf.set_random_seed(seed) doesn't work, but I don't know how to solve it.

Can you give an example with what you expected to happen and what actually happened?
If tensorflow is failing to set a set, maybe you should file a bug on tensorflow.
For gym environments and numpy (for example) are seeded differently, and you should not expect seeding tensorflow to affect either of them.

@machinaut @jachiam ,I run the common python ddpg.py -s 2, the first time I got AverageTestEpRet=-594 of epoch one,but the second time I got the AverageTestRet=-314 of epoch one. And I tried to set
env.seed(seed)
test_env.seed(seed)

but the AverageTestEpRet of epoch one was still different.

@XFFXFF, hmm, that's a bit unfortunate. I appreciate that you tried this out.
I am not fully sure what could be going wrong here. My suspicion is that it might involve the Python hash seed used to prevent dict collision attacks. See here for an explanation of the issue, and see here for more info. Can you try export PYTHONHASHSEED=0 and then try running your experiment again?

@XFFXFF I tried out export PYTHONHASHSEED=0, in addition to setting env seed and test_env seed, and did two runs of DDPG with python -m spinup.run ddpg --hid [32] --env HalfCheetah-v2 --steps_per_epoch 1000.
Results from first run:
---------------------------------------
|             Epoch |               1 |
|      AverageEpRet |            -260 |
|          StdEpRet |               0 |
|          MaxEpRet |            -260 |
|          MinEpRet |            -260 |
|  AverageTestEpRet |            -533 |
|      StdTestEpRet |            3.39 |
|      MaxTestEpRet |            -525 |
|      MinTestEpRet |            -536 |
|             EpLen |           1e+03 |
|         TestEpLen |           1e+03 |
| TotalEnvInteracts |           1e+03 |
|      AverageQVals |           0.508 |
|          StdQVals |            1.17 |
|          MaxQVals |            5.88 |
|          MinQVals |           -6.88 |
|            LossPi |           -1.39 |
|             LossQ |           0.964 |
|              Time |            6.86 |
---------------------------------------

Results from second run:
---------------------------------------
|             Epoch |               1 |
|      AverageEpRet |            -260 |
|          StdEpRet |               0 |
|          MaxEpRet |            -260 |
|          MinEpRet |            -260 |
|  AverageTestEpRet |            -533 |
|      StdTestEpRet |            4.89 |
|      MaxTestEpRet |            -526 |
|      MinTestEpRet |            -541 |
|             EpLen |           1e+03 |
|         TestEpLen |           1e+03 |
| TotalEnvInteracts |           1e+03 |
|      AverageQVals |           0.508 |
|          StdQVals |            1.17 |
|          MaxQVals |            5.88 |
|          MinQVals |           -6.88 |
|            LossPi |           -1.39 |
|             LossQ |           0.964 |
|              Time |              16 |
---------------------------------------

Looks like this solves the issue. I'm going to mark this as closed.

Scratch that, double-checking and it looks like things diverge after Epoch 1. Don't know where this nondeterminism is coming from.

With env seed setting and export PYTHONHASHSEED=0, TRPO/PPO/VPG are deterministic through at least the first three epochs. I have no idea why DDPG would be different.

Is this issue happening in both tensorflow and pytorch? Have the operation-level seeds been properly set?

With following env seed setting
    env.seed(seed)
    env.action_space.seed(seed)
    test_env.seed(seed)
    test_env.action_space.seed(seed)

the result is same in same seed.


FAQ and TROUBLESHOOTING Documentation

Added Documentation of FAQ and Troubleshooting page.

How to delete the chaos engine if it gets stuck in finalizer?
How to cleanup predefined workflows?

@ajeshbaby @ksatchit found these two questions in the doc, it is relevant still, and if yes can we add them?

Yes good to add these.


DatePicker: Fix onChange flowtype

Flowtype fix (exact type)
TODO

[ ] Accessibility checkup
[ ] Update documentation
[ ] Add/update Tests
[ ] Pinterest Designer (for design changes)
[ ] Format PR title: ComponentName: Description.

@AlbertCarreras thanks for the fix, could you make sure to always use the following format: ComponentName: Description
So
DatePicker: Fix onChange flowtype
instead of
[DatePicker] Flowtype fix


Add new enum to specify the direction of a relationship

Introduce a new enum Direction in the @Relationship annotation to replace String based relationship direction
Description

add Relationship.Direction enum and change type of "direction" member to that
new enum
introduce converter branch and new method in AnnotationInfo class to support
enum members of annotations
adapt return type of methods which previously returned the relationship direction string
to now return an enum constant to only use the new enum constants internally
adapt @Relationship annotations in node entities for integration tests
adapt the docs to reflect the changed return type

Related Issue
#631
Motivation and Context
Specifying the direction of a relationship through a simple string may lead to confusion and might cause unexpected behaviour when a typo is overlooked
How Has This Been Tested?
Running existing test cases and 4 newly added ones
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[x] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] My code follows the code style of this project.
[x] My change requires a change to the documentation.
[x] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[x] I have added tests to cover my changes.
[x] All new and existing tests passed.

Codecov Report

Merging #785 into master will increase coverage by 0.02%.
The diff coverage is 95.78%.

@@             Coverage Diff              @@
##             master     #785      +/-   ##
============================================
+ Coverage     80.35%   80.38%   +0.02%
- Complexity     3002     3005       +3
============================================
  Files           274      275       +1
  Lines          9169     9176       +7
  Branches       1379     1379
============================================
+ Hits           7368     7376       +8
  Misses         1299     1299
+ Partials        502      501       -1

Impacted Files
Coverage Δ
Complexity Δ

...neo4j/ogm/context/DirectedRelationshipForType.java
57.14% <ø> (ø)
3.00 <0.00> (ø)

.../java/org/neo4j/ogm/context/GraphEntityMapper.java
90.93% <ø> (ø)
84.00 <0.00> (ø)

...ore/src/main/java/org/neo4j/ogm/cypher/Filter.java
90.56% <ø> (ø)
47.00 <0.00> (ø)

...iler/builders/node/DefaultRelationshipBuilder.java
91.89% <ø> (ø)
19.00 <0.00> (ø)

...g/neo4j/ogm/session/delegates/SessionDelegate.java
93.15% <75.00%> (ø)
23.00 <0.00> (ø)

...eo4j/ogm/metadata/reflect/EntityAccessManager.java
68.83% <88.46%> (ø)
68.00 <10.00> (ø)

...in/java/org/neo4j/ogm/annotation/Relationship.java
100.00% <100.00%> (ø)
0.00 <0.00> (?)

...va/org/neo4j/ogm/context/DirectedRelationship.java
62.50% <100.00%> (ø)
5.00 <1.00> (ø)

...in/java/org/neo4j/ogm/context/EntityCollector.java
82.97% <100.00%> (ø)
10.00 <0.00> (ø)

.../java/org/neo4j/ogm/context/EntityGraphMapper.java
93.38% <100.00%> (ø)
130.00 <3.00> (ø)

... and 13 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a48d593...2a44a4f. Read the comment docs.

Thanks for letting me push into your branch.

Looks very good to me now… I'm gonna go through one more time and might push a thing or two.
Would you please sign our CLA? https://neo4j.com/developer/cla/ so that I can whitelist you in our CI? Thank you.

Done ✔


Calculate time between last PR commit and merge

Rough implementation computing the desired metrics.
Not yet implemented: the ability to include/exclude repos by name

Is this ready for review, or should this be a draft?

@ryanmoran The current implementation does what is outlined in the acceptance criteria, but what it does could be further refined in a few ways we identified along the way of implementing a first iteration. For instance:

including/excluding repos by name (so we can remove Java repos if we want).
doing something more clever to filter out PRs that sat un-reviewed for a long time

I'm happy to hear feedback from folks about whether we want those and/or other refinements, or if we're good with this initial implementation.

Just realized we should probably add some error statement if the PAKETO_GITHUB_TOKEN isn't set so the user knows to set it

I changed it up and added concurrency! Also added a check for the github token being set before it's off to the races.
Unfortunately, it's really easy to hit the github API abuse rate limit making concurrent requests, so the efficiency gains may be purely intellectual/theoretical here.
Not yet added: Reasonable error handling. Still figuring out how to do that well with channels.

Looks pretty good to me, although I think Ryan should probably leave his review as well because I'm not entirely familiar with channels.
I'm curious if you ran into the API abuse rate limit while testing this out locally?

Yeah, I saw it locally. It seems like it's easy to trigger the rate limit if you're making concurrent requests with the same auth token. They expressly indicate that concurrent requests isn't something they want you to do :'(

@fg-j This looks cool! Have you thought about adding tests, we usually have them for our go in this repo.

When you run this, are you using your own PAT or the Paketo bot token?

@ryanmoran I was using the Paketo token when i was testing it. Haven't tried it with my own token, actually.

I was wondering because I tried with my own token and haven't hit any sort of concurrency limit. I wonder if they are detecting requests from different hosts and you could run into that if you are running the script and an action on a repo is also running.

In the new implementation changes, I've set the default number of concurrent workers to 1 -- @ryanmoran try upping that with your own or the Paketo token and see if you hit the abuse limit. I still saw the same problem when using my PA.


Browsersync not hot-reloading

I run 'gulp'
and then
cd public/
browser-sync start --server --files "js/.js, css/.css"
but hot-reload doesn't happen. I have to run gulp and then browser-sync again to reflect the changes.

Hi, I have had this issue before with a simple gulp watch. Usually this means your system does not allow for this many files to be "watched" at the same time. You can change those limits but this depends heavily on your system setup. It is also not an issue with the actual code I think. If this pops up for somebody else, please reopen.


Update Overview.md

Added NuGet Server (Open Source) which is a very similar implementation, however it uses IISExpress instead of Cassini

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/Hosting-Packages/Overview.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Thanks for the addition--we appreciate it!


Update basics.md

Fixes #12921

At the same time Can you fix the alignment issue of the itemgroup snippets at https://docs.microsoft.com/en-us/aspnet/core/grpc/basics?view=aspnetcore-3.0#add-a-proto-file-to-a-c-app and https://docs.microsoft.com/en-us/aspnet/core/grpc/basics?view=aspnetcore-3.0#generated-c-assets
It looks like they have extra  on the last lines.

@JunTaoLuo thanks, I'll push fixes live now.


Update all file to the newest version

Update all file to the newest version

Update all file to the newest version


DataGrid: Make extension methods to apply filters

Description
Added some extension methods on IEnumerable in DataGridExtensions.
This makes it more obvious and convenient to apply filters in cases such as when using the ServerData parameter.
Instead of having to iterate over the IEnumerable and apply .Where(filter.GetnerateFilterFunction()) for each filter,
users will have a more obvious method to accomplish this common behaviour, which is precedented by the already
existing IEnumerable.OrderBySortDefinitions extension method.
How Has This Been Tested?
Changes tested via integration and experimentation with my own code.
Type of Changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation (fix or improvement to the website or code docs)

Checklist

[x] The PR is submitted to the correct branch (dev).
[x] My code follows the code style of this project.
[x] I've added relevant tests.

Hi. Thank you for your contribution.
Please add unit tests for these extension methods.

I have already opened a pull request a while ago that covers this topic, was discussed and is fully covered with unit tests: #8254
I just rebased it on the dev branch, it's ready to merge.

I have already opened a pull request a while ago that covers this topic, was discussed and is fully covered with unit tests: #8254
I just rebased it on the dev branch, it's ready to merge.

Can this PR be closed then?

This one is a little different as it operates on IEnumerable<T> rather than IQueryable<T>.
I'd still simplify it to this (also renaming ApplyFilters simply into Where) and, as already mentioned, adding tests.
public static IEnumerable<T> Where<T>(this IEnumerable<T> source, GridState<T> state, FilterOptions? filterOptions = null)
    => Where(source, state.FilterDefinitions, filterOptions);

public static IEnumerable<T> Where<T>(this IEnumerable<T> source, ICollection<IFilterDefinition<T>> filterDefinitions, FilterOptions? filterOptions)
    => filterDefinitions.Aggregate(source, (current, filter) => current.Where(filter.GenerateFilterFunction(filterOptions)));


Incomplete SSL certificate chain

Hi there, I wasn’t sure where to send a technical issue like this, but, I’m having some problems with the API stemming from the way your server is configured.
Your server is returning your site's SSL certificate, but not GoDaddy’s intermediate certificate, which means it works okay on desktop systems which have clever storage of intermediate certificates they’ve seen on the web, but on server-type systems presents a problem as the intermediate certificate is intended to be sent as part of the certificate chain so your certificate can be verified to be trusted against the root certificates.
You can take a look at the SSL Labs report yourself: https://www.ssllabs.com/ssltest/analyze.html?d=collections.museumvictoria.com.au
The pertinent part is here:

It looks like your server requires some reconfiguration to include the intermediate certificate (that “Go Daddy Secure Certificate Authority - G2” one), as described in GoDaddy’s documentation: https://ca.godaddy.com/help/what-is-an-intermediate-certificate-868

Oh hey thanks for the heads up will forward this on to the server admin : )

Ok apparently they are all over this issue and will be sorting it out soon....

Sorry bout the wait, but we have moved the site from our old domain to the new one that has the correctly managed certs.  Old domain is still there but you should now get a 301 redirect to the new url which is now...
https://collections.museumsvictoria.com.au/

A 301 won’t do anything if the old domain still can’t be connected to, unfortunately!

The new domain does, in fact, include the necessary intermediate certificate. I’ll update my code to specify the new domain! Thanks for looking into this. ☺


LSP[markdown_oxide]: Error INVALID_SERVER_MESSAGE: -32700 Parse error

Often when I paste large chunk of text into current .md file I got this:
LSP[markdown_oxide]: Error INVALID_SERVER_MESSAGE: {
  error = {
    code = -32700,
    message = "Parse error"
  },
  jsonrpc = "2.0"
}

This happens only with markdown-oxide (I've also efm LSP connected to run markdownlint), so I don't really believe Neovim actually sends an invalid JSON request. All other kinds of "markdown parse errors" shouldn't use that code and shouldn't crash the server.
Another issue is LSP doesn't restart automatically after that error, so I need to manually do :LspStart markdown_oxide - but this is probably a general Neovim issue.
Let me know if you need LSP logs with request/response traces, I'll try to reproduce this issue with enabled logging.

Thank you for reporting! Sorry you are experiencing this.
More LSP logs may help if you don't mind. Possibly relevant neovim configuration too.

It turns out to be racy! I repeat the same simple repro sequence a lot of times:

vi test.md (non-existing file)
A<Enter><Enter><Esc>p (I don't think A<Enter><Enter><Esc> part is needed, but as it's racy I repeat the one I know for sure may trigger the issue and don't waste more time experimenting here).

and it triggers the issue about 1/10 times.
I've temporary switched off markdownlint to avoid unrelated messages in a log.
markdown-oxide.lsp1.log

Mason says I'm using markdown-oxide v0.23.1.

hmmmm thank you
I wonder if it's related to the non-ascii characters

I wonder if it's related to the non-ascii characters

Me too. Tried to repro using english, but as it's racy I give up after some fair amount of attempts.

Which LSP client are you using?

I'm finding that this is an error in parsing the JSON of the RPC

"the message from the server cannot be parsed"

I'm finding that this is an error in parsing the JSON of the RPC

Yeah. At least it shouldn't be used for other reasons: https://www.jsonrpc.org/specification#error_object

Which LSP client are you using?

Just Neovim (I'm using neovim/nvim-lspconfig).

Hm. So no nvim-cmp?

I've nvim-cmp, but it has nothing with this. It doesn't affect existing LSP requests and I didn't used completion keymap so it shouldn't send it's own LSP requests too.

Anyway, it shouldn't and doesn't matter which plugins I've in Neovim. JSON-RPC 2 is a network protocol, and you see all network messages in a log, so if sent from Neovim messages are valid then it doesn't matter who sends them.

This is related: https://github.com/ebkalderon/tower-lsp/issues/267

I don't believe I am able to reproduce this.
Could you send a neovim configuration and version information with your issue?

Still trying to produce a minimal repro.lua, no luck so far.
But I noticed something interesting! This is request which result in an error:
[DEBUG][2024-07-26 20:35:52] .../vim/lsp/rpc.lua:286	"rpc.send"	{
  jsonrpc = "2.0",
  method = "textDocument/didChange",
  params = {
    contentChanges = { {
        text = "\n\n  - п╢п╬п╠п╟п╡п╦я┌я▄ PostgreSQL\n  - п╢п╬п╠п╟п╡п╦я┌я▄ HSTS, CSP п╦ п©я─п╬я┤п╦п╣ /etc/nginx/conf.d/headers.conf\n  - п╢п╬п╠п╟п╡п╦я┌я▄ LPC я┤п╣я─п╣п╥ gRPC\n    - https://github.com/fullstorydev/grpchan\n      - https://github.com/apssouza22/grpc-production-go я┌я┐я┌ п╣я│я┌я▄ п©я─п╦п╪п╣я─\n    - п╨п╟п╨ п╢п╣п╩п╟я┌я▄ discovery/mock?\n  - п╢п╬п©п╦я│п╟я┌я▄ ms/example:\n    - п╠п╬п╩я▄я┬п╣ API (п╦ я│п╪я▀я│п╩п╟)\n    - п╠п╬п╩я▄я┬п╣ я┌п╣я│я┌п╬п╡\n    - п╟п╢п╟п©я┌п╣я─ п╨ ms/auth\n  - п╢п╬п╠п╟п╡п╦я┌я▄ п╡п╟я─п╦п╟п╫я┌ я│ legacy-п╨п╬п╢п╬п╪ п╡ old/ я┐ п╬п╢п╫п╬пЁп╬ п╦п╥ example?\n  - п©п╬п©я─п╬п╠п╬п╡п╟я┌я▄ consul п╡п╪п╣я│я┌п╬ env.sh\n    - п╠п╬п╩я▄я┬п╦п╫я│я┌п╡п╬ п╫п╟я│я┌я─п╬п╣п╨ п╦п╪п╣я▌я┌ defaults\n    - п©я─п╦ п╥п╟п©я┐я│п╨п╣ п╬я┌я│я┐я┌я│я┌п╡я┐я▌я┴п╦п╣ п╫п╟я│я┌я─п╬п╧п╨п╦ я│п╬п╥п╢п╟я▌я┌я│я▐ я│ default-п╥п╫п╟я┤п╣п╫п╦я▐п╪п╦\n    - п╨п╟п╨ я─п╣я┬п╦я┌я▄ host-specific п╫п╟я│я┌я─п╬п╧п╨п╦?\n    - п╨п╟п╨ п╦п╥п╠п╟п╡п╩я▐я┌я▄я│я▐ п╬я┌ п╫п╣п╦я│п©п╬п╩я▄п╥я┐п╣п╪я▀я┘ п╫п╟я│я┌я─п╬п╣п╨?\n      - я│ я┐я┤я▒я┌п╬п╪ я┌п╬пЁп╬, я┤я┌п╬ пЁп╢п╣-я┌п╬ п╪п╬п╤п╣я┌ я─п╟п\n"
      } },
    textDocument = {
      uri = "file:///home/powerman/doc/Main/test.md",
      version = 6
    }
  }
}

and this is same request with about same content which does not result in error:
[DEBUG][2024-07-26 23:01:02] .../vim/lsp/rpc.lua:286	"rpc.send"	{
  jsonrpc = "2.0",
  method = "textDocument/didChange",
  params = {
    contentChanges = { {
        text = "\n\n\n  - добавить PostgreSQL\n  - добавить HSTS, CSP и прочие /etc/nginx/conf.d/headers.conf\n  - добавить LPC через gRPC\n    - https://github.com/fullstorydev/grpchan\n      - https://github.com/apssouza22/grpc-production-go тут есть пример\n    - как делать discovery/mock?\n  - дописать ms/example:\n    - больше API (и смысла)\n    - больше тестов\n    - адаптер к ms/auth\n  - добавить вариант с legacy-кодом в old/ у одного из example?\n  - попробовать consul вместо env.sh\n    - большинство настроек имеют defaults\n    - при запуске отсутствующие настройки создаются с default-значениями\n    - как решить host-specific настройки?\n    - как избавляться от неиспользуемых настроек?\n      - с учётом того, что где-то может работать инстанс старой версии,\n        ещё использующий эти настройки? или он может в этот момент\n        перезапускаться…\n"
      } },
    textDocument = {
      uri = "file:///home/powerman/doc/Main/test.md",
      version = 6
    }
  }
}

It turns out both are the same! No matter they didn't look the same in Vim and here on Github - they're mostly the same, or, more precisely, they begins with same bytes but ends differently.
The problem is first one ends with incomplete Unicode byte sequence. Because of this it detected in different way by file tool (and by Vim):
$ file  /tmp/lsp-encoding.txt  /tmp/lsp-encoding-fixed.txt
/tmp/lsp-encoding.txt:       Non-ISO extended-ASCII text, with LF, NEL line terminators
/tmp/lsp-encoding-fixed.txt: Unicode text, UTF-8 text

The second "-fixed" file is same as first but with last (incomplete UTF-8) symbol removed from the file.
So, it looks like Neovim sends incomplete UTF-8 byte sequence just before closing ", which breaks parsing JSON in that library (I suppose it eats " byte as part of previous UTF-8 sequence and then fails to find closing ").

Resume. Sending incomplete UTF-8 sequence is a Neovim bug for sure. Still, LSP servers shouldn't crash on such an input.

:exploding_head:
That's quite the bug! Your commentary is exceptionally quality.
tower-lsp returns the Parse error right here as a result of serde_json not being able to parse. The library author chose to make the LS crash for this input.
I think from a UX perspective, it would be better if we log an error instead. I can modify the library.
NOTE: the project seems to be dead, and I have been using my own fork for a while

https://github.com/neovim/neovim/issues/29898

This is not an issue with oxide, so closing (cleaning)

Sometimes while editing large file with unicode text single unicode letters gets replaced with ��.
Any chance this happens because of same issue?

Unfortunately I don't think so; it is probably your font?

Font? No, it's not font. I think single Unicode letter's bytes was send to LSP split into two separate chunks, so LSP get incomplete Unicode at the end of first chunk and at the beginning of second chunk - and return both "bad" Unicode symbols back to Neovim as � (it's \uFFFD actually), replacing single Unicode letter with two � as result.

odd; are those question marks? They appear as question marks to me:

I don't have any ideas for this at the moment.

Actually it's a unicode symbol designed exactly for such use cases: https://www.fileformat.info/info/unicode/char/fffd/index.htm


[WIP] return InferenceData objects when sampling and forward sampling

I still need to update documentation and tests

one more commit and we can merge! I still have to fix a couple of details :-)

Oy, while playing around with examples, the other problem I saw was that this makes posterior predictive sampling a bit awkward, right? I would love to be able to do
trace = pm.sample(model())
ppc_trace = pm.sample_posterior_predictive(trace)

or even
trace = pm.sample(model())
pm.sample_posterior_predictive(trace, in_place=True)

but right now sample_posterior_predictive expects a dictionary of numpy arrays (with a certain shape!)


Better defaults for maxNumRetries

The motivation is to have more number of retries when there's just 1 uri and also limit the maximum number of retries in case there's a lot of uris.

We haven't talked yet
On Nov 9, 2017 12:34 PM, "Diogo Holanda"<EMAIL_ADDRESS>wrote:

@diogoholanda commented on this pull request.
In http-clients/src/main/java/com/palantir/remoting3/
clients/ClientConfigurations.java
https://github.com/palantir/http-remoting/pull/611#discussion_r150032872
:

@@ -42,6 +42,8 @@
private static final Duration DEFAULT_WRITE_TIMEOUT = Duration.ofMinutes(10);
private static final Duration DEFAULT_BACKOFF_SLOT_SIZE = Duration.ofMillis(500);
private static final boolean DEFAULT_ENABLE_GCM_CIPHERS = false;

private static final int DEFAULT_MAX_NUM_RETRIES_LOWER_LIMIT = 3;

@uschi2000 https://github.com/uschi2000 @jared2501
https://github.com/jared2501 any conclusions from your talks?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/palantir/http-remoting/pull/611#discussion_r150032872,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACu-Mo4gkdqUPCPd-__P5ygtyA5GJRruks5s0zeWgaJpZM4QWXB6
.

Rob's out of the office today. Likely will chat tonorrow.
On Nov 9, 2017 12:34 PM, "Jared Newman"<EMAIL_ADDRESS>wrote:

We haven't talked yet
On Nov 9, 2017 12:34 PM, "Diogo Holanda"<EMAIL_ADDRESS>wrote:

@diogoholanda commented on this pull request.
In http-clients/src/main/java/com/palantir/remoting3/clients/
ClientConfigurations.java
https://github.com/palantir/http-remoting/pull/611#discussion_r150032872
:

@@ -42,6 +42,8 @@
private static final Duration DEFAULT_WRITE_TIMEOUT = Duration.ofMinutes(10);
private static final Duration DEFAULT_BACKOFF_SLOT_SIZE = Duration.ofMillis(500);
private static final boolean DEFAULT_ENABLE_GCM_CIPHERS = false;

private static final int DEFAULT_MAX_NUM_RETRIES_LOWER_LIMIT = 3;

@uschi2000 https://github.com/uschi2000 @jared2501
https://github.com/jared2501 any conclusions from your talks?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/palantir/http-remoting/pull/611#discussion_r150032872,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACu-Mo4gkdqUPCPd-__P5ygtyA5GJRruks5s0zeWgaJpZM4QWXB6
.

I'd propose to punt on this for now: Main reason is that's it's not likely that increasing the number of retries without more advanced global throttling, timeouts, and circuit breaking is going to improve the performance of the overall system. We're going to implement this in a more first-class way in the Mesh and I think it's our best bet to leave the behavior unchanged until we have better data to base a decision on.

I think that's fine. I just wanted to comment here about the actual use case that motivated the change:
In general, what @jared2501 said about retrying the same node being likely to fail again and the retry being undesirable is true. However, this is not the case for QoS exceptions. The service is literally asking us to just wait a little bit and retry later, but if the number of URLs is 1, then our request will simply fail. Maybe we should special case the retries for QoS then?

I'd say we probably want to put a lot of this retry logic into the mesh
later rather than spending too much time trying to get it right in the http
remoting layer. The mesh layer is not far off! We already have it deployed
to our internal platform release stack, and have it comfortably sitting
between a couple services.
On Nov 14, 2017 3:07 AM, "Diogo Holanda"<EMAIL_ADDRESS>wrote:

I think that's fine. I just wanted to comment here about the actual use
case that motivated the change:
In general, what @jared2501 https://github.com/jared2501 said about
retrying the same node being likely to fail again and the retry being
undesirable is true. However, this is not the case for QoS exceptions. The
service is literally asking us to just wait a little bit and retry later,
but if the number of URLs is 1, then our request will simply fail. Maybe we
should special case the retries for QoS then?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/palantir/http-remoting/pull/611#issuecomment-344176857,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACu-MtX_eYzL1OnPoy5No-8Whjp5VzJtks5s2UozgaJpZM4QWXB6
.

I am fine with the decision here. Just wanted to expose the actual problem we had to make sure it will be addressed in the mesh solution.

Thanks for getting this resolved.


Strange behavior of the slider in homekit

homebridge.log.gz
homebridge-hue.json.gz
When you slide the slider up at a higher speed to about 70 percent, it returns to zero and then goes back to 70 percent.
To understand me correctly I am attaching this video.

Is it possible to correct it?

Is it possible to correct it?

No, it's a fundamental design flaw in HomeKit.  It doesn't distinguish between current and target brightness (and other light attributes such as hue, saturation, and colour temperature).
I think you're using deCONZ and an IKEA or OSRAM light (one that supports attribute reporting)?
You set the target brightness using the slider, causing Homebridge Hue to send a Zigbee command to change the brightness.  The light starts transitioning from it's current brightness (or off position) to the target brightness.  Depending on the transition time, and the attribute reporting settings, the API updates the reported brightness several times during this transition.  Homebridge Hue uses this to update the slider.

Yes this is an IKEA bulb.  I noticed that this behavior does not appear with  Chinese bulb and also with the Trådfri gateway for this I assumed that the problem could be corrected in deconz or homebridge-hue

If you disable attribute reporting for the IKEA bulb, or remove the binding from its Level Control cluster to the coordinator, you won't see this behaviour for the default transition time of 0.4 seconds.  However, I think the REST API plugin might restore these settings.

Is it possible while the slider is moving, Homebridge-Hue to ignore the returned state from api and after some time to apply the last returned state from api?

I noticed that this problem only appears when the light bulb is off, but if it is 1 percent and you slide the slider, the problem is gone.

Is it possible while the slider is moving

No, HomeKit doesn’t provide any info as to whether the slider is still moving.  It just reports the current position.  You could try and tweak waitTimeUpdate to make Homebridge Hue wait a little longer to see if HomeKit sends another value before sending the command to deCONZ.

Homebridge-Hue to ignore the returned state from api and after some time to apply the last returned state from api?

I don’t have the equivalent of waitTimeUpdate to set the HomeKit state.  I could ignore updates from the API for a brief period (0.5 seconds) after sending a command, but that probably won’t work out well, as you might need to wait until Homebridge Hue next polls the gateway before the actual state is reflected.
What version of deCONZ are you on?  How long have you been seeing this behaviour?  I suspect this is one of the unforeseen side-effects of sending the same commands as the Hue bridge, see https://github.com/dresden-elektronik/deconz-rest-plugin/issues/2475.  I think I’d better change that back, if only for non-Hue lights.

My deconz is with version .77 and the latest firmware of conbee 2.  I have been using deconz recently and the problem was with the .75 version

I experimented with the time of waitTimeUpdate there is no effect on the problem.

My deconz is with version .77 and the latest firmware of conbee 2. I have been using deconz recently and the problem was with the .75 version

Sorry is the problem with .75 and with .77, or only with .75?  In the first case: my recent changes in deCONZ handling the light state don't influence the issue.  I'm afraid there's nothing I can do without introducing other issues elsewhere.

The problem is present in .75 and .77  versions.
In the examples I have with esp8266 esp_homekit_demo I notice that the slider does not receive feedback on its status during the up or down drag.
I update its status only when I enter the home application or when I increase the brightness with a button that calls this function https://github.com/Kristian8606/simple_dimming_LED/blob/1518e95f16db6044c25a1697716d7eb5b9587327/main.c#L232

I would do a test. Would you direct me where in your code  to stop updatings the slider to test?

The Brightness characteristic is updated here:
https://github.com/ebaauw/homebridge-hue/blob/e9a8ec165983a8d61b5585f78674594124c899b4/lib/HueLight.js#L998
Updating the light when moving the slider is handled here:
https://github.com/ebaauw/homebridge-hue/blob/e9a8ec165983a8d61b5585f78674594124c899b4/lib/HueLight.js#L1555

for test I managed to ignore the slider feature updates, and it started to behave steadily and move smoothly.
The question is how do I ignore updates only when the slider is moving?

this.obj.state.bri is a global variable?

this.obj in an instance variable, of course.  It points to the light or group object from the full bridge state created during startup.  It's maintained only by the HueLight instance for the light or group.

I analyzed the behavior of Ikea bulbs and realized that when the bulb receives a brightness update, it immediately returns to its current brightness and after a few seconds returns to its already updated brightness, this behavior makes the slider in the home app jump.  I will try to give an example: if the slider is 20% and touch its upper end to 100% the bulb immediately returns its current brightness 20% which updates the slider to 20 and after a few seconds returns the already updated value 100%

I made a slight change to the code that eliminates this behavior.  I'm sure it's not the best solution but I hope you look at it and I'm also sure you'll find a good way to implement

The logic was already implemented in HueLight.js for ct and xy.  I just wasn't applied to bri.

Great

I tested but sometimes it doesn't work

What do you mean by “Doesn’t work”?  Please attach a debug log file.
The code just ignores any updates of the value by deCONZ for 500ms after sending the changed value to deCONZ.  It doesn’t count the number of updates.
Afaik no light is capable of sending multiple updates in 300 ms.

I'm sorry I was in a hurry and I couldn't explain what I meant.
I restarted my Raspberry pi because I have observed that after a long use (3-5 days) the reaction of sensors and bulbs slows down and after the restart they work quickly again.
After the restart I started testing the slider and everything seems to be fine but sometimes the function obviously fails to ignore the update and the slider jumps again.
As far as I remember the bulbs on the ikea, websocket send several status messages, 3-5 for a period of 300-350 milliseconds after touching the slider and another one after about 4-5 seconds.
example: first message 2ms after touching the slider, second - 8ms third 9ms fourth 112ms fifth last 4-5000ms.
I suspect that the function does not ignore the first 2 commands one updates the slider to 1% the next to 100%.
I hope for now this information will be useful to you, as soon as I have time I will try to recreate the problem and upload the logs.

Did you manage to re-create the issue while capturing a log?

I tested several times after restarting the radpberry pi.  I can confirm that the problem is still present, most often I reproduce it on a bulb TRADFRI bulbE27 WS opal 1000lm.
Unfortunately I still can't save log files

https://youtu.be/ySZlL_eTMjQ

I really need a debug log file to see what's happening.  Latest version of Homebridge Config UI X now allows downloading the log file from the View Logs screen, if you use hb-service.

homebridge.log.txt.gz
Sorry for the delay! Here is the file

What is the attribute reporting configuration for the Current Level attribute of the bulb?  It seems to take the light 5 seconds to report the (final) brightness.  Meanwhile, Homebridge Hue is polling the gateway, receiving the initial brightness (for which the attribute report was ignored alright).  With standard 1/300/1 settings, I would expect the light to report the final brightness after a second, not after 5 seconds.  You could try a setting of 0/300/1.  That might cause multiple reports, but they should appear in the 0.4 seconds of the transition.
I cannot do anything about this in Homebridge Hue.  As workaround, you might disable attribute reporting for the bulb (but the deCONZ REST API Plugin will probably re-configure it), or you might increase the heartrate to 30 (you can do this runtime in Eve for testing), so there's a smaller chance that the gateway is polled in between the reports.
Here you set the light to on and 87% brightness.
[2020-8-27 17:54:17] [Hue] TRADFRI bulb Bedroom: homekit brightness changed from 66% to 87%
[2020-8-27 17:54:17] [Hue] TRADFRI bulb Bedroom: homekit on changed from 0 to 1
[2020-8-27 17:54:17] [Hue] Phoscon-GW: gateway request 11: put /lights/6/state {"bri":221,"on":true}
[2020-8-27 17:54:17] [Hue] Phoscon-GW: gateway request 11: ok

deCONZ updates it's cache and echoes the values back to Homebridge Hue:
[2020-8-27 17:54:17] [Hue] TRADFRI bulb Bedroom: state changed event: {"alert":null,"bri":221,"colormode":"ct","ct":250,"on":true,"reachable":true}

There's no update needed, as HomeKit already reflects that brightness.
Next, deCONZ receives an attribute report from the bulb and issues a web socket notification.  Homebridge Hue rightfully ignores this change, due to the recent update from HomeKit:
[2020-8-27 17:54:17] [Hue] TRADFRI bulb Bedroom: state changed event: {"alert":null,"bri":2,"colormode":"ct","ct":250,"on":true,"reachable":true}
[2020-8-27 17:54:17] [Hue] TRADFRI bulb Bedroom: light bri changed from 221 to 2
[2020-8-27 17:54:17] [Hue] TRADFRI bulb Bedroom: recently updated - ignore changed bri

When next polling the gateway, Homebridge Hue notices the brightness differs from HomeKit and updates HomeKit:
[2020-8-27 17:54:22] [Hue] Phoscon-GW: gateway request 15: get /lights
[2020-8-27 17:54:22] [Hue] Phoscon-GW: gateway request 15: ok
[2020-8-27 17:54:22] [Hue] TRADFRI bulb Bedroom: light bri changed from 221 to 2
[2020-8-27 17:54:22] [Hue] TRADFRI bulb Bedroom: set homekit brightness from 87% to 1%

Only thereafter, the attribute from the light arrives, and Homebridge Hue updates the HomeKit brightness:
[2020-8-27 17:54:22] [Hue] TRADFRI bulb Bedroom: state changed event: {"alert":null,"bri":221,"colormode":"ct","ct":250,"on":true,"reachable":true}
[2020-8-27 17:54:22] [Hue] TRADFRI bulb Bedroom: light bri changed from 2 to 221
[2020-8-27 17:54:22] [Hue] TRADFRI bulb Bedroom: set homekit brightness from 1% to 87%

I hope IKEA for updates this bulb.
thanks for the help.