[question] isolation + threadlocal + test selection with miri

Hi - this is a question about miri, and not a bug. It's quite likely that I'm doing something wrong,
I'm using miri with concread due to the amount of unsafe + potential concurrency issues that can exist in a datastructure. So far I believe that miri is "passing" but I'm seeing a couple of odd behaviours.
First, I can't seem to run miri again "targeted" tests. For example:
cargo miri test -- -Zmiri-disable-isolation -- bptree2

This still runs "all tests" rather than the subset I want to target. What's the correct way to run miri against only a subset of my test suite?
I'm also seeing test failures such as:
---- collections::bptree::tests::test_bptree2_map_from_iter_1 stdout ----
thread 'main' panicked at 'assertion failed: is_empt', src/collections/bptree/node.rs:70:5

failures:
 ...
    collections::bptree::tests::test_bptree2_map_from_iter_1
   ...

Without miri, this test passes. The failure is occuring in a memory leak check that runs inside of my tests, but it relies on thread-locals / globals and I'm wondering if disable isolation is potentially causing the threads to stomp on each other somehow. There are no leaks in the full tree or stress tests, so I'm not sure it's a leak in my datastructure, but it's probably my fault in how I'm doing the leak check or using miri here.
A way to fix this could be if there is a cfg(miri) flag (similar to cfg test) then I could skip the custom leak check. Does such a flag exist?
Finally, I see my system memory grow to huge amounts - I exceeded 8GB of ram during one attempt and started to swap. Does miri not release it's data until the end of execution or something similar? Or could it be due to the size of the tests I'm attempting to run (I have some multithreaded stress tests still in here)?
Anyway, thanks for your time, and again, thanks for making such an excellent tool!

This still runs "all tests" rather than the subset I want to target. What's the correct way to run miri against only a subset of my test suite?

The command you gave works for me and we are even testing that on CI. So I think we need some more details here to figure out what is happening.

The failure is occuring in a memory leak check that runs inside of my tests, but it relies on thread-locals / globals and I'm wondering if disable isolation is potentially causing the threads to stomp on each other somehow.

Miri forces the number of threads to be 1, which you can replicate outside of Miri using cargo test -- --test-threads=1. Also see https://github.com/rust-lang/miri/issues/1360. Maybe that explains your problem here?

A way to fix this could be if there is a cfg(miri) flag (similar to cfg test) then I could skip the custom leak check. Does such a flag exist?

It does, Ctrl-F for "miri config flag" in our README. But of course it would be better t figure out what actually happens.

Finally, I see my system memory grow to huge amounts - I exceeded 8GB of ram during one attempt and started to swap. Does miri not release it's data until the end of execution or something similar? Or could it be due to the size of the tests I'm attempting to run (I have some multithreaded stress tests still in here)?

Miri releases program memory when the program frees it. However, when the program does a lot of work on the same memory, a lot of metadata for the aliasing tracking can accumulate. You could try to see if -Zmiri-disable-stacked-borrows helps, but of course that means aliasing violations will not be detected any more.
Also since you mention concurrency, you should have gotten a warning that concurrency support is still experimental. In particular, thread-local storage is not freed currently (https://github.com/rust-lang/miri/issues/1369). (We also keep some thread metadata around forever after the thread dies, but that's just ~100 bytes per thread so I doubt it will be a problem.)

This still runs "all tests" rather than the subset I want to target. What's the correct way to run miri against only a subset of my test suite?

The command you gave works for me and we are even testing that on CI. So I think we need some more details here to figure out what is happening.

As mentioned it was likely my fault, and as soon as I write reproduction instructions, it works  ...
I think what happens is when you run a single test it passes, then it immediately compiles and tests a second binary, so the output of the second binary shows 0 tests, and that's what confused me! Really sorry about that.

The failure is occuring in a memory leak check that runs inside of my tests, but it relies on thread-locals / globals and I'm wondering if disable isolation is potentially causing the threads to stomp on each other somehow.

Miri forces the number of threads to be 1, which you can replicate outside of Miri using cargo test -- --test-threads=1. Also see #1360. Maybe that explains your problem here?

It works with many threads and threads=1 outside of miri, even under stress, which is why I'm a bit confused and possibly assuming incorrectly what the cause could be.

It does, Ctrl-F for "config flag" in our README. But of course it would be better to figure out what actually happens.

Sure, but I think the other thing is that in my mind is that the "failure" right now happens in leak-detection, which miri will do for me. So while maybe it's an unknown, this does seem like the correct option to disable my own leak detector when running under miri. That also will remove the need to have thread-locals etc which as you point out, could also be an issue.
Another thought that comes to mind is I have been previously testing on macos and now I'm testing on Linux so I wonder if there is some platform specific issue that may be occuring.

Also since you mention concurrency, you should have gotten a warning that concurrency support is still experimental. In particular, thread-local storage is not freed currently (#1369). (We also keep some thread metadata around forever after the thread dies, but that's just ~100 bytes per thread so I doubt it will be a problem.)

This could be related, as mentioned my leak-tracking tracks every single allocated datastructure node id, and that set could grow quite large on some tests, and of course, miri would also check those leak tracking sets for correctness too. So I'm thinking the correct answer as before - disable my leak tracker while under miri as it's double handling the same problem.
Anyway, I've use the cfg there to disable miri during this test and indeed I can see a memory leak, so again - very likely a mistake in my program rather than miri :)
Thanks again!

Right, I understand the cause of this issue now.
So, during a normal test as I allocate and recreate each tracking set for the allocated nodes, it would only show leaks "from that test case". There were 5 tests in my iter.rs module where I was not assigning a tree root correctly, causing those tests to leak. But in those 5 tests I was not calling my "assert_released" function to ensure everything was cleaned. So on the "next test" the allocation set was replaced and those leaks were "missed". That means subsequent tests also did correctly report they were not leaking (because they weren't!)
However, when running under miri, I think this wasn't occuring. So once the leafiter tests had been executed, the allocation sets persisted, which caused subsequent tests to then fail - this is because the tracking set contained the leaked nodes from leafiter tests, but the later tests like from_iter were checking assert_released and were detecting the nodes leaked from previous tests.
So I have now fixed the leak in my iter tests, and 100% pass once again (I'll leave it to you to decide if this leak counts as a trophy, as the leak only affected tests, not the core datastructure, but miri was still invaluable in helping to find it and give me trust in my code!).
During the process I did notice one "quality of life improvement" though. I noticed with a subset of tests you get out put like this:
running 10 tests
test collections::bptree::iter::tests::test_bptree2_iter_leafiter_1 ... ok
test collections::bptree::iter::tests::test_bptree2_iter_leafiter_2 ... ok
test collections::bptree::iter::tests::test_bptree2_iter_leafiter_3 ... ok
test collections::bptree::iter::tests::test_bptree2_iter_leafiter_4 ... ok
test collections::bptree::iter::tests::test_bptree2_iter_leafiter_5 ... ok
test collections::bptree_legacy::iter::tests::test_bptree_iter_leafiter_1 ... ok
test collections::bptree_legacy::iter::tests::test_bptree_iter_leafiter_2 ... ok
test collections::bptree_legacy::iter::tests::test_bptree_iter_leafiter_3 ... ok
test collections::bptree_legacy::iter::tests::test_bptree_iter_leafiter_4 ... ok
test collections::bptree_legacy::iter::tests::test_bptree_iter_leafiter_5 ... ok

test result: ok. 10 passed; 0 failed; 0 ignored; 0 measured; 175 filtered out

The following memory was leaked:
alloc876074 (Rust heap, size: 120, align: 8) {
    0x00 │ 07 00 00 00 00 00 00 20 0a 00 00 00 00 00 00 00 │ ....... ........
    0x10 │ 0b 00 00 00 00 00 00 00 0c 00 00 00 00 00 00 00 │ ................
    0x20 │ 0d 00 00 00 00 00 00 00 0e 00 00 00 00 00 00 00 │ ................
    0x30 │ 0f 00 00 00 00 00 00 00 10 00 00 00 00 00 00 00 │ ................
    0x40 │ 0a 00 00 00 00 00 00 00 0b 00 00 00 00 00 00 00 │ ................
    0x50 │ 0c 00 00 00 00 00 00 00 0d 00 00 00 00 00 00 00 │ ................
    0x60 │ 0e 00 00 00 00 00 00 00 0f 00 00 00 00 00 00 00 │ ................
    0x70 │ 10 00 00 00 00 00 00 00                         │ ........
}
... <many more leaks below>

Now this is all well and good, we know we have leaks, but:

My allocations apparently are quite "random". The alloc would never be the same between two executions, which meant that -Zmiri-track-alloc-id=<id> was unable to help me find the leaking allocation and leaking test
You can't see which allocation relates to what test, so it's impossible to tell which test leaked what memory

It would be "interesting" to perhaps track at the least, the top level function or test case that each allocation belongs to, so that you can more easily isolated what test is leaking. Ie alloc<id> + test_id or similar.
Alternately, perhaps a "bisect" function where if you have a leak, the tests are re-run and reduced to the set of cases that do have the leak.
Anyway, once again, thanks for your time and patience with my questions, and I'm glad to have solved this. Thanks mate!

My allocations apparently are quite "random". The alloc would never be the same between two executions, which meant that -Zmiri-track-alloc-id= was unable to help me find the leaking allocation and leaking test

Miri is deterministic unless you access the operating system. We have a seed (-Zmiri-seed=0) that by default is 0, and you can change it to test different szenarios. Other than that, every execution should be the same with the same alloc ids and everything.

Alternately, perhaps a "bisect" function where if you have a leak, the tests are re-run and reduced to the set of cases that do have the leak.

that seems like something that the testing infrastructure could benefit from independently of miri.

So on the "next test" the allocation set was replaced and those leaks were "missed". That means subsequent tests also did correctly report they were not leaking (because they weren't!)
However, when running under miri, I think this wasn't occuring.

That should never happen. We do not deallocate #[thread_local] statics, but that is not observable by the program unless the program is UB. Other than that, if a program behaves differently under Miri and "for real" (and this is not caused by e.g. there being just one CPU core in Miri and thus everything running on the same thread), that's a critical bug.
Why would whatever code "resets" your allocation statistics not run in Miri? Or was that "resetting" relying on being in a new thread with clean TLS? But then outside Miri with just a single thread for the test harness you should have the same problem.

My allocations apparently are quite "random". The alloc would never be the same between two executions

That should only happen if your program actually does something differently -- Miri itself introduces no non-determinism, as @oli-obk said. But if e.g. the exact number or size of allocations depends on things returned by the OS and those differ between runs, then alloc IDs can start to diverge.

It would be "interesting" to perhaps track at the least, the top level function or test case that each allocation belongs to, so that you can more easily isolated what test is leaking. Ie alloc + test_id or similar.

Please open a separate issue for that. However, the test harness for Miri is just another program, and the top-level function is "main" of the test harness, so I have no good idea for how to achieve this. Miri doesn't even know it is running a test suite, it just runs the MIR that is generated by rustc --test.

So on the "next test" the allocation set was replaced and those leaks were "missed". That means subsequent tests also did correctly report they were not leaking (because they weren't!)
However, when running under miri, I think this wasn't occuring.

That should never happen. We do not deallocate #[thread_local] statics, but that is not observable by the program unless the program is UB. Other than that, if a program behaves differently under Miri and "for real" (and this is not caused by e.g. there being just one CPU core in Miri and thus everything running on the same thread), that's a critical bug.

While there is certainly unsafe in my library, I only use std collections and safe constructs in this tracker. The definition is:
thread_local!(static NODE_COUNTER: AtomicUsize = AtomicUsize::new(1));
thread_local!(static ALLOC_LIST: Mutex<BTreeSet<usize>> = Mutex::new(BTreeSet::new()));

At this point I'm questioning now that it's very likely something I am doing wrong though, rather than a fault in miri, so I will think about this more to see if I can identify a reproducer for you.

Why would whatever code "resets" your allocation statistics not run in Miri? Or was that "resetting" relying on being in a new thread with clean TLS? But then outside Miri with just a single thread for the test harness you should have the same problem.

My allocations apparently are quite "random". The alloc would never be the same between two executions

That should only happen if your program actually does something differently -- Miri itself introduces no non-determinism, as @oli-obk said. But if e.g. the exact number or size of allocations depends on things returned by the OS and those differ between runs, then alloc IDs can start to diverge.

I'm using Box, which are the allocations in question. So I suspect you are correct that this involves the OS allocator.

It would be "interesting" to perhaps track at the least, the top level function or test case that each allocation belongs to, so that you can more easily isolated what test is leaking. Ie alloc + test_id or similar.

Please open a separate issue for that. However, the test harness for Miri is just another program, and the top-level function is "main" of the test harness, so I have no good idea for how to achieve this. Miri doesn't even know it is running a test suite, it just runs the MIR that is generated by rustc --test.

That's true. Thought needed on how to achieve this. But I will open the issue.

I'm using Box, which are the allocations in question. So I suspect you are correct that this involves the OS allocator.

miri injects its own allocator. By "using the OS" ralf means reading files or accessing the file system. Hypothetically even using syscalls, but I think we have no problematic ones there.

Right, if e.g. the content of a file you are reading changes, that could lead to different program behavior and thus different allocation IDs. The allocator itself is fully deterministic in Miri.

I'm not reading any files, using any time sources or changing environment variables. I do have isolation disabled though because in a seperate test it does need clock monotonic. Perhaps this is related.

Even if you don't change them, disabling isolation causes Miri to read all env vars on startup. That can already introduce differences.

That probably explains that. Thanks for your time!

The thing we still haven't explained is why your leak checker ceased to work under Miri -- or rather, why its state wasn't reset the way we'd expect. Would still be interesting if you could minimize that into code that you can share.


UserWarning with num_iterations

A UserWarning consistently appears: UserWarning: Found 'n_estimators' in params. Will use it instead of argument.
According to the documentation, num_iterations is the parameter, with the following aliases: num_iteration, n_iter, num_tree, num_trees, num_round, num_rounds, nrounds, num_boost_round, n_estimators, max_iter.
When I change the parameter to num_iterations, this warning should not appear.
However, the warning now states: UserWarning: Found 'num_iterations' in params. Will use it instead of argument.
_log_warning(f"Found '{alias}' in params. Will use it instead of argument")
The bug can be in engine.py within the train function:
for alias in _ConfigAliases.get("num_iterations"):
    all_aliases = _ConfigAliases.get("num_iterations") ### I debugged this, and the set contains also num_iterations. I think without this string the warning would not occur.
    if alias in params:
        num_boost_round = params.pop(alias)
        _log_warning(f"Found `{alias}` in params. Will use it instead of argument")

With this modification the warning occurs only if an alias was used.
for alias in _ConfigAliases.get("num_iterations"):
    all_aliases = _ConfigAliases.get("num_iterations")
    if alias == "num_iterations":
        continue
    if alias in params:
        num_boost_round = params.pop(alias)
        _log_warning(f"Found `{alias}` in params. Will use it instead of argument")

My params:
lightgbm:
objective: "regression"
metric: "rmse"
boosting: "gbdt"
lambda_l1: 0.01
lambda_l2: 0.01
num_leaves: 16
max_depth: -1
num_iterations: 2000
learning_rate: 0.01
early_stopping_rounds: 50
verbosity: 1
LightGBM Version: 4.3.0

Did you set 'num_iterations' to 'params' and 'n_estimators' to function call parameters at the same time?
e.g.
param = {
    'num_iterations': 100
}

model = lgb.train(params, n_estimators=50)

No, i dont use num_iterations and aliases at the same time.

Thanks for using LightGBM.
This is already being discussed #6324, please subscribe to that issue for updates. You can also find more details on strategies for suppressing that warning at https://stackoverflow.com/a/78213401/3986677.
For the future, you might also find that these are helpful for working on GitHub:

how to format code with markdown: https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax
how to link to code: https://docs.github.com/en/repositories/working-with-files/using-files/getting-permanent-links-to-files


[Bug] Some series in MG or DX are not being displayed as series when using DICOMJSON

Describe the Bug
When using DX or MG modalities and having series. The series are displayed as individual instances instead of series. This didn't happen previously in OHIF v2.
Steps to Reproduce

Download the ZIP file or JSON.
View it using DICOMJSON on OHIF v3.

The current behavior
The series are dispalyed as separate images.
The expected behavior
The series should be displayed as a series.
OS
macOS 14.5
Node version
20
Browser
Arc 1.47.1 (50893)

the same  issue

When using DICOM JSON, you are responsible for splitting the data. Since you are providing the JSON file statically, you can edit the JSON as needed, and it will work.


Add Search Admin Role details to the Common admin table

User provided feedback regarding missing Search Admin role  details in the table - can you please review  if make sense to add this Admin role to the most common admin role table - thank you

@gemmadev Please indicate the document you are referring to and wish to give the feedback for its improvement. Thanks

hi there-  sorry I may have submitted incorrectly and also provided feedback to the page itself  . this is the page I am referring to https://learn.microsoft.com/en-us/microsoft-365/admin/add-users/about-admin-roles?view=o365-worldwide

@yogkumgit This is a duplicate of https://github.com/MicrosoftDocs/microsoft-365-docs/issues/12820. We have already addressed this via PR#https://github.com/MicrosoftDocs/microsoft-365-docs/pull/12849


Test and document AWS EFS home directory restore functionality

Backups are only useful if we know they work, and can actively restore them.
https://docs.aws.amazon.com/aws-backup/latest/devguide/restoring-efs.html has documentation on how to restore an EFS backup.
We should test to make sure we can, and then write the smallest amount of documentation possible to make sure we can do so in times of disaster. We should rely on linking out to the cloud provider documentation as much as possible, this doesn't have to be super detailed.
In the long run, we should test these once every 6 months or so. But that's out of scope for this ticket

How did I test?
I created another backup to reduce the amount of dataloss and then followed the restore instructions and they worked.


Configurable branding of dspfront

Describe the feature you'd like and what it will do
Update all site branding to be configurable and document how to do it.  This will be useful when other groups come along that would like to collect and submit standardized submissions across repositories.
Why is this feature important?
This allows us to manage multiple Data Submission Portals for different groups.

This has been implemented by leveraging internationalization techniques and Vuetify theming capabilities. Documented in README file.


verify the app with google

Google App Authorization

This task is in progress

see also the user cap so we don't have an issue in the future!!!

Privacy policy updated waiting on merge to main

@Gitongamiriam

make sure to reply to the email thread once you can see the new policy in "https://opencreds.net/privacy"

Yes, sure, I am on the lookout.

Responded to the Google email and resubmitted the updated Privacy policy on the Google console

Already responded to the email awaiting feedback .


Fishing, MLM and woodcutting overlays showing duplicate informations

Currently, fishing, woodcutting and motherlode mine plugin overlays track logs/fish/pay-dirt cut/fished/mined and also the actions/hr. These informations are now obsolete, because exactly same informations are now present in the XP tracker.

I just mentioned this on #569, but figured I ought to leave a comment here as well, just in case: the MLM info isn't (and shouldn't be) duplicated by the XP tracker, since the tracker is tracking "XP Drops(/hr)" while the MLM overlay is showing "paydirt(/hr)". These two things aren't the same, because there's an xp drop from washing the dirt as well as from mining it.

Fixed in #569

This isn't an "issue" much as it is personal preference. and it wasn't "fixed". This need to be reopened until a proper toggle is implemented  Or only show it while the sidebar is closed.

1st: It was enhancement
2nd: The merge made this issue "solved" because it was talking about removing obsolete informations that are no longer present
If the merge would be reverted, this issue can be reopened. If not, making new issue to display the informations while panel is closed would be correct follow-up. Or, I guess, you can open it also right now, and then we can track possible solutions, including the possible revert there.


chore: use dev-net chain config for local node

Close: https://github.com/FuelLabs/fuels-ts/issues/2303

Summary:

Updated the 2 chains config to use the same values from the dev-net.
Updated the base asset ID to be the same value as the dev-net.
Updated funding amounts within our tests, as the calculated fee value increased.

@nedsalk I wonder if merging this PR now will result in breaking https://github.com/FuelLabs/fuels-ts/pull/1356.

@Torres-ssf I don't think it will break because only the values of the configs changed, not the interface.


Direct float texture reading

WebGL class support methods for padded and unpadded textures
shaders: direct and deferred padding
syntactic sugar for uniform binding

Initial contribution. Hopefully dealing correctly with this github cooperative workflow. Learning as stepping through it, open to be advised on best practices or weblas conventions.

It's probably my fault for not understanding, but can you go into more detail on what this enables? I'm curious about the benefits you're seeing (or hoping to see). The last thing I want to do is waste all of the work you've put into this.

tl:dr The aim is to skip float encoding if possible and availble, saving those compute cycles.
This modifies your Tensor.transfer() function firstly, based on your webgl class "gl.hasFloat" support is available or not, if the float computation results must indeed be encoded into sets of four gl.UNSIGNED_BYTE in the output, otherwise an appropriate output texture is created instead (gl.FLOAT) and float results are output directly when read via native gl.readpixels(), or not so directly if the data shape requires padding given the texture constraints (columns as multiples of four), in which case, an alternate shader to keep individual values contiguous along the readPixels() buffer is used instead. Aside of the new method bindUniform(), this PR is all that is required to provide this functionality.
Bypassing float encode is a natural option for the unpacked format, encoding comes only as a necessity of the level of webgl support, not only or maybe as a convenience given the chosen data format ie. packed.
I gather that weblas aims to be used with smaller factor browsers, and the relevant extension(s)
OES_texture_float and WEBGL_color_buffer_float (more info on chrome) are already enabled in some browsers. Looking forward webgl 2.0 will also account for this.
My personal aim is to keep as much of any arbitrary computation an algorithm requires at the GPU level, reducing transfers, if internaly the GPU deals with floats natively, encoding comes only as a requirement for CPU reads which I'm avoiding from the begining, hence my intent on a format which is free from encoding imposed/encouraged constraints.
Feel free if you need more clarifications.

Are you available for Google Hangout or Skype some time? I want to understand this and feel like I need higher bandwidth to do so. Email me:  AT gmail

Good conversation. Going to go ahead close this.


Programmatically toggle auto-contrast for image layers.

🚀 Feature

Currently, it is not possible to reliably set auto-contrast on via a line of code. It would be nice to do something like
layer = viewer.add_image(image_data)
layer._keep_auto_contrast = True

However, this currently exhibits weird behavior as described in https://forum.image.sc/t/programmatically-set-auto-contrast-for-image-layer/71631
Motivation

Sometimes, I want to look at a series of images that are automatically loaded in the background by a script. I do not want the user to everytime click on the continuous button. It would be better if that button is already toggled on because of a line of code.
Pitch

layer = viewer.add_image(image_data)
Pseudo-code: the continuous button in the viewer is toggled on and the auto-contrast is adjusted when jumping around in the axes of the image.
Once the user decides to adjust the contrast limits, click on the once button or toggle off the continuous button, the contrast limits shouldnt change anymore when moving through the axes of the image.

Alternatives

Additional context
As proposed (https://forum.image.sc/t/programmatically-set-auto-contrast-for-image-layer/71631/5) additional synchronization between _keep_auto_contrast and the GUI needs to be added.

I think that this issue should also contain the proposition of making it part of public API.

@Czaki Do you need me to change something in the feature request, or should I just wait?

It looks ok. You could wait or make a PR.


Miri fix: Rename invalid_mut to without_provenance_mut

Which issue does this PR close?

Closes #.
Rationale for this change

Nightly renamed the function: https://github.com/rust-lang/rust/pull/117658

Introduced by https://github.com/apache/arrow-rs/pull/5387

What changes are included in this PR?

Are there any user-facing changes?

cc @jhorstmann

Thanks, looks good. That's the downside of using nightly features, although renaming happens very seldomly.


Cannot use import statement outside a module & DiceManager is not defined

Hi, after cloning and installing the project, the examples don't work for me. I get the errors
Uncaught SyntaxError: Cannot use import statement outside a module and Uncaught ReferenceError: DiceManager is not defined.
I am not sure if I'm missing some transpiling here or anything, it would be much appreciated if you could help me out.
Best Regards!

Hey,
Did you figure this out? I'm facing the same problem having just started to try using this code.
Thanks!

Hey,
Did you figure this out? I'm facing the same problem having just started to try using this code.
Thanks!

The examples only work if you remove the import statements and remove the "export" key word from the library file

The examples only work if you remove the import statements and remove the "export" key word from the library file


filenamePatterns auto complete fills in foo*bar

Actually I wonder why there is "filenamePatterns" suggested as it seems I can use glob patterns for "filename" too, so maybe a leftover?

@bpasero Unsure what you mean? filename is equally check and filenamePattern is the glob-style check. The bar*foo.txt is the default value I've put in the schema

@jrieken when I tried it, only "filename" worked for me with a glob pattern. I filed the bug originally because I think we cannot have "foo" and "bar" as words in visible strings, but @SofianHn would know.

@bpasero, I think you meant someone else?

@jrieken I tried using the filenamePattern to have the editor recognize the dockerfile.dev and similar files, but it seems like it doesn't work; it's not recognizing the files.
Here's the example I have in my package.json file:
 "id": "dockerfile",               "filenamePatterns": [                   "dockerfile.*",                   "dockerfile*.*"               ]

@jrieken is this for workspace settings? Because I do not get 'languages' as a suggestion

@isidorn when adding a language as extension


Potential blocking risks of the solution 当前的风险

我是该项目的早期贡献者，事实上，对于中国服务器的支持也是由我共同实现的 627e7e0，在13.5版本更新之后，我向社区指出了为什么在死亡时，换肤会失效，因为拳头在玩家重生后，会重置 character_data_stack。这似乎是13.5新增的。
I was an early contributor to the project, and in fact co-implemented the support for the Chinese servers(see commit 627e7e0). After the 13.5 update, I pointed out to the community that why the software was disabled on reborn, because riot will resets the character_data_stack
目前我看到社区的结局方案都是类似的，他们检测玩家的状态，并在重生时，调用函数，这么做首先会导致性能问题，和奇怪的BUG，你会在重生时候有卡顿，其次更严重的问题是，这将导致你频繁修改内存中的位置，不断覆盖和重写，这会导致一系列风险，导致客户端崩溃（已经看到有人反馈在游戏结束时候客户端崩溃），极大的增加了封号的风险，尤其是中国服务器。
The solutions I've seen from the community so far are similar in that they detect the state of the player and, on reborn, call the change skin function. Doing this will firstly cause performance issues, you will have small lag during reborn, and secondly a more serious issue is that this will cause you to frequently modify locations in memory, constantly overwriting and rewriting, which will lead to a series of risks that will cause the client to crash(already seen the problems in the issues when game end client will crash) and greatly increase the risk of blocking.
这是因为，拳头会在每次游戏客户端崩溃的时候，或者发生性能问题的时候，Dump整个游戏内存并且上报到服务器，这会极大的增加你的封号风险，这也是我跟作者早期在Discord的讨论，其实拳头并不是每次都会重置内存，我已经做了很多测试，其实这里存在一个一个简单的解决方案，只需要修改一行代码，就能完美运行，也不需要在局内频繁读写，但是还需要更多的测试。
This is because the riot will be in each game client crash, or when performance problems occur, Dump the entire game memory and reported to the server, which will greatly increase your risk of ban, this is also the author and I early discussion in Discord, in fact, the riot does not always reset the memory, I have done a lot of testing, in fact, here exists a simple solution, only need to modify one line of code, you can run perfectly, and do not need to read and write frequently in  gameing, but there also needs more testing.

我在实现中国服务器支持的时候跟作者的交流
My discussion with the author

感谢！

thank you so much!

Indeed, when debugging the code, my game crashed dozens of times, I modified the code to only update the player's own skin, and minimized the interaction between tools and game memory.
waiting for your better solution!

Thanks!

Let's say hypothetically I decide to implement @cjd0901 solution, would that risk my account being banned, or is it just a normal fix that could be better?

thanks

Let's say hypothetically I decide to implement @cjd0901 solution, would that risk my account being banned, or is it just a normal fix that could be better?

Greatly increase the risk of being ban

Let's say hypothetically I decide to implement @cjd0901 solution, would that risk my account being banned, or is it just a normal fix that could be better?

Greatly increase the risk of being ban

So @hotline1337 just uploaded an update, and is not that much different from the fix @cjd0901 had. Even that would greatly increase the chance of a ban?

thanks

As I mentioned before, more and more side effects have started to be exposed recently, as the extra state detection, and frequent function calls, can make a mess of your memory, and the detection functions can cause a lot of errors and crashes. Currently getting community feedback, FPS lag issues, game client crash issues, playback system crash issues, skin disappearance issues. And so on.
These are all caused by the side effects of extra detection and frequent function calls, and prove that the current solution is immature, I am sure that frequent crashes will lead to your account being banned, so we need to work together to find a better solution.
正如我之前提到的那样，最近越来越多的副作用开始暴露，因为额外的状态检测，和频繁调用函数，会让你的内存变得一团糟，检测函数会导致大量的错误和崩溃。目前得到的社区反馈，FPS卡顿问题，游戏客户端崩溃问题，回放系统崩溃问题，皮肤消失问题。等等。
这些都是由于额外的检测和频繁的调用函数的副作用所导致的，也证明了当前的解决方案是不成熟的，我相信，频繁的崩溃一定会导致您的帐号被封禁，所以，我们需要努力一起找到更优的解决方案。

Understandable, in my experience dealing with these type of "changes" it's going to be hard to find the "perfect" solution, before like patch 13.6 comes out, but that is my experience, this could be different. I hope that this does not mean the end for R3nz skins as it is a very nice tool to be experiment on, for myself at least, also for the community.

waiting for your better solution

Appreciate your selfless spirit.
Salute!

感谢您！中国有很多lol玩家受益于你们的工作，希望r3nzskin可以继续下去。

Thanks
Salute! 🙏

注意，中国服务器仍然会在很长一段时间内使用X86架构，我也有能力在64架构到来之后继续维护更新。这里没什么可担心的。

我怎么觉得，干脆hook掉他内存上报更简单一点。。。崩溃大不了重启嘛，他上报了对这局游戏也无关紧要。而且有点纳闷没注意过崩溃后出现异常流量啊，还是说他不是立即上传的。全部进程内存上传肯定不现实，只上传堆栈的话，也许可以hook皮肤加载函数换肤来避免被检测？


Getting json files 404, seem not put into release

This module will try to load modules/tokencover/json/dnd5e/half_token.json, but tokencover/json does not exist.
I downloaded json files from GitHub, and it works.

Fixed in v0.9.2. Thanks!


Latest Changes

Pre wag mo akuin lahat, group tayo and malaki din yung part ng bawat isa, may issue din saken di ko na sunod ung plano and di ko mas naayos yung group naten and actually need rin naten maka adopt sa reality so wag mo dibdibin yung nangyare if hanggang ngayon iniisip mo parin

If tutuloy parin naten, want ko rin kasi tapusin yung program, hindi pa kasi talaga tapos kaya nangyare yan, here lapag ko yung na notes ko that time.

Current Issue:
Front End

UI/UX Revamp, I'll use the default color palette
Built in Hover Issue
Resolution is better in Fullscreen
Icons are bigger than text
Text cannot be seen
Inconsistent resolution
Too many font types
Too much color
Too much space

Back End

System is actually slow, might be a leaks somewhere (ref: 'the Mouse issue is not really an issue, rip mouse forever be missed')
Problem with default table width (ref: POS)
Editable tables (ref: Invoice Table)
Find date (ref: Income Monitoring) does not locate any date
Problem with some calculations raw value instead of mark up or just replace the Mark Up: into Raw Price:(ref: Product Information)
Remove the expiration date of the product (ref: Product Information)
Problem with saving (ref: Product Information)
No functions yet (ref: Most selling product)
Stocks should be the raw materials instead of the product itself (ref: Stocks)
Unnecessary decimal (Quantity)
Placeholder does not update when it is active (ref: Search bar)
Customer profiling on Guest (ref: Shop Check Out)
No validation for deleting, making changes and exiting
The product quantity can go pass the stocks left and still went through check out (ref: Shop)
No Validation for no input payment (ref: Shop Check Out)
Problem with pay buttons (ref: Customer Invoice)
No option for the profit of selected day (ref: Income Monitoring)
Current session data (ref: quantity) does not update
Use more buttons rather than event click
Too much functions (we need to remove some)
Did not met the scopes of the program

To be added

[ ] QR Code generator and Scanner
[ ] Importing and Exporting data
[ ] Receipt
[ ] JCalendar

Thank you for reassurance but okayy naman na sakin yung defense natin bro. Na shock lang ako kahapon and im tired already + lack of sleep kaya hinde ako energetic mag defense. srry if u see my emotionally fragile side ko kahapon. Im also planning adding more features and developing fixing the current bugs and errors sa project bro but there will be limitations.


Add headers support to Flow method

And also fix a bug with the function return type when no response types are specified

it seems like the failed job is unrelated to this PR


Implement commands to add and delete attendees from an event

[x] AddProfilesToEventCommand
[x] DeleteProfilesFromEventCommand
[x] add tests for commands
[ ] add tests for command parsers
[ ] add tests for attendees

Known issues:

attendees list stops refreshing immediately after the first change made to it fixed by replacing event with a new copy after every change
attendees list does not listen to changes (name edits and deletion) to a profile

Codecov Report
Base: 72.21% // Head: 71.19% // Decreases project coverage by -1.01% :warning:

Coverage data is based on head (a65d443) compared to base (8adc6de).
Patch coverage: 63.78% of modified lines in pull request are covered.

Additional details and impacted files
@@             Coverage Diff              @@
##             master     #118      +/-   ##
============================================
- Coverage     72.21%   71.19%   -1.02%
- Complexity      685      731      +46
============================================
  Files           100      105       +5
  Lines          2120     2354     +234
  Branches        276      302      +26
============================================
+ Hits           1531     1676     +145
- Misses          514      593      +79
- Partials         75       85      +10

Impacted Files
Coverage Δ

...in/java/seedu/address/commons/core/LogsCenter.java
78.37% <ø> (ø)

...main/java/seedu/address/commons/core/Messages.java
0.00% <ø> (ø)

.../address/logic/commands/event/AddEventCommand.java
76.47% <ø> (ø)

...dress/logic/commands/event/DeleteEventCommand.java
100.00% <ø> (ø)

...edu/address/logic/commands/event/EventCommand.java
100.00% <ø> (ø)

...in/java/seedu/address/logic/parser/ParserUtil.java
81.35% <0.00%> (-9.22%)
:arrow_down:

.../parser/event/AddProfilesToEventCommandParser.java
0.00% <0.00%> (ø)

...er/event/DeleteProfilesFromEventCommandParser.java
0.00% <0.00%> (ø)

...address/logic/parser/event/EventCommandParser.java
47.05% <0.00%> (-6.28%)
:arrow_down:

src/main/java/seedu/address/model/Model.java
100.00% <ø> (ø)

... and 20 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


use safe-buffer

This way we don't explode Node<4.5.0

needs patch release

needs patch release

$ npm owner ls hash-base
cwmma<EMAIL_ADDRESS>dcousens<EMAIL_ADDRESS>fanatid<EMAIL_ADDRESS>
:)

Published as v3.0.4


⚠ Fortnite has degraded performance

In e28efc6, Fortnite (https://fortnite.com) experienced degraded performance:

HTTP code: 200
Response time: 1519 ms

Resolved: Fortnite performance has improved in bff70b2.


Fix stack overflow

This replaces a spread call validDocs.push(...docs) where validDocs was empty with simple assignment validDocs = docs to avoid a stack overflow with a big docs array.
RangeError: Maximum call stack size exceeded

Spread calls use stack space, so foo.push(...REALLY_BIG_ARRAY) uses LOTS of stack space and it can easily cause a stack overflow.

How is this going?

This fix makes a lot of sense as there's no need to push to an empty array when you can just reassign it. Just thinking on optimizations it makes a lot of sense. How is it going @tex0l

This fix makes a lot of sense as there's no need to push to an empty array when you can just reassign it. Just thinking on optimizations it makes a lot of sense. How is it going @tex0l @arantes555


Code suggestion duplicates last character

Whenever I use the code action to replace an error with the GHC suggestion, I get the last letter of the suggestion repeated at the end. You cans see a short video with the error here:

I'm using Nvim 0.3 and GHC 8.2.2, if that helps clarify anything

Thanks! it is working great now


Adding Draco II upgrade guide

/staging

should it also be linked from the other upgrade guide ?

Sure, added some links with the pervious upgrade guide  and also the fullnode guide.

/staging


update dependencies

Problem solved
Short description of the bug fixed or feature added

PR-Codex overview
This PR primarily updates the version numbers of various dependencies across multiple package.json files and modifies the export syntax for types in several TypeScript files.
Detailed summary

Updated @cloudflare/workers-types from 4.20241011.0 to 4.20241022.0.
Updated @biomejs/biome from 1.9.3 to 1.9.4.
Updated turbo from 2.1.3 to 2.2.3.
Updated @tanstack/react-query from 5.59.13 to 5.59.16 in multiple packages.
Updated lucide-react from 0.452.0 to 0.453.0 in multiple packages.
Updated tailwindcss from 3.4.13 to 3.4.14 in multiple packages.
Export syntax changed from export { type ... } to export type { ... } for various types in files under packages/thirdweb/src/exports/.

The following files were skipped due to too many changes: pnpm-lock.yaml

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

#5150  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @jnsdls and the rest of your teammates on  Graphite


Bug: Incorrectly reports android deeplinks as invalid for app.links urls

Resolves #79
Bug: Incorrectly reports android deeplinks as invalid if contains anything other than application/json in the header
Fix: services such as app.link return content-type headers: application/json; charset=utf-8 and work fine with ios and android. Android yURL results currently wrongly state this is invalid.
Switched: contentType[0] != "application/json" to  isJSONMimeType := strings.Contains(contentType[0], "application/json")

Closing due to lack of response in issue #79.
cc: @aceslick911


Adding zone overwrites existing one

What happened?
I'm experiencing with the integration and not using in "production" yet. I have added and deleted multiple zones. After some time I could not add new zone becaue when I added a new zone it overwrote an existing one.
Finally I have opened the storage file and I saw the ID of my zones was 3, 4, 5, etc.. I changed them to 0, 1, 2 and restarted Home assistant. After that I was able to add new zone.
How to reproduce
See above.
Relevant log output
No response
Which version are you running?
v2024.4.3
Diagnostics file

[X] I have attached a diagnostics file

Additional information
No response

was this a fresh install or did you already have a previous version installed?

Solved. Found the storage file, added zones in it and they appeared in my instance.

closing - there is no diag file.


Capture image with Aspect Ratio 4:3

Hi,
I want to take a photo with aspect ratio 4:3. Currently, some device it capture with 4:3 and some with 16:9, so i have problem while scale image which were captured.
Sorry for my bad English. Hope i can get some advice soon.

Can you post a screenshot? And what's the picture size passed to the TakePictureRequest ?

Hi there,
Awesome library, found an issue in the TakePictureRequest class
private Camera.Size findClosetPictureSize(List<Camera.Size> sizeList, int width, int height) {         if (sizeList == null || sizeList.size() <= 0) {             return null;         }         int minDiff = Integer.MAX_VALUE;         Camera.Size bestSize = null;         for (Camera.Size size : sizeList) {             int diff = Math.abs(size.width - width) + Math.abs(size.height);             if (diff < minDiff) {                 minDiff = diff;                 bestSize = size;             }         }         return bestSize;     }
height never been used, so I believe it should be:
int diff = Math.abs(size.width - width) + Math.abs(size.height - height);
Regards, d

@nonameden you are right, I will fix this

Thank you,
Do you have in plans add support for Camera2?

Thank you so much.

@nonameden Not yet, but may be I will release a new called RxCamera2 to support the camera2 API

Is this push to com.ragnarok.rxcamera:lib:0.0.5?
It's in master but not 0.05.


Generated from 8f0794106d76d899fe6e152218380831e3340727

Ran prettier check on all edited files

Can one of the admins verify this patch?


tests: move {Disk,}CustomHostPath into storage package

Only storage tests need these paths, so they should not be exposed elsewhere.
/sig code-quality
NONE

/approve

/lgtm


Fix #959 - Add Webjar support

Here's my implementation for the Webjar support  (#959) - let me know what you think!

Codecov Report

Merging #960 into master will decrease coverage by -0.04%.
The diff coverage is 95%.

@@            Coverage Diff             @@
##           master     #960      +/-   ##
==========================================
- Coverage   69.01%   68.98%   -0.04%
==========================================
  Files         243      244       +1
  Lines        5461     5481      +20
  Branches      596      596
==========================================
+ Hits         3769     3781      +12
- Misses       1692     1700       +8

Impacted Files
Coverage Δ

...cala/org/http4s/server/staticcontent/package.scala
100% <100%> (ø)
:white_check_mark:

...rg/http4s/server/staticcontent/WebjarService.scala
94.73% <94.73%> (ø)

.../scala/org/http4s/testing/ArbitraryInstances.scala
53.33% <ø> (-26.67%)
:x:

core/src/main/scala/org/http4s/HttpVersion.scala
87.5% <ø> (-6.25%)
:x:

core/src/main/scala/org/http4s/Uri.scala
83.63% <ø> (-3.64%)
:x:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 32ef7b9...55bdd44. Read the comment docs.

You're right, there's not really a difference. I started renaming from File -> Asset but wasn't completely consistent.

@bryce-anderson I made changes according to you suggestions, thanks!
@rossabaker I believe I found what you meant with the slash logic - a path like prefixwebjar/v1.0/file.js was actually served. I added a test and fixed it. I also reordered some args in the methods to make the code a bit more readable.

Removing the pathPrefix from Config as @reactormonk suggested means that I can get rid of some code, which is cool.
Just for my understanding - what is it used for in the other services in staticcontent, if mountService provides this feature already?

... I didn't realize the other had pathPrefix too. Not sure, might be worth refactoring?

mountService, Router, and URITranslation (which should really be UriTranslation) are all means to relocate a service underneath another.  Multiple mountService calls is natural in servlet containers, but Blaze assembles them into one service.
I think this should be consistent with the other two.  I guess the question is a breaking change vs. something we don't strictly need.  I could argue this either way.  Does anybody feel strongly?

All of Router, URITranslation and mountService are Service => ..., whereas this one is Service, so for less-ways-to-do-a-thing, I'd add a deprecation notice with a link to the other ones.


Feature Requests: Flags

Following flags could be convenient:

quiet output (e.g. for use w/ cron), e.g. -q

add list-file to the system's installed packages for building (e.g. synth upgrade-repository -l /usr/local/etc/synth/portlist.conf to upgrade repository with host packages and the ones supplied in the file).
Maybe additionally to a config option for such a file

choose specific profile for this build, e.g. -p libressl

i can tell you right now I'm not interested in putting flags in the command line.
I don't want it to look like a git man page.
It's got a simple interface intentionally.

cron: isn't it possible to write a shell script that launches synth and sends stdout to /dev/null ?

the profile suggestion is interesting but not sure how to implement it cleanly atm.

yes, for sure.
There are quite a few programs that have a -qflag so I thought it might be useful for some / something to think about.
It's just some ideas I wanted to suggest.

maybe profile could be sent with environmental variable.  It's something only an advanced user would need

--quiet would be a pain to implement.

I think I can close this:
quiet: no, just redirect stdout to /dev/null, even in cron.  (profile would not use curses)
list-file: I don't urderstand the point of this at all.  You would just use a master list of files; I see no reason to "append" what's on the host system.  Seems to add complexity for its own sake.
selectable profile: Will be available in next release (soon)

the idea of the list file was to make something like
synth just-build /listfile && synth prepare-system
into
synth prepare-system -l /listfile
So it would rebuild the system's ports and the list's port with just one command.
It's probably confusing and unnecessary.

that makes no sense, those are two opposing commands
"synth prepare-system" has a ports list, it's generated from the system.
Thus "just-build" and "prepare-system" are not compatible.
prepare-system and system-upgrade are just specialty cases of "synth build", "synth install"

thus "prepare-system" and a build list are also incompatible

Yes but I assumed that prepare-systemwould check for the system's ports, rebuild them if necessary, and then rebuild the repository INCLUDING the ports built from the list file.
And I thought that it had worked that way when I had used rebuild-repositorybefore that command was repurposed (to add the ports I had built for the jails to the repository).

it's a specialty case.  If you want host system + more, then you create a ports file that is host system + more.  pkg can produce the initial list (e.g. "pkg prime-list" should do it) and the user would just augment that.
People are also asking for "exclude lists" too, e..g exclude from host system.  The answer is the same, make a master list and forget prepare-system, upgrade-system commands.

You're probably right. I was trying to avoid listing the host system's ports to stay flexible.
It worked the way I did it though.
otherwise I might use
> synth just-build /list && synth rebuild-repository

And update the host separately… (synth prepare-system)

yes, that combination will do what you want.

no, i mean this one:
synth just-build /list && synth prepare-system

that's what you want.

thanks


Add To() test base class

Added the base class for the To() tests.
Added the simplest tests that use the base class.

Part of fix #69


Embeddings for protein structures

Hello @joshim5, @tomsercu , @sbonner0 😇 ! I am new to this repository. I was looking for something that can be used to get embeddings for structures of proteins and was suggested to use this repository. Would you please let me know if the models mentioned and available in the repository can be used to get embeddings for protein structures ??

Yes you can look at the README page for instructions. Most models are sequence-first, but you could use ESM-IF1 for a kind of structure embeddings.
Feel free to use this place for discussions if you get stuck: https://github.com/facebookresearch/esm/discussions


Nail down by-value vs. by-reference semantics

{
  let obj1 = { field: 1 };
  let obj2 = obj1;
  obj2.field = 2;
  // what is obj1.field's value?
}

Also, how does this work with closures? And what about objects, especially since those desugar to having their methods as fields with function values?
class Person
{
  constructor(name)
  {
    this.name = name;
  }

  printName()
  {
    print(this.name);
  }
}

let bill = Person("Bill");
let jane = Person("Jane");
bill.printName = jane.printName;
bill.printName(); // what does this print?

See discussion in http://craftinginterpreters.com/classes.html#methods-on-classes (look for "bound methods")


terra-time-input - the return of the terra-time-input.

There was a prop-type error in react-text-mask (https://github.com/cerner/terra-core/issues/340) causing us to have to pull out terra-time-input. The error is now fixed in react-text-mask 5.0.2. This pull request is to pull in react-text-mask 5.0.2 and merge terra-time-input back in terra-core.
Resolves https://github.com/cerner/terra-core/issues/424
Thanks for contributing to Terra.
@cerner/terra
Deployment Link: https://terra-core-deployed-pr-365.herokuapp.com/

Our other issue with text-mask was prop-types comes pre-bundled through webpack.  https://github.com/text-mask/text-mask/issues/529
Since prop-types comes pre-bundled with text-mask, it will be duplicated in our bundle. However, the duplication should be only a couple KBs.
http://stackoverflow.com/questions/43981041/webpack-dedupe-webpack-bundle
To fix the duplication, we would need to go contribute to text-mask.

This is the same review that we approved earlier. This is probably good to get functionally and UX reviewed.

@mmalaker @neilpfeiffer This PR is ready for functional and UX Review. :)

@mmalaker @neilpfeiffer. terra-time-input has been deployed to http://engineering.cerner.com/terra-core/#/site/time-input. You may so functional and UX review.

Functional verification completed

Notes from 5/24/17 Terra-core meeting:

Change the hours & minutes nodes to independent inputs so they are easier to interact with
Use default placeholder attribute for "hh" and "mm" text-mask
For 24 time, if user enters 3-9 for hours: auto-rewrite to two digits ("03") on deblur, if user enters 6-9 for minutes: auto-rewrite to ("07") on deblur
Listen for arrow-left and arrow-right to move between the independent hour and time inputs

@benbcai  With #414 merged down to master, we'll want to delete the lib directory in packages/terra-time-input.

These updates look good to me @benbcai. Let's run this through another round of functional verification and UX verification since we changed the time input a bit based on @neilpfeiffer's original UX verification. cc/ @mmalaker

Keyboard navigation issues found during functional verification:

Chrome and Firefox both work as expected.
Desktop Safari, IE, and Edge do not allow navigation between the hour and minute time fields with backspace/delete or the left and right arrow keys.
Mobile Safari on iPad does not allow navigation with delete key. Also, you have to switch to the numeric keyboard manually, both when initially entering values and when switching between the hour field and the minute field.
Android on the MC40 allows entry of time values but also does not allow navigation between fields with backspace.
Most surprising, the latest Android version on Galaxy S6 does not allow time entry into the field at all, either through touch or on-screen keyboard actions.

@mmalaker The issue in Safari/IE/Mobile where using backspace/delete does not navigate from the minute input back to the hour input should now be fixed. Mobile device should now show the numeric keyboard. I didn't test on an Android on the MC40 because I don't have one. I tested on a Galaxy S6 with Andriod 7.0 and I was able to enter values in the hour and minute inputs before the code change. Is your S6 also on Andriod 7.0?

Functional verification completed. All time examples and test pages display and function correctly. Keyboard access issues fixed. Note that there is a minor, cosmetic, character alignment issue present in IE10 and 11. That issue will be fixed in the next forms PR.

Tested: PR #365
Reviewed:

Follows pattern considerations documented in the current Cerner Design Standard:
Date/Time Format and  Time Input
Follows UX defined Terra initial design implementation for Time Input.

Pass

UX Review

Suggestion (for Future Enhancement)

Have text entry overwrite current populated values when cursor is in the first or second position (before the “tens” or “ones” position) inside of the input element, so user doesn’t have to forcibly delete or remove any existing values.

🎉 🎈


Uncaught SyntaxError: Unexpected token /

[Enter steps to reproduce below:]

...
...

Atom Version: 1.0.19
System: Linux Mint
Thrown From: Remote-FTP package, v0.7.8
Stack Trace
Uncaught SyntaxError: Unexpected token /
At file:///opt/atom/resources/app.asar/static/index.html#%7B%22locationsToOpen%22%3A%5B%7B%22pathToOpen%22%3A%22%2Fhome%2Fmrx%2FDownloads%22%2C%22initialLine%22%3Anull%2C%22initialColumn%22%3Anull%7D%5D%2C%22bootstrapScript%22%3A%22%2Fopt%2Fatom%2Fresources%2Fapp.asar%2Fsrc%2Fwindow-bootstrap.js%22%2C%22resourcePath%22%3A%22%2Fopt%2Fatom%2Fresources%2Fapp.asar%22%2C%22devMode%22%3Afalse%2C%22safeMode%22%3Afalse%2C%22appVersion%22%3A%221.0.19%22%2C%22shellLoadTime%22%3A1113%2C%22initialPaths%22%3A%5B%22%2Fhome%2Fmrx%2FDownloads%22%5D%7D:3

SyntaxError: Unexpected token /
  at Object.parse (native)
  at /home/mrx/.atom/packages/Remote-FTP/lib/client.js:78:17
  at FSReqWrap.readFileAfterClose [as oncomplete] (fs.js:380:3)

Commands
     -1:52.6.0 remote-ftp:toggle (div.settings-view.pane-item)
     -0:56.4.0 core:select-right (atom-text-editor.editor.is-focused)
  2x -0:55.9.0 core:paste (atom-text-editor.editor.is-focused)
  2x -0:43.3.0 core:backspace (atom-text-editor.editor.is-focused)
  2x -0:37.8.0 core:save (atom-text-editor.editor.is-focused)
     -0:10.3.0 core:backspace (atom-text-editor.editor.is-focused)
     -0:09.6.0 core:save (atom-text-editor.editor.is-focused)

Config
{
  "core": {}
}

Installed Packages
# User
Remote-FTP, v0.7.8

# Dev
No dev packages

running on mint, 3.16.0-38-generic #52~14.04.1-Ubuntu

I have exacte the same error on Windows 7 64-bit

Same error happens for me

I can confirm the existence of this bug in Windows 10 64 Bit, and in Ubuntu 15.04 64 Bit too.

in Osx yosemite 10.10.4

There is an error parsing your .ftpconfig file...
Since the latest release we are pre-validating the JSON, so I don't understand why this error happens.
Can anybody of you post his .ftpconfig (without user&pw of course) please?

It's ok i'm sorry -_- , but I do not understand why we need to open a project for creates a file .ftpconfig, the button "create sftp file" not working if is no open project.
Thanks

@snettah Remote-FTP is project based and creates a local copy of all files you edit on the remote server in your project directory on your disk (if you haven't already). Hence, a project directory is required.
But this is off-topic.
When the error Uncaught SyntaxError: Unexpected token / still occurs, post your .ftpconfig file (without credentials) here please.

I don't have access to my pc but I've used the package's built in function to create the config file.

I was having trouble with "Uncaught SyntaxError: Unexpected token /". I have removed all comments and now the FTP config example work like a charm!

conf file copied from here > https://atom.io/packages/remote-ftp
{     "protocol": "ftp",     "host": "<IP_ADDRESS>", // string - The hostname or IP address of the FTP server. Default: 'localhost'     "port": 21, // integer - The port of the FTP server. Default: 21     "user": "user", // string - Username for authentication. Default: 'anonymous'     "pass": "pass", // string - Password for authentication. Default: 'anonymous@'     "promptForPass": false, // boolean - Set to true for enable password dialog. This will prevent from using cleartext password in this config. Default: false     "remote": "/home/user/",     "secure": false, // mixed - Set to true for both control and data connection encryption, 'control' for control connection encryption only, or 'implicit' for implicitly encrypted control connection (this mode is deprecated in modern times, but usually uses port 990) Default: false     "secureOptions": null, // object - Additional options to be passed to tls.connect(). Default: (null) see http://nodejs.org/api/tls.html#tls_tls_connect_options_callback     "connTimeout": 10000, // integer - How long (in milliseconds) to wait for the control connection to be established. Default: 10000     "pasvTimeout": 10000, // integer - How long (in milliseconds) to wait for a PASV data connection to be established. Default: 10000     "keepalive": 10000 // integer - How often (in milliseconds) to send a 'dummy' (NOOP) command to keep the connection alive. Default: 10000 }
Stack Trace :
SyntaxError: Unexpected token /
at Object.parse (native)
at /home/egg/.atom/packages/Remote-FTP/lib/client.js:78:17
at FSReqWrap.readFileAfterClose [as oncomplete] (fs.js:380:3)

{
"protocol": "ftp",
"host": "45.xx.xx.xxx", // string - The hostname or IP address of the FTP server. Default: 'localhost'
"port": 21, // integer - The port of the FTP server. Default: 21
"user": "root", // string - Username for authentication. Default: 'anonymous'
"pass": "somepw", // string - Password for authentication. Default: 'anonymous@'
"promptForPass": false, // boolean - Set to true for enable password dialog. This will prevent from using cleartext password in this config. Default: false
"remote": "_myground",
"secure": false, // mixed - Set to true for both control and data connection encryption, 'control' for control connection encryption only, or 'implicit' for implicitly encrypted control connection (this mode is deprecated in modern times, but usually uses port 990) Default: false
"secureOptions": null, // object - Additional options to be passed to tls.connect(). Default: (null) see http://nodejs.org/api/tls.html#tls_tls_connect_options_callback
"connTimeout": 10000, // integer - How long (in milliseconds) to wait for the control connection to be established. Default: 10000
"pasvTimeout": 10000, // integer - How long (in milliseconds) to wait for a PASV data connection to be established. Default: 10000
"keepalive": 10000 // integer - How often (in milliseconds) to send a 'dummy' (NOOP) command to keep the connection alive. Default: 10000
}

For some reason (for me, at least), this error was getting triggered due to the comment ( // string - the... ) slashes. Removing all the comments has everything working just fine.
Not sure if this is due to the way the .ftpconfig file is being parsed or is somehow related to Atom, but either way everything works great now.
I also had to blow away my machine a few nights ago and had the exact same problem again, resolved the same way with a completely fresh environment. Couldn't remember how I did it so hoping this comment will at a minimum remind me in the future, but ideally help others.
Setup:
Mac OSX 10.11
Atom 1.1.0
Remote-FTP 0.7.8, 0.7.9

Upon further review, I'm 99% confident the comments are causing the issue.
On line 78, /lib/client.js:
JSON.parse is called on the "streamed" .ftpconfig string, which is not to spec for JSON.parse
Here's a demonstration of what's happening so you can play with it a bit:
http://codepen.io/jwp/pen/QjxzKB
The tricky thing is, I'm not sure the best approach to resolve, aside from trying to write some massive function or half-baked regex to wipe the comments. At a minimum, it seems like a user-friendly error would be a good approach to start.

this config fike has worked for me
{
"protocol": "ftp",
"host": "xxx.xxx.xxx.xxx",
"port": 21,
"user": "xxx.xx",
"pass": "xyzabcpoi",
"promptForPass": false,
"remote": "/",
"secure": false,
"secureOptions": null,
"connTimeout": 10000,
"pasvTimeout": 10000,
"keepalive": 10000
}

In JSON you can't use comments and thats why the parsing fails. But there is a module for that called strip-json-comments which now sanitizes the config file before parsing. Fixed in commit 2e8677f9857f92b103ca1180f6177f4ee24fd1b7
Simply remove the comments in .ftpconfig or update Remote-FTP to resolve the issue :smile:

Woohoo, thanks, Jan!


Add Icon on App profile page

Summary of the Pull Request
Add an icon to the top of the profile page, next to the name
References and Relevant Issues
Closes #9694
Detailed Description of the Pull Request / Additional comments
This PR is being opened in a not-finalized state to allow collaboration (specifically for image scaling issues)
Validation Steps Performed
PR Checklist

[x] Closes #xxx
[ ] Tests added/passed
[ ] Documentation updated

If checked, please file a pull request on our docs repo and link it here: #xxx

[ ] Schema updated (if necessary)

@microsoft-github-policy-service agree

Hey thanks! What's it look like?


Trying again

This is a new API script for use with the Fate of the Norns game and (preferably) character sheet. It handles playmat management and the game's unique "rune drawing" mechanism for character actions. My last attempt drew the comment "In the future please remove the DS_Store update in this pull request." I don't think I have a DS_Store update in this request, but I didn't know I did last time.

Looks like you have fixed the folder structure as requested. Merging!


WIP: Let gateways specify their own route hints

This PR implements:

Route hints to the GW LN node for users

The gateway fetching all its channels from CLN and giving them to the client as route hints
The client builds two-hop hints from these [to_gw_ln_node, fake_channel]
Route hints are only updated on gateway restart, so restart CLN after opening/closing channels

Gateway registration times out

fedimintd will only return gateways whose expiry time hasn't been hit
The gateway spawns a process that keeps re-registering itself with the federation to stay active
The client will re-fetch gateways once its active gateway expires (problem: no preference is given to the previously chosen one, TODO for later)
This together allows switching gateways in live deployments and changing their route hints without involving users

fixes #1498

This is pretty much ready except for the WASM incompatibility and maybe some clippy nits. I don't know the WASM ecosystem that well, what would you suggest to use for SystemTime @Maan2003? Does wasm_timer sound good?

Does wasm_timer sound good?
nope that SystemTime is just re-export of std's SystemTime (see 1.8.0 near [src] in docs)
The one Justin suggested should be good.

I think this closes #1094


Implement sector concept of Bundes-Klimaschutzgesetz

Closes #580
Sector division individuals added:

KSG sector division

CRF sector individuals added:

CRF sector (IPCC 2006): fugitive emissions from fuels
CRF sector (IPCC 2006): CO2 transport and storage
CRF sector (IPCC 2006): domestic aviation
CRF sector (IPCC 2006): road transportation
CRF sector (IPCC 2006): railways
CRF sector (IPCC 2006): domestic navigation
CRF sector (IPCC 2006): other transportation
CRF sector (IPCC 2006): pipeline transport

KSG sector individuals added:

KSG sector energy industry
KSG sector industry
KSG sector buildings
KSG sector transport
KSG sector agriculture
KSG sector waste management and other
KSG sector land use, land-use change and forestry

Thanks for your approval, @sfluegel05. As this is quite a large pull request adding more than a dozen individuals I wait for at least a second approval before merging.


How to use on Nest monorepo structure ?

Question
Hi!
I need to use Nestia on a Nest project with monorepo structure. Sadly, I get this error after enable monorepo mode.
Error: Error on nestia.core.TypedRoute.Get(): no transform has been configured. Run "npx typia setup" command, or check if you're using non-standard TypeScript compiler like Babel or SWC.

Running npx typia setup don't help, by default, nest seems to use webpack [1] and the "plugin" [2] option is setup in tsconfig.json
// project-name-npm-mono/nest-cli.json:
...
  "compilerOptions": {
    "deleteOutDir": true,
    "webpack": true, // <<---- [1]
    "tsConfigPath": "apps/project-name-npm/tsconfig.app.json"
  },
...

// project-name-npm-mono/tsconfig.json:
...
    "plugins": [ // <<--- [2]
      {
        "transform": "@nestia/core/lib/transform",
        /**
         * Validate request body.
         *
         *   - "assert": Use typia.assert() function
         *   - "is": Use typia.is() function
         *   - "validate": Use typia.validate() function
         *   - "assertEquals": Use typia.assertEquals() function
         *   - "equals": Use typia.equals() function
         *   - "validateEquals": Use typia.validateEquals() function
         */
        "validate": "assert",
        /**
         * Validate JSON typed response body.
         *
         *   - "assert": Use typia.assertStringify() function
         *   - "is": Use typia.isStringify() function
         *   - "validate": Use typia.validateStringify() function
         *   - "stringify": Use typia.stringify() function, but dangerous
         *   - null: Just use JSON.stringify() function, without boosting
         */
        "stringify": "assert"
      },
      {
        "transform": "typia/lib/transform"
      }
    ],
...

I don't find any resource about using Nestia on monorepo, did I miss something ?
Here I've created a repo with the walkthrough of my test https://github.com/fluffy-kaiju/nestia-v3.2.1-nestjs-monorepo-issue
Thanks for your help :smile:

Currently, to make monorepo SDK generation working, you should generate SDK library per each package.
I'll study how to, but cannot sure to be succeeded.

@fluffy-kaiju When use nestia you must compile your app by tsc and the default repo nestjs generate use webpack: true. So you have two way to make run correctly:
// project-name-npm-mono/nest-cli.json:
...
  "compilerOptions": {
    "deleteOutDir": true,
    "webpack": false, // <<---- [1]
    "tsConfigPath": "apps/project-name-npm/tsconfig.app.json"
  },
...

or create config webpack config use tsc as compiler and transpileOnly: false


Command PhaseScriptExecution failed with a nonzero exit code

Hello,
I tried to build this project. But I have an error at build.
This is the error:
Thank you.
/Users/myuser/Library/Developer/Xcode/DerivedData/Velik-ezhahudpeualwshfsnpeqbtzlvtb/Build/Intermediates.noindex/Velik.build/Debug-iphonesimulator/Velik.build/Script-4B10F19B249A84830050EEF4.sh: line 3: ./make.sh: No such file or directory
Command PhaseScriptExecution failed with a nonzero exit code

Please try to pull latest changes from master branch. And also verify that you have make.sh file in place:
https://github.com/avdyushin/Velik/blob/master/Velik/Resources/Localizable/make.sh

thx for your feedback.
I did pull latest changes from master branch.
I verified the files are present. I don't know what is wrong.
I did not managed to make it work :(


feat(profile): ipns avatar -> redirect to the linked url at /avatar_id

sweet

will work on any textile node gateway... desktop can use it's own gateway, but mobile should use the cafe


fix: systemd service -- handle 40x and block indefinitely

Closes #22173
Fixes systemd wrapper script behavior to handle configurations with http, https with tls, and user auth enabled.

[x] CHANGELOG.md updated with a link to the PR (not the Issue)
[x] Well-formatted commit messages
[x] Rebased/mergeable
[x] Tests pass

Can we also add the test cases from #22052 to this one?

@danxmoran done.


Broken link

The link  05.a.multi-turn-prompt-with-language-fallback sample is broken (https://github.com/microsoft/BotBuilder-Samples/tree/master/experimental/language-generation/csharp_dotnetcore/05.a.multi-turn-prompt-with-language-fallback). It leads to a 404 page on GitHub.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 59573a90-7a7a-418c-f661-90f53a00e47d
Version Independent ID: cec084c0-0823-0c68-887c-bc660ac109cc
Content: Language Generation - Bot Service
Content Source: articles/v4sdk/bot-builder-concept-language-generation.md
Service: bot-service
GitHub Login: @Kamrani
Microsoft Alias: kamrani

Thanks for letting us know @roberi. Here is the link to the sample.

Closing now that the changes are live.


TaxonomySortMutation

Definition
Message (TypeScript definition)
interface Message {
  type: "TaxonomySortMutation",
  payload: Payload
}

interface Payload {
  childrenIds: number[]
  taxonomyTermId: number
  userId: number
}

Output (TypeScript definition)
interface Output {
  success: boolean
}

Description what happens in the database layer
Sets the position of the linked entities in taxonomy term with taxonomyTermId according to childrenIds + position of children taxonomy ids. The position of the linked entities / taxonomy elements can be the index in childrenIds.
Necessary checks / possible errors

[x] childrenIds is the same set of uuids as the union of entities linked to taxonomyTermId and the taxonomy term children of taxonomyTermId

[ ] Add constraints to the term_taxonomy_entity. Right now (at least in the dump), it doesn't check if  entity_id and term_taxonomy_id have their counterparts in the respective tables.

Hey @kulla
just to be sure: it doesn't trigger any event, does it? Not even taxonomy/term/update?

just to be sure: it doesn't trigger any event, does it? Not even taxonomy/term/update?
If it doesn't we don't need userId in payload.

Do a "taxonomy/term/update" for the root taxonomy


Disallow robots crawling the old docs.

This should fix the issues we have with search engines.
Being that we have search for 4.0+ and it's working in the future, I'd say this is the only way.
Please chime in.
To test it locally set HUGO_ENV=production. The Netlify preview won't show this, but the result should be the following:
# www.robotstxt.org
# Allow crawling of all content
User-agent: *
Sitemap: http://localhost:9001//sitemap.xml
Disallow:

Disallow: /1.0.0/
Disallow: /1.1.0/
Disallow: /1.1.1/
Disallow: /1.2.0/
Disallow: /1.3.0/
Disallow: /1.4.0/
Disallow: /2.0.0/
Disallow: /2.0.1/
Disallow: /2.0.2/
Disallow: /2.0.3/
Disallow: /2.0.4/
Disallow: /2.1.0/
Disallow: /2.1.1/
Disallow: /2.2.0/
Disallow: /2.2.1/
Disallow: /2.2.2/
Disallow: /2.3.0/
Disallow: /2.3.1/
Disallow: /2.3.2/
Disallow: /docs/3.3/
Disallow: /docs/3.4/
Disallow: /docs/4.0/
Disallow: /docs/4.1/
Disallow: /docs/4.2/

Awesome! Happy to see this is being dealt with. I haven't tested the above locally - but the output robots.txt above looks good - though is it intentional to not have indexing for Bootstrap 3.4? I'd have thought it'd be good if someone searching for Bootstrap 3 would get the 3.4 docs. I know 3.x is end-of-life now but it's still widely used and could mean that top search position for 'Bootstrap 3' is lost to w3schools or something....

Yeah, I was actually thinking about it too. Perhaps I should allow the latest version of each major version, although I'm not sure if this will help us with the issue we are having.

Perhaps I should allow the latest version of each major version

yes, this would be the best option I think. I think it'd be worthwhile adding rel="nofollow" to the docs blocked in robots.txt on that docs version dropdown too... e.g.
        Latest (4.3.x)
        v4.2.1
        v4.0.0

        v4 Alpha 6
        v3.4.1
        v3.3.7

Too troublesome to do at this point and I hate editing the old docs manually all the time. Feel free to open a new issue or if you find the original issue comment there.


Declined charge not being stored in DB, previous charge still active

Expected Behavior
I'm triggering the next billing plan, when a user sync products to the DB and it is over a certain amount. When the user click cancel, it assumed it would store the charge in the DB I assume and marked it as declined.
This way I can add next steps to make sure they have an active plan at the right level or constantly ask them to accept the billing to do anything or lock down parts of the app until they do.
Current Behavior
The app is being redirected to the billing page and back to app homepage if they decline but nothing in the DB is happening. Old charge is still there and still "active", new charge which was declined is not even saved in the DB so no record of it.

Steps to Reproduce
Please provide detailed steps for reproducing the issue.

Create two plans
Add user to first plan
Trigger next plan and cancel it
User is pushed to apps page with no decline message and nothing happens in the app to store that it has happened.

Context
Please provide any relevant information about your setup. This is important in case the issue is not reproducible except for under certain conditions.

Package Version: v8.0
Laravel Version: v5.7
PHP Version: v7.2

I'm looking at a workaround for this but wondered, what is the best way of adding a custom method to the Shop Model?
I've added a "upgrade_to_plan_id" column in the Shop table. Then will use this to see if the plan_id and this match in a custom middle ware. If not redirect them to the billing screen.
This seems like the best way around the declining as I think it changed recently?

@Kyon147 If someone clicks decline on Shopify for the charge, to my knowledge there is no flag or anything sent back for us to detect.

@ohmybrew no worries, was not 100% sure!
In the end I've added a migration which adds a field to the Shop model called upgrade_to_plan_id this then gets updated when the products are sync to the database. Depending on the amount it sets this to the plan it needs to be. This field could be used for any logic though.
Then I have made a middleware which checks to make sure plan_id and upgrade_to_plan_id are the name or upgrade_to_plan_id is null.
If they are not the same or null, then it redirects using your billing route to trigger the plan. I've also added a method for the blade view so that it removes certain buttons and replaces them with "upgrade".
This seemed like the best way to do it and thought I'd outline in in case someone else wants to do something similar.


Individual fields whitelisting option added.

added an option to whitelist individual fields
refactored code slightly
added some basic tests for whitelisting

@tractorcow Code updated based on feedback. Please review.

Cleaned up and replaced with https://github.com/tractorcow/silverstripe-fluent/pull/303


Why 5 seconds wait for new cookie it takes hours to download 1700 tickers

Ran, thank you for creating this. I was about to do the same, but you saved me a lot of time. One question, why wait 5 seconds for the new cookie request? It takes hours to download the 1700 tickers I used to download in 30 minutes... every 2 or 3 tickers, the code lags for 5  seconds... Thank you again

I removed the 5s wait in version 0.0.8 and instead lowered the cookie ttl to 3 min. I also keep track of all failed downloads and retries them later.
Upgrade using:
$ pip install fix_yahoo_finance --upgrade --no-cache-dir

and let me know if you see better results.

Ran,
I've cleaned up my ticker list to remove all tickers that weren't good anymore and used your 0.8 version and it took about 10-12 min to dowload +/- 1200 tickers with a dsl connection. This is incredible! Thank you so much! Keep up the great work.


Add getTransactions to soroban-rpc-client crate

The new method was recently added (https://github.com/stellar/soroban-rpc/pull/174) but the rust crate needs to be updated to use it.

Blocked for now till rpc 21.4.0

@BlaineHeffron @willemneal RPC new endpoints are live now on testnet, like getTransactions. Platform upgraded the RPC to 21.4.0 this AM.

Thanks @janewang I'll test it out.

Fixed by https://github.com/stellar/rs-stellar-rpc-client/pull/3


Move docs from sandbox

Description
Move these documents:
https://github.com/eclipse/microprofile-sandbox/tree/master/proposals/microprofile-telemetry
To the spec folder.

[ ] Application user
[X] Specification implementer

...I need to be able to:
See the Telemetry documentation created in the sandbox
...which enables me to:
Evolve the spec documentation

Still needed:

I need to send the preliminary work to the README.
add licenses to the spec
fix commit email.

I think this should be closed as all tasks were done. Please reopen if you think something is still missing.


isodatetime issue

Hello,
I am converting date '31 Mar 2017' to '2017-03-31T00:00:00'. I found solution as per your documentation: dateFormat(now, "isoDateTime");
so when i am using this :dateFormat('31 Mar 2017', "isoDateTime");
it is giving me output like:2017-03-31T00:00:00+0530
but i don't want +0530 part with datetime. please provide me the solution asap.

I has the same problem

You can specify your own pattern instead of using isoDateTime:
console.log(dateformat('31 Mar 2017', "yyyy-mm-dd'T'HH:MM:ss"));

thanks


Add chance to create a form sheet with just a view

Instead of having to create a view controller, the user will have the option to add just a view making for simple cases as showing an image zoom or a message.

Thanks


Add using Siddhi as Docker Micro Service documentation

Purpose

Add using Siddhi as Docker Micro Service user guide

Goals

Guide users to run Siddhi as a micro service on Docker

Codecov Report

Merging #1225 into master will increase coverage by <.01%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #1225      +/-   ##
============================================
+ Coverage     61.44%   61.45%   +<.01%
  Complexity       35       35
============================================
  Files           619      619
  Lines         24942    24942
  Branches       3909     3909
============================================
+ Hits          15326    15327       +1
+ Misses         8291     8288       -3
- Partials       1325     1327       +2

Impacted Files
Coverage Δ
Complexity Δ

...ore/util/persistence/InMemoryPersistenceStore.java
53.84% <0%> (-5.13%)
0% <0%> (ø)

.../io/siddhi/core/util/snapshot/SnapshotService.java
38.3% <0%> (-0.21%)
0% <0%> (ø)

...rocessor/stream/window/SessionWindowProcessor.java
75.37% <0%> (+1.49%)
0% <0%> (ø)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 92c384e...ab33527. Read the comment docs.


Add qq_plot

IMPORTANT: Please do not create a Pull Request without creating an issue first.
Before submitting (must do checklist)

[x] Did you read the contribution guide?
[x] Did you update the docs? We use Numpy format for all the methods and classes.
[ ] Did you write any new necessary tests?
[x] Did you update the CHANGELOG?

Type of Change

[ ] Examples / docs / tutorials / contributors update
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] Improvement (non-breaking change which improves an existing feature)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Proposed Changes
Look #591.
Related Issue
#591.
Closing issues
Closes #591.

Codecov Report

Merging #604 (e5575b1) into master (c309803) will decrease coverage by 31.87%.
The diff coverage is 41.93%.

:exclamation: Current head e5575b1 differs from pull request most recent head 413f8eb. Consider uploading reports for the commit 413f8eb to get more accurate results

@@             Coverage Diff             @@
##           master     #604       +/-   ##
===========================================
- Coverage   85.20%   53.32%   -31.88%
===========================================
  Files         117      118        +1
  Lines        5818     5840       +22
===========================================
- Hits         4957     3114     -1843
- Misses        861     2726     +1865

Impacted Files
Coverage Δ

etna/analysis/eda_utils.py
19.33% <21.42%> (+0.21%)
:arrow_up:

etna/analysis/utils.py
53.33% <53.33%> (ø)

etna/analysis/__init__.py
100.00% <100.00%> (ø)

etna/analysis/plotters.py
11.78% <100.00%> (-7.46%)
:arrow_down:

etna/commands/__init__.py
0.00% <0.00%> (-100.00%)
:arrow_down:

etna/commands/backtest_command.py
0.00% <0.00%> (-96.43%)
:arrow_down:

etna/commands/forecast_command.py
0.00% <0.00%> (-92.00%)
:arrow_down:

etna/commands/__main__.py
0.00% <0.00%> (-87.50%)
:arrow_down:

etna/commands/resolvers.py
0.00% <0.00%> (-80.00%)
:arrow_down:

etna/analysis/outliers/density_outliers.py
22.44% <0.00%> (-75.52%)
:arrow_down:

... and 69 more

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more


Job management

For now it seems things are more or less working if you assume that the user is running on a single node (but even then there are the issues of selecting multiple GPUs). I'm mostly opening this as a place for discussing on how to improve the situation.

Has a similar problem already been solved?
Some kind of job management library could turn out to be very useful, e.g. http://ccl.cse.nd.edu/software/workqueue/
What's the most portable way to detect GPUs on a given node?

On the queuing side: PBS is able to support allocating gpus; it will create a file PBS_GPUFILE which is similiar to PBS_NODEFILE. On degennes, I wrote some scripts to parse this file so you can get which gpu you should run on.  However, Raham doesn't use the first-in, first-out scheduler that comes with PBS, and instead uses Maui for scheduling, which sadly, breaks the PBS_GPUFILE aspect of PBS, but gives you other useful features, in particular, the ability to assign priority to queues (and implement fairshare for users). I talked with Will about SLURM, but the effort to get that working seems too much, and they didn't have much insight into dealing with the gpu allocation, since they just set their nodes to be job exclusive (that is, if you request a gpu node, you get all the gpus/cores).  All of the gpu-enabled systems I've worked on have all basically been job exclusive, which is what I think we will stick with.
However, related to figuring out the gpus on a given node, you can always parse the output from nvidia-smi.  Although I've not tried to figure out how to do this on a multi-node run, since you'd need this info from all the different nodes you are asking for.
Since MSIBI is effectively spawning jobs, I agree some sort of job management code would be useful to keep track of them.   I'm not sure the best way to do this, since you'd obviously want this code to be rather self-contained (i.e., not need to be installed on the cluster itself, but as part of the code).

you can always parse the output from nvidia-smi
@tcmoore3, didn't you find a case where this broke? If not then I guess that problem is already solved for nvidia systems.

They intentionally broke some of the nicer features for the consumer cards, so you can't see which processors are using which gpu.  However, you can still figure this out which gpu is being tasked via the memory usage.
Check out:
https://developer.nvidia.com/nvidia-management-library-nvml

So we should probably be using this if it turns out to be stable instead of parsing the output of nvidia-smi. Hopefully it's also actively maintained...

That software looks reasonable.  Presumably we could write a simple piece of C/C++ code to access the library functions we specifically need as well, if it turns out not to be maintained.

nvidia-smi has been working on accre. Right now our jobs aren't running in parallel in the most sophisticated manner (e.g., all jobs in a "chunk" finish before the next chunk of jobs is launched). That's something we need to figure out on the MS IBI side. As far as allocating resources, this nvidia-ml-py seems to do what we want, and is somewhat actively maintained.
On a positive note, I got MSIBI up and running on rahman. On a less positive note, it's not properly allocating GPUs. It's doing the different simulations in parallel, 2 at a time (2 GPUs), but it's assigning the same GPU to each job. Clearly, something is amiss, but I haven't yet taken the time to thoroughly investigate.

When we were running on keeneland, there were 3 gpus per node.  These were nanowire elongation runs that would finish whenever the wire broke (so each job walltime varied a lot).  What I did was have my script break up the jobs into 3 subscripts (one per gpu).  So gpu=0 got a list of jobs to run, gpu=1, got a list, etc.  Each list was backgrounded and thus didn't depend on the others (until they were all finished and encountered the wait statement, since I was using bash).   We were running ~100 jobs, so this approach worked, I'm assuming MSIBI will have <10 that need to run concurrently, so that might not be the best plan.

Clearly, what we need is something like PBS but for each gpu that the code can call. But for portability, it needs to be stand alone software because most people can't install stuff on clusters.
I'm sure we can do this if we have a shared datastructure that keeps track of which jobs are running, on what gpu, and which are done, all embedded in a parallel while loop.  This should be relatively easy with openmp: asynchronous thread execution (including the ability to lock the shared array if you are grabbing a job from it or changing the status of a job), with only one blocking statement after the entire loop has been completed.
I'm going to mess around with writing a simple code (that doesn't actually really run anything, just does the threading).

It looks like nvidia-smi is working properly on Rahman.  I basically plucked out the essential bits of code from the worker script and tested them.  It was telling me two gpus, and assigning me the right one based on idx in the loop.
I think someone needs double check what idx is being assigned in the actual worker routine and also set up a minimal test of the multiprocessing part (make sure there isn't a bug in the python version we are using).

So the issue seems to be related to setting "chunk_size" in the imap function.  When you define the pool, you are setting the number of threads that can be run at a time.  When you define chunk_size, you are telling it how to partition the total number of tasks, as presumable you can get some speed increase by smart partitioning (for large number of jobs).  However, since idx, which defines which card to use, is not defined based on what chunk you end up in, you end up having cases where it's trying to run all jobs that were given an even idx (and hence running on the same card).
Basically, if you sent chunk_size = 1, it solves the problem, since it doesn't try to group jobs, and it will just do them sequentially (i.e., the same manner idx is assigned).

Ok so let's keep the general discussion on ideas and resolve specifics in their own issues.

Fixes for intra-node GPU selection: #20
Job management across multiple nodes will require some more brainstorming.
Once we start supporting more engines, we may have to talk about divvying up CPU simulations as well.

Well, more on the general topic of the multiprocessing library...can we spawn processes asynchronously?  Like if we tell it to run 10 total threads, 2 threads at a time, is the blocking operation at the end of executing all 10 threads, or after executing 2?
Furthermore, can we control how it breaks up the threads into chunks?  We can obviously calculate the time to complete each process...these times are relatively consistent during the iteration.  We could group jobs by the approximate wall time they take, in order to load balance better.   By grouping it could be: (1) which two jobs run at the same time or (2) which jobs go into which chunk.   The gpu id to use could be dynamically set (have a shared memory array that says which gpu is idle/used), rather than being assigned initially.
In terms of running on multiple nodes, gnu parallel might do the trick, since we are basically talking about a bunch of "serial" jobs.  This is also pretty lightweight code someone could build in their home account if it's not available on a cluster. This is nice because it just uses the pbs_nodefile, so there is nothing really tricky going on. We would probably have to assume a homogeneous environment (i.e., each node has the same number of gpus), since the current code queries nvidia_smi.
https://wiki.scinet.utoronto.ca/wiki/index.php/User_Serial#GNU_Parallel

@tcmoore3 started messing with the asynchronous threading. Here's actually a good explanation that might help clear up some of the confusion we had: http://stackoverflow.com/a/26521507/2597468
However, let's say we get the workers submitted asynchronously, we're still managing the gpu_id manually which means, e.g., on a 2 GPU node job 1 and job 2 start running, 2 finishes and 3 gets launched but because of how we're defining gpu_id, it will try to run on top of job 1. To solve that we would either need to do some kind of approximation on the wall time like you suggested or otherwise we'd have to get the GPU activity somehow?
GNU parallel looks pretty nice. Sounds like it'd be worth playing with on Rahman.

In the current code, the gpu is determined by doing the modulus of your job id and the number of available cards.   As such, if things are asynchronous, a job may run on an occupied gpu.
I think the simple solution is to change the gpu assignment to make it dynamically assigned by which one isn't being used both other threads. We need a shared memory array, where each index represents a gpu.  If the value of the array = 0, the gpu corresponding to that index available, if it's 1, it's not available.  The thread just picks out which gpu is available and increments the array to 1 (and when it's done, it sets it to 0).  Obviously, this requires a thread to lock the array before reading (i.e., only one thread can access the array at a time, both for read and write operations)...making sure that is correct is probably the tricky part (again, I know how to do that in openmp, need to make a test script in python to get the logic correct).

That's actually much simpler. I think you can just use a normal dictionary since this part of the code is just threaded:
http://stackoverflow.com/questions/1312331/using-a-global-dictionary-with-threads-in-python
It may require using the lock like in the example shown but I don't think it will be particularly complicated.

Or maybe a queue is simpler? http://stackoverflow.com/a/1312704/2597468


chore(main): release 1.1.1

:robot: I have created a release beep boop
1.1.1 (2022-11-07)
🧹 Miscellaneous

add sponsor button (3d04dab)
change owner in license from unjs to myself (b5bcf4a)
deps: update all non-major dependencies (#10) (cb2fb2a)
deps: update all non-major dependencies (#13) (3462b74)
deps: update all non-major dependencies (#15) (21d536f)
don't run prettier on pnpm lock file (58504ac)
provide npm token in correct form (cd85147)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/apollo-server-integrations/apollo-server-integration-h3/releases/tag/v1.1.1 :sunflower: