GPU memory issue

I used the train script from README to kick off 125M model training for 10 batches, exactly as in the example and was surprised to see this used almost all of my GPU memory (3080 16GB)
Here's the output from the last batch:
[batch=10/10]:
Train time/batch: 9
Train time/sample: 2304
Train time/batch_in_epoch: 9
Train time/sample_in_epoch: 2304
Train time/token: 4718592
Train time/token_in_epoch: 4718592
Train memory/allocated_mem: 3.8951
Train memory/active_mem: 3.8951
Train memory/inactive_mem: 5.5924
Train memory/reserved_mem: 13.5500
Train memory/alloc_retries: 3
Train trainer/device_train_microbatch_size: 8
Train loss/train/total: 10.9654
Train metrics/train/LanguageCrossEntropy: 10.9651
Train metrics/train/LanguagePerplexity: 57818.7188
Train time/train: 0.0606
Train time/val: 0.0000
Train time/total: 0.0606
Train lr-DecoupledAdamW/group0: 0.0001
Train time/remaining_estimate: 0.0000
I would ideally want to train/fine-tune a larger model, but even trying the 350M model fails:
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 1.54 GiB (GPU 0; 16.00 GiB total capacity; 13.53 GiB already allocated; 6.00 MiB free; 14.01 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
ERROR:composer.cli.launcher:Rank 0 crashed with exit code 1.
I checked with nvidia-smi before running the train script and the GPU had no memory allocated.

Hi @zhranj , please try reducing the device_train_microbatch_size. This will keep the math the same (within numerics) but will reduce the memory usage. The current settings have all been tuned to A100-40GB cards.
If you're unsure how to tune the microbatch size, try device_train_microbatch_size: auto and Composer will automatically catch OOMs and tune the microbatch size to fit on your hardware.
In general, using close-to-max GPU memory is not a bad thing, in fact you should try to use as much as possible (aka as large of a microbatch size as possible) to maximize throughput.
Let us know how this goes!

Thanks for the answer @abhi-mosaic, this helped with the 350M, but trying 1b or 3b or 7b gives a different error now which I'm not sure how to categorize:
root@8e801cb1b9dc:/workspaces/llm-foundry/scripts# composer train/train.py   train/yamls/mpt/1b.yaml   data_local=my-copy-c4   train_loader.dataset.split=train_small   eval_loader.dataset.split=val_small   max_duration=10ba   eval_interval=0   save_folder=mpt-1b
Initializing model...
cfg.n_params=1.32e+09
Building train loader...
Building eval loader...
Building trainer...
/usr/lib/python3/dist-packages/composer/callbacks/speed_monitor.py:120: UserWarning: gpu_flop count not found for None with precision: amp_bf16; MFU cannot be calculated and reported. gpu_flops_available can be manuallyoverridden by setting gpu_flops_available in SpeedMonitor.
warnings.warn(
Logging config...
data_local: my-copy-c4
data_remote: null
max_seq_len: 2048
global_seed: 17
run_name: llm
model:
name: mpt_causal_lm
init_device: meta
d_model: 2048
n_heads: 16
n_layers: 24
expansion_ratio: 4
max_seq_len: ${max_seq_len}
vocab_size: 50368
attn_config:
attn_impl: triton
tokenizer:
name: EleutherAI/gpt-neox-20b
kwargs:
model_max_length: ${max_seq_len}
train_loader:
name: text
dataset:
local: ${data_local}
remote: ${data_remote}
split: train_small
shuffle: true
max_seq_len: ${max_seq_len}
shuffle_seed: ${global_seed}
drop_last: true
num_workers: 8
eval_loader:
name: text
dataset:
local: ${data_local}
remote: ${data_remote}
split: val_small
shuffle: false
max_seq_len: ${max_seq_len}
shuffle_seed: ${global_seed}
drop_last: false
num_workers: 8
scheduler:
name: cosine_with_warmup
t_warmup: 100ba
alpha_f: 0.1
optimizer:
name: decoupled_adamw
lr: 0.0002
betas:

0.9
0.95
eps: 1.0e-08
weight_decay: 0.0
algorithms:
gradient_clipping:
clipping_type: norm
clipping_threshold: 1.0
max_duration: 10ba
eval_interval: 0
eval_first: false
eval_subset_num_batches: -1
global_train_batch_size: 512
seed: ${global_seed}
device_eval_batch_size: 4
device_train_microbatch_size: 2
precision: amp_bf16
fsdp_config:
sharding_strategy: FULL_SHARD
mixed_precision: PURE
activation_checkpointing: false
activation_checkpointing_reentrant: false
activation_cpu_offload: false
limit_all_gathers: true
verbose: false
progress_bar: false
log_to_console: true
console_log_interval: 1ba
callbacks:
speed_monitor:
window_size: 10
lr_monitor: {}
memory_monitor: {}
runtime_estimator: {}
save_folder: mpt-1b
dist_timeout: 600.0
n_gpus: 1
device_train_batch_size: 512
device_train_grad_accum: 256
n_params:<PHONE_NUMBER>

Starting training...

Config:
enabled_algorithms/GradientClipping: true
node_name: unknown because NODENAME environment variable not set
num_gpus_per_node: 1
num_nodes: 1
rank_zero_seed: 17

Traceback (most recent call last):
File "", line 21, in _bwd_kernel
KeyError: ('2-.-0-.-0-d82511111ad128294e9d31a6ac684238-2b0c5161c53c71b37ae20a9996ee4bb8-c1f92808b4e4644c1732e8338187ac87-d962222789c30252d492a16cca3bf467-12f7ac1ca211e037f62a7c0c323d9990-5c5e32ff210f3b7f56c98ca29917c25e-06f0df2d61979d629033f4a22eff5198-0dd03b0bd512a184b3512b278d9dfa59-d35ab04ae841e2714a253c523530b071', (torch.bfloat16, torch.bfloat16, torch.bfloat16, None, torch.bfloat16, torch.float32, torch.bfloat16, torch.bfloat16, torch.float32, torch.float32, 'fp32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32'), ('none', True, 128, False, True, True, True, 128, 128), (True, True, True, (False,), True, True, True, True, True, True, (False,), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False)))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/workspaces/llm-foundry/scripts/train/train.py", line 254, in
main(cfg)
File "/workspaces/llm-foundry/scripts/train/train.py", line 243, in main
trainer.fit()
File "/usr/lib/python3/dist-packages/composer/trainer/trainer.py", line 1766, in fit
self._train_loop()
File "/usr/lib/python3/dist-packages/composer/trainer/trainer.py", line 1940, in _train_loop
total_loss_dict = self._train_batch(use_grad_scaling)
File "/usr/lib/python3/dist-packages/composer/trainer/trainer.py", line 2115, in _train_batch
optimizer.step(closure=lambda **kwargs: self._train_microbatches(
File "/usr/lib/python3/dist-packages/torch/optim/lr_scheduler.py", line 68, in wrapper
return wrapped(*args, **kwargs)
File "/usr/lib/python3/dist-packages/torch/optim/optimizer.py", line 140, in wrapper
out = func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/composer/optim/decoupled_weight_decay.py", line 288, in step
loss = closure()
File "/usr/lib/python3/dist-packages/composer/trainer/trainer.py", line 2115, in
optimizer.step(closure=lambda **kwargs: self._train_microbatches(
File "/usr/lib/python3/dist-packages/composer/trainer/trainer.py", line 2213, in _train_microbatches
microbatch_loss_dict = self._train_microbatch(use_grad_scaling, current_batch_size, is_final_microbatch)
File "/usr/lib/python3/dist-packages/composer/trainer/trainer.py", line 2340, in _train_microbatch
microbatch_loss.backward(create_graph=self._backwards_create_graph)
File "/usr/lib/python3/dist-packages/torch/_tensor.py", line 488, in backward
torch.autograd.backward(
File "/usr/lib/python3/dist-packages/torch/autograd/init.py", line 197, in backward
Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass
File "/usr/lib/python3/dist-packages/torch/autograd/function.py", line 267, in apply
return user_fn(self, *args)
File "/usr/lib/python3/dist-packages/flash_attn/flash_attn_triton.py", line 827, in backward
_flash_attn_backward(do, q, k, v, o, lse, dq, dk, dv,
File "/usr/lib/python3/dist-packages/flash_attn/flash_attn_triton.py", line 694, in _flash_attn_backward
_bwd_kernel[grid](
File "/usr/lib/python3/dist-packages/triton/runtime/jit.py", line 106, in launcher
return self.run(*args, grid=grid, **kwargs)
File "/usr/lib/python3/dist-packages/triton/runtime/autotuner.py", line 73, in run
timings = {config: self._bench(*args, config=config, **kwargs)
File "/usr/lib/python3/dist-packages/triton/runtime/autotuner.py", line 73, in
timings = {config: self._bench(*args, config=config, **kwargs)
File "/usr/lib/python3/dist-packages/triton/runtime/autotuner.py", line 63, in _bench
return do_bench(kernel_call)
File "/usr/lib/python3/dist-packages/triton/testing.py", line 140, in do_bench
fn()
File "/usr/lib/python3/dist-packages/triton/runtime/autotuner.py", line 62, in kernel_call
self.fn.run(*args, num_warps=config.num_warps, num_stages=config.num_stages, **current)
File "/usr/lib/python3/dist-packages/triton/runtime/autotuner.py", line 200, in run
return self.fn.run(*args, **kwargs)
File "", line 43, in _bwd_kernel
RuntimeError: Triton Error [CUDA]: invalid argument
ERROR:composer.cli.launcher:Rank 0 crashed with exit code 1.
Waiting up to 30 seconds for all training processes to terminate. Press Ctrl-C to exit immediately.
Global rank 0 (PID 5236) exited with code 1
ERROR:composer.cli.launcher:Global rank 0 (PID 5236) exited with code 1

Hi @zhranj , this looks like an issue with Triton and your particular hardware.
Could you try these two options:

using our docker image from the top README: mosaicml/pytorch:1.13.1_cu117-python3.10-ubuntu20.04
if this still fails, revert attn_impl: triton to attn_impl: torch which will just use PyTorch' attention and I would expect this to work on any NVIDIA GPU.

Note:
Traceback (most recent call last):
File "", line 21, in _bwd_kernel
KeyError: ('2-.-0-.-0-d82511111ad128294e9d31a6ac684238-2b0c5161c53c71b37ae20a9996ee4bb8-c1f92808b4e4644c1732e8338187ac87-d962222789c30252d492a16cca3bf467-12f7ac1ca211e037f62a7c0c323d9990-5c5e32ff210f3b7f56c98ca29917c25e-06f0df2d61979d629033f4a22eff5198-0dd03b0bd512a184b3512b278d9dfa59-d35ab04ae841e2714a253c523530b071', (torch.bfloat16, torch.bfloat16, torch.bfloat16, None, torch.bfloat16, torch.float32, torch.bfloat16, torch.bfloat16, torch.float32, torch.float32, 'fp32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32', 'i32'), ('none', True, 128, False, True, True, True, 128, 128), (True, True, True, (False,), True, True, True, True, True, True, (False,), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False), (True, False)))

means there is a mismatch with the triton version required for llm-foundry and the triton version installed on your system.
As Abhi noted, this means the environment setup is incorrect.


Update "outs" counts for RETURN and SUICIDE opcodes

Sources: http://gavwood.com/Paper.pdf and https://github.com/ethereum/yellowpaper/blob/master/Paper.tex#L1940 .

ping


feat: update collateral tab

Related issues 🔗
Closes #847 & #1366
Description ℹ
Update the collaterals tab.
Demo 📺
After:

Before:

Technical 👨🔧
Details of technical implementation that reviewers may need to be aware of, if applicable.

@MadalinaRaicu @PeteWilliams I don't think we should be making the collapsible rows right now. It requires big work arounds to make it work with AgGrid and the complexity overhead is not worth it. It will make filtering and sorting very difficult if not impossible. Additionally we end up with scroll bars all over the place and multiple ag grid instances rendered (which is quite heavy).
What I propose for now is to just render the rows as they come back from the API and have buttons to open the withdraw and deposit dialogs (general account rows only). If we really deem the progress bar and other changes necessary how about showing that on the asset dialog?


Segfault in alpine docker image while initializing client

This issue can only be reproduced in Alpine docker image: alpine or Rust alpine image: rust:alpine. Debian based images work just fine with the same code.
Min code
Running in actix-web::main
use reqwest::Client;
...
let client = Client::new(); // segfault (docker exist code 139) here
// Or using builder
let client = Client::builder().build(); // segfault (docker exist code 139) here
...

Error log
There isn't any log whatsoever. According to docker exit code 139, there is a segfault occurred.
Possible cause
I don't think it's caused by missing libraries or the log should show what's missing but instead, there are no logs.
Maybe something wrong with alpine's musl libc vs glibc?

Well that's pretty strange. Could you try debugging a little more, perhaps using strace to see what calls the program gets to before crashing?

Here is the strace:
Before crash
...
rt_sigprocmask(SIG_BLOCK, [QUIT], [], 8) = 0
brk(0x55555714e000)                     = 0x55555714e000
rt_sigaction(SIGQUIT, {sa_handler=0x7f64849224e0, sa_mask=[], sa_flags=SA_RESTORER|SA_RESTART|SA_SIGINFO|SA_NOCLDSTOP, sa_restorer=0x7f648496510f}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
fcntl(46, F_DUPFD_CLOEXEC, 0)           = 51
fcntl(51, F_SETFD, FD_CLOEXEC)          = 0
fcntl(51, F_GETFD)                      = 0x1 (flags FD_CLOEXEC)
epoll_ctl(3, EPOLL_CTL_ADD, 51, {EPOLLIN|EPOLLPRI|EPOLLOUT|EPOLLET, {u32=3, u64=3}}) = 0
epoll_pwait(3, [{EPOLLOUT, {u32=0, u64=0}}, {EPOLLOUT, {u32=1, u64=1}}, {EPOLLOUT, {u32=2, u64=2}}, {EPOLLOUT, {u32=3, u64=3}}], 1024, -1, NULL, 8) = 4

Crash
epoll_pwait(3,  <unfinished ...>)       = ?
+++ killed by SIGSEGV (core dumped) +++

Additionally, I have Valgrind log here:
==1== Thread 2 actix-rt:worker::
==1== Jump to the invalid address stated on the next line
==1==    at 0x0: ???
==1==    by 0x31EC3D: native_tls::imp::TlsConnector::new (in /fwcheck)
==1==    by 0x31F2B4: native_tls::TlsConnectorBuilder::build (in /fwcheck)
==1==    by 0x31C423: reqwest::connect::Connector::new_default_tls (in /fwcheck)
==1==    by 0x305706: reqwest::async_impl::client::ClientBuilder::build (in /fwcheck)
==1==    by 0x20A7F0: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1B06A1: <actix_web::handler::HandlerServiceResponse<T,R> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1BA4A8: <actix_web::handler::ExtractResponse<T,S> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1BACC9: <actix_web::handler::ExtractResponse<T,S> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1D6556: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1EBAA8: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x357780: <core::pin::Pin<P> as core::future::future::Future>::poll (in /fwcheck)
==1==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==1==
==1==
==1== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==1==  Bad permissions for mapped region at address 0x0
==1==    at 0x0: ???
==1==    by 0x31EC3D: native_tls::imp::TlsConnector::new (in /fwcheck)
==1==    by 0x31F2B4: native_tls::TlsConnectorBuilder::build (in /fwcheck)
==1==    by 0x31C423: reqwest::connect::Connector::new_default_tls (in /fwcheck)
==1==    by 0x305706: reqwest::async_impl::client::ClientBuilder::build (in /fwcheck)
==1==    by 0x20A7F0: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1B06A1: <actix_web::handler::HandlerServiceResponse<T,R> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1BA4A8: <actix_web::handler::ExtractResponse<T,S> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1BACC9: <actix_web::handler::ExtractResponse<T,S> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1D6556: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x1EBAA8: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll (in /fwcheck)
==1==    by 0x357780: <core::pin::Pin<P> as core::future::future::Future>::poll (in /fwcheck)
==1==
==1== HEAP SUMMARY:
==1==     in use at exit: 0 bytes in 0 blocks
==1==   total heap usage: 0 allocs, 0 frees, 0 bytes allocated
==1==
==1== All heap blocks were freed -- no leaks are possible
==1==
==1== Use --track-origins=yes to see where uninitialised values come from
==1== For lists of detected and suppressed errors, rerun with: -s
==1== ERROR SUMMARY: 5 errors from 2 contexts (suppressed: 0 from 0)

valgrind: the 'impossible' happened:
   main(): signal was supposed to be fatal

host stacktrace:
==1==    at 0x58031AD2: show_sched_status_wrk (m_libcassert.c:388)
==1==    by 0x58031D26: report_and_quit (m_libcassert.c:459)
==1==    by 0x58031F3D: panic (m_libcassert.c:535)
==1==    by 0x58031F3D: vgPlain_core_panic_at (m_libcassert.c:540)
==1==    by 0x58031F45: vgPlain_core_panic (m_libcassert.c:545)
==1==    by 0x58081326: shutdown_actions_NORETURN (m_main.c:2268)
==1==    by 0x580AA3C7: run_a_thread_NORETURN (syswrap-linux.c:203)
==1==    by 0x580AA465: vgModuleLocal_start_thread_NORETURN (syswrap-linux.c:320)
==1==    by 0x5807FA72: ??? (in /usr/lib/valgrind/memcheck-amd64-linux)
==1==    by 0xDEADBEEFDEADBEEE: ???
==1==    by 0xDEADBEEFDEADBEEE: ???
==1==    by 0xDEADBEEFDEADBEEE: ???

sched status:
  running_tid=2

Note: see also the FAQ in the source distribution.
It contains workarounds to several common problems.
In particular, if Valgrind aborted or crashed after
identifying problems in your program, there's a good chance
that fixing those problems will prevent Valgrind aborting or
crashing, especially if it happened in m_mallocfree.c.

If that doesn't help, please report this bug to: www.valgrind.org

In the bug report, send all the above text, the valgrind
version, and what OS and version you are using.  Thanks.

Ah interesting. It could be the crash is from native-tls... You could file an issue there. Also, you could try disabling native-tls in reqwest and enabling rustls.

In the past there have been caveats cross-compiling with musl around getting openssl to link correctly. Workarounds involved explicitly adding an extern crate openssl; to all the binary targets in your project and somehow the ordering was significant... https://github.com/emk/rust-musl-builder/issues/69#issuecomment-487027545
Using rustls should sidestep the issue since you'd no longer need to link openssl.

Enabling rustls works for me! Thanks guys!

Can you expand on what you mean by enable rustls to sidestep the tls issue?
I have been stuck on this for days, and can't get my head wrapped around how to fix this.
I have a really simple implementation here where I just create an actix-web route, initialize a new reqwest client, and I segfault when calling the route. I can see that it is related to the TLS failing to do something, but can't seem to get anything to work to fix it.
https://github.com/Daktic/mock_reqwest


Update/eduard #vc 2of1

Added new vaccination article to the FAQ

PREVIEW:
DE:

EN:

Please also update https://www.coronawarn.app/de/faq/#vac_cert_jj or at least add a reference.

@Ein-Tim Thanks for the suggestion, yes, we should reference/link these two articles mutually.

@dsarkar

Thanks for the suggestion, yes, we should reference/link these two articles mutually.

Will you take care of this?

@dsarkar
The merged article uses the term "immediate" (sofort). When will the fix be released? If it is not today, then I suggest changing the wording.


[v2 event.json] move services -> social_links

new api

old api

Please give us text instead of screenshots of code. You can use markup, if you want it to look different.


where is fully demo for this plugin

i want to read and write message to nfc device. could you please give some example?

Added a example app.
You can check it in the example directory.

@okadan thanks for your fully demo, it works.


devconfig.cgi RSSI List empty

The following command returns an empty RSSI list while the same command allows to display the current RSSI list on a standard CCU2:
http://homematic-raspi/tools/devconfig.cgi?cmd=show_rssi&sid=XXXXXX

After some short investigation the following line seems to be returning an empty result:
https://github.com/eq-3/occu/blob/master/WebUI/www/tools/devconfig.cgi#L359
Thus, on line 359 in devconfig.cgi the rssi_list variable seems to be empty after the xmlrpc call.

Please note that the following short tclsh script seems to work fine on a CCU2 but fails on RaspberryMatic:
#!/bin/tclsh
source once.tcl
loadOnce tclrpc.so
puts [xmlrpc "xmlrpc_bin://<IP_ADDRESS>:2001" rssiInfo]

On a CCU (within the /www/tools directory) it returns proper list of RSSI values, but on RaspberryMatic it returns an empty list (no RSSI values).

This problem seems to be solved with the latest 2.27.x release candidate version. Thus, closing it.

Moin Jens
i have the same problem
i ran an PI3 with a HM-MOD-RPI-PCB an <IP_ADDRESS>71118
i open : "http://192.168.xxx.xxx/tools/devconfig.cgi?cmd=show_rssi&sid=@xxxxxxxx@"
but i got only this
"
  | RSSI List
-- | --
  |
  | RSSI List
"
What can i do ?

Have exactly the same problem. Using the latest RaspberryMatic Version (<IP_ADDRESS>80225) but RSSI-List is empty. Maybe you can fix that? Or what else can I do?


Far-field spectra comparison

Thanks for releasing this new version!
I just happened to be needing a comparison with our TERMS program so I tested this new version. I was a bit confused by the different cross-sections in the output file (not sure what the " sphere-based total extinction, absorption, scattering efficiencies (unpolarized)" correspond to?) but eventually got there by summing the individual extinction and absorption cross-sections.
The structure is a helical strand of 8 Au spheres, 50nm in radius, in vacuum. I set Lmax to 12 (Mie truncation), which seems just enough. In TERMS I used a particle-centred formulation by Stout et al, which requires lower Lmax (no expansion about a common origin).

comparison.pdf
The temp_pos file is as follows
  0.120000000000E+03,  0.000000000000E+00, -0.175000000000E+03,  0.500000000000E+02, (  0.148077717605E+00,  0.492529106276E+01)
  0.848528137424E+02,  0.848528137424E+02, -0.125000000000E+03,  0.500000000000E+02, (  0.148077717605E+00,  0.492529106276E+01)
  0.734788079488E-14,  0.120000000000E+03, -0.750000000000E+02,  0.500000000000E+02, (  0.148077717605E+00,  0.492529106276E+01)
 -0.848528137424E+02,  0.848528137424E+02, -0.250000000000E+02,  0.500000000000E+02, (  0.148077717605E+00,  0.492529106276E+01)
 -0.120000000000E+03,  0.146957615898E-13,  0.250000000000E+02,  0.500000000000E+02, (  0.148077717605E+00,  0.492529106276E+01)
 -0.848528137424E+02, -0.848528137424E+02,  0.750000000000E+02,  0.500000000000E+02, (  0.148077717605E+00,  0.492529106276E+01)
 -0.220436423847E-13, -0.120000000000E+03,  0.125000000000E+03,  0.500000000000E+02, (  0.148077717605E+00,  0.492529106276E+01)
  0.848528137424E+02, -0.848528137424E+02,  0.175000000000E+03,  0.500000000000E+02, (  0.148077717605E+00,  0.492529106276E+01)

and for TERMS, the input file (full spectrum) reads
ModeAndScheme 2 3
Wavelength 380 820 100

MultipoleCutoff 12
Medium 1.0

ScattererCentredCrossSections

OutputFormat HDF5 terms

Scatterers 8
Au   1.200000e+02  0.000000e+00 -175 50
Au   8.485281e+01  8.485281e+01 -125 50
Au   7.347881e-15  1.200000e+02  -75 50
Au  -8.485281e+01  8.485281e+01  -25 50
Au  -1.200000e+02  1.469576e-14   25 50
Au  -8.485281e+01 -8.485281e+01   75 50
Au  -2.204364e-14 -1.200000e+02  125 50
Au   8.485281e+01 -8.485281e+01  175 50

I'll try a near-field comparison as well.

Hi Baptiste!
Ha!   I never expected the github mstm code to be discovered so fast!    I am still working on the manual, and am preparing a couple of papers (to jqsrt) that will describe the new features.      I’m completely new to github and am not even sure if this reply will reach you.
It looks like mstm and TERMs are agreeing perfectly – unless I am missing something.       Such a relief to have confirmation.
Is your Lmax=12 the truncation for the sphere-based expansions, or for the cluster as a whole (i.e., that for the cluster T matrix)?   Sphere size parameters go from pi/4 to pi/8, correct?      Seems odd that C_abs << C_sca in the IR – I would have expected Rayleigh behavior to start emerging (i.e., C_abs -> C_ext, for metals), yet I guess the size parameters are not quite small enough for this limit to take effect.
Please send a reply when you get this – like I said, I’m not sure where the email is going.
Cheers,
Dan
From: baptiste @.>
Sent: Tuesday, January 25, 2022 1:30 PM
To: dmckwski/MSTM @.>
Cc: Subscribed @.***>
Subject: [EXT] [dmckwski/MSTM] Far-field spectra comparison (Issue #1)
CAUTION: Email Originated Outside of Auburn.
Thanks for releasing this new version!
I just happened to be needing a comparison with our TERMS program<nano-optics.ac.nz/terms/> so I tested this new version. I was a bit confused by the different cross-sections in the output file (not sure what the " sphere-based total extinction, absorption, scattering efficiencies (unpolarized)" correspond to?) but eventually got there by summing the individual extinction and absorption cross-sections.
The structure is a helical strand of 5 Au spheres, 50nm in radius, in vacuum. I set Lmax to 12 (Mie truncation), which seems just enough. In TERMS I used a particle-centred formulation by Stout et al, which requires lower Lmax (no expansion about a common origin).
[Screenshot 2022-01-25 at 20 20 35]https://user-images.githubusercontent.com/18970/151045382-94b46220-caf4-4152-a9e7-4f7917d2df9f.png
comparison.pdfhttps://github.com/dmckwski/MSTM/files/7936561/comparison.pdf
The temp_pos file is as follows
0.120000000000E+03,  0.000000000000E+00, -0.175000000000E+03,  0.500000000000E+02, (  0.148077717605E+00,  0.492529106276E+01)
0.848528137424E+02,  0.848528137424E+02, -0.125000000000E+03,  0.500000000000E+02, (  0.148077717605E+00,  0.492529106276E+01)
0.734788079488E-14,  0.120000000000E+03, -0.750000000000E+02,  0.500000000000E+02, (  0.148077717605E+00,  0.492529106276E+01)
-0.848528137424E+02,  0.848528137424E+02, -0.250000000000E+02,  0.500000000000E+02, (  0.148077717605E+00,  0.492529106276E+01)
-0.120000000000E+03,  0.146957615898E-13,  0.250000000000E+02,  0.500000000000E+02, (  0.148077717605E+00,  0.492529106276E+01)
-0.848528137424E+02, -0.848528137424E+02,  0.750000000000E+02,  0.500000000000E+02, (  0.148077717605E+00,  0.492529106276E+01)
-0.220436423847E-13, -0.120000000000E+03,  0.125000000000E+03,  0.500000000000E+02, (  0.148077717605E+00,  0.492529106276E+01)
0.848528137424E+02, -0.848528137424E+02,  0.175000000000E+03,  0.500000000000E+02, (  0.148077717605E+00,  0.492529106276E+01)
and for TERMS, the input file (full spectrum) reads
ModeAndScheme 2 3
Wavelength 380 820 100
MultipoleCutoff 12
Medium 1.0
ScattererCentredCrossSections
OutputFormat HDF5 terms
Scatterers 8
Au   1.200000e+02  0.000000e+00 -175 50
Au   8.485281e+01  8.485281e+01 -125 50
Au   7.347881e-15  1.200000e+02  -75 50
Au  -8.485281e+01  8.485281e+01  -25 50
Au  -1.200000e+02  1.469576e-14   25 50
Au  -8.485281e+01 -8.485281e+01   75 50
Au  -2.204364e-14 -1.200000e+02  125 50
Au   8.485281e+01 -8.485281e+01  175 50
I'll try a near-field comparison as well.
—
Reply to this email directly, view it on GitHubhttps://github.com/dmckwski/MSTM/issues/1, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AU45VD5FH7UAIDK5OKF4KVTUX324PANCNFSM5MZEVCWA.
Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you are subscribed to this thread.Message ID<EMAIL_ADDRESS>

Hi Baptiste! Ha! I never expected the github mstm code to be discovered so fast!

Funny coincidence – I'm just finishing up our TERMS user guide (https://arxiv.org/abs/2111.12556) and needed a comparison with other programs, when I saw on your website you'd just posted a new version of MSTM.

I am still working on the manual, and am preparing a couple of papers (to jqsrt) that will describe the new features.

Great – if you have a working title for MSTM v4 please let me know, and I'll use it in our reference list.

It looks like mstm and TERMs are agreeing perfectly – unless I am missing something. Such a relief to have confirmation.

Yes, very pleasing indeed. Of course TERMS claims to implement the same algorithms as MSTM (in Scheme=3), though we've also mixed some bits and pieces from Stout et al (Scheme=0,1,2), such as "balancing" (pre-conditioning).

Is your Lmax=12 the truncation for the sphere-based expansions, or for the cluster as a whole (i.e., that for the cluster T matrix)?

For sphere-based expansions; in MSTM I set
mie_epsilon
-12

but I'm not too sure what the Nmax of the whole cluster becomes in MSTM. In TERMS I used the keyword ScattererCentredCrossSections, which implements formulas by Stout for far-field cross-sections that don't require a larger Nmax, regardless of particle separation (but it's slower).

Sphere size parameters go from pi/4 to pi/8, correct? Seems odd that C_abs << C_sca in the IR – I would have expected Rayleigh behavior to start emerging (i.e., C_abs -> C_ext, for metals), yet I guess the size parameters are not quite small enough for this limit to take effect.

Au nanoparticles sustain strong plasmon resonances in this size range, so it's not quite the standard Rayleigh regime, and it's typical above ~50nm in size to have a cross-over with Cabs < Csca).
PS: welcome to github – it's a pretty good platform for code sharing (by the way, you might want to put the source files rather than the zip archive, it makes it easier to view the contents etc.)

(closing because it wasn't an actual issue :)


Extract common RUN blocks to shared scripts

We have a lot of common RUN blocks that are copied from project to project for things like package installation and system configuration. We could create some shared bash scripts to cut down on these repeated blocks and possibly make debugging easier.
Some common RUN blocks:

Updating and upgrading system packages.
Installing a list of system packages.
Installing an R version.
Installing a list of R packages.
Installing a Python version.
Installing a list of Python packages.
Installing and configuring the Jupyter installation on Workbench projects.
Installing a Posit product (Workbench, Connect, or PPM).
Installing a custom LM for a service-based product.
Installing Quarto.
Installing tini.
Installing wait-for-it.sh.

Elevating this issue to due needs from internal requirements.

I've made significant progress on this ticket by borrowing scripts from rstudio/marketplaces. Marking as blocked for now though because our build workflow is too broken to make these updates a simple process.
https://github.com/rstudio/rstudio-docker-products/pull/716
Blocker: https://github.com/rstudio/rstudio-docker-products/issues/717


Phosanis HighScores suggestion

Just a suggestion about the in game high-scores feature to change the icon of phosanis nightmare to the same icon featured on the official osrs high scores
I just think along with the nex logo and the 2 nightmares side by side it would be cool to change it up and make it more definable

Also not sure if this is the right place to post or if this has been brought up before I am new to GitHub and have no idea what I’m doing lol 😂

Duplicate of #14214


Update Hardware.md

As promised hereby a new infographic and a short description about read-only pumps. Hopefully you consider it an improvement. If there is anything als it can do please ask. The png is located at my own server. Don't know how to get it at the right place. My english is not flawless. There could be some typos in the text.

Thanks @statuur! I can't open the raw file (but thank you for sharing it), so I made some notes on top of the image file: Dana edits_hardware.md.pdf
Key comments:

Fixed some typos and otherwise edited text that I had been wanting to edit!
Can we add a 640 (same as 670) pump visual to the top as well?
Let's make the 5/723 and 5/754 not have the color split, as it makes it look like 7-series is not compatible.
I like the read-only section addition! Let's also add a green arrow there to flow that step to OpenAPS.org

I'm wondering if it would be too much to also add the DANAR and DANARS and Combo pumps on the right, to point to AndroidAPS? Example poorly drawn :) below.

Let me know if you have any questions about the edits I suggested! Thanks again for working on this! (Also feel free to PM me on Gitter if it's easier to discuss in real-time.)


Keras load_model fails after training

Please make sure that the boxes below are checked before you submit your issue. If your issue is an implementation question, please ask your question on StackOverflow or join the Keras Slack channel and ask there instead of filing a GitHub issue.
Thank you!

[x] Check that you are up-to-date with the master branch of Keras. You can update with:
pip install git+git://github.com/fchollet/keras.git --upgrade --no-deps

[x] If running on TensorFlow, check that you are up-to-date with the latest version. The installation instructions can be found here.

[x] If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:
pip install git+git://github.com/Theano/Theano.git --upgrade --no-deps

[x] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

I tried the code given here:  https://github.com/fchollet/keras/issues/2295
I am unable to load my model after training it. I am getting the following error:
ValueError: Optimizer weight shape (3, 3, 512, 512) not compatible with provided weight shape (256, 43)
I tried the solution here: https://github.com/fchollet/keras/issues/3964 using HDFView and deleting the optimiser weights and then reloading. But then I get the error:
ValueError: ('shapes (10,4224) and (1128,256) not aligned: 4224 (dim 1) != 1128 (dim 0)', (10, 4224), (1128, 256))
Apply node that caused the error: Dot22(Reshape{2}.0, lstm_2_W_i)
Toposort index: 249
Inputs types: [TensorType(float32, matrix), TensorType(float32, matrix)]
Inputs shapes: [(10, 4224), (1128, 256)]
Inputs strides: [(16896, 4), (1024, 4)]
Inputs values: ['not shown', 'not shown']
Outputs clients: [[Elemwise{Add}[(0, 0)](Dot22.0, InplaceDimShuffle{x,0}.0)]]

I also tried the solution by barrykui here: https://github.com/fchollet/keras/issues/4044 saving the architecture to JSON and saving weights separately and then loading but even that fails.

I was able to devise a workaround. I followed the steps below:

Create the Sequential model.
Example:

model = Sequential()
model.add(...)
model.add(...)

model.compile(...)
model.fit(...)

After training save only model weights using model.save_weights()
Example:
model.save_weights(SaveLocation)
To load the model weights, create the model programmatically just as in step 1 but do not use the model.compile function.
Example:

model = Sequential()
model.add(...)
model.add(...)
model.load_weights(weightFile)

The weights are now loaded successfully.

I reopened this issue just in case someone else is able to provide a better solution.

Another solution is to name each layer in the model.
This helps if you're using checkpointing or something similar, where you cannot save the weights separately.
For ex: model.add(LSTM(10, name='lstm1'))
This worked for me in keras 1.1.1 and theano 0.8.2

I observed this issue on Keras 1.2.2 with TensorFlow 0.12.1 backend on Win64 (. The best solution so far has been from @AahanSingh (without needing to upgrade my setup). Other solutions have alluded to naming each layer and retraining before saving model JSON and weights, which for me was not a convenient option. I have redefined the model as per the suggestion - and this seems to be the simplest workaround till date (at least without having to upgrade my setup).


feat: refactor shared structure

The Problem
Imports for Web or Mobile specific files cause errors in IDEs and build time.
Example:

You have service that has platform specific implementations for wen and mobile in service.tns.ts and service.web.ts.
The service is imported in another module with import { xxx } from ".../service"
IDE (ex. VsCode) will (rightfully) complain that it cannot find the service module

The Solution
Configure typescript to resolve platform specific files (with .tns.ts/.web.ts extension) with priority over shared files (just *.ts). The configuration is done by setting paths in tsconfig. Here is how the tsconfig.tns.json (used for mobile) looks:
"compilerOptions": {
   //...
   "paths": {
     "@src/*": [
       "src/*.android.ts",
       "src/*.ios.ts",
       "src/*.tns.ts",
       "src/*"
     ]
   }
 }

Important Remarks

The common config contains mapping for both web and mobile so that paths can work and be resolved inside IDE (ex. VS Code) while developing.
Platform specific configs contain mapping only for the platform (see example above).
We map @src/ to the src folder of the project. Other symbols be used - like @proj/. However, don’t use “~/" - this leads to confusing build-time errors and suffering. The reason: “~/" is reserved and is already used as an alias in webpack configuration it is needed for CSS files resolution.
The “paths” mapping only work for non-relative imports. This means that all imports that should be resolved to a plat-specific file should be defined with as non-relative paths starting with @src. Our suggestion is to adopt a convention to use @src for all project files imports and enforce it with a tslint rule.
Sometimes when building for NativeScript there is a problem with AOT compiler picking up “*.ts” extension that are intended for web-only. It then fails with a component/module not used error. To solve that we explicitly specify files in tns and web ts-configurations. These are the entry points to TS compilations. All lazy modules should be specified in “files” list in tsconfigs, as they are not directly imported in other TS files.

Tasks

[x] Modify ng-new schematic to create apps using the ts-configuration described above.
[x] Create tslint rule that enforces non-relative imports. It should have an auto-fix option that replaces relative imports with the @src scheme
[x] There are a couple of places in schematics that we need to resolve imports. Resolve them using the ts-configurations so that they can work with @src and basePath mappings
[x] Make sure all schematics work and produce code with @src imports (instead of relative). We can just reuse the tslint rule. We can execute the rule with --fix to make it fix relative imports that we have generated

[x] ng-new
[x] add-ns
[x] shared
[x] generate component
[x] generate module
[x] master-detail template
[x] migrate-module
[x] migrate-component

[ ] Update the documentation

Fixes #172

test


RichTextUploadingField is not showing up

After the recent update (around 4 days ago), the RichTextUploadingField is not showing up.
The RichTextField works just fine, but with the exact same settings, just changing the model's field to RichTextUploadingField, it doesn't show up. Instead, Django's standard TextField appears.
I've tested this on both Linux and Windows.
I see no error in browser's developer console.

Does it work with django-ckeditor==6.4.0?

Good point. Thanks  Will update this shortly.
On Fri, Apr 15, 2022, 12:09 AM Matthias Kestenholz @.***>
wrote:

Does it work with django-ckeditor==6.4.0?
—
Reply to this email directly, view it on GitHub
https://github.com/django-ckeditor/django-ckeditor/issues/716#issuecomment-1099560916,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHSFM6LCFVJKU2PQQRQXQD3VFBX7FANCNFSM5TOU3GUA
.
You are receiving this because you authored the thread.Message ID:
@.***>

Same issue.   RichTextField work fine, but RichTextUploadingField show only basic textarea.   In browser debug console - no errors.
django-ckeditor==6.3.2
Django==4.0.4

It works fine with 6.4.0. Have you updated the package?

Yep, work fine after update to latest 6.4.0

Great, thanks.


Possibility to show pins on MapView (MKPointAnnotation)

Hi there,
first of all thanks for open sourcing this great library, it really made my day (and will likely make my month).
I would like to ask if there is yet a possibility to show markers / pins (MKPointAnnotation is the real name I guess)  on the map? If not and if you would like, I would be glad to work an a PR for this feature.
Regards and thanks for the great work!

+1

:+1:

Hey @DanielMSchmidt, there isn't any api for that so far... but please, if you want to work on it, go ahead! :)

Great, I will do so ;)

:+1:

@tadeuzagallo Would this syntax be okay?
<MapView pins={this.state.pins} />

It's good to me, although I'm not the best for naming... if someone is against it, it should be fairly easy to rename the property later, so I'd say go for it!

@DanielMSchmidt What's the latest on this? Anything I can do to help as I need it for my app?

The current status is, that it's possible to add pins to the map, but after some seconds the map freezes. I haven't updated any documentation yet, nor written any tests. As I am (unfortunately) on vacation, I won't have time to fix this, but I could and likely will write the documentation and some tests.
I would be really happy if you would take a look at the current status and fix the freezing. My hint would be, that I simply added the annotations at the wrong time of the maps lifecycle or in a method that gets called too often. I added you as collaborator to my fork and if you need an example implementation, see here. Thanks for your help!

Update: the code is working now, I will soon add the documentation and tests and then I will open a PR

Pins seem to be implemented as annotaions while I have been working on this PR, just FYI.

Any tips on how to change the design of the pins to incorporate say an image or a custom view?

@dominicthomas did you find out how to style custom markers/pins?

Does it freeze on the device too? I've been doing map work with RN and the simulator sucks compared to the device for maps

Yes, same problem on my device (iPhone 6).

Am 14.11.2015 um 00:37 schrieb Jay Garcia<EMAIL_ADDRESS>Does it freeze on the device too? I've been doing map work with RN and the simulator sucks compared to the device for maps
—
Reply to this email directly or view it on GitHub.

@seb0zz no not yet. I was about to start looking at creating a custom MkAnnotationView and extend the RN MapView to pass in an image URL either for a custom pin or an image in the custom annotation view. Will then need to try a similar approach on android.

:+1:  would be awesome to have custom annotations @dominicthomas

@GantMan Totally. to get going I've had to duplicate the RCTMap and RCTMapManager as well as the MapView js so I can make changes. Shouldn't be too hard to change the view type being rendered as annotation. I'll post back if I get anywhere.

Unfortunately, this is something that has to be done due to the patterns that react native uses.  :(
I documented this problem and have not gotten confirmation on if this is the best/only way to "extend" react native  components.
https://github.com/facebook/react-native/issues/3444

@dominicthomas and @jaygarcia I admire your willingness to jump in!  I'm focussed on an MVP right now, but if either of you completely patch RN to get this feature, I'd love to hear about the process.  I'm a fan of contributing to OSS and would be enthralled to hear all that's involved to get an extension like this.


Removed ambiguous TextAnnotatorPlugin export

Issue
I accidentally added an ambiguous/duplicated definition for the TextAnnotatorPlugin within the @recogito/react-text-annotator package. Now I would say that users should use the explicitly re-exported AnnotoriousPlugin component

However, that still might create confusion on why the TextAnnotator plugin is used for the plugins definition, but then still gets inserted into the AnnotoriousPlugin component 🤔
I can't think of a good name to draw a line between the TextAnnotatorPlugin and "text annotator plugin component". Like it's done for the Annotator/Annotorious


Button 3 and button 6 not detected

In latest version (3.19) I can't bind button 3. Previous version was fine. I have similar issue in CEMU.
My log: https://pastebin.com/r64Ht2Jb

Is it only Button 3 or is there any more buttons that does not work?

Button 6 also doesn't work. No, it's simple bluetooth gamepad with binary buttons and triggers, two analogs and 1 POV.

Can you please send me log from current and previous versions?

And I can see from the log that neither SCPToolkit nor ViGEm is working correctly on your computer. Make sure that you have everything installed including:

DirectX
Visual C++ Redistributable for Visual Studio 2015 32-bit
Visual C++ Redistributable for Visual Studio 2015 64-bit
all the drivers for your controllers
.Net Framework 4.5.2 if you do not have Windows 10
the official XBox 360 controller drivers.

XOutput_last_working_3.18.log
XOutput_not_working_3.19.1.log
I'm 100% sure that everything is correctly installed on my laptop. Latest versions (3.19.X) don't detect my gamepad correctly.

As you can see some buttons is not detected

Any chance to fix that bug ?

We will see, I will try for sure.

You can try 3.20 now, and if it still does not work then please send me a new log file, as I added more info.

XOutput.log
Still doesn't work. I have similar bug in CEMU, so maybe you use same library in your app ?

settings.txt

Can you confirm that you these are all working? Do you miss the button 13 as well?

6 axes

X
Y
Z
Z rotation
Accelerator
Brake

12 buttons

1
2
4
5
6
7
8
9
10
11
12
14
15

My gamepad doesn't have 14 and 15, and 13 too

Okay then I think I know where the problem is. I will revert the latest change, and fix this for you today.

Please try it in version 3.22.

It's working now ! Thanks. Can you tell me where issue was ?

The name and the number of the buttons are calculated from the of the ObjectId. They are usually in a continuous increasing order (like button 1, button 2, ...). In your case there are some missing ones (3, 6, 13).
SharpDX gives you an array of the button values (0-11), and I was indexing it with numbers (0,1,3,4,6,7,8,9,10,11,13,14). So the issue was, that I was indexing the array with the ObjectIds and not their actual position.
I hope it can be understood. but the change can be found here as well d4af27d.

Thanks a lot. I got your version from SourceForge and exact issue was encountered. Glad to see you fixed it.

hey I have the same issue with my right and left triggers (my triggers are working perfect without xoutput) can you fix mine too?

Which version are you using right now?
Been a while, I don't even have that controller to test now.

I think our controllers are not same. Mine is steelseries nimbus+. Im using v3.32 I think.


[VS Code] Joining collaboration session: Unable to connect to the host. Verify both you and the host are online.

Error:
Joining collaboration session: Unable to connect to the host. Verify both you and the host are online.
Steps to Reproduce:
1.
2.

Version Data

extensionName
VSLS

extensionVersion
1.0.509

protocolVersion
2.2

applicationName
VSCode

applicationVersion
1.36.1

platformName
Windows

platformVersion
10.0.17134

Hey @waseeld , is it possible that this was a valid error message? Were you eventually able to connect?

yes

Great! I'll close for now. If you hit something that looks like an issue with the product, or have suggestions on how we could improve the experience, please reopen or create a new issue.


Improve CICD

add test_matrix  for testing in paralell
rename to better names

rename gplugins.eme to gplugins.meow
rename gplugins.fem to gplugins.femwell
rename gplugins.gtidy3d to gplugins.tidy3d

enable dispersive materials in tidy3d by default

fixes #7

@momchil-flex
something in tidy3d seems to be failing now that I enabled dispersive materials by default

I get these errors when I try to run locally
2023-08-03 11:11:25.619 | INFO     | gdsfactory.component:_write_library:1968 - Wrote to '/tmp/gdsfactory/straight_length2.gds'
2023-08-03 11:11:25.621 | WARNING  | gdsfactory.klive:show:48 - Could not connect to klive server. Is klayout open and klive plugin installed?
____________________________________________________________________________________ test_sparameters_grating_coupler _____________________________________________________________________________________
gplugins/gtidy3d/tests/test_write_sparameters_grating_coupler.py:19: in test_sparameters_grating_coupler
    dfs = [
gplugins/gtidy3d/tests/test_write_sparameters_grating_coupler.py:20: in <listcomp>
    gt.write_sparameters_grating_coupler(
../../../miniconda3/envs/gdsfactory/lib/python3.10/site-packages/gplugins/gtidy3d/write_sparameters_grating_coupler.py:117: in write_sparameters_grating_coupler
    sim = get_simulation_grating_coupler(
../../../miniconda3/envs/gdsfactory/lib/python3.10/site-packages/gplugins/gtidy3d/get_simulation_grating_coupler.py:318: in get_simulation_grating_coupler
    medium=get_medium(name_or_index=clad_material_name_or_index),
../../../miniconda3/envs/gdsfactory/lib/python3.10/site-packages/gplugins/gtidy3d/materials.py:98: in get_medium
    elif name_or_index in material_name_to_medium:
E   TypeError: unhashable type: 'numpy.ndarray'
------------------------------------------------------------------------------------------ Captured stdout call -------------------------------------------------------------------------------------------
2023-08-03 11:11:25.642 | INFO     | gdsfactory.component:_write_library:1968 - Wrote to '/tmp/gdsfactory/grating_coupler_ellipti_3efea4b2.gds'
2023-08-03 11:11:25.658 | WARNING  | gdsfactory.klive:show:48 - Could not connect to klive server. Is klayout open and klive plugin installed?
========================================================================================= short test summary info =========================================================================================
FAILED gplugins/gtidy3d/tests/test_simulation.py::test_simulation - TypeError: unhashable type: 'numpy.ndarray'
FAILED gplugins/gtidy3d/tests/test_write_sparameters.py::test_sparameters_straight_3d - TypeError: unhashable type: 'numpy.ndarray'
FAILED gplugins/gtidy3d/tests/test_write_sparameters.py::test_sparameters_straight_2d - TypeError: unhashable type: 'numpy.ndarray'
FAILED gplugins/gtidy3d/tests/test_write_sparameters_grating_coupler.py::test_sparameters_grating_coupler - TypeError: unhashable type: 'numpy.ndarray'

Oh right sorry. By the way how do you develop locally? If I try running pytest gplugins/gtidy3d/tests/ it complains that I don't have gplugins installed. And I can't pip install -e ./, so I ended up just doing pip install ./, but then I have to redo it every time the code changes.

pip install -e . installs it in developer mode so you link the code and you only need to do it once

Yeah that's what I wanted to do but it told me
ERROR: File "setup.py" or "setup.cfg" not found. Directory cannot be installed in editable mode: /home/momchil/Drive/flexcompute/gplugins
(A "pyproject.toml" file was found, but editable mode currently requires a setuptools-based build.)

are you on python>=3.10 ?

Yeah python 3.10 works thanks.
Ok so I figured out what causes the issue, even though I don't fully understand it. But the underlying reason is that the mode monitor terminates exactly at the substrate on the z- direction:

For some reason, this causes it to error at one of the frequencies. I assume something to do with the very last pixel being different, and that's where we apply the PEC boundary condition. I found two possible solutions

move the mode solver plane a little or make it smaller in that direction so that it doesn't touch the substrate (or if you want to take the substrate into account, then expand it so that it covers more of it, but then you may have to also make sure you're looking at the right mode and not a substrate mode).
change this line to mode_spec=td.ModeSpec(num_modes=1, precision="double").

Ideally it's better to use the first approach throughout gdsfactory to make sure that mode planes are not exactly aligned with substrates. Similarly if the second approach is taken, it would probably be best to set double precision everywhere. Running this may be a little bit slower (and a bit more expensive) though.
We're also in the process of refactoring our remote mode solver. It may be that this will also fix this particular error.
By the way: regarding using dispersive models by default - they are also slower and more expensive to simulate. They also have a higher chance of diverging in some cases. Are you sure you want to start using those by default instead of making it easy for the user to toggle them on?

Thank you Momchil, the tests are passing now, i implemented both of your suggestions, thank you for your help
I also went back to non dispersive models by default

actually the tests are still failing remotely
not sure why they pass locally
https://github.com/gdsfactory/gplugins/actions/runs/5756010767/job/15604642415


Azure MFA using WHFB no longer supported

We were informed by MSFT support today that following these instructions when using Azure AD Connect even though it is fully functional is no longer supported. Please update the documentation to provide a warning to users wanting to use the on-premise model with Azure MFA using Azure AD Connect that they cannot use this method. Instead they must re-deploy with one of the Hybrid models.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 12c2ca3e-9d15-5a4a-971a-bc1eb3cb2637
Version Independent ID: ca1f6f75-8129-24d1-2426-21effa0ab151
Content: Validate and Deploy MFA for Windows Hello for Business with key trust - Microsoft 365 Security
Content Source: windows/security/identity-protection/hello-for-business/hello-key-trust-validate-deploy-mfa.md
Product: w10
GitHub Login: @mapalko
Microsoft Alias: mapalko

Thank you for this feedback. Do you still have the MS Support ticket number stored somewhere, in case any of the Microsoft Docs team members would like to ask for it later? (I don't need it, I just thought I should check, just in case.)
I also presume someone from the MS Docs team of tech writers will post questions at a later time.

( ref. https://github.com/MicrosoftDocs/windowsserverdocs/issues/5519 )

@illfated Yes the ticket number is<PHONE_NUMBER>006727

@e0i : FYI - maybe invite one of the tech writers or the page author to have a look at the report?

Under investigation. Thank you for the feedback.

@jgallucci32 Do you feel like the note at the beginning of the article is adressing this topic?

If not, what statement would you like to see added? Could you come up with an example?

@VLG17 No, that statement addresses a different topic. It addresses MFA Server which is different than the Microsoft Azure MFA extension to AD FS described in Microsoft and third-party additional authentication methods
A more appropriate statement would be something like this:

As of Month Date, Year , Microsoft will no longer support Microsoft Azure MFA as a third-party authentication method to AD FS for the on-premises deployment model when Active Directory is connected to Azure using Azure AD Connect. You must use one of the Hybrid deployment models for WHFB when Azure AD Connect is configured in your environment.

@mapalko Can you take a look? Thanks.

I am no longer the area owner for this docs section.

@paolomatarazzo

I don't think the existing article needs to be updated. It clearly calls out that it applies to on-premises deployment type and after the important note: "Windows Hello for Business requires all users perform multi-factor authentication prior to creating and registering a Windows Hello for Business credential. On-premises deployments can use certificates, third-party authentication providers for AD FS, or a custom authentication provider for AD FS as an on-premises MFA option.".
I'm not sure if I misinterpreted the ask, but it seems correct.

It's generally assumed if third-party and/or custom auth providers are supported then so would a first-party plugin. In this case the first-party MFA adapter for Azure is not supported, but other third party or custom ones are. This should be explicitly called out.
Most people purchasing Microsoft products expect them to integrate with each other first before non-microsoft products.
Sent from my iPhone

On Dec 6, 2022, at 10:30, Paolo Matarazzo @.***> wrote:

I don't think the existing article needs to be updated. It clearly calls out that it applies to on-premises deployment type and after the important note: "Windows Hello for Business requires all users perform multi-factor authentication prior to creating and registering a Windows Hello for Business credential. On-premises deployments can use certificates, third-party authentication providers for AD FS, or a custom authentication provider for AD FS as an on-premises MFA option.".
I'm not sure if I misinterpreted the ask, but it seems correct.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.

@jgallucci32 thank you for clarifying. However, I don't agree with your statement. The MFA adapter is not supported in the context of on-premises deployments. Please check the definition of on-premises deployment here: https://learn.microsoft.com/en-us/windows/security/identity-protection/hello-for-business/hello-how-it-works-technology#on-premises-deployment


the webpage https://github.com/osrf/vrx/wiki/propulsion_characteristics_tutorial is still empty

Describe the bug
As the title, the webpage https://github.com/osrf/vrx/wiki/propulsion_characteristics_tutorial is still empty.

Is it linked from any other page?

Removed in any case.


loadTranslations does not load the default namespace correctly if not specified in config

I have started using ni18n. I had the following ni18nConffig:
import type { Ni18nOptions } from 'ni18n'

export const ni18nConfig: Ni18nOptions = {
  supportedLngs: ['en', 'es'],
  ns: ['layout-default','specific-page'],
}

I understood that the first item is the default namespace and always gets loaded, and maybe this is were I was wrong.
In fact it worked ok until I used loadTranslations on getServerSideProps to load the specific-page namespace for a specific page. It loaded specific-page correctly, but not default-layout.
My getServerSideProps:
export const getServerSideProps = async (props) => {
  return {
    props: {
      ...(await loadTranslations(ni18nConfig, props.locale, ['specific-page'])),
    },
  }
}

I have solved it by including the defaultNS entry in ni18nConfig:
import type { Ni18nOptions } from 'ni18n'

export const ni18nConfig: Ni18nOptions = {
  supportedLngs: ['en', 'es'],
  ns: ['layout-default','specific-page'],
  defaultNS: 'layout-default'
}

Now it works correctly. default-layout gets loaded on every page and specific-page only on specific page.
Also question: when I use loadTranslations, is the default namespace loaded on server side rendering? I'm assuming it does, but I'm not sure as the default behavior is to load client-side.

Hi @Pryrios , some behaviour have changed in the last major version of i18next and I'm yet to take a proper look at it to know if any further changes to the documentation are necessary. But glad to hear you were able to solve your problem. :)
The loadTranslations function should make the defaultNS available as props to the server side renderer. If you check the network tab you will see that some .json payloads are loaded with the page, these json should have all the namespaces used when rendering on the server.
Let me know if you encounter any issue. 👍

Nothing else for now, thank you for your input and for your module!


Endless loop in IUpdateHandler, HandleErrorAsync

Here is code sample of IUpdateHandler implementation:
public class TelegramHandler : IUpdateHandler
    {
        public async Task HandleUpdateAsync(ITelegramBotClient botClient, Update update, CancellationToken cancellationToken)
        {
            try
            {
                //disable internet connection at this point

                if (update.Type == UpdateType.Message)
                    await botClient.SendTextMessageAsync(update.Message.Chat.Id, "test");
            }
            catch { }
        }

        public Task HandleErrorAsync(ITelegramBotClient botClient, Exception exception, CancellationToken cancellationToken)
        {
            //this will be called in endless loop, while internet is off
            Console.WriteLine("error");
            return Task.CompletedTask;
        }
    }

Now, do the following:

Put breakpoint at line if (update.Type == UpdateType.Message)
Send message to the bot to hit breakpoint
Disable your internet connection
Release breakpoint
Watch at console output and CPU load 😊

This happens because of call _botClient.MakeRequestAsync inside loop:

Which is going to exception handler:

Probably some workaround can be done by catching HttpRequestException and invoking Task.Delay(1000)

Polling loop in case of error has been changed in v21.3
Comment/reopen if you still see an issue


feat: visually hide search if less than five elements

Ensures we show the search only when necessary:

Keyboard interaction remains fully operational, including selecting + filtering of entries.

Updated so it can be incorporated independently.


Use self.logger.debug to replace print to make code clean

Description
Replace some print statements with self.logger.debug for better code consistency.
How Has This Been Tested?

[x] Tested in a local development environment

Checklist

[ ] The PR is tagged with proper labels (bug, enhancement, feature, documentation)
[x] I have performed a self-review of my own code
[ ] I have added unit tests that prove my fix is effective or that my feature works
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation

cc:

@wangxiaoyou1993

@wangxiaoyou1993 Please take a look at these minor refinement changes.


Database backup restore [RDO-214]

What

[x] change chart to use as source of database authentication information k8s secret
[x] split chart to dirs backup, semaphore, postgresql
[x] add database restore feature
[x] change values.yml structure to database, semaphore, backup, restore "primary keys"

@Teebor-Choka rebased, please take a look when possible.

Please, deploy to test that everything works properly.

Please, deploy to test that everything works properly.

tested, working


cs: (experimental) add support for IF and FOR

Idea:
// signature
func (cs *CS) IF(condition *Constraint, ifBlock func(), elseBlock ...func())

	// example:
	circuit.IF(condition, func() {
		circuit.IF(circuit.NOT(condition2), func() {
			// if block
			circuit.MUL(...)
		}, func() {
			// optional else block
			circuit.DIV(...)
		})
	})

        const maxLoops = 100
	for i := 0; i < maxLoops; i++ {
		circuit.IF(circuit.IS_SMALLER(myContraint, i), func() {
		})
                myConstraint = circuit.ADD(myConstraint, 1)
	}

Work in progress. Likely needs better syntax, may need new type of constraints /  "boolean predicates".

In the example that you gave: the IS_SMALLER(..) that would return a bool is actually difficult to write as a r1c (I didn't find how to do it at the time I was writing the cs.go file).
I suggest to keep things simple  at first step and write an
IF(c, func(..) []*constraints, func(..) []constraints) []*constraints
where c is a variable that has been boolean constrained. Writing the IF like this would be syntaxic sugar to avoid writing by hand the two paths and then the SELECT at the end.


feat: HugrView::extract_hugr to extract regions into owned hugrs.

Add a extract_hugr method to extract regions into owned Hugrs.
The implementation is short-circuited for Hugr to avoid unnecessary clones.
Closes #1171.

We have many ways of obtaining a hugr from a view (there's base_hugr already), so I prefer to keep the explicit method to disambiguate.


Support upload by dragging files to exercise name

@yammesicka can I handle this one?

Sure.

[ ] Be sure to have some feedback ondragover - something like the border goes dotted instead of solid.
[ ] The element should be updated on real time - no page refreshes.
[ ] It should be able only on the main page (and disabled on the 'archive' page)
[ ] If the upload fails for some reason, we should indicate it somehow to the user. Red border maybe?

The whole exercise area :)


Adding 200 response to LRO operation

Data Plane API - Pull Request
Adding 200 response to LRO operation that should return final resource value
API Info: The Basics
Most of the information about your service should be captured in the issue that serves as your engagement record.

Link to engagement record issue:

Is this review for (select one):

[ ] a private preview
[ ] a public preview
[x] GA release

Change Scope
This section will help us focus on the specific parts of your API that are new or have been modified. Please share a link to the design document for the new APIs, a link to the previous Open API document (swagger) if applicable, and the root paths that have been updated.

Design Document: n/a
Previous Open API Doc: https://github.com/Azure/azure-rest-api-specs/blob/main/specification/deviceupdate/data-plane/Microsoft.DeviceUpdate/stable/2022-10-01/deviceupdate.json
Updated paths:

❔Got questions? Need additional info?? We are here to help!

 Contact us!
The Azure API Review Board is dedicated to helping you create amazing APIs. You can read about our mission and learn more about our process on our wiki.

💬 Teams Channel
💌 email

Click here for links to tools, specs, guidelines & other good stuff
Tooling

Open API validation tools were run on this PR. Go here to see how to fix errors
Spectral Linting
Open API Hub

Guidelines & Specifications

Azure REST API Guidelines
OpenAPI Style Guidelines
Azure Breaking Change Policy

Helpful Links

Azure DevTools Wiki

Approving this breaking change as the 200 response is (currently) required by our client generators.


mui.com crash on Safari

Duplicates

[X] I have searched the existing issues

Latest version

[X] I have tested the latest version

Current behavior 😯
mui.com crashes on Safari
Expected behavior 🤔
mui.com works on Safari
Steps to reproduce 🕹
Steps:

open https://master--material-ui.netlify.app/
see error page

Context 🔦
Safari doesn't support Lookbehind in JS regular expressions which was introduced in https://github.com/mui-org/material-ui/pull/30107
https://github.com/mui-org/material-ui/blob/7767fb8f46abbe56f652bc59ed9637a8a1af9634/docs/src/modules/components/AppSearch.js#L308-L313
If I understand the intention, that part of the code is supposed to prefix url with material/ if it doesn't start with it already.
If so, I think lookbehind can be replaced with if condition like this:

I will submit a PR :)
Your environment 🌎

`npx @mui/envinfo`
  Safari 15.2, both desktop and mobile

Got reported on a tweet https://twitter.com/ludvigerikss0n/status/1473287869242429453 and 3 PMs

Just did a quick patch https://github.com/mui-org/material-ui-docs/commit/bf5628da28bc2321d97bd3347183e8b8c2c0214d so we have more time to fix it right in #30345

I'm adding the "bug" label back so we can find all the bugs by only searching with this label. The "regression" label is a sub-type of bugs (regressions are usually x2-x5 more important relative to the other bugs).


Enhancement: Cache directory

In ImageCache.java you are not using the android built-in getExternalCacheDir(), to retrieve a path to save the files.
Do you have a reason to do this?
Maybe it is better to use getExternalCacheDir(). It has some benefits like clearing this directory on uninstall and using the packagename within the path. Maybe you should check the docs on this: http://developer.android.com/reference/android/content/ContextWrapper.html#getExternalCacheDir()

After reading the docs again, maybe getExternalFilesDir() would be a better choice, because these files are kept private to your app only and are excluded from the mediascanner. If getExternalCacheDir() is used, you have to create a ".nomedia" file to exclude it from the mediascanner.

@OKA133, these are both good points. I've split them off into separate issues (#6 and #7 respectively). I'm on holidays for another week, but feel free to make a pull request for either one if you like.

BTW both of these issues are now fixed in edc6132. Next version will probably be 2.0; I need some time to consider the API changes before I make a release.


Regenerate due to upstream changes

This change:
https://github.com/ugorji/go/commit/54210f4e076c57f351166f0ed60e67d3fca57a36
Causes this during build:
src/github.com/coreos/go-etcd/etcd/response.generated.go:242:27: too many arguments in call to r.Dec
have ([]byte, bool, bool)
want ([]byte, bool)
src/github.com/coreos/go-etcd/etcd/response.generated.go:261:28: too many arguments in call to r.Dec
have ([]byte, bool, bool)
want ([]byte, bool)
src/github.com/coreos/go-etcd/etcd/response.generated.go:326:26: too many arguments in call to r.Dec
have ([]byte, bool, bool)
want ([]byte, bool)
src/github.com/coreos/go-etcd/etcd/response.generated.go:572:27: too many arguments in call to r.Dec
have ([]byte, bool, bool)
want ([]byte, bool)
src/github.com/coreos/go-etcd/etcd/response.generated.go:1066:27: too many arguments in call to r.De
have ([]byte, bool, bool)
want ([]byte, bool)
src/github.com/coreos/go-etcd/etcd/response.generated.go:1379:14: cannot assign 1 values to 2 variab
src/github.com/coreos/go-etcd/etcd/response.generated.go:1494:21: cannot assign 1 values to 2 variab
I've regenerated response.generated.go and now it compiles.  I did this to allow to compilation of gliderlabs/registrator: https://github.com/gliderlabs/registrator/issues/591

As the README says, this project has been deprecated since at least two years. If this is breaking any consumer, please migrate it to the official client instead.

This project is used by registrator. We'd like to make these changes - or at least add some doco for how to work around them.

I agree that this is the better solution. If there is a desire for "minimal change" then the polite thing to do is to help your users with a workaround. Please see the pull request here: https://github.com/coreos/go-etcd/pull/251


Body tracking distrib files not stamped with version numbers -- need version numbers

Critical redistrib files distributed with the new Body Tracking v1.1.0 are missing version numbers.
We need version numbers for building, cmake package control, runtime app version control, validation, etc.
Setup

Windows 10 x64
Download body tracking 1.1.0 from https://www.microsoft.com/en-us/download/details.aspx?id=102901

Repro

Install from the downloaded MSI, accept defaults.
cd  C:\Program Files\Azure Kinect Body Tracking SDK\tools
in windows explorer, right click on onnxruntime.dll
click on details tab

Result
onnxruntime.dll file version is <IP_ADDRESS>
onnxruntime_providers_tensorrt.dll has nothing
onnxruntime_providers_shared.dll has nothing
nvinfer.dll has nothing
etc...
Expected
The file versions and/or production versions to be something useful on all critical files. Naturally, these versions must also be available using the standard Windows version APIs like GetFileVersionInfo()
Pics

Updated list of files in OP having no version info or not useful (e.g. <IP_ADDRESS>) version info


set ENUM for authorized users

Change strings for username verification to ENUM.
What is ENUM?
An Enum is a set of symbolic names bound to unique values. They are similar to global variables, but they offer a more useful repr() , grouping, type-safety, and a few other features. They are most useful when you have a variable that can take one of a limited selection of values.
Lines 82/97/160 should need changing.

Resolved with latest push.


How incremental refresh is affected when re-publishing the dataset?

I would like to see some explicit information about the impact (or non impact) of re-publishing of dataset that has incremental refresh active. Are there any limitations? How are the changes in the dataset managed? Thank you very much.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: fdb026fd-1515-7484-1acd-3d363e143cc8
Version Independent ID: 62650e83-de5c-0323-cacc-31bad6e46361
Content: Incremental refresh in Power BI Premium - Power BI
Content Source: powerbi-docs/service-premium-incremental-refresh.md
Service: powerbi
Sub-service: powerbi-admin
GitHub Login: @mgblythe
Microsoft Alias: mblythe

Hi, @ruanonline -- thanks for your question! Assigning to the author to follow up.

I would like to see some clarification around re-publishing a dataset that has incremental refresh set.  It seems that if you make a change and re-publish then it will need another full refresh on the server which is not ideal
Would be good to have options similar to SSAS to Publish FULL or just PROCESS

This is incredibly annoying and our biggest pain point - we cannot use incremental and iteratively deliver new functionality within that data model to our business without publishing over the dataset with the subset of data and then manually kicking off full refresh again.  The functionality is ONLY usable and scalable for a static dataset now - which is not acceptable for an enterprise adoption.

Any plans/workarounds for this problem?

Thank you for your feedback. We've put this doc request into our queue, and will review it with other documentation requests.
Meanwhile, there are many community and support options for you to find answers to your questions. Getting an answer or feedback on your question from the community (https://community.powerbi.com) or from support at https://support.powerbi.com are options. You can also provide feedback using the Give Product Feedback button at the bottom of the article page.


Updated Greek messages.

Q
A

New feature?
no

Breaks BC?
no

Fixed issues
comma-separated list of tickets # fixed by the PR, if any

Translation update.

Σας ευχαριστώ :)

Παρακαλώ! :)


Travis CI build is failing

The Travis CI build is now failing after switching to npm for managing the frontend build.

Resolved by b8b71c124dca4508f033cc796ec44dcc05e86350


Do not modify serializer import path

It can save users from typing .serializers but breaks when you use a little bit different structure: app/serializers/<name>.py

Related to https://github.com/rsinger86/drf-flex-fields/pull/34

@rsinger86

Hi. What if we tried both ways and caught the exception for an import error? That might be better for keeping things compatible with existing codebases?

Ran up against this too in our fairly large api project, lots of non-standard import paths for serializers, IMO it should be left up to the developer to specify a fully qualified import path

Fixed in the latest release, thanks for the bump.


minor issue: missing a close paren ) in chocolateyInstall.ps1

A missing ) on:
Write-Host "You will need to restart your PowerShell session (or reload your profile using "". `    $PROFILE"""

Thanks Kory. Fixed in 17412ead97d02ea22432fc0e1e123079c8e2c1c6


A question about ElastoDyn

Dear,
What parameters should be modified in ElastoDyn if you want to set a tilt Angle for the wind turbine？

Dear @May0048,
The tilt angle of the shaft and rotor in OpenFAST are defined via ElastoDyn input ShftTilt.
Best regards,


feature request: add support for format file via autopep8/yapf

Are them supported by pyright itself?

Probably not, but I think it would be better if this plugin can support it. see microsoft/pyright#213

It’s an explicit non-goal of pyright to enforce any kind of style. I’m sure there are already yapf or autopep8 formatter plugins on package control.

Since LSP-pylsp supports formatters, I'd rather look into running them simultaneously and using LSP-pylsp for formatting only. With ability to disable capabilities it would hopefully be possible but I haven't tried.
Please create an issue in LSP-pylsp repo if you care. If we figure out the exact settings then we can add that to documentation.


eth_getTransactionCount returning 502 error

Description
while working on testnet and using the method: eth_getTransactionCount sometimes, randomly but often we are getting the following error:

<meta http-equiv="content-type" content="text/html;charset=utf-8">
<title>502 Server Error</title>

Error: Server Error

The server encountered a temporary error and could not complete your request.
Please try again in 30 seconds.

Steps to reproduce
Consume method eth_getTransactionCount many times until you get it.
Additional context
No response
Hedera network
testnet
Version
v0.25.0
Operating system
None

@AlfredoG87 thanks, I would just add that it affects more than the eth_getTransactionCount method specifically eth_sendRawTransaction(for contract deploys and txs) and is potentially related to hashgraph/hedera-json-rpc-relay/issues/1139

Hey @AlfredoG87 just came across paradigmxyz/flood and thought it might be useful for load testing the relay and ensuring no serious performance degradation occurs with new releases.

@mshakeg will take a look into it, Thank you!! 👍

@mshakeg
502 ServerError is thrown after timeouts due to a limitation of the used LoadBalancer/Gateway that will consume the actual error, and always throw a 502 instead of the actual status code and error message. This "bug" is only on HashIO Environments and not when running the relay instance locally.
Seems like with the recent rollout of some method optimizations and also load type normalized and methods stopped timing out, and throwing 502 errors.
During the exploration of the error timeout was moved from 30s to 60s but since load diminished and optimizations were released there is a very low timeouts above 30s and none above 60s. For that reason we might consider leaving it at 60s for a little longer while we add more optimizations that can further avoid getting timeouts.
Please could you confirm that the error is no longer happening so we can close this issue?

@AlfredoG87 thanks for the update. I can confirm that I no longer have this issue on testnet but that's likely because even large contract deployments take <60s, however, on mainnet those same contracts would take close to 2 minutes and so I would suspect that the issue(though not major as the contracts do eventually deploy successfully) would still exist on mainnet.

Hi @mshakeg  we believe the timeout issues are solved. I think we should close this ticket based on current statistics.
We have been doing a lot of optimizations to improve performance, tweaking some internal parameters, and also found and fixed a known issue of NodeJS that was responsible for delays when many simultaneous outgoing requests are made.

I will be closing this ticket but if you believe this is happening again, please let us know! 👍


Update README.md

[中文版模板 / Chinese template]
🤔 This is a ...

[ ] New feature
[ ] Bug fix
[ ] Site / documentation update
[ ] Demo update
[ ] Component style update
[ ] TypeScript definition update
[ ] Bundle size optimization
[ ] Performance optimization
[ ] Enhancement feature
[ ] Internationalization
[ ] Refactoring
[ ] Code style optimization
[ ] Test Case
[ ] Branch merge
[ ] Other (about what?)

🔗 Related issue link

💡 Background and solution

📝 Changelog

Language
Changelog

🇺🇸 English

🇨🇳 Chinese

☑ Self Check before Merge
⚠ Please check all items below before review. ⚠

[ ] Doc is updated/provided or not needed
[ ] Demo is updated/provided or not needed
[ ] TypeScript definition is updated/provided or not needed
[ ] Changelog is provided or not needed

Codecov Report

:exclamation: No coverage uploaded for pull request base (feature@744ddc4). Click here to learn what that means.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             feature        #1   +/-   ##
===========================================
  Coverage           ?   100.00%
===========================================
  Files              ?       394
  Lines              ?      7471
  Branches           ?      2103
===========================================
  Hits               ?      7471
  Misses             ?         0
  Partials           ?         0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 744ddc4...7e1042e. Read the comment docs.


fix(otlp-exporter): add support for signal-specific env vars

Which problem is this PR solving?
Make all otlp-related env vars work as expected.
Fixes #3193, #3747, #3748
Short description of the changes
Type of change
Please delete options that are not relevant.

[x] Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

[x] Existing tests

Checklist:

[x] Followed the style guidelines of this project
[x] Unit tests have been added
[x] Documentation has been updated

FIY I'm currently changing the way we're handling configuration in the exporters as part of fixing #3748

FIY I'm currently changing the way we're handling configuration in the exporters as part of fixing #3748

OK, I'll see what I can help after that.


update buildah image to v1.23.1

Signed-off-by: wanjunlei<EMAIL_ADDRESS>Changes

Submitter Checklist

[ ] Includes tests if functionality changed/was added
[ ] Includes docs if changes are user-facing
[x] Set a kind label on this PR
[x] Release notes block has been filled in, or marked NONE

See the contributor guide
for details on coding conventions, github and prow interactions, and the code review process.
/kind bug
Release Notes
Update Buildah to 1.13.1, picking up fix #containers/buildah#3126

May you also update https://github.com/shipwright-io/build/blob/main/samples/buildstrategy/source-to-image/buildstrategy_source-to-image-redhat_cr.yaml#L21 ?

Done

We are now back to the failure for which we changed to config to stay on the older release. @wanjunlei do you know a Buildah configuration that we need to set to make it behave like docker = to lookup from DockerHub without prompting?
[1] [1/2] STEP 1/4: FROM golang:1.16 AS build
[1] [2/2] STEP 1/4: FROM scratch
[1] error creating build container: short-name resolution enforced but cannot prompt without a TTY

We are now back to the failure for which we changed the strategies originally to stay on the older release. @wanjunlei do you know a Buildah configuration that we need to set to make it behave like docker = to lookup from DockerHub without prompting?
[1] [1/2] STEP 1/4: FROM golang:1.16 AS build
[1] [2/2] STEP 1/4: FROM scratch
[1] error creating build container: short-name resolution enforced but cannot prompt without a TTY

See https://github.com/containers/buildah/blob/main/docs/samples/registries.conf#L8-L14
That is the classic v1 format I believe.
There is even a "v2" for the format of that file.
For a practical example of how openshift builds sets up a default for /etc/containers/registries.conf, see https://github.com/openshift/builder/blob/master/imagecontent/etc/containers/registries.conf

For a practical example of how openshift builds sets up a default for /etc/containers/registries.conf, see https://github.com/openshift/builder/blob/master/imagecontent/etc/containers/registries.conf

Nice, so, we would maybe hard-code this to get it green. Probably just an echo to that file in the build strategy script. Assumes that we run as root there which I think is the case.
Eventually, we can make that setting configurable through a strategy defined param. Unqualified registries requires an array = we'll need the implementation of that ship (hoping to get this done soon) before we can do that.

e2e

I think it is more reasonable to define the registry config file with parameters. With this pr, I add three parameters and a prepare step to the buildah cluster build strategy to generate the registry config file.
  parameters:
    - description: The registries for searching short name images such as `golang:latest`, separated by commas.
      name: REGISTRIES_SEARCH
      default: docker.io,quay.io
    - description: The fully-qualified name of insecure registries. An insecure registry is one that does not have a valid SSL certificate or only does HTTP.
      name: REGISTRIES_INSECURE
      default: ""
    - description: The registries that need to block pull access.
      name: REGISTRIES_BLOCK
      default: ""

Adding suggestion for the e2e failure. Will we need the same params in the s2i strategy as well ?

I will add these params in the s2i strategy.
The e2e failure because of the build of buildah-golang-build-mutate timeout. Is there any way to locate the cause of the timeout?

Adding suggestion for the e2e failure. Will we need the same params in the s2i strategy as well ?

I will add these params in the s2i strategy.
The e2e failure because of the build of buildah-golang-build-mutate timeout. Is there any way to locate the cause of the timeout?

@HeavyWombat I know you just replaced the source-bundle to use quay instead of DockerHub to pull the golang image. Any idea why the build now seems to hang?

This time the mistake is here, My local test is successful, I think this is caused by insufficient resources.
https://github.com/shipwright-io/build/blob/main/test/e2e/e2e_bundle_test.go#L106
I checked the previous e2e test of pr and did not find the bundle test. Is bundle test newly added?

This time the mistake is here, My local test is successful, I think this is caused by insufficient resources. https://github.com/shipwright-io/build/blob/main/test/e2e/e2e_bundle_test.go#L106
I checked the previous e2e test of pr and did not find the bundle test. Is bundle test newly added?

The bundle test is in there since a couple of months ago.

Changes were picked up in a separate PR #959 to include additional fixes. Thanks again for your contribution @wanjunlei. We will soon modify the strategy again once array parameters are available. See #988 for details.


Add HTMLrev

Add HTMLrev to UI Libraries, Components & Templates section.

Name
Link

HTMLrev
Resource link here

HTMLrev provides free HTML templates for websites and landing pages.

[X] My item is in the right category
[X] My item is logically grouped below similar items
[X] My item's name and description respects the conventions of the list
[X] My item is awesome
[X] My item is in line with the Tailwind brand usage guidelines
[X] I have read and followed the contribution guidelines

I'm not sure about this. Some links are dead, some links lead to the same GitHub repository without linking a specific directory, most are not easily previewable.
I'd rather have the outstanding templates added directly to the list instead of linking to another list.


Feat: Block instance replication of itself and it's own replicas

WIP: A patch to block an instance from replicating itself and it's own replicas
Fixes: #1484

Thanks @uds5501, you need to fix the clang-tidy error: https://github.com/apache/incubator-kvrocks/actions/runs/5199313230/jobs/9376587287?pr=1488#step:8:1166 to pass the CI. And can you add some Go tests to make sure it works?

@git-hulk , added integration test and clang-tidy error

@git-hulk , added integration test and clang-tidy error

Thanks for your efforts

Thanks all, merging...

@uds5501 Thanks for your contribution.