Agent Unable to write node config header: interrupted system call

Is there an existing issue for this?

[X] I have searched the existing issues

Version
equal or higher than v1.16.0 and lower than v1.17.0
What happened?
Agent fails to regenerate endpoints due to failing to write node config header, which leads to all bpf compilations to fail.
How can we reproduce the issue?
Happens randomly, not easy to reproduce.
Cilium Version
Cilium 1.17.0-dev 8fa83406 2024-08-22T14:10:51+02:00 go version go1.23.1 linux/amd64
Kernel Version
Linux/x86 6.10.0-rc6
Kubernetes Version
N/A
Regression
No response
Sysdump
RuntimeAgentFQDNPolicies_With_verbose_policy_logs_Validates_DNSSEC_responses.zip
Relevant log output
time="2024-09-26T13:15:45.933969654Z" level=error msg="Unable to write node config header" error="failed to write node configuration file at /var/run/cilium/state/globals/node_config.h: rendering vlan filter macros: listing network interfaces: interrupted system call" subsys=datapath-loader
...
time="2024-09-26T13:15:46.100906287Z" level=error msg="Failed to compile bpf_lxc.o: exit status 1" compiler-pid=793 subsys=datapath-loader
time="2024-09-26T13:15:46.100944558Z" level=warning msg="In file included from /var/lib/cilium/bpf/bpf_lxc.c:18:" subsys=datapath-loader
time="2024-09-26T13:15:46.100961449Z" level=warning msg="In file included from /var/lib/cilium/bpf/lib/auth.h:6:" subsys=datapath-loader
time="2024-09-26T13:15:46.100969916Z" level=warning msg="In file included from /var/lib/cilium/bpf/lib/common.h:1279:" subsys=datapath-loader
time="2024-09-26T13:15:46.100978361Z" level=warning msg="In file included from /var/lib/cilium/bpf/lib/overloadable.h:10:" subsys=datapath-loader
time="2024-09-26T13:15:46.100987608Z" level=warning msg="In file included from /var/lib/cilium/bpf/lib/overloadable_skb.h:8:" subsys=datapath-loader
time="2024-09-26T13:15:46.100997066Z" level=warning msg="/var/lib/cilium/bpf/lib/clustermesh.h:18:23: error: use of undeclared identifier '__config_identity_length'" subsys=datapath-loader
time="2024-09-26T13:15:46.101011563Z" level=warning msg="   18 |         __u32 identity_len = CONFIG(identity_length);" subsys=datapath-loader
time="2024-09-26T13:15:46.101018676Z" level=warning msg="      |                              ^" subsys=datapath-loader
time="2024-09-26T13:15:46.101138079Z" level=warning msg="/var/lib/cilium/bpf/lib/static_data.h:75:10: note: expanded from macro 'CONFIG'" subsys=datapath-loader

Anything else?
No response
Cilium Users Document

[ ] Are you a user of Cilium? Please add yourself to the Users doc

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Happened again here: https://github.com/cilium/cilium/actions/runs/11067434628

Happened her again: https://github.com/cilium/cilium/actions/runs/11352893938

Hit again at https://github.com/cilium/cilium/actions/runs/11351409408


Prepare for iOS 10 Beta 2

We'll need screen shots for:

[ ] New iMessage Store
[ ] iMessage Sticker Templates
[ ] New layout for drawing in Messages
[ ] Activity Widget
[ ] Home Control Centre Widgets
[ ] Control Centre 3D Touch Actions
[ ] Music App

Did you/someone spotted any visual differences with the official release of iOS 10?


Differentiate between time spent on Spring related actions and the rest

Currently the time spent on Spring related actions is calculated as follows:
Spring time = total time - test time
This way, all the time spent on other things that are not necessarily Spring related is included in this time.

When starting a spring context with Spring Boot, there is a log output saying "XyContext: initialization completed in 4711 ms"
If we find this in the spring boot code, we might be able to extract this information the same way that Spring Boot does.


Remove static MimetypeHelper.

Fix issue #4
This will allow you to use whatever mimetype helper you want.

Ping @dbu

looks good. i seem to not have been subscribed to this repo, watching it now.

Thank you


Many missing tile warning. missing interiors and tiles.

when running i receive a lot of missing tile warning during the base and base_top steps. after the process completes the map has a number of black regions in both top and isometric. the black regions are primarily in interiors, water, and field tiles.
there is also missing images for some chunks of the most zoomed out regions.
missing tile: blends_street_01_85
missing tile: blends_street_01_80
missing tile: blends_street_01_87
missing tile: blends_street_01_80
missing tile: blends_street_01_87
missing tile: blends_grassoverlays_01_16
missing tile: blends_street_01_86
missing tile: blends_grassoverlays_01_16

Also, how much free memory do you have on your computer?
A typical 16 render threads setting will use about 16GB of memory. Tiles may not be able to load when you run out of memory.
You can follow the instructions on the readme file to use fewer render threads, and that will reduce the memory needed.

No missing textures or errors. i got that many jobs as well, which i assume corresponds to textures.
i have 64 GB of ram so thats not a problem

I just fixed a bug that may cause the missing tile error. Can you try the latest code and see if the problem still exists?

ok running base over again and its looking good so far, let you know how it goes. had to increase my ulimit to get it to run.

also i am running on linux. i converted some of the batch scripts to python.
would you be open to PRs to convert the project to pure python to be more portable?

also i am running on linux. i converted some of the batch scripts to python.
would you be open to PRs to convert the project to pure python to be more portable?

I'm OK with PRs. The batch scripts are barely passing parameters to main entries. It is possible to add similar bash scripts for Linux support. Also, be aware that some libraries behave differently between Linux and Windows. For example, the shared_memory in Windows does not need unlink() to fully release as the OS does the clean-up when the last process closes the handle, but not calling unlink() accordingly on Linux will lead to a memory leak.

looks like it worked.


Translate pt-BR o readme

Olá @diego3g, I would like to know if I can do the translation of the README to have an i18n improve! What do you think?

@alexandreramosdev Hi Alexandre, we want to keep all issues and contributions to this repo in english so can you please translate the original message here?
We have thoughts about i18n but nothing decided yet. Maybe we could create an docs website instead of using only Github because here we have really poor support for i18n as new markdowns would be created for every language.

I suggest create others docs translated on folder translated, and in the README link to the doc translated.

For docs website, maybe we can use Docusaurus, they allows easy translation and localization and the site can be host by github pages. Sound good?

@leonardobrito Sounds great. We are using Docosaurus here: http://docs.rocketseat.dev
Repository: https://github.com/rocketseat/docs

Nice @diego3g , how to proceed with that? We have to send a pull request to https://github.com/rocketseat/docs or unform will have his own repository/website of docs. Or is too early to do that. If possible i will like to contribute with that chore.

@leonardobrito I think Unform can have it's own documentation

So the @rocketseat will create another repository for his documentation? With a docusaurus project.

@leonardobrito Maybe will be better if we have a docs folder inside the same repo.

@diego3g The Rockeatseat have the styleguide for create website doc do unform! I like about think that create website with Docosaurus in folder docs inside this repor.

@alexandreramosdev i guess a folder named docs or documentation, like the example folder will be great. What do you folks think?

I have seen others repositories and docs folder would be great.
Inside that folder we can create a readme for each language

If is okay for all, i can publish an PR with it soon, the folder docs with Docusaurus project and the initial documentation.

This issue is being treated in #80.

O próprio Google, acredito que outros navegadores também permitam, que o usuário adicione uma extensão para tradução automática de outras línguas, como inglês para português brasileiro.
Basta acessar as configurações para adicionar a extensão.

Hey guys, we have decided now to keep our efforts on the Unform API, so the docs will keep in English. As the project grows, we are going to focus on i18n. As we finish it we will create new docs where you all can contribute (in any language). We really appreciate your efforts in making this happen 💜


Dropbox Smart Sync issue

Description
When accessing a Dropbox folder with "Online only" files, it will automaticaly change files to "Local", even if your Dropbox Smart Sync configuration is set to "Allow admin to manage this setting" (or something similar)
Steps To Reproduce

Set Smart Sync to  "Allow admin to manage this setting" https://ibb.co/jGfDHDC (screencap)
Go to any Dropbox folder with "Online Only" files

Expected behavior
"Online Only" files will automatically set to "Local"
Files Version
Versión: <IP_ADDRESS> OS Version: 10.0.22000.556
Windows Version
Windows 11 21H2 22000.556
Relevant Assets/Logs
debug_fulltrust.log
debug.log

Do you have the preview pane open in Files? Any difference if you close it?

I also was seeing this, but it wasn't happening right away. If I left a folder open, periodically I'd see files start changing to Local. It was almost like a periodic scan on the files was causing them to be opened/modified

Do you have the preview pane open in Files? Any difference if you close it?

Hi gave. How do I check that?

Sorry I think I misunderstood. Do the files change to Local when you select them or is it enough to open a folder and all files inside it are changed to Local?
The documents will change its status once you access the folder.

Do you happen to have the "file tags" feature enabled (under Files settings > Experimental)?

Is this an issue in the preview version? https://files.community/download/preview


Problem with continue training

When the training interrupted ,I choose keep to countinue to previous training . The training will be start at Epoch 67 ,but the error of training down from 1.0 rather than previous training .
log.log
[1119 20:35:29 @concurrency.py:36] Starting EnqueueThread QueueInput/input_queue ...
[1119 20:35:30 @monitor.py:251] Found existing JSON at train_log/alexnet-dorefa/stat.json, will append to it.
[1119 20:35:30 @monitor.py:262] Found training history from JSON, now starting from epoch number 67.
[1119 20:35:30 @base.py:209] Start Epoch 67 ...
[1119 21:10:27 @base.py:219] Epoch 67 (global_step 10000) finished, time:2097.31 sec.
[1119 21:10:33 @saver.py:82] Model saved to train_log/alexnet-dorefa/model-10000.
[1119 21:11:28 @monitor.py:363] QueueInput/queue_size: 50
[1119 21:11:29 @monitor.py:363] cost: 5.9455
[1119 21:11:29 @monitor.py:363] cross_entropy_loss: 5.8991
[1119 21:11:29 @monitor.py:363] param-summary/conv0/W-rms: 0.067988
[1119 21:11:29 @monitor.py:363] param-summary/conv1/W-rms: 0.040989
[1119 21:11:29 @monitor.py:363] param-summary/conv2/W-rms: 0.02956
[1119 21:11:29 @monitor.py:363] param-summary/conv3/W-rms: 0.034119
[1119 21:11:29 @monitor.py:363] param-summary/conv4/W-rms: 0.034149
[1119 21:11:29 @monitor.py:363] param-summary/fc0/W-rms: 0.014811
[1119 21:11:29 @monitor.py:363] param-summary/fc1/W-rms: 0.022189
[1119 21:11:29 @monitor.py:363] param-summary/fct/W-rms: 0.022158
[1119 21:11:29 @monitor.py:363] regularize_cost: 0.046381
[1119 21:11:29 @monitor.py:363] train-error-top1: 0.95903
[1119 21:11:29 @monitor.py:363] train-error-top5: 0.88012
[1119 21:11:29 @monitor.py:363] val-error-top1: 0.93398
[1119 21:11:29 @monitor.py:363] val-error-top5: 0.82844
[1119 21:11:29 @monitor.py:363] validation_cost: 5.5529
[1119 21:11:29 @group.py:42] Callbacks took 61.329 sec in total. InferenceRunner: 55.482sec
[1119 21:11:29 @base.py:209] Start Epoch 68 ...
[1119 21:46:23 @base.py:219] Epoch 68 (global_step 20000) finished, time:2093.86 sec.
[1119 21:46:30 @saver.py:82] Model saved to train_log/alexnet-dorefa/model-20000.
[1119 21:47:24 @monitor.py:363] QueueInput/queue_size: 50
thanks a lot.

You start with the epoch number but didn't load the previous model. The example has a --load option.

I try load model-xxx.data-xx   or model-xx.index but failed. Should i tranform them into .npy and load it?

"--load model-xxx"    is ok, don't need postfix.

Thanks . It's does't work .
information is as following  @dongzhuoyao
Traceback (most recent call last):
File "alexnet-dorefa.py", line 306, in
config.session_init = SaverRestore(args.load)
File "/home/wa/.local/lib/python2.7/site-packages/tensorpack/tfutils/sessinit.py", line 106, in init
model_path = get_checkpoint_path(model_path)
File "/home/wa/.local/lib/python2.7/site-packages/tensorpack/tfutils/varmanip.py", line 167, in get_checkpoint_path
assert tf.gfile.Exists(model_path) or tf.gfile.Exists(model_path + '.index'), model_path
AssertionError: /train_log/alexnet-dorefa/model-660000
(tensorflow) wangchao@haoyu-Precision-Tower-7910:~/tensorflow/BNN/DoReFa-Net$ python alexnet-dorefa.py --dorefa 1,32,32 --load /train_log/alexnet-dorefa/model-660000 --data /resources/data/ILSVRC2012/images/ --gpu 1

check whether or not this file "/train_log/alexnet-dorefa/model-660000.index"  exists.

yes .  File "/train_log/alexnet-dorefa/model-660000.index" exists.

@erdollar
Are you aware that "/train_log/alexnet-dorefa/model-660000.index" and "./train_log/alexnet-dorefa/model-660000.index" are two different paths?

wuuwu,I realized where I was wrong. I realize that is the absolute path and the relative path difference.Thanks a lot.


Revert "Make _initialize and static _initializers private (#3895)"

This reverts commit 338e1b1ce49f307f37f91876643bed400e687c6d.
An internal subclass of ReactiveElement still needs to be able to delay calling _initialize until hydration, long after construction.
FYI @justinfagnani

This won't land in 3.0 but it should be landable in a few months, so whenever we do the next breaking change it'll qualify.


Unnecessary parsing of IRIs every time an RDF parser is used

Every time a parser is used AbstractRDFParser.resolveURI() uses new ParsedIRI() to determine if the IRI is relative. This is an expensive operation and has a noticeable overhead when many IRIs need to be created. A simpler check such as indexOf(':') will suffice to determine if something is a relative IRI.

Sorry for that. It seems I only ran "test" instead of "verify". I've modified the test to match the new behaviour, which I believe is very much expected since the IRI instance is created directly via SimpleValueFactory. The previous behaviour was a side effect of passing the string through new ParsedIRI(string).toString() before calling the value factory.
PS I can't seem to run the full integration suite. Tests that rely on HTTP fail with java.lang.NoSuchMethodError: org.springframework.beans.factory.config.ConfigurableListableBeanFactory.hasEmbeddedValueResolver()Z. I'm not sure why that is.


feat: check for headless in runtime

Check for headless environment at runtime instead of compile-time.
If we rely on #if UNITY_SERVER then we don't properly detect headless mode when I start the game like this:
MyGame.exe -batchmode -nographics

Also, the #if hides compiler errors. We might delete a using statement and not realize we broke headless builds.  This has happened to me a lot in Cubica with other defines.

I think this PR was from before our discord round yesterday?
so we can probably close?


Added Vector ds

Have you read the Contributing Guidelines on Pull Requests?
Yes
Description
Vector is one of the most commonly used data structure from the C++ STL library. But many people are not aware how it works. The implementation has all the functions like push_back(), pop_back(), front() etc.
Checklist

[x] I've read the contribution guidelines.
[x] I've checked the issue list before deciding what to submit.
[x] I've edited the README.md and link to my code.

Related Issues or Pull Requests
#1447

[x] Menu added for users
[x] Prompted users to enter values in the vector
[x] If i entered is an index out of bounds then, we cannot predict the output.


Use IterateBatchedSend on azure queue storage output

Hi!
I've made this very tiny change to use IterateBatchedSend on Azure Queue Storage output

Looks good, thanks @mfamador


Add the latest Twilio SDK version

I replaced the custom Twilio SDK build for ARM that you guys used to have with the latest Twilio SDK build from https://www.twilio.com/docs/api/ip-messaging/changelogs/android.
About my employer (One University Network): we are building educational Android app, and chat is an important part of it. We believe in and want to use Twilio IP Messaging as a chat solution for us.

@taydakov Thanks for the contribution! But, as this is a library version bump I'll have to add those changes myself. I'll introduce the new version as soon as possible. Let us know if you need something else regarding your Twilio Powered chat application.

@taydakov I have updated the app. Now it uses the latest SDK version. Thanks!


fix(cli): allow passing n_ctx=0 to openAI API server CLI arguments

What?
This PR updates the OpenAI API server's command line arguments to allow passing --n_ctx 0 as a command-line argument. Currently, --n_ctx parameter has a minimum of 1: in llama_cpp/server/settings.py you can see the following context CLI parameter configuration: n_ctx: int = Field(default=2048, ge=1, description="The context size.")
Why?
As of<EMAIL_ADDRESS>per #1015 by @DanieleMorotti, passing n_ctx=0 to the LLama class in llama_cpp/llama.py automatically sets the n_ctx to the model's n_ctx_train paramter from KV, and also updates the model's n_batch to min(n_ctx, n_batch). This is intentional to allow llama-cpp-python to infer the context size from the GGUF model file's KV parameters.
However, when this change was made, the OpenAI API server's CLI argument configuration was not updated, so the minimum value for the option remained at 1 - making the patch in #1015 unavailable to users of the OpenAI API serve.r

@K-Mistele thank you!


DWIConvert: Nrrd -> FSL -> Nrrd != Nrrd

Using the most recent commit as well as 3 others, I find that this transformation changes 'space thicknesses' and the last axis of 'space origin' of a given nrrd DWI.
I've attached a summary:
DWIConvertTest.pdf
(dwinrrd is the original nrrd and dwifslnrrd is the one generated via nrrd->fsl->nrrd)

DWIVisualizationFixing.pptx
@reckbo  We did a lot of testing last year to ensure that the results were consistent across several data types for FSL and NRRD.  https://github.com/BRAINSia/BRAINSTools/tree/master/DWIConvert/TestSuite
I can't claim that all the bugs are gone, but we did identify several results that were wrong.
We did the nrrd->fsl->nrrd conversions as well.  It is not possible to retain some orientation informations through that conversion process because fsl format is not as flexible as nrrd.

@Hui-Xie
Hui,  Please monitor this thread to ensure that DWIConvert is working reliably for new data types.
Hans

I think that this issue was addressed in a previous message.
We did the nrrd->fsl->nrrd conversions as well. It is not possible to retain some orientation informations through that conversion process because fsl format is not as flexible as nrrd.


Yikes! Ask timed out on [ActorSelection[Anchor(akka://kafka-manager-system/), Path(/user/kafka-manager)]] after [5000 ms]

The following error will happen at log file when  open topic link or others. could you help me? thanks.

[error] k.m.ApiError - error : Ask timed out on [ActorSelection[Anchor(akka://kafka-manager-system/), Path(/user/kafka-manager)]] after [5000 ms]
akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://kafka-manager-system/), Path(/user/kafka-manager)]] after [5000 ms]
at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:334) ~[com.typesafe.akka.akka-actor_2.11-2.3.10.jar:na]
at akka.actor.Scheduler$$anon$7.run(Scheduler.scala:117) ~[com.typesafe.akka.akka-actor_2.11-2.3.10.jar:na]
at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:599) ~[org.scala-lang.scala-library-2.11.7.jar:na]
at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109) ~[org.scala-lang.scala-library-2.11.7.jar:na]
at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:597) ~[org.scala-lang.scala-library-2.11.7.jar:na]
[error] k.m.ApiError - error : Ask timed out on [ActorSelection[Anchor(akka://kafka-manager-system/), Path(/user/kafka-manager)]] after [5000 ms]
akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://kafka-manager-system/), Path(/user/kafka-manager)]] after [5000 ms]
at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:334) ~[com.typesafe.akka.akka-actor_2.11-2.3.10.jar:na]
at akka.actor.Scheduler$$anon$7.run(Scheduler.scala:117) ~[com.typesafe.akka.akka-actor_2.11-2.3.10.jar:na]
at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:599) ~[org.scala-lang.scala-library-2.11.7.jar:na]
at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109) ~[org.scala-lang.scala-library-2.11.7.jar:na]
at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:597) ~[org.scala-lang.scala-library-2.11.7.jar:na]

Wondering if your issues is similar to mine: https://github.com/yahoo/kafka-manager/issues/143
I fixed it by enabling Active Offset Cache thats radiobox option when modifying clusters

thanks. It works by enabling Active offset Cache

Hello,How to enabling Active offset Cache?

How to enabling Active offset Cache @josephfrancis ,please help me

I'm seeing the same issue even with Active offset cache enabled. I have the same config as @josephfrancis.
Any thoughts?

Is it kafka manager can be used for existing kafka cluster?@josephfrancis.

@Jonathan-Wei As I see it yes, it can only be used for an existing cluster, as it lacks the options to create a new one.
Unless I misunderstood the question.

Thank you! @LeePorte   I can use it now!

Thanks @josephfrancis

Not working for me. I still see the same error when I click topic list or consumer even enabled the Active offset Cache.

[error] k.m.ApiError$ - error : Ask timed out on [ActorSelection[Anchor(akka://kafka-manager-system/), Path(/user/kafka-manager)]] after [5000 ms]
akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://kafka-manager-system/), Path(/user/kafka-manager)]] after [5000 ms]
at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:334) ~[com.typesafe.akka.akka-actor_2.11-2.3.14.jar:na]
at akka.actor.Scheduler$$anon$7.run(Scheduler.scala:117) ~[com.typesafe.akka.akka-actor_2.11-2.3.14.jar:na]
at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:599) ~[org.scala-lang.scala-library-2.11.7.jar:na]
at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109) ~[org.scala-lang.scala-library-2.11.7.jar:na]
at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:597) ~[org.scala-lang.scala-library-2.11.7.jar:na]
at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(Scheduler.scala:467) ~[com.typesafe.akka.akka-actor_2.11-2.3.14.jar:na]
at akka.actor.LightArrayRevolverScheduler$$anon$8.executeBucket$1(Scheduler.scala:419) ~[com.typesafe.akka.akka-actor_2.11-2.3.14.jar:na]
at akka.actor.LightArrayRevolverScheduler$$anon$8.nextTick(Scheduler.scala:423) ~[com.typesafe.akka.akka-actor_2.11-2.3.14.jar:na]
at akka.actor.LightArrayRevolverScheduler$$anon$8.run(Scheduler.scala:375) ~[com.typesafe.akka.akka-actor_2.11-2.3.14.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_72]

Seeing this as well.

I have the same error
Yikes! Ask timed out on [ActorSelection[Anchor(akka://kafka-manager-system/), Path(/user/kafka-manager)]] after [5000 ms]
Kafka Manager - <IP_ADDRESS>
Apache Kafka <IP_ADDRESS>, <IP_ADDRESS>
Zookeeper 3.4.6, 3.4.8
OS - RHEL 6.7
Starup string - ./kafka-manager -Dconfig.file="/opt/kafka-manager/conf/application.conf" -Dhttp.port=9001 -Dapplication.home="/opt/kafka-manager/" -java-home /opt/java/jdk8
My issue is very strange. I have 3 hosts (MBP OS X 10.11.3, VM1 RHEL 6.7, VM2 RHEL 6.7) and all of them has been deployed with the same kafka-manager-<IP_ADDRESS>.zip file. All instances of Kafka Manager are connected to the same zookeeper instance. RHEL 6.7 was deployed with the same VM template and differ only IPv4 address. I'm facing issue only with VM1 RHEL 6.7 but two others (OS X and VM2 RHEL 6.7) works without any errors.
Please help me to define steps to debug the issue.
Exception:
[error] k.m.ApiError$ - error : Ask timed out on [ActorSelection[Anchor(akka://kafka-manager-system/), Path(/user/kafka-manager/kdc_cluster_ss1/kafka-state)]] after [2000 ms]
akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://kafka-manager-system/), Path(/user/kafka-manager/kdc_cluster_ss1/kafka-state)]] after [2000 ms]
at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:334) ~[com.typesafe.akka.akka-actor_2.11-2.3.14.jar:na]
at akka.actor.Scheduler$$anon$7.run(Scheduler.scala:117) ~[com.typesafe.akka.akka-actor_2.11-2.3.14.jar:na]
at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:599) ~[org.scala-lang.scala-library-2.11.7.jar:na]
at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109) ~[org.scala-lang.scala-library-2.11.7.jar:na]
at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:597) ~[org.scala-lang.scala-library-2.11.7.jar:na]
at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(Scheduler.scala:467) ~[com.typesafe.akka.akka-actor_2.11-2.3.14.jar:na]
at akka.actor.LightArrayRevolverScheduler$$anon$8.executeBucket$1(Scheduler.scala:419) ~[com.typesafe.akka.akka-actor_2.11-2.3.14.jar:na]
at akka.actor.LightArrayRevolverScheduler$$anon$8.nextTick(Scheduler.scala:423) ~[com.typesafe.akka.akka-actor_2.11-2.3.14.jar:na]
at akka.actor.LightArrayRevolverScheduler$$anon$8.run(Scheduler.scala:375) ~[com.typesafe.akka.akka-actor_2.11-2.3.14.jar:na]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_74]

I have cloned the VM2 RHEL 6.7 and reconfigured it same as VM1 RHEL 6.7 and all Kafka Manager works without any exceptions.

the same problem persists with me, none of the above does not help to solve it.
error occurs when you try to view the consumers.
launched on vagrant with apashe-kafka quickstart
anyone can tell what I'm doing wrong?

I have an identical setup for preproduction and production. At preproduction everything is working as expected, but when I use the link for Broker List or Topic List at production, the following error message shows up.
2016-03-04 08:24:54,917 - [ERROR] - from kafka.manager.ApiError$ in pool-1-thread-1
error : Ask timed out on [ActorSelection[Anchor(akka://kafka-manager-system/), Path(/user/kafka-manager)]] after [5000 ms]
akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://kafka-manager-system/), Path(/user/kafka-manager)]] after [5000 ms]
	at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:334) ~[com.typesafe.akka.akka-actor_2.11-2.3.14.jar:na]
	at akka.actor.Scheduler$$anon$7.run(Scheduler.scala:117) ~[com.typesafe.akka.akka-actor_2.11-2.3.14.jar:na]
	at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:599) ~[org.scala-lang.scala-library-2.11.7.jar:na]
	at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109) ~[org.scala-lang.scala-library-2.11.7.jar:na]
	at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:597) ~[org.scala-lang.scala-library-2.11.7.jar:na]
	at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(Scheduler.scala:467) ~[com.typesafe.akka.akka-actor_2.11-2.3.14.jar:na]
	at akka.actor.LightArrayRevolverScheduler$$anon$8.executeBucket$1(Scheduler.scala:419) ~[com.typesafe.akka.akka-actor_2.11-2.3.14.jar:na]
	at akka.actor.LightArrayRevolverScheduler$$anon$8.nextTick(Scheduler.scala:423) ~[com.typesafe.akka.akka-actor_2.11-2.3.14.jar:na]
	at akka.actor.LightArrayRevolverScheduler$$anon$8.run(Scheduler.scala:375) ~[com.typesafe.akka.akka-actor_2.11-2.3.14.jar:na]
	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_72-internal]

The only difference is that the link for Broker List will show the brokers without the metrics for bytes and messages etc and the link for Topic List will show a error message.
Yikes! Ask timed out on [ActorSelection[Anchor(akka://kafka-manager-system/), Path(/user/kafka-manager)]] after [5000 ms]

Is it possible that these timeouts are related to the size of the Kafka topics? This could be an explanation why this is working for preproduction, but not for production.
I am using the following software versions:
Kafka: <IP_ADDRESS>
Kafka-Manager: <IP_ADDRESS>
ZooKeeper: 3.4.5

This is weird. Forget what I said. I just moved Kafka-Manager to another EC2 instance and now everything is working as expected.

+1

could it be that kafka-manager.zkhosts= is not correctly set in conf/application.conf?

It was correct in my case.

+1, v <IP_ADDRESS>

It seems that once an "ask timeout" occoured to a cluster, this cluster will always be "ask timeout" (other cluster is still available), unless I restart the KM.
Simple way to duplicate it:

open KMhost/clusters/{cluster}/consumers/{group_id}/topic/{topic_name}/type/KF
press F5 and hold on for a few seconds

The cluster returns "ask timeout", and never will it availbale again unless I restart KM.
Am I mistaking something?

+1, similar issue here. v<IP_ADDRESS>-1
It works fine for a while, but eventually it seems to hit this problem and never recover. I have to restart the service for it to start working again, and then it works fine right away.
[ESC[31merrorESC[0m] k.m.ApiError$ - error : Ask timed out on [ActorSelection[Anchor(akka://kafka-manager-system/), Path(/user/kafka-manager)]] after [5000 ms]
akka.pattern.AskTimeoutException: Ask timed out on [ActorSelection[Anchor(akka://kafka-manager-system/), Path(/user/kafka-manager)]] after [5000 ms]
        at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:334) ~[com.typesafe.akka.akka-actor_2.11-2.3.14.jar:na]
        at akka.actor.Scheduler$$anon$7.run(Scheduler.scala:117) ~[com.typesafe.akka.akka-actor_2.11-2.3.14.jar:na]
        at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:599) ~[org.scala-lang.scala-library-2.11.7.jar:na]
        at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109) ~[org.scala-lang.scala-library-2.11.7.jar:na]
        at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:597) ~[org.scala-lang.scala-library-2.11.7.jar:na]
        at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(Scheduler.scala:467) ~[com.typesafe.akka.akka-actor_2.11-2.3.14.jar:na]
        at akka.actor.LightArrayRevolverScheduler$$anon$8.executeBucket$1(Scheduler.scala:419) ~[com.typesafe.akka.akka-actor_2.11-2.3.14.jar:na]
        at akka.actor.LightArrayRevolverScheduler$$anon$8.nextTick(Scheduler.scala:423) ~[com.typesafe.akka.akka-actor_2.11-2.3.14.jar:na]
        at akka.actor.LightArrayRevolverScheduler$$anon$8.run(Scheduler.scala:375) ~[com.typesafe.akka.akka-actor_2.11-2.3.14.jar:na]
        at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]

+1, reproduced in <IP_ADDRESS>
it seems to hit this problem and never recover. I have to restart the service for it to start working again, and then it works fine right away.

My suggest is to check the network conditions to zk and kafka cluster.

Yeah, same here. I have Active Offset Cache enabled and sometimes I get this timeout issue, which is only restored after a restart of the service.

i used to get same error, i did not have zookeeper host configured correctly.
there 2 ways you can set this either by command line
kafka-manager -Dkafka-manager.zkhosts="localhost:2181" -Dhttp.port=9999
or
via application.conf property file.
kafka-manager.zkhosts="kafka-manager-zookeeper:2181" # this is default value, change it to point to zk instance.

Still getting this, works fine for days, maybe even a week, but they you will hit it and it will never recover.....

i get the same error,just add offsetCacheThreadPoolSize this parameter  2 to 5

@mosesyou you are correct after increasing offsetCacheThreadPoolSize parameter 2 to 5. this issue fixed.
+1

I get the 5000 sec timeout pretty frequently and it never recovers unless kafka manager process is restarted.
in Zookeeper logs I see error corresponding by time to the first timeout:
2017-01-16 17:35:51,923 [myid:2] - INFO  [ProcessThread(sid:2 cport:-1)::PrepRequestProcessor@649] - Got user-level KeeperException when processing sessionid:0x59a827ff1d0000 type:create cxid:0x18 zxid:0x10000008f txntype:-1 reqpath:n/a Error Path:/kafka-manager/mutex/locks Error:KeeperErrorCode = NoNode for /kafka-manager/mutex/locks
I understand the path /kafka-manager/mutex/locks in zookeeper doesn't exist, is that normal?
Does kafka manager handles that or it's a symptom of the issue?

Use Future[PartitionOffsetsCapture]] in KafkaStateActor.scala send OffsetRequest to periodicity get last offset from broker, but the action possible timeout lead to the bug.
set a timeout parameter in 'val partitionOffsets: Option[PartitionOffsetsCapture] = Await.ready()' codes in the getTopicPartitionIdentity in ActorModel.scala to fixed the bug, note that this Await.ready is used three times in ActorModel.scala file.

i am using both <IP_ADDRESS> and kafka-manager-<IP_ADDRESS> compiled version from latest master branch. i  haven't found any issue related to configuration. now kafka 10 supported on kafka-manager-<IP_ADDRESS> version.

I had run in the same issue and all of the workarounds above did not worked for me. The strange thing is that same compiled code is working on OS X but not working on Ubuntu. I've double/triple/quadruple... checked configuration ¯(ツ)/¯. What could be the reason for facing with this problem.

@meinac Please show your log.

I had disabled the Poll Consumer Information and Enable Active OffsetCache options and it worked.

Please try <IP_ADDRESS>, i fixed the blocking call for producer offset polling which was causing timeout.

Restarting kafka-manager resolved my issue.
Below link will help
Starting the service
$ bin/kafka-manager
By default, it will choose port 9000. This is overridable, as is the location of the configuration file. For example:
$ bin/kafka-manager -Dconfig.file=/path/to/application.conf -Dhttp.port=8080
Again, if java is not in your path, or you need to run against a different version of java, add the -java-home option as follows:
$ bin/kafka-manager -java-home /usr/local/oracle-java-8
URL for refer : https://github.com/yahoo/kafka-manager

+1
And restart does not work sometimes. Still have no clue about how to configure to avoid this error.

i get the same error,just add offsetCacheThreadPoolSize this parameter 8 to 16,then fixed.

Had the same isssue
Restarted many times and did all the config above
Just changed the default port from 9000 to run with this option
$ bin/kafka-manager -Dconfig.file=/path/to/application.conf -Dhttp.port=8080
it works now

All those solutions imply on kafka manager or the ThreadPools being restarted. This is not a the solution. We have latest version here with kafka <IP_ADDRESS> and the problem still occurs. Maybe increase the timeout time from 5000 to 10000 would solve this. The size of the kafka cluster and topics quantity matter for this problem to occurs.
If dont know scala very well. If I do, I would already started to fix this.

Also make sure the port of jmx is reachable to kafka-manager, otherwise it will not work

Please try the pull request https://github.com/yahoo/kafka-manager/pull/456

We have built kafka-manager with #456 and still seeing the issue of timeout when trying to an ZK quorum of an existing Kafka cluster with large number of topics/partitions. Any other pointers to resolve this issue?

@bijugs get topic offsets is not related to zk, Can you provide more details?

@DavidLiuXh .. here are the details of the issue we are facing. Let me know if you see any issues with the set-up or suggestions to fix.

Kafka-Manager is able to talk to a multi-node Kafka/ZooKeeper cluster set-up on the node where it is running from. Kafka-Manager is set-up to use the local ZK Quorum.
Trying to add a remote cluster to Kafka-Manager and the cluster ZK host property is set to zk.host1.com of the target cluster's 3 node ZK Quorum which is reachable from the machine where Kafka-Manager is running. This was verified by running telnet zk.host1.com 2181
When we navigate to the cluster view the following response is displayed

Yikes! Ask timed out on [ActorSelection[Anchor(akka://kafka-manager-system/), Path(/user/kafka-manager/DNJ2-Kakfa/kafka-state)]] after [2000 ms]

In the Kafka-Manager log we see this warning

[warn] o.a.z.ClientCnxn - Session 0x4d5ea507bdfb52c4 for server zk.host1.com/nnn.nnn.nnn.nnn:2181, unexpected error, closing socket connection and attempting reconnect
java.io.IOException: Unreasonable length = 1227014

@bijugs try  use  ip to replace zk hostname and monitor your zk

i am still at a loss as to what the issue is, I have never seen any consumer data here, ever. I tried all combinations of settings etc.

I came across this issue while connecting the KM to this image I made recently: https://github.com/nicovillanueva/docker-kafka
When working locally on my Mac, the KM works fine (well, mostly, I think; still didn't properly test the image). It even picks up JMX metrics of brokers.
But, when replicating my setup on a Vagrant environment or on a Marathon (on DC/OS), this error comes up.
It'd be reasonable to say that I'm running this within a Docker container: https://github.com/nicovillanueva/kafka-manager-docker

Make sure you have zookeeper service up and without any problem and then you have to run kafka-manger with the correct value at zkHosts configuration variable. Until I ran this command, it was not running for me, after that everything was ok
helm install stable/kafka-manager --set zkHosts=six-kafka-zookeeper:2181 --name six-kafka-manager
remember, the service six-kafka-zookeeper:2181 has to be up and running well

Hi All,
I just solved it. If you have extra strings other than zk host and port you will face this issue

The reason was different here. In my case, the ZK that I used when I set kafka-manager didn't exist anymore...that's why it was timing out.

Kafka-manager requires a zookeper to function
I faced this issue as kafka-manager was not able to reach to the zookeeper provided in the conf file.

In my case I can't even get the list of clusters to change ZK settings for one cluster that fails. Circle closed...

I had to update zookeeper hosts in the cluster and that fixed this problem.
Ping every host given in Cluster Zookeeper environment  and fix those which are not up.

in my case, it's worked to edit application.conf by changing from <IP_ADDRESS> to <IP_ADDRESS>.

Make sure that the zookeeper configurations set in the application.conf file works correctly.
If the cluster configuration set in the application.conf file is broken you will get the above "Yikes!" error on all of your attempts to add new clusters via the web UI.
I tried all of the above fixes and they didn't help me.
Once I setup a working application.conf file I was able to add new clusters via the UI.

I have the same problem.
When I set the Cluster ZooKeeper Hosts with its local IP such as "<IP_ADDRESS>:2181" and without any other extra string, the problem was resolved. I think the lack of a step by step "getting started" documentation causes to have this issue for many of us.

I have had this problem and I solved it
Make sure your zookeeper string while creating on Kafka Cluster page should be zk_host:2181 that zookeeper should have all relevant nodes like /config/topics etc..

For me this was the resolution: https://github.com/yahoo/CMAK/issues/485#issuecomment-374899619
I was trying to start cmak on a different zkroot path and was getting the same error.

I had this problem before, I has it resolved after changing my zookeeper host configuration to the one I created the container with.
I used a docker compose file
which had this line:
environment:
ZK_HOSTS: "zookeeper:2181"
but i was setting different host in the cluster i created, everything worked well after I used same and correct zhost setting

after 5 months I got the same problem again, seeing the comment I dropped months ago and didn't even see I wrote it... My comment saved me again

This problem can be solved by:

make sure zookeeper on the server is running
Make sure kafka on the server is running

If everything has been executed then execute the cmak command, it will work, good luck


[Bug] App crashes when trying to open a manga externally

Device information

Tachiyomi version: <IP_ADDRESS>
Android phone & version: OP6-A10

Steps to reproduce

Add a manga to library
Enable hide library entries from browse (top right ...)
Then open the manga externally using Neko

Expected behavior
Manga Info page will be opened
Actual behavior
App Crashes
Other details
If i re-enable Show Library entries, it works fine

<IP_ADDRESS>


fix case where multiple http_addr

with patch applied...
Notice: /Stage[main]/Consul::Config/File[/etc/init.d/consul]/content:
--- /etc/init.d/consul	2019-01-30 11:59:11.133195366 -0800
+++ /tmp/puppet-file20190213-127228-1atiisu	2019-02-13 10:57:09.024947000 -0800
@@ -17,7 +17,7 @@
CONFIG=/etc/consul
PID_FILE=/var/run/consul/consul.pid
LOG_FILE=/var/log/consul
-HTTP_ADDR=-http-addr=<IP_ADDRESS> <IP_ADDRESS>:8500
+HTTP_ADDR=-http-addr=<IP_ADDRESS>:8500
[ -e /etc/sysconfig/consul ] && . /etc/sysconfig/consul

Hi @robmbrooks, aren't multiple ip addresses allowed here? If so, should we instead enforce it in the datatype of the parameter instead of choosing always the first one? Can you also have a look at the used email address in your commits? It isn't associated with your github account.

aren't multiple ip addresses allowed here?

within config.json yes, but used within the init script they don't work

$ service consul status
/etc/init.d/consul: line 20: <IP_ADDRESS>:8500: command not found

the following content in the init script is the issue:

HTTP_ADDR=-http-addr=<IP_ADDRESS> <IP_ADDRESS>:8500

which is used here...

"$CONSUL" info ${HTTP_ADDR}

given both addresses are listening I chose the first,
I can fix my email address if it is an issue, it just happens I committed from work.

edit my commit email and force pushed

@bastelfreak can you review when you have a chance

@solarkennedy I'm fine with this. Can you take a look as well?

I'm fine with it as well.
It is kinda related to
https://github.com/solarkennedy/puppet-consul/pull/406
In fact, it is kinda a regression. @robmbrooks can you look at the above PR and see if you can solve it in a general way like that, instead of only in the systemd file?

@solarkennedy I will adjust the PR to fix all the distro init/upstart scripts. Ironically systemd is the only one that needs no adjustment because systemd does not rely on a scripted status and instead relies on systemd process monitoring.

@solarkennedy hopefully this approach is more tasteful, and it copies the approach taken for multiple $config_hash_real['addresses']['http'] values.

wfm


Fix websocket ping random failed on macos

@CodiumAI-Agent /review

PR Analysis

🎯 Main theme: Fixing random test failures in websocket ping on macOS
📝 PR summary: This PR aims to fix random test failures in websocket ping on macOS by waiting for the http server to start before running the tests. It also modifies the test assertions to include error messages for better debugging.
📌 Type of PR: Bug fix
🧪 Relevant tests added: No
⏱ Estimated effort to review [1-5]: 2, because the changes are straightforward and limited to a single test file.
🔒 Security concerns: No security concerns found

PR Feedback
💡 General suggestions: The changes in this PR seem to be addressing the issue at hand. However, it would be beneficial to add more context in the PR description about why these changes are necessary and how they solve the problem. Also, it would be helpful to add tests that specifically check the new changes.

 ✨ Usage guide:
Overview:
The review tool scans the PR code changes, and generates a PR review. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on any PR.
When commenting, to edit configurations related to the review tool (pr_reviewer section), use the following template:
/review --pr_reviewer.some_config1=... --pr_reviewer.some_config2=...

With a configuration file, use the following template:
[pr_reviewer]
some_config1=...
some_config2=...

  Utilizing extra instructions
The review tool can be configured with extra instructions, which can be used to guide the model to a feedback tailored to the needs of your project.
Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Specify the relevant sub-tool, and the relevant aspects of the PR that you want to emphasize.
Examples for extra instructions:
[pr_reviewer] # /review #
extra_instructions="""
In the code feedback section, emphasize the following:
- Does the code logic cover relevant edge cases?
- Is the code logic clear and easy to understand?
- Is the code logic efficient?
...
"""

Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.

  How to enable\disable automation

When you first install PR-Agent app, the default mode for the review tool is:

pr_commands = ["/review", ...]

meaning the review tool will run automatically on every PR, with the default configuration.
Edit this field to enable/disable the tool, or to change the used configurations

  About the 'Code feedback' section
The review tool provides several type of feedbacks, one of them is code suggestions.
If you are interested only in the code suggestions, it is recommended to use the improve feature instead, since it dedicated only to code suggestions, and usually gives better results.
Use the review tool if you want to get a more comprehensive feedback, which includes code suggestions as well.

  Auto-labels
The review tool can auto-generate two specific types of labels for a PR:

a possible security issue label, that detects possible security issues (enable_review_labels_security flag)
a Review effort [1-5]: x label, where x is the estimated effort to review the PR (enable_review_labels_effort flag)

  Extra sub-tools
The review tool provides a collection of possible feedbacks about a PR.
It is recommended to review the possible options, and choose the ones relevant for your use case.
Some of the feature that are disabled by default are quite useful, and should be considered for enabling. For example:
require_score_review, require_soc2_review, enable_review_labels_effort, and more.

  More PR-Agent commands

To invoke the PR-Agent, add a comment using one of the following commands:

/review: Request a review of your Pull Request.
/describe: Update the PR title and description based on the contents of the PR.
/improve [--extended]: Suggest code improvements. Extended mode provides a higher quality feedback.
/ask <QUESTION>: Ask a question about the PR.
/update_changelog: Update the changelog based on the PR's contents.
/add_docs 💎: Generate docstring for new components introduced in the PR.
/generate_labels 💎: Generate labels for the PR based on the PR's contents.
/analyze 💎: Automatically analyzes the PR, and presents changes walkthrough for each component.

See the tools guide for more details.
To list the possible configuration parameters, add a /config comment.

See the review usage page for a comprehensive guide on using this tool.

Persistent review updated to latest commit https://github.com/samanhappy/easeprobe/commit/ecf255a7aedd2f7f6d0a99fa70eaffce6528f60c

@CodiumAI-Agent /review

Persistent review updated to latest commit https://github.com/samanhappy/easeprobe/commit/7b2f63cb9b06e6f8a6e58db128ff3f4d7b2b0711

Persistent review updated to latest commit https://github.com/samanhappy/easeprobe/commit/7b2f63cb9b06e6f8a6e58db128ff3f4d7b2b0711


BLE Host is only able to receive one packet per interval

I currently have two pico ws, one configured as a HOG Host, and one configured as a HOG Device, using code very similar to the hog keyboard hog host examples.
If i connect the device over bluetooth from a PC, i can get to poll rate around 850hz or so, which suggests that multiple packets are being sent per connection interval. However, if i instead connect to the HOG host running on a pico w, i end up with a poll rate of around 32hz, which lines up with the default 30ms connection interval.
If I then add the following line to the HOG host, I can lower the connection interval to the minimum 7.5ms, and I do end up seeing a poll rate of around 130hz or so, which also lines up with the 7.5ms connection interval.
gap_set_connection_parameters(0x0060, 0x0030, 0x06, 0x06, 0, 0x0048, 2, 0x0030);
I also ran BTStack locally on my PC using the libusb port, and found that that did get to around 850hz as well, so it does appear to just be an issue with the pico w port specifically.
My eventual goal is to use this to allow a user to create a usb adapter for a bluetooth controller i am working on that can  then be plugged into a console, hence the need for a very low latency.

You'll have to provide more context about the code you're running. What method are you using polling, background or are you using an rtos? Do you have any code that demonstrates a problem. Note that we're not actually bluetooth experts here so your report is assuming lots of prior knowledge. I don't know what "interval" you're referring to for example. Unless you can clarify what you're reporting in a way I can investigate a problem you're unlikely to get any help.

Ok, thanks for adding some context. Examples (or modifications to existing examples) would be useful. In the background mode, we get an interrupt via a gpio to tell us there might be work to do (cyw43_gpio_irq_handler). This ends up in a call to cyw43_poll_func. We ask the chip if there's any work to do (cyw43_ll_bt_has_work) and call cyw43_bluetooth_hci_process which signals that a btstack data source has work to do. BTstack calls hci_transport_cyw43_process which keeps reading hci commands. So if the chip thinks there's nothing to do or there are no hci commands to read we won't try and "receive" anything. I would avoid debugging via logging as it's likely to affect timings. Probably best to use a logic analyser to work out what's going on.

@peterharperuk if there are more than one ACL packet stored in the cyw43 ring buffer, will all of them get processed in a row until it's empty? or does this require another interrrupt via the GPIO line?

@mringwal I think it should yes. hci_transport_cyw43_process keeps reading hci commands until there's no data (or an error). It's possible this loop is faulty somehow. It should perhaps keep going while cyw43_ll_bt_has_work is true. It might be worth debugging this area (without using printf).

@sanjay900 Could you run gatt_streamer_server on one pico and le_streamer_client on the other. le_streamer_client should be configured to use TEST_MODE_ENABLE_NOTIFICATIONS - so it's only receiving. As the demo prints the throughput, we can check how many packets are sent per interval.
I still don't see how only a single packet could get received in a connection interval. The sender in peripheral role will send as long as the central allows it to send and the central will keep the connection event open until it's buffers overflow. event with controller to host flow control, we should get a least a few packets per conn interval.

Let me just build  both of those and i will give it a try

Hmm, doesn't seem to be working right, here are the logs from the le_streamer_client
BTstack up and running on 43:43:A2:12:1F:AC.
Start scanning!
Stop scan. Connect to device with addr 43:43:A2:12:1F:AC.
Connection Interval: 30.00 ms
Connection Latency: 4
Search for LE Streamer service.
SERVICE_QUERY_RESULT, ATT Error 0x7f.
Disconnected 43:43:A2:12:1F:AC
Start scanning!
Stop scan. Connect to device with addr 43:43:A2:12:1F:AC.
Connection Interval: 30.00 ms
Connection Latency: 4
Search for LE Streamer service.

And from the gatt_streamer_server:
BTstack up and running on 43:43:A2:12:1F:AC.
To start the streaming, please run the le_streamer_client example on other device, or use some GATT Explorer, e.g. LightBlue, BLExplr.
A: ATT connected, handle 0x0040, test data len 20
- LE Connection 0x0040: connected - connection interval 30.00 ms, latency 4
- LE Connection 0x0040: request 15 ms connection interval
A: ATT MTU = 1688 => use test data of len 200
- LE Connection 0x0040: connection update - connection interval 15.00 ms, latency 0

@peterharperuk did anybody complain about all pico w using the same BD ADDR? In @sanjay900's logs, both have 43:43:A2:12:1F:AC - the first 6 hex digits indicate the used controller, a 4343A2.
I don't remember exactly, but BTstack avoids connections to devices with the same BD ADDR, as that allows for some sneaky security hacks.
@sanjay900 Thanks for trying. I can check myself tomorrow or Friday.

It does however seem like something is broken with updating the connection parameters, as commenting out gap_request_connection_parameter_update(con_handle, 12, 12, 0, 0x0048); is enough to make it run. I actually found this myself too when i was looking for ways to increase my connection interval.
BTstack up and running on 43:43:A2:12:1F:AC.
To start the streaming, please run the le_streamer_client example on other device, or use some GATT Explorer, e.g. LightBlue, BLExplr.
 In @sanjay900's logs, both have 43:43:A2:12:1F:AC - the first 6 hex digits indicate the used controller, a 4343A2.
I don't remember exactly, but BTstack avoids connections to devices with the same BD ADDR, as that allows for some sneaky security hacks.
@sanjay900 Thanks for trying. I can check myself tomorrow or Friday.

@peterharperuk did anybody complain about all pico w using the same BD ADDR? In @sanjay900's logs, both have 43:43:A2:12:1F:AC - the first 6 hex digits indicate the used controller, a 4343A2.
I don't remember exactly, but BTstack avoids connections to devices with the same BD ADDR, as that allows for some sneaky security hacks.
@sanjay900 Thanks for trying. I can check myself tomorrow or Friday.

I know there are fixes for that in the develop branch, and I also had that thought myself but i ran it with the develop branch and still had issues.

Yes 43:43:A2:12:1F:AC is the default btstack address. I have a fix on develop that makes sure the btstack address is +1 the wifi mac address.
However I still haven't got to the bottom of why this issue is happening. I was unsure if it was because the OTP had not been set in the factory or if we had devices with duplicate addresses. @sanjay900 I would appreciate it if I could see the lines that say something like this for both the devices you have...
Version: 7.95.61 (abcd531 CY) CRC: 4528a809 Date: Wed 2023-01-11 10:29:38 PST Ucode Ver: 1043.2169 FWID 01-7afb9
cyw43 loaded ok, mac 28:cd:c1:07:86:f7
BT FW download, version = CYW4343A2_<IP_ADDRESS>1.0000_Generic_SDIO_37MHz_wlbga_BU_dl_signed
BTstack up and running on 28:CD:C1:07:86:F8.

Yes 43:43:A2:12:1F:AC is the default btstack address (hardcoded in the infineon firmware to always be the same). I have a fix on develop that makes sure the btstack address is +1 the wifi mac address.
However I still haven't got to the bottom of why this issue is happening. I was unsure if it was because the OTP had not been set in the factory or if we had devices with duplicate addresses. @sanjay900 I would appreciate it if I could see the lines that say something like this for both the devices you have...
Version: 7.95.61 (abcd531 CY) CRC: 4528a809 Date: Wed 2023-01-11 10:29:38 PST Ucode Ver: 1043.2169 FWID 01-7afb9
cyw43 loaded ok, mac 28:cd:c1:07:86:f7
BT FW download, version = CYW4343A2_<IP_ADDRESS>1.0000_Generic_SDIO_37MHz_wlbga_BU_dl_signed
BTstack up and running on 28:CD:C1:07:86:F8.

what defines do i need to set to make those lines show up?

@sanjay900 good idea to skip the conn param update. With 30 ms and about 6.5 kB/s, we end up with 200 bytes per connection interval, which would indicate a single packet with 200 bytes (if DLE is supported and I think it is).

Start scanning!
Stop scan. Connect to device with addr 00:15:83:ED:65:E8.
Connection Interval: 30.00 ms
Connection Latency: 4
Search for LE Streamer service.
SERVICE_QUERY_RESULT, ATT Error 0x1f.
Disconnected 00:15:83:ED:65:E8
Start scanning!
Stop scan. Connect to device with addr 00:15:83:ED:65:E8.
Connection Interval: 30.00 ms
Connection Latency: 4
Search for LE Streamer service.
Search for LE Streamer RX characteristic.
Search for LE Streamer TX characteristic.
A: ATT MTU = 1691 => use test data of len 200
Start streaming - enable notify on test characteristic.
Notifications enabled, ATT status 0x00
A: 16800 bytes -> 5.586 kB/s
A: 18400 bytes -> 6.131 kB/s
A: 18600 bytes -> 6.200 kB/s
A: 18600 bytes -> 6.154 kB/s
A: 19400 bytes -> 6.449 kB/s
A: 19000 bytes -> 6.333 kB/s
A: 19000 bytes -> 6.333 kB/s
A: 19200 bytes -> 6.400 kB/s
A: 19400 bytes -> 6.402 kB/s
A: 19400 bytes -> 6.466 kB/s
A: 18600 bytes -> 6.200 kB/s
A: 19000 bytes -> 6.285 kB/s
A: 20000 bytes -> 6.648 kB/s
A: 18800 bytes -> 6.266 kB/s
A: 19200 bytes -> 6.351 kB/s
A: 18400 bytes -> 6.119 kB/s
A: 19200 bytes -> 6.400 kB/s
A: 19600 bytes -> 6.533 kB/s

Interestingly, even when running the gatt_streamer_server on my pc with the libusb port, I still needed to comment out the conn param line for the connection to work

@peterharperuk not sure if you still need this but heres that line from the other pico:
Version: 7.95.61 (abcd531 CY) CRC: 4528a809 Date: Wed 2023-01-11 10:29:38 PST Ucode Ver: 1043.2169 FWID 01-7afb0879
cyw43 loaded ok, mac 28:cd:c1:07:b8:11
BT FW download, version = CYW4343A2_<IP_ADDRESS>1.0000_Generic_SDIO_37MHz_wlbga_BU_dl_signed
BTstack up and running on 43:43:A2:12:1F:AC.
To start the streaming, please run the le_streamer_client example on other device, or use some GATT Explorer, e.g. LightBlue, BLExplr.

If there is anything else i can provide let me know

@mringwal Out of curiosity, any idea why the conn param update breaks things?

Well, the controllers seem to fail in synchronizing the change. You can try to use slave latency = 4 with the connection interval = 15 ms, too.

Minor update: I've repeated the test with le_streamer_client on Pico W and gatt_streamer_server on my Mac with an USB CSR8510 dongle and as able to get a air trace using our Raccoon nRF5x sniffer.
I get the same (low) throughput. Interestingly, as the CSR8510 does not support DLE, the 200 bytes test packets are split into several smaller packets over the air. Still, only one of these packets are send per connection interval. The packets indicate that the peripheral (CSR8510) always has data to send, but the central (pico_w) does not/cannot receive it. While not impossible, I cannot imagine that the CYW43 can only receive this 200 bytes per connection interval. Quick test replacing the pico_w/cyw43439 with a CYW20704 on my Mac, I get 25 kB/s as well.
My hypothesis here is that they get stuck in the CYW43 on the Pico W. Will review the code a bit more to see if we can more details on this.

Thanks for looking into it :)

To get deeper, I've reviewed the code (no issues there) and toggled a GPIO while in a) hci_transport_cyw43_process, b) cyw43_bluetooth_hci_read, and c) the processing in BTstack. Please see attached traces.
When testing against a CYW20704, I did get 66 kB/s when sending 200 byte packets. With lower packet sizes, the throughput goes down. One fact is that the CYW20704 has 8 buffers for LE ACL packets. If we receive them faster than we can read the with the RP2040, this could be a bottleneck.
@peterharperuk : in the traces for smaller packet sizes, hci_transport_cyw43_process is entered, but cyw43_bluetooth_hci_read seems to report no data to read. I don't know what to make of that.
@sanjay900 : what's your project requirements? Is it pico<->pico only? You can 'play' with the connection interval and the packet size (e.g. sending multiple reports in one packet if you don't need to comply to a standard). You could also try to use Bluetooth Classic HID to see if that works better for your case.
I still don't know what's really going on. One idea would be to ask Infineon if they can increase the number of LE ACL buffers (right not 8 x 251 ~ 2 kB are used and the Controller should have way more memory). e.g. 16 buffers? :0) this might help. I've also tried shorted connection intervals and in one session just got broken links while in another, it worked as expected with the same hardware on my table.
traces.zip

@mringwal If i can i would like to stick with something HID based, as my goal is for my game controller to still work with a PC normally, and then for the pico <-> pico bit to be only necessary if a user would like to use  their controller on a game console.

@sanjay900 In that case, I would suggest to try the Classic HID examples :)

Ill give that a go again tomorrow and see if i can work out a way to get them working at a decent speed.

it seems going to classic HID cuts the poll rate to around 374hz with a pc

Well, that's one report every 3 ms.  isn't that good enough? :)
you can compare to same code with libusb.

See it would be if BLE wasn't giving me 850hz haha

Is it possible to ask infineon if they have any idea whats going on / if they can increase the number of LE ACL buffers?

Hi
What about this: the problem only happens when receiving on the Pico W. If you receive HOG from your PC it's fine, right?
In this case, you could provide a separate mode for Pico W <-> Pico W communication. Instead of sending a single report in single packet, you could add a second 'Burst HOG Service' where multiple reports are send in a single packet (up to 250 bytes..). By that, you should be able to send even more reports per second than previously.
best
Matthias

On 29 Apr 2023, at 10:23, Sanjay Govind @.***> wrote:
Is it possible to ask infineon if they have any idea whats going on / if they can increase the number of LE ACL buffers?
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: @.***>

ill see if i can figure out how to do that and give it a go, are there any examples of doing anything remotely similar to this?

nothing that contains the logic, but the gatt_streamer_server and le_streamer_client show how to send data as fast as possible. You only need the notifications on the server side.
For your use case, you can follow le streamer and try to check if the streamer service is available. If yes, you use that. If not, you use the existing hog client as before.

On 29 Apr 2023, at 11:30, Sanjay Govind @.***> wrote:
ill see if i can figure out how to do that and give it a go, are there any examples of doing anything remotely similar to this?
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: @.***>

Is it possible to ask infineon if they have any idea whats going on / if they can increase the number of LE ACL buffers?

If someone can summarise the problem and question we need to ask, I can send something to Infineon.

Ok, let me try.
When trying to send many small packets (e.g. HID reports over GATT), the CYW43439 receives less than expected - the connection interval isn't fully used even when using the minimal connection interval of 7.5 ms. An air trace shows that CYW43439 in Central role ends the connection event.
Our hypothesis here is that we cannot get the packets out of the Controller via the shared bus fast enough which causes all incoming ACL buffers to get full. When sending larger data, we can achieve optimal throughput. From HCI log, it looks like there are current 8 ACL LE buffers of 251 bytes. Is there any RAM left to increase the numbers of buffers, e.g. to double the numbers of buffers to validate this hypothesis?

@mringwal  thanks for that
@peterharperuk thank you, have you had a chance to send any comms to infineon yet?

Thank you

Would you guys be able to test this updated firmware? https://github.com/peterharperuk/cyw43-driver/tree/Updated_BLE_ACL_Config
Please just use this for testing. It seems faster to me but I'm not sure how to test for your use case. You will have to go into pico-sdk/lib/cyw43-driver and checkout the Updated_BLE_ACL_Config branch from my repo. In debug on boot it should say the following...
BT FW download, version = CYW4343A2_<IP_ADDRESS>8.0000_Generic_SDIO_37MHz_wlbga_BU_dl_signed + Updated BLE ACL Config
They say it "has  host_claim_host_to_device_count,  host_to_device_count and device_to_host_count updated value to 16 (from default value 8)."

I used this version of the firmware but unfortunately it does not appear to have fixed the problem.

@sanjay900 Could you provide some measurements? If you're using the HOG device/host: how man HID reports are sent/received per connection interval? how many bytes are in one report?

@mringwal Out of curiosity, any idea why the conn param update breaks things?

Well, I might have now. BTstack was using a max ce length of 0xffff and while nobody really knows what this value is used for, it looks like 0xffff causes a disconnect, while all other values are ok.
Could you update to the current BTstack develop branch (which uses max ce length = 0) and verify that the conn param update works now?

@sanjay900 Could you provide some measurements? If you're using the HOG device/host: how man HID reports are sent/received per connection interval? how many bytes are in one report? Would you have a packet log with the test firmware?

With HOG device/host, it is still just one packet per interval. I don't really have a easy way to get a packet log, but there are 27 bytes per report.

With HOG device/host, it is still just one packet per interval. I don't really have a easy way to get a packet log, but there are 27 bytes per report.

Thanks, that's a start. Which connection interval is this? And how many packets do you get per conn interval on the PC with the same connection interval?

@mringwal Can confirm, using the latest develop branch did fix the conn param update works now. With that, i set my transmitter to request a 7.5ms interval with a latency of 0.
For testing, i have just been emulating a gamepad, and assigning a random value to the joystick, and this then ends up sending out a new report as often as possible.
For HOG device/host, i end up with a average poll rate of 129hz or so, which ends up being close to around 1 packet per 7.5ms interval
For HOG device/pc, i end up with an average poll rate of around 750hz, which ends up being just over 5 and a half packets per interval

Did you check if 7.5 ms connection interval was used in both cases?

Yeah, i have the firmware printing out the connection params

@mringwal were there possibly any settings in btstack_config.h that i should play around with with the new firmware?

@sanjay900 No, there aren't. I'd need to get an air trace to confirm that it's still the central side that slows down, then the question is why. I was busy with other stuff though.

@peterharperuk should this be in 1.5.1 milestone?

@mringwal have you had a chance to grab an air trace? Sorry that i can't help there i don't have the tools for that myself

@kilograham No - it shouldn't be on 1.5.1

@sanjay900 Sorry, nope, was busy otherwise.
How urgent is this for your project? Could you use a custom solution, e.g. sending multiple reports in a single notification to get full throughput if the connection is between two Pico Ws for the time being.
It's unclear what exactly is causing this (stack, interface to controller, controller itself, radio link). The air trace might narrow this a bit down.

Yeah for the moment I'll do something custom, was just curious if you'd had
any time to look into it or not
On Tue, 13 Jun 2023, 9:19 pm Matthias Ringwald, @.***>
wrote:

@sanjay900 https://github.com/sanjay900 Sorry, nope, was busy otherwise.
How urgent is this for your project? Could you use a custom solution, e.g.
sending multiple reports in a single notification to get full throughput if
the connection is between two Pico Ws for the time being.
It's unclear what exactly is causing this (stack, interface to controller,
controller itself, radio link). The air trace might narrow this a bit down.
—
Reply to this email directly, view it on GitHub
https://github.com/raspberrypi/pico-sdk/issues/1342#issuecomment-1588891041,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAOGYQCSEX2C2XZ4GST4RYTXLAWCBANCNFSM6AAAAAAW3JVOAU
.
You are receiving this because you were mentioned.Message ID:
@.***>

@sanjay900 I'm glad to hear. The nice folks at Raspberry will get to this as well, but there's the 1.5.1 release upcoming now - I've heard :)

Nice, good to hear they will get to it at some point

@mringwal Back on April 12, you said

"...which would indicate a single packet with 200 bytes (if DLE is supported and I think it is)."

Have you resolved whether DLE is supported? The documentation for the BlueKitchen stack says "Don't Know" for DLE support for the CYW43439. The file pico-examples/pico_w/bt/config/btstack_config.h does not contain #define ENABLE_LE_DATA_LENGTH_EXTENSION but the BlueKitchen btstack/port/raspi/btstack_config.h directory does. Is there some work to do to enable DLE that requires calling the HCI commands that ENABLE_LE_DATA_LENGTH_EXTENSION enables?

@rppicomidi The 43439 supports DLE. I have a HCI trace which contains a LE Data Length Change to 251 bytes, which indicates that DLE has been activated for that connection.
You are correct that pico-examples/pico_w/bt/config/btstack_config.h does not defines ENABLE_LE_DATA_LENGTH_EXTENSION, which is needed for DLE. To enable DLE, you can add it to btstack_config.h yourself.
@peterharperuk could you add ENABLE_LE_DATA_LENGTH_EXTENSION to the default configuration in the pico-examples repo?

Yes - raise this so I don't forget https://github.com/raspberrypi/pico-examples/issues/406

@mringwal @peterharperuk  Thank you!

@sanjay900 Sorry, nope, was busy otherwise.
How urgent is this for your project? Could you use a custom solution, e.g. sending multiple reports in a single notification to get full throughput if the connection is between two Pico Ws for the time being.
It's unclear what exactly is causing this (stack, interface to controller, controller itself, radio link). The air trace might narrow this a bit down.

how exactly would one send more than one report in a single notification?

I've setup a test environment with a CSR8510 as Peripheral connected to my Mac and the Pico W with the CYW43439 in Central role.
The CSR8510 is an older Bluetooth v4.0 USB Bluetooth dongle, which does not support DLE or 2M-PHY, but is able to send the max amount of PDUs per connection interval (hence, it's trustworthy from my side).
The Peripheral is sending GATT notifications of 20 bytes, similar to the HID reports in the original issue, the Central only receives the Notifications.
With the new firmware, I got 500-900 packets per second, depending on the connection interval (30ms, 15ms, 7.5ms). With 15 m I got 850 and with 7.5 ms even 900 packets.
This is a reasonable number of packets per interval, and clearly more than one packet per interval.
@sanjay900 Could you try to repeat this with the firmware @peterharperuk posted above https://github.com/raspberrypi/pico-sdk/issues/1342#issuecomment-1541775040 and use the same settings for GATT Streamer as well as LE Streamer client?

le_streamer_client: receive only

#define TEST_MODE TEST_MODE_ENABLE_NOTIFICATIONS

gatt_streamer_server: 20 byte PDUs

limit packet size by changing the size of the test_data arary
char test_data[20];

Thanks @mringwal. Is the new firmware behaving better than the current release firmware?

@peterharperuk To be honest, I didn't go back. As mentioned in the support case, more buffers doesn't hurt. Would it help if repeat the same test with the firmware from master?

Thanks @mringwal. Is the new firmware behaving better than the current release firmware?

Should I replace the UF2 on the currently hosted on the datasheets site with an updated one?

@aallan The firmware was for debug / test purposes only. We would need an official release from Infineon
@mringwal I would like to know how the current firmware is behaving before we spend time getting a new official version of the firmware.

@peterharperuk I see. I'll re-run the same tests with the other firmware then.

I've setup a test environment with a CSR8510 as Peripheral connected to my Mac and the Pico W with the CYW43439 in Central role. The CSR8510 is an older Bluetooth v4.0 USB Bluetooth dongle, which does not support DLE or 2M-PHY, but is able to send the max amount of PDUs per connection interval (hence, it's trustworthy from my side).
The Peripheral is sending GATT notifications of 20 bytes, similar to the HID reports in the original issue, the Central only receives the Notifications.
With the new firmware, I got 500-900 packets per second, depending on the connection interval (30ms, 15ms, 7.5ms). With 15 m I got 850 and with 7.5 ms even 900 packets. This is a reasonable number of packets per interval, and clearly more than one packet per interval.
@sanjay900 Could you try to repeat this with the firmware @peterharperuk posted above #1342 (comment) and use the same settings for GATT Streamer as well as LE Streamer client?
* le_streamer_client: receive only

  * #define TEST_MODE TEST_MODE_ENABLE_NOTIFICATIONS

* gatt_streamer_server: 20 byte PDUs

  * limit packet size by changing the size of the test_data arary
  * char test_data[20];

Just so i can be sure, what are you using for your btstack_config.h?

@sanjay900 btstack_config.h is was not modified.

@peterharperuk As expected/feared, I get pretty much the same throughput values with the firmware on master/v1.0.1

@sanjay900 btw. I'm using the 'poll' version of the Pico run loop variants. Please use that as well to minimize the differences.

I wonder if it's dle related, as I was able to replicate your results, so I went back to hid, and it seems that with a report size of 20 things work, but with a report size of 28 I get limited to the interval again. I'll play around with this more tomorrow, and I'm happy to just shorten my record size and call it a day but it's an interesting outcome

@sanjay900 Thanks for the feedback. When you play around a bit more, I'd be interested what's the smallest size where it slows down. Just enabling DLE doesn't cause an issue, right?
If we can find a clear test case, we can report it back.
I haven't tried 28, but that would requires DLE which is not supported by the CSR8510. I'll need/can use a different dongle then.

As for a test case, this works perfectly. settting test_data to 20 results in a rate of around 15 kB/s, while with it set to 21, the rate drops to <2Kb/s

@mringwal Im just glad you somehow picked 20 bytes and that just happend to be the fix haha, don't think i would have figured this out as that wasn't even a consideration on my mind! Weird though that its 20 bytes and not 27, that means something else is at play here i guess.

I just redesigned all my HID reports to be within 20 bytes and now im getting >850hz between the two picos haha, awesome.

@sanjay900 Glad we found this workaround for you. The limit is 20 bytes as there's a 3 byte ATT header for notifications and a 4 byte header for L2CAP, so the ACL packet will be 27 bytes, which is the maximum packet size without DLE.
I'll probably re-run the tests with 21 bytes payload tomorrow. If that becomes slow, as in your tests, we might have something to show Infineon.

@mringwal that makes sense, i figured there were headers in play. It was kinda good cause it gave me an excuse to do some things more properly, so now im sending 7 byte reports instead of 28 byte ones lol.

I've tested this again without DLE. The issues shows there as well. Throughput drops from 11 kB/s to 0.7 kB/s when increasing from 20 to 21 bytes. I would have tolerated up to half the throughput (as two packets have to be sent over the air), but the air trace shows that only one 28-byte packet is sends as a 27 bytes and a 1 byte packet is sent per connection interval.

Good to have it replicated,.and in a way that's easy to demonstrate
On Fri, 21 Jul 2023, 1:59 am Matthias Ringwald, @.***>
wrote:

I've tested this again without DLE. The issues shows there as well.
Throughput drops from 11 kB/s to 0.7 kB/s when increasing from 20 to 21
bytes. I would have tolerated up to half the throughput (as two packets
have to be sent over the air), but the air trace shows that only one
28-byte packet is sends as a 27 bytes and a 1 byte packet is sent per
connection interval.
—
Reply to this email directly, view it on GitHub
https://github.com/raspberrypi/pico-sdk/issues/1342#issuecomment-1643981652,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAOGYQFE2QAXYQIKV3EBSOLXRE2VZANCNFSM6AAAAAAW3JVOAU
.
You are receiving this because you were mentioned.Message ID:
@.***>

Any updates?

No. The issue is still open against the wifi vendor.


cannot build branch: hygieiacloud2.1 - PMD error

I tried to build the branch today and I get this PMD error doing a mvn clean package
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-pmd-plugin:3.6:check (default) on project api: You have 1 PMD violation. For more details see: /Users/jaz/work/Hygieia/api/target/pmd.xml -> [Help 1]
<?xml version="1.0" encoding="UTF-8"?>
<pmd version="5.3.5" timestamp="2016-05-09T07:09:15.996">
<file name="/Users/jaz/work/Hygieia/api/src/main/java/com/capitalone/dashboard/rest/CloudSubnetController.java">
<violation beginline="66" endline="66" begincolumn="27" endcolumn="43" rule="LooseCoupling" ruleset="Coupling" package="com.capitalone.dashboard.rest" class="CloudSubnetController" method="getSubNetworkDetailsBySubnetIds" externalInfoUrl="https://pmd.github.io/pmd-5.3.5/pmd-java/rules/java/coupling.html#LooseCoupling" priority="3">
Avoid using implementation types like 'ArrayList'; use the interface instead
</violation>
</file>
</pmd>

Yes WIP will fix it in next commit ..

Fixed...


Add S3 version of duplicate-event

duplicate-event only uses the download distribution. The new operation
duplicate-event-aws uses the aws distribution instead.
Your pull request should…

[ ] have a concise title
[ ] close an accompanying issue if one exists
[ ] be against the correct branch
[ ] include migration scripts and documentation, if appropriate
[ ] pass automated tests
[ ] have a clean commit history
[ ] have proper commit messages (title and body) for all commits

I was under the impression that duplicate event works on the asset manager level rather than the distribution side of things so I don't quite understand why we dont just distribute it again to AWS in the following start-workflow wfh?

@ts23 the duplicate-event operation will only publish the new event to the internal channel. Yes, we could remove that and then handle everything in the start-workflow operation. The latter did not exist when duplicate-event was first implemented.
One problem might be that typically the elements for the internal publication are not archived making it harder (impossible?) to publish them again. But maybe this is not a good enough argument and we should archive those elements?

If I understand it correctly we duplicate from archive, so we get the stuff out of the archive and put the copies back into the archive to be processed later so we can do whatever we like with the duplicates, ie. running an arbitrary WF on them.
I think I turned the functionality of duplicate-wf around:
the original:  ...  make copy(ies) and modify the original  ... to ... make copy(ies) and keep orig and run a WF on all copies

we could actually consider sticking the startWF into the duplication

Summary from the OC dev meeting: duplicate-event should no longer deal with publications and we thus don't need two versions of duplicate-event.


Interpolate on a path

I'm trying to animate a marker on a path, but from reading the documentation it appears that I can only interpolate() between two points. Does anyone know how to interpolate on a Polyline with many points?

Hi @nevi-me , did you find any solution for this?

Hi @pingothedoer, I sort of did, I ended up using linear interpolation over very short distances. It however didn't work well, so I stopped trying.


leadingTrivia indentation should not discard zero indentation values

With v1.3.1 documentation is now also included for the generated AllCasePaths struct.
In order to achieve this the leadingTrivia for each case is appended to the generated properties. To ensure indentation matches, the code discards leading whitespaces from the trivia.
When determining how many characters to trim, the code loops over each line in the leadingTrivia, discarding empty lines and counting the amount of prefixed whitespaces. The smallest value counted is then taken and used as the amount of whitespace to trim from each line.
There is an issue in this last bit of logic where lines without prefixed whitespaces are always ignored. This will cause these lines to be trimmed erroneously. Take the following example:
@CasePathable
enum MyEnum {
    // TODO: Implement this case
//    case one
    case two
}

Because the line //    case one is ignored when calculating the minimum amount of whitespace, it is trimmed in the generated code:
public struct AllCasePaths {
    // TODO: Implement this case
      case one
    public var two: CasePaths.AnyCasePath<MyEnum, Void> {
        ...
    }

    …
}

This obviously causes a build failure.
The change removes the bit of logic where lines with zero whitespacing are ignored when calculating the minimum amount of whitespace. As far as I can tell this doesn’t have a negative impact. Tests still pass includingtestDocumentation.

The underlying problem here is that Swift considers the first commented out case to be trivia relating to the second case even though it isn't. I don't think that's easy to fix though :)

@djangovanderheijden Thanks! Any chance you can add a macro test case for this? Would be good to capture the fix in test coverage.

Hey @stephencelis, no problem! I added a small test that fails in the old situation and passes with the changes made.
I also noticed that the indentation of the line on which the property itself is declared (e.g. public var two: CasePaths.AnyCasePath<MyEnum, Void> {) was slightly different from the indentation found in other tests with the change. To combat this, I added a small tweak that removes trailing whitespaces excluding newlines from the very end of the leadingTrivia. All tests still pass and the change seems harmless enough so I figured I'd include it.


merge latest master to dev-v2.0 (do not squash)

I suggest merging via git locally rather than a PR. Or at least it shouldn't be squashed. "Squash and merge" will cause conflicts in future when merging back.

Is this PR necessary?

Is this PR necessary?

if we want to include new updates from master branch in dev-v2.0 in order to avoid possible merge conflict. Or some features depend on new features, bug fix in v1.9.

Is this PR necessary?

if we want to include new updates from master branch in dev-v2.0 in order to avoid possible merge conflict. Or some features depend on new features, bug fix in v1.9.

It seems there is no any commit on dev-2.0 by now,  if you want merge now, you can just delete dev-2.0 and recreate it.

Is this PR necessary?

if we want to include new updates from master branch in dev-v2.0 in order to avoid possible merge conflict. Or some features depend on new features, bug fix in v1.9.


Tests passing for d3 v4

Fixing #54.

[x] All tests passing.
[x] The example shows an animated bar chart and there are no errors in the console.

Attempted to give a birds-eye view of what was broken here.
Ultimately most of the changes to the API do not affect this package as they are passed onto the user; the fixes were to the tests. Engineers will need to rewrite their React components to use the new v4 d3 APIs.

This is awesome, reading now :)

Sweet, let's go for it. Looks good to me locally.


Bug fixed in \analyze command and reuse code.

The \analyze command issues SQL queries. Due to a recent change in the execution engine (PR #14), there was a bug in issuing the queries, which is fixed in this branch. The issue is described below:
The workflow to issue queries is as follows:

Create a TMB message consisting of the query.
Send the TMB message from main thread to the Foreman thread.
Foreman executes the query.
Foreman sends a completion message back to the main thread.
Main thread receives the completion message and proceeds.

In the relevant function where \analyze queries are issued, we were not capturing the response in step 5, which gave a segmentation fault sometimes.
This branch also puts a common code to receive feedback from Foreman upon query completion in a function, which gets used in multiple places. There are some minor inclusion fixes in the touched files.

Assigning @jianqiao

LGTM! Merging.


Can't upload with rclone

For some reason I can upload to teldrive with rclone-browser and rclone itself but its fast until 70mb in where it goes down to 400-600 kb/s. This has been happening since I have started using it, but I would combat it by uploading through the rclone mount, and it seemed  to be at my normal speeds. However, I can't upload to rclone mount anymore, as it shows that its goiong at decent speeds in task manager, but nothing happens, it looks like it just keeps on restarting, and since the rclone copies are basically unusable with the very slow upload speeds, I can't do anything. How do I fix the issues of extremely slow copies and my rclone mount straight up not working?

Try adding a bot in teldrive webui settings.

I got 20 bots, I dont use them all tho cause my internets upload is like 3mb/s, so its pointless

I left it to upload overnight with the mount to see if there is a difference and I get this error:
Failed to copy: multi-thread copy: failed to write chunk: error sending chunk 1: error sending chunk 1
2024/07/19 07:53:56 ERROR : file_instert_here: vfs cache: failed to upload try #2, will retry in 20s: vfs cache: failed to transfer file from cache to remote: multi-thread copy: failed to write chunk: error sending chunk 1: error sending chunk 1
Teldrive logs:
2024-07-19 10:49:16 19/07/2024 12:49 AM ERROR   group: task readLoop: read: read: read intermediate: read length: EOF
2024-07-19 10:49:16 19/07/2024 12:49 AM INFO            {"status": 500, "method": "POST", "path": "/api/uploads/167c5c23d8d162e78a526601f15dfc06", "query": "channelId=0&encrypted=false&fileName=file_yes&partName=4bf4e458d8e7ee21b9718a65ef44e445&partNo=1", "ip": "<IP_ADDRESS>", "user-agent": "rclone/v1.67.4", "latency": 1123.472102627, "time": "2024-07-19T00:49:16Z"}
2024-07-19 10:55:33 19/07/2024 12:55 AM DEBUG   upload failed   {"fileName": "file_yes", "partName": "c12b241d72c58e649fd9ee14387807b1", "chunkNo": 3}
2024-07-19 10:55:33 19/07/2024 12:55 AM ERROR   callback: upload part: send upload part 313 RPC: retry middleware skip: invoke pool: rpcDoRequest: rpc error code 400: FILE_PART_SIZE_INVALI


Native support for traffic/metrics plugins in Argo Rollouts

Within upstream Argo Rollouts, one can enable traffic management plugins and metric plugins via the standard Argo Rollouts configuration ConfigMap.
However, at present, within Argo Rollouts operator there is no mechanism to specify these via the RolloutManager CR.
As of this writing (once PR #40 has merged), the current operator behaviour is:

If there is an existing traffic management plugin defined in the ConfigMap, it will be preserved when we add the OpenShift GitOps Route plugin (not overwritten)
Any custom metric plugin in the ConfigMap are overwritten (lost).

However, requiring users to modify the ConfigMap to add plugins is not as desirable as our supporting it 'natively', within the RolloutManger CR.
Native support within the RolloutManager CustomResource would look like this:
For example, something like:
kind: RolloutManager
metadata:
  name: my-rollout-install
  namespace: rollouts
spec:
  # (...)
  plugins:
    trafficManagement:
      - name: "argoproj-labs/gatewayAPI"
        location: "https://github.com/argoproj-labs/rollouts-plugin-trafficrouter-gatewayapi/releases/download/<version>/gateway-api-plugin-<arch>" # file url also supported

    metric:
      - name: "argoproj-labs/sample-prometheus"
        location: "https://github.com/argoproj-labs/rollouts-plugin-metric-sample-prometheus/releases/download/v0.0.4/metric-plugin-linux-amd64" # file url also supported
        sha256: "dac10cbf57633c9832a17f8c27d2ca34aa97dd3d" #optional sha256 checksum of the plugin executable

This will cause the specified values to be inserted into the ConfigMap that is generated by the RolloutManager controller.
Open Questions to be discussed and/or investigated by implementor:

Do we need to preserve existing plugins within the ConfigMap, or is it safe to overwrite them now this replacement mechanism is available?  My vote is to no longer preserve existing plugins in the ConfigMap.
Do we also need to restart the Rollouts install? Does Rollouts automatically detect the new plugin, once the ConfigMap changes?

Check whether Rollouts processing plugins on startup, or at any time. If on startup, we will need to restart the Pod.
Check how Argo CD operator handles similar cases (if they exist)

Issue Criteria:

Add Traffic/metric plugins fields to the RolloutManager custom resource
Modify operator to support new fields: addition/modification/removal of contents of these fields should update the ConfigMap respectively.
Unit tests to verify behaviour as expected: Addition/removal/modification of the fields
E2E tests to verify that Rollouts operator works as expected

Additional/removal/modification of the fields

For testing traffic management plugin, we can use the Gateway plugin as an example. An E2E test might do this:

Create a RolloutManager pointing to Gateway plugin

Wait for RolloutsManager to install Argo Rollouts with the plugin, and wait for the Rollout Deployment to be healthy

Create an E2E test that follows the Traefik Gateway API steps: https://github.com/argoproj-labs/rollouts-plugin-trafficrouter-gatewayapi/tree/main/examples/traefik

This allows us to verify that the plugin has been successfully loaded, and is running/

For testing metric plugin, there don't seem to be many plugins that exist. I've found these:

https://github.com/argoproj-labs/rollouts-plugin-metric-honeycomb
https://github.com/argoproj-labs/rollouts-plugin-metric-sample-prometheus
Of these, my preference would be a test case (similar to traffic management plugin) using rollouts-plugin-metric-sample-prometheus

Red Hat external issue tracker: https://issues.redhat.com/browse/GITOPS-3987


Bio, photo and social  - Lucas

Bio Lucas
Hi, I'm Lucas.
I'm a PhD researcher in synthetic biology at DTU Bioengineering. After doing my BSc at the University of Copenhagen and my MSc from the Technical University of Denmark, with exchange semesters at the National University of Singapore (NUS) and École Polytechnique Fédérale de Lausanne (EPFL) in Switzerland, I dove into projects like teemi, an open-source literate programming framework for bioengineering. Currently, I'm developing web apps and high-throughput genetic engineering platforms for Streptomyces and Aspergillus. I'm passionate about combining biology and programming to accelerate synthetic biology and automate workflows. Reach out if you are interested in collabs.
Photo:
https://media.licdn.com/dms/image/v2/C4D03AQH1iAa_NnAFJw/profile-displayphoto-shrink_800_800/profile-displayphoto-shrink_800_800/0/1611761701213?e=1736380800&v=beta&t=maUe_0tTtK6-sIbcOLpsczWY_XBpXofExMVVFJCSrcM
Linkedin: https://www.linkedin.com/in/lucas-levassor/
Orchid: https://orcid.org/0000-0002-3261-9911
twitter: https://x.com/Colonellucas27

Updated!


Develop some automated test cases

Pull together some automated test cases that can verify that changes to the RaSCSI applications. These can be incorporated into Github Actions that are run with every commit/merge.

@uweseimet has started this project. This old issue isn't doing anyone any good except making the backlog bigger. Closing.


NASNeta_mobile

Hi @Cadene !
I rewrote https://github.com/Cadene/pretrained-models.pytorch/blob/master/pretrainedmodels/models/nasnet.py#L487 , so now two options are available NASNetALarge (6 @ 4032) and NASNetAMobile (4 @ 1056).
I can do a PR if it would be useful.

Hey Veronika,
Out of curiosity, do you know what the size of a saved NASNetAMobile model turns out to be?

That's so cool! Please do :)

@achaiah it is 21M

NASNetAMobile is available at https://github.com/wandering007/nasnet-pytorch
Thanks to @wandering007 !
https://github.com/Cadene/tensorflow-model-zoo.torch/issues/6

Hi, all!
@Cadene I created a PR https://github.com/Cadene/pretrained-models.pytorch/pull/46 with ported weights to PyTorch.
The results  after porting are:  ** Acc@1 74.080 Acc@5 91.740**


DGT-DMF Integration

Related Issue
Supports #ISSUE_NUMBER
Related PRs
This PR is not dependent on any other PR
What this PR does / why we need it
The purpose of this PR is to bring an integration between DGT and DMF where inputs and outputs can be stored in  DMF / Lakehouse
Special notes for your reviewer
If applicable**

[ ] this PR contains documentation
[ ] this PR contains unit tests
[ ] this PR has been tested for backwards compatibility

Won't proceed on this way. Closing it!


Unable to run any application

Hello,
First, thanks a lot for creating this great tool;
I am contacting you since, since I acquired a new mac, I am not able to run any application from the NeuroDesk. I tried to reinstall both Docker and Neurodesk without any success.
When launching an application, a consol will appear and then close without launching the software. This only concerns the application not already included in Neurodesk (Firefox, text editors.. works) but not SPM, brainstorm...
I tried to launch the following command:

user@neurodesktop-20220128:~$  /neurocommand/local/fetch_and_run.sh spm12 r7771 20211206
[DEBUG] fetch_and_run.sh: Script name : /neurocommand/local/fetch_and_run.sh
[DEBUG] fetch_and_run.sh: Current working dir : /home/user
[DEBUG] fetch_and_run.sh: Script location path (dir) : /neurocommand/local
[DEBUG] fetch_and_run.sh: SINGULARITY_BINDPATH : /afm01,/afm02,/cvmfs,/90days,/30days,/QRISdata,/RDS,/data,/short,/proc_temp,/TMPDIR,/nvme,/local,/gpfs1,/working,/winmounts,/state,/autofs,/cluster,/local_mount,/scratch,/clusterdata,/nvmescratch,/vnm,/neurodesktop-storage
/neurocommand/local/config.ini
CVMFS detected and Container seems to be available
Mounting containers from CVMFS directly.
Due to MODULEPATH changes the following have been reloaded:

spm12/r7771

[DEBUG] fetch_and_run.sh: fetching containers done.
[DEBUG] fetch_and_run.sh: MOD_NAME:  spm12
[DEBUG] fetch_and_run.sh: MOD_VERS:  r7771
[DEBUG] fetch_and_run.sh: Module 'spm12/r7771' is installed. Use the command 'module load spm12/r7771' outside of this shell to use it.
[DEBUG] fetch_and_run.sh: looking for /cvmfs/neurodesk.ardc.edu.au/containers/spm12_r7771_20211206/spm12_r7771_20211206.simg
[DEBUG] fetch_and_run.sh: Attempting to launch container spm12_r7771_20211206
ERROR  : Failed to spawn stage 1
ERROR  : Failed to spawn stage 1
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
the container /cvmfs/neurodesk.ardc.edu.au/containers/spm12_r7771_20211206/spm12_r7771_20211206.simg experienced an error.
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
[DEBUG] fetch_and_run.sh: module load spm12/r7771
[DEBUG] fetch_and_run.sh: Running command ''.

and

user@neurodesktop-20220128:~$ module load spm12/r7771
user@neurodesktop-20220128:~$ spm12
ERROR  : Failed to spawn stage 1
user@neurodesktop-20220128:~$

user@neurodesktop-20220128:~$ cat which spm12
#!/usr/bin/env bash
export PWD=pwd -P
singularity exec  --pwd $PWD /cvmfs/neurodesk.ardc.edu.au/containers/spm12_r7771_20211206/spm12_r7771_20211206.simg spm12 "$@"

The main error seems to be this "Failled to spawn stage 1" but i have no idea to what that corresponds.
Thanks a lot,
Edouard

Hi @Edouard2laire. Do you have the model for the new Mac?

It’s MacBook Pro 2020. Do you think that the fact that it is using an apple M1 chip might be the issue ?

Dear @Edouard2laire - yes, most neuroimaging applications do not yet work on the M1 chip. We are aiming to get the ARM architecture supported but this is a multi-month effort where we need a lot of help from the community.

Here is the issue where we would like to support M1: https://github.com/NeuroDesk/neurodesktop/issues/44 - but we are currently lacking the people power (and the money to purchase an M1) in the project to work on this. @Edouard2laire - are you keen to work on this?

yes, would be happy to help. Would you have any pointer on where to start?

Dear @Edouard2laire,
great to hear you are keen to help :) The best point to get started is at the application container level: Since you would like to run SPM, but SPM requires Matlab, which is not yet supported on M1, I would start with trying to run SPM in Octave, which supports Arm. Could you try to build this container on your machine and see if SPM works?
https://github.com/NeuroDesk/neurocontainers/blob/master/recipes/niistat/build.sh
I am happy to schedule a zoom call in the next days to get you started on this. Get in touch via email<EMAIL_ADDRESS>Cheers
Steffen


Added the new nla.ld and nla.gd collections types.

Added the new Legal deposit and Government deposit collections need for:
http://ourweb.nla.gov.au/apps/jira/browse/DDG-105
http://ourweb.nla.gov.au/apps/jira/browse/DDG-112
http://ourweb.nla.gov.au/apps/jira/browse/DDG-128

Looks good.


[Suggestion] Notification of new comments

Notification of new comments for your mods. Now, in order for me to receive a notification about a new comment in my mod, it is necessary to ping me :(
So it would be nice to be able to just subscribe to the mod, like on a forum thread. Subscription may also include notification of new releases.

hm, strange. you're supposed to be receiving a notification on all comments on mods that you uploaded. I'll have to check.

Perhaps the problem is that I don't receive notifications at all now :(