Display bug with "allowTimes: []" - values hidden in scroll view

If you dynamically change allowTimes: to set of time with less than 6 items - those items are hidden and to show them, you need to click "down scrollbar".
If there are 6 or more items - list is displayed correctly. Also if list is initialized with less then 6 items - it works fine... This situation accurs only when allowTimes is changed after first onShow:
http://jsfiddle.net/konieckropka/sLe94/2/

Im also having this issue, its a pretty big fail and so far proving to be really hard to fix.

Found a better temporary solution.
After changing the allowed times just change the css of the position of the times back to the top.
setTimeout(function(){
                                 $(".xdsoft_time_variant").css({"margin-top": 0});
                                }, 10)

I had to put it in a setTimeout because the position of the times were getting set after the onSelectDate method finished running. (I think)
But this works and its pretty easy, you can still scroll afterwards.

I know this is ancient, but still relevant.
For future Google warriors, try this after you've dynamically changed the allowTimes:
setTimeout(() => datetimepicker.trigger('afterOpen.xdsoft'), 20);
Example:
onSelectDate: async function (date) {
	timePicked = false;
	const slots = await ctl.getTimeSlots(date.toISOString().split('T')[0]);
	if (!slots || !slots.length) {
		$('.xdsoft_time_variant').empty();
	} else {
		this.setOptions({
			allowTimes: slots,
		});
	}
	setTimeout(() => this.trigger('afterOpen.xdsoft'), 20);
},


[TMS] Migrate to app server

Please migrate That's My Song to the app server and mark it as HOT. Vanderbilt is ramping up the data collection now.

@jaburgoyne , it's running on app.amsterdammusiclab.nl/thats_my_song now!


Make Row component with custom element type

Wondering if it is possible to make the Row component with possibility to use a custom element type, something like in react-bootstrap:
https://react-bootstrap.github.io/components.html#grid-props-row
cheers.

Use the tag prop <Row tag="div">. You can also pass in other components, not just strings.

Very nice, thank you.


Issues with the model list after modifying the model

Describe the bug

Steps to reproduce
Issue 1: After renaming the model, the name in the model list has not been updated.
Issue 2: After filtering the model names and modifying the model, the filter is not taking effect.
https://github.com/user-attachments/assets/b60641f6-11ac-45e2-b5a9-ae30fbc5f4b1
Expected behavior

Environment

GPUStack version: main(65da7a7)
UI version: 0317a37
OS: macOS 14.6.1
GPU: Apple M2, 24G

Additional context
Add any other context about the problem here.

Environment:

GPUStack version: main(1578ff0)
UI version: 0adca93

Test Result:
Issue 1: pass
Issue 2: pass


[BUG] fixed issue #132

Closes: #132
Description of Changes
Applied changes in CSS.

No worries.. can we continue login page in this pR?? and please pull the latest changes...

No worries.. can we continue login page in this pR?? and please pull the latest changes...

Yes you may merge this, sure I'll pull it


Huawei LTE fails during setup

Home Assistant release with the issue:

0.102.2
Last working Home Assistant release (if known):
Operating environment (Hass.io/Docker/Windows/etc.):

Debian 10 Buster
Home assistant in virtual environment per the home assistant guide.
Integration:

https://www.home-assistant.io/integrations/huawei_lte/
Description of problem:
Huawei B535 setup fails. I have tested huawei-lte-api manually and it works with the B535. I am running the B535 in bridge mode, so it cannot see any devices. I want to use the integration to send notifiations and monitor the LTE
Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):
It is configured via the GUI, with the proper username and password. It also fails if I run it in non-authenticated mode.

Traceback (if applicable):
Error setting up entry B535-232 for huawei_lte
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/config_entries.py", line 192, in async_setup
    hass, self
  File "/etc/homeassistant/custom_components/huawei_lte/__init__.py", line 313, in async_setup_entry
    **device_data,
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/helpers/device_registry.py", line 123, in async_get_or_create
    for key, value in connections
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/helpers/device_registry.py", line 123, in <setcomp>
    for key, value in connections
  File "/srv/homeassistant/lib/python3.7/site-packages/homeassistant/helpers/device_registry.py", line 56, in format_mac
    if len(to_test) == 17 and to_test.count(":") == 5:
TypeError: object of type 'NoneType' has no len()

Additional information:

@scop let me know if I can assist in further debugging.

It looks like platform independent problem (I'm using hass.io) - after update from 0.101.3 to 0.102.2 this integration is broken.
I've read Breaking Changes section before update, but I already have configuration like in example
 - url: http://<IP_ADDRESS>/     username: !secret huawei_lte_user     password: !secret huawei_lte_pass and it was working till update.
I've also tried GUI configuration, but also without success.

@larsvinc the problem originates from that we're unable to grab mac address for your router. What does the URL you're using look like? If you're using an IP address in the URL, can you try what this outputs?
python -c 'import getmac;print(getmac.get_mac_address(ip="<IP_ADDRESS>"))' # replace your ip address

...or if you're using a hostname in it:
python -c 'import getmac;print(getmac.get_mac_address(hostname="your-hostname-here"))' # replace your hostname

@larsvinc have you got the integration to work with earlier Home Assistant versions, or is this the first one you tried with? IIRC we have been using mac address identifiers since HA 0.97, so I believe something could have failed already with those versions.
@nepozs please confirm that the traceback you're seeing is essentially the same as in the original comment. Also please specify if you're using the router in bridged mode or not.

I have not gotten this to work with earlier versions.
(homeassistant) homeassistant@kraftkar:/srv/homeassistant$ python3 -c 'import getmac;print(getmac.get_mac_address(ip="<IP_ADDRESS>"))' # replace your ip address
None
See the output above. Can this be caused by the router sitting outside the firewall? It is kind of interesting because when I run it on devices that does not exist in the local network it returns all zeros:

getmac.get_mac_address(ip="<IP_ADDRESS>")
'00:00:00:00:00:00'

Looks like the fix is to handle the returned "None"?

Maybe, but I'd first like to understand when does it end up returning None and to see if this is something that can be addressed in getmac. We'd really like a usable mac, even if it would be the one of the firewall.
Could you make the getmac module within your HA to produce debug output in your HA log and see what's there in case it gives some hints? There are two parts to this, first the logging config in configuration.yaml:
logger:
  default: info
  logs:
    getmac: debug

...as well as a dirty code hack in some module, for example near the rest of the imports in homeassistant/components/huawei_lte/init.py, add these lines:
import getmac
getmac.getmac.DEBUG = 9

It should start producing lines containing [getmac].

First I changed the router address to <IP_ADDRESS> to make absolutely sure that there was no issue in the 192.168-ranges. The problem persisted. Then I ran the full debug per your request. Output:
2019-12-03 21:31:26 DEBUG (SyncWorker_19) [getmac] Trying: 'arp -a <IP_ADDRESS>'
2019-12-03 21:31:26 DEBUG (SyncWorker_19) [getmac] Running: '/usr/sbin/arp -a <IP_ADDRESS>'
2019-12-03 21:31:27 DEBUG (SyncWorker_19) [getmac] Output from '/usr/sbin/arp' command: b'arp: in 5 entries no match found.\n'
2019-12-03 21:31:27 DEBUG (SyncWorker_19) [getmac] Result: None
2019-12-03 21:31:27 DEBUG (SyncWorker_19) [getmac] Trying: '_uuid_ip' (to_find: '<IP_ADDRESS>')
2019-12-03 21:31:27 DEBUG (SyncWorker_19) [getmac] Result: None
2019-12-03 21:31:27 DEBUG (SyncWorker_19) [getmac] Raw MAC found: None
I tried the arp command manually with sudo, (sudo arp -a <IP_ADDRESS>), and it seems like the problem is that the router is not in the local arp cache.

Thanks for testing. Can you reach the device via arping? I.e. arping -f -c 1 <IP_ADDRESS>? I submited that to getmac upstream, let's see what they think. https://github.com/GhostofGoes/getmac/pull/52
Anyway I suppose we need to handle the case where None is returned nevertheless.  #29455 hopefully accomplishes that.

:~$ sudo arping -F -c 1 <IP_ADDRESS>
ARPING <IP_ADDRESS>
Timeout
--- <IP_ADDRESS> statistics ---
1 packets transmitted, 0 packets received, 100% unanswered (0 extra)
I also tested arping on a different valid IP and there it worked.

@scop my traceback (looks nearly identical)
2019-12-01 01:16:30 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry B525s-23a for huawei_lte Traceback (most recent call last):   File "/usr/src/homeassistant/homeassistant/config_entries.py", line 192, in async_setup     hass, self   File "/usr/src/homeassistant/homeassistant/components/huawei_lte/__init__.py", line 287, in async_setup_entry     await hass.async_add_executor_job(router.update)   File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run     result = self.fn(*self.args, **self.kwargs)   File "/usr/src/homeassistant/homeassistant/components/huawei_lte/__init__.py", line 176, in update     get_data(KEY_WLAN_HOST_LIST, self.client.wlan.host_list)   File "/usr/src/homeassistant/homeassistant/components/huawei_lte/__init__.py", line 152, in get_data     self.data[key] = func()   File "/usr/local/lib/python3.7/site-packages/huawei_lte_api/api/WLan.py", line 69, in host_list     host = hosts.get('Hosts', {}).get('Host') AttributeError: 'NoneType' object has no attribute 'get'
My Huawei LTE works as gateway, but is in in more complicated configuration: 2 routers in "cascade" connection - Huawei has LAN IP <IP_ADDRESS> in network <IP_ADDRESS>/24 but hassio works behind secondary router (also in gateway mode) with LAN <IP_ADDRESS>/24 so no arp is possible (router after router), but before upgrade to 0.102.x this integration had worked properly.

@nepozs your backtrace indicates that it's a different issue from this one, it's almost certainly https://github.com/home-assistant/home-assistant/issues/28922 which has been fixed and I believe will appear in 0.102.4 or 0.103, whichever comes first.

@larsvinc oh well, arping doesn't rescue all cases it seems. But BTW, which arping is the one you use? The one I have installed on Ubuntu 18.04 (from iputils-arping package) and hass.io doesn't have the -F argument, they both have just -f ("quit on first arp reply"). Does yours understand the lowercase one?
Anyway I hope the fix from #29455 covers the None case.

(Forgot to note that in case #29455 doesn't, please let me know.)

Nevermind the arping version, my patch was accepted in getmac and there's some discussion about that there, see https://github.com/GhostofGoes/getmac/pull/52#issuecomment-561989761

@scop - I am using ARPing 2.19 (running debian buster amd64). Glad to hear that there is a patch in getmac.


msi build of latest toolbox

Is it possible to build a .msi of the latest released version of toolbox
Thanks,
Shaun

Done: https://github.com/3scale-labs/3scale_toolbox_packaging/releases/tag/v0.20.0


Moving data from CPU to GPU is slow

I have noticed that when moving MNIST data from the CPU to the GPU, there is a significant time lag when using TensorFlow in comparison to Theano. Specifically, we have noticed this problem in the context of feed_dict, which moves information from the CPU to the GPU when running minibatches. I am using python 2.7. Our current solution to this problem is to move all of the data directly to the GPU at the beginning of the program, which is of course not sustainable unless one has a significant amount of space on their GPU.
When we time 25000 minibatches of size 100 each, TensorFlow is approximately four times as slow as Theano. I have attached both files, and the time difference is evident in the final output.
Environment info
Operating System: Ubuntu 14.04
CPU Information:
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                12
On-line CPU(s) list:   0-11
Thread(s) per core:    2
Core(s) per socket:    6
Socket(s):             1
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 63
Stepping:              2
CPU MHz:               1246.328
BogoMIPS:              6995.89
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              15360K
NUMA node0 CPU(s):     0-11
GPU: NVidia GeForce GTX TITAN X Graphics Card (12GB)
Installed version of CUDA and cuDNN: 7.5.17
Logs or other output that would be helpful
(If logs are large, please upload as attachment).
Relevant files (I apologize for the poor naming conventions):
1.mnist_softmax.txt
This file contains the relevant program that uses TensorFlow.

mnist.pkl.gz
This file contains the dataset for the TensorFlow file.

[mnist_softmax_theano.txt(https://github.com/tensorflow/tensorflow/files/370314/mnist_softmax_theano.txt)
This file contains the relevant program that uses Theano.

tf_data.pkl.gz
This file contains the dataset necessary for the Theano file.

Just curious: have you (and how have you) isolated the problem to host-to-device transfer with feeding?
Search for StepStats and "chrome timeline" in the repo.  It should tell you how to get a basic performance profile out.

Lemme clarify the question a little bit.
We took the tutorial code for logistic regression for MNIST from the tensorflow website.
We used the same MNIST dataset from the tensorflow repository and used an equivalent logistic regression model in Theano.
We're not sure where the four-time timing difference comes from, although I suspect it's from the feed_dict and placeholder feature.
I don't see any documentation for the StepStats and timeline function. So I don't know how to use those to do profiling.
Can you please provide more instruction as to how to profile?
Thank you.

You can look at report #3320 as I have example code there to capture a run in the "discrete_deepq_model.py" file starting near line 257.  This will give you a trace file to open in Crome with:
chrome://tracing
There is a link to the reference I used to create the trace file in that post as well.

We generated a basic performance profile and opened it in chrome://tracing, but we are unsure of how to interpret it.
Here is the JSON file:
timeline.ctf.json.zip

@zheng-xq would you take a look?

The above is for a single step, here is a JSON file with 500 steps concatenated together:
timeline_500.ctf.json.zip

@Brandon-Tai, the best performing TensorFlow models tend to feed the examples into a queue with a different Python thread, or use one of the reader op to read in the examples, instead through feed_dict.
https://www.tensorflow.org/versions/r0.10/api_docs/python/io_ops.html#readers
We can take a look a this model and see if there are other issues beside feed_dict.

@zheng-xq It seems that if we only use one GPU to train the model, then even if we load the data from disk to memory in parallel threads we still have to load the data into the GPU sequentially. Do you mean we should train the models using multiple GPUs so that we can also load the data into GPU in parallel? But in that case for each GPU the loading still has to be sequential.

feed_dict has more overhead than just copying CPU to GPU. It also moves the data from Python to TensorFlow among other things. Someone will have to look further to determine whether this is the only issue.
But even with only one GPU, you can hide data transfer between python and TensorFlow through:
1.a Having one python thread to read the data, and feed them into a tensorflow queue, managed by queue runner.
1.b Having your model read from that queue in parallel, instead of through feed_dict. That way, the data copy from Python to TensorFlow is parallelized behind the real training.
When you have many replicas running on multiple GPU over multiple machines, you want to use the reader class to read the data locally on each replica, instead of through feed_dict. For example:
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/models/image/cifar10/cifar10_input.py#L79
https://github.com/tensorflow/models/blob/master/inception/inception/dataset.py#L95

@lizallendorf, is your original issue resolved? Did @zheng-xq's suggestions help?

BTW, official MNIST examples on the website Howto are not tuned. /mnist/fully_connected_feed.py using feed_dict runs is 10x faster than other 3 examples which use queues, due to how Python schedules pre-fetching threads

Closing due to lack of activity.


The diversity loss has no effect?

Hi, authors! Thank you for doing such an inspiring job and opening the source code! There is a problem when I using your code, the diversity loss seems not having a great effect.
I run your code "RIDE Without Distill (Stage 1)" of 3 experts on CIFAR100-LT using your config, and got validation accuracy 47.8%. And I tried to do some ablation, I make "additional_diversity_factor"=0.45 (the original setting is -0.45), got validation accuracy 48.0%, which is even 0.2% higher than 47.8%. I didn't change any thing else of your codes. Could you help me figure out the problem?
Thanks a lot!

Update:
I also found a bug in the code:https://github.com/frank-xwang/RIDE-LongTailRecognition/blob/39ab2c832a10baba22a2e1943b1a60c9b492bfe0/model/loss.py#L224 means using "collaborative loss" if diversity loss is 0 (if I didn't misunderstand), while this comparison is unfair (as paper said "individual loss" is better than "collaborative loss"). So I change this setting, using "individual loss" when w/o diversity, achieve 48.7% accuracy when testing.
Here are some logs of my experiments:

individual loss w/ -0.45 KL divergence
'loss': 2.590021451187134, 'accuracy': 0.4772, 'many_class_num': 35, 'medium_class_num': 35, 'few_class_num': 30, 'many_shot_acc': 0.6554285, 'medium_shot_acc': 0.50257146, 'few_shot_acc': 0.23966669
individual loss w/ 0.45 KL divergence
'loss': 2.487060943412781, 'accuracy': 0.4798, 'many_class_num': 35, 'medium_class_num': 35, 'few_class_num': 30, 'many_shot_acc': 0.68057144, 'medium_shot_acc': 0.49142852, 'few_shot_acc': 0.23200001
collaborative loss w/o KL divergence
'loss': 3.084108729171753, 'accuracy': 0.4549, 'many_class_num': 35, 'medium_class_num': 35, 'few_class_num': 30, 'many_shot_acc': 0.64285713, 'medium_shot_acc': 0.46028566, 'few_shot_acc': 0.22933334
individual loss w/o KL divergence
'loss': 2.51549779586792, 'accuracy': 0.4872, 'many_class_num': 35, 'medium_class_num': 35, 'few_class_num': 30, 'many_shot_acc': 0.68, 'medium_shot_acc': 0.49628574, 'few_shot_acc': 0.25166667

all experiments are of 3 experts.
Thanks.

Update:
I also found a bug in the code:https://github.com/frank-xwang/RIDE-LongTailRecognition/blob/39ab2c832a10baba22a2e1943b1a60c9b492bfe0/model/loss.py#L224 means using "collaborative loss" if diversity loss is 0 (if I didn't misunderstand), while this comparison is unfair (as paper said "individual loss" is better than "collaborative loss"). So I change this setting, using "individual loss" when w/o diversity, achieve 48.7% accuracy when testing.
Here are some logs of my experiments:

individual loss w/ -0.45 KL divergence
'loss': 2.590021451187134, 'accuracy': 0.4772, 'many_class_num': 35, 'medium_class_num': 35, 'few_class_num': 30, 'many_shot_acc': 0.6554285, 'medium_shot_acc': 0.50257146, 'few_shot_acc': 0.23966669
individual loss w/ 0.45 KL divergence
'loss': 2.487060943412781, 'accuracy': 0.4798, 'many_class_num': 35, 'medium_class_num': 35, 'few_class_num': 30, 'many_shot_acc': 0.68057144, 'medium_shot_acc': 0.49142852, 'few_shot_acc': 0.23200001
collaborative loss w/o KL divergence
'loss': 3.084108729171753, 'accuracy': 0.4549, 'many_class_num': 35, 'medium_class_num': 35, 'few_class_num': 30, 'many_shot_acc': 0.64285713, 'medium_shot_acc': 0.46028566, 'few_shot_acc': 0.22933334
individual loss w/o KL divergence
'loss': 2.51549779586792, 'accuracy': 0.4872, 'many_class_num': 35, 'medium_class_num': 35, 'few_class_num': 30, 'many_shot_acc': 0.68, 'medium_shot_acc': 0.49628574, 'few_shot_acc': 0.25166667

all experiments are of 3 experts.
Thanks.

Hi yypurpose, thanks for your insightful question and attention to our project. I have carefully read the numbers that you give as well as the analysis. However, it is very interesting of us to see that your runs have different results than what we have. Here I will briefly explain how we run our experiments, how the results were, and how we get to the current conclusion: we run the experiments that you run five times in total, two with -0.45, the current hyperparm, and two with 0.45, the hyperparam that you offer, and we run the same as your settings with hyperparam as 0 and modify the code the same as the way you describe.
In the case that hyperparam=-0.45, we get 48.82% and 49.00% as our first pair. In the case that hyperparam=0.45, we get 47.74% and 47.75% as our second pair. In the experiment you suggest with a hyperparam as 0, we get 48.25%. The mean of our runs with a hyperparam=-0.45 is about 1.2% higher than the runs with hyperparam=0.45, which is enough to prove that our loss is effective in our runs.
In answer to the results you have, I see that your result is lower than both our formally reported one which is from an old codebase (48.0%), and the ones above that use the new codebase, which is higher than the old codebase that we use in our report (the above ones actually have 48.9% as an average), and this may mean something is actually in a mismatch. However, there are many issues that could lead to the differences. I suggest you try the latest codebase and upgrade your dependencies to make sure we are on the same page, although I admit that this does not solve all the problems and it is hard to give much assistance.
To let you know and compare with our previous experiments, I have put logs of our runs in the following gist for you to take a look and you could check them to validate your environment or process: https://gist.github.com/TonyLianLong/885c8d3d0feb9f775c3a2e5794ba0450, https://gist.github.com/TonyLianLong/9656ca2ce75eb5f167ba3349c69b518e, https://gist.github.com/TonyLianLong/87ef86db3195b243846f8ca024441e31. If you would like, I can also send you the checkpoints so that you could test whether our models behave the same and are evaluated to the same results in evaluation time and there is no precision error at least in inference between us for these models. We also propose to give several updates to our report and codebase (see below), and we will then perform a check when we update them to make sure each part works on our end, so if you think we should look further into it, then don't worry, we have it on our plan already. We will give you updates for anything we discover and would like to tell you. Hope this answers your question.
In addition, I would also like to give you some information on the upcoming revision of our report. This serves as a preview for those who are curious, too (so if you are one of these people, you probably would like to read this paragraph). We acknowledge that there are many things that are not clear in the previous one such as the analysis, presentation, organization, and the formula, which may be misleading, and we will give an update in about several weeks. To relieve you from waiting for the full revised report, I would like to point out the way that we organize our contributions. I will write in a more detailed way because this also answers other questions about potential clarifications, which also potentially solves other people's concerns. The way that we organize our contributions has the following: 1. the multi-experts framework that is applicable to many backbone networks and allows parallel voting and agreements and is designed to efficiently use parameters, 2. the loss function to encourage diversity (this includes two components that provide the same goal and purpose: 2.1. individual loss, which we proposed to replace the collaborative one that was used in former multi-branch models in works like BBN, we found individual loss would then decouple most of the correlation among experts since we apply loss individually to each expert, which contributes about 2% to our improvement without adding computation or change the evaluation or inference formula at test time. 2.2. we found that although implicit random initialization is already effective to encourage diversity, adding an explicit KL divergence to further encourage the disentanglement is beneficial most of the time, although only to a small increase as listed above if you compare to the 5% to 7% overall improvement. We admit that we did not offer enough details and we will add more things, give details, and provide necessary explanations to make it clearer on the effects of the loss.), 3. the routing experts module which effectively uses the multi-expert property to reduce the number of computation in running the model: the diversity will matter if we need it to reduce variance, which is especially useful for tail classes and leads to the preservation of result with fewer computational costs. We are also going to add more experiments and analysis which will bring more details as well as make corrections to several known issues. In addition, we present tricks specific to multi-expert models, such as using a model with 6 experts as a teacher to distill into a model with few experts, as well as analysis such as bias-variance tradeoff in long-tailed datasets. We will update the codebase accordingly to reflect our new update. I think the updated report will present you with a clear view of our project in which we clarify a lot of things.

Hi yypurpose, thanks for your insightful question and attention to our project. I have carefully read the numbers that you give as well as the analysis. However, it is very interesting of us to see that your runs have different results than what we have. Here I will briefly explain how we run our experiments, how the results were, and how we get to the current conclusion: we run the experiments that you run five times in total, two with -0.45, the current hyperparm, and two with 0.45, the hyperparam that you offer, and we run the same as your settings with hyperparam as 0 and modify the code the same as the way you describe.
In the case that hyperparam=-0.45, we get 48.82% and 49.00% as our first pair. In the case that hyperparam=0.45, we get 47.74% and 47.75% as our second pair. In the experiment you suggest with a hyperparam as 0, we get 48.25%. The mean of our runs with a hyperparam=-0.45 is about 1.2% higher than the runs with hyperparam=0.45, which is enough to prove that our loss is effective in our runs.
In answer to the results you have, I see that your result is lower than both our formally reported one which is from an old codebase (48.0%), and the ones above that use the new codebase, which is higher than the old codebase that we use in our report (the above ones actually have 48.9% as an average), and this may mean something is actually in a mismatch. However, there are many issues that could lead to the differences. I suggest you try the latest codebase and upgrade your dependencies to make sure we are on the same page, although I admit that this does not solve all the problems and it is hard to give much assistance.
To let you know and compare with our previous experiments, I have put logs of our runs in the following gist for you to take a look and you could check them to validate your environment or process: https://gist.github.com/TonyLianLong/885c8d3d0feb9f775c3a2e5794ba0450, https://gist.github.com/TonyLianLong/9656ca2ce75eb5f167ba3349c69b518e, https://gist.github.com/TonyLianLong/87ef86db3195b243846f8ca024441e31. If you would like, I can also send you the checkpoints so that you could test whether our models behave the same and are evaluated to the same results in evaluation time and there is no precision error at least in inference between us for these models. We also propose to give several updates to our report and codebase (see below), and we will then perform a check when we update them to make sure each part works on our end, so if you think we should look further into it, then don't worry, we have it on our plan already. We will give you updates for anything we discover and would like to tell you. Hope this answers your question.
In addition, I would also like to give you some information on the upcoming revision of our report. This serves as a preview for those who are curious, too (so if you are one of these people, you probably would like to read this paragraph). We acknowledge that there are many things that are not clear in the previous one such as the analysis, presentation, organization, and the formula, which may be misleading, and we will give an update in about several weeks. To relieve you from waiting for the full revised report, I would like to point out the way that we organize our contributions. I will write in a more detailed way because this also answers other questions about potential clarifications, which also potentially solves other people's concerns. The way that we organize our contributions has the following: 1. the multi-experts framework that is applicable to many backbone networks and allows parallel voting and agreements and is designed to efficiently use parameters, 2. the loss function to encourage diversity (this includes two components that provide the same goal and purpose: 2.1. individual loss, which we proposed to replace the collaborative one that was used in former multi-branch models in works like BBN, we found individual loss would then decouple most of the correlation among experts since we apply loss individually to each expert, which contributes about 2% to our improvement without adding computation or change the evaluation or inference formula at test time. 2.2. we found that although implicit random initialization is already effective to encourage diversity, adding an explicit KL divergence to further encourage the disentanglement is beneficial most of the time, although only to a small increase as listed above if you compare to the 5% to 7% overall improvement. We admit that we did not offer enough details and we will add more things, give details, and provide necessary explanations to make it clearer on the effects of the loss.), 3. the routing experts module which effectively uses the multi-expert property to reduce the number of computation in running the model: the diversity will matter if we need it to reduce variance, which is especially useful for tail classes and leads to the preservation of result with fewer computational costs. We are also going to add more experiments and analysis which will bring more details as well as make corrections to several known issues. In addition, we present tricks specific to multi-expert models, such as using a model with 6 experts as a teacher to distill into a model with few experts, as well as analysis such as bias-variance tradeoff in long-tailed datasets. We will update the codebase accordingly to reflect our new update. I think the updated report will present you with a clear view of our project in which we clarify a lot of things.

Thank you for your detailed answer! My opinion is that, if w/o diversity loss, the performance is still at a good level, it's a good thing. No need to concern about the hyper-parameters!
I'm also looking forward to see the new version of your report!
Thanks.

Thank you for your detailed answer! My opinion is that, if w/o diversity loss, the performance is still at a good level, it's a good thing. No need to concern about the hyper-parameters!
I'm also looking forward to see the new version of your report!
Thanks.

@yypurpose @TonyLianLong My ablation experiments are consistent with @yypurpose
on cifar100-LT, with all other factors the same, the accuracies on test set are

individual loss + KL loss(without any change of the code): 48.51
collaborative loss(set additional_diversity_factor=0): 46.39
individual loss only(comment the line that add diversity loss): 48.95
it seems diversity loss has no effect, although making multiple experts focus on different things sounds reasonable.

Although this issue has been addressed above, we have checked the checkpoints of all the results above and attached them here, including the results of several runs, log files, and checkpoints: According to them, our results are consistent with the results reported in the paper, and in fact are often higher than the reported results..
Here we provide a summary on the previous experiment results:

Diversity loss (individual+KL) with weight_factor=-0.45:

Results: 48.82% and 49.00% (reported in the paper: 48.0)
Log files: https://gist.github.com/TonyLianLong/885c8d3d0feb9f775c3a2e5794ba0450 https://gist.github.com/TonyLianLong/5fad89c7578809f9b861d361e25cb9c4
Checkpoints: https://drive.google.com/drive/folders/1S2tOAoCSIs7-1aZKRcKEB_5kAC1IOhL4?usp=sharing https://drive.google.com/drive/folders/1MShjBhOntsIiTaz4eq24mPT8E0e-I8xO?usp=sharing

Diversity loss (individual+KL) with weight_factor=0.45:

Results: 47.74% and 47.75%
Log files: https://gist.github.com/TonyLianLong/87ef86db3195b243846f8ca024441e31 https://gist.github.com/TonyLianLong/340757296ba05b13480a23955f56636a
Checkpoints: https://drive.google.com/drive/folders/1rD3FSF914fkbn5BAp0E-bXSrQf_bv8Ll?usp=sharing https://drive.google.com/drive/folders/164mHLPlHolL1C7yu86Ihz2alcGqfiX_U?usp=sharing

Diversity loss (individual only):

Results: 48.25%
Log files: https://gist.github.com/TonyLianLong/9656ca2ce75eb5f167ba3349c69b518e
Checkpoints: https://drive.google.com/drive/folders/1XiuSbIckPgIx2hjY_X6eYIKh65Lj-d9z?usp=sharing

Vanilla (collaborative):

Results: 45.83%
Log files: https://gist.github.com/TonyLianLong/f644a8e4843fef7ff3cfc250fcffea89
Checkpoints: https://drive.google.com/drive/folders/1_qJhDbCjYufHU2ZXesVwruNlyhD7XK7q?usp=sharing

You could test our models with the checkpoints above. According to results above, the results that we obtained above strengthen the belief on the diversity loss to be effective. Again, I suggest you try the latest codebase and upgrade your dependencies to make sure we are on the same page, although I admit that it does not solve all the problems and it is hard to give much assistance.
In addition, the intuition is that diversity is very sensitive to many factors. As a thought experiment, if the branches' initializations are highly correlated (to the extreme, same initialization), and then experts will have similar even the same SGD update and same output (since we train on same data), and no improvement will be observed, unless we apply a loss that encourage diversity.
Our loss, which encourages KL divergence, encourages this diversity and removes the bad effect since we are not using other things to control these factors currently.
For the diversity part, in order to encourage multiple experts to make supplementary decisions, we also adopted "individual loss" instead of collaborative losses like BBN. The decopling effect in the individual loss also encourages the multiple experts to produce diversified opinions and the above experiment shows the effectiveness of diversity loss with two parts.
As you can see, the models above were trained half a year ago, and I don't have much time to dig even deeper on this issue right now. We are planning an extension of this project, so we will definitely perform an in-depth analysis to address your concern.
Hope this answers your question and wish you a nice day.


add multireader

Provides option to wrap multiple stores to get a single content.Provider that can be used anywhere a Provider is needed.

Yeah, this one, I think, should be doable.

Updated with a test


coqPackages.hydra-battles: fix version checking logic

Follow up of #160479 (cc @vbgl).

Motivation for this change
With this change, the derivation can be relied on without overlay in the upstream project.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Can’t isGe and co. treat dev as +∞?

It would be better if it would, indeed.


[1.13 rc3 Swarm] When restarting a local container on a worker which connects to a overlay network the worker is added as peer every time.

Description
I am using 1.13 rc3 to test the newest swarm features. Currently I am using overlay networks in the swarm to let local running containers communicate with each other. To be precise, the manager runs zabbix-server, zabbix-web and zabbix-postgres container. One of the worker runs zabbix-agent which should connect to the zabbix-server. They are all in the same zabbix-swarm-network.
Now I used docker restart to restart the docker container on the worker. After inspecting the overlay network via docker network inspect zabbix-swarm-network I found out that the current worker is added every restart to the network. This is ONLY on the worker! The manager shows everything correct.
Steps to reproduce the issue:

Make overlay network networkName
Create container with --network networkName
restart container with docker restart containerName
docker network inspect networkName

Describe the results you received:
root@jar:~# docker network inspect zabbix-swarm-network
[
    {
        "Name": "zabbix-swarm-network",
        ...,
        "Containers": {
            "3a2c1e4b122eb5358deb3b67120874717592c8f7508c917b47d90dcb1dafd95c": {
                "Name": "zabbix-agent-jar",
                ...
            }
        },
        ...
        "Peers": [
            {
                "Name": "jar-b6c5612d7641",
                "IP": "<IP_ADDRESS>"
            },
            {
                "Name": "bottle.haembi.de-74a51b212452",
                "IP": "<IP_ADDRESS>"
            },
            {
                "Name": "jar-b6c5612d7641",
                "IP": "<IP_ADDRESS>"
            },
            {
                "Name": "jar-b6c5612d7641",
                "IP": "<IP_ADDRESS>"
            }
        ]
    }
]

Describe the results you expected:
Only one entry of jar.
Additional information you deem important (e.g. issue happens only occasionally):
I don't know if this is important but the zabbix-server (which runs on the manager) can't reach the zabbix-agent on the worker. Maybe this is due to the duplicated peers entry? Or I am doing something wrong here.
Output of docker version on manager:
Client:
 Version:      1.13.0-rc3
 API version:  1.25
 Go version:   go1.7.3
 Git commit:   4d92237
 Built:        Mon Dec  5 18:49:08 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.13.0-rc3
 API version:  1.25 (minimum version 1.12)
 Go version:   go1.7.3
 Git commit:   4d92237
 Built:        Mon Dec  5 18:49:08 2016
 OS/Arch:      linux/amd64
 Experimental: false

Output of docker info: on manager
Containers: 4
 Running: 4
 Paused: 0
 Stopped: 0
Images: 4
Server Version: 1.13.0-rc3
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 33
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
Swarm: active
 NodeID: 2roi10et2p908xc60w8ap8i6x
 Is Manager: true
 ClusterID: 69a3wiv7ja5bxs7isbndnmglv
 Managers: 1
 Nodes: 6
 Orchestration:
  Task History Retention Limit: 5
 Raft:
  Snapshot Interval: 10000
  Number of Old Snapshots to Retain: 0
  Heartbeat Tick: 1
  Election Tick: 3
 Dispatcher:
  Heartbeat Period: 5 seconds
 CA Configuration:
  Expiry Duration: 3 months
 Node Address: <IP_ADDRESS>
 Manager Addresses:
  <IP_ADDRESS>:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 03e5862ec0d8d3b3f750e19fca3ee367e13c090e
runc version: 51371867a01c467f08af739783b8beafc154c4d7
init version: 949e6fa
Kernel Version: 3.16.0-4-amd64
Operating System: Debian GNU/Linux 8 (jessie)
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 15.62 GiB
Name: bottle.haembi.de
ID: W4ZV:L457:5MAK:WYYG:UE3Y:ECG6:A5UI:7EFA:IW7Q:5BE2:UPBP:G5YW
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No memory limit support
WARNING: No swap limit support
WARNING: No kernel memory limit support
WARNING: No oom kill disable support
WARNING: No cpu cfs quota support
WARNING: No cpu cfs period support
Experimental: false
Insecure Registries:
 <IP_ADDRESS>/8
Live Restore Enabled: false

Output of docker version on worker:
Client:
 Version:      1.13.0-rc3
 API version:  1.25
 Go version:   go1.7.3
 Git commit:   4d92237
 Built:        Mon Dec  5 18:49:08 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.13.0-rc3
 API version:  1.25 (minimum version 1.12)
 Go version:   go1.7.3
 Git commit:   4d92237
 Built:        Mon Dec  5 18:49:08 2016
 OS/Arch:      linux/amd64
 Experimental: false

Output of docker info: on worker
Containers: 2
 Running: 1
 Paused: 0
 Stopped: 1
Images: 7
Server Version: 1.13.0-rc3
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 34
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
Swarm: active
 NodeID: zqe2l9ft41guntg448vqxtpf5
 Is Manager: false
 Node Address: <IP_ADDRESS>
 Manager Addresses:
  <IP_ADDRESS>:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 03e5862ec0d8d3b3f750e19fca3ee367e13c090e
runc version: 51371867a01c467f08af739783b8beafc154c4d7
init version: 949e6fa
Kernel Version: 3.16.0-4-amd64
Operating System: Debian GNU/Linux 8 (jessie)
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 3.854 GiB
Name: jar
ID: N47Z:CACH:MYSN:7R2P:YO74:O2AP:CCJA:GSVL:NPPN:4VXQ:SZPS:HS2R
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No memory limit support
WARNING: No swap limit support
WARNING: No kernel memory limit support
WARNING: No oom kill disable support
WARNING: No cpu cfs quota support
WARNING: No cpu cfs period support
Experimental: false
Insecure Registries:
 <IP_ADDRESS>/8
Live Restore Enabled: false

This is expected; if a container is part of a service (i.e. it's a "task" for a service created through docker service create ....), then that container is managed/monitored by swarm.
Running docker restart .. is a combination of docker stop .., and docker start ..; Swarm is monitoring those containers, and when it detects a container is stopped (in this case, due to doing a manual docker restart), it starts a new task (container) to replace the "failing" one. The new container becomes visible in docker network inspect on the host that the container is running, and those are the endpoints you see being added.

The containers aren't running as a service.

Ah, thanks for the extra info. I'm able to reproduce this indeed;
$ docker network create -d overlay --attachable foo

$ docker run -d --name foo --net foo nginx:alpine

$ docker network inspect foo --format '{{ json .Peers }}'
[{"Name":"moby-8375ed025e1b","IP":"<IP_ADDRESS>"}]

Then, when restarting the foo container, another peer entry is added;
$ docker restart foo

$ docker network inspect foo --format '{{ json .Peers }}'
[{"Name":"moby-8375ed025e1b","IP":"<IP_ADDRESS>"},{"Name":"moby-8375ed025e1b","IP":"<IP_ADDRESS>"}]

ping @sanimej, could you have a look?

@Michael-Hamburger Thanks for the report. As you have observed having the entry twice is more of a display issue because it won't have an impact on the connectivity. But I will check whats going on.


README and Project Management Improvements

Hi @ctandi and @merlinschumacher
I've been keeping up with issues on the repo here and I'm noticing a trend with users seeking Tasmota support here where they may be better helped by existing support channels. With that may I recommend some additions to the README.

Clarify that this project's function is only to flash firmware of your choice over stock Tuya firmware
Direct users to seek assistance with their custom firmware to their respective projects
Ask that flashing successes and failures be added directly to the wiki instead of opening an issue

For increased visibility these items could also be made part of the issue template.
I'd also ask you to consider the following, as a significant amount of energy and time has been dedicated by several others in the parallel research and development of these techniques, independently of your paper. After this project was published, much of the support for maintaining and supporting the project has come from the community.

Add mentions to parallel projects (#1) and credit contributors in the README

For ongoing support of the project, you may wish to consider asking one or more community members to be a member of this repository so they may better assist with answering and closing issues, pull requests, and maintenance. I would be happy to take on a more active role to keep the project moving and address support issues faster.
Thanks for your consideration
// CK

Everyone flashing Tasmota to a unknow devices is doing that at his own risk
To clarify. There is no support for devices from Tasmota that are not listed on Github side.
Since there are so many different devices....
Everyone flashing Tasmota to a unknow devices is doing that at his own risk!

Hi @Jason2866
I'm not sure what you're getting at, would you please clarify? Sounds like you maybe misunderstood my post, as it does not concern awareness of the risk of flashing custom firmware. tuya-convert is not specific to Tasmota, it can flash any custom firmware on stock Tuya devices.
Many issues opened here are reporting successful flashing (which is the goal of this project) but then going on to ask for support setting up Tasmota (which this project is not responsible for) and they would be better served by directing them to the existing support channels for the firmware they're trying to set up, whatever that may be.

Maybe change from support to help, and have a few different people look at the description? For example, if someone wanted help setting up Tasmota (popular as it is) on a new device, does the Tasmota wiki have a page of advice for that?

Hi @dshanske

Maybe change from support to help, and have a few different people look at the description?

I'm not sure what you're asking.
Whatever the firmware that a user chooses to install, once installed, the job of tuya-convert is done. Any issues after that should be directed to the relevant project. If a user has an issue with Tasmota, there are no actionable items as tuya-convert does not own or maintain Tasmota.

For example, if someone wanted help setting up Tasmota (popular as it is) on a new device, does the Tasmota wiki have a page of advice for that?

Indeed, the Sonoff-Tasmota wiki has several pages dedicated to initial configuration of devices.

I was brainstorming how such an entry in the readme might read.

What i want to say. Just flashing Tasmota to devices means not the will work with.
We (the Tasmota Support Team) cant help with devices we dont know
So it makes no sense to open a issue device xy doesnt work with Tasmota! This issue will be closed.
How to setup devices initial in general is explained in wiki. For general!! questions is Tasmota Discord

I was brainstorming how such an entry in the readme might read.

Ahah, thank you for your input.

Just flashing Tasmota to devices means not the will work with

There is no claim that it will

We (the Tasmota Support Team) cant help with devices we dont know

Neither can tuya-convert. If there is any place to ask for support for any given software, it is from that software's community.

So it makes no sense to open a issue device xy doesnt work with Tasmota! This issue will be closed.

That is your prerogative

How to setup devices initial in general is explained in wiki. For general!! questions is Tasmota Discord

Then this is where users should go for support, not tuya-convert

We have to many users who bricked there devices with a Tuya flash tool and coming to Tasmota and ask for help. We cant give support for that....

I'm not saying that devices bricked through the flashing process itself should be issues of Tasmota. If the issue is with flashing then indeed, tuya-convert is the place to ask for support. If the issue is with setup of Tasmota, tuya-convert is not the place to ask.

If the issue is with setup of Tasmota, tuya-convert is not the place to ask.
And that is the point, we cant help in most cases too. We dont know which GPIO is used for what
or if there is a additional MCU build in for something...
So we are not the place to ask too.  This are 90% of the questions.

If the issue is with setup of Tasmota, tuya-convert is not the place to ask.
And that is the point, we cant help in most cases too. We dont know which GPIO is used for what
or if there is a additional MCU build in for something...
So we are not the place to ask too.  This are 90% of the questions.

Sure, if you wish you limit the scope of your project (a good idea, and exactly what I'm doing here) then you may wish to indicate which devices you will support. I'm not arguing with you here. Again, that is entirely your prerogative and not relevant to this particular issue.
The point made here is only that the scope of tuya-convert does not include setting up the firmware you choose to flash. Issues here should only pertain to the flashing process.

I think where your confusion lies @Jason2866 is that I'm not suggesting the Sonoff-Tasmota community do anything. That is entirely up to you/them. I am simply declaring what the tuya-convert community will not do.

I agree with you 100%   All users should know that the get support here for Software Tuya not for Tasmota Software.  And Tasmota cant provide support for unknown specific device questions.

Great @Jason2866, I'm glad we have found common ground and agree with the approach. Thanks for your input!

The Sonoff User Forum is a perfect place to discuss new devices that have had Tasmota flashed to them.

Resolved


[Chore] : Improve the removal of issuers

What does this PR do?
Removed the call to updateMatchingSchemaIssuers in remove issuers
Added two separate methods below

updateSchemaIssuerWithPortalOwner
bulkUpdateSchemasIssuersWithPortalOwner

Related ticket
Fixes #653
Type of change

[x] Chore
[ ] Bug fix
[ ] New feature
[ ] Documentation update

Check list

[ ] Unit tests for any smart contract change
[ ] Contracts and functions are documented

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 94.20%. Comparing base (3c68622) to head (669ff45).
Report is 3 commits behind head on dev.

Additional details and impacted files
@@            Coverage Diff             @@
##              dev     #666      +/-   ##
==========================================
+ Coverage   94.14%   94.20%   +0.05%
==========================================
  Files          30       30
  Lines         495      500       +5
  Branches      115      115
==========================================
+ Hits          466      471       +5
  Misses         28       28
  Partials        1        1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Docker: unable to run container

I am trying to set up docker in my new laptop but I cant get my container running. I am able to get the docker-tutorial container to run but every time I run other containers they will only load for like one second and go back to excited state.

Actually, does anyone know if docker is required of us? I don't have it on my exam laptop and I'm not sure if I should set it up for finals.

@wjiayis  I opened a new issue about that

@wjiayis, docker is not required. If you can get java 17 working on your laptop feel free to use that.
@ziyi105, I am not sure what that issue is. Can I check if you are using a macbook with M1 or M2 chip?

Hi @danqiye1 and just for your reference, I am using M1 Pro and docker is able to run normally on my desktop, although it occupies a lot of memory space and jshell sometimes fails to start with the command.

@danqiye1 hmm but I am using a windows laptop with a Ryzen 7 processor tho

I did some additional testing and I think it is because the latest docker desktop security update introduced some additional security features. I created a new image that works for me on the new docker desktop. Try downloading that and see if it works.
For the rest who did not encounter this issue, my advice is "if it is not broken, don't fix it".


JSON parser errors when retrieving C++ code from archival memory.

Describe the bug
I have loaded some C++ source code using memgpt load directory .... So far I am loading just one component directory that has just a few source files so that I can experiment and work through issues. The first issue is that the JSON parser has difficulty parsing the response after searching archival memory. My hypothesis is that the code contains character sequences that are not properly encoded/escaped which causes the stringified JSON to be malformed.
I will attach a larger capture from the session, but the initial part of the error message is:
> Enter your message: Search your archival memory for any references to WaitingForAction, which is a C++ class template (a.k.a template class). Please summarize what you know about WaitingForAct
ion.
An exception ocurred when running agent.step():
Traceback (most recent call last):
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/local_llm/json_parser.py", line 154, in clean_json
    data = json.loads(raw_llm_output)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
               ^^^^^^^^^^^^^^^^^^^^^^
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 6 column 33 (char 144)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/local_llm/json_parser.py", line 158, in clean_json
    data = json.loads(raw_llm_output + "}")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
               ^^^^^^^^^^^^^^^^^^^^^^
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 6 column 33 (char 144)

During handling of the above exception, another exception occurred:
...

Please describe your setup

[ ] MemGPT version

What is the output of memgpt version? (eg "0.2.4")

I am using pymemgpt-nightly from the night of Dec 27th/28th. memgpt version doesn't reflect this -- perhaps you can add some suffix to the semver string?
jim.lloyd@jimsm2 memgpt % memgpt version
0.2.10

[ ] How did you install memgpt?
Basically I did:  pip install pymemgpt-nightly
However this was after a pip uninstall pymemgpt and before that I had done pip install pymemgpt followed by pip install pymemgpt[local].

[ ] Describe your setup

What's your OS (Windows/MacOS/Linux)?
Apple Powerbook M2 64Gb ram running Sonoma 14.2.

How are you running memgpt? (cmd.exe/Powershell/Anaconda Shell/Terminal)
Terminal session inside VS Code.

Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
I created the data source using this command:
memgpt load directory --recursive --name test_action_state_machine --input-dir ~/{project}/cpp/eta/action_state_machine

I then ran memgpt and did a /attach and selected test_action_state_machine.
If you're not using OpenAI, please provide additional information on your local LLM setup:
Local LLM details
Everything running locally:

llama.cpp

./server -m models/openhermes-2.5-mistral-7b-16k.Q8_0.gguf -c 16384

text-embeddings-router

model=BAAI/bge-large-en-v1.5
revision=refs/pr/5
text-embeddings-router --model-id $model --revision $revision --port 3000

> Enter your message: You are an expert with the programming language C++. You will be my C++ programming assistant. Your archival memory contains some C++ code now and I want your help to 1) un
derstand that code better, and 2) be able to make enhancments to the code. Please summarize this new instructions and update your core memory.
💭 Locating the user's request in conversation history.
💭 Searching for C++ keywords in archival memory.
⚡🧠 [function] updating memory with archival_memory_search
💭 Updating core memory with the user's request and archival search results.
⚡🧠 [function] updating memory with core_memory_append
> Enter your message: Search your archival memory for any references to WaitingForAction, which is a C++ class template (a.k.a template class). Please summarize what you know about WaitingForAct
ion.
An exception ocurred when running agent.step():
Traceback (most recent call last):
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/local_llm/json_parser.py", line 154, in clean_json
    data = json.loads(raw_llm_output)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
               ^^^^^^^^^^^^^^^^^^^^^^
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 6 column 33 (char 144)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/local_llm/json_parser.py", line 158, in clean_json
    data = json.loads(raw_llm_output + "}")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
               ^^^^^^^^^^^^^^^^^^^^^^
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 6 column 33 (char 144)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/local_llm/json_parser.py", line 162, in clean_json
    data = json.loads(raw_llm_output + "}}")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
               ^^^^^^^^^^^^^^^^^^^^^^
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 6 column 33 (char 144)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/local_llm/json_parser.py", line 166, in clean_json
    data = json.loads(raw_llm_output + '"}}')
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
               ^^^^^^^^^^^^^^^^^^^^^^
json.decoder.JSONDecodeError: Unterminated string starting at: line 6 column 33 (char 144)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/local_llm/json_parser.py", line 171, in clean_json
    data = json.loads(repaired)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
               ^^^^^^^^^^^^^^^^^^^^^^
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 6 column 33 (char 144)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/local_llm/json_parser.py", line 176, in clean_json
    data = json.loads(repaired)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
               ^^^^^^^^^^^^^^^^^^^^^^
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 6 column 33 (char 144)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/local_llm/json_parser.py", line 23, in extract_first_json
    return json.loads(string[start_index : i + 1])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/3.11.6/lib/python3.11/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
               ^^^^^^^^^^^^^^^^^^^^^^
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 5 column 33 (char 143)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/local_llm/json_parser.py", line 180, in clean_json
    data = extract_first_json(raw_llm_output + "}}")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/local_llm/json_parser.py", line 25, in extract_first_json
    raise LLMJSONParsingError(f"Matched closing bracket, but decode failed with error: {str(e)}")
memgpt.errors.LLMJSONParsingError: Matched closing bracket, but decode failed with error: Expecting property name enclosed in double quotes: line 5 column 33 (char 143)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/local_llm/json_parser.py", line 184, in clean_json
    data = clean_and_interpret_send_message_json(raw_llm_output)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/local_llm/json_parser.py", line 70, in clean_and_interpret_send_message_json
    raise LLMJSONParsingError(f"Couldn't manually extract send_message pattern from:\n{json_string}")
memgpt.errors.LLMJSONParsingError: Couldn't manually extract send_message pattern from:

{
  "function": "archival_memory_search",
  "params": {
    "inner_thoughts": "Looking for WaitingForAction.",
    "query": "WaitingForAction",

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/local_llm/llm_chat_completion_wrappers/chatml.py", line 279, in output_to_chat_completion_response
    function_json_output = clean_json(raw_llm_output)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/local_llm/json_parser.py", line 186, in clean_json
    raise LLMJSONParsingError(
memgpt.errors.LLMJSONParsingError: Failed to decode valid MemGPT JSON from LLM output:
=====

{
  "function": "archival_memory_search",
  "params": {
    "inner_thoughts": "Looking for WaitingForAction.",
    "query": "WaitingForAction",
=====

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/local_llm/chat_completion_proxy.py", line 126, in get_chat_completion
    chat_completion_result = llm_wrapper.output_to_chat_completion_response(result, first_message=first_message)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/local_llm/llm_chat_completion_wrappers/chatml.py", line 281, in output_to_chat_completion_response
    raise Exception(f"Failed to decode JSON from LLM output:\n{raw_llm_output} - error\n{str(e)}")
Exception: Failed to decode JSON from LLM output:

{
  "function": "archival_memory_search",
  "params": {
    "inner_thoughts": "Looking for WaitingForAction.",
    "query": "WaitingForAction", - error
Failed to decode valid MemGPT JSON from LLM output:
=====

{
  "function": "archival_memory_search",
  "params": {
    "inner_thoughts": "Looking for WaitingForAction.",
    "query": "WaitingForAction",
=====

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/main.py", line 284, in run_agent_loop
    new_messages, user_message, skip_next_user_input = process_agent_step(user_message, no_verify)
                                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/main.py", line 260, in process_agent_step
    new_messages, heartbeat_request, function_failed, token_warning = memgpt_agent.step(
                                                                      ^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/agent.py", line 683, in step
    raise e
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/agent.py", line 614, in step
    response = self.get_ai_reply(
               ^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/agent.py", line 812, in get_ai_reply
    raise e
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/agent.py", line 793, in get_ai_reply
    response = create(
               ^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/openai_tools.py", line 361, in wrapper
    raise e
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/openai_tools.py", line 334, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/openai_tools.py", line 413, in create
    return get_chat_completion(
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/jim.lloyd/.pyenv/versions/memgpt/lib/python3.11/site-packages/memgpt/local_llm/chat_completion_proxy.py", line 131, in get_chat_completion
    raise LocalLLMError(f"Failed to parse JSON from local LLM response - error: {str(e)}")
memgpt.errors.LocalLLMError: Failed to parse JSON from local LLM response - error: Failed to decode JSON from LLM output:

{
  "function": "archival_memory_search",
  "params": {
    "inner_thoughts": "Looking for WaitingForAction.",
    "query": "WaitingForAction", - error
Failed to decode valid MemGPT JSON from LLM output:
=====

{
  "function": "archival_memory_search",
  "params": {
    "inner_thoughts": "Looking for WaitingForAction.",
    "query": "WaitingForAction",
=====
? Retry agent.step()? No
> Enter your message: /memory

Dumping memory contents:

### CORE MEMORY ###
=== Persona ===
My name is MemGPT.
I am an AI assistant designed to help human users with document analysis.
I can use this space in my core memory to keep track of my current tasks and goals.

The answer to the human's question will usually be located somewhere in your archival memory, so keep paging through results until you find enough information to construct an answer.
Do not respond to the human until you have arrived at an answer.
Just received a detailed explanation of the WaitingForAction template.
I will assist you in understanding C++ code, enhancing it.

=== Human ===
First name: Chad
first name: Jim

### ARCHIVAL MEMORY ###
Jim confirmed his full name, first name: Jim
#include "eta/error/panic.hpp"
#include "eta/chrono/Chrono.hpp"

namespace eta::action_state_machine {
using namespace eta::chrono;

template<typename MachineState, typename DataClock>
typename ActionStateMachine<MachineState, DataClock>::CoverageMap::Map ActionStateMachine<MachineState, DataClock>::CoverageMap::sMap;

template<typename MachineState, typename DataClock>
ActionStateMachine<MachineState, DataClock>::CoverageMap::~CoverageMap()
{
    ETA_DEFINE_LOGGER(logger, "ActionStateMachine", "coverage");

    for (const auto state : wise_enum::range<MachineState>)
    {
        for (const auto status: wise_enum::range<Status>)
        {
            const auto key = std::make_tuple(state.value, status.value);
            auto it = sMap.find(key);
            auto count = it==sMap.end() ? 0 : it->second;
            ETA_DBG(logger, "{}\t{} \t{}\n", count, state.name, status.name);
        }
    }
}

template<typename MachineState, typename DataClock>
float ActionStateMachine<MachineState, DataClock>::CoverageMap::coverage() const
{
    const auto total = wise_enum::size<MachineState> * wise_enum::size<Status>;
    assert(total > 0);
0 : it->second;
            ETA_DBG(logger, "{}\t{} \t{}\n", count, state.name, status.name);
        }
    }
}

...
...

The above shows only some of the C++ code from archival memory. This is from a proprietary code base so I am not at liberty to freely share more code.

I think there are likely issues with parsing JSON when MemGPT is trying to code (generating code), but all of the error traces you seem to be more standard local LLM JSON decoding errors, eg:
Failed to decode valid MemGPT JSON from LLM output:
=====

{
  "function": "archival_memory_search",
  "params": {
    "inner_thoughts": "Looking for WaitingForAction.",
    "query": "WaitingForAction",
=====

^this indicates that the local LLM (dolphin mixtral) outputted that string, which the MemGPT parsing code had trouble turning into valid JSON (which is odd - that looks pretty salvageable to me, but I'll try adding it to a unit test to check).
More related to putting c++ in archival memory, I tried putting the code snippet you included in a directory and ran load + run and wasn't able to reproduce any JSON errors:
Working w/ OpenAI gpt-4 backend:

"Working" w/ quickstart backend:

Just confirmed via pytest it looks like our current JSON parser breaks on that string, will add a patch for it now:
E                                       memgpt.errors.LLMJSONParsingError: Failed to decode valid MemGPT JSON from LLM output:
E                                       =====
E
E
E                                       {
E                                         "function": "archival_memory_search",
E                                         "params": {
E                                           "inner_thoughts": "Looking for WaitingForAction.",
E                                           "query": "WaitingForAction",
E
E                                       =====

memgpt/local_llm/json_parser.py:186: LLMJSONParsingError

Patch for the specific LLM output in your trace is here (so you shouldn't see that exact error anymore): https://github.com/cpacker/MemGPT/pull/739

I do not know if this has been reported yet, but when I talk to the bot in English, there are no JSON errors. However, as soon as I speak in French (thus with accents and characters that do not exist in English), I get a lot of errors. There may be an issue due to this.
Surprising in 2024...


error 500

Expected behavior
Actual behavior
Information about the Issue
Steps to reproduce the behavior

...
...

No information provided or steps to replicate


Create normalize.scss

It's just a copy of normalize.css.
After downloading normalize via npm it creates only normalize.css, so developer needs to change the file extension each time because SCSS supports only .scss files import.

https://github.com/necolas/normalize.css/search?q=scss&type=Issues&utf8=✓


Cannot use MemoryCacheManager

Brief bug description
There is an example in https://github.com/Kentico/kontent-delivery-sdk-net/wiki/Caching-responses#usage-without-di which suggest to use DeliveryClientCache but we can't do that because that class is marked as internal.
var cachedClient = new DeliveryClientCache(new MemoryCacheManager(new MemoryCache(memoryOptions), cacheOptions), client);
Expected behavior
Make the class public or suggest new way to register delivery client with cache (without DI).
Test environment

.NET 6,  16.0.0-beta5

Hi @Enngage,
thanks for finding. We checked this with @gormal (thx for help) and found out, that the problem was probably forgotten update of Wiki. The right approach to create instance of MemoryCacheManager in this example is by usage of public factory class CacheManagerFactory method Create. Please see the updated example in Wiki https://github.com/Kentico/kontent-delivery-sdk-net/wiki/Caching-responses#usage-without-di.
I will also verify, whether there is a need of other update (like to describe usage of this factory more), but this I will do in a future.


externalize the config.go file

Signed-off-by: ayush_gitk<EMAIL_ADDRESS>Description
externalize the config.go file
Notes for Reviewers
Signed commits

[x] Yes, I signed my commits.

I am confused with  DefaultGenerationURL .There are many .yaml  files at this location ,so i don't know exactly which file i should refrence to.

Thanks for your review @b0id .Could you suggest some changes or point out the necessary code that i might have removed .
I mainly referenced this pr from here .

@ayushthe1 Can you remove this directory https://github.com/meshery/meshery-kuma/tree/master/templates/meshmodel/components/traits as well. This was supposed to be some particular version but in some previous propagation change was named to traits

@ayushthe1 The build tests are failing as well.

hey @Revolyssup ,i did this some changes and now the 'Meshery-kuma' workflow is passing but for the the 'Test kuma adapter' workflow i'm getting this error invalid workload of type: KumaMesh .Can you suggest how can i fix this.

i have camel cased the the keys in metadata.json as well as  across  the whole repo so that they remain consistent. If this was not required ,i will undo the changes.
@Revolyssup

Also i have confusion with the DefaultGenerationURL. Could you check once if the correct yaml file has been referenced to.

Hey @Revolyssup ,could you please take a look at this pr and suggest any changes if required.

@ayushthe1 Please take care of the merge conflicts.

@ayushthe1 Please take care of the merge conflicts.

@Revolyssup Thanks for pointing that out. It's fixed now.

@ayushthe1 lint tests failing


Return correct href when bulk assigning/unassigning tags

Fixes #15076
The add_parent_href_to_resultis using the request to form the href assigned to an action result, but with bulk tagging, there is no collection id which is causing the href to be returned incorrectly.
This update appends a :parent_id to the action_result options so that the correct id is referenced and appended to the href
This occurs in Fine as well.
@miq-bot add_label bug, api, fine/yes
@miq-bot assign @abellotti

Thanks @jntullo for the update, this LGTM!! 👍

@jntullo BZ link please

@simaishi BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1460257

Fine backport details:
$ git log -1
commit f9deb1d4116713aeae5a3f95bec4a1294ba1a185
Author: Alberto Bellotti<EMAIL_ADDRESS>Date:   Mon May 22 17:19:36 2017 -0400

    Merge pull request #15083 from jntullo/bulk/incorrect_href_bulk_tag

    Return correct href when bulk assigning/unassigning tags
    (cherry picked from commit da20831cb3e18e74ac84c9c7ae633ca8c50783d7)

    https://bugzilla.redhat.com/show_bug.cgi?id=1460294


Selection-level highlighting of TTS words (initial, for comment)

This adds by-word highlighting for selections that fill within the same block (selections spanning multiple blocks are more complex and will need further exploration). It's rough right now, but at a point where I think comments would be useful. In particular, I am fairly new to TypeScript, and want to make sure code I contribute is appropriately idiomatic in how it's being used.
Thoughts/observations:

the raw Range was added to the ISelectionInfo interface. I suspect this may not be appropriate long-term, and the ISelectionInfo / IRangeInfo interfaces should have additional functionality added to support what's done with the Range in the speak function of TTSModule.
the approach taken uses Splitting.js in a somewhat different way, and does some DOM manipulation to temporarily replace the text within the selection with word-splitting markup, then restore the original content.
I think initial refactoring should focus on extracting a common model of operation for the use case (highlighting individual words as TTS speaks); this is complicated by the realities of how the Selection/Range features work, but for the moment there appear to be three main areas to consider:

speaking and highlighting the entire current resource, word by word (implemented)
speaking and highlighting a selection within a single block (implemented)
speaking and highlighting a selection spanning multiple blocks (not implemented)

@waharnum I am merging this into the TTS branch, for now until the Splitter POC is done will leave it in that branch and we'll make decisions after the Splitter POC.


Missing subprotocols field in the websocket scope

Describe the bug
Trying to use httpx-ws to test GraphQL subscriptions provided by the fastapi + strawberry combo using an example from the Testing ASGI section of the docs.
Establishing WebSocket connection always fails with the following error:
self = <httpx_ws.transport.ASGIWebSocketAsyncNetworkStream object at 0x7f0eee7210>

    async def __aenter__(self) -> "ASGIWebSocketAsyncNetworkStream":
        self.exit_stack = contextlib.ExitStack()
        self.portal = self.exit_stack.enter_context(
            anyio.start_blocking_portal("asyncio")
        )
        _: "Future[None]" = self.portal.start_task_soon(self._run)
        await self.send({"type": "websocket.connect"})
        message = await self.receive()
        if message["type"] == "websocket.close":
            await self.aclose()
>           raise WebSocketDisconnect(message["code"], message.get("reason"))
E           httpx_ws._api.WebSocketDisconnect: (<CloseReason.INTERNAL_ERROR: 1011>, "'subprotocols'")

.venv/lib/python3.11/site-packages/httpx_ws/transport.py:53: WebSocketDisconnect

Turns out, Strawberry relies on Starletter Websocket scope to contain subprotocols entry: https://github.com/strawberry-graphql/strawberry/blob/c6eb8656670ba859dd2dd959c9f7a84e6d80ef84/strawberry/fastapi/router.py#L271C6-L271C6
So it fails with a KeyError("subprotocols").
Comparing the scopes generated by httpx-ws test transport and the actual queries made by GraphQL clients I found these discrepancies:

So seems like for the real calls, subprotocols field is being added to the scopes, while in httpx-ws transport it is not: https://github.com/frankie567/httpx-ws/blob/f0e4d4b124d3ca7b0a85178f8294c22542841ce5/httpx_ws/transport.py#L149-L161
Not being a WebSocket guru, I'm not sure if it is actually an httpx-ws or strawberry bug... :thinking:
To Reproduce
Test code:
from httpx import AsyncClient
from httpx_ws import aconnect_ws
from httpx_ws.transport import ASGIWebSocketTransport

@pytest.mark.anyio
async def test_subscriptions(fastapi_app):
    async with AsyncClient(transport=ASGIWebSocketTransport(fastapi_app)) as client:
        async with aconnect_ws("ws://test/graphql", client, subprotocols=["graphql-transport-ws"]) as ws:
            pass

This raises an error described above.
Expected behavior
WebSocket connection is expected to be established successfully.
Potential fix
If I add something like this to the scope creation code in the httpx-ws transport:
scope = {
    ...,
    "subprotocols": [subprotocol for subprotocol in headers.get("sec-websocket-protocol", "").split(",") if subprotocol],
}

the connection is successful.
Configuration

Python version: 3.11.4
httpx-ws version: 0.4.0
fastapi version: 0.98.0
strawberry-graphql version: 0.190.0

Hi @YaraslauZhylko 👋
Yep, definitely a bug on our side! Actually, ASGIWebSocketTransport "emulates" an ASGI server so we need to make sure all the data is correctly passed in scope, including subprotocols.

Checked it im pur project.
The fix works perfectly.
Thank you.


Disable remove first ListField element

Coverage decreased (-0.1%) to 99.724% when pulling 96eac9102a72f5015be7b9743e420c2ddc1d5520 on ksobolewski-netins:master into a89a03f2ae09a4a6edc8e91210b8e98b572b4b77 on mprzodala:master.

Coverage increased (+0.001%) to 99.862% when pulling 354aa20ab147309aed5e86956cdd933ff719040c on ksobolewski-netins:master into a89a03f2ae09a4a6edc8e91210b8e98b572b4b77 on mprzodala:master.

I dont know why you give only posibility to disable remove first list element. Please change this logic and give someone posibility to declarate minimal lenght and maximum lenght of list.

Remove first element disallow is the most popular case for that action type(demo example: book author >= 1) that's why I created it. I'll change it with your suggestions.

ok thanks. I waiting for your commit.

Coverage increased (+0.006%) to 99.867% when pulling 790f94d24b588bb7865f6e474b6f19a9c6164751 on ksobolewski-netins:master into a89a03f2ae09a4a6edc8e91210b8e98b572b4b77 on mprzodala:master.

Coverage increased (+0.003%) to 99.864% when pulling d81309df150f0a3049b1134f42c1ffdeca847788 on ksobolewski-netins:master into a89a03f2ae09a4a6edc8e91210b8e98b572b4b77 on mprzodala:master.

Thanks for support and quick response 👍


it8xxx2_evb: kernel: tests/kernel/timer/timer_api/kernel.timer.tickless Failed

Describe the bug
During test run on it8xxx2_evb platform, there is a Fail test case: test_sleep_abs. This is output from git bisect:
7a5fcb8c60e2e6ba26704fabc28fe2d4e0106513 is the first bad commit
commit 7a5fcb8c60e2e6ba26704fabc28fe2d4e0106513
Author: Keith Packard<EMAIL_ADDRESS>Date:   Tue Sep 12 23:51:33 2023 -0700
libc/picolibc: Support 'long long' and 'minimal' printf variants

To Reproduce
Steps to reproduce the behavior:

cd <path/to>zephyrproject/zephyr
source zephyr-env.sh
twister -p it8xxx2_evb --device-testing --hardware-map <path/to>/map.yaml -T tests/kernel/timer/timer_api/ -v

Expected behavior
Test pass
Impact
annoyance
Logs and console output
START - test_sleep_abs
Assertion failed at WEST_TOPDIR/zephyr/tests/kernel/timer/timer_api/src/main.c:807: timer_api_test_sleep_abs: (late >= 0 && late <= MAX(2, k_us_to_ticks_ceil32(250)) is false)
expected wakeup at 157, got 160 (late 3)
FAIL - test_sleep_abs in 0.034 seconds

Other test cases pass:
PASS - [timer_api.test_time_conversions] duration = 0.643 seconds
PASS - [timer_api.test_timeout_abs] duration = 0.003 seconds
PASS - [timer_api.test_timer_duration_period] duration = 0.327 seconds
PASS - [timer_api.test_timer_expirefn_null] duration = 0.327 seconds
PASS - [timer_api.test_timer_k_define] duration = 0.703 seconds
PASS - [timer_api.test_timer_period_0] duration = 0.203 seconds
PASS - [timer_api.test_timer_period_k_forever] duration = 0.203 seconds
PASS - [timer_api.test_timer_periodicity] duration = 0.254 seconds
PASS - [timer_api.test_timer_remaining] duration = 0.052 seconds
PASS - [timer_api.test_timer_restart] duration = 0.277 seconds
PASS - [timer_api.test_timer_status_get] duration = 0.002 seconds
PASS - [timer_api.test_timer_status_get_anytime] duration = 0.327 seconds
PASS - [timer_api.test_timer_status_sync] duration = 0.355 seconds
PASS - [timer_api.test_timer_user_data] duration = 0.303 seconds
Environment (please complete the following information):

OS: Linux
Toolchain Zephyr SDK 16.4

@keith-packard Any ideas?

@Chenhongren Can you take a look at this issue?

hi @pkunieck,
The it8xxx2 chip needs more time to resume from the low-power mode and the default CONFIG_PM for it8xxx2_evb is disabled. However, it's enabled in the prj_tickless.conf. We would disable PM for the specific platform(it8xxx2_evb) in testcase.yml to address this issue.(disable-pm-for-it8xxx2-evb)
Is this okay for you?

This PR: #69891 seems to be working, tests pass with those changes


Extendable 'SetupWallet' method.

Need to add functionality to this method in other PRs to write more tests for the WalletTransactionHandler. Not keen on adding to that multiple return.

@codingupastorm Can you check please, I don't the solution builds?


Feature request - Gem socketing window (TBC Classic)

Please add! ^_^

@johanruda - I just added gem socket window for tbc classic in v1.0.14. I do not have any gear with socket, though, so I'm not able to test. Can you confirm it's working for you?

It's working perfectly! Thank you! 😁


[kinect2_head_kinect2_bridge-2] process has died

Hi,
Whenever I try to run kinect2_bridge i receive the error below repeatedly. I am running kinect2_bridge on Ubuntu 14.04 (on a dual boot computer alongside Windows 10).
While trying to get to the bottom of this, I noticed that kinect2_bridge is not receiving any depth data. However, when I run the kinect sdk depth test in Windows 10 on the same computer, it seems to detect depth data via my USB 3.0 fine.
Any ideas as to what might be the problem or what I might do to explore the problem further? I've been hacking at it for hours...
I am extremely new  at ros, so if I'm leaving out any relevant information please let me know and I'll send it over.
Thanks!
SUMMARY
PARAMETERS

/kinect2_head_kinect2_bridge/sensor_id:
/kinect2_head_kinect2_bridge/sensor_name: kinect2_head
/kinect2_head_points_xyzrgb_highres/queue_size: 2
/kinect2_head_points_xyzrgb_lowres/queue_size: 2
/rosdistro: indigo
/rosversion: 1.11.16

NODES
/
kinect2_head_kinect2_bridge (kinect2_bridge/kinect2_bridge)
kinect2_head_points_xyzrgb_highres (nodelet/nodelet)
kinect2_head_points_xyzrgb_lowres (nodelet/nodelet)
auto-starting new master
process[master]: started with pid [2712]
ROS_MASTER_URI=http://localhost:11311
setting /run_id to 599687aa-938b-11e5-be87-5c514f70e57c
process[rosout-1]: started with pid [2725]
started core service [/rosout]
process[kinect2_head_kinect2_bridge-2]: started with pid [2728]
process[kinect2_head_points_xyzrgb_lowres-3]: started with pid [2729]
process[kinect2_head_points_xyzrgb_highres-4]: started with pid [2730]
[DepthRegistration::New] Using OpenCL registration method!
[DepthRegistration::New] Using OpenCL registration method!
[OpenCLDepthPacketProcessor::init] ERROR: clGetPlatformIDs(-1001)
terminate called after throwing an instance of 'cl::Error'
what():  clGetPlatformIDs
[kinect2_head_kinect2_bridge-2] process has died [pid 2728, exit code -6, cmd /home/shanee/workspace/ros/catkin/devel/lib/kinect2_bridge/kinect2_bridge __name:=kinect2_head_kinect2_bridge __log:=/home/shanee/.ros/log/599687aa-938b-11e5-be87-5c514f70e57c/kinect2_head_kinect2_bridge-2.log].
log file: /home/shanee/.ros/log/599687aa-938b-11e5-be87-5c514f70e57c/kinect2_head_kinect2_bridge-2*.log
[kinect2_head_kinect2_bridge-2] restarting process
process[kinect2_head_kinect2_bridge-2]: started with pid [2810]
[DepthRegistration::New] Using OpenCL registration method!
[DepthRegistration::New] Using OpenCL registration method!
[OpenCLDepthPacketProcessor::init] ERROR: clGetPlatformIDs(-1001)
terminate called after throwing an instance of 'cl::Error'
what():  clGetPlatformIDs

Hello @sshonig ,
This problem is not a problem with kinect2_bridge but with freenect2, but anyway I'll try to help you here!
It seems that the problem is with OpenCL, so you need to read carefully the installation procedure about OpenCL with libfreenect2, here at paragraph 5:
https://github.com/OpenKinect/libfreenect2#debianubuntu-1404
Once you have done so, recompile and reinstall libfreenect2.
In libfreenect2/build do :
make && sudo make install
Then you can try running the Kinect2 by using Protonect.
In libfreenect2/build/bin try to run these commands:
./Protonect cpu
./Protonect cl
./Protonect gl

That way you will try to run the Kinect2 with the cpu, opencl and opengl.
If one of these fails, you can try to fix it by reading again the installation instructions, or you can post an issue to libfreenect2.
If you are able to run the kinect2 with Protonect but not with kinect2_bridge @wiedemeyer will try to help you here
Good luck !

no updates, so I close the issue

@wiedemeyer @JimmyDaSilva @sshonig @mintar
Hi, I just met the same problem with @sshonig , but i am using NVIDIA Jetson TX1, the OPENCL can't support it! what should i do, i feel very sad, anyone kown the solutions?


remove subscribe button in header

What type of PR is this?
/kind cleanup

What this PR does / why we need it:
remove subscribe button in header
Which issue(s) this PR fixes:

Fixes #
How is this PR tested:

[x] unit test
[ ] e2e test
[ ] other (please specify)

Special notes for your reviewer:
Does this PR introduce a user-facing change?

- remove stale subscribe button from upper left header

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

Fixed in https://github.com/Edgenesis/shifu.dev/pull/311


package.json Version out of sync with Tag

package.json still lists: "Version": 0.5.2
should be: "version": 0.6.0

We have not merged 0.6.0 into master.  Our release strategy is to only merge tags into master that have been proven to be stable.  This means that our next push to master will most likely be 0.6.1.


Add source character encoding definition.

@jputrino @richbrowne
What issues does this address?
What's this change do?
Adds character encoding definition to top of files: # coding=utf-8
Where should the reviewer start?
Any file.
Any background context?
Previous change to add F5 trademarks included non-ASCII characters which causes imports to fail.

Looks good to me! I noticed that the copyright blurb is missing from one of the modules (lbaasv2/drivers/bigip/icontrol_driver.py); I created a new issue for it ( #54 ) and should be able to get that fix in today.

Jodie - Thanks for catching the missing copyright.
From: Jodie Putrino <notifications<EMAIL_ADDRESS>Reply-To: F5Networks/f5-openstack-agent <reply<EMAIL_ADDRESS>Date: Wednesday, March 30, 2016 at 9:48 AM
To: F5Networks/f5-openstack-agent <f5-openstack-agent<EMAIL_ADDRESS>Cc: Jeff Longstaff <j.longstaff<EMAIL_ADDRESS>Subject: Re: [F5Networks/f5-openstack-agent] Add source character encoding definition. (#53)
Looks good to me! I noticed that the copyright blurb is missing from one of the modules (lbaasv2/drivers/bigip/icontrol_driver.py); I created a new issue for it ( #54https://github.com/F5Networks/f5-openstack-agent/issues/54 ) and should be able to get that fix in today.

You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHubhttps://github.com/F5Networks/f5-openstack-agent/pull/53#issuecomment-203497693


Feature/bphh 1236/external api admin screens

Description
Feature/bphh 1236/external api admin screens

Adds external api key index screen with a table of api keys for jurisdictions. Viewable as super admin and admin
Adds ability to enable or disable api keys for a whole jurisdiction as a super admin
Add's ability to create, update, expire and revoke api key's as super admin and review manager is api keys are enabled for the jurisdiction

What type of PR is this? (check all applicable)

[x ] 🍕 Feature
[ ] 🐛 Bug Fix
[ ] 📦 Chore (Release)
[ ] ✅ Test

Related Tickets & Documents

https://hous-bssb.atlassian.net/browse/BPHH-1236
https://hous-bssb.atlassian.net/browse/BPHH-1237

Steps to QA

First enable api keys for a jurisdiction as a Super admin

Navigate to jurisdictions page
Select any jurisdiction and click manage -> External Api keys
Should navigate to index screen
Enable api key's using the switch top right of screen

To perform CRUD operations for external api key when api keys are enabled for the jurisdiction:

Navigate to external api key index page

As a super admin - Navigate to Jurisdictions -> Click Manage -> External API keys
As a review manager - Navigate to Configuration Management -> Click edit on "Manage Configuration Management"

To Create click "Create new API key " button which should open up a modal

Add required details then click "Create"
On success should navigate to Manage modal

To Manage/Edit External API key, click "Manage" on any of the table keys

Should open up the Manage modal
Should be able to change everything except token, and revoked at
Should be able to revoke if already not revoked.  Should show a confirmation modal
And save

https://github.com/bcgov/HOUS-permit-portal/assets/32022335/a7480500-b3ae-4390-a26d-f8e3a4d19c40

to disable api keys for a jurisdiction as a Super admin

Navigate to jurisdictions page
Select any jurisdiction and click manage -> External Api keys
Should navigate to index screen
Disable api key's using the switch top right of screen
The Create and Manage buttons should become disabled
Should be the same for review manager once disabled, but should also show a warning banner

https://github.com/bcgov/HOUS-permit-portal/assets/32022335/af43b674-aaac-4cbb-9ad2-926ebcd9767a

please add ACTIVE_RECORD_ENCRYPTION env vars to example


Draw a route (lines)

Hi!
I would like to draw a polyline between two points, like a route.
This path would be shown in the camera view. Is it possible?
There is a topic in the forum (http://beyondar.com/node/99) which is exactly what I'm looking for. Unfortunately it seems that it has not been finalized yet.
Is there any progress on this feature?
I really need this in my project.
Thank you!

Hi, did you manage to resolve this?, I also need this feature for my project. Thanks!

Hi. I'm having a similar issue, to draw a lined pathed on the road instead of floating GeoObjects.
Can you help me on this? That would be a great favor and highly appreciable.