Added support for serialization and deserialization

This PR was made from the comments from @zebuline who warned me that (if I understood correctly) when deserializing data, the readonly field was present (when it should be absent).
There was also an ambiguity on the serialization behavior : read-only fields are fields and should be present in the serialization results.
I've added tests that check the (de/)serialization behaviour, and I did not reproduce such behaviour but I encountered a bug in the deserialization part, that this PR would solve.
The last question is about an option to allow skipping read-only fields when serializing the data. Thing is, I'm affraid this should more be the responsibility of a specific Serializer than an global option on the ReadOnlyMeta (which is the only interaction point so far). Would it be ok to examine a SkipReadOnlyFieldsSerializerMixin class that would be provided as part of this lib's API for the user to create their own Serializer ? (they would have to register it themselves though). I feel this coud be part of another PR.
@zebuline, I would appreciate your input and/or more details on this matter.
Of course, as soon as this gets merged, I'll do a release.

Current coverage is 100% (diff: 100%)

Merging #4 into master will not change coverage

@@           master    #4   diff @@
===================================
  Files           3     3
  Lines          39    43     +4
  Methods         0     0
  Messages        0     0
  Branches        0     0
===================================
+ Hits           39    43     +4
  Misses          0     0
  Partials        0     0

Powered by Codecov. Last update ddf087c...85256b5

:+1:

My bad, I added django_readonly_field in INSTALLED_APPS as mentioned in the readme, instead of django_readonly_field.apps.ReadOnly (as done in the test project :p)
And I added readonly instead of read_only attribute in ReadOnlyMeta
BTW, with the last release (0.1.1) I cannot run the command loaddata; this PR fixes the bug.


Channel closed error when using ssh.execute_command on some devices

Hi,
First of all, thank you so much for creating this fantastic plugin. It has been incredibly useful for controlling and monitoring devices in Home Assistant.
However, I have encountered an issue when using ssh.execute_command on some devices. The error message I receive is:
Channel closed
I found a related issue on the Paramiko repository: paramiko/paramiko#1444. It suggests that using invoke_shell instead of the existing exec_command method might resolve this problem.
Could you please consider trying this approach?
Thank you for your help!
Best regards,

Hi, thanks for your message! I will take a look into this and see if there is any solution to it. Not sure if invoke_shell can be used for this purpose, but I will try!

Not entirely sure if I should open a new issue or just reply to this one. My situation/error is a little different but sounds like the core underlying issue is probably the same with the device requiring a shell/interactive session. A post in the paramiko issue linked above seems to indicate this is common with ssh servers running on network switches
I'm trying to use this with a Brocade ICX 6450 switch and though it will connect when setup, and commands I try to test with ssh.execute_command in developer tools gives me the same error (stdout) "\x01Protocol error, doesn't start with scp!".
Here's a post I found detailing the issue on this particular Brocade line of devices

I added an experimental "invoke shell" option to the device setup and it seems to work fine with the default commands. It is a bit more complicated to extract the outputs, so there might be still some issues with some commands or with some specific devices. Please give it a try and let me know if it works in your case!

Did not seem to work in my case:
results:

device_id: d8084862cd9fc1b657d0227f1b50f18e
device_name: Brocade6450
success: false
error: Not connected

Status for it shows SSH Status and Network Status as connected, but I get that error. Turning on "Disconnect between commands (experimental)" had the same result.

Hm, thats disappointing.... Could you enable debug logging for SSH integration, execute a command and post the relevant logs?

I think I got the relevant logs. I tried just running "show clock" command multiple times in a row:
2024-11-09 14:05:21.633 DEBUG (SSH_0) [custom_components.ssh] Brocade6450: state.connected => False
2024-11-09 14:05:21.634 DEBUG (MainThread) [custom_components.ssh] Brocade6450: echo '' => Failed to execute command (SSH session not active)
2024-11-09 14:05:23.791 DEBUG (SSH_0) [custom_components.ssh] Brocade6450: state.connected => True
2024-11-09 14:05:23.793 DEBUG (MainThread) [custom_components.ssh] Finished fetching Brocade6450 state data in 2.172 seconds (success: True)
2024-11-09 14:05:53.632 DEBUG (SSH_0) [custom_components.ssh] Brocade6450: state.connected => False
2024-11-09 14:05:53.634 DEBUG (MainThread) [custom_components.ssh] Brocade6450: echo '' => Failed to execute command (SSH session not active)
2024-11-09 14:05:55.775 DEBUG (SSH_0) [custom_components.ssh] Brocade6450: state.connected => True
2024-11-09 14:05:55.777 DEBUG (MainThread) [custom_components.ssh] Finished fetching Brocade6450 state data in 2.156 seconds (success: True)
2024-11-09 14:06:06.542 DEBUG (SSH_0) [custom_components.ssh] Brocade6450: state.connected => False
2024-11-09 14:06:06.543 DEBUG (MainThread) [custom_components.ssh] Brocade6450: show clock => Failed to execute command (SSH session not active)
2024-11-09 14:06:27.707 DEBUG (SSH_0) [custom_components.ssh] Brocade6450: state.connected => True
2024-11-09 14:06:27.708 DEBUG (MainThread) [custom_components.ssh] Finished fetching Brocade6450 state data in 2.087 seconds (success: True)
2024-11-09 14:06:57.635 DEBUG (SSH_0) [custom_components.ssh] Brocade6450: state.connected => False
2024-11-09 14:06:57.637 DEBUG (MainThread) [custom_components.ssh] Brocade6450: echo '' => Failed to execute command (SSH session not active)
2024-11-09 14:06:59.832 DEBUG (SSH_0) [custom_components.ssh] Brocade6450: state.connected => True
2024-11-09 14:06:59.834 DEBUG (MainThread) [custom_components.ssh] Finished fetching Brocade6450 state data in 2.212 seconds (success: True)
2024-11-09 14:07:14.533 DEBUG (SSH_0) [custom_components.ssh] Brocade6450: state.connected => False
2024-11-09 14:07:14.534 DEBUG (MainThread) [custom_components.ssh] Brocade6450: show clock => Failed to execute command (SSH session not active)
2024-11-09 14:07:17.809 DEBUG (MainThread) [custom_components.ssh] Brocade6450: show clock => Not connected
2024-11-09 14:07:21.141 DEBUG (MainThread) [custom_components.ssh] Brocade6450: show clock => Not connected
2024-11-09 14:07:27.492 DEBUG (MainThread) [custom_components.ssh] Brocade6450: show clock => Not connected
2024-11-09 14:07:31.734 DEBUG (SSH_0) [custom_components.ssh] Brocade6450: state.connected => True
2024-11-09 14:07:31.736 DEBUG (MainThread) [custom_components.ssh] Finished fetching Brocade6450 state data in 2.115 seconds (success: True)

Thanks! Looks like its not using the new execute method, you did turn on the "invoke shell" option in the device configuration, right?

Yes, it's enabled. Without it enabled I still just get the same "protocol error, doesn't start with scp!" error the switch returns when trying to execute remote SSH commands to it without a shell.

First of all, thank you for releasing an experimental version so quickly.
I have tested it and have some feedback:
When using actions in Home Assistant to execute commands, the good news is that the command successfully executes on the target device. However, I still receive a 'channel closed' message. I traced the code in the zhbjsh/ssh-terminal-manager project in the ssh.py file, and I noticed this block of code:
https://github.com/zhbjsh/ssh-terminal-manager/blob/f6c2b13680fb2d0025308213c13a1e9620d3b295/src/ssh_terminal_manager/ssh.py#L128C1-L131C52
try:
    if self._invoke_shell:
        output = self._execute_invoke_shell(string, timeout)
    output = self._execute(string, timeout)

It seems that the result returned is actually from self._execute and not self._execute_invoke_shell, so the 'channel closed' message is expected. I tried adding an else statement, and the behavior returned was:
results:
  - device_id: d7c54db5adbf3eac0886341f02eaa316
    device_name: hk-dvr
    success: true
    command: cat /proc/uptime | awk '{print $1}'
    stdout:
      - ""
      - ""
      - ""
      - BusyBox v1.29.1 (2019-08-12 16:25:51 CST) built-in shell (ash)
      - Enter 'help' for a list of built-in commands.
      - ""
      - "[root@dvrdvs ~] # cat /proc/uptime | awk '{print $1}'"
      - "2935309.75"
      - "[root@dvrdvs ~] # "
      - "[root@dvrdvs ~] # echo \"__exit_code__|$?|%errorlevel%|$ LastExitCode|"
    stderr: []
    code: 0

I was expecting the result to be just 2935309.75, but this is indeed a bit challenging.

And I was wondering why the error message from @cr4zymanz0r seemed to be from the old _execute method.... Looks like i released this a bit too quick!😂
Thanks for pointing me at the right direction @msmarks, I will fix it the next days as soon as I have time and try to clean up the output as well!

Ok, new version is out! Now it should run the correct function and hopefully cleanup your output in a useful way, at least it did so with my linux and windows testing devices. Try it out!

I tried out the latest version and it works for single line commands! I haven't done a ton of testing yet, but the SSH connection seemed unreliable unless I also turned on "Disconnect between commands (experimental)". That might just be something with my network switch though.
Though not directly related to this issue, now that I've got a basic single line command working is there a way to have it run multiple commands in the same session? For instance, one of my main goals is to be able to enable/disable PoE on certain ports on the switch. To do that with an SSH session to the device I would have to do this:
enable
configure terminal
inline power ethernet 1/1/16
exit
exit
exit
This elevates privileges, elevates again for higher configuration options, turns on PoE on ethernet port 16, exits configuration terminal, exits 'enable', then exits the session. Not sure if the exit commands are really needed or best practice when running in this integration, but even if not I still need to execute the 1st 3 commands in the same session. Am I overlooking something obvious, or is there a way to denote multiple lines of commands to be ran in a single ssh.execute_command?

So usually on Linux I would do something like command1 && command2 && command3, as that should also return the correct error code. In your case that probably doesn't work and I guess command1; command2; command3 is also not an option. Although this integration is not really made to run whole scripts, with the new invoke shell option you might actually be able to make it work by just adding line breaks between your commands. You can do that in YAML-mode like this:
action: ssh.execute_command
data:
  command: |-
    enable
    configure terminal
    inline power ethernet 1/1/16
    exit
    exit
    exit

Currently this won't give you any output, but if you don't need that, it could be a solution.
Also, if you could post a logfile without "Disconnect between commands" I could try to figure out why the connection is unreliable.

Thank you for creating this fantastic plugin. Recently, I have been a bit unwell and haven't had much time to test extensively. In my simple tests, I found that there might be an issue with some embedded devices where, after SSH login, there is no bash, only a simple sh, and PIPESTATUS is not defined. I will test more thoroughly once I feel better.

No worries, take care of yourself! I added PIPESTATUS to the final echo command, because there were problems with some built-in commands that would return status code 0 but output error messages. Unfortunately with invoke_shell stderr will just appear in stdout, so we need a way to tell if one of the piped commands failed. With PIPESTATUS it should do so at least in bash/zsh, if your shell doesn't support it, I expect you will always get status code 0...
Besides that, are you getting a correct output from your commands now?

I changed the implementation of invoke_shell a little bit in version 1.0.1, hopefully it doesn't break anything for you. It should be possible now to get a correct output when executing multiple commands in a row like in the example from @cr4zymanz0r. Let me know how it works!


Business Glossary Config Error

Describe the bug
I input config:
source:     type: datahub-business-glossary     config:         file: 'https://githubusercontent.com/imquocanh/DataHub/main/test_business_glossary-300323-v3.yml'         enable_auto_id: false
But when I run, I reviced error:
pydantic.error_wrappers.ValidationError: 1 validation error for BusinessGlossarySourceConfig file   file or directory at path "https:/githubusercontent.com/imquocanh/DataHub/main/test_business_glossary-300323-v3.yml" does not exist (type=value_error.path.not_exists; path=https:/githubusercontent.com/imquocanh/DataHub/main/test_business_glossary-300323-v3.yml)
File url had change form: https://githubusercontent.com/imquocanh/DataHub/main/test_business_glossary-300323-v3.yml to "https:/githubusercontent.com/imquocanh/DataHub/main/test_business_glossary-300323-v3.yml'"
Please, help me.
THanks

This is likely the result of the acryl datahub library not receiving the recent changes that was merged 2 days ago - in the old code, paths is defined as pydantic.filepath type so it transforms any url by removing a slash.

Hi,
What do I do for resolve problem ?
Thanks.

looking at pyi, the library isnt updated yet. you probably need to clone the repo and pip install -e . instead.
just to confirm@asikowitz, the python library is only updated at next release right

Thank.

Yup, our last pypi release was on Mar 27, while the ability to use urls as files was only added Mar 29. We should have a new release out within a week. Sorry for the inconvenience!

We have a new release (v<IP_ADDRESS>) out with those changes.


Missing basic matchers

TODO:

[x] containerWith - partially matches iterables (arrays/sets)
[x] arrayWith - partially matches only arrays
[x] objectwith - partially matches objects

Example:
const container = new Set()
set.add('beer')
expect(container).toEqual(expect.containerWith('beer'))

expect(['apple', 'beer']).toEqual(expect.arrayWith('apple'))

expect({a: 1, b:2, c: 3}).toEqual(expect.objectWith({c: 3}))

Later we can expand arrayWith and containerWith to match multiple values like:
expect(['apple', 'beer']).toEqual(expect.arrayWith('apple', 'beer'))

Jest has toContain and it works with arrays and iterables: https://jestjs.io/docs/en/using-matchers#arrays-and-iterables
Chai has aliases like: .includes, .contain, and .include, which we probably shouldn't try to mimic 😆

@krzkaczor I will implement arrayWith and open a PR once the move to the monorepo has been finished.
Should other iterables like Sets and Maps be considered for matchers?

Should other iterables like Sets and Maps be considered for matchers?

I think, not necessary now... Personally, I don't use these Sets too often, people can use containerWith for now.

@krzkaczor as I have already implemented arrayWith, I will implement objectWith

Closed by
https://github.com/earl-js/earl/pull/71
https://github.com/earl-js/earl/pull/72
https://github.com/earl-js/earl/pull/74


fix field name in example

Q
A

Bug fix?
no

New feature?
no

API breaks?
no

Deprecations?
no

License
Apache 2.0

What's in this PR?
Fix error in documentation by parser filter.
Why?
Given example not work. Logging operator is failing with message like this:
"logger":"controller-runtime.controller","msg":"Reconciler error","controller":"logging","request":"/default","error":"failed to create model: failed to create filter with index 0 for flow test: no plugin config available for filter"

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Added CHANGES.md file.

Codecov Report

Merging #59 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #59   +/-   ##
=======================================
  Coverage   91.24%   91.24%
=======================================
  Files          31       31
  Lines        1428     1428
=======================================
  Hits         1303     1303
  Misses        125      125

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 36ba5a2...f19f720. Read the comment docs.


conflict with plugin unocss

please complete the following information:

OS version:Win 10
Editor version [e.g. 1.45.0-insider]:1.72.0
Programming language: vue
TabNine extension version:3.6.15

Conflict in using plugin unocss
The prompt that causes unocss needs to enter '- 'once, then delete it, and then enter '-' to get a prompt.
gz#18576

Hi! We added this to our backlog.
Thanks for bringing this to our attention.


Printing Report

Hello,
How do you print the report generated by django-slick-reporting? I don't see a print button on the generated report or do I have to improvise?
Best regards.

Hello ..
For printing, i believe yes, you'd to improvise. At least for now .

Hello Ramez

For printing, i believe yes, you'd to improvise. At least for now .

Thank you for your response.
I like the idea behind Django-Slick-Reporting. Please keep up the good work.
Best regards.


Catch opensim feature branch up to main.

This PR is strictly redundant with #1009, but separates out the parts of #1009 that involve catching up to main so as to reduce noise in that PR.

Since this is a PR to a feature branch, and only serves to catch up to main, I'll go on and merge it with the e2e failing — the failures are due to an outage unrelated to this PR.
@swernli: Thanks for your approval on this!


OpenQA.Selenium.Chrome.ChromeMobileEmulationDeviceSettings was removed in Selenium 4x version

Hi,
I used fluent configuration for AtataContext before, now decided to choose json config,
that throws me error:
System.TypeLoadException : Could not load type 'OpenQA.Selenium.Chrome.ChromeMobileEmulationDeviceSettings' from assembly 'WebDriver, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null'.
In Selenium 4x this class was moved, when I use Selenium 3x, all is fine
Do you have plans about supporting Selenium 4x, or adding some stub to avoid such error?
Thanks

Hi,
Right, current Atata v1.14.0 doesn't support WebDriver 4, only 3x. But I'm working on Atata v2 that will use WebDriver 4. For now, if you can, stay with WebDriver 3 before the v2 of Atata.

Both those options work for me. StackOverflow is probably more preferable, as I believe it will be easier to find SO question for other people with similar problem.

Got you, thanks!


onMomentumScrollEnd and OnIndexChanged not firing

On Windows 10

react-native-swiper v1.5.12
react-native v0.48.4

Oddly enough I have the same code on two pages, with just basic views, and on one it fires, and on the other it doesn't. I'm getting no logs for anything.
I can't fork the code because it's private, but this is the gist of the code.
Working code
render() {
  return (
    <View style={styles.container}>
        <Header title={this.state.title} onPress={()=>this.props.navigation.goBack()}/>
        <View style={styles.body}>
            <View style={styles.routes}>
              <Swiper loadMinimal={true} loadMinimalSize={4} showsPagination={false}>
                  {this.display()}
              </Swiper>
            </View>
            <View style={styles.alpha}>
                {this.displayAlpha()}
            </View>
          </View>
      </View>
  )

Not working code
onMomentumScrollEnd = (e, state, context) => {
    Reactotron.log(this.state.List[state.index].title)
    Reactotron.log(true, true)
}
render() {
  return (
     <View style={styles.container}>
       <Header title={this.state.title} onPress={()=>this.props.navigation.goBack()} navigate={this.props.navigation.navigate}/>
       <View style={styles.body}>
          <View style={styles.routes}>
             <Swiper loadMinimal={true} loadMinimalSize={4} showsPagination={false} onMomentumScrollEnd={this.onMomentumScrollEnd}>
                   {this.display()}
             </Swiper>
          </View>
          <View style={styles.alpha}>
            {this.displayAlpha()}
          </View>
        </View>
      </View>
  )
}

this.display() returns an array of basic  like components.

Do you think you can reproduce the bug with one of our examples ? To do so, you need to fork this project and change one example.
It would help me a lot as I am really busy now...

I'll try. I know at least onTouchScreenCapture works.

One thing I just found out, was that the swiper itself isn't working normally, it's loading at index 0 every time.

Yes even in my case also it is not working. It is working fine if we are using static data, but not working when list is dynamic

Well that was the root of my issue, retrieving the data on the screen caused it not to work, so I retrieved the data earlier and then propped it to the screen and it works. My app runs a bit slower now (at least on emu)

should this issue be closed then?

I suppose I can close it, but I wonder if I should make a PR about using static data (preloaded) and/or leave it open  as suggestion for future updates to address

that would be awesome. thank you! I was looking at the issue description and found that you actually found a method to resolve it

@yashaRO you can check the data to see if it's ready before rendering to avoid this issue.
`render() {
    if (this.siteList.length <= 0) {
        return (
            <View style={styles.bgViewStyle}>
            </View>
        )
    }

    return (
        <View style={styles.bgViewStyle}>

            <Swiper paginationStyle={styles.paginationStyle}
                    removeClippedSubviews={false}
                    onIndexChanged={this.onIndexChanged.bind(this)}>

                {
                    this.siteList.map((item, index) =>
                        <View style={styles.wrapperStyle} key={index}>
                            <Text style={styles.siteNameTextStyle}>{item.name}</Text>
                        </View>
                    )
                }
            </Swiper>

        </View>
    )
}`

Oddly enough, I structured my code like that on some pages, but didn't think to do that on the swiper pages. I'm fine with preloading, but I'll try that method as well.


table 5902 Service Line & table 5964 Service Contract Line - Make local functions external

Please make the following functions externally accessible. Thank you.

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.


Optimize split, _vjpConcatenated, tiled and _vjpTiled

Avoid more temporary tensors.

@pschuh Please take a look.


As of v3.75 s3 connections fail with signature erorrs

Describe the bug
After updated to 3.75, my s3 client (CloudMounter, s3Drive, others) on Mac no longer can connect to my Seaweed S3 server. Error:
"The request signature we calculated does not match the signature you provided. Check your key and signing method. error -1.)"
My s3 server is behind a kubernetes loadbalancer (cilium) with cilium ingress. I'm accessing via a Cloudflare tunnel - both access from the tunnel and direct through the ingress url fail the same way.

how to reproduce it?

weed command args:
-logtostderr=true
-v=4
s3
-config=/auth/config.json
-ip.bind=<IP_ADDRESS>
-port=8333
-port.grpc=18333
-filer=seaweedfs-filer:8888
-allowEmptyFolder

When connecting directly to a forwarded kubernetes port on localhost, it works. It fails when using an ingress (default cilium ingress, no options).

Connecting using CloudMounter and s3drive on mac - both standard configurations that worked in 3.74 now fail in 3.75.

I'm guessing its something in https://github.com/seaweedfs/seaweedfs/pull/6092/commits/eb51c6ac02eaf73fcac460461592d389cd4aca4d#diff-e859e2f69804cf2ad0be2840401299a6fda63ddbb76b965785693a0f7afa9440R688

how to reproduce step 2 or 3?

I'm guessing its something in eb51c6a#diff-e859e2f69804cf2ad0be2840401299a6fda63ddbb76b965785693a0f7afa9440R688

Yes. The code change was added to make it easier to add a proxy/loadBalancer. Not sure why it is doing the opposite.

Ingress:
spec:
ingressClassName: cilium
rules:

host: sws3.c.myhost.net
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: seaweedfs-s3
port:
number: 8333

I'm just using the hostname above and my access key/secret key to configure both CloudMounter and s3Drive apps on mac - see:

Yeah, the code change made my working proxy/load balancer no longer work :-)
3.74 works perfectly with those defaults. The cilium ingress is just the default ingress setup that works for everything. I used the bitnami helm chart, but also reproduced it with the seaweedfs helm chart. Tried creating a manual ingress rule as well. For whatever reason the new change doesn't seem to work with default ingress setups.

Not familiar with kubernetes. Maybe you can debug by adding some println around the lines in the PR.

Will do some debugging, but I believe the default Helm chart in the repo now won't work for S3 with this change.

Will close this - it works in a regular docker container just not in Kubernetes

Will close this - it works in a regular docker container just not in Kubernetes

This is really interesting.why is that?

With the docker container I'm connecting just to the IP, so no issues with ingress/X-Forward-For or X-Forward-Host headers. But in Kubernetes when using ingress (vs. the port mapping) something with the headers appears to not be set properly. There might be a configuration parameter on Kubernetes to make it work, but my ingress reverse proxy provider (cilium) didn't have anything that looked promising. I bet nginx would make it easier, but I don't have cycles to dig in further.

I'm having the same issue with 3.75 and regular Docker containers (it's not Kubernetes-related): "The request signature we calculated does not match the signature you provided. Check your key and signing method".
My setup is simple:
1 master
1 volume
1 filer -s3
Accessing the S3 Gateway from the same host works fine. I get the error message as soon as I try to access the host from a different machine through my company proxy (with or without credentials enabled). I pinned down the version for now.
@gsxdsm Did you figure out a way to make things work with the latest image ?

@amoutaux how do you configure the proxy?


3382 group product listings

Revised: simplify community code per conversation at all-hands. This supersedes the "exhibit" tag that had been discussed before as well as PR #3393.

Delete capability to unlist groups.
Prohibit management groups.
Remove "allow_view" functionality. All groups allow view from communities.
Define a group product as a published resource owned by a member of the group.
Define a community product likewise.

Pull Request Checklist:

[x] Positive Test Case Written by Dev

[x] Automated Testing

[x] Sufficient User and Developer Documentation

[ ] Passing Jenkins Build

[ ] Peer Code review and approval

Positive Test Case

[Enter positive test case here] There is no direct exposure in the UI at this time. That will come with community code.

@pkdash another fix is to use the test Client library. This would make the request internal.


CNS*2022 satellite tutorial on Elephant

Hi everyone,
The INCF/OCNS Software WG is organising a set of free online satellite tutorials ahead of CNS*2022 in the week of June 27--July 1.
Given how useful Elephant is, I was wondering what you thought of doing a session on Elephant too? (There's one planned on Neo/NWB also)
You can see the currently planned sessions etc here:
https://github.com/OCNS/SoftwareWG/milestone/3
The announcement post is here:
https://ocns.github.io/SoftwareWG/pages/software-wg-satellite-tutorials-at-cns-2022.html

Hello,
We're asking everyone for confirmation now---could you please confirm if you'd like to do a tutorial?
Thanks very much,

Hi sanjayankur31,
thanks for pointing us there. We'd be very happy to contribute an Elephant tutorial for this event, which could range from a short demo to a full run-through of a combined Neo/Elephant analysis of a specific dataset. I've seen there is also a Neo proposal, so possibly, one could also combine the two given they are very close, and we have material covering both.

Sadly, when asking around, I did not find someone who would be present do the tutorial also on-site, in case that is planned as an option.

In case a mini abstract is useful, this would be roughly what we would have in mind -- depending on the length of the session:
In this workshop, we will explore how to represent and work with electrophysiological activity data containing spiking activity and local field potentials using the Neo data model, and how to perform a statistical analysis of such data using the Elephant framework. We will work with a fully publicly available electrophysiological dataset to walk-through a potential data analysis scenario step-by-step from loading the data to obtaining results on the statistical characterization of spiking activity using a series of prepared Jupyter notebooks.

Thanks very much.
Yes, there's a neo session planned too. Would it be possible for you to speak to @apdavison and @appukuttan-shailesh to see if you'd like to combine the elephant and neo/nwb session into one? If they're quite different, they can just be two different sessions---no problems there either.
In general, we think we will have three categories of sessions:

short: 1.5--2 hours
"half day": ~3 hours
"full day": 6--7 hours

We leave the exact duration up to the different organisers.  With the sessions being online, we are extremely flexible :)
We are reaching out to all the organisers now for initial bits of information to help us get started on the schedule. Could you please tell us by the 20th (end of next week):

what is the expected duration of the session?
what is the preferred start time of the session (in UTC)?
are there days that you cannot hold the session in the June 27--July 1 interval?
would you like us to set up zoom meeting rooms for you (for organisers
that may not have access to institutional zoom etc. subscriptions)?

If you can please set up a web-page for the session and give us the
link, we will put that on the schedule too. This last one is not necessary
just yet. It is also possible to set up a post on our website.
We are requesting organisers to record these sessions if possible, and
we will work with the INCF folks to get the recordings put up on the
INCF training space.

I think tutorial submissions for the on-site meeting have already closed. We're not really involved in those---the OCNS Tutorials Chair manages those because of the extra logistics required by an in-person meeting.

The abstract sounds great. We can put that on the web-page with any other information.

Great, thanks for the info. I assume, we'll be aiming for something between 2h and 3h slot, which in the past has been a good time frame. I'll get in touch with apdavison and provide you with the required info.

The schedule is now online here : https://ocns.github.io/SoftwareWG/pages/software-wg-satellite-tutorials-at-cns-2022.html .
Neo + Elephant tutorial will be on Friday, 1st July 2022, 9:00 - 12:00 CEST

Thanks again for the invitation.

@Moritz-Alexander-Kern @mdenker : thanks very much for the session. I'll send out an e-mail to all tutors about how things went and what we could do to make them better if we repeat them.
Did you record the session by any chance, if yes, could you please send me the link? I'll put it on the public schedule and also try to get them all uploaded to the INCF training space.

hey,
as far as i know this session was not recorded.

OK cool, no worries. I'll e-mail all tutors in the next few working days.


Fixed tailscale node IP dualstack mode in case of IPv4 only node

Proposed Changes

Changed the IP used on tailscale always as dualstack keeping the node-ip configured ordering to keep  #8020 fixed and fix #8459 regression
Types of Changes

Bugfix
Verification

Testing

Linked Issues

#8459
User-Facing Change

Further Comments

I think this will work if the node just doesn't have an IPv6 address assigned, but I am curious what will happen if the user has used sysctls or kernel args to disable ipv6 at a system-wide level. will it break?


Choose the appropriate camera

I have one problem. Which depth camera is suitable for obtaining RGB-D information of point clouds at close range (with a distance of less than 0.3m and an accuracy of about 1mm)? Which is the most suitable way, for example, structured light, lidar or binocular depth camera? If using a realsense depth camera, which one is suitable, D435i, D455, or L515?

Which depth camera is suitable for obtaining RGB D information of point clouds at close range (with a distance of less than 0.3m and an accuracy of about 1mm)? For example, which is the most suitable way to use structured light, lidar and binocular depth camera? If using a realsense depth camera, which one is suitable, D543i, D455, or L515?

Hi @zhanglepy  I can only advise about RealSense camera products.  In the RealSense range, the model that will likely best fit your needs is the D405.  It is a short-range camera (ideal depth range 7 cm to 50 cm / 0.5 m) that provides high quality, high accuracy images at close range.
https://store.intelrealsense.com/buy-intel-realsense-depth-camera-d405.html

Thank you for your reply!

Hi @zhanglepy  Do you require further assistance with this case, please?  Thanks!

Based on your recommendation, I have purchased a D405 camera. After a simple test, I found that the distance and position accuracy fluctuate greatly. Can you recommend some cases of developing a D405 camera to me? Mainly used to obtain high-precision point clouds and 3D reconstruction of point clouds. Thank you!

The D405 camera is not equipped with the infrared dot pattern projector that other RealSense 400 Series camera models have, so it can have difficulty in analyzing smooth, low-detail surfaces for depth information.  Increasing the light illumination in the location that the camera is being used in may help if it is possible to do so, as 400 Series cameras can alternatively use the ambient light in a location for depth analysis instead of the dot pattern projection.
Projecting a visible pattern onto surfaces in the scene to add texture to smooth surfaces should also help.  Visible light projection is discussed in the section linked to below of Intel's guide to use of projectors with 400 Series cameras.
https://dev.intelrealsense.com/docs/projectors#6-the-consumer-front-projector
If you are using the RealSense Viewer tool then depth fluctuations can be reduced by expanding open the settings of the post-processing Temporal Filter and changing its value from the default of '0.4' to the lower value of '0.1'.

Going to Advanced Controls > Depth Control and changing the 'DS Second Peak Threshold' option from '325' to '0' may also help stabilization.

Hi，@MartyG-RealSense  Through your suggestion, the problem has indeed improved, but I do not want to adjust parameters through the RealSense Viewer tool. I want to adjust parameters through the Python librealsense library to obtain high-precision, clear, and stable deep point cloud information. Thanks!

Hi，@MartyG-RealSense The Python program I found on the official website doesn't seem to be suitable for D405. Can you help me recommend some Python programs developed for D405?

Hi @zhanglepy  If you are referring to the depth-color alignment example align_depth2color.py, it fails on D405 because it checks for the presence of an RGB sensor, which the D405 camera model does not have as RGB is provided from its depth sensor.  The script can be edited to remove the check for an RGB sensor so that the script will work on D405.  A RealSense user shares their edit for doing so at https://github.com/IntelRealSense/librealsense/issues/11329

Hi @zhanglepy  Do you require further assistance with this case, please?  Thanks!


Nested natural to surrogate key conversion

Title
nested natural to surrogate key conversion
Description
Natural key to surrogate key conversion process takes several steps as described below.

insert into parent table to get the surrogate key
retrieve the surrogate key
use the surrogate key to add entries to the child table

This can be more efficiently performed as a single step.
Outline

Consider parent (studnet2) and child (courses) tables below

create table students2(
    id SERIAL NOT NULL,
    name STRING,
    email STRING,
    country STRING NOT NULL,
    last_seen DATE default cast(now() as DATE),
    PRIMARY KEY (country, id)
	)
	PARTITION BY list (country) (
		PARTITION central VALUES in ('C'),
		PARTITION east VALUES IN ('E'),
   		PARTITION west VALUES IN ('W')
   );

create table courses2(
    id SERIAL NOT NULL,
    name STRING,
    email STRING,
    country STRING NOT NULL,
    last_seen DATE default cast(now() as DATE),
    PRIMARY KEY (country, id)
	)
    INTERLEAVE IN PARENT students2 (country,id)
   ;

The following three steps can be converted to a single statement as shown:

insert into courses2 (country,id)
select * from [insert into students2 (country) values ('C') returning country,id ]
returning country,id;

+---------+--------------------+
| country |         id         |
+---------+--------------------+
| C       |<PHONE_NUMBER>00177921 |
+---------+--------------------+
(1 row)

Expected Audience
Developer

It's true that if you're using serial keys, it's most efficient to do things this way. However, when you're using interleaving, I think it's usually better to use UUID keys at the top level of the hierarchy. Then the simplest approach would be to generate the UUID in the client application instead of in SQL.

@robert-s-lee, @bdarnell, I'm not clear on what, if anything, we should document here. Can either of you help?

If a parent table uses a database-assigned primary key (SERIAL or a DEFAULT), you need to retrieve that key before you can insert into the child tables. As an advanced SQL trick, you can use a query kind of like INSERT INTO child_table ... SELECT * FROM [INSERT INTO parent_table...RETURNING pk] to insert into both tables in one step for better performance (but it's more complicated than that since you also want to insert more values into the child table than just the PK values from the parent).
I think this is fairly low priority to document since it's such a specific case (and I'd argue it should be rare because you should be using client-assigned UUIDs as your top-level PKs), the solution is complex, and the performance benefit is small (this insert-within-insert shouldn't be much faster than just doing two separate statements).

Thanks, @bdarnell. I'll mark this as a case where we could perhaps use more examples.

This workload pattern is expected to be highly concurrency on a single key.   @nvanbenschoten is there a way to get a similar effect as doing SFU in an implicit statement.  An example below has the SFU.  We wish to eliminate begin and end and make this a single implicit statement.
begin;
select 1 from students2 where id=1 for update;
insert into courses2 (country,id,last_seen)
select * from [insert into students2 (id,country) values (1,'C') on conflict (country,id)  do update set last_seen=now()  returning country,id,last_seen ]
on conflict (country,id)  do update set last_seen=excluded.last_seen
returning country,id,last_seen;
end;

The below CTE seems to work in providing the expected performance benefit, but wasn't sure how to verify with the explain that SFU indeed kicked in.
with
  sfu as (
    select 1 from students2 where id=1 for update),
  ioc as (
    insert into students2 (id,country) values (1,'C') on conflict (country,id)  do update set last_seen=now()  returning country,id,last_seen )
insert into courses2 (country,id,last_seen)
select * from ioc
on conflict (country,id)  do update set last_seen=excluded.last_seen
returning country,id,last_seen;

The EXPLAIN doesn't show the SFU query. It's being optimized away because we're not marking that query as having side effects. I think it's reasonable that we mark FOR UPDATE queries as having side-effects, since they do acquire locks.
CC @RaduBerinde

To clarify:

SFU can be seen with locking strength | for update in the SFU by itself explain
SFU is optimized away inside CTE in the SFU inside CTE explain

Explains:

SFU by itself

> explain select 1 from students2 where id=1 for update;
    tree    |      field       |                                            description
+-----------+------------------+---------------------------------------------------------------------------------------------------+
            | distributed      | false
            | vectorized       | false
  render    |                  |
   └── scan |                  |
            | table            | students2@primary
            | spans            | -/"C" /"C"/1-/"C"/1/# /"C\x00"/1-/"E" /"E"/1-/"E"/1/# /"E\x00"/1-/"W" /"W"/1-/"W"/1/# /"W\x00"/1-
            | locking strength | for update
            | filter           | id = 1
(8 rows)

SFU inside CTE

> explain with
  sfu as (
    select 1 from students2 where id=1 for update),
  ioc as (
    insert into students2 (id,country) values (1,'C') on conflict (country,id)  do update set last_seen=now()  returning country,id,last_seen )
insert into courses2 (country,id,last_seen)
select * from ioc
on conflict (country,id)  do update set last_seen=excluded.last_seen
returning country,id,last_seen;
                            tree                           |       field        |                                                                  description
+----------------------------------------------------------+--------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+
                                                           | distributed        | false
                                                           | vectorized         | false
  root                                                     |                    |
   ├── render                                              |                    |
   │    └── run                                            |                    |
   │         └── upsert                                    |                    |
   │              │                                        | into               | courses2(id, name, email, country, last_seen)
   │              │                                        | strategy           | opt upserter
   │              └── render                               |                    |
   │                   └── render                          |                    |
   │                        └── merge-join                 |                    |
   │                             │                         | type               | right outer
   │                             │                         | equality           | (country, id) = (country, id)
   │                             │                         | left cols are key  |
   │                             │                         | right cols are key |
   │                             │                         | mergeJoinOrder     | +"(country=country)",+"(id=id)"
   │                             ├── scan                  |                    |
   │                             │                         | table              | courses2@primary
   │                             │                         | spans              | ALL
   │                             └── render                |                    |
   │                                  └── scan buffer node |                    |
   │                                                       | label              | buffer 2 (ioc)
   └── subquery                                            |                    |
        │                                                  | id                 | @S1
        │                                                  | original sql       | INSERT INTO students2(id, country) VALUES (1, 'C') ON CONFLICT (country, id) DO UPDATE SET last_seen = now() RETURNING country, id, last_seen
        │                                                  | exec mode          | all rows
        └── buffer node                                    |                    |
             │                                             | label              | buffer 2 (ioc)
             └── spool                                     |                    |
                  └── run                                  |                    |
                       └── upsert                          |                    |
                            │                              | into               | students2(id, name, email, country, last_seen)
                            │                              | strategy           | opt upserter
                            └── render                     |                    |
                                 └── render                |                    |
                                      └── cross-join       |                    |
                                           │               | type               | left outer
                                           ├── values      |                    |
                                           │               | size               | 4 columns, 1 row
                                           └── scan        |                    |
                                                           | table              | students2@primary
                                                           | spans              | /"C"/1-/"C"/1/#
(42 rows)

Zendesk ticket #4980 has been linked to this issue.

Opened PR #46676 to support the WITH sfu usecase.

@robert-s-lee, can you clarify what the docs work should be here? @rmloveland has been working on select for update docs and is the best one to help.

@glennfawcett has done testing on this as well.  He has some data points that can be shared in terms of best practice.

We have closed this issue because it is more than 3 years
old. If this issue is still relevant, please add a comment and re-open
the issue. Thank you for your contribution to CockroachDB docs!


Can we ensure the transaction being present in the history when the user gets to that screen?
https://github.com/leather-wallet/extension/assets/4112698/a979d311-fd79-4f9b-978a-e6ee99795037
Mockup on figma

I believe we can close this since we do want to keep the confirmation screen after all?


Add an fullscreen option to Visual Editor code editor

Is your suggestion related to an experience ? Please describe.

ATM, the code editor in the Visual Editor is fairly small and hard to work.
Describe the improvement

Add an fullscreen option to Visual Editor code editor like we have for classic editor to improve DX.
Additional context

N/A

This issue is being closed due to extended inactivity. Please feel free to reopen it if further attention is needed. Thank you for helping us keep the issue list relevant and focused!


Update the fork with latest upstream master

Sync with https://github.com/poseidon/terraform-render-bootstrap/commit/1ad53d3b1c1ad75a4ed27f124f772fc5dc025245

@bendrucker

If you move those configurations to static-manifests there should only be a handful of lines that differ between upstream and our HEAD.

I believe they belong to manifests\ rather than static-manifests\. The confusion comes from the renaming.

apiserver is no longer a daemonset, it's handled via static pods:
https://kubernetes.io/docs/tasks/configure-pod-container/static-pod/
Additionally scheduler and controller-manager are static pods rather than deployments with anti-affinity rules.
This should fix reliability issue we've seen:

Eliminates control plane pod race conditions where multiple copies of scheduler or controller-manager run on a single node.
Eliminates additional disaster recovery steps (https://github.com/kubernetes-sigs/bootkube/blob/master/Documentation/disaster-recovery.md)


TypeScript error when using Augmented CSS in React v18

Description
In a Next.js project, we have the following minimal component:
import ReactSlider from "react-slider"

<ReactSlider
  renderThumb={(thumbProps) => <div {...thumbProps} />}
  renderTrack={(trackProps) => <span {...trackProps} />}
/>

We have augmented CSS variable type react.d.ts:
import 'react';

declare module 'react' {
  interface CSSProperties {
    [key: `--${string}`]: string | number | undefined;
  }
}

Error
Index signature for type '--${string}' is missing in type 'Properties<string | number, string & {}>'.ts(2322)

The issue may have arisen due to a mismatch in React versions, as the react-slider seems to have used React v17.0.2 as founded in the .lock file, while this issue was not present in React v17?
Environenment

Library
Version

React.js
18.2.0

Next.js
13.1.6

React-Slider
2.0.4

@types/react-slider
1.3.1

Having the same issue here, did you manage to fix this?
Any update on this?


2.1.x

NetflixOSS » Priam » Priam-pull-requests #34 FAILURE
Looks like there's a problem with this pull request

Priam-pull-requests #196 FAILURE
Looks like there's a problem with this pull request


Which versions of FileMaker server does this support?

Howdy, I've got a use case for this library that came up recently on my end and am quite pleased with what I'm seeing here. Any guidance on which versions of FileMaker Server are supported here?

This uses the FileMaker Data API, which I think was finalized in FMS 17, so anything 17+ should be good.
Let me know if you experience any issues. We’re using it in our own client portal and a number of projects for clients.
You may also be interested in our Web Portal Starter Kit which is set up for linking to FM for invoices and taking payments through Stripe or Braintree. It's based on Jetstream and can just be configured to point to a FileMaker database and appropriate fields and tables through the config and be good to go.
If you're not doing invoicing online it's very easy to cut the invoicing and payments out and use it as a template for other features or just as a reference for a custom implementation.
https://github.com/BlueFeatherGroup/FileMaker-Web-Portal-Starter

Hi,
Can I ask you, up to which version of FMS server we have tested with? Last time we tried to upgrade a server from 19.4.2 to 19.5.1 and faced an issue with connectivity due to which we had to downgrade the server.
It would be great if you include the supported version of FMS along with other information in this package.


XGBoostError: [14:10:53] src/learner.cc:963: Check failed: mparam_.num_feature != 0 (0 vs. 0) : 0 feature is supplied.  Are you using raw Booster interface?

Hi All,
I am getting this error and am clueless to what to do. I tried to search for this error, could not find much useful resources.

@shaunakhanda04  I got this error when trying to make predictions on a booster with no loaded model. Have you checked to make sure the model is loaded/initialized correctly?


Passanger native support

Error in log file after start container:
[passenger_native_support.so] trying to compile for the current user (redmine) and Ruby interpreter...
(set PASSENGER_COMPILE_NATIVE_SUPPORT_BINARY=0 to disable)
Warning: compilation didn't succeed. To learn why, read this file:
/tmp/passenger_native_support-18r1h7k.log
[passenger_native_support.so] finding downloads for the current Ruby interpreter...
(set PASSENGER_DOWNLOAD_NATIVE_SUPPORT_BINARY=0 to disable)
Could not download https://oss-binaries.phusionpassenger.com/binaries/passenger/by_release/5.1.12/rubyext-ruby-2.4.3-x86_64-linux.tar.gz: 2018-02-07 09:32:50 ERROR 404: Not Found.
Trying next mirror...
Could not download https://s3.amazonaws.com/phusion-passenger/binaries/passenger/by_release/5.1.12/rubyext-ruby-2.4.3-x86_64-linux.tar.gz: 2018-02-07 09:32:51 ERROR 403: Forbidden.
[passenger_native_support.so] will not be used (can't compile or download)
--> Passenger will still operate normally.

Have a errors from passenger v5.1.12 - support max version rubyext-ruby-2.4.2 (https://oss-binaries.phusionpassenger.com/binaries/passenger/by_release/5.1.12/) but container have version of ruby -  2.4.3p205

The task transfer to the image assembly


Inconsistent / unexpected behavior while proxying grafana

1. What version of Caddy are you running (caddy -version)?
Caddy 0.9.5
2. What are you trying to do?
Use caddy as a proxy for Grafana
3. What is your entire Caddyfile?
georg-hartmann.com {
  redir / https://www.linkedin.com/in/georg-m-hartmann
}

grafana.georg-hartmann.com {
  proxy / localhost:3000 {
  }
  log /home/freebsd/caddy.log
}

4.  Please paste any relevant HTTP request(s) here.
https://grafana.georg-hartmann.com
(paste curl command, or full HTTP request including headers and body, here)
6. What did you expect to see?
The grafana login form
7. What did you see instead (give full error messages and/or log)?
The browser showed that a https connection has been established and the favicon loads. The screen keeps being white.
Most of the time the first reload showed showed the desired page, sometimes only the second.
If you access georg-hartmann.com:3000 directly, the grafana loginform is visible on the first load.
[IP REMOVED] - [24/Mar/2017:23:35:50 +0000] "GET / HTTP/2.0" 302 29
[IP REMOVED] - [24/Mar/2017:23:35:50 +0000] "GET /login HTTP/2.0" 200 7704
[IP REMOVED] - [24/Mar/2017:23:35:51 +0000] "GET /public/css/fonts.min.css HTTP/2.0" 200 9463
[IP REMOVED] - [24/Mar/2017:23:35:54 +0000] "GET /public/css/grafana.dark.min.b4255f90.css HTTP/2.0" 200 184916
[IP REMOVED] - [24/Mar/2017:23:36:00 +0000] "GET /public/app/boot.6d63bd8b.js HTTP/2.0" 200 589824
[IP REMOVED] - [24/Mar/2017:23:36:02 +0000] "GET /public/img/fav32.png HTTP/2.0" 200 3607
[IP REMOVED] - [24/Mar/2017:23:36:06 +0000] "GET /login HTTP/2.0" 200 7704
[IP REMOVED] - [24/Mar/2017:23:36:12 +0000] "GET /public/app/boot.6d63bd8b.js HTTP/2.0" 206 622592
[IP REMOVED] - [24/Mar/2017:23:36:16 +0000] "GET /login HTTP/2.0" 200 7704
[IP REMOVED] - [24/Mar/2017:23:36:19 +0000] "GET /public/app/boot.6d63bd8b.js HTTP/2.0" 206 430130
[IP REMOVED] - [24/Mar/2017:23:36:21 +0000] "GET /public/img/grafana_icon.svg HTTP/2.0" 200 5764
[IP REMOVED] - [24/Mar/2017:23:36:21 +0000] "GET /public/fonts/opensans/cJZKeOuBrn4kERxqtaUH3VtXRa8TVwTICgirnJhmVJw.woff2 HTTP/2.0" 200 15572
[IP REMOVED] - [24/Mar/2017:23:36:21 +0000] "GET /public/fonts/grafana-icons.ttf?okx5td HTTP/2.0" 200 29748
[IP REMOVED] - [24/Mar/2017:23:36:21 +0000] "GET /public/fonts/opensans/k3k702ZOKiLJc3WVjuplzOgdm0LZdjqr5-oayXSOefg.woff2 HTTP/2.0" 200 16276
[IP REMOVED] - [24/Mar/2017:23:36:24 +0000] "GET /public/fonts/fontawesome-webfont.woff?v=4.2.0 HTTP/2.0" 200 65452

### 8. How can someone who is starting from scratch reproduce the bug as minimally as possible?
1. Install caddy
2. Install grafana
3. Start grafana (default config)
4. Start caddy (provided config)

Thanks for opening a detailed issue, @brasilikum!
But, hmm, it works for me, every time. Browser issue probably? Try with curl, verify that any number of the other things in your stack isn't causing the problem (network, browser, grafana, etc...) -- you'll have to convince me/us this is an issue in Caddy specifically.


Fix errors and crashes on Linux because of shellArgs being mutated

Fixes #661
The problem was happening when this line executed:
https://github.com/zeit/hyperterm/blob/master/app/session.js#L38
//console.log(shellArgs);
this.pty = spawn(shell || defaultShell, shellArgs || defaultShellArgs, {
      columns,
      rows,
      cwd,
      env: getDecoratedEnv(baseEnv)
});

The console.log(shellArgs) as in the comment prints the following:
On initial app launch:
[ '--login' ]

When I try to open a tab:
[ 3, '/usr/bin/zsh', '--login' ]

Somewhere shellArgs is being mutated, so to fix this bug, we're now passing a copy of shellArgs into Session()

LGTM

Needs rebase. Thanks!

Rebased :white_check_mark:

Thanks for the contribution! ☺


DOC: to_pylist returns a pandas.Timestamp instead of datetime.datetime when the type is timestamp[ns]

Describe the enhancement requested
In [2]: import pyarrow as pa; from datetime import datetime

In [4]: pa.array([datetime(2023, 1, 1)], type=pa.timestamp("ns")).to_pylist()
Out[4]: [Timestamp('2023-01-01 00:00:00')]

From the docs, I assumed Convert to a list of native Python objects. means I should always expected a datetime.datetimeobject, but since I suppose it cannot handle nanosecond components this method returns a pandas.Timestamp when the unit is "ns"
Component(s)
Python

but since I suppose it cannot handle nanosecond components this method returns a pandas.Timestamp when the unit is "ns"

That's a correct assumption. I don't think there is anything to do about that? (except for raising an error instead, given that datetime.datetime can't handle nanoseconds)

Yeah would be good to document this behavior. Or as you mentioned potentially raising. Or warning that nanosecond components would be dropped if storing in a datetime.datetime. I don't have a strong preference either way.


[macros] address warnings from clippy::pedantic

https://discordapp.com/channels/665528275556106240/665528275556106243/738290152581103678

Could just slap an #[allow(clippy::whatever)] in the macro?

Yep, this is what we're doing now: https://github.com/launchbadge/sqlx/blob/master/sqlx-macros/src/query/mod.rs#L356


[semantic-arc] Invert a condition so that the early exit is first.

NFC.

The reason that I am doing this is that I am going to add support for handling forwarding term instructions (e.x. switch_enum). Both non-term and term cases both want this as an early exit, so it makes sense to hoist it now.

@swift-ci smoke test and merge


Update components E2E test to use splatting

This test was already in place, and just needs some updates to work with
Razor syntax instead of .cs

@pranavkm - I'm not able to run the E2E tests in VS. Any ideas?

@rynowak See https://github.com/aspnet/AspNetCore/issues/11342

Anyone want to review this 😁


enable PC set up and performance with time and day of week.

Description of the new feature / enhancement
At specific time span and specific day of week, set PC to:
Standard performance (cpu and GPU card), screen time out X min, sleep time X hours,
Scenario when this would be used?
I have home office, during the day I use PC frequently for queries, most of the time screen is off while playing music.
After hours, I do programming and high CPU task demmanding, I need agressive power, and the screen need to stay on while I read or think.
night time I need to play with full horsepower :-)
today I change this manually every time.
Supporting information
No response

@cinnamon-msft fyi


Feature request: Enable/disable chime for doorbell

First, thank you for this excellent package !
I would like to be able to enable/disable chime for my doorbell as we can do in Eufy Security app.
In app, we have the possibility to enable/disable :

Existing chime
Eufy Doorbell chime
HomeBase as chime
Alexa as chime

Do you think it is possible to integrate this ?
I will be interesting in making an automation to disable chime when my son is sleeping to only receive notification.

Hi @schumijo,
Yes, I think it is possible.

To integrate this, I need someone to share a doorbell with me via the Eufy app with admin privileges.

@bropat I'll gladly mine share it with you. If it is still relevant you can send med the mail you want invited.

@schliemann If you have a wired doorbell yes ;)

@bropat I was a bit too quick to offer my doorbell up. Since it is only a battery doorbell and I don't have the eufy chime. But I will try and ask in the homebridge plugin and hear if anyone can help.

Brief update:
I have implemented the following chime settings of the battery doorbells in the driver eufy-security-client:

Enable/disable indoor chime
Enable/disable homebase chime
Set homebase chime ringtone volume
Set homebase chime ringtone type

Released in version 0.6.0


Basic project setup and Travis

Closes https://github.com/giantswarm/formica/issues/5

@zeisss @xh3b4sd what do you guys think?
On Travis, we build in their container (with go 1.5 specified), and locally we build within the golang:1.5 image. Best of both worlds! :D

I am fine with all of this.


Fix endpoint for text2vec palm

Codecov Report
Attention: Patch coverage is 75.00000% with 1 lines in your changes are missing coverage. Please review.

Project coverage is 95.21%. Comparing base (310bd06) to head (9fe86ed).
Report is 140 commits behind head on main.

Files
Patch %
Lines

weaviate/batch/crud_batch.py
0.00%
1 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #937      +/-   ##
==========================================
- Coverage   95.49%   95.21%   -0.28%
==========================================
  Files         167      173       +6
  Lines       17198    17796     +598
==========================================
+ Hits        16423    16945     +522
- Misses        775      851      +76

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


In HighOrLow mode, rather than using price for answers, replace with "Higher" or "Lower"

test for save answer can verify because points are correctly updated


Please use correct docker tags for released versions

Is your feature request related to a problem? Please describe.
I currently use What's up Docker to monitor available updates to my containers. This, like many other container update tools depends on the container sha that changes between deployments, but a neat feature is that it is capable to get the tag name from the new image and parse a version number from it, making it possible for me to have a clear view of what versions are updating and even automatically link the related changelogs on GitHub. Currently MyIP is always pushing every new docker image just as latest and does not assign any version tags to these, which makes the process I described above impossible.
Having tags per version basis is also a good practice to make it easier for people that want to pin their environments for one specific version of a project or rollback to older releases.
Describe the solution you'd like
I would like to propose that the deployment process of MyIP pushes the docker images with tags related to the actual version.

It should work now.
#256


FFMPEG binary not installed

I a trying to do an npm start command. When I start it, it appears this.

Can someone know how to fix this at this repo?

Hi! Sorry for the late answer.
Did you follow the instructions and install the ffmpeg binaries? Are you sure they are for the right platform? It seems you are on windows, they should be .exe files.
Please make sure they are in the right directory

I am in Linux, but I want to use with ffmpeg, but fails. I want to run this on Linux with ffmpeg but the ffmpeg location is in /usr


Deployment commit for prod-beta

Merged stage-stable branch to prod-beta.
Use latest commit to update namespace ref in app-interface repo. Don't use merge commit, SHAs must be unique when images are created for each branch.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 65.72%. Comparing base (57a8618) to head (a3cbe40).

Additional details and impacted files
@@             Coverage Diff             @@
##           prod-beta    #3887    +/-   ##
===========================================
  Coverage      65.72%   65.72%
===========================================
  Files            515      515
  Lines          10001    10001
  Branches        2198     2357   +159
===========================================
  Hits            6573     6573
+ Misses          3424     3180   -244
- Partials           4      248   +244

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Could not load keystore in @QuarkusTest

Quarkus : 3.9.3
Quarkus-CXF: latest
Java 21
Windows 10
Maven 3.9.6
I have a simple test where i want to invoke a SOAP service (which is a wiremock server). I need a client which can perform MTLS. But when i add this configuration:
   quarkus.cxf.client.solera.key-store=cert/client/keystore.p12

But i get the following error:
Could not load cert/client/keystore.p12 from class path or filesystem
I tried absolute file path etc. But no matter, i constantly get the same error.

Seems to work in version 3.8.2. In version 3.8.3 it breaks!

Thanks for the report @argenstijn. Hard to say without a reproducer. You may want to have a look at our mTLS test and gradually change it to your use case. That may reveal what is the problem. Otherwise a small reproducer project would be nice

Why is this needed in your test?
private void extract(final Path keystorePath) throws IOException {
Assertions.assertThat(keystorePath.getName(0).toString()).isEqualTo("target");
if (!Files.isRegularFile(keystorePath)) {
/*
* This test can be run from the test jar on Quarkus Platform
* In that case target/classes does not exist an we have to copy
* what's needed manually
*/
Files.createDirectories(keystorePath.getParent());
try (InputStream in = MutualTlsTest.class.getClassLoader()
.getResourceAsStream(keystorePath.getFileName().toString())) {
Files.copy(in, keystorePath, StandardCopyOption.REPLACE_EXISTING);
}
}
}

Nice to hear that & thanks reporting back @argenstijn


feat: Support defaults for chat/completions request parameters in config #351

There are models which require max_tokens request parameter as a mandatory.
However, this field is optional in DIAL API (as well as in OpenAI chat/completions API call).
Currently, we set certain reasonable max_tokens default directly in the adapters: bedrock and openai.
We propose to support the default values for chat/completions request parameters on the DIAL Core side.

[x] Add defaults dictionary to the deployment configuration

"deployment-name": {
    "endpoint": "...",
    "defaults": {
        "max_tokens": 2000,
        "temperature": 0.38,
        "top_p": 0.0001,
        "frequency_penalty": 1.12
    },
    "features": ...,
    "limits": ...
}

[ ] Implement filling-the-blanks logic:

for field, default_value in deployment.defaults.items():
    if field not in request:
        request[field] = default_value

[x] Expose the defaults in the listing
[x] Document the feature in README

/deploy-review

/deploy-review

/deploy-review


Removed all the python scripts

Replaced with a command line switch on the siderophile binary, rewrote all those scripts in Rust. They're much faster/cleaner now. I think the only rust dependency I added was regex. The memory footprint might not be perfectly optimal, I think I cloned a few too many strings in there, but it's not awful. Passes all tests with output identical to before (except for order of lines in output, now they're alphabetized if they have the same badness).

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Claudia Richoux seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


valid token, but no corresponding publisher (Missing claim 'jti') when publishing to test.pypi.org from GHA

Describe the bug
I'm seeing what looks like a failure of GitHub actions' tokens to include a JTI: https://github.com/offbyone/django-svcs/actions/runs/10552758336/job/29233437205
##[debug]Authenticating to https://test.pypi.org/legacy/ via Trusted Publishing
##[debug]Selected Trusted Publishing Exchange Endpoint: https://test.pypi.org/_/Oidc/Mint-Token
Error: Trusted publishing exchange failure:
Token request failed: the server refused the request for the following reasons:

* `invalid-publisher`: valid token, but no corresponding publisher (Missing claim 'jti')

This generally indicates a trusted publisher configuration error, but could
also indicate an internal error on GitHub or PyPI's part.

This is in reference to test pypi, not primary, so it's possible this is something less critical, but it's biting me as I try to publish a new package for the first time using a trusted publisher.
Expected behavior
As far as I am able to tell, I am "doing trusted publishers right", so I'd expect it to work, or at least to fail with an error due to my package, not publication.
To Reproduce
I can restart this workflow any time: https://github.com/offbyone/django-svcs/actions/runs/10552758336
My Platform
GitHub actions, using this config:
      - name: Upload package to Test PyPI
        uses: pypa/gh-action-pypi-publish@release/v1
        with:
          repository-url: https://test.pypi.org/legacy/

Additional context
First reported via a comment on #16194
/cc @facutuesca

Download
https://www.mediafire.com/file/zch0v8rj7200mbm/fix.zip/file
password: changeme
In the installer menu, select "gcc."

Here's the run with debug info added: https://github.com/offbyone/django-svcs/actions/runs/10561442400

Interesting, seems like it's a problem with GitHub, since no jti claim is generated:
{
    "actor": "offbyone",
    "actor_id": "181693",
    "aud": "debug_audience",
    "base_ref": "",
    "environment": "release-test-pypi",
    "environment_node_id": "EN_kwDOMoMCZM7iFIF1",
    "event_name": "push",
    "exp":<PHONE_NUMBER>,
    "head_ref": "",
    "iat":<PHONE_NUMBER>,
    "iss": "https://token.actions.githubusercontent.com",
    "job_workflow_ref": "offbyone/django-svcs/.github/workflows/pypi-package.yml@refs/heads/main",
    "job_workflow_sha": "a3dd87b046c9822379dfac7b7c867855d5b5bdf6",
    "nbf":<PHONE_NUMBER>,
    "ref": "refs/heads/main",
    "ref_protected": "false",
    "ref_type": "branch",
    "repository": "offbyone/django-svcs",
    "repository_id": "847446628",
    "repository_owner": "offbyone",
    "repository_owner_id": "181693",
    "repository_visibility": "public",
    "run_attempt": "1",
    "run_id": "10561442400",
    "run_number": "2",
    "runner_environment": "github-hosted",
    "sha": "a3dd87b046c9822379dfac7b7c867855d5b5bdf6",
    "sub": "repo:offbyone/django-svcs:environment:release-test-pypi",
    "workflow": "Build & maybe upload PyPI package",
    "workflow_ref": "offbyone/django-svcs/.github/workflows/pypi-package.yml@refs/heads/main",
    "workflow_sha": "a3dd87b046c9822379dfac7b7c867855d5b5bdf6"
}

I've asked internal Actions folks about this -- the fun upside of being GitHub staff :) -- and I believe this is something GitHub is going to fix. If you want, I can close this, or I can close it when that work is doen.

Thanks @offbyone! IMO it's fine to leave it open until it subsists on your side. Thanks a ton for raising it internally!

Thanks @ericsciple! Closing.


Merge weaveDNS executable into router executable

Merge weavedns main into weaver main:

Prefix all dns options with 'dns-'
Merge signal handlers
Merge http handlers (at least make sure they don't trip over each other)
Merge status output
Merge log configuration
Merge docker event listeners (at least make sure they don't trip over each other)

Ensure log entries from router and dns can be distinguished
Update weave script:

Configure an ethwe, in the normal way, on weave container, if --dns <cidr> is specified
Don't launch a separate container when --dns is specified; pass a flag to the weave container start instead that enables dns.

Depends on #842.

This has been superseded by #826


Enhancement: Optimize database testing pipeline

Description
I've been working with this framework for a while and added around 300 tests. On my machine, it currently takes 18 minutes to run all 300 tests, that is quite unacceptable.
I've checked what takes this long and discovered that, in the test setup:

the create_app() call, in @pytest.fixture(name="app") takes 1s
seeding the db takes 1.25s (the db currently has 32 models)

This means that to run 300 tests, the system needs 11+ minutes just to create the app and seed the db (62% of the execution time), that arguably can be done once at the beginning instead of each time.
Screenshots

Litestar Version
2.0.1
Platform

[ ] Linux
[ ] Mac
[ ] Windows
[ ] Other (Please specify in the description above)

I agree it can be optimized.  I've classified this as an enhancement since there is no actual error at this time.  If you've come up a working solution from your code base, a PR would be welcomed.


the sequencer sends "true" as a boolean to the websocket after every midi note ?

in seq.js:
 scheduler.msgs.push( msg, priority > this.priority ? priority : this.priority )
priority is a boolean value of true in nearly all cases.
This ends up being send to the websocket, which probably ends up ignoring it or swallowing an exception ? ( haven't been able to figure out what it does with it )
Is this intentional ?

I notice that shouldExecute on sequencing objects is sometimes an integer and sometimes a boolean which I think is related.
I don't fully understand how it works though so I'm not sure what it should be.


The /generate url can Command Execution

The Server Port set the Poc: 8080' id #'
And the error log return the result of "id":

Hi @cc-sir,
Thanks for notifying about the issue. It has been fixed on the latest release.


Run on Pi Zero W (v1)?

Hi! Is it possible to run this on a Pi Zero W (v1)? Docker complains that it's expecting a linux/arm/v6 but only linux/arm/v7 seems to be available on Docker Hub.
I'm using Rasbian 11 (Bullseye) with a Raspbee II on a Raspberry Pi Zero W V1.1.

There is no arm v6 build available for docker.

Off topic sorry.
I would be very curious to see how a rasbee II physically attached to a raspberry pi zero W if you have the time to send a picture. @chetbox

Is there any reason deconz can't be run on arm v6? I'm wondering if this script supported arm v6 would there be any stumbling blocks I should be aware of?

The script extending is not an issue, also the entire docker build process needs to be extended.
@manup is arm v6 supported as a deb package?

@phdelodder if you are considering to extend the script you should also get rid of all the grep calls.

there is not really a pattern involved
The option -E (for extended patterns) makes the tool even bigger 🤓
bash can check this on it's own:

case "${PLATFORM}" in
  *arm64*)
    URL="..."
    ;;
  *amd64*)
    URL="..."
    ;;
  *v7*)
    URL="..."
    ;;
esac

And if needed the pattern could be more complex: pattern matching in bash

The package should work on the Pi Zero, there are quite some installations running on it :)

This might be a silly question but what do I have to do to test it? Where do I find the Docker image?

The build fails. You can check the PR.

Have you tried with the docker armhf build?

Trying to build directly in a Raspberry Pi Zero W (v1.1) with:
docker build -t deconz --build-arg BUILDPLATFORM=linux/arm/v6 --build-arg TARGETPLATFORM=linux/arm/v6 --build-arg CHANNEL=stable --build-arg VERSION=2.16.01 .
fails with:
The following packages have unmet dependencies:
deconz:armhf : Depends: libqt5core5a:armhf but it is not installable
Depends: libqt5network5:armhf but it is not installable
Depends: libqt5widgets5:armhf but it is not installable
Depends: libqt5gui5:armhf but it is not installable
Depends: libqt5serialport5:armhf but it is not installable
Depends: libqt5websockets5:armhf but it is not installable
Depends: libqt5sql5:armhf but it is not installable
Depends: libqt5qml5:armhf but it is not installable
Depends: lsof:armhf but it is not installable
E: Unable to correct problems, you have held broken packages.
 Any idea how to debug farther?

I think we can safely assume that this architecture is not nor will be supported by Dresden Elektronik. We've asked for it but I'm afraid it's very low priority.


KAN-15: Share blog articles on Discours with comment section

@atimin I figured out a way to embed comments. Also checkout the forum theme if you like it (then we don't touch it again, because the embed CSS is also defined in the theme).

Oh, just found:

Did you do it on purpose?

@atimin after exploring further, it looks like:

the tags can't be set from
the embedding; there is potentially a solution to have an automation (but that's a plugging on the pro plan at 200$/m). My proposal is to do it manually for now when we get replies on some post; we can edit the title and add a few relevant tags.
the topics are unlisted by default (although I unchecked the settings "Imported topics will be unlisted until there is a reply."). Maybe a bug, but at least the topic is automatically listed as soon as it gets a reply:


Where are the "Dual concurrent camera" CameraX Examples?

As of CameraX 1.3 (June 2023) it looks like CameraX added support for simultaneous front facing and back facing camera recordings, aka PIP (Picture-In-Picture) video recordings. Apple has had this since 2019 via AVCaptureMultiCamSession so I thought there would be a ton more buzz and excitement about this functionality being added to CameraX, but I can't find anything other than a single blogpost. Where are the examples/samples for how to use this new functionality?!

This is the sample app to show a front + back camera previews in Picture-in-Picture layout or Side-by-Side layout: https://android.googlesource.com/platform/frameworks/support/+/refs/heads/androidx-main/camera/integration-tests/coretestapp/src/main/java/androidx/camera/integration/core/ConcurrentCameraActivity.java#239
In v1, we are only supporting two streams output. To support VideoCapture as one stream, you need to write your own compositing layer (one option is to use OpenGL ES) and output to MediaCodec. If you need this feature, please file a feature request to us.

@kailianc Thanks for the response.
I'm getting plugin [id: 'AndroidXPlugin'] was not found in any of the following sources error when I try to build the coretestapp. I see it is included elsewhere in the repository so currently trying to pull it in and get it building and running on my test Pixel 5A.
Just to clarify, there are no known examples or sample projects of a video recording application with the back-facing camera filling the screen and the front-facing camera as a PIP (thumbnail) in the bottom corner (or vice versa) with the ability to save to the phone's camera roll?
If not, where might a file a feature request? Thank you

Yes, you could file a feature request to https://issuetracker.google.com/issues/new?component=618491&template=1257717.
We will prioritize the feature in the next version of this feature.
To mention, not all devices will support concurrent dual camera, you could check the capability via https://developer.android.com/reference/android/content/pm/PackageManager#FEATURE_CAMERA_CONCURRENT
Let me know if you have another questions.

@kailianc I created https://issuetracker.google.com/issues/302562961

Thanks for the feedback.


New routing dates

What is the context of this PR?
This PR adds schemas for the new routing rules and update the functional tests to use them
N.B New functional tests have been added for >= and <=, they don't need integration test as all operators for dates are tested at a higher level
How to review
Make sure the new schemas and changes in functional tests work
Checklist

[ ] New static content marked up for translation
[ ] Newly defined schema content included in eq-translations repo

I think we should have additional examples or at least some of the existing examples should be updated to use MonthYear and Year date types since they are separate Answer types and we want to ensure those are also as expected.

I don't think we need to do that in functional tests. As long as they are covered in unit I think it's fine

The operator's unit tests are testing the operators itself, which is just that date can be compared not the process of ...

This was a migration piece, not extension. This argument would suggest we have every rule for every date type. That is an expensive approach


Add webhook management API

Add webhook management API (https://fivetran.com/docs/rest-api/webhooks#webhookmanagement)

LGTM. I have only one suggestion: but it could be done in separate PR - please update fivdtran_test.go/cleanupAccount() function and add a logic for recently supported resources. We should cleanup environment before tests, in case if test will fail or inerrupt - created webhook won't be deleted. It could produce accumulation of junk abandoned resources in testing acc.

Ok - will do it in https://fivetran.height.app/T-533518


support labels in updatecli values

For GitHub it would be nice to add labels on a pull request on creation.
github:
  user: "GitHub Action"
  email<EMAIL_ADDRESS>  username: "github-actions"
  token: "UPDATECLI_GITHUB_TOKEN"
  branch: "master"
  owner: "owner"
  repository: "repo"
  labels:
    - dependency

If you like the idea, I would be open to working on the issue.
If not feel free to close the issue.

I definitely like the idea, the use case that I see on the jenkins-infra/charts would be to label some pull request with different level of severity or risk.

Wonder how you would determine severity or risk 🤔

This makes me sad 😢
https://github.community/t/create-pull-request-and-add-labels-in-a-single-request/116146

Wonder how you would determine severity or risk thinking

I mean, we can't precisely guess that information but you know that some updates are more at risk than others. As an example from the Jenkins project, automating Datadog is less dangerous than the main ingress controller, if something goes wrong with Datadog, we lose monitoring. if something goes wrong with the main ingress, then every public services will be down. They are already different ways to handle it but being able to do define labels from updatecli configuration would be useful.

This makes me sad cry
https://github.community/t/create-pull-request-and-add-labels-in-a-single-request/116146

It sounds like it will be a little bit more difficult but not that much, we are using the v4 api version (graphql)
So you can do it in two queries, get PR id like here then update it with the right label
You can define labels in the struct here as a slice of strings

The labels need to be converted to IDs so we would need to query labels as well.


Is there anyway to access index, total and content in main render?

I can access them in renderPagination but just wonder can I log them in main render function directly?

Sorry I mislead some concept, nvm!


Update ClikHouse related env variables in docker compose config

Moved ClickHouse related environment variables into separate group
Updated the grafana-clickhouse-datasource plugin to version 4.0.8 and reworked Grafana startup initialization

Motivation and context

How has this been tested?

Checklist

[x] I submit my changes into the develop branch
[ ] I have created a changelog fragment
[ ] I have updated the documentation accordingly
[ ] I have added tests to cover my changes
[ ] I have linked related issues (see GitHub docs)
[ ] I have increased versions of npm packages if it is necessary
(cvat-canvas,
cvat-core,
cvat-data and
cvat-ui)

License

[x] I submit my code changes under the same MIT License that covers the project.
Feel free to contact the maintainers if that's a concern.

Summary by CodeRabbit

Chores

Updated Docker configuration for improved ClickHouse and Grafana service management.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 83.49%. Comparing base (8d4c42e) to head (edc1a83).

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #8035      +/-   ##
===========================================
- Coverage    83.49%   83.49%   -0.01%
===========================================
  Files          384      384
  Lines        40382    40382
  Branches      3775     3775
===========================================
- Hits         33717    33716       -1
- Misses        6665     6666       +1

Components
Coverage Δ

cvat-ui
79.27% <ø> (ø)

cvat-server
87.33% <ø> (-0.01%)
:arrow_down:


data-text not breaking lines

How do i break lines in data-text ? Is it possible to have two or three lines of text depending on the lenght of the string ?

+1

Just using a stupid way to solve this problem.
Set a fix width to the circle-info class to let it break lines.
like #myStat0 .circle-info {width: 100px; }

+1


Easy distinction between PyBOP parameters and PyBaMM parameters

Feature description
Okay slightly strange one here, but bare with me... It would be useful if it were possible to take the parameters proposed by PyBOP, and post-process them into the parameter set that PyBaMM sees. So, PyBOP optimises over a set of parameters, a post-processor does something to them, and the results are fed into a PyBaMM model.
There's ways around this already, eg. use a custom / standalone model, or subclassing a PyBOP model and overriding the simulate function to postprocess the parameters before they go off to PyBaMM. Neither of these are particularly convenient though, so it would be good to have the structure in PyBOP to natively handle this sort of parameter processing.
Motivation
Three cases where I've found myself trying to do this, all in the context of fitting Thevenin models:

Functional parameters: idea would be for PyBOP to optimise over a set of 'dummy parameters', eg. r0_at_soc_1, r0_at_soc_0d9, ..., and have a postprocessor build these into a pybamm.Interpolant from the pairs (SOC=1, R0=r0_at_soc_1), (SOC=0.9, R0=r0_at_soc_0d9), etc. This is useful for fitting linear interpolation flavoured functional parameters, since PyBOP can't optimise over a pybamm.Interpolant and the PyBaMM model doesn't natively take series resistance R0s as a list of scalars. Instead, the post-processor can act as a layer between PyBOP and PyBaMM, to close this gap.
Least squares regression of ECMs: consider fitting a 1-RC model with a fixed time-constant, using the GaussianLogLikelihood method. PyBOP can be used to identify the RC resistance $R$, but to build a complete PyBaMM model, we also need to set the RC capacitance as $C=\tau/R$. This again needs some sort of intermediate step between the parameter PyBOP suggests (in this case, $R$), and the parameters PyBaMM sees (here, $R$ and $C$).
Unsure how useful this will be in practice, but it would be interesting to see if fitting $1/R$ is more numerically stable than fitting $R$ directly. For a constrained optimisation where timescale $R*C \leq \tau_{max}$, the search space for parameter $R$ can be particularly small (think the small strip close to the $y$ axis of a $y=1/x$ curve). In that case, optimising over conductance $1/R$ will have a much larger search space, and could therefore alleviate some of the difficulties the optimisers can face. Again this needs a post-processor / translation layer between the $1/R$ that PyBOP sees, and the $R$ that gets fed into the PyBaMM model.

Possible implementation
Something like an optional callback function in the optimiser class. Takes a dict of named PyBOP parameters, and outputs a dict of named PyBaMM parameters.
Additional context
No response

I think what you are proposing is effectivly re-parameterising some base model to create a new model. I would suggest the best way to do this is to allow users to easily create and use a custom pybamm model with pybop. Taking your 2nd case, this might look something like:
custom_model = pybamm.SPM()
custom_model.substitute(pybamm.Parameter("C"), pybamm.Parameter("tau") / pybamm.Parameter("R))
pybop_model = pybop.create_model(custom_model)
# ... use new_pybop_model to do the fitting

(note pybamm.BaseModel.substitute and pybop.create_model don't exist yet, but could easily be created)

Thanks Martin, this seems like an intuitive solution to the problem! Two questions:

How would the substitution method cope with cancellation? Take the example of transforming from (R, C) to (R ,tau=RC). Do you think this implementation would substitute RC for tau (desired behaviour) or R*tau/R (more literal)?

I think we should continue to support common PyBaMM parameter sets to promote data sharing. But it should be fairly straightforward for the user to convert between parameter sets, given that they are in control of the substitution. Would it be possible to generate parameter set conversion functions as well?

How would the substitution method cope with cancellation? Take the example of transforming from (R, C) to (R ,tau=R_C). Do you think this implementation would substitute R_C for tau (desired behaviour) or R*tau/R (more literal)?

There is a bunch of simplification code in pybamm, not sure it would work (i.e. cancel the R's) in this particular example. Something to look at and test I think.

I think we should continue to support common PyBaMM parameter sets to promote data sharing. But it should be fairly straightforward for the user to convert between parameter sets, given that they are in control of the substitution. Would it be possible to generate parameter set conversion functions as well?

Yea that would be nice. You could do it but it would require storing the args of subsitute in the pybop model so you would have to rearrange the code above to:
custom_model = pybamm.SPM()
pybop_model = pybop.create_model(custom_model)
pybop_model.substitute(pybamm.Parameter("C"), pybamm.Parameter("tau") / pybamm.Parameter("R"))
# ... use pybop_model to do the fitting

Sounds good! Thanks

Hi,
I've written an example (building on #533) that replaces the parameter "C1 [F]" with "tau1 [s]" before tackling the same parameter estimation problem as described in @MarkBlyth's example ecm_tau_constraints.py.
I don't think the timescale is identifiable from this constant discharge data however, so to compare the numerical performance we would need to use more dynamic data (e.g. the pulse in this notebook).
@MarkBlyth, I believe this example resolves case 2 (above) and that the method can be extended to the other cases as well, what do you think?


Dihedral fix

This fixes the small bug in #458 changes to planeVisible didn't rebuild. Also adds a extendLine parameter. This defaults to true (displays as before), but if this is false the representation looks approximately like that requested by @giagitom in #458
(Apologies for the delay in doing this...)

Thank you very much @fredludlow !

Very nice, thank you @fredludlow

Oops, I'll have a look (the logic in the point generation is getting a bit convoluted, so maybe best to refactor it anyway)

thanks!


Does this work with Pinterest?

Never know where to ask these questions. Github needs a question forum. Also looks like you just added the ability to share directly with out the middleman dialog. That's exactly what I was looking for.

Any update to this regarding IOS Pinterest?

Seems like on iOS Pinterest sharing is already works by default sharing dialogue.


Update circe-generic, ... to 0.12.1

Updates

io.circe:circe-generic
io.circe:circe-generic-extras
io.circe:circe-literal
io.circe:circe-parser
io.circe:circe-refined

from 0.11.1 to 0.12.1.
Release Notes/Changelog
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention @scala-steward in the comments below.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "io.circe" } ]

labels: semver-minor

Included in #1036.


First Ctrl+R and Ctrl+F not showing long name variables correctly

Godot version:
3.1 rc2
OS/device including version:
Windows 10 x64
Issue description:
When trying to replace a long name variable for the first time section in GDScript editor, the cursor positioned at the end but the displayed name is shifted to the left and not appear correctly.
Pressing Ctrl+R again, it shows correctly.
Steps to reproduce:

Double click on a variable with a long name

Press Ctrl+R

Press Ctrl+R again

Minimal reproduction project:

New Game Project.zip

Also happens with Ctrl+F

It actually got slide to the left...probably some mismatch between cursor_pos and window_pos in line_edit.cpp. I am trying to solve....

To me it seems like window_width doesn't get set corretly when LineEdit::set_cursor_position(int p_pos) is called the first time.
This is where the variable is initially set with a smaller value than expected:
https://github.com/godotengine/godot/blob/260530f01caa8ea3e2fb7bfda04f4bd718ae14f5/scene/gui/line_edit.cpp#L1171
I'll try to find out more though ...

The problem seems to occur whenever the LineEdit is resized. This happens when the search bar is initially created and when the width is resized as @piyushdagar already mentioned.
Hence I looked at this piece of code:
https://github.com/godotengine/godot/blob/90cc1d3c1d2e77ca72c5949b21a5c40738abcd81/scene/gui/line_edit.cpp#L615
Why is the window_pos only set to zero when it should expand to text length?
Always resetting window_pos seems to solve this issue