Cannot create a stable cluster

I am using docker swarm. I did exactly as in the docker swarm example, except removing remoing the NODE_ADDRESS variable to let it autoconfigure.
I also feel like i have tried everything. Including making the network first and inspecting it to get the ip range, and using that for NODE_ADDRESS variable.
Does this project work in its current state?
galera_node.2.hfps0ofumgos@swarm-master-03    | 2018-02-27  9:38:23<PHONE_NUMBER>86624 [Note] WSREP: save pc into disk
galera_node.2.hfps0ofumgos@swarm-master-03    | 2018-02-27  9:38:23<PHONE_NUMBER>93920 [Note] WSREP: New COMPONENT: primary = yes, bootstrap = no, my_idx = 1, memb_num = 2
galera_node.2.hfps0ofumgos@swarm-master-03    | 2018-02-27  9:38:23<PHONE_NUMBER>93920 [Note] WSREP: STATE EXCHANGE: Waiting for state UUID.
galera_node.2.hfps0ofumgos@swarm-master-03    | 2018-02-27  9:38:23<PHONE_NUMBER>86624 [Note] WSREP: forgetting b663bab1 (tcp://<IP_ADDRESS>:4567)
galera_node.2.hfps0ofumgos@swarm-master-03    | 2018-02-27  9:38:23<PHONE_NUMBER>93920 [Note] WSREP: STATE EXCHANGE: sent state msg: f3f2f61c-1ba1-11e8-a872-5f4a2014eb6d
galera_node.2.hfps0ofumgos@swarm-master-03    | 2018-02-27  9:38:23<PHONE_NUMBER>93920 [Note] WSREP: STATE EXCHANGE: got state msg: f3f2f61c-1ba1-11e8-a872-5f4a2014eb6d from 0 (5dcd2316d4f6)
galera_node.2.hfps0ofumgos@swarm-master-03    | 2018-02-27  9:38:23<PHONE_NUMBER>93920 [Note] WSREP: STATE EXCHANGE: got state msg: f3f2f61c-1ba1-11e8-a872-5f4a2014eb6d from 1 (f6c9099d7e52)
galera_node.2.hfps0ofumgos@swarm-master-03    | 2018-02-27  9:38:23<PHONE_NUMBER>93920 [Note] WSREP: Quorum results:
galera_node.2.hfps0ofumgos@swarm-master-03    |         version    = 4,
galera_node.2.hfps0ofumgos@swarm-master-03    |         component  = PRIMARY,
galera_node.2.hfps0ofumgos@swarm-master-03    |         conf_id    = 11,
galera_node.2.hfps0ofumgos@swarm-master-03    |         members    = 1/2 (joined/total),
galera_node.2.hfps0ofumgos@swarm-master-03    |         act_id     = 0,
galera_node.2.hfps0ofumgos@swarm-master-03    |         last_appl. = 0,
galera_node.2.hfps0ofumgos@swarm-master-03    |         protocols  = 0/8/3 (gcs/repl/appl),
galera_node.2.hfps0ofumgos@swarm-master-03    |         group UUID = a156782c-1ba0-11e8-9506-f3d6550b8b55
galera_node.2.hfps0ofumgos@swarm-master-03    | 2018-02-27  9:38:23<PHONE_NUMBER>93920 [Note] WSREP: Flow-control interval: [23, 23]
galera_node.2.hfps0ofumgos@swarm-master-03    | 2018-02-27  9:38:23<PHONE_NUMBER>93920 [Note] WSREP: Trying to continue unpaused monitor
   joiner: => Rate:[   0 B/s] Avg:[   0 B/s] Elapsed:0:01:40  Bytes:    0 B ] Avg:[   0 B/s] Elapsed:0:00:10  Bytes:    0 B
galera_node.2.hfps0ofumgos@swarm-master-03    | WSREP_SST: [ERROR] Possible timeout in receving first data from donor in gtid stage (20180227 09:38:28.182)
galera_node.2.hfps0ofumgos@swarm-master-03    | WSREP_SST: [ERROR] Cleanup after exit with status:32 (20180227 09:38:28.186)
galera_node.2.hfps0ofumgos@swarm-master-03    | WSREP_SST: [INFO] Cleaning up fifo file /tmp/mysql-console/fifo (20180227 09:38:28.191)
galera_node.2.hfps0ofumgos@swarm-master-03    | rm: cannot remove '/tmp/mysql-console/fifo': Permission denied
galera_node.2.hfps0ofumgos@swarm-master-03    | 2018-02-27  9:38:28<PHONE_NUMBER>01216 [ERROR] WSREP: Failed to read 'ready <addr>' from: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '<IP_ADDRESS>' --datadir '/var/lib/mysql/'   --parent '41'  ''
galera_node.2.hfps0ofumgos@swarm-master-03    |         Read: '(null)'
galera_node.2.hfps0ofumgos@swarm-master-03    | 2018-02-27  9:38:28<PHONE_NUMBER>01216 [ERROR] WSREP: Process completed with error: wsrep_sst_xtrabackup-v2 --role 'joiner' --address '<IP_ADDRESS>' --datadir '/var/lib/mysql/'   --parent '41'  '' : 1 (Operation not permitted)
galera_node.2.hfps0ofumgos@swarm-master-03    | 2018-02-27  9:38:28<PHONE_NUMBER>10144 [ERROR] WSREP: Failed to prepare for 'xtrabackup-v2' SST. Unrecoverable.
galera_node.2.hfps0ofumgos@swarm-master-03    | 2018-02-27  9:38:28<PHONE_NUMBER>10144 [ERROR] Aborting
galera_node.2.hfps0ofumgos@swarm-master-03    |
galera_node.1.h1m8gi66g9va@node-02    | Error in my_thread_global_end(): 1 threads didn't exit
galera_node.1.h1m8gi66g9va@node-02    | MariaDB exited with return code (0)
galera_node.1.h1m8gi66g9va@node-02    | # GALERA saved state
galera_node.1.h1m8gi66g9va@node-02    | version: 2.1
galera_node.1.h1m8gi66g9va@node-02    | uuid:    00000000-0000-0000-0000-000000000000
galera_node.1.h1m8gi66g9va@node-02    | seqno:   -1
galera_node.1.h1m8gi66g9va@node-02    | safe_to_bootstrap: 1
galera_node.1.h1m8gi66g9va@node-02    | Goodbye

I forgot to mention that the nodes only live for 2 minutes. After that new containers replaces them.

I am having exactly the same issue. I also noticed that once this has occurred, then the seed node starts failing it's health check. Portainer reports that curl: (22) The requested URL returned error: 503 Service Unavailable which results in the seed container being cycled. I'm trying to read more about galera and xtrabackup in general to try and understand this part of the process and why the comms hang.
Some insight would very much be appreciated.

@cowai @smidge84 You're probably hitting https://jira.mariadb.org/browse/MDEV-15383?workflowName=MariaDB+v3&stepId=1 and https://jira.mariadb.org/browse/MDEV-15254 . It seems there are more patches coming, similar to the 10.1.31 release. My personal advice is using the 10.1.31 Dockerfile - I've already spent a few hours trying to get 10.2 running, without success.

@peter-slovak
Thank you for the tip.
Just to confirm. Would the tag "10.1.31-2018-02-20" work okay?

Yes, I'm fairly confident that 10.1.31-2018-02-20 is stable as it applies some patches to fix the major regressions:
https://github.com/colinmollenhour/mariadb-galera-swarm/blob/master/Dockerfile-10.1#L22
I was able to bootstrap my Kontena cluster using this patched 10.1.31 version. I've been told 10.2 works fine but I have not tried it myself.

@colinmollenhour I tried that image, and it has been stable for days now. Thanks! You may want to remove the 10.2 image or warn against it not working. Just a thought.

In my case I found out that in the 10.2 Mariadb image, the tail command used to watch /tmp/mysql../fifo fails, if the swarm node use the overlayfs. The tail from coreutils (8.23.4) in this image has a known bug, that this file system type is unknown. A newer debian brings coreutils 8.24.1, where that is fixed. Perhaps someone can confirm this in a test against aufs as overlay fs.

I use aufs and had other problems so this is probably not related. Thanks for the heads up as I wanted to go overlayfs very soon.

Yes, I'm fairly confident that 10.1.31-2018-02-20 is stable as it applies some patches to fix the major regressions:
https://github.com/colinmollenhour/mariadb-galera-swarm/blob/master/Dockerfile-10.1#L22
I was able to bootstrap my Kontena cluster using this patched 10.1.31 version. I've been told 10.2 works fine but I have not tried it myself.

It seems that now that image is not working. Setting colinmollenhour/mariadb-galera-swarm:10.1.31-2018-02-20 in the docker-compose.yml file will abort with this error:
InnoDB: No valid checkpoint found.
InnoDB: A downgrade from MariaDB 10.2.2 or later is not supported.
InnoDB: If this error appears when you are creating an InnoDB database,
InnoDB: the problem may be that during an earlier attempt you managed
InnoDB: to create the InnoDB data files, but log file creation failed.
InnoDB: If that is the case, please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/error-creating-innodb.html
2019-04-18 16:01:46<PHONE_NUMBER>98816 [ERROR] Plugin 'InnoDB' init function returned error.
2019-04-18 16:01:46<PHONE_NUMBER>98816 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed.
2019-04-18 16:01:46<PHONE_NUMBER>98816 [Note] Plugin 'FEEDBACK' is disabled.
2019-04-18 16:01:46<PHONE_NUMBER>98816 [ERROR] Unknown/unsupported storage engine: innodb
2019-04-18 16:01:46<PHONE_NUMBER>98816 [ERROR] Aborting

Error in my_thread_global_end(): 1 threads didn't exit
MariaDB exited with return code (0)
# GALERA saved state
version: 2.1
uuid:    7d806cd0-61ee-11e9-b306-43433cf9bcf9
seqno:   -1
safe_to_bootstrap:

I'm having a hard time to make the example work, so I found this issue and tried this image. Seems that this doesn't work either.


Cannot use install.sh, the url is 404

Hi, I  am following the server self host guide:  https://docs.plandex.ai/hosting/self-hosting
The guide says:
curl -sL https://plandex.ai/install.sh | bash

But that URL currently 404s
Thanks.

@irthomasthomas Sorry about that! Should now be fixed.


Keybase error when dragging files into private shared folder

Keybase GUI Version: 1.0.17-20161024112838+4d

cf351

Hi @juicetoon -- could you please run keybase log send from the Keybase command prompt and post the resulting log ID here?  Thanks!

Hi Jeremy - the log ID is a3af385e1dd632d4456671c
On 10 November 2016 at 17:07, Jeremy Stribling<EMAIL_ADDRESS>wrote:

Hi @juicetoon https://github.com/juicetoon -- could you please run keybase
log send from the Keybase command prompt and post the resulting log ID
here? Thanks!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/keybase/client/issues/4825#issuecomment-259609314,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIgMahXqaFxy48NZr3zBs7_x0oIujQ2Bks5q8rS-gaJpZM4KuTLi
.

CC @zanderz @taruti

The real log ID is a3af385e1dd632d445667f1c -- the one above is somehow missing a character.

I see two different sources for errors in the log, and I'm not sure which one you're reporting here.
The first is that it seems KBFS is struggling to upload a 300MB file to our servers without a certain timeout, even on the lowest bandwidth setting.  I think either your network connection or your CPU is the bottleneck.  The good news is we are currently testing fixes for this to make KBFS less reliant on the network connection and CPU performance.  In the meantime, maybe try again?  Sorry for the inconvenience.
The second is a timeout while trying to verify a user's identity while you were looking at a public folder.  It looks a little weird to me since the timeout occurs after only a second, whereas the KBFS timeout on Windows should be 29 seconds.
@patrickxb @maxtaco: this occurs in the service log of a3af385e1dd632d445667f1c:
31703 2016-11-10T19:37:33.704177 - [DEBU keybase notify_router.go:269] 5d00 FS activity: {true context deadline exceeded 2 3 4 map[tlf:XXX mode:read] 0}

(username redacted from privacy)
Is this "deadline exceeded" error coming from a deadline set by the service?  I assume so, since I don't think the KBFS deadline is transferred via RPC.  Is the deadline intentionally so short?  I recall something about canceling identifies quickly and retrying them, but in that case the error shouldn't be transferred back to KBFS right?  KBFS sees a "deadline exceeded" just after it appears in the service log:
2016-11-10T19:37:32.887014 - [DEBU kbfs(FBO 7f4f5b66) folder_branch_ops.go:837] 257d Running identifies on /keybase/public/XXX [tags:DID=7SW5GwLTF2C98F-mv5SYSw]
2016-11-10T19:37:33.704177 - [DEBU kbfs(FBO 7f4f5b66) folder_branch_ops.go:1322] 257e Done: context deadline exceeded [tags:DID=7SW5GwLTF2C98F-mv5SYSw]
2016-11-10T19:37:33.704177 - [DEBU kbfs kbfs_ops.go:402] 2580 Done: context.deadlineExceededError{} [tags:DID=7SW5GwLTF2C98F-mv5SYSw]
2016-11-10T19:37:33.704177 - [DEBU kbfs(FBO 7f4f5b66) folder_branch_ops.go:841] 257f Identify finished with error: context deadline exceeded [tags:DID=7SW5GwLTF2C98F-mv5SYSw]
2016-11-10T19:37:33.704177 - [DEBU kbfs dir.go:131] 2581 context deadline exceeded [tags:DID=7SW5GwLTF2C98F-mv5SYSw]

@patrickxb @maxtaco: d'oh, I just realized that the service log I'm citing is a notification for FS activity -- i.e., it's coming in from KBFS.  So the timeout must be happening there.
@songgao: there is a weird one-second deadline happening during an identify call, when it really should be 29 seconds.  I wonder if this is related to the delayed cancellation stuff?

@strib If you search for "/keybase/public/chris", the getMaybeCreateRootNode is called 29 seconds ago before the context is canceled. It looks like it's indeed the 29 timeout that kicked in?

@songgao: d'oh you are right, thanks.  Don't know how I missed that.
So, this does seem to be another instance of identifies of a certain well-known user taking a long time.  @patrickxb @maxtaco, starting at 2016-11-10T19:37:04.702367 in the log mentioned above.

Network calls in this trace are going extremely slowly:
31362 2016-11-10T19:37:20.649230 - [DEBU keybase notify_router.go:269] 5bab FS activity: { XXX }
31363 2016-11-10T19:37:32.335932 - [DEBU keybase api.go:644] 5bac - API call sig/get success

And:
31923 2016-11-10T19:37:37.177651 - [DEBU keybase identify2_with_uid.go:306] 5ddc - CheckCompleted
31924 2016-11-10T19:37:44.037705 - [DEBU keybase api.go:196] 5ddd | doRequestShared(Done) for XXXXX some web site

Not much that can be done. Since presumably the user is being tracked, all components have to be checked.  Is KBFS using a lot of bandwidth here BTW in another TLF?

@maxtaco: QR was happening but that shouldn't be bandwidth-intensive.  Ok, guess there's not much else to do here for now.

Looks like timeouts unfortunately.
We should degrade better when there is not enough network speed.


Adapter for loading Excel spreadsheet

We should allow for an adapter that is able to load Excel spreadsheets

Now part of CLARIAH/wp4-requirements#7

This adapter is already part of the code, but not 'active' yet.


Make your first barch & pull request

Each one needs to make there own branch, make a change (or add new files)
then push to origin  & send a pull request

Great!
Good practice is to add a comment when you close the issue.
Explaining what was done. What was fixed, enhanced, modified, documented, etc.


Issue with the REST API WorkItemUpdates

I don't know whether this is the right place to create this issue. But I am using TFS Rest API to lock the work items so as to stop other users from adding or removing relations from the work item.
To find the relation being added or deleted I am using the TFS WorkItemUpdates API which gets all the updates applied to the work item.
The problem I am facing are:
When a user adds a relation:
I assume that revisedDate corresponds to the date when the change occurred. But this date is sometimes MAX Datetime 9999-01-01T00:00:00Z. I want to know why?
When a user deletes a relation:
I see that the revisedBy in the corresponding update is representing the user who added this relation rather than the user who deleted the relation. This is stoping me from identifying the user to who deleted the relation.
This history when accessed from the history tab in the work item editor using TFS Website is correct. But when calling the REST API doesnot seem to reflect the same.
I feel this might not be right place to raise the issue. But any help or direction will be appreciated.

Sukrat - I believe what you are looking for is "changed date"..
Revised date is largely an internal field, that is used to track when a particular revision was replaced with a newer revision... (and the last revision, which is not replaced yet, has max date set for revised date)..

@amitgup-devops So, is there a way to access the changed date and changed by of all the work updates through the REST api?

@Sukrat yes, you can update changed by/date fields using REST APIs, if you have permissions to 'bypass rules' (as these fields are automatically populated by rules during work item update)
Some useful pointers:-
https://docs.microsoft.com/en-us/rest/api/vsts/wit/work items/update?view=vsts-rest-4.1
https://docs.microsoft.com/en-us/vsts/release-notes/2017/nov-28-vsts#grant-the-bypassrule-permission-to-specific-users
https://visualstudio.uservoice.com/forums/330519-visual-studio-team-services/suggestions/7173706-vso-rest-api-provide-a-way-to-disable-rules-for-m


update fog-openstack to 0.1.15

This fog-openstack update adds fixes to the workflow service lookup and various network updates.

@miq-bot add_label euwe/yes

@tzumainn there are some OpenStack InfraManager Refresh failures for this one, can you take a look?

@blomquisg Fixed - I talked to Ladislav and he mentioned that setting up an undercloud environment exactly as before is extremely difficult, so it's okay to change the spec tests.

@tzumainn, that's great! Thank you for addressing this issue.

@dclarizio , could you please review? This is blocking some of the network provisioning features recently added. Thanks!

@tzumainn @gildub seems like the specs might be pointing to some non working code after all?

@Ladas, assigning over to you ... if the specs are in good order, send it back my way

@Ladas hi!  most of the commented out specs have to do with attributes that simply don't exist in my undercloud setup.  I don't know if the original specs were done against a baremetal environment that might have more information available, but I don't have the capacity to duplicate that environment.

@Ladas I've made the changes you talked about - thanks!

looks good to me now :+1:

@miq-bot assign @blomquisg

Euwe Backport details:
$ git log -1
commit dfaa14a0d4a0c9a4c5c73a3f741d7dbf1015f671
Author: Greg Blomquist<EMAIL_ADDRESS>Date:   Mon Oct 17 10:42:29 2016 -0400

    Merge pull request #11747 from tzumainn/fog-openstack-update

    update fog-openstack to 0.1.15
    (cherry picked from commit dc17dc9f511490adbb1ecd622dc1a9d6f6a731f2)


FS_FILTER_QUERY_OPEN operation

There's no mention of the FS_FILTER_QUERY_OPEN operation

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f3aed758-d5dc-0128-62fb-2bc69c635540
Version Independent ID: 8351a690-5a4c-8c30-11f6-eb0a62ed3ded
Content: FsRtlRegisterFileSystemFilterCallbacks function
Content Source: wdk-ddi-src/content/ntifs/nf-ntifs-fsrtlregisterfilesystemfiltercallbacks.md
Service: unspecified
Product: windows-hardware
GitHub Login: @windows-driver-content
Microsoft Alias: windowsdriverdev

Hi @21street, thanks for letting us know. I've added the operation to the page, and it should be visible shortly. Thanks!


Add support for token auth and subdirectories.

Added two additional (optional) parameters to the SonarAPIHandler constructor:

subdirectory: In the event that sonar is not hosted under a root directory (/), allows the specification of a subdirectory (e.g. '/sonar').
token: Sonar authentication token. Can be provided in place of/addition to username and password.

Closes #46. Closes #47.

Coverage increased (+0.1%) to 95.683% when pulling d3bcda5c2a810174530d4b495cb203c2cbfb0936 on FireEater64:master into 5a9bc2a718fda994f0a4942168a453c1a23ff8e3 on kako-nawao:master.

Coverage increased (+0.1%) to 95.683% when pulling 30f191620cc992dc23b871fbada4170574421357 on FireEater64:master into 5a9bc2a718fda994f0a4942168a453c1a23ff8e3 on kako-nawao:master.

Token authentication requires an empty password field
curl https://localhost:9000/api/authentication/validate -u f21eccd163721442f7e371754ee047ed8a586307:
{"valid":true}%

Coverage increased (+0.1%) to 95.683% when pulling 6dd0ee3ecffa07a8452538842735936b9165a4d1 on FireEater64:master into 5a9bc2a718fda994f0a4942168a453c1a23ff8e3 on kako-nawao:master.

Coverage increased (+0.1%) to 95.683% when pulling 7f95d4c1753ba2e4e9883f3c5f55df5616bc93f5 on FireEater64:master into 5a9bc2a718fda994f0a4942168a453c1a23ff8e3 on kako-nawao:master.

Coverage increased (+0.1%) to 95.683% when pulling 7f95d4c1753ba2e4e9883f3c5f55df5616bc93f5 on FireEater64:master into 5a9bc2a718fda994f0a4942168a453c1a23ff8e3 on kako-nawao:master.

Coverage increased (+0.1%) to 95.683% when pulling 7f95d4c1753ba2e4e9883f3c5f55df5616bc93f5 on FireEater64:master into 5a9bc2a718fda994f0a4942168a453c1a23ff8e3 on kako-nawao:master.

I guess the only other thing that might be worth changing, is exposing base_path as a command-line parameter in the cmd projects - if you think that's worth doing?

Coverage increased (+0.2%) to 95.745% when pulling 08f048529b7aa3f3c4f1b485811da298f09960a4 on FireEater64:master into 5a9bc2a718fda994f0a4942168a453c1a23ff8e3 on kako-nawao:master.


Add module-info.java?

Can we have module-info.java in the release? Currently, when I try to import source buddy in my project (code is simplified):
module rua {
  requires SourceBuddy;
}

It gives me the following error:
¿\src\main\java\module-info.java:12: error: module not found: SourceBuddy
  requires SourceBuddy;
           ^

I should have made it a module from the beginning. I will take a look.

I created the release 2.5.2, which should address this issue. It is available from Maven Central.
The module name is not SourceBuddy, as opposed to what your comment implicitly suggested. That is not compliant with the usual namings. It is com.javax0.sourcebuddy, which is still not fully compliant with the conventions because of the training digit, but ... okay.
It was a great idea to make it modular, and I was happy to make it happen.

Thank you!
The module name was generated by IntelliJ IDEA autocompletion.


Update core-amqp, event hubs, service bus packages to use v1.1.0 of rhea-promise

v1.1.0 of rhea-promise is about to be released. We need to

[x] Run live tests for core-amqp, event hubs and service bus packages against the commit https://github.com/amqp/rhea-promise/commit/68fdda18a12f0140a59c418c23341e6e86c94538 . This is the branch where I have the relevant changes, but build is currently failing here: https://github.com/Azure/azure-sdk-for-js/tree/rhea-promise-update
[x] Get v1.1.0 of rhea-promise released
[ ] Update core-amqp, event hubs and service bus packages to use v1.1.0 as the min version. This will allow us to make use of the new features which were added

https://github.com/Azure/azure-sdk-for-js/pull/13683 for the last TODO.


Create abstract base class for inverters

Pull Request
Description

Aims to create an abstraction across different inverter types to simplify dealing with different models. This should also help with testing as we can treat "mock" inverters and real ones in the same way.
Aims to centralise pulling configuraiton from environment variables. Again this should help with testing as a settings object can be instantiated without needing env variables if required.

How Has This Been Tested?
Kind of difficult to test as I don't have access to all of these inverter types. However I have managed to get access to a GivEnergy system and I've managed to test this one out.
It would be good to mock out some API calls to various inverters so that they can be tested.

[x] Yes

If your changes affect data processing, have you plotted any changes? i.e. have you done a quick sanity check?

[x] Yes

Checklist:

[x] My code follows OCF's coding style guidelines
[x] I have performed a self-review of my own code
[x] I have made corresponding changes to the documentation
[x] I have added tests that prove my fix is effective or that my feature works
[x] I have checked my code and corrected any misspellings

Thanks @mduffin95, this is very neat, Really appreciate all this work youve done.
@aryanbhosale would you like to review this?

Thanks @mduffin95, this is very neat, Really appreciate all this work youve done.
@aryanbhosale would you like to review this?

Yup sure, I'll have a look at this right now

Thanks @mduffin95, this is very neat, Really appreciate all this work youve done.
@aryanbhosale would you like to review this?

Hey @peterdudfield I've added some reviews , do you want me to review something again or is this fine?

@aryanbhosale I can't see any comments. Have you submitted the review?

@aryanbhosale I can't see any comments. Have you submitted the review?

Hey yes I've added the reviews, check the file changes tab probably it'll show up there

@aryanbhosale any comments added should show up on this page.

@aryanbhosale any comments added should show up on this page. You'd also show up in the reviewers section if you'd submitted a review. Do the comments say "pending" next to them for you?
Yeah they say pending next to mine

Yeah they say pending next to mine

@aryanbhosale this means you need to submit the review before I can see the comments.

Yeah they say pending next to mine

@aryanbhosale this means you need to submit the review before I can see the comments.

Ah my bad, thank you for that(bad network on my side so didn't get submitted)

Have these been tested @mduffin95 - with the api and the dashboard?

With the givenergy credentials that you had ?

Have these been tested @mduffin95 - with the api and the dashboard?

With the givenergy credentials that you had ?

We havent shared the credentials, only you have them @aryanbhosale , would it be worth @aryanbhosale you testing out this solution with credentials, otherwise we should write more tests to make sure its working

Have these been tested @mduffin95 - with the api and the dashboard?

With the givenergy credentials that you had ?

We havent shared the credentials, only you have them @aryanbhosale , would it be worth @aryanbhosale you testing out this solution with credentials, otherwise we should write more tests to make sure its working

Yup sure will test

@mduffin95 you havent pulled the latest code btw, the api code has been updated since you'd last forked it, can you pull the latest code and make the relevant changes wherever applicable?

I think having separate directories for each inverter inside the /inverter folder and then further separating the files or folders(up to you) into classes/actual functionalities using those classes

I've implemented your requested changes @aryanbhosale :)

@mduffin95 you havent pulled the latest code btw, the api code has been updated since you'd last forked it, can you pull the latest code and make the relevant changes wherever applicable?

Pretty sure this branch is ahead of main? main hasn't been updated for about 5 days:

I think having separate directories for each inverter inside the /inverter folder and then further separating the files or folders(up to you) into classes/actual functionalities using those classes

I've implemented your requested changes @aryanbhosale :)

@mduffin95 you havent pulled the latest code btw, the api code has been updated since you'd last forked it, can you pull the latest code and make the relevant changes wherever applicable?

Pretty sure this branch is ahead of main? main hasn't been updated for about 5 days (highlighted is latest on main):

Your api/app folder looks like this, in the main it had a couple of other files,

I think having separate directories for each inverter inside the /inverter folder and then further separating the files or folders(up to you) into classes/actual functionalities using those classes

I've implemented your requested changes @aryanbhosale :)

@mduffin95 you havent pulled the latest code btw, the api code has been updated since you'd last forked it, can you pull the latest code and make the relevant changes wherever applicable?

Pretty sure this branch is ahead of main? main hasn't been updated for about 5 days (highlighted is latest on main):

I think having separate directories for each inverter inside the /inverter folder and then further separating the files or folders(up to you) into classes/actual functionalities using those classes

I've implemented your requested changes @aryanbhosale :)

@mduffin95 you havent pulled the latest code btw, the api code has been updated since you'd last forked it, can you pull the latest code and make the relevant changes wherever applicable?

Pretty sure this branch is ahead of main? main hasn't been updated for about 5 days (highlighted is latest on main):

thanks for these @mduffin95 ! i'll review them

@mduffin95 when i run python api/main.py, i see these errors, could you try to fix the circular import?

@mduffin95 when i run python api/main.py, i see these errors, could you try to fix the circular import?

the problem is line 5 in enphase_models.py, and line 12 in enphase.py

@mduffin95 when i run python api/main.py, i see these errors, could you try to fix the circular import?

the problem is line 5 in enphase_models.py, and line 12 in enphase.py

you basically cannot reference file a from file b and file b from file a at the same time

Yep I should have run that myself!
In that case I'm not sure I see the benefit of splitting the inverter class and settings class out into their own files. I think it overcomplicates it. The files aren't that big, and they all contain related logic (about one inverter type).

Yep I should have run that myself!
In that case I'm not sure I see the benefit of splitting the inverter class and settings class out into their own files. I think it overcomplicates it. The files aren't that big, and they all contain related logic (about one inverter type). I've reverted the changes.

I see these errors in your latest commit when i try to run api/main.py

I get these errors with enphase

I get these errors with enphase

It's the same issue with all the inverters - not just enphase

It looks like that's because the env file has multiple variables set up. I only tried it with variables for GivEnergy. If we want to allow that I should be able to tweak the settings.

@mduffin95 error with enphase:
INFO:     <IP_ADDRESS>:49020 - "POST /forecast/ HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 406, in run_asgi
result = await app(  # type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in call
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in call
await super().call(scope, receive, send)
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/applications.py", line 123, in call
await self.middleware_stack(scope, receive, send)
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in call
raise exc
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in call
await self.app(scope, receive, _send)
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/middleware/cors.py", line 85, in call
await self.app(scope, receive, send)
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 65, in call
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/routing.py", line 754, in call
await self.middleware_stack(scope, receive, send)
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/routing.py", line 774, in app
await route.handle(scope, receive, send)
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/routing.py", line 295, in handle
await self.app(scope, receive, send)
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/routing.py", line 77, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/routing.py", line 74, in app
response = await f(request)
^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/fastapi/routing.py", line 193, in run_endpoint_function
return await run_in_threadpool(dependant.call, **values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/concurrency.py", line 42, in run_in_threadpool
return await anyio.to_thread.run_sync(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 2177, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 859, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/api/app/api.py", line 45, in forecast
predictions_with_live = run_forecast(site=site, ts=timestamp)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/quartz_solar_forecast/forecast.py", line 112, in run_forecast
return predict_ocf(site, None, ts, nwp_source)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/quartz_solar_forecast/forecast.py", line 30, in predict_ocf
pv_xr = make_pv_data(site=site, ts=ts)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/quartz_solar_forecast/data.py", line 183, in make_pv_data
live_generation_kw = site.get_inverter().get_data(ts)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/quartz_solar_forecast/inverters/enphase.py", line 32, in get_data
return get_enphase_data(self.__settings)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/quartz_solar_forecast/inverters/enphase.py", line 163, in get_enphase_data
access_token = get_enphase_access_token(settings)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/quartz_solar_forecast/inverters/enphase.py", line 106, in get_enphase_access_token
conn.request(
File "/usr/lib/python3.11/http/client.py", line 1303, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.11/http/client.py", line 1314, in _send_request
self.putrequest(method, url, **skips)
File "/usr/lib/python3.11/http/client.py", line 1148, in putrequest
self._validate_path(url)
File "/usr/lib/python3.11/http/client.py", line 1248, in _validate_path
raise InvalidURL(f"URL can't contain control characters. {url!r} "
http.client.InvalidURL: URL can't contain control characters. "/oauth/token?grant_type=authorization_code&redirect_uri=https://api.enphaseenergy.com/oauth/redirect_uri&code=client_id='b5427210de994aaefbf6174523f16e59' system_id='3136663' api_key='67fd8c27327e0c982a24b224d860f509' client_secret='7d5161173a9d30757c9647303d54a545'" (found at least ' ')
GivEnergy plot with live kw might be inconsistent because of an extra/1000 - please check your code since the output should not be this far from the forecasted one

for Solis it doesnt fetch any live data: (Error retrieving Solis data: Timeout error occurred)

Solarman: this is the same as givenergy, please look out for the extra /1000 which is causing this inconsistency

i think once these errors are solved this can e merged in

It looks like that's because the env file has multiple variables set up. I only tried it with variables for GivEnergy. If we want to allow that I should be able to tweak the settings.

@mduffin95 error with enphase:
INFO:     <IP_ADDRESS>:49020 - "POST /forecast/ HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 406, in run_asgi
result = await app(  # type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in call
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in call
await super().call(scope, receive, send)
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/applications.py", line 123, in call
await self.middleware_stack(scope, receive, send)
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in call
raise exc
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in call
await self.app(scope, receive, _send)
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/middleware/cors.py", line 85, in call
await self.app(scope, receive, send)
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 65, in call
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/routing.py", line 754, in call
await self.middleware_stack(scope, receive, send)
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/routing.py", line 774, in app
await route.handle(scope, receive, send)
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/routing.py", line 295, in handle
await self.app(scope, receive, send)
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/routing.py", line 77, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/routing.py", line 74, in app
response = await f(request)
^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/fastapi/routing.py", line 193, in run_endpoint_function
return await run_in_threadpool(dependant.call, **values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/starlette/concurrency.py", line 42, in run_in_threadpool
return await anyio.to_thread.run_sync(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 2177, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 859, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/api/app/api.py", line 45, in forecast
predictions_with_live = run_forecast(site=site, ts=timestamp)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/quartz_solar_forecast/forecast.py", line 112, in run_forecast
return predict_ocf(site, None, ts, nwp_source)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/quartz_solar_forecast/forecast.py", line 30, in predict_ocf
pv_xr = make_pv_data(site=site, ts=ts)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/quartz_solar_forecast/data.py", line 183, in make_pv_data
live_generation_kw = site.get_inverter().get_data(ts)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/quartz_solar_forecast/inverters/enphase.py", line 32, in get_data
return get_enphase_data(self.__settings)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/quartz_solar_forecast/inverters/enphase.py", line 163, in get_enphase_data
access_token = get_enphase_access_token(settings)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/aryan/OneDrive - BITS Pilani K K Birla Goa Campus/Desktop/myLab4Prac/GSoC_openclimatefix/Open-Source-Quartz-Solar-Forecast/quartz_solar_forecast/inverters/enphase.py", line 106, in get_enphase_access_token
conn.request(
File "/usr/lib/python3.11/http/client.py", line 1303, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/lib/python3.11/http/client.py", line 1314, in _send_request
self.putrequest(method, url, **skips)
File "/usr/lib/python3.11/http/client.py", line 1148, in putrequest
self._validate_path(url)
File "/usr/lib/python3.11/http/client.py", line 1248, in _validate_path
raise InvalidURL(f"URL can't contain control characters. {url!r} "
http.client.InvalidURL: URL can't contain control characters. "/oauth/token?grant_type=authorization_code&redirect_uri=https://api.enphaseenergy.com/oauth/redirect_uri&code=client_id='CLIENT_ID' system_id='SYSTEM_ID' api_key='API_KEY' client_secret='CLIENT_SECRET'" (found at least ' ')
GivEnergy plot with live kw might be inconsistent because of an extra/1000 - please check your code since the output should not be this far from the forecasted one

for Solis it doesnt fetch any live data: (Error retrieving Solis data: Timeout error occurred)

Solarman: this is the same as givenergy, please look out for the extra /1000 which is causing this inconsistency

i think once these errors are solved this can be merged in

please look out for the extra /1000 which is causing this inconsistency

@aryanbhosale have you checked this functionality on the main branch? My change doesn't alter this part of the code.
Regarding this issue:
URL can't contain control characters. "/oauth/token?grant_type=authorization_code&redirect_uri=https://api.enphaseenergy.com/oauth/redirect_uri&code=client_id='CLIENT_ID' system_id='SYSTEM_ID' api_key='API_KEY' client_secret='CLIENT_SECRET'"

I've pushed a chane that I think might fix this. If it doesn't work do you think you might be able to take a quick look to find the fix yourself? As I don't have enphase credentials it's difficult for me to do.

please look out for the extra /1000 which is causing this inconsistency

@aryanbhosale have you checked this functionality on the main branch? My change doesn't alter this part of the code.
Regarding this issue:
URL can't contain control characters. "/oauth/token?grant_type=authorization_code&redirect_uri=https://api.enphaseenergy.com/oauth/redirect_uri&code=client_id='CLIENT_ID' system_id='SYSTEM_ID' api_key='API_KEY' client_secret='CLIENT_SECRET'"

I've pushed a chane that I think might fix this. If it doesn't work do you think you might be able to take a quick look to find the fix yourself? As I don't have enphase credentials it's difficult for me to do.

What do you think could be the issue here?

@mduffin95 everything works currently but with enphase its a little weird,
this is the streamlit ui:

even after i type the url, when i go back to my CLI, i see this:

This shouldnt happen, if im using something in the API/dashboard context everything should be done on the dashboard side itself, and when im using the CLI context, it should show up in the CLI, there shouldnt be a mismatch bw the 2, the code for this is currently in the recently pushed main branch of this repo
anyway if i paste the url in the cli after its prompted, it says this:
enphase.py", line 115, in get_enphase_access_token
access_token = data_json["access_token"]
~~~~~~~~~^^^^^^^^^^^^^^^^
KeyError: 'access_token'
We're using env vars to store the access/refresh token as of now when using the cli context, and conditionally checking if they exist, then dont go into the CLI context(in the current main branch in enphase.py)
Would be great if you could fix this, everything else works perfectly fine now

one more thing @mduffin95 , when No Inverter is chosen, the columns should have only power_kw like this:

currently it has both power_kw and power_no_live_kw, and theyre both the same which doesnt make sense to have, it would be good if that change is made too before merging this in

Great work here @mduffin95 ! It's fixed and works as expected. Thanks a lot for this.
@peterdudfield I'm happy for this to be merged in, would you like to have a look at it before we merge this in?

hey @mduffin95 , do you mind having a look at the current docs for the api, as well as the dashboard_2 docs, and see if your changes affect anything there? and update it accordingly, it would be great if we could get this done in the same relevant PR

I've added a line to the docs about making sure new inverter models extend AbstractInverter. Other than this I don't think any changes are required really because this is a refactoring and doesn't add new functionality.

I've added a line to the docs about making sure new inverter models extend AbstractInverter. Other than this I don't think any changes are required really because this is a refactoring and doesn't add new functionality.

Thank you for this

Nice work on this!

Can this be merged?

Nice work on this!

Can this be merged?

If yourself and @peterdudfield are happy with it too then yes I would say go ahead, maybe squash and merge as it's quite a few commits

In fact I would say go ahead now, you worked on this most @aryanbhosale so your approval was the key thing, so feel free to merge @mduffin95! Thanks again


tests(cdc): fix data race in TestMessageClientBasics

What problem does this PR solve?

Issue Number: ref #3825
What is changed and how it works?

Made mockSendMessageClient thread-safe.

Check List
Tests

Unit test

Related changes

Need to cherry-pick to the release branch

Release note
None

/run-integration-test
/run-kafka-integration-test

/merge

/run-integration-tests

/run-integration-tests /tidb=pr/30558
/run-kafka-integration-test /tidb=pr/30558


Records and Webhooks occurs even if event did't happend

Problem:
If some exception happens during request handling (such /api/update-user and e.t.c. ) we still will have new Record and webhook call.
Reason:
beego.InsertFilter("*", beego.BeforeRouter, routers.RecordMessage)
Proposal:
Change to beego.InsertFilter("*", beego.AfterExec, routers.RecordMessage)
Benefits:

No false records / webhook calls in case of panic
There is option to include response status code to record  / webhook
No possibility to race condition ( when webhook calls lead to API request before the entity is updated in the database

@ISulimanov sounds good, can you make a PR?


File system is not available the first time the app is built

Whenever I build my all for the first time "npm run dev", filesystem operations are not successful.
In the example below, the callback is never called. However, when I refresh the window the file is read successfully. The reason for this test is because I did not have much success with Nedb either (from the docs). It seems to only work after I hit refresh as well. Setting a timeout did not help either, so I don't know if it's not ready or I'm doing something wrong here.
From my main.js file:
// I know this file exists
const myFile = path.join(remote.app.getPath('userData'), '/lastConnection.db');
const fs = require("fs");

fs.readFile(myFile , function (err, data) {
  if (err) {
    return console.error(err);
  }
  // This is never logged until I hit F5
  console.log("Asynchronous read: " + data.toString());
});

Node version: 8.8.1
Operating System: Windows

npm install --save-dev read-file-utf8

import Read from 'read-file-utf8';
import path from 'path';

try {
  const file = Read(path.join(remote.app.getPath('userData'), '/data.json'))
} catch (error) {}

I hope it helps you.

@PeachT Unfortunately I still can't read the file the first time the app is loaded. The strange thing is that this only happens in developpement, I did not have this problem in production. This is an almost clean project and I had this problem on two machines hence the bug repport here. I'll keep investigating the issue.


Start developing cli unit tests

Created 2 basic unit tests for the CLI to start out.
Ref: #72

Yay.

Awesome, more tests the better in my book!

As you're adding a new test file, please make sure you also add it to the linting tests


MODELIX-708: find a configuration for open api generator where the compiler helps finding missing implementations

Picks up #526 by adapting the OpenAPI generator templates such that abstract base classes for the different APIs are generated. This ensures that the implementation can't diverge much from the spec.
Notable mentions:

I've dropped useless OpenAPI specs for things that cannot be formalized via OpenAPI
A v1 spec has been extracted to free the v2 spec from these concerns
Where possible, handlers where implemented via the generated base classes
The generation / retrieval of ID has been extracted into a separate API as an example of how we can continue with the remaining routes later.
The duplication of the Problem data type and having to serve multiple Swagger UIs is not ideal. I'll try to resolve this in a follow-up PR.

There would be more to do to get to a clean state. However, I want to prevent this PR from growing with increased likelihood for conflicts. Everything is functional, so this should be fine to merge in the current state.
Review is probably easiest on a per-commit basis.
To be verified by reviewers

[ ] Relevant public API members have been documented
[ ] Documentation related to this PR is complete

[ ] Boundary conditions are documented
[ ] Exceptions are documented
[ ] Nullability is documented if used

[ ] Touched existing code has been extended with documentation if missing
[ ] Code is readable
[ ] New features and fixed bugs are covered by tests

:tada: This PR is included in version 8.3.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Wireguard does not start at boot time

Hello everyone,
I have already reported the error in the Opnsense forum. Unfortunately without success.
Error: If a tunnel endpoint with an FQDN is set up in the Wireguard plugin, the Wireguard service does not start after a firewall reboot.
Reproduction:
If I remove the FQDN from the VPN tunnel, the Wireguard service starts directly when the firewall boots.
I have already described everything in the following topic:
https://forum.opnsense.org/index.php?topic=30981.0
Regards

It was found that the plugin does not register any "newwanip" event which means that once DHCP or PPPoE comes up WireGuard will not reconnect as it should. If anyone want's to take a stab at it please be my guest @jkellerer @patschi @mimugmail

Ok, that means the phenomenon is completely normal at the moment and may be fixed in future updates.

Hi @fichtner, this means technically, in plugins.inc.d/wireguard.inc we need:
function wireguard_configure()
{
    return [
        'newwanip' => ['wireguard_reload'],
    ];
}

... and an implementation of wireguard_reload in the same file?

@jkellerer Basically, yes. However, to avoid spurious reloads said wireguard_reload function will receive as a parameter the interface name reloading at the moment. I was wondering if an optional interface name selection (multiple would be sane as well for multi-wan perhaps) and then try to reload only the the ones that are bound to said interface(s) to reload when an IP change is happening on them.
It's obviously not a key feature for the current user base so trying to keep it optional would be best?

(I don't have a setup to test but if unconditional reload does not disrupt the tunnelling that would be fine as well and a lot simpler)

I was wondering if an optional interface name selection (multiple would be sane as well for multi-wan perhaps) and then try to reload only the the ones that are bound to said interface(s) to reload when an IP change is happening on them.

I agree
I have tested this briefly by using non existing hostnames, editing the wireguard configuration files directly and starting the service.
Some findings:

If a tunnel has multiple peers and at least one of them fails dns resolution, the tunnel interface is not bringed up.

If there are multiple tunnels and at least one of them is up, the service is considered up in the webui, though service wireguard status shows an error for the failed interface and returns 1.

You can't use any wg ... <interface> command if the interface isn't up.

You can't use service wireguard reload for the same reason.

The only way that I found is to use wg-quick up <interface>, like wireguard_start does in /usr/local/etc/rc.d/wireguard
If the interface already exists it just errors out doing nothing, otherwise it brings up the interface with its configuration.
To check which interfaces are up you can use wg show interfaces, its output is just a line with interfaces separated by space.
Then there is the need to group the interface in the wireguard group like implemented in /etc/rc.conf.d/wireguard
While looking at that I discovered a new bug which I'll report soon.
The script suggested in the forum resolve-dns.bash, which is meant to be run with cron won't work in this case because it uses wg.

I just ran into this issue of WireGuard not connection properly at startup. As this issue seems not resolved, can we reopen it, please?

I'm still experiencing this on OPNsense 23.7.12-amd64. If I use WireGuard with an FQDN it stalls during boot. If I use an IPv4 address it works fine. It would be nice to be able to use an FQDN instead of needing to use a static IP or risk the endpoint dynamic IP changing.

Your issue may be symptomatic to sloppy DNS resolution or intermittent reconnects depending on your WAN type. The result may be that WireGuard isn't up, but it's not the root cause. Given there is no more information it's impossible to speculate (and it doesn't fit this closed ticket).

Thank you, I was going to fill out a new bug report but upon testing this I discovered that my DNS server (AdGuard Home) wasn't up early enough in boot to resolve DNS. Upon testing using an external DNS server, WireGuard resolves as expected.

Ok, glad to hear. 😊


vs code stopped showing syntax errors, I work in laravel, vs code inbuilt IntelliSense stopped working when I downgrade my vs code to JAN 2021 version it throws error the problem is with the new version of vs code only

Does this issue occur when all extensions are disabled?: Yes

VS Code Version:  April 2021 version 1.56
OS Version: Windows 10 Home 20H2

vs code stopped validating the syntax errors when I degrade it to March 2021 version it shows errors automatically. The problem is with the vs code April 2021 version. I work in laravel framework later this week i got an vs code update and i did it but suddenly vs code stopped validating the syntax error and again i degraded my vs code to the last version and it started validating the problem is with the the new version of vs code now i'm using php extensions to validate the syntax errors

/needsMoreInfo

Same problem, PHP syntax error stopped working after updating to 1.56.2. I downgrade to 1.55.2 and syntax error is working again.
Problem with:
Version: 1.56.2
OS: Linux x64 5.8.0-50-generic

Confirm the issue on Debian/Linux 9.13 (stretch) and PHP 7.0.33. VSCode version is 1.56.2

Confirm the issue both using the php installed from the repository and using php by xampp 7.4 and 8.0.6.
My OS is Linux Mint 20.1 (Ulyssa)
Vscode info:
Version: 1.56.2
Commit: 054a9295330880ed74ceaedda236253b4f39a335
Date: 2021-05-12T16:45:26.313Z
Electron: 12.0.4
Chrome: 89.0.4389.114
Node.js: 14.16.0
V8: <IP_ADDRESS>-electron.0
OS: Linux x64 5.4.0-73-generic

@roblourens Now I am using 1.61.0, and it's working without problems.


[gn] introduce build_root variable

Problem
The build folder has to be always present in the GN build root directory which in some cases requires creating a symbolic link for it (which is not available under Windows).
Summary of Changes
This PR:

Introduce build_root variable and build.gni file.
Modifies the criteria of importing ot_br_posix.gni file.
Modifies the output dir for shell library,
Defines a custom platform's build_overrides files for nrfconnect.

Rebased, and introduced a build_root variable and build.gni file after getting feedback from @mspang.
Additionally, bumped pigweed version, so now it is possible to build firmware under Windows OS as well.

Rebased, and introduced a build_root variable and build.gni file after getting feedback from @mspang.
Additionally, bumped pigweed version, so now it is possible to build firmware under Windows OS as well.

New changes:

Rebas to master
Bump pigweed and GN versions to support --root-target option and building under Windows
Build nrfconnect platform from CHIP root directory

Note that bumping pigweed will not casue problems like with #4490.

New changes:

Rebas to master
Bump pigweed and GN versions to support --root-target option and building under Windows
Build nrfconnect platform from CHIP root directory

Note that bumping pigweed will not casue problems like with #4490.

@woody-apple @bzbarsky-apple - could you add the checkmark from Apple side?

@woody-apple @bzbarsky-apple - could you add the checkmark from Apple side?


Improve performance of ChiselEnum annotations

ChiselEnums check if they should create annotations every time an instance of them is bound. Because so many annotations would be created, they check to see if an equivalent annotation has already been added to the annotations. Previously, this used a linear search of the annotations, now it uses a HashSet.
I noticed this inefficiency glancing at a Flamegraph of a very large design. In that case, just eyeballing where this is obviously showing up, it's at least 18% of the runtime. As with all things that are fundamentally O(n*m), the benefits from this change will be greatest in large designs with lots of annotations (n) and lots of ChiselEnums (m).
Contributor Checklist

[NA] Did you add Scaladoc to every public function/method?
[ ] Did you add at least one test demonstrating the PR?
[x] Did you delete any extraneous printlns/debugging code?
[x] Did you specify the type of improvement?
[ ] Did you add appropriate documentation in docs/src?
[x] Did you state the API impact?
[x] Did you specify the code generation impact?
[x] Did you request a desired merge strategy?
[x] Did you add text to be included in the Release Notes for this change?

Type of Improvement

performance improvement

API Impact
No impact
Backend Code Generation Impact
No impact
Desired Merge Strategy

Squash

Release Notes
Improve performance of ChiselEnum
Reviewer Checklist (only modified by reviewer)

[ ] Did you add the appropriate labels?
[x] Did you mark the proper milestone (Bug fix: 3.4.x, [small] API extension: 3.5.x, API modification or big change: 3.6.0)?
[ ] Did you review?
[ ] Did you check whether all relevant Contributor checkboxes have been checked?
[ ] Did you do one of the following when ready to merge:

[ ] Squash: You/ the contributor Enable auto-merge (squash), clean up the commit message, and label with Please Merge.
[ ] Merge: Ensure that contributor has cleaned up their commit history, then merge with Create a merge commit.

@jackkoenig This would be a good use case for providing a Builder local cache for libraries. Maybe with Keys similar to how the CDE works. There are many different reasons why a library might want to cache things local to a Builder context, e.g., Diplomacy fails to elaborate in parallel because it has its own global cache.

@ekiwi I agree, and Instantiate being another good sample: https://github.com/chipsalliance/chisel3/pull/2717


tests culture

Test scripts should be moved away from the root directory. Also, why there is two files, tests_run.sh and run_tests.sh and what is their purpose? This should be all done in the tests directory. Also with unit-tests_run.sh, mypy_run.sh, lint_run.sh and others.
Also, one of the run tests scripts (tests_run i think) should use the -P (or --directory-prefix) option of wget instead of cd-ing there and back again. The test-output directory should  be generated somewhere else than in the root of the repository, preferable in a temporary location either in /tmp or in a tmp dir in subdirectory of tests.

It is also needed to add documentation for these tests.

Some (http://as.ynchrono.us/2007/12/filesystem-structure-of-python-project_21.html) even say that directory for unit tests should be inside the package.

I was planning on moving these files, will do that after #3. Unit tests already are inside the package. What should be in the documentation?

The question is what shouldn't...
These things definitely should: How the tests are run, which is which and what do they test.

OK, I will write a readme file after I move the files.

All done.


wrong changes

Fix wrong path for 4.15

Hello @dmrost. This dockerfile is no longer in use. In 4.15 we changed the way to operator images are build. We actually need to delete Dockerfile.metadata. Do you have any use for it?

/ok-to-test

@dmrost the file can stil be used for testing purposes. Though, only for that.

/override ci/prow/e2e-aws-operator


[SPARK-36201][SQL][FOLLOWUP] Schema check should check inner field too

What changes were proposed in this pull request?
When inner field have wrong schema filed name should check field name too.

Why are the changes needed?
Early check early faield
Does this PR introduce any user-facing change?
No
How was this patch tested?
Added UT

ping @dongjoon-hyun @HyukjinKwon @cloud-fan

Test build #141218 has started for PR 33409 at commit cda8c31.

Test build #141218 has finished for PR 33409 at commit cda8c31.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141218/

Test build #141220 has started for PR 33409 at commit 471d1b8.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45731/

Kubernetes integration test unable to build dist.
exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45733/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45731/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45731/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45733/

Test build #141229 has started for PR 33409 at commit 8285ef0.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45743/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45743/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45743/

Test build #141220 has finished for PR 33409 at commit 471d1b8.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141220/

@AngersZhuuuu would you mind retriggering the test (https://github.com/AngersZhuuuu/spark/actions/runs/1044234803)?

@AngersZhuuuu would you mind retriggering the test (https://github.com/AngersZhuuuu/spark/actions/runs/1044234803)?

Done

Test build #141245 has started for PR 33409 at commit 3303ccb.

Kubernetes integration test unable to build dist.
exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45759/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45759/

Test build #141229 has finished for PR 33409 at commit 8285ef0.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141229/

Test build #141245 has finished for PR 33409 at commit 3303ccb.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141245/

hm, GA test seems failing.

hm, GA test seems failing.

Seems all my UT failed like this, can't find why..

Test build #141291 has started for PR 33409 at commit c43d1b4.

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45805/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45805/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45805/

Wow, the GitHub Action failures look really weird.

The error code is the following. It looks like OOM happening again.
 ./build/mvn: line 178:  1699 Killed                  "${MVN_BIN}" "$@"
2021-07-20T04:37:47.6486105Z ##[error]Process completed with exit code 137.

It seems that the master branch's Java 17 job is suffering with the same reason.

By the way, when this can be fixed and merge this then I can start the follow up work?

Test build #141315 has started for PR 33409 at commit 6250d3d.

Test build #141291 has finished for PR 33409 at commit c43d1b4.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
protected abstract class ConnectionProviderBase extends Logging

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141291/

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45830/

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45830/

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45830/

Test build #141315 has finished for PR 33409 at commit 6250d3d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141315/

GA is quite unstable now. I'm merging it to master/3.2 since jenkins has passed


Grid Mode Participant Camera Seems to be turned off after a while

Hello, i ve been using you application in production and i ve also implemented some of the  extra features you have implemented like the grid mode.
When the participants are above a certain number, the participants camera's seem to close when in reality they are open.
I 've had some trouble with this and cannot find the cause, i ve changed clientTrackSwitchOffControl: manual' but it seems the problem keep persisting.
Thank you for your time
(Sorry for my english)

Hi @Zeusakos - thanks for the question!
It sounds like tracks are still getting switched off even though you have set clientTrackSwitchOffControl to manual, and to me this sounds like expected behavior.
The clientTrackSwitchOffControl setting controls a relatively new feature that was added to the twilio-video SDK. When clientTrackSwitchOffControl is set to auto (which is the default setting in this app), the SDK will automatically detect when a track is not visible (either because the track's video element has left the viewport, or because the user has switched browser tabs), and it will switch off the track to save bandwidth. It's a great feature that can really optimize the performance of a video app, and we recommend that this setting be set to auto.
Developers can change this setting to manual so that their application can control whether or not a track is switched off (with remoteTrack.switchOff() or remoteTrack.switchOn()). This is usually only done when the SDK cannot detect the visibility of a track, which is not a very common occurrence.
 The SDK may still chose to switch off tracks in cases where a user's available bandwidth may be low. In these cases, the SDK will switch off lower priority tracks, so that the higher priority tracks can still be seen. I suspect that this is what is happening in your app.
The SDK does offer some settings that will change when tracks are switched off due to network congestion. Please see this section of our documentation for more information about these settings: https://www.twilio.com/docs/video/tutorials/using-bandwidth-profile-api#understanding-trackSwitchOffMode
I'd recommend that clientTrackSwitchOffControl be set to auto. With this setting, all off-screen tracks will be switched off, and this will free up bandwidth so that the SDK can display the tracks that are on-screen. With clientTrackSwitchOffControl set to manual, the SDK will try to display all video tracks (even the ones off screen), and this will likely cause some of the on-screen tracks to be switched off due to insufficient bandwidth.
Please let me know how this works out for you! I can further assist if you can provide more information about the settings that you are using, in addition to the changes that you have made to the app. The more technical information that you can provide us, the better. Thanks!

Hey @Zeusakos!
If you're looking for less agressive track switch-offs, I think it can be worth it to try trackSwitchOffMode: detected. This is what our docs have to say about the detected mode:

When detected is used, track switch-offs will only happen after congestion actually happens. Thanks to this, detected will avoid any unnecessary switch-offs. However, as congestion is detected through packet loss, a downlink bandwidth drop when using this mode will typically cause choppy audio and frozen video during a few seconds. We only recommend to use detected if video continuity is very important for your use-case.

So it will eliminate unnecessary switch-offs, at the risk of introducing choppy audio. Could be worth a try! Let me know how it works for you.


[bugfix] checkbox click not working, upgrade vuetify etc. #32, #24

Also removes yarn.lock because I totally forgot that this project uses NPM and adding yarn.lock in #25 was a mistake.
fixes #32, #24

@BryonLewis or @AlmightyYakob I'd really appreciate if someone else could regression-test this since it's a minor-version upgrade of Vuetify.


poc: create avatar renderer cli

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (337b37f) 14.55% compared to head (da78663) 14.55%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #81   +/-   ##
=======================================
  Coverage   14.55%   14.55%
=======================================
  Files          96       96
  Lines       11542    11542
=======================================
  Hits         1680     1680
  Misses       9862     9862

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Make service-account secrets in kubectl form.

Changes approach taken in #5470
Instead of creating a kubernetes_auth file,
which we are tring to get away from, creates
a kubeconfig file, which is the new hotness.
Instead of creating the kubeconfig file in the
kube-addon script on the master, it creates
it at the time of salt-overlay generation.
More information is available at this time.
In particular, the master certs and master address
are handy at this point, so those are included in
the kubeconfig file.
The kube-addons script is simplified because the secret objects
are now just plain olf yaml files, which it knows how to
create, just like it creates pods and services.
Kubectl is used to generate the kubeconfig file.
This ensures correct format and is more self-documenting,
and matches how the admin credentials are done.
TODO(erictune): do this for kubelet.

@zmerlynn reviewed the last PR like this.  If he is available, he is a good reviewer for this.
@cjcullen is starting in this area and could review, and @jlowdermilk has some context too.

Sure. I need to grab a beer anyways.

LGTM except some nits.

This doesn't work at all.
cluster/common.sh is not available from within cluster/gce/configure-vm.sh.  The latter runs on the VM.
Also, kubectl is not available on the VM, at least at the point where this runs, which is before salt starts.
Also, the certs are not available at this point.
So, configure-vm is the wrong place to be setting this stuff up.  I am going to have to go back to the kube-addons approach.

@erictune: Sorry for not noticing. I blame Friday afternoon.
 I feel like the create-salt-auth call is way early in configure-vm.sh - I put it in that order for legacy reasons when I factored all the things into functions, but honestly, there's no reason the auth files need to get created that early. I think, but don't quote me, that you could actually move create-salt-auth right before run-salt.

(In fact, I suspect the logical thing is to move ensure-kube-token / create-salt-pillar / create-salt-auth all down to just before run-salt.)

Doing it before run-salt is still too early to use kubectl and too early to read the certs.


added a definition for Lundau Length

This adds a definition for Lundau Length, which is a length on one of the  Plasma parameters: https://en.wikipedia.org/wiki/Plasma_parameters, suggested by Will DeShazer.

This has updated documentation for the Lundau_Length (also called the classical distance of closest approach).


Fails to start debugging on Arch.

Environment data
dotnet --info output:
.NET Core SDK (reflecting any global.json):
 Version:   2.1.403
 Commit:    04e15494b6

Runtime Environment:
 OS Name:     manjaro
 OS Version:
 OS Platform: Linux
 RID:         arch-x64
 Base Path:   /opt/dotnet/sdk/2.1.403/

Host (useful for support):
  Version: 2.1.5
  Commit:  290303f510

.NET Core SDKs installed:
  2.1.403 [/opt/dotnet/sdk]

.NET Core runtimes installed:
  Microsoft.NETCore.App 2.1.5 [/opt/dotnet/shared/Microsoft.NETCore.App]

VS Code version: 1.27.2
C# Extension version: 1.16.2
Steps to reproduce

Run vs code on a new console application
Add default VSCode launch.json for .NET Core, fix paths.
Press f5.

Expected  behavior
For debugging to begin.
Actual behavior
Debugging fails to start with a popup in the corner saying
Error processing 'configurationDone' request. Unknown Error: 0x89720010

@ThisDrunkDane This means you are not using an official version of VS code. Only official versions from Microsoft support .NET Debugging.


Promtool should validate service discovery files

Proposal
Offer a way to validate json and yml files loaded by file_sd_configs, either by:

updating promtool check config
adding a dedicated command promtool check file_sd_configs

A dedicated command would be better suited because changing the targets list dynamically does not require a prometheus restart/reload, meaning the files themselves are not required @ prometheus start.
Situation
promtool check config validates that one or more files exist for a specific file-sd configuration, but their contents are not checked.
There is no way to validate the files before publishing them.
The only way to detect the issues without checking the logs is adding an alert on prometheus_sd_file_read_errors_total.
Example bad config that passes check config
prometheus.yml
scrape_configs:
  - job_name: 'node'
    file_sd_configs:
    - files: ['./service_discovery/node.yml']

./service_discovery/node.yml
- targets:
  - <IP_ADDRESS>:9100
  labels: { example: nocomma between:labels }

Current behaviour
Output of promtool check config
Checking prometheus.yml
  SUCCESS: 0 rule files found

If the file is missing, a warning is issues:
Checking prometheus.yml
  WARNING: file "service_discovery/node.yml" for file_sd in scrape job "node" does not exist
  SUCCESS: 0 rule files found

Both scenarios currently have an exit code of 0.

Something like this could also be extended to the new HTTP SD, and both could also be made available as a standalone command for those developing SD mechanisms.

I would not go HTTP calls during check config, however checking the format of file_sd seems a good idea.
cc @dgl

I can work on this.

@roidelapluie @LeviHarrison can I help with the PR in any way ?

If you could test it out that'd be great! It's always good to get feedback.


Change or read options after object creation

Sorry, this is really a support question but I don't know where else to post.
Given the following code:
ppsMonitor = settingsPane.addMonitor(laserSimulator, 'ppsActual', {view: 'graph', min: 0, 100, interval: 1000});

Is there anywhere information how I can change the "max" property in the above example?
This is such a great looking UI. It makes web programming fun. Documentation and examples are a bit light.
I just have no idea how to navigate these UI objects in code. I tried digging through
ppsMonitor.controller_
ppsMonitor.emitter_

The only thing that makes sense is ppsMonitor.label :-)
https://user-images.githubusercontent.com/1192916/146966898-ef1783bc-ec07-4a9b-b74d-5898ffb6acab.mp4

Added changes in #404, and you'll be able to change min/max value in the next version like this:
const pane = new Tweakpane.Pane();
const m = pane.addMonitor(params, 'foo', {
  view: 'graph',
  min: 0,
  max: 100,
});

m.controller_.valueController.props_.set('minValue', 40);
m.controller_.valueController.props_.set('maxValue', 60);

These are still private fields and may change in future updates. Please be careful.

for anyone who is trying to automatically set the range of their graph, as of 3.1.6 you can access the array of values in the graph via
m.controller_.valueController.value_.value_

and, for the FPS graph specifically, you can set minValue and maxValue via
m.controller_.valueController.graphC_.props_.set


Create HelloWorld3

@miki164 Give me a request to review this.

How can I do that? Sorry, but I'm very new in contributing to someone else's repositories.

In the top right, u will see this 👇

Click on settings and add me to you review your code

I tried that, but I don't have settings(geer wheel)


Replace bignum.h

Currently, we handle bignum by using a wrapper around OpenSSL implementation.
I saw several drawbacks to that:

we have to maintain the wrapper
no support for 256-bit integer (it is commented out), and this is the default integer type in the blockchain.
we have to handle different version of OpenSSL (see https://github.com/vrancurel/quadiron/pull/4)
seems to rely a lot on memory allocation (not a big deal for now)

An alternative would be  cpp_int from the "multiprecision" package of Boost.
Advantage:

almost no maintenance on our side
native support for 128 and 256-bit integer
support for 160-bit integer (address in the blockchain) is as easy as a typedef
efficient code (it's Boost + it seems like default types are implemented with no new/malloc)

Replaced by an homemade implementation (see #46)


Install core components with resource.limits

It would be nice to add ressource.limits to the K8s core components which got installed by KubeOne
components would be:

canal
coredns
kube-proxy
any other k8s core component

identified manifests that we manage that need limits:

daemonset/canal
deployment/digitalocean-cloud-controller-manager
deployment/hcloud-cloud-controller-manager
deployment/machine-controller
deployment/machine-controller webhook
deployment/metrics-server

system manifests:

deployment/coredns
daemonset/kube-proxy

/assign @kron4eg
/lifecycle active

This is out of the scope for KubeOne. Beside that, it's hard to determine limits for some components.
/close


Update SSH2 module to newest version

@butenkor @PhilippSpo can you review? Thanks.

Coverage remained the same at 24.053% when pulling 733c3b935252f10ced36e8efa7885c4a571fc780 on update-ssh2-module into b42da4f451bce3ab5345d1d5ed2d01305f7dad61 on master.


Get started guideline for contributors

This issue just note down the getting start guidelines for contributors, will close after we move these to documentations.
for contributors to start nodejs environment

prepare nodejs >= 16
prepare pnpm npm install -g pnpm
install without scripts (to ignore preinstall binary build) pnpm install --ignore-scripts
prepare github cli https://cli.github.com/
download binary artifacts pnpm run artifacts

for native addon development, install

rust
cmake
clang/gcc

https://llama-node.vercel.app/contribution


[MRG] Update documentation with more cross references

This adds a few additional cross-references, mentions Python 2.7 support and removes a level from the TOC for How To guides inn the index.

Rendered docs: https://278-90722372-gh.circle-artifacts.com/0/html/index.html

nice!


API doc mentions cleanSourcesFilter, did you mean cleanSource or cleanSourceWith?

I was reading through https://github.com/ipetkov/crane/blob/master/docs/API.md#libcleancargosource trying to figure out how to include *.lalrpop files into my build, and I noticed the doc mentions cleanSourcesFilter but this does not exist in Nixpkgs nor in Crane code.
In Crane code, the only similar Nix code used is:

cleanSourceWith
cleanSource (which is just a special case of cleanSourceWith)

In Nixpkgs, there exists a function cleanSourceFilter (notice the singular Source), but it does not appear to be directly used by Crane code.

Ah, thanks for the report @sir4ur0n! Yes this is a typo, cleanCargoSource will internally use cleanSource so I'll update the docs to correct the name!


Decker (C) crashes on undoing pasting a contraption after undoing moving it

Ran into this while playing with some of the contraptions in the contraption bazaar.
On Windows and Decker 1.45, create a new deck, then paste the widgets from this contraption: https://itch.io/post/9887819.
Contraption code for ease of reproduction

%%WGT0{"w":[{"name":"dialogizer styler1","type":"contraption","size":[289,187],"pos":[49,48],"show":"transparent","def":"dialogizer styler","widgets":{"fontlist":{},"fct":{},"bct":{},"fclr":{"value":"47"},"bclr":{"value":"32"},"tft":{},"tfnt":{},"patpal":{},"traplist":{"row":18},"tbr":{},"brdr":{},"button1":{},"idi":{"value":"p"},"result":{},"sti":{}}}],"d":{"dialogizer styler":{"name":"dialogizer styler","size":[289,187],"margin":[0,0,0,0],"description":"generate style code","script":"on view do\nfontlist.value:select name:key from deck.fonts\nfprv.font:first fontlist.rowvalue\nfprv.text:first fontlist.rowvalue\ntfnt.font:first fontlist.rowvalue\ntfnt.text:first fontlist.rowvalue\ntraplist.value:select name:key from deck.contraptions\nbrdr.text:first traplist.rowvalue\nend","image":"%%IMG2ASEAuwD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wAUIQEA/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wD/AP8A/wDZ","widgets":{"fontlist":{"type":"grid","size":[97,176],"pos":[96,11],"locked":1,"script":"on click row do\n view[]\nend","headers":0,"value":{"name":["body","menu","mono"]},"row":0},"fct":{"type":"field","size":[48,16],"pos":[0,11],"locked":1,"font":"mono","border":0,"align":"right","value":"fcolor:"},"bct":{"type":"field","size":[48,16],"pos":[0,25],"locked":1,"font":"mono","border":0,"align":"right","value":"bcolor:"},"fclr":{"type":"field","size":[49,16],"pos":[48,11],"font":"mono","border":1,"value":"32"},"bclr":{"type":"field","size":[49,16],"pos":[48,26],"font":"mono","border":1,"value":"47"},"tft":{"type":"field","size":[48,16],"pos":[0,40],"locked":1,"font":"mono","border":0,"align":"right","value":"tfont:"},"tfnt":{"type":"field","size":[49,16],"pos":[48,41],"border":1,"value":"body"},"patpal":{"type":"canvas","size":[288,10],"pos":[0,1],"locked":1,"border":0,"image":"%%IMG2ASAACiASARMhESISIxIkEiUSJhInEigSKRIqEisSLBItEi4SIBIBEyERIhIjEiQSJRImEicSKBIpEioSKxIsEi0SLhIgBAEEIAIBBCAEAQQgAQEFIAUBBCEDAQQhAgEEIQQiBAEEIgIBASIHIwQBBCMDAQMjBCQEAQQkAgEBJAclBAEEJQIBBSUDJgQBBCYDAQImBScEAQQnAgEEJwQoBAEBKAUBBCgEKQQBASkFAQMpBSoEAQEqBQEEKgQrBAEBKwUBBCsELAQBASwFAQEsBy0EAQEtBgEDLQQuBAEBLgUBAS4HIAcBASAFAQEgBAEEIAEBASABAQcgAQEEIQYBASEFAQEhBCIHAQEiAgEBIgEBASIFIwcBASMDAQEjBiQHAQEkAgEBJAclBwEBJQYBASUDJgcBASYDAQImBScHAQEnAgEBJwIBAScEKAQBASgBAQEoAwEBKAIBASgEKQQBASkBAQEpBQEBKQUqBAEBKgEBASoGAQEqBCsEAQErAQEBKwYBASsELAQBASwBAQEsAwEBLAEBASwFLQQBAS0BAQEtBAEBLQYuBAEBLgEBAS4DAQEuByAHAQEgBQEBIAQBBCABAQEgAQEHIAEBBCEGAQEhBQEBIQQiBwEBIgIBASIBAQEiBSMHAQEjAwEBIwYkBwEBJAIBASQHJQcBASUGAQElAyYHAQEmAwECJgUnBwEBJwIBBCcEKAQBASgBAQEoAwEBKAIBASgEKQQBASkBAQEpBQEBKQUqBAEBKgEBASoGAQEqBCsEAQErAQEBKwYBASsELAQBASwBAQEsAwEBLAEBASwFLQQBAS0BAQEtBAEBLQYuBAEBLgEBAS4DAQEuByAFAQIgAwEEIAQBBCADAQUgAgEFIQQBAiEEAQIhBSIFAQIiAwEDIgUjBQECIwQBAyMEJAUBAiQDAQQkBCUFAQIlBQECJQQmBQECJgMBBCYEJwUBAicGAQEnBCgEAQMoAwEBKAIBASgEKQQBAykFAQEpAQEBKQMqBAEDKgMBBCoEKwQBAysEAQIrBSwEAQMsAwEDLAUtBAEDLQQBAy0ELgQBAy4DAQQuBCAHAQEgAgEBIAcBBiABAQUgAQEGIQYBASEFAQEhBCIHAQEiBAEBIgUjBwEBIwUBASMEJAcBASQCAQEkAgEBJAQlBwEBJQQBASUFJgcBASYCAQEmAgEBJgQnBwEBJwUBAScEKAYBASgDAQEoAgEBKAQpBgEBKQUBASkBAQEpAyoGAQEqAwEBKgcrBgEBKwYBASsELAYBASwFAQEsBS0GAQEtBgEBLQQuBgEBLgMBAS4CAQEuBCAEAQQgAgEEIAQBBiABAQUgAQEGIQMBBCECAQQhBCIEAQQiBAEBIgUjBAEEIwIBBCMEJAQBBCQCAQQkBCUEAQQlBAEBJQUmBAEEJgIBBCYEJwQBBCcFAQEnBCgGAQEoAwEEKAQpBgEBKQMBBSkDKgYBASoDAQQqBCsGAQErAwEEKwQsBgEBLAUBASwFLQYBAS0DAQQtBC4GAQEuAwEELgQgEgETIREiEiMSJBIlEiYSJxIoEikSKhIrEiwSLRIuEiASARMhESISIxIkEiUSJhInEigSKRIqEisSLBItEi4S","scale":1},"traplist":{"type":"grid","size":[96,176],"pos":[192,11],"locked":1,"script":"on click row do\n view[]\nend","headers":0,"value":{"name":["dialogizer styler"]},"row":2},"tbr":{"type":"field","size":[48,16],"pos":[0,55],"locked":1,"font":"mono","border":0,"align":"right","value":"border:"},"brdr":{"type":"field","size":[49,16],"pos":[48,56],"border":1,"value":"dialogizer styler"},"button1":{"type":"button","size":[96,16],"pos":[0,89],"script":"on click do\nresult.text:\"%s:()\n%s.fcolor:%s\n%s.bcolor:%s\n%s.tfont:%s\n%s.border:deck.contraptions.%s\ndd.style[%s]\" format idi.text,idi.text,fclr.text,idi.text,bclr.text,idi.text,tfnt.text,idi.text,brdr.text,idi.text\nend","text":"generate","style":"rect"},"idi":{"type":"field","size":[49,16],"pos":[48,71],"font":"mono","border":1,"value":"o"},"result":{"type":"field","size":[97,80],"pos":[0,107],"script":"on change val do\n \nend"},"sti":{"type":"field","size":[48,16],"pos":[0,70],"locked":1,"font":"mono","border":0,"align":"right","value":"ID:"}}}}}

After pasting the widgets, move the selection around on the page, then undo to move them back to the original position when the widgets were pasted. Then undo again, which should undo pasting the widgets. On my system, Decker crashes at this point reproducibly. Looking at the crash in a debugger, I can see the last instruction before the crash seems to be at decker + 501CAh which is call qword ptr [rbx+50h]. rbx is set to decker + 1679B0h so it works out to call qword ptr [decker + 167A00h] but that entire block of memory is almost entirely zeroes, causing it to crash with an access violation because it then tries to execute code at 0x0. I have a few windows minidump files (a meg or two in size) and a WinDbg "time travel" recording (4gb in size). I'll update this issue if I can reproduce it on other platforms.

Actually it looks like this happens with any contraption pasted into Decker and moved on screen. Here's a minimal example that I can reliably reproduce the crash with:
%%WGT0{"w":[{"name":"crasher1","type":"contraption","size":[100,100],"pos":[206,121],"def":"crasher","widgets":{"field1":{}}}],"d":{"crasher":{"name":"crasher","size":[100,100],"margin":[0,0,0,0],"widgets":{"field1":{"type":"field","size":[83,84],"pos":[8,8],"value":"once you paste me, move me once, then undo to put me back, then undo again to crash decker"}}}}}

And I can reproduce it on Mac OS as well - looks like the stack trace from the crash report on Mac is more useful than what Windows'll provide - crash happens inside is_resizable.
Relevant crash information from Problem Report

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   ???                           	               0x0 ???
1   app                           	       0x106a9025a is_resizable + 181
2   app                           	       0x106a906ec object_editor + 1144
3   app                           	       0x106a99122 main_view + 3171
4   app                           	       0x106a697f0 tick + 405
5   app                           	       0x106a69646 io_run + 26
6   app                           	       0x106a9aa84 main + 3570
7   app                           	       0x106a30e70 start + 52
Thread 0 crashed with X86 Thread State (64-bit):
rax: 0x0000000106b384b8  rbx: 0x0000000106b30060  rcx: 0x0000000000000201  rdx: 0x0000000000000000
rdi: 0x0000000106b30060  rsi: 0x0000000106b384b8  rbp: 0x00007ff7b94cca00  rsp: 0x00007ff7b94cc9d8
r8: 0x0000000000000064   r9: 0x0000000000000340  r10: 0x0000000000004000  r11: 0xffff80094d677a18
r12: 0x0000000106a59564  r13: 0x000000130000005e  r14: 0x0000000000000000  r15: 0x0000000106a9c377
rip: 0x0000000000000000  rfl: 0x0000000000210246  cr2: 0x0000000000000000
Logical CPU:     2
Error Code:      0x00000014 (no mapping for user instruction read)
Trap Number:     14
Binary Images:
0x106a2f000 -        0x106aabfff com.beyondloom.decker (1.45) <058b3fe6-297d-37d2-ae94-b35ea075a4d4> /Users/USER/*/Decker.app/Contents/MacOS/app
{
"captureTime" : "2024-07-01 13:59:53.4130 +0100",
"pid" : 8589,
"cpuType" : "X86-64",
"roots_installed" : 0,
"bug_type" : "309",
"procLaunch" : "2024-07-01 13:59:37.9350 +0100",
"procStartAbsTime" :<PHONE_NUMBER>5559,
"procExitAbsTime" :<PHONE_NUMBER>0728,
"procName" : "app",
"procPath" : "/Users/USER/*/Decker.app/Contents/MacOS/app",
"bundleInfo" : {"CFBundleShortVersionString":"1.45","CFBundleIdentifier":"com.beyondloom.decker"},
"vmRegionInfo" : "0 is not in any region.  Bytes before following region:<PHONE_NUMBER>\n      REGION TYPE                    START - END         [ VSIZE] PRT/MAX SHRMOD  REGION DETAIL\n      UNUSED SPACE AT START\n--->  \n      __TEXT                      106a2f000-106aac000    [  500K] r-x/rwx SM=COW  ...nts/MacOS/app",
"exception" : {"codes":"0x0000000000000001, 0x0000000000000000","rawCodes":[1,0],"type":"EXC_BAD_ACCESS","signal":"SIGSEGV","subtype":"KERN_INVALID_ADDRESS at 0x0000000000000000"},
"termination" : {"flags":0,"code":11,"namespace":"SIGNAL","indicator":"Segmentation fault: 11","byProc":"exc handler","byPid":8589},
"vmregioninfo" : "0 is not in any region.  Bytes before following region:<PHONE_NUMBER>\n      REGION TYPE                    START - END         [ VSIZE] PRT/MAX SHRMOD  REGION DETAIL\n      UNUSED SPACE AT START\n--->  \n      __TEXT                      106a2f000-106aac000    [  500K] r-x/rwx SM=COW  ...nts/MacOS/app",

Thanks for the report.
I believe I've resolved the problem: destroying objects did not necessarily de-select them, which in certain circumstances could result in an attempt to access the fields of an object which was already garbage-collected.

This fix has been incorporated into release v1.46.