App crash when plugging in Yubikey during smart card extension configuration

When trying to use the smart card extension configuration function in Yubico Authenticator (three-dot menu at upper right > Configuration > Smart card extension), the app immediately crashes when I plug my Yubico 5Ci into my device. (Of note, I am using a FIPS device.)
I've attached the crash log from iOS here.
Device info: iPhone 13 Pro, iOS 16.3.1.
Yubico Authenticator: v1.7.1 (build 96)
YubiKey: YubiKey 5Ci FIPS, firmware 5.4.2
Authenticator-2023-03-20-133002.ips.zip

I was asked to provide more info about our Yubikey keys, so here that info is.
First, we have no issues using the derived PIV-D certs on the keys for system login, either on Mac or Windows; they work perfectly. The Yubikey Manager app (on macOS) can't read the PIV certs, though — when going to Applications > PIV, it reports "Failed connecting to the YubiKey.Make sure the application has the required permissions.". We've worked through all the advice on this page, but it doesn't look like it's a permissions issue per se — all the necessary permissions are granted to the app, so that must be a more generic error message that doesn't reflect the actual issue the app has in reading the certs/keys.
Second, the certs are written to the keys using a Deloitte-written application that uses Entrust's PIV-D system as its underpinnings.
Finally, I was asked about the key size of the keys; according to the macOS security/smartcard support infra (at the command line, security export-smartcard), this is the info about the first private key on the card (there are two, and they're similar):
==== private key #1
                crtr : 0
                esiz : 0
                decr : 0
                atag : ""
                kcls : 1
                agrp : "com.apple.token"
                pdmn : "dk"
                bsiz : 2,048
                type : 42
                klbl : <ce 20 3e 93 24 c5 bb e1 1b ae d7 66 28 0d 66 5e 25 bc 8d a5>
                edat : 2001-01-01 00:00:00 +0000
                sign : 1
                mdat : 2023-04-13 12:45:43 +0000
                drve : 0
                labl : "Key For PIV Authentication (Jason E. Levine -A11)"
                sync : 0
                musr : <>
                sha1 : <ff c7 8b b1 e3 77 c0 5b d8 7d 5b 01 cf 0a 29 70 48 db 54 86>
                cdat : 2023-04-13 12:45:43 +0000
                tkid : "com.apple.pivtoken:00000000000000000000000000000000"
                sdat : 2001-01-01 00:00:00 +0000
                tomb : 0
                priv : 1
                accc : constraints: {
                                                osgn : "PIN"
                                }
                                protection: {
                                                tkid : "com.apple.pivtoken:00000000000000000000000000000000"
                                }
                unwp : 0
====

@jensutbult Will there be a new version of Yubico Authenticator released to the iOS App Store so that we can test whether this resolves the issue on our end?

@delfuego there's a new release coming out next week that includes this fix.

@jensutbult Excellent! If you do any app testing with TestFlight, we'd be happy to enroll and test out the new build before you release it to confirm that it resolves our issue.


[question] cannot invoke 'decode' with an argument list of type '((AnyObject))'

Hi, I'm trying to get the decode function to work. I have a decode function that I think conforms to Decodable:
static func decode(j: JSON) -> Decoded<User> {
    return User.create
        <^> j <| "id"
        <*> j <| "username"
        <*> j <| "access_token"
        <*> j <| "refresh_token"
        <*> j <| "token_type"
}

Now I'm trying to call User.decode like this:
let user:User? = User.decode(json)

The json object is an AnyObject from the Alamofire's responseJson callback. As far as I can tell my code matches all the Argo examples I've found, but I'm getting this error when I build:
cannot invoke 'decode' with an argument list of type '((AnyObject))'

I'm using Swift 1.2 and Argo 1.0.3

The decode function takes

The decode function takes a JSON type and since your json variable is an AnyObject, this won't work. You can use JSON.parse to transform the AnyObject into a JSON type.

What I think you want is the global decode function. Use it like so: let user: User? = decode(json)

Ah, thank you! That worked, I didn't catch that there was a global function.

No problem, glad that's all it was!


BUG: Missing VTS Policy Manager unit tests and mocks

Present services build does not unit test vts/policymanager tests and also generated mocks needs to be submitted to the services repository.
Issue linked to #24

I think this should be looked at in the context of #16

I think, this code establishes the policy management framework, so we can include the same. Issue #16 add meat to the framework? We just need enough unit tests to cover the framework?


Snapshotting with invert colors doesn't respect snapshot identifier

Specifically the Swift version of the snapshot method in the iOSSnapshotTestCase subspec. The Objective-C version of this behaves correctly.
To reproduce:
func testView() {
    let view = UIView(frame: .init(x: 0, y: 0, width: 100, height: 100))
    SnapshotVerifyWithInvertedColors(view, identifier: "someIdentifier")
}

When run in record mode, this should produce a reference image named testView_someIdentifier_ followed by the device properties. Instead it produces a reference image named testView_ (omitting the identifier).

This should be fairly straightforward to address (the actual fix is a one-liner).
I think it's worth updating the snapshot test a bit at the same time. Specifically, we should add an identifier to the test to verify this fix, and it's also probably worth adding some verification around the state restoration, e.g.:
    func testInvertColors() {
        let viewController = InvertColorsViewController()
        viewController.view.frame = UIScreen.main.bounds

        FBSnapshotVerifyView(viewController.view, identifier: "disabled")

        SnapshotVerifyWithInvertedColors(viewController.view)

        // Run the plain snapshot a second time to ensure that the view was restored to its original state.
        FBSnapshotVerifyView(viewController.view, identifier: "disabled")
    }

Due to a view life cycle issue with the snapshot test framework, you'll need to modify where the status get checked in the sample view controller in order to get the initial check to pass. The easiest way to accomplish this is to move the code from viewWillAppear(_:) to the end of viewDidLoad().


Cosmos DB metrics page has been updated

The metrics blade of Cosmos DB looks different from what is presented on this doc page. The tabs shown in the first image (Overview, Throughput, etc) has been moved to the Insights blade.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 4535fb42-55f9-4d8c-52d2-30b5eb0ef726
Version Independent ID: f5cccae0-a752-4065-605a-4179016a5a42
Content: Monitor and debug with metrics in Azure Cosmos DB
Content Source: articles/cosmos-db/use-metrics.md
Service: cosmos-db
Sub-service: cosmosdb-sql
GitHub Login: @kanshiG
Microsoft Alias: govindk

@felixtsem5 We are looking into this and We will get back to you.

@felixtsem5  I was able to see the tabs you mentioned above (Overview, Throughput, ect) on the Metrics(classic) blade in the Azure portal. Please let me know if that is not the case from your end. Please see the screen below

@felixtsem5 I have assigned the issue to the content author for further review and update the doc as appropriate.
cc @kanshiG Could you please look into this issue.

This may be affected some other pages as well. I'm flipping through the Cosmos DB docs and I started seeing the same item. Here's another example. If you need me to open another issue, please let me know.
https://docs.microsoft.com/en-us/azure/cosmos-db/create-mongodb-java

Please assign to @StefArroyo

@felixtsem5 thanks for your feedback, I submitted a PR above to update the doc. It will be reviewed and published by end of this week. We will now close this issue.
#please-close


Catkin fix

Add support to build catkin packages on the target platform.
The main issue I identified with cross compiled catkin packages is that various config files are generated containing the absolute paths of the host/build system. I added a pre-process step before the packaging to remove these path components. However, this will only work as long as only the standard catkin files are generated (other generated configuration files might be missed).
Additionally, I added a patch for the cmake module FindBoost which is the diff to Ubuntu's FindBoost version and I added some missing dependencies for catkin-dev.

See my inline comments.

I understand the issue now, and I will run some bitbake core-image-ros-world on the different architectures to make everything still works. You can consider providing these patches also on the openembedded-core mailing list, so that the issue is solved for a larger user group.

Could this be extended to enable cross compiling from an ADT? I tried it, but lots of stuff like all the Python libraries are missing in the x86 sysroot, and I'm also having trouble telling catkin where to find non-ROS libraries like Boost. I'd be glad for any hints I you have no time to work on this.

@bulwahn The patch for the Python dependency has been added here.
The FindBoost.cmake patch is obsolete with the HEAD of openembedded-core, because they are no longer using version 2.8.12 and in newer cmake versions this has been fixed.
@sephalon I'm not clear on what you want to do and therefore what the problem is.

@dominiquehunziker Sorry, I'll elaborate this a little bit more: Using the populate_sdk task, one can build an Application Development Toolkit (ADT) for an image. You can install this on your development machine to cross-compile stuff for the target machine. Basically an ADT contains two sysroots, one matching the root filesystem of your target (including all libraries and headers) and another one with the cross toolchain (compiler, linker, debugger and so on) compiled for the (most likely x86) development machine architecture. This way you could build catkin packages on-the-fly and even skip deployment by just mounting the build folder from the target using SSHFS or so.
So far I wrote a toolchain.cmake file pointing to the cross compiler. Most importantly this has a CMAKE_SYSROOT variable set telling CMake to only look for stuff in the x86/development machine sysroot. When I run catkin (locally installed on my machine, but with -DCMAKE_TOOLCHAIN_PATH set) the compiler is detected nicely, but lots of Python dependencies like EmPy and the nosetests binary are missing in the x86 sysroot. After I just copied those from the arm into the x86 sysroot, I got a little bit further until getting errors for missing Boost libraries; and that's when I stopped, not sure what other problems are to come. Well, this post got a bit long; basically I was just hoping that you'd have some idea what has to be changed in order for Yocto to add all missing requirements to the development machine sysroot.

@sephalon Ok, now I get it. Didn't realize that this even existed. We do basically the same thing, but use a standalone shell script for fast cross-compiling which piggybacks on the two sysroots of bitbake. Let me check whether I can build an ADT from this.

I tested this pull request with the current openembedded-core repository version, and the cmake bbappend additions lead to the following failures, as cmake is updated to 3.4.2 by now:
ERROR: No recipes available for:
  /home/lukas/dev/github.com/bmwcarit/meta-ros/recipes-devtools/cmake/cmake-native_2.8.12.%.bbappend
  /home/lukas/dev/github.com/bmwcarit/meta-ros/recipes-devtools/cmake/cmake_2.8.12.%.bbappend

I would suggest to move the needed changes for cmake 2.8.12 into a directory backports or backport-patches, and then add a section in the README.md to explain what changes must be done to apply the changes on the older versions of poky. We can certainly integrate the first patch and the other parts of the second patch (besides the cmake bbappend and related changes).

@sephalon & @dominiquehunziker Are there any progress on the populate_sdk stuff? We are thinking about a ROS instance running on a embedded target and want to provide the customers a cross-toolchain which allows the customer to build his own ros packages.

I have something working. First, however, I have clean it up a bit and test it more thoroughly, but I don't have the time to do it just now.

I'll start a new issue so we do not need to hijack this PR

This PR has been superseded by #386


Support issuance and verification for SD-JWT

Reference links

RFC-001 - Issue Verifiable Credentials Workflow
RFC-002 - Present Verifiable Credentials Workflow
RFC-100 - EWC Interoperability Profile Towards ITB - v1.0

Fixed in https://github.com/EWC-consortium/eudi-wallet-oidc-android/pull/3


Enable CI

Using travis ci GSCAN is automatically run
Also, GSCAN is added to lock git hooks for commit and push via the husky lib.

This was a really nice idea thanks @mrosenquist! It wouldn't have worked with the old version of GScan because there were no exit codes. I've fixed that, tested it thoroughly and got an updated (slightly simpler) version of this here #470


podman running kata containers is not cleanup up correctly.

# podman ps
# podman run -ti --runtime=/usr/bin/kata-runtime --rm  fedora sh
# ^d
#
# podman ps
ERRO[0000] OCI Runtime kata is in use by a container, but is not available (not in configuration file or not installed)
CONTAINER ID  IMAGE  COMMAND  CREATED  STATUS  PORTS  NAMES

just curious, does buildah containers show anything or podman ps -a?

Why are you specifying full path to --runtime? Why not --runtime=kata? I think our default configuration file includes it now...
IMO, NOTABUG, --runtime=FULL-PATH is reserved for CI only.

podman run -ti --runtime=kata --rm  fedora echo hello
Error: error creating libpod runtime: default OCI runtime "kata" not found: invalid argument

Anyways this error went away, since I think something was screwed up.

The root libpod.conf we ship by default includes the runtimes:
# Kata Containers with the default configured VMM
kata-runtime = [
    "/usr/bin/kata-runtime",
]

# Kata Containers with the QEMU VMM
kata-qemu = [
    "/usr/bin/kata-qemu",
]

# Kata Containers with the Firecracker VMM
kata-fc = [
    "/usr/bin/kata-fc",
]

Containers.conf includes only one of these (kata) - https://github.com/containers/common/blob/master/pkg/config/containers.conf#L372-L379


DocsArchive-<IP_ADDRESS>Auto archive content

Please don't merge this PR until target archive repo PR https://github.com/OPS-E2E-PPE/docs-archive-test-target/pull/796 is merged into live branch.
Auto archive content to https://github.com/opstest2/docs-archive-test-target.git

Docs Build status updates of commit 172c520:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error-RuningBuildFailed] Some unexpected errors happened when running build, please open a ticket in https://SiteHelp and include the error report for our team to troubleshoot

For more details, please refer to the build report.
If you see build warnings/errors with permission issues, it might be due to single sign-on (SSO) enabled on Microsoft's GitHub organizations. Please follow instructions here to re-authorize your GitHub account to Docs Build.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
Note: Your PR may contain errors or warnings unrelated to the files you changed. This happens when external dependencies like GitHub alias, Microsoft alias, cross repo links are updated. Please use these instructions to resolve them.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


Not possible to deploy latest edge hub schema using az cli or portal

When deploying via the portal, it is not possible to define TTL or priority per route, it's just a single line for route:

When deploying a template using az iot edge deployment create, it also fails with an error that seems consistent with what the portal shows:
az iot edge deployment create --hub-name cnc-dev --deployment-id locker-edge-deployment-$BITBUCKET_BUILD_NUMBER --content locker-deployment-manifest.json --priority 0 --target-condition "$TARGET_CONDITION" --subscription $CNC_AZURE_SUBSCRIPTION_ID
{
  "validationErrors":[
    {
      "description":"{'route': 'FROM /messages/* INTO $upstream', 'priority': 0, 'timeToLiveSecs': 1200} is not of type 'string'",
      "contentPath":"modulesContent.$edgeHub.properties.desired.routes.upstream",
      "schemaPath":"properties.modulesContent.properties.$edgeHub.properties.properties.desired.properties.routes.patternProperties.^[^\\.\\$# ]+$.type"
    }
  ]
}

The above error is generated from a build pipeline that runs within container mcr.microsoft.com/azure-cli and az cli version within that is 2.1.0 with azure iot extension 0.10.3
The part within our manifest that fails looks like this:
"$edgeHub": {
  "properties.desired": {
    "schemaVersion": "1.1",
    "routes": {
      "upstream": {
        "route": "FROM /messages/* INTO $upstream",
        "priority": 0,
        "timeToLiveSecs": 1200
      },
      ....

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: ad3731a8-f0df-f1a1-d1a1-e89a769a0908
Version Independent ID: 50a5190e-eafa-7dde-c18f-be9505b00a0a
Content: Deploy module & routes with deployment manifests - Azure IoT Edge
Content Source: articles/iot-edge/module-composition.md
Service: iot-edge
GitHub Login: @kgremban
Microsoft Alias: kgremban

@jokarl
Thanks for the feedback! We are currently investigating and will update you shortly.

@jokarl - thank you for raising the issue.  We are investigating the CLI issue you raised.  In the meantime, the portal does allow the Edge Hub 1.1 settings to be applied to both devices and deployments.  These fields are available by default on new deployments and devices.  If you'd like to set a device or cloned deployment to use the new schema:
1.) from the Create Deployment or Set Modules view, click Modules
2.) click runtime settings
3.) select the target schema version from the Schema Version under Edge Hub
4.) click save.
5.) the expanded options should now be available on the Routes tab for each route.

@jokarl Until there is a fix for the CLI, as a short-term workaround you can try using the --layered switch for the az iot edge deployment create command.

@jokarl - thank you for raising the issue.  We are investigating the CLI issue you raised.  In the meantime, the portal does allow the Edge Hub 1.1 settings to be applied to both devices and deployments.  These fields are available by default on new deployments and devices.  If you'd like to set a device or cloned deployment to use the new schema:
1.) from the Create Deployment or Set Modules view, click Modules
2.) click runtime settings
3.) select the target schema version from the Schema Version under Edge Hub
4.) click save.
5.) the expanded options should now be available on the Routes tab for each route.

Completely missed this, we'll use this until our pipeline works with the new schema. Will also give the - -layered switch a try.
Thanks for the quick responses!

Small follow up, we released v0.10.4 of the IoT extension today which among other things should resolve this issue you are running into.

Small follow up, we released v0.10.4 of the IoT extension today which among other things should resolve this issue you are running into.

Confirmed that this is working in our pipeline. Thanks for the quick help

Totally off topic, but this feature is great and it solved a lot of issues for us. We can now throw away events that we don't want ghosting around in our device if we fail to ack them on the edgeHub. It also allowed us to to remain offline for long periods of time and still recover from it and having all telemetry arrive at our endpoints.

Hello @jokarl , Thanks for sharing your experience on this product document.
Please let us know if we are good at this point and may be able to close this issue?


Rechunk method for uncompressed arrays

@rsignell this is inspired by your blog post (still a WIP for now)
The idea is that you can simply do
vds = open_virtual_dataset('uncompressed_netcdf3.nc')
subchunked_vds = vds.chunk(time=1)

[x] Closes #86
[x] Tests added
[ ] Tests passing
[x] Full type hint coverage
[ ] Changes are documented in docs/releases.rst
[ ] New functions/methods are listed in api.rst
[ ] New functionality has documentation

Note to self: we should add more validation to the ZArray class to check that the chunks attribute is a tuple of positive integers, and move the zarray.replace call to the start of the method to catch invalid input early.

we can rechunk in however many dimensions we want!

I love that aspect!

but the new integration test fails because Dataset.chunk is dispatching to dask's version of .rechunk somewhere. This may require a change to xarray's ChunkManagerEntrypoint upstream to fix.

https://github.com/pydata/xarray/pull/9286 has now progressed far enough that this PR works for me at least! Passing all tests locally 🟢


Add /calculate_pp and improve performance usecase

Edit: Accidentally put the GDPR stuff in here so I guess merge #341 before this
Added the /calculate_pp endpoint. It takes the api key in the Authorization header.
In the query parameters, beatmap_id is the only required parameter. mods, mode, misses etc. all have a default value of 0, combo has the maximum combo of the beatmap as the default.
The return type of the endpoint is a single JSON object if the n100 and n50 parameters were used. If the acc parameter is used instead, the user can specify multiple accuracies in the format "99.1,98.38,88.23". In that case, a list of JSON objects is returned.
While changing return types on API endpoints are usually bad, we can do it here because the developer explicitly either requests a singlg object via n100 and n50 or multiple via the acc parameter. Therefore the developer can expect what the return type is.
The extra n100 and n50 is necessary as the pp system has a 50 hit penalty. If you specify accuracies, rosu-pp automatically assumes the best-case-scenario for the 50 count. This is usually desired when requesting a list of accuracies, e.g. for visualizing the pp values for common accuracies (100, 99, 98, 95, ...)
I think adding the possibility of specfying multiple n100 and n50 pairs would bloat the endpoint with parsing and does not really have much real world applications. And since the performance usecase does not optimize for calculating multiple scores anyways (e.g. by caching star ratings) the user could just send multiple requests to the endpoint if more than one n100/n50 pair is necessary.
I also reworked the return of the performance usecase and adapted the code that uses the calculate_performances() method to it. It now returns all performance and difficulty-related values instead of just the pp and star rating.

This PR should be rebased to master and force pushed to remove data dumping functionality. I myself do not approve of this PR otherwise as it would depend on a completely unrelated PR which is not even guaranteed for merging. @cmyui would like your thoughts on this as well.


Lock co-body version for koa1

Lock co-body to 5.x
Specify dev dependency versions
Update failing tests

Ref https://github.com/dlau/koa-body/issues/85


Cardigan: Images detecting a slight difference

What
We've had this issue more often in the past; we added diffThreshold: 0.2 (more info) to the troublemakers, it fixed it pretty well.
Now ImageGallery: Frames is asking for Approval on almost every commit.
I've tried diffThreshold: 0.3 and diffThreshold: 0.2, I currently have delay: 1000 to try and see if it helped but no, could've tried a longer delay though, I just don't think that's the issue.
I talked to Chromatic, here's the full message:

You are using loading the font Inter. Are you preloading your fonts or using a loader, as we suggest? If you do, it will reduce inconsistencies related to fonts.
 You fetch and  with several versions (srcset) that take too long to load and cause inconsistencies. Please, follow the instructions in this sample project to improve image inconsistencies:
https://github.com/yannbf/storybook-image-loading-best-practices

Should we just use a placeholder image? I feel like we talked about this, the image not being cached, but I can't fully remember the conversation. @gestchild @davidpmccormick I could use your opinion!
Example of a build saying there's a difference

I'm going to close this, we have some ways of fixing, and we're looking towards a new instance of Storybook 7 too. Can always reopen if it comes back.


Main

Thanks but I do not accept new handlers like this which are not standalone and generic. Please submit it as a third party package and list it here if you like: https://github.com/Seldaek/monolog/wiki/Third-Party-Packages


Docker env for Arm® Ethos™-U55 Port

This is the first PR for the Arm Ethos-U Integration RFC (https://github.com/apache/tvm-rfcs/pull/11)

Added Arm® Corstone™-300 Reference System for testing
Added Arm® Ethos™-U driver stack
Added installation of Arm® Vela.

Co-authored-by: Manupa Karunaratne<EMAIL_ADDRESS>@manupa-arm @areusch @Mousius
Thanks for contributing to TVM!   Please refer to guideline https://tvm.apache.org/docs/contribute/ for useful information and tips. After the pull request is submitted, please request code reviews from Reviewers by @ them in the pull request thread.

@grant-arm i believe we have resolved the CI issue that broke this

Hi @areusch, could you please take a look at this again. I've addressed your latest comment now and the CI tests have all passed.

thanks @grant-arm, the PR is now merged!

We need a docker file update for this .


Found a new endpoint, but it returns a binary file

https://teg/api/devices/vitals
Good old strings returns some interesting tidbits:
THC_STATE_AUTONOMOUSCONTROL
THC_AmbientTemp!

I'd love to get temperature info from my gateway/inverter/battery since I was having problems with my brand-new Tesla 7.6kW inverter because it was installed with low coolant, and until it got topped off last week, was throwing "Inverter Over Temperature" alerts (which after a recent update, were only visible after logging on as Installers instead of Customers. Super-shady.)

Nice find!  (And nice work on your project https://github.com/darryllee/teslog).
Feel free to issue a pull request with what you know so far, I'm sure the rest of the community will be interested and can help learn more.
Thanks!

DOH, I just saw you already published a bash script that's far more elegant than mine. I will create a PR for the vitals endpoint I discovered. :-}

I'm taking a look at the binary file output on my powerwall now.  It looks like the various serial numbers of the gateway and the battery units are plastered all over this file.  So I suggest you scrub those out before posting an example.
I don't know if there is some special encoding or not yet. But I suspect that these are key-value pairs and probably some standard format from Redis or something similar.    I'll put into a hex editor and see what I can reverse engineer.
example:  I'm looking at the output in VIM:
Line 24:
^OTHC_AmbientTemp!^@^@^@^@^@^@=@
THOC_AmbientTemp is likely the key, and the value is encoded in the hexidecimal value here: ^@^@^@^@^@^@=@ ;  I'll need to see the hex values vs the ASCII translation to see if the numbers make sense.
These would be interesting to have the values too:
37 ^XPOD_nom_energy_remaining!^@^@^@^@^@~Á@^R#
38 ^XPOD_nom_full_pack_energy!^@^@^@^@^@×Ê@^R%
39 ^ZPOD_available_charge_power!^@^@^@^@^@X»@^R%
40 ^ZPOD_available_dischg_power!^@^@^@^@^@X»@^R%
41 ^ZPOD_curtailed_charge_power!^@^@^@^@^@^@^@^@^R%
42 ^ZPOD_curtailed_dischg_power!^@^@^@^@^@^@^@^@^R^_
Also, setting up a "charge completed" notification would be useful in my automation to then enable charging for the car on those days when I'm making more solar than I can use/store in the powerwall.  But it looks like these are broken out for each powerwall (if you have multiples on the same gatway).
Powerwall 1:
Line 50:
50 ^RPOD_ChargeComplete0^@^R^Y
Others:
51 ^UPOD_DischargeComplete0^@^R^[
52 ^WPOD_PersistentlyFaulted0^@^R^Z
53 ^VPOD_PermanentlyFaulted0^@^R^U
54 ^QPOD_ChargeRequest0^@^R^U
55 ^QPOD_ActiveHeating0^@^R^O
56 ^KPOD_Resting0^@^R^O
57 ^KPOD_CCVhold0^@^R^Z
58 ^VPOD_HeatZeroCurrentReg0^@^R^
59 ^XPOD_ExtendedCapacityMode0^@^R^Y
60 ^UPOD_IpcRequestedBusUp0^@^R^Y
61 ^UPOD_SelfTestRequested0^A
Powerwall 2:
134 ^RPOD_ChargeComplete0^@^R^Y
135 ^UPOD_DischargeComplete0^@^R^[
136 ^WPOD_PersistentlyFaulted0^@^R^Z
137 ^VPOD_PermanentlyFaulted0^@^R^U
138 ^QPOD_ChargeRequest0^@^R^U
139 ^QPOD_ActiveHeating0^@^R^O
140 ^KPOD_Resting0^@^R^O
141 ^KPOD_CCVhold0^@^R^Z
142 ^VPOD_HeatZeroCurrentReg0^@^R^
143 ^XPOD_ExtendedCapacityMode0^@^R^Y
144 ^UPOD_IpcRequestedBusUp0^@^R^Y
145 ^UPOD_SelfTestRequested0^A
This is also interesting:
172 ^SPINV_MainRelayState*^LRELAY_CLOSED^R&
173 ^VPINV_NeutralRelayState*^LRELAY_CLOSED^R^V

Ha yes, I just created a PR without including my vitals file because of it contained my serial numbers, and I didn't feel like loading up a hex editor and scrubbing that info.
But yes, tons of very interesting info that I've not seen in other endpoints.

Not sure if I'm going the right direction but I'm looking at this:
THC_AmbientTemp!......<@
0x0A (Hex value 0A) plus another hex byte always precedes each Value.  (maybe 0x0A indicates the end of the line?)
The ! = 0x21 - This looks like a the delimiter for the end of each of the keys.
The @ = 0x40 - This also looks like a delimiter as well.
So the hex in between those delimiters should be the value.
In my case I see the following HEX as the value:
00 00 00 00  00 80 3C
0x80  = 128
0x3C = 60
Does this mean 128.60 °F ?
Seems a bit hot.  My ambient outside of the chassis is 80.24 °F.

Ha yes, I just created a PR without including my vitals file because of it contained my serial numbers, and I didn't feel like loading up a hex editor and scrubbing that info.
But yes, tons of very interesting info that I've not seen in other endpoints.

Merged.

Here's a bit of data between THC_AmbientTemp! up to " TEPOD:
At 2:36PM (ambient temp 88ºF)
00 00 00 00 40 42 40 0A EF 06 0A 97 01 0A 94 01 0A
At 6:06PM (82ºF)
CE CC CC CC CC 4C 42 40 0A 80 07 0A 97 01 0A 94 01 0A
At 6:26PM (80ºF)
9A 99 99 99 99 19 42 40 0A 80 07 0A 97 01 0A 94 01 0A
So this seems consistent for end of data:
0A 97 01 0A 94 01 0A

From user power.saver on the teslamotorclub.com forums:

The binary data after the text strings is the value, in 64-bit IEEE-754 Floating Point notation. There is a ! character separating the text from the number, and the 8 bytes are in little-endian format.
For example, in the vitals file is the text:
THC_AmbientTemp!
This is followed by 8 bytes (shown here as hexadecimal digits):
CE CC CC CC CC CC 40 40
Here they are in big-endian format (again shown as hexadecimal digits):
40 40 CC CC CC CC CC CE
Those 8 bytes comprise the 64-bits of the IEEE-754 Floating Point number.
The decimal equivalent of that is 33.6 which is the temperature in Celsius. (92.5°F)
Text fields that are followed by a * character have a text value after them (enumeration).
The object:
THC_State*
has the value:
THC_STATE_AUTONOMOUSCONTROL
An interesting one in this file is:
POD_DC_Bus_Voltage!
which has a value of 428.1 volts.

I'm power.saver on the TMC forums.
Some additional info on the objects with * after them (text enumerations). The first byte after the * character is the length of the text string, followed by the actual text. That wasn't explained in my post on the TMC forums.

With a little work, I was able to parse out most of the vitals file. Here is the text result. The first column is the number of bytes into the file where the parser found the start of an object. This will probably vary since differences in the text string lengths affects the file size.
A new delimiter was found: if a 0 (ascii zero) appears after the object then the value is TRUE or FALSE, and is represented by a binary 1 or 0 in the next byte. I show TRUE or FALSE when this occurs.
0031 : 1118431-00-J =
00b4 : 1092170-03-G =
0128 : THC_State = THC_STATE_AUTONOMOUSCONTROL
0152 : THC_AmbientTemp = 33.600000
019b : 1081100-79-J =
020b : POD_nom_energy_to_be_charged = 2756.000000
0234 : POD_nom_energy_remaining = 11115.000000
0259 : POD_nom_full_pack_energy = 13707.000000
027e : POD_available_charge_power = 7000.000000
02a5 : POD_available_dischg_power = 7000.000000
02cc : POD_curtailed_charge_power = 0.000000
02f3 : POD_curtailed_dischg_power = 0.000000
031a : POD_est_DC_Bus_Power = -2320.000000
033b : POD_Ibatt = -50.600000
0351 : POD_Vbatt = 46.600000
0367 : POD_DC_Bus_Voltage = 428.100000
0386 : POD_state = POD_ACTIVE
039f : POD_enable_line = TRUE
03b4 : POD_ChargeComplete = FALSE
03cc : POD_DischargeComplete = FALSE
03e7 : POD_PersistentlyFaulted = FALSE
0404 : POD_PermanentlyFaulted = FALSE
0420 : POD_ChargeRequest = FALSE
0437 : POD_ActiveHeating = FALSE
044e : POD_Resting = FALSE
045f : POD_CCVhold = FALSE
0470 : POD_HeatZeroCurrentReg = FALSE
048c : POD_ExtendedCapacityMode = FALSE
04aa : POD_IpcRequestedBusUp = FALSE
04c5 : POD_SelfTestRequested = TRUE
050e : 1081100-79-J =
057e : PINV_EnergyDischarged = 4173130.000000
05a0 : PINV_EnergyCharged = 4827370.000000
05bf : PINV_VSplit1 = 123.300000
05d8 : PINV_VSplit2 = 123.200000
05f1 : PINV_PllFrequency = 59.995000
060f : PINV_PllLocked = TRUE
0623 : PINV_Pout = 2.320000
0639 : PINV_Qout = -0.020000
064f : PINV_Vout = 246.500000
0665 : PINV_Fout = 60.011000
067b : PINV_ReadyForGridForming = TRUE
0699 : PINV_State = PINV_GridFollowing
06bb : PINV_GridState = Grid_Compliant
06dd : PINV_MainRelayState = RELAY_CLOSED
0702 : PINV_NeutralRelayState = RELAY_CLOSED
072a : PINV_Iavail = 32.400000
0742 : PINV_Savail = 7.990000
075a : PINV_Vdc = 428.100000
076f : PINV_HardwareEnableLine = TRUE
078c : PINV_PowerLimiter = PWRLIM_No_Power_Limit
080d : 1092170-03-G =
0881 : THC_State = THC_STATE_AUTONOMOUSCONTROL
08ab : THC_AmbientTemp = 29.100000
08f4 : 1081100-59-J =
0964 : POD_nom_energy_to_be_charged = 2758.000000
098d : POD_nom_energy_remaining = 11207.000000
09b2 : POD_nom_full_pack_energy = 13810.000000
09d7 : POD_available_charge_power = 7000.000000
09fe : POD_available_dischg_power = 7000.000000
0a25 : POD_curtailed_charge_power = 0.000000
0a4c : POD_curtailed_dischg_power = 0.000000
0a73 : POD_est_DC_Bus_Power = -2350.000000
0a94 : POD_Ibatt = -51.200000
0aaa : POD_Vbatt = 46.600000
0ac0 : POD_DC_Bus_Voltage = 428.000000
0adf : POD_state = POD_ACTIVE
0af8 : POD_enable_line = TRUE
0b0d : POD_ChargeComplete = FALSE
0b25 : POD_DischargeComplete = FALSE
0b40 : POD_PersistentlyFaulted = FALSE
0b5d : POD_PermanentlyFaulted = FALSE
0b79 : POD_ChargeRequest = FALSE
0b90 : POD_ActiveHeating = FALSE
0ba7 : POD_Resting = FALSE
0bb8 : POD_CCVhold = FALSE
0bc9 : POD_HeatZeroCurrentReg = FALSE
0be5 : POD_ExtendedCapacityMode = FALSE
0c03 : POD_IpcRequestedBusUp = FALSE
0c1e : POD_SelfTestRequested = TRUE
0c67 : 1081100-59-J =
0cd7 : PINV_EnergyDischarged = 4179150.000000
0cf9 : PINV_EnergyCharged = 4857010.000000
0d18 : PINV_VSplit1 = 123.400000
0d31 : PINV_VSplit2 = 123.100000
0d4a : PINV_PllFrequency = 60.007000
0d68 : PINV_PllLocked = TRUE
0d7c : PINV_Pout = 2.330000
0d92 : PINV_Qout = -0.030000
0da8 : PINV_Vout = 246.600000
0dbe : PINV_Fout = 60.013000
0dd4 : PINV_ReadyForGridForming = TRUE
0df2 : PINV_State = PINV_GridFollowing
0e14 : PINV_GridState = Grid_Compliant
0e36 : PINV_MainRelayState = RELAY_CLOSED
0e5b : PINV_NeutralRelayState = RELAY_CLOSED
0e83 : PINV_Iavail = 32.400000
0e9b : PINV_Savail = 7.980000
0eb3 : PINV_Vdc = 428.100000
0ec8 : PINV_HardwareEnableLine = TRUE
0ee5 : PINV_PowerLimiter = PWRLIM_No_Power_Limit
0f67 : 1118431-00-J =
0fdb : ISLAND_VL1N_Main = 122.500000
0ff8 : ISLAND_FreqL1_Main = 60.010000
1017 : ISLAND_VL1N_Load = 122.500000
1034 : ISLAND_FreqL1_Load = 60.010000
1053 : ISLAND_PhaseL1_Main_Load = 0.000000
1078 : ISLAND_VL2N_Main = 121.000000
1095 : ISLAND_FreqL2_Main = 60.010000
10b4 : ISLAND_VL2N_Load = 121.000000
10d1 : ISLAND_FreqL2_Load = 60.010000
10f0 : ISLAND_PhaseL2_Main_Load = 0.000000
1115 : ISLAND_VL3N_Main = 0.000000
1132 : ISLAND_FreqL3_Main = 0.000000
1151 : ISLAND_VL3N_Load = 0.000000
116e : ISLAND_FreqL3_Load = 0.000000
118d : ISLAND_PhaseL3_Main_Load = -256.000000
11b2 : ISLAND_L1L2PhaseDelta = -256.000000
11d4 : ISLAND_L1L3PhaseDelta = -256.000000
11f6 : ISLAND_L2L3PhaseDelta = -256.000000
1218 : ISLAND_GridState = ISLAND_GridState_Grid_Compliant
124d : ISLAND_L1MicrogridOk = TRUE
1267 : ISLAND_L2MicrogridOk = TRUE
1281 : ISLAND_L3MicrogridOk = FALSE
129b : ISLAND_ReadyForSynchronization = TRUE
12bf : ISLAND_GridConnected = TRUE
12d9 : SYNC_ExternallyPowered = FALSE
12f5 : SYNC_SiteSwitchEnabled = TRUE

Unsubscribe
On Sat, Aug 7, 2021 at 9:47 PM sean-user @.***> wrote:

With a little work, I was able to parse out most of the vitals file. Here
is the text result. The first column is the number of bytes into the file
where the parser found the start of an object. This will probably vary
since differences in the text string lengths affects the file size.
A new delimiter was found: if a 0 (ascii zero) appears after the object
then the value is TRUE or FALSE, and is represented by a binary 1 or 0 in
the next byte. I show TRUE or FALSE when this occurs.
0031 : 1118431-00-J =
00b4 : 1092170-03-G =
0128 : THC_State = THC_STATE_AUTONOMOUSCONTROL
0152 : THC_AmbientTemp = 33.600000
019b : 1081100-79-J =
020b : POD_nom_energy_to_be_charged = 2756.000000
0234 : POD_nom_energy_remaining = 11115.000000
0259 : POD_nom_full_pack_energy = 13707.000000
027e : POD_available_charge_power = 7000.000000
02a5 : POD_available_dischg_power = 7000.000000
02cc : POD_curtailed_charge_power = 0.000000
02f3 : POD_curtailed_dischg_power = 0.000000
031a : POD_est_DC_Bus_Power = -2320.000000
033b : POD_Ibatt = -50.600000
0351 : POD_Vbatt = 46.600000
0367 : POD_DC_Bus_Voltage = 428.100000
0386 : POD_state = POD_ACTIVE
039f : POD_enable_line = TRUE
03b4 : POD_ChargeComplete = FALSE
03cc : POD_DischargeComplete = FALSE
03e7 : POD_PersistentlyFaulted = FALSE
0404 : POD_PermanentlyFaulted = FALSE
0420 : POD_ChargeRequest = FALSE
0437 : POD_ActiveHeating = FALSE
044e : POD_Resting = FALSE
045f : POD_CCVhold = FALSE
0470 : POD_HeatZeroCurrentReg = FALSE
048c : POD_ExtendedCapacityMode = FALSE
04aa : POD_IpcRequestedBusUp = FALSE
04c5 : POD_SelfTestRequested = TRUE
050e : 1081100-79-J =
057e : PINV_EnergyDischarged = 4173130.000000
05a0 : PINV_EnergyCharged = 4827370.000000
05bf : PINV_VSplit1 = 123.300000
05d8 : PINV_VSplit2 = 123.200000
05f1 : PINV_PllFrequency = 59.995000
060f : PINV_PllLocked = TRUE
0623 : PINV_Pout = 2.320000
0639 : PINV_Qout = -0.020000
064f : PINV_Vout = 246.500000
0665 : PINV_Fout = 60.011000
067b : PINV_ReadyForGridForming = TRUE
0699 : PINV_State = PINV_GridFollowing
06bb : PINV_GridState = Grid_Compliant
06dd : PINV_MainRelayState = RELAY_CLOSED
0702 : PINV_NeutralRelayState = RELAY_CLOSED
072a : PINV_Iavail = 32.400000
0742 : PINV_Savail = 7.990000
075a : PINV_Vdc = 428.100000
076f : PINV_HardwareEnableLine = TRUE
078c : PINV_PowerLimiter = PWRLIM_No_Power_Limit
080d : 1092170-03-G =
0881 : THC_State = THC_STATE_AUTONOMOUSCONTROL
08ab : THC_AmbientTemp = 29.100000
08f4 : 1081100-59-J =
0964 : POD_nom_energy_to_be_charged = 2758.000000
098d : POD_nom_energy_remaining = 11207.000000
09b2 : POD_nom_full_pack_energy = 13810.000000
09d7 : POD_available_charge_power = 7000.000000
09fe : POD_available_dischg_power = 7000.000000
0a25 : POD_curtailed_charge_power = 0.000000
0a4c : POD_curtailed_dischg_power = 0.000000
0a73 : POD_est_DC_Bus_Power = -2350.000000
0a94 : POD_Ibatt = -51.200000
0aaa : POD_Vbatt = 46.600000
0ac0 : POD_DC_Bus_Voltage = 428.000000
0adf : POD_state = POD_ACTIVE
0af8 : POD_enable_line = TRUE
0b0d : POD_ChargeComplete = FALSE
0b25 : POD_DischargeComplete = FALSE
0b40 : POD_PersistentlyFaulted = FALSE
0b5d : POD_PermanentlyFaulted = FALSE
0b79 : POD_ChargeRequest = FALSE
0b90 : POD_ActiveHeating = FALSE
0ba7 : POD_Resting = FALSE
0bb8 : POD_CCVhold = FALSE
0bc9 : POD_HeatZeroCurrentReg = FALSE
0be5 : POD_ExtendedCapacityMode = FALSE
0c03 : POD_IpcRequestedBusUp = FALSE
0c1e : POD_SelfTestRequested = TRUE
0c67 : 1081100-59-J =
0cd7 : PINV_EnergyDischarged = 4179150.000000
0cf9 : PINV_EnergyCharged = 4857010.000000
0d18 : PINV_VSplit1 = 123.400000
0d31 : PINV_VSplit2 = 123.100000
0d4a : PINV_PllFrequency = 60.007000
0d68 : PINV_PllLocked = TRUE
0d7c : PINV_Pout = 2.330000
0d92 : PINV_Qout = -0.030000
0da8 : PINV_Vout = 246.600000
0dbe : PINV_Fout = 60.013000
0dd4 : PINV_ReadyForGridForming = TRUE
0df2 : PINV_State = PINV_GridFollowing
0e14 : PINV_GridState = Grid_Compliant
0e36 : PINV_MainRelayState = RELAY_CLOSED
0e5b : PINV_NeutralRelayState = RELAY_CLOSED
0e83 : PINV_Iavail = 32.400000
0e9b : PINV_Savail = 7.980000
0eb3 : PINV_Vdc = 428.100000
0ec8 : PINV_HardwareEnableLine = TRUE
0ee5 : PINV_PowerLimiter = PWRLIM_No_Power_Limit
0f67 : 1118431-00-J =
0fdb : ISLAND_VL1N_Main = 122.500000
0ff8 : ISLAND_FreqL1_Main = 60.010000
1017 : ISLAND_VL1N_Load = 122.500000
1034 : ISLAND_FreqL1_Load = 60.010000
1053 : ISLAND_PhaseL1_Main_Load = 0.000000
1078 : ISLAND_VL2N_Main = 121.000000
1095 : ISLAND_FreqL2_Main = 60.010000
10b4 : ISLAND_VL2N_Load = 121.000000
10d1 : ISLAND_FreqL2_Load = 60.010000
10f0 : ISLAND_PhaseL2_Main_Load = 0.000000
1115 : ISLAND_VL3N_Main = 0.000000
1132 : ISLAND_FreqL3_Main = 0.000000
1151 : ISLAND_VL3N_Load = 0.000000
116e : ISLAND_FreqL3_Load = 0.000000
118d : ISLAND_PhaseL3_Main_Load = -256.000000
11b2 : ISLAND_L1L2PhaseDelta = -256.000000
11d4 : ISLAND_L1L3PhaseDelta = -256.000000
11f6 : ISLAND_L2L3PhaseDelta = -256.000000
1218 : ISLAND_GridState = ISLAND_GridState_Grid_Compliant
124d : ISLAND_L1MicrogridOk = TRUE
1267 : ISLAND_L2MicrogridOk = TRUE
1281 : ISLAND_L3MicrogridOk = FALSE
129b : ISLAND_ReadyForSynchronization = TRUE
12bf : ISLAND_GridConnected = TRUE
12d9 : SYNC_ExternallyPowered = FALSE
12f5 : SYNC_SiteSwitchEnabled = TRUE
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://urldefense.com/v3/__https://github.com/vloschiavo/powerwall2/issues/51*issuecomment-894742452__;Iw!!IKRxdwAv5BmarQ!MX696kmbCCiWOdDVri9n4kbcZNAfTDfRuzw6YSf3m6zBFGBGfo2OsV01g5ngPg$,
or unsubscribe
https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/ABU4JYCDTKZRP35PQGK7GHDT3YD4NANCNFSM5BVJR2CQ__;!!IKRxdwAv5BmarQ!MX696kmbCCiWOdDVri9n4kbcZNAfTDfRuzw6YSf3m6zBFGBGfo2OsV3uEC3dQw$
.
Triage notifications on the go with GitHub Mobile for iOS
https://urldefense.com/v3/__https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675__;!!IKRxdwAv5BmarQ!MX696kmbCCiWOdDVri9n4kbcZNAfTDfRuzw6YSf3m6zBFGBGfo2OsV0M-v6DlA$
or Android
https://urldefense.com/v3/__https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email__;!!IKRxdwAv5BmarQ!MX696kmbCCiWOdDVri9n4kbcZNAfTDfRuzw6YSf3m6zBFGBGfo2OsV3ihpVmiQ$
.

Nice work everyone!  Thanks!
Is anyone up for creating a sample decoder script?

Not perfect..  but close:
`syntax = "proto3";
option go_package = "github.com/example/path/gen;gen";
package teslapower;
import "google/protobuf/timestamp.proto";
message DeviceVital {
optional string name = 1;
oneof value {
uint32 intValue = 3;
float floatValue = 4;
string stringValue = 5;
bool boolValue = 6;
}
}
message ConnectionParameters {
optional string ipAddress = 1;
optional string serialPort = 2;
optional string serialBaud = 3;
optional uint32 modbusId = 4;
}
message TeslaHardwareId {
optional uint32 pcbaId = 1;
optional uint32 assemblyId = 2;
optional uint32 usageId = 3;
}
message TeslaEnergyEcuAttributes {
optional int32 ecuType = 1;
optional TeslaHardwareId hardwareId = 2;
}
message GeneratorAttributes {
optional float nameplateRealPowerW = 1;
optional float nameplateApparentPowerVa = 2;
}
message PVInverterAttributes {
optional float nameplateRealPowerW = 1;
}
message MeterAttributes {
repeated uint32 meterLocation = 1;
}
message Device {
optional string din = 1;
optional string partNumber = 2;
optional string serialNumber = 3;
optional string manufacturer = 4;
optional string siteLabel = 5;
optional string componentParentDin = 6;
optional string firmwareVersion = 7;
optional google.protobuf.Timestamp firstCommunicationTime = 8;
optional google.protobuf.Timestamp lastCommunicationTime = 9;
optional ConnectionParameters connectionParameters = 10;
oneof deviceAttributes {
TeslaEnergyEcuAttributes teslaEnergyEcuAttributes = 11;
GeneratorAttributes generatorAttributes = 12;
PVInverterAttributes pvInverterAttributes = 13;
MeterAttributes meterAttributes = 14;
};
}
message SiteControllerConnectedDevice {
optional Device device = 1;
}
message SiteControllerConnectedDeviceWithVitals {
optional SiteControllerConnectedDevice device = 1;
repeated DeviceVital vitals = 2;
repeated string alerts = 3;
}
message DevicesWithVitals {
repeated SiteControllerConnectedDeviceWithVitals devices = 1;
}`

`syntax = "proto3";
option go_package = "github.com/example/path/gen;gen";
package teslapower;
import "google/protobuf/timestamp.proto";
message DeviceVital {
optional string name = 1;
oneof value {
int64 intValue = 3;
double floatValue = 4;
string stringValue = 5;
bool boolValue = 6;
}
}
message StringValue {
string value = 1;
}
message UInt32Value {
uint32 value = 1;
}
message ConnectionParameters {
optional string ipAddress = 1;
optional string serialPort = 2;
optional string serialBaud = 3;
optional uint32 modbusId = 4;
}
message TeslaHardwareId {
optional UInt32Value pcbaId = 1;
optional UInt32Value assemblyId = 2;
optional UInt32Value usageId = 3;
}
message TeslaEnergyEcuAttributes {
optional int32 ecuType = 1;
optional TeslaHardwareId hardwareId = 2;
}
message GeneratorAttributes {
optional uint64 nameplateRealPowerW = 1;
optional uint64 nameplateApparentPowerVa = 2;
}
message PVInverterAttributes {
optional uint64 nameplateRealPowerW = 1;
}
message MeterAttributes {
repeated uint32 meterLocation = 1;
}
message DeviceAttributes {
oneof deviceAttributes {
TeslaEnergyEcuAttributes teslaEnergyEcuAttributes = 1;
GeneratorAttributes generatorAttributes = 2;
PVInverterAttributes pvInverterAttributes = 3;
MeterAttributes meterAttributes = 4;
};
}
message Device {
optional StringValue din = 1;
optional StringValue partNumber = 2;
optional StringValue serialNumber = 3;
optional StringValue manufacturer = 4;
optional StringValue siteLabel = 5;
optional StringValue componentParentDin = 6;
optional StringValue firmwareVersion = 7;
optional google.protobuf.Timestamp firstCommunicationTime = 8;
optional google.protobuf.Timestamp lastCommunicationTime = 9;
optional ConnectionParameters connectionParameters = 10;
repeated DeviceAttributes deviceAttributes = 11;
}
message SiteControllerConnectedDevice {
optional Device device = 1;
}
message SiteControllerConnectedDeviceWithVitals {
repeated SiteControllerConnectedDevice device = 1;
repeated DeviceVital vitals = 2;
repeated string alerts = 3;
}
message DevicesWithVitals {
repeated SiteControllerConnectedDeviceWithVitals devices = 1;
}`

Unsubscribe
On Mon, Sep 20, 2021 at 8:13 PM brianhealey @.***>
wrote:

`syntax = "proto3";
option go_package = "github.com/example/path/gen;gen";
package teslapower;
import "google/protobuf/timestamp.proto";
message DeviceVital {
optional string name = 1;
oneof value {
int64 intValue = 3;
double floatValue = 4;
string stringValue = 5;
bool boolValue = 6;
}
}
message StringValue {
string value = 1;
}
message UInt32Value {
uint32 value = 1;
}
message ConnectionParameters {
optional string ipAddress = 1;
optional string serialPort = 2;
optional string serialBaud = 3;
optional uint32 modbusId = 4;
}
message TeslaHardwareId {
optional UInt32Value pcbaId = 1;
optional UInt32Value assemblyId = 2;
optional UInt32Value usageId = 3;
}
message TeslaEnergyEcuAttributes {
optional int32 ecuType = 1;
optional TeslaHardwareId hardwareId = 2;
}
message GeneratorAttributes {
optional uint64 nameplateRealPowerW = 1;
optional uint64 nameplateApparentPowerVa = 2;
}
message PVInverterAttributes {
optional uint64 nameplateRealPowerW = 1;
}
message MeterAttributes {
repeated uint32 meterLocation = 1;
}
message DeviceAttributes {
oneof deviceAttributes {
TeslaEnergyEcuAttributes teslaEnergyEcuAttributes = 1;
GeneratorAttributes generatorAttributes = 2;
PVInverterAttributes pvInverterAttributes = 3;
MeterAttributes meterAttributes = 4;
};
}
message Device {
optional StringValue din = 1;
optional StringValue partNumber = 2;
optional StringValue serialNumber = 3;
optional StringValue manufacturer = 4;
optional StringValue siteLabel = 5;
optional StringValue componentParentDin = 6;
optional StringValue firmwareVersion = 7;
optional google.protobuf.Timestamp firstCommunicationTime = 8;
optional google.protobuf.Timestamp lastCommunicationTime = 9;
optional ConnectionParameters connectionParameters = 10;
repeated DeviceAttributes deviceAttributes = 11;
}
message SiteControllerConnectedDevice {
optional Device device = 1;
}
message SiteControllerConnectedDeviceWithVitals {
repeated SiteControllerConnectedDevice device = 1;
repeated DeviceVital vitals = 2;
repeated string alerts = 3;
}
message DevicesWithVitals {
repeated SiteControllerConnectedDeviceWithVitals devices = 1;
}`
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://urldefense.com/v3/__https://github.com/vloschiavo/powerwall2/issues/51*issuecomment-923574346__;Iw!!IKRxdwAv5BmarQ!LTuYq4U0-M37z9UhvlyLhI5EqgLPKxRN6t5ooRYeRD7pUn_FKXYz6nePTWtq3Q$,
or unsubscribe
https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/ABU4JYGT53TXE3GNAW4TNJTUC7Z65ANCNFSM5BVJR2CQ__;!!IKRxdwAv5BmarQ!LTuYq4U0-M37z9UhvlyLhI5EqgLPKxRN6t5ooRYeRD7pUn_FKXYz6nfBCvfKRQ$
.
Triage notifications on the go with GitHub Mobile for iOS
https://urldefense.com/v3/__https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675__;!!IKRxdwAv5BmarQ!LTuYq4U0-M37z9UhvlyLhI5EqgLPKxRN6t5ooRYeRD7pUn_FKXYz6ne6enLNpA$
or Android
https://urldefense.com/v3/__https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign*3Dnotification-email*26utm_medium*3Demail*26utm_source*3Dgithub__;JSUlJSU!!IKRxdwAv5BmarQ!LTuYq4U0-M37z9UhvlyLhI5EqgLPKxRN6t5ooRYeRD7pUn_FKXYz6nfTWBXPQA$.

I logged into my TEG on its internal wifi with installer access. the "System" page shows all the vital statistics nicely represented. The reason I was in there was to figure out how much electricity each solar string was generating. But it would be nice to parse out the additional values like temp and frequency as well.
Since the website shows it clear text, I assume there is a javascript on the page that does the decoding but I haven't looked at it in detail yet :)

@sean-user Thanks for your code. I ported a version to python here but it isn't quite right. The protobuf path seems very interesting.
@brianhealey Were you able to parse with that?  Any quick tips for someone just learning protobuf? :)  I just brew install protobuf and checking out how to get a proto definition into python to parse these vitals payloads.  Going through protoc now...

Update: I used the great work by @brianhealey (tesla.proto definition) to process the vitals payload and produce a json representation. Work in progress: https://github.com/jasonacox/pypowerwall/tree/main/examples/vitals

What would be the best way to format the output a bit better?
I am Pre-PTO I think there are issues with my MCI - Maybe once a week / day etc. I have to keep an eye on my strings.
Sometimes 1 or maybe 2 strings are not functioning so I have to do a reboot of the inverters to make them show up again.
If getting the fromat correct and comparing it from a baseline everyday if something turns False for a string some type of email notifaction would be amazing for a lot of people I believe when they think there solar is under producing but in the end it could be a MCI problem or a string issue since people complain alot about under performance this could be the reason why!
After executing this is what I get for the Dispaly String Data:
{'A': {'Current': 1.1300000000000001, 'Voltage':<PHONE_NUMBER>0000002, 'Power': 173.0, 'State': 'PV_Active', 'Connected': True}, 'B': {'Current': 0.54, 'Voltage': 339.0, 'Power': 178.0, 'State': 'PV_Active', 'Connected': True}, 'C': {'Current': 0.59, 'Voltage': 324.6, 'Power': 197.0, 'State': 'PV_Active', 'Connected': True}, 'D': {'Current': 0.48, 'Voltage': 99.4, 'Power': 64.0, 'State': 'PV_Active', 'Connected': True}, 'A1': {'Current': 1.1, 'Voltage':<PHONE_NUMBER>0000003, 'Power': 327.0, 'State': 'PV_Active', 'Connected': True}, 'B1': {'Current': 1.09, 'Voltage': 303.5, 'Power': 320.0, 'State': 'PV_Active_Parallel', 'Connected': True}, 'C1': {'Current': 0.0, 'Voltage': -1.299999999999999, 'Power': 0.0, 'State': 'PV_Active', 'Connected': False}, 'D1': {'Current': 0.0, 'Voltage': -0.29999999999999893, 'Power': 0.0, 'State': 'PV_Active', 'Connected': False}}

Hi @reli3 - I want to keep an eye on my strings as well, which is why I wrote this.  I also wanted to keep an eye on the Neruio solar meter which is also in the vitals payload.   For string data, if you want to see it in human readable format, just specify jsonformat as True like this:
strings = pw.strings(jsonformat=True)
print(strings)

Output:
{
    "A": {
        "Connected": true,
        "Current": 0.48,
        "Power": 115.0,
        "State": "PV_Active",
        "Voltage": 251.8
    },
    "B": {
        "Connected": false,
        "Current": 0.0,
        "Power": 0.0,
        "State": "PV_Active",
        "Voltage": -2.3999999999999995
    },
    "C": {
        "Connected": true,
        "Current": 1.82,
        "Power": 526.0,
        "State": "PV_Active",
        "Voltage": 293.6
    },
    "D": {
        "Connected": true,
        "Current": 1.81,
        "Power": 521.0,
        "State": "PV_Active_Parallel",
        "Voltage": 294.0
    }
}

I push that into a tiny installation of Splunk (see TinySplunk) and get a graph like this:

I have a lot of trouble with the Neruio solar meter that Tesla installed in my Powerwall+ so I also have scripts to watch that.  It shows up in the vitals payload as a device that I monitor and capture to create alerts (e.g. if it doesn't show up in the vitals payload that means it has lost connection with the gateway and shut down solar production):
"NEURIO--VAHxxxxxxxxxx": {
        "NEURIO_CT0_InstRealPower": 1333.5606370817025,
        "NEURIO_CT0_Location": "solar",
        "Parent": "STSTSM--xxxxxxx-00-E--TGxxxxxxxxxxxx",
        "firmwareVersion": "1.6.1-Tesla",
        "lastCommunicationTime": "1638720520",
        "manufacturer": "NEURIO",
        "partNumber": "",
        "serialNumber": "VAHxxxxxxxxxx"
    }

@jasonacox omg thata is amazing exactly what I think I am looking for....any chance for a write up to set this up via tinysplunk on how you are getting the data in? Does it pull the string data every minute?
This output will help in the morning and ill run it via cron to put to what I need instead of having to login via installer and flipping the powerwall switch when I think its not producing the correct amount of solar as expected!
I loved your other dashboard with grafana was thinking of making a PI display just for that eventually.
I believe I have the NEURIO as well since I have Two Powerwall+ and 4x Powerwalls. I haven't had any issues with the Neurio at least so far besides sometimes the inveter likes to discount from the Wi-Fi every once in a while which what I found was to re join it via the App surpisingly and it fixes it.
Thanks for your amazing work so far with this!

Thanks @reli3 !  I cross posted this as an issue under pypowerwall for Strings Data- (see https://github.com/jasonacox/pypowerwall/issues/1) since I don't know how relevant it would be for this issue.  I included the script I use to gather and push the string data into Splunk.  I plan to add this to the Grafana dashboard as well. :)

Glad to see that the protocol buffer example helped. :) There were some errors in what I put together, so I forked your branch to update it.  Hopefully this update will help flush out any issues your might be seeing and/or include fields I didn't have before that might be silently ignored.


Descriptions truncated in collections list

Describe the bug
The description of the collections, which should be complete, is truncated in the list of collections.
To Reproduce
Just go in collection and watch descriptions.
Expected behavior
Complete description.
Screenshots
https://github.com/user-attachments/assets/8b7d9616-71b1-4382-abbb-a7241da61f57
Device (please complete the following information):

Browser: Firefox
Maintainerr version: 2.2.1

It's purely cosmetic, it helps me remember which collection it is, but otherwise everything works fine.

The truncation is on purpose, but it shouldn't be scrollable like that.

This should be resolved in v2.5.0.


Bump project version to next snapshot

https://github.com/mark-vieira/gradle-maven-settings-plugin/blob/master/gradle.properties#L2 is currently 0.5-SNAPSHOT although 0.5 version has been released. Please bump it to 0.6-SNAPSHOT or similar. Consider also automating this as part of the release process.

Done. Yes, release automation could use some work. Right now this is manual given how infrequently I release.


"Contribute to this page" link broken everywhere

I think this is related to the new structure setup?

Yes but that's a haxe.org issue.

I'll take a look now.
On Tue, May 19, 2015 at 6:51 PM, Simon Krajewski<EMAIL_ADDRESS>wrote:

Yes but that's a haxe.org issue.
—
Reply to this email directly or view it on GitHub
https://github.com/HaxeFoundation/HaxeManual/issues/145#issuecomment-103436196
.

Nice. Sorry, sometimes I don't exactly know what is part of what.

HaxeManual: content and everything related to how the markdown export looks
haxe.org: presentation on the website


The import org.apache.commons.collections.Unmodifiable is never used.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


SSCSSI-404 hearing api merge with enable flag

Before creating a pull request make sure that:

[ ] commit messages are meaningful and follow good commit message guidelines
[ ] README and other documentation has been updated / added (if needed)
[ ] tests have been updated / new tests has been added (if needed)

Please remove this line and everything above and fill the following sections:
JIRA link (if applicable)
Change description
Does this PR introduce a breaking change? (check one with "x")
[ ] Yes
[ ] No

closing, its not require


Do Captions Currently Work?

Hello,
Do captions currently work? The documentation talks about the interface for them, but as far as I can tell they are not implemented yet?

Resolved.


Question: How to run a large Network in the simple way without TVM and LLVM?

I am trying to write a FASTER RCNN network by TVM. I would like to run the network by only one shared library to run faster. I am thinking about how to save these modules.
One way is using different schedule method and using different building.
For example, I can save the modules one by one. And then run by many shared library.
        s4 = topi.cuda.schedule_pool(pool)
        s5 = topi.cuda.schedule_injective(change)
        s6 = topi.cuda.schedule_dense(dense)
        s7 = topi.cuda.schedule_softmax(Softmax)
        s8 = topi.cuda.schedule_depthwise_conv2d_nchw(NewDepthwiseConv2d)

    f4 = tvm.build(s4, [Pool, pool], device)
    f5 = tvm.build(s5, [reshape, change], device)
    f6 = tvm.build(s6, [reshape1, weight, dense], device)
    f7 = tvm.build(s7, [softmax, Softmax], device)
    f8 = tvm.build (s8,[Input, Filter, Scale, Shift, NewDepthwiseConv2d], device)

But the problem of this method is how to link different libraries, I have no idea about saving output of one module and then using it on the next module as input.
And another way is build all modules using the same schedule so that I can save everything in one shared library.
For example，
        s1 = topi.cuda.schedule_depthwise_conv2d_nchw(DepthwiseConv2d)
        s2 = topi.cuda.schedule_depthwise_conv2d_nchw(ScaleShift)
        s3 = topi.cuda.schedule_depthwise_conv2d_nchw(Relu)
    f1 = tvm.build(s1, [Input, Filter, DepthwiseConv2d], device)
    f2 = tvm.build(s2, [Input, Filter, Scale, Shift, ScaleShift], device)
    f3 = tvm.build(s3, [Input, Filter, Scale, Shift, Relu],device)

But this method make the program run fastest.
I dont know which method is suitable for the large network(e.g. Faster RCNN), or two methods both are wrong. I am trying to run only the testing stage of Faster Rcnn, So I dont need train them. That's why I want to run by shared library and Cpp file.

You should use nnvm compiler which compiles and save a network for you, instead per operator

I know NNVM can help to save a network as a shared library, but it has not the example for running the saved module by C++. And my C++ is not good.
And I would like to run a network without TVM and NNVM, ONLY by shared library and Cpp file, does it work well?

I saw the example for running .o file in /home/aries/nnvm/tutorials/deploy_model_on_rasp.py.
Does it include the device module?

nnvm compiles the module directly into a module that only depend on TVM runtime. You will be able to run it with only the all in one runtime or the runtime built with tvm

Thank you so much!!!


0.8.0: pytest is failing in ipython unit

Issue
I'm trying to package your module as an rpm package. So I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

python3 -sBm build -w --no-isolation
because I'm calling build with --no-isolation I'm using during all processes oly locally installed modules
install .whl file in </install/prefix>
run pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>

In below output is as well one pytest warning.
Here is pytest output:
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-sphinx-codeautolink-0.8.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-sphinx-codeautolink-0.8.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/sphinx-codeautolink-0.8.0, configfile: pyproject.toml, testpaths: tests
collected 186 items

tests/__init__.py ...                                                                                [  1%]
tests/extension/__init__.py ........................................F...............................                                                                 [ 40%]
tests/parse/__init__.py ..................x..                                                                                [ 51%]
tests/parse/assign.py ....................x...............                                                                                [ 70%]
tests/parse/chain.py ...                                                                                [ 72%]
tests/parse/scope.py ..........x......x....x...x..........x.............                                                                                [100%]

================================================================================= FAILURES =================================================================================
_________________________________________________________________________ test_references[file40] __________________________________________________________________________

file = PosixPath('/home/tkloczko/rpmbuild/BUILD/sphinx-codeautolink-0.8.0/tests/extension/ref/ref_ipython.txt')
tmp_path = PosixPath('/tmp/pytest-of-tkloczko/pytest-725/test_references_file40_0')

    @pytest.mark.parametrize('file', ref_tests)
    def test_references(file: Path, tmp_path: Path):
        """
        Basic extension tests for reference building.

        The tests are structured as .txt files, parsed and executed here.

The HTML pages are in ../../../../../tmp/pytest-of-tkloczko/pytest-725/test_references_file40_0/build/html.
--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------
/tmp/pytest-of-tkloczko/pytest-725/test_references_file40_0/src/index.rst:5: WARNING: Pygments lexer name 'ipython3' is not known
============================================================================= warnings summary =============================================================================
tests/parse/__init__.py::TestSimple::test_import_star_invalid
  /home/tkloczko/rpmbuild/BUILDROOT/python-sphinx-codeautolink-0.8.0-2.fc35.x86_64/usr/lib/python3.8/site-packages/sphinx_codeautolink/parse.py:351: UserWarning: Could not import module `not_a_module` for parsing!
    warn(f'Could not import module `{node.module}` for parsing!')

-- Docs: https://docs.pytest.org/en/stable/warnings.html
========================================================================= short test summary info ==========================================================================
XFAIL tests/parse/__init__.py::TestSimple::test_overwrite_dotted_not_tracked
  Assignments to imports are not tracked.
XFAIL tests/parse/assign.py::TestAssign::test_partially_overwrite_dotted_import
  Following assigns into imports would be a pain.
XFAIL tests/parse/scope.py::TestFunction::test_func_uses_overrided_later
  Assignments are not tracked.
XFAIL tests/parse/scope.py::TestFunction::test_global_in_outermost_scope
  No reason to do this in the real world.
XFAIL tests/parse/scope.py::TestFunction::test_global_overwritten_then_used_in_outer
  Global assigns not tracked to outer scopes.
XFAIL tests/parse/scope.py::TestFunction::test_global_deleted_then_used_in_outer
  Global deletes not tracked to outer scopes.
XFAIL tests/parse/scope.py::TestComprehension::test_comp_leaks_walrus
  Assignments are not tracked.
FAILED tests/extension/__init__.py::test_references[file40] - AssertionError: assert 0 == 2
=========================================================== 1 failed, 178 passed, 7 xfailed, 1 warning in 40.55s ===========================================================

Expected behavior
pytest should not fail.

Hi, thanks for the report! Did you install ipython?

Hi, thanks for the report! Did you install ipython?

And that was missing bit. Looking on error message I was a bit hard to guess that ipython was missing :P
Thank you.
In below output I see UserWarning. Is it something which is expected?
(feel free to close this ticket)
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-sphinx-codeautolink-0.8.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-sphinx-codeautolink-0.8.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/sphinx-codeautolink-0.8.0, configfile: pyproject.toml, testpaths: tests
plugins: mock-3.6.1, rerunfailures-10.2, cov-3.0.0, flake8-1.0.7, black-0.3.12
collected 186 items

tests/__init__.py ...                                                                                [  1%]
tests/extension/__init__.py ........................................................................                                                                 [ 40%]
tests/parse/__init__.py ..................x..                                                                                [ 51%]
tests/parse/assign.py ....................x...............                                                                                [ 70%]
tests/parse/chain.py ...                                                                                [ 72%]
tests/parse/scope.py ..........x......x....x...x..........x.............                                                                                [100%]

============================================================================= warnings summary =============================================================================
tests/parse/__init__.py::TestSimple::test_import_star_invalid
  /home/tkloczko/rpmbuild/BUILDROOT/python-sphinx-codeautolink-0.8.0-2.fc35.x86_64/usr/lib/python3.8/site-packages/sphinx_codeautolink/parse.py:351: UserWarning: Could not import module `not_a_module` for parsing!
    warn(f'Could not import module `{node.module}` for parsing!')

-- Docs: https://docs.pytest.org/en/stable/warnings.html
========================================================================= short test summary info ==========================================================================
XFAIL tests/parse/__init__.py::TestSimple::test_overwrite_dotted_not_tracked
  Assignments to imports are not tracked.
XFAIL tests/parse/assign.py::TestAssign::test_partially_overwrite_dotted_import
  Following assigns into imports would be a pain.
XFAIL tests/parse/scope.py::TestFunction::test_func_uses_overrided_later
  Assignments are not tracked.
XFAIL tests/parse/scope.py::TestFunction::test_global_in_outermost_scope
  No reason to do this in the real world.
XFAIL tests/parse/scope.py::TestFunction::test_global_overwritten_then_used_in_outer
  Global assigns not tracked to outer scopes.
XFAIL tests/parse/scope.py::TestFunction::test_global_deleted_then_used_in_outer
  Global deletes not tracked to outer scopes.
XFAIL tests/parse/scope.py::TestComprehension::test_comp_leaks_walrus
  Assignments are not tracked.
================================================================ 179 passed, 7 xfailed, 1 warning in 42.03s ================================================================

It is! And actually that reminds me we should change that to a Sphinx warning, which was done for other warnings in #89.


chore: removing std/secrets.

Current dependencies on/for this PR:

main

PR #1032

PR #1034   👈

This comment was auto-generated by Graphite.


Switchery and jquery-fullscreen Type Definitions

Switchery: https://github.com/abpetkov/switchery
jquery-fullscreen: https://github.com/kayahr/jquery-fullscreen-plugin

@bgrieder thanks mate!


Add flexibility option for constraints [review]

It would be something like this :
constraints.add(
Constraint.ALIGN_MARKERS,
tol=10e-4
instant=Instant.START,
first_marker_idx=Bow.frog_marker,
second_marker_idx=violon_string.bridge_marker,
)
for g :
max_bound=0+tol, min_bound=0-tol

apparently we can already!

it should be add in some examples !

There is already some :)


x/website/_content/doc: add note about how much memory the Go runtime retains to the GC guide

Hello,
I spent a lot of time to find the reason of a "memory leak" which did not exist. There was a fundamental misunderstanding about memory management. We thought that the garbage collector returns unused heap memory back to the operating system, which was wrong.

I think is is very important to know.

This is intended to be explained in https://tip.golang.org/doc/gc-guide#GOGC, though that section has a lot more words and could perhaps have a more direct, succinct mention of the total heap size.
cc @mknyszek

My application allocates memory on demand, e.g.when it receives a web request. The annoying part was, that it does not return all used memory back to the OS regardless how long we wait. So the memory usage (from OS point of view) did not go back to the level that we had before the web request.
The exact amount which gets returned is not so important to me, but the facts that it does not return all free memory. To be honest, I understood the gc-guide only partially. Maybe that's the reason why the final result from OS point of view was not clear to me.
Anyway, I ensured already that the memory usage does not constantly increase over time. This works fine.

I waited 2 days


std.traits: scope is only for types with pointers

This is blocking https://github.com/dlang/dmd/pull/5897
Please do not delay pulling it.

Auto-merge toggled on

That was quick! Thanks!


Fix Bug #2224

Description
解决多次创建同一个数据库，不能创建同样存储元数据的向量库的问题
How Has This Been Tested?
多次创建，删除同一个数据库，存储元数据的向量库可以正确创建

Hi, @dusx1981 Thank you very much for your contribution, Can you sink the chromadb operation into ChromaStore? Otherwise, if you change it here, other vector databases will have problems.
chromadb.api.client.SharedSystemClient.clear_system_cache()

Hi, @dusx1981 Thank you very much for your contribution, Can you sink the chromadb operation into ChromaStore? Otherwise, if you change it here, other vector databases will have problems.
chromadb.api.client.SharedSystemClient.clear_system_cache()

Good suggestion, adjustments have been made.


Field of view

Hi, I test the algorithm on my dataset and find that the field of view is only about 120 degrees in front of the ego car.
I am wondering if this algorithm could detect 360 degrees of view.

  That's why the code removes the points outside the front image when loading the point cloud run.py Line 210. A quick hack would be replacing get_cam_points_in_image_with_rgb with get_cam_points to process all points. But since the checkpoints are not trained with objects outside the view, the results would likely be messy. If you have your own dataset, it's better to retrain the model in the complete view.

 That's why the code removes the points outside the front image when loading the point cloud run.py Line 210. A quick hack would be replacing get_cam_points_in_image_with_rgb with get_cam_points to process all points. But since the checkpoints are not trained with objects outside the view, the results would likely be messy. If you have your own dataset, it's better to retrain the model in the complete view.

Make sense, and thank you very much!


[Moderation] Setting package to be excluded from testing doesn't save permanently (or does it?)

Click the 'Exempt the package from verification testing' checkbox and add a reason
Leave package status as 'Submitted'
Clear 'If Submitted status, require maintainer to make changes?'
Hit Save
Page reloads and message pop-up appears saying changes have been saved (and Verification Testing Exemption is shown)
Go to browser location bar, click and press enter (to reload page without re-posting)
Page shows no testing exemption

Hmm.. tried it with another package and didn't clear the 'If Submitted status, require maintainer to make changes` flag, and it seems to have stuck that time.

Yep.. just confirmed - if I check 'If Submitted status, require maintainer to make changes', then it doesn't save.
That would be the bug.

And because that check is not cleared, the package is now in a 'Waiting for maintainer' state, which is not ideal

Not sure when I will get a chance to look at this one, but I will try as soon as I can.

Looks like a workaround is to make sure you add a moderation comment. If I do that, then the changes seem to stick


Builder must be able to pickle/unpickle sdata structure to save/load precompiled normalization units

Resolved with PR #8


The recommend target template size used by sot

I find that if the size of target template is small,such as 150*150,the sot get poor performance.I would like to know what target template size is proper for sot?

SOT needs the rich feature representation of the target for tracking, and the template image will be resized to 128*128 in the pipeline. However, the appropriate raw input size depends on your target itself.

Closed as no further discussion. Feel free to reopen it again.


DynamoDBRepositoryFactory logs confusing warning if Spring Data Version.class.getPackage().getImplementationVersion() returns null

Spring Data DynamoDB Version: 4.5.5
Spring Data Version: 1.13.11.RELEASE
AWS SDK Version: 1.11.289
Java Version: 1.8.x
Platform Details: Mac OS, 10.13.4 (High Sierra)

During the application startup, a confusing warning is logged (see the last line logged at the WARN level):
 018-06-12 16:02:39 INFO  o.s.s.d.d.r.s.DynamoDBRepositoryFactory --- Spring Data DynamoDB Version: 4.5.5 (2.0)
 2018-06-12 16:02:39 INFO  o.s.s.d.d.r.s.DynamoDBRepositoryFactory --- Spring Data Version:          null
 2018-06-12 16:02:39 INFO  o.s.s.d.d.r.s.DynamoDBRepositoryFactory --- AWS SDK Version:              1.11.289
 2018-06-12 16:02:39 INFO  o.s.s.d.d.r.s.DynamoDBRepositoryFactory --- Java Version:                 1.8.0_144 - Java HotSpot(TM) 64-Bit Server VM 25.144-b01
 2018-06-12 16:02:39 INFO  o.s.s.d.d.r.s.DynamoDBRepositoryFactory --- Platform Details:             Mac OS X 10.13.4
 2018-06-12 16:24:10 WARN  o.s.s.d.d.r.s.DynamoDBRepositoryFactory --- This Spring Data DynamoDB implementation might not be compatible with the available Spring Data classes on the classpath! NoDefClassFoundExceptions or similar might occur!

This is caused by the fact that the result of the following operation in DynamoDBRepositoryFactory.java (line 48) is null - although the legitimate version of Spring Data is used, apparently:
  		String springDataVersion = Version.class.getPackage().getImplementationVersion();

Not sure why the value is not set, and whether it is the same for all Spring Data versions, but the warning, while benign, may be scary...

Actually the version is only reported from Spring-Data 2.0 going forward.
Little we can do here to change that.
But as a matter of fact, this was version check was not meant to be backported to the 4.5.x line to be honest.

Fixed in 4.5.6 that was just pushed to Maven Central

Any Idea why I receive this warning with the combination below?
o.s.s.d.d.r.s.DynamoDBRepositoryFactory  : Spring Data DynamoDB Version: 5.0.4 (2.0)
o.s.s.d.d.r.s.DynamoDBRepositoryFactory  : Spring Data Version:          2.1.3.RELEASE
o.s.s.d.d.r.s.DynamoDBRepositoryFactory  : AWS SDK Version:              1.11.473
o.s.s.d.d.r.s.DynamoDBRepositoryFactory  : Java Version:                 1.8.0_191 - Java HotSpot(TM) 64-Bit Server VM 25.191-b12
o.s.s.d.d.r.s.DynamoDBRepositoryFactory  : Platform Details:             Windows 10 10.0
o.s.s.d.d.r.s.DynamoDBRepositoryFactory  : This Spring Data DynamoDB implementation might not be compatible with the available Spring Data classes on the classpath!
NoDefClassFoundExceptions or similar might occur!

This may be due to version problem in the class path. Did you try changing the version?


35975: jsoup:HtmlFuzzer: Uncaught exception in org.jsoup.parser.HtmlTreeBuilder.process

https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=35975
Detailed Report: https://oss-fuzz.com/testcase?key=4820007715471360
Project: jsoup
Fuzzing Engine: libFuzzer
Fuzz Target: HtmlFuzzer
Job Type: libfuzzer_asan_jsoup
Platform Id: linux
Crash Type: Uncaught exception
Crash Address:
Crash State:
org.jsoup.parser.HtmlTreeBuilder.process
org.jsoup.parser.HtmlTreeBuilderState$9.process
java.base/java.lang.String.compareTo
Sanitizer: address (ASAN)
Recommended Security Severity: Low
Crash Revision: https://oss-fuzz.com/revisions?job=libfuzzer_asan_jsoup&revision=202107050606
Reproducer Testcase: https://oss-fuzz.com/download?testcase_id=4820007715471360
Issue filed automatically.

Haven't been able to repro - I will wait for the next execution of the fuzzer with updated fixes and review.

Shows as still reproducing, but I can't repro with the attached sample. Thinking that it might be a character encoding issue, I tried every charset on my platform but still could not repro. I'm not sure what I'm missing.
 By dropping my stack size way down (256K hits it). I guess the fuzzer is optimizing the sample data for its specific stack size :)

Fixed -- I think this is a gap in the spec? AFAICT am processing according to the rules in https://html.spec.whatwg.org/multipage/parsing.html#parsing-main-intable - but there exists states where the resetInsertionMode will leave in the InTable state, and so reprocessing the current token will recurse and eventually overflow. Fixed by testing that the mode changed and if not, inserting directly.
Also fixed up a couple other processing states.


using OpticSim doesn't cause AGFGlassCat.jl to be rebuilt

Just pulled the latest version of OpticSim (already had a previous version installed) and then did using OpticSim at the repl and got this error:
julia> using OpticSim
[ Info: Precompiling OpticSim [24114763-4efb-45e7-af0e-cde916beb153]
ERROR: LoadError: LoadError: SystemError: opening file "/home/brian/repos/OpenSource/OpticSim.jl/src/GlassCat/data/jl/AGFGlassCat.jl": No such file or directory

The same thing happened on my windows laptop and on my linux desktop. Had to build OpticSim to get it to work.
Is this the expected behavior or should using OpticSim cause AGFGlassCat.jl to be built if it doesn't already exist?

This is expected.
using serves one purpose: to load existing source files. This is a good design principle.
AGFGlassCat.jl needs to be built. There are three sensible ways to run the build script:

include("deps/build.jl")
using Pkg; Pkg.build() or ] build
using Pkg; Pkg.add("OpticSim") or ] add OpticSim

In a way, 3 ends up executing 2, and 2 ends up executing 1. Neither cloning, dev-ing nor using the package will trigger the build steps. I don't think it makes sense for any of them to do so.

This is expected.
Build steps are triggered by Pkg.build. Neither using, cloning, pulling, or dev-ing (as someone's previously asked) will trigger Pkg.build. The only other canonical commands that run the build steps are Pkg.add("OpticSim") and include("deps/build.jl").
We could potentially check for AGFGlassCat.jl at precompilation and trigger a build from there. But this is a messy solution for various reasons. I've added a nice error message that tells you what to do if AGFGlassCat.jl isn't found.


How to reproduce results using i3d for ActivityNet?

Dear authors,
First of all, great work with ActionFormer. I wish to reproduce results on ActivityNet using i3d features. Could you please make available the features and the config file that I can use to train the model?
Thanks in advance

We can work on this, yet probably with a lower priority. Those I3D features are much larger (in storage) than the TSP features, and produce slightly worse results.

Thanks for a prompt response @happyharrycn. Are these I3D features the standard ones used by other repos or are they something that you specifically pre-process for actionformer? Is there a way you can prioritize making them available? If storage is an issue, I can provide a link to you to make them available as I have generous amount of storage on my end. Thanks!

Hi, for ActivityNet I3D features, we use the feature provided by CMCS. You can still download the original feature from CMCS link. But it will need a BaiduYun VIP account and around 500G disk. After downloading the feature, you can find the I3D feature in it.

Thanks @tzzcl for providing the features. Can you also provide the config file that I can use for i3d features? And also provide guidance on any other change I might need to do in the code to run with i3d features?

As far as I recall, we downsampled the features and uses a config file similar to our TSP features. I can dig into our internal repo and find the config file this weekend.

Also looking forward to the I3D-based configuration and related code files. Thanks a lot!

This should now been addressed by commit bc9c4c62d9471b711cefcfd3fc8e8da3e5e89cb5 (config file) and 88f6110a25d43f92ee9c4fb193eea8564296590e (documentation).
Let us know if there is any further issue.

@g1910 @AwalkZY I will mark this as resolved (see the latest commits) unless further issues are raised.


Nondeterministic weights in formula_containments

Does the test here pass with #653

The nondeterministic behavior is fixed by your proposed changes!
However, the tests cannot pass as the Union of provenance sets is currently implemented as a sum of probabilities while it should be a noisy or. This leads to the wrong probabilistic calculation. This is fixed by the LQP Disjoint PR that changes the default projection to be an independent-project (noisy or of probabilities) and therefore the union is fixed as well.

I'm closing this PR and moving the test to the LQP disjoint project PR


[Feature Req] Add setting to delete/move a funscript along with the video.

Great idea! I will look into that. Multi-axis comes to mind, where there are multiple scripts alongside a video, those should be included as well.

Added the feature.


BUG blackdoc modifies but returns "left unchanged"?

nbqa-blackdoc............................................................Failed
- hook id: nbqa
- files were modified by this hook

All done! ✨ 🍰 ✨
1 file left unchanged.
All done! ✨ 🍰 ✨
1 file left unchanged.

noticed with on commit 624027ec8858bf2051a58e7aeea1a43c2c8cec9d from git@github.com:MarcoGorelli/birdsong-recognition.git

this is because when you save a notebook in vscode, it removes the endline at the end of the file

actually, I should probably make sure that the endline is the same as what it was before

Resolved - I'd rather be opinionated and always leave a trailing newline if --nbqa-mutate is passed


Show a progress bar when fetching events.

There's no way to make progress bars compatible with the terminal that don't crash jupyter notebooks.

i think @selik mentioned that we could use logging to make things terminal/jupyter compatible

Another option I was thinking of for jupyter notebooks is to use an notebook-specific widget. Something like https://ipywidgets.readthedocs.io/en/latest/examples/Widget List.html#IntProgress


IT worked now it doesnt

this plugin did work for me then out of no where when I click on the sign it doesnt give me anything, it says i purchased the item but my balance is the same so is my invatory it didnt give me anything, please help I tryed using diffrent Carrot shop verzions but it still didnt work !please!

What version of minectaft, sponge and plugin are you using?

I'll close the ticket as you did not answer.
If you still have the problem you can join https://discord.gg/3ZSt2cq for quick help


In SzAbstractFactory change from createSz* to create*

See https://github.com/senzing-garage/sz-sdk-go/issues/194

Pull requests:

https://github.com/senzing-garage/playground/pull/181


Fix (some) VMCNET nans

Background
We for a long time been observing NaNs in VMCNET much more commonly than in the DeepMind FermiNet repository. For example when running the N2 molecule most runs would generate NaNs. I investigated this by checkpointing on NaNs and then reloading the checkpoint and checking the amplitude, parameters, etc.
The proximal cause of the NaNs I investigated was that a walker reached a very improbable location (amplitude i.e. 1e-70), which resulted in grad(log(Psi)) becoming extremely large, causing the parameters to explode and resulting in NaNs in the following epoch.
The key the figuring out why this happened was turning on the flag to log the min and max amplitude at each epoch and noticing that the min amplitude was stuck at 1e-21 for many epochs before it suddenly jumped to 1e-70. When I say stuck I mean that the min amplitude was EXACTLY the same over and over for thousands of epochs. This is not a reasonable behavior since the parameters change every epoch, so even if the walker is not moving, we would expect the min amplitude to change at least somewhat epoch to epoch.
This led me to realize that we were not updating the walker amplitudes when we updated the parameters. This meant a walker could get stuck if the optimization process caused the position it was in to become extremely unlikely, because the walker would retain its old amplitude which would make the acceptance probability of moving to a nearby location extremely small. Furthermore, if this extremely low probability event happened, then the amplitude would become extremely small and cause NaNs as I described above.
Fix
Okay the explanation above is a bit complex, but the fix is quite simple: update the amplitudes when the parameters are updated!
I tested the N2 molecule with this fix and got 4 runs in a row with no NaNs - much better than before!

FYI @jeffminlin  thought you might be interested too in knowing at least one cause of the NaNs that were plaguing us.

Amazing, thanks for the detective work @ggoldsh! These kinds of problems plagued us enough last year that I wonder if the models + data should just be encapsulated in such a way so that the parameters can't be (easily) updated without the wavefn amplitudes also being updated.

Mm maybe, like some object where you have to update params via a method, and it automatically updates the amplitudes when you do? That could be nice.

@jeffminlin  Oh yeah, I had this strange feeling I had seen this bug before but couldn't figure out why we wouldn't have fixed it if we had seen it. Must be because it had showed up in the reloading params case.
Good points, I'll think about the best way to do that and take a note for later.


Add support for placing workers on AWS outpost

What this PR does / why we need it:
The PR adds support for creating the nodes on AWS Outpost.
For users who wish to run the nodes and their workloads on their premises, either for privacy or to enhance the AWS Outpost.
Creating the private subnet in AWS Outpost will lead to creating the instances on it as well.
The option to specify the AWS Outpost ARN is added to infra create command and to cluster create command by flag --outpost-arn $OUTPOST_ARN
AWS Outpost has several limitations:

AWS Outpost supports only gp2 for root volume type. The unsupported value will be overridden by gp2.
AWS Outpost is homed to a specific availability zone. Multi AZ will not be supported. The zone should match the AWS Outpost AZ. The Zone is determined according to AWS Outpost AZ if not provided.

aws-sdk-go version was upgraded to support outposts API.
Which issue(s) this PR fixes
https://github.com/openshift/hypershift/issues/1935
Checklist

[x] Subject and description added to both, commit and PR.
[x] Relevant issues have been referenced.
[ ] This change includes docs.
[ ] This change includes unit tests.

thanks!
can you please break out the PR in two commits, one for vendoring one for actual changes so it's easier for reviewers?
Can we also update the desc to say something like "Enable creating infra for aws outpost through the cli"? (Reason is this is not proposing API changes as far as I can see. The cli is just convenient automation, it does not define hypershift in any way. And the infra is a pre requirement we happen to automate via cli for convenience as a consumer choice. So using "support" here might be confusing).

thanks! can you please break out the PR in two commits, one for vendoring one for actual changes so it's easier for reviewers? Can we also update the desc to say something like "Enable creating infra for aws outpost through the cli"? (Reason is this is not proposing API changes as far as I can see. The cli is just convenient automation, it does not define hypershift in any way. And the infra is a pre requirement we happen to automate via cli for convenience as a consumer choice. So using "support" here might be confusing).

Makes sense. Split to 2 commits (and to another PR for typos that weren't related to this one) + updated message to be more accurate

/approve


Error when repeatedly recording

Discussed in https://github.com/katspaugh/wavesurfer.js/discussions/3251

Originally posted by nguyenhuyhoang241298 October 12, 2023
https://wavesurfer.xyz/examples/?record.js
When I record 2 or more times, I have warning like this

@AmitJoki looks like a regression after the pause feature (#3249).
Version 7.3.4 works fine.

The line that's causing the error is this https://github.com/katspaugh/wavesurfer.js/blob/main/src/plugins/record.ts#L97 which is not related to pause feature but rather the issue of the tabs showing recording is still on even when stopRecording has been called. I'll see if it can be solved in another way or that line could be removed.


Bump eslint-config-react-app, fixes #142

This is an attempt to fix the problem described in #142. Summary:

There seems to be a problem with the dependency resolution during npm install.
It seems the culprit is<EMAIL_ADDRESS>which transitively depends on<EMAIL_ADDRESS>|| ^6.0.0", while<EMAIL_ADDRESS>was already specified as a dependency already.
Bumping to<EMAIL_ADDRESS>seems to fix the problem.

thank you for your contribution!
the dependency which you are trying to upgrade will be centrally managed in another module and removed from this repository, therefore the fix will probably not be working in the future. Please check for alternative solution in https://github.com/inveniosoftware/invenio-assets/issues/142#issuecomment-1110904136

thank you for your contribution!
the dependency which you are trying to upgrade will be centrally managed in another module and removed from this repository, therefore the fix will probably not be working in the future. Please check for alternative solution in https://github.com/inveniosoftware/invenio-assets/issues/142#issuecomment-1110904136


[TWINFACES-91] Experimental crud data table on twin classes page

Also please rebase on dev to solve merge conflicts


fix: do not route to plugin when command is invalid

Description of changes
When commands are deemed invalid by the input-manager, the command is set to "help" but the plugin is not changed. This leads to invalid commands still being routed to non-core plugins, which sometimes results in users being prompted, even after a message showing that the command is invalid is printed. This change stops commands from being routed to plugins by setting the plugin to core. The plugin that executes the help command does not matter, as all plugins yield the same top-level help. See the first photo below for an example of a prompt appearing after an invalid command message and the second photo for the new behavior.

Issue #, if available
#12118

Description of how you validated changes
Checklist

[x] PR description included
[x] yarn test passes
[] Tests are changed or added
[ ] Relevant documentation is changed or added (and PR referenced)
[ ] New AWS SDK calls or CloudFormation actions have been added to relevant test and service IAM policies
[ ] Pull request labels are added

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Excellent fix!

Codecov Report

Merging #12132 (562e192) into dev (984c0f8) will decrease coverage by 3.14%.
The diff coverage is 56.28%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff             @@
##              dev   #12132      +/-   ##
==========================================
- Coverage   49.45%   46.32%   -3.14%
==========================================
  Files         692      821     +129
  Lines       33424    37587    +4163
  Branches     6840     7643     +803
==========================================
+ Hits        16529    17411     +882
- Misses      15370    18514    +3144
- Partials     1525     1662     +137

Impacted Files
Coverage Δ

...sync-simulator/src/data-loader/opensearch/index.ts
45.45% <0.00%> (ø)

...s/amplify-appsync-simulator/src/type-definition.ts
100.00% <ø> (ø)

...sync-simulator/src/velocity/util/dynamodb-utils.ts
6.55% <0.00%> (ø)

...y-appsync-simulator/src/velocity/util/map-utils.ts
10.00% <ø> (ø)

...tor/src/velocity/util/transform/dynamodb-filter.ts
4.93% <0.00%> (ø)

...ync-simulator/src/velocity/util/transform/index.ts
21.05% <0.00%> (ø)

...appsync-simulator/src/velocity/value-mapper/map.ts
86.88% <ø> (ø)

...ackages/amplify-category-auth/src/commands/auth.ts
0.00% <0.00%> (ø)

...es/amplify-category-auth/src/commands/auth/help.ts
0.00% <0.00%> (ø)

.../amplify-category-auth/src/commands/auth/import.ts
0.00% <0.00%> (ø)

... and 539 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Review utility classes and propose new utility classes

Prior to Launch we should review the utility classes we currently have to ensure they are up to date, and also propose new utility classes that may be useful.
In the display utility for display: none , visibility: hidden should also be used.
Another suggestion:
.pf-u-truncate {   white-space: nowrap;   overflow: hidden;   text-overflow: ellipsis; }

.pf-u-nowrap { white-space: nowrap; } would be a good one, for example to wrap around 2 or more things so they wrap together when the content around it wraps (like 2 words in a paragraph you want to keep together)

We can remove

.pf-u-flex
.pf-u-display-inline-flex
.pf-u-display-inline-grid
.pf-u-display-flex-row, .pf-u-display-flex-row-reverse, .pf-u-display-flex-columns, .pf-u-display-flex-column-reverse
.pf-u-justify-content-*
.pf-u-align-items-*
.pf-u-align-content-* i suspect this will rarely be used
.pf-u-flex-wrap, .pf-u-flex-nowrap, .pf-u-flex-wrap-reverse

Add to utilities

.pf-u-display-initial

Add to .pf-l-flex

.pf-m-inline-flex
.pf-m-align-items-{flex-start, flex-end, center, baseline}
.pf-m-align-self-{flex-start, flex-end, center, baseline, stretch}
.pf-m-flex-wrap, .pf-m-flex-nowrap

Add to .pf-l-grid

.pf-m-inline-grid to .pf-l-grid

Update

documentation for .pf-u-hidden

I also question the need for !important for all utilities..

+1 for .pf-u-truncate, would have used that today

It would be really helpful if the utility styles  were extends and the class extend them. That would make it easier for the web component project to directly extend styles from PF.