Feature: implement tests

theres currently no tests in place for the code, this should be implemented.

Hi, @deggja! I am interested in working on this issue. As a first step, for which functions should I write tests? It'll be helpful if you share your thought process.

Also, there are testing frameworks in Go that mock K8s cluster so that we can create resources in the mock to test whether the desired outcomes are met. @deggja

Hello @roopeshsn!
Brilliant. I am super happy to hear that you want to contribute here. Help is welcomed with open arms. As far as this issue goes and writing tests for the code, theres nothing in place as of today - so we would essentially need to write tests for all core functions. As far as using a testing framework goes, I am all ears and open to suggestions.
I will go through the code the in the next few days and write up a list of which functions you can prioritize/start with. Sound good?

Hello @roopeshsn! Brilliant. I am super happy to hear that you want to contribute here. Help is welcomed with open arms. As far as this issue goes and writing tests for the code, theres nothing in place as of today - so we would essentially need to write tests for all core functions. As far as using a testing framework goes, I am all ears and open to suggestions.
I will go through the code in the next few days and write up a list of which functions you can prioritise/start with. Sound good?

Yes, it sounds good!

Hello @roopeshsn! Brilliant. I am super happy to hear that you want to contribute here. Help is welcomed with open arms. As far as this issue goes and writing tests for the code, theres nothing in place as of today - so we would essentially need to write tests for all core functions. As far as using a testing framework goes, I am all ears and open to suggestions.
I will go through the code in the next few days and write up a list of which functions you can prioritise/start with. Sound good?

Yes, it sounds good!

I think for starters. The obvious big ones are the scanning functions. That would be:
ScanNetworkPolicies # scanner.go
ScanCiliumNetworkPolicies # cilium-scanner.go
ScanCiliumClusterwideNetworkPolicies # cilium-scanner.go

In addition to those, the dashboard specific functions:
gatherVisualizationData # visualizer.go
GatherClusterVisualizationData # visualizer.go
getNetworkPolicyYAML # visualizer.go

I think starting with these would make sense. Once tests for these are in place, we can move on to the rest of the logic.

Thanks! The ScanNetworkPolicies function have CLI interaction in-between. I am thinking how we can handle that.

Thanks! The ScanNetworkPolicies function have CLI interaction in-between. I am thinking how we can handle that.

Hmm. Yes. You will find that all the scanners have CLI logic in them. I have thought about taking them apart and splitting them up into smaller and separate functions, but I have not gotten to that yet.
I think theres three options;

Write tests for the parts of the functions that makes sense.
Start writing tests for other less complex functions.
Start rewriting the functions and split CLI logic into separate functions (this is definately a separate issue in itself..)

Disclaimer: I have not looked into writing tests yet, so not completely sure what it entails. Especially concerning the CLI logic. I can look more into this in the next few days and share my thoughts sometime this weekend.
What do you think? What are your thoughts on this?

Alright, I'll also look into how we can write tests for scanner functions. In the meantime, I'll start writing tests for dashboard specific functions.

Hi @deggja! I wrote a test for gatherVisualizationData function. The test function I wrote will create a fake clientset from fake library and create a pod and a networkpolicy. Then we can pass this fake clientset to the gatherVisualizationData function so that it will return the VisualizationData struct. We can compare both results.
Now the gatherVisualizationData function creates a clientset. We need to remove that line and pass the clientset as an argument. So that when we run the test we'll pass the fake clientset. Can I make that change?

Hi @deggja! I wrote a test for gatherVisualizationData function. The test function I wrote will create a fake clientset from fake library and create a pod and a networkpolicy. Then we can pass this fake clientset to the gatherVisualizationData function so that it will return the VisualizationData struct. We can compare both results.
Now the gatherVisualizationData function creates a clientset. We need to remove that line and pass the clientset as an argument. So that when we run the test we'll pass the fake clientset. Can I make that change?

Nice, sounds good. Make the required changes. I can help you test in your branch once its in place if you need it.

I'm done with writing tests for visualizer.go. Shall I proceed with refactoring scanner.go, so that I can write a test for ScanNetworkPolicies?

I'm done with writing tests for visualizer.go. Shall I proceed with refactoring scanner.go, so that I can write a test for ScanNetworkPolicies?

Very nice!
You can. However, there are other functions and helper functions in scanner.go that might be easier to start with if you want.

Done with writing tests for helper functions in scanner.go. I'll start by doing the same for cilium-scanner.go.

Now, the remaining functions to write tests are the main ones,

ScanNetworkPolicies
ScanCiliumNetworkPolicies

How we can proceed with them?

Now, the remaining functions to write tests are the main ones,

ScanNetworkPolicies
ScanCiliumNetworkPolicies

How we can proceed?

Hey @roopeshsn!
Great job so far, its looking good.
I have had to take some time off due to some personal reasons, so have not had time to look at all your commits yet. Hopefully I'll get time to do so in the near future.
Considering the task involving refactoring the main functions is quite tedious, maybe it makes sense to write tests for the non-CLI parts of those functions for the time being? We can rewrite the tests at a later stage, once the main functions have been rewritten as well.
Thoughts?

Now, the remaining functions to write tests are the main ones,

ScanNetworkPolicies
ScanCiliumNetworkPolicies

How we can proceed?

Hey @roopeshsn! Great job so far, its looking good. I have had to take some time off due to some personal reasons, so have not had time to look at all your commits yet (but I have skimmed through). Hopefully I'll get time to do so in the near future.
Considering the task involving refactoring the main functions is quite tedious, maybe it makes sense to write tests for the non-CLI parts of those functions for the time being? We can rewrite the tests at a later stage, once the main functions have been rewritten as well.
Thoughts?

Sure


chore: refactor diag logger

/cc @MSNev
Per request of @Flarna this is the diag refactoring split out of #6. There may be some relevant discussion in the previous PR.
This PR is a prerequisite for #10

The filtered logger is now the only global that is saved

log level and the logger are now enclosed in the filtered logger

setLogLevel is now removed in favor of a second optional argument to setLogger
setLogger()/setLogger(undefined)/setLogger(null) are now disallowed and replaced with disable

Calling this way anyway results in a noop logger

Constructing a log level filtered logger results in a logger which ignores the global log level and only uses the one it is specifically constructed with
Methods to create log level loggers and noop loggers are now not exported to the user and treated as internal concerns. The only exports are diag, DiagLogger, DiagLogLevel, and LogFunction.

@MSNev I can't add you as a reviewer because it seems you aren't a member of the OpenTelemetry community?

Ok, I know I said id mention that "m" word less, buuuutttt...
Does this result in smaller output? Looks like it might be close -- enough?
As this definitely looks easier to read. And while I can think of a few additional tricks, I'm not sure its worth it, basically

keep the diagLoggerFunctions but don't export from main project (effectively keeping internal)
still use for loop for diag.ts, here (which some funcky toUpper() for the level lookup) and in the noopLoger.ts.

The looping might trim a tiny bit off, but you also add the array itself and the import statements.

@MSNev I can't add you as a reviewer because it seems you aren't a member of the OpenTelemetry community?

https://github.com/open-telemetry/community/issues/666


Filter Options cant be changed?!

Hello,
i am trying to change the filter options depending on a state. I got the following code in my render method:
f(this.state.mode == "monthly") {
var filter_options = {
type: 'TextFilter',
defaultValue: myCurrentMonth,
};
} else {
var filter_options = {
type: 'TextFilter',
defaultValue: null,
};
}
and in the return statement, i got my TableHeaderColumn defined like this:
<TableHeaderColumn headerAlign='center' row='0' rowSpan='2' width="0" dataField='monat' filter={filter_options} editable={ false } hidden>Monat</TableHeaderColumn>
The TableHeaderColumn always use the first defined value for defaultValue. This might be intended to be. but how can i change the filter options? Any Idea or workaround?

Sorry for lately reply and let me check this out, it might be a bug.

hello @medianetic, this bug was fixed on v3.4.4, please let me know if the problem still remain, thanks

Hello @AllenFang
unfortunately the problem still remain. For testing i've tried the following (in my render method):
     if(this.state.mode == "monthly") {
        var filter_options = {
            type: 'TextFilter',
            defaultValue: myCurrentMonth,
        };
      } else {
        var filter_options = {
            type: 'TextFilter',
            defaultValue: '09',
        };
      }

Now if the value of this.state.mode is not 'monthly' the table sets the filter to null (not as expected to '09')

@medianetic, could you please describe your reproduce step on UI?  I think this bug was fixed but maybe we're not in the same line for reproducing this issue, thanks


Tab key does not give four spaces

Jodit Version:  3.2.xxxxx
Browser:  Chrome
OS:  Linux
Used Jodit here: https://xdsoft.net/jodit/

Type something
Press the 'tab' key

Actual behavior:
Focus in the editor is switched to another button or element.
Expected behavior:
I expect a tab to be four spaces

Nope, for form navigation - tab works correctly.


Support passing store in via props for testing

Allow you to pass the redux store in via props to make it easier to unit test (Instead of having to mess with Provider's or contexts)
See also #38
Closes #33

Thank you for your work on this. Released in 0.8.0.


Type Definitions

Hi! I would like to use type definitions with this library and other k6 official libraries.

Where can I find the type definitions for this?
Would be good if there could be some instructions on how to add the type definitions and install via npm if possible.

I would expect we can just replace esbuild with tsup and get everything we need

I have raised this as with Grafana and it is now a Feature Request at their end.

I added a PR that provides a way get to types along with a local installation of the plugin.


Add dataset classes mike nolan

Adds ECoG data reader functions to data.py
load_ecog_clfp_data() reads in clfp files from goose_wireless dataset. Will work on ECoG or LFP data. Must have experiment.json files in local directory.

this would be a great addition to aopy! i suggest adding "goose" to the names of some of these functions, or something similar to differentiate them from the ecube data loading functions.
question about the DataFile() class: could we get the same functionality just by converting the data into HDF format? so we can read portions of the data without loading the whole thing into memory?

@m-nolan Thanks for sharing the code in a timely manner for me to use it. I think we need to include tests for the function and update docstrings before we can merge.
Could you update docstrings for the functions ? @leoscholl what is the difference between these loading functions and our ECoG loading functions? If this is specific for Goose dataset, I recommend a separate goose_data.py script instead. Let me know what you think.

I'll add docstrings to functions I've added.
As for the DataFile() class, it would definitely work better if simply reading from an HDF5 file instead of parsing/reparsing the structure of binary files every time you recreate the class interface. I haven't created HDF5 versions of the goose_wireless data, but I've been meaning to do that for some time. Having the data in compressed array formats would also enable much, much more flexible pytorch dataset/dataloader creation methods than what I'm currently using.
What is our current data conditioning and preprocessing pipeline?

#181 is the beginning of our preprocessing pipeline. not much else has been written. we have some functions for filtering, band power estimates, spike detection, binning. but no formal pipeline to string those things together.

@pavi-rajes @leoscholl
I've added docstrings and resolved the merge conflict. There are some package import overlaps, but the code should run without interference.

Pavi wrote some passing tests for this. Can you review and approve, @leoscholl ?
Thanks!

@leoscholl
read_lfp is a more updated version of load_ecog_clfp_data using the xarray interface and that's suited to loading more general file_name.*lfp file formats than just *.clfp.

thanks for adding the data filtering / quality / outlier functions! they are super helpful broadly, i don't think they belong in this aopy.data.peslab subdirectory. would you mind taking them out of here and making a new branch and pull request for those functions?

do you have a small (<100MB) test dataset that you could upload so we can write some short tests to demo the lfp loading code?

@leoscholl

Moved data quality check code to /aopy/preproc/quality.py
Refactored other peslab functions to utils
corrected import calls

The quality code currently uses a homebrewed multitaper spectrogram generation function. There is already one of these written in precondition for individual PSDs. I'll rewrite the preproc.quality.mt_sgram function to use the nitime-based MTPSD estimator.

cool! the peslab stuff looks good, only exception is the docs will not be complete after refactoring. can you add peslab section to docs/source/data.rst? see preproc.rst as an example. also while you're at it can you fix the tqdm import in docs/source/conf.py -- see #289
for the data quality functions, it would be really helpful if we had some tests or examples. would you mind moving quality.py to a different pull request so we can merge all the peslab code now?

@leoscholl I would mind doing that because (1) the peslab data readers depend on the quality check functions and (2) I don't know how to split pull requests like that.

Got it. Added some basic tests for the quality check functions and swapped the data axes around to match convention. Could you update the peslab code to follow the same rules? And there are a few errors coming from quality.py but I think they are due to the window for spectrogram being too big for the test data.

Implemented changes and test alterations made to reflect data array shape changes. Should pass on server.


[🔨for root]:  make height fixed and update grid layout

For desktop resolition, place the Editor under the Preview and Settings modules as in the screenshot above.

To limit Playground height - set a fixed Root height. Let it be 500px. Note that you don't need to do this on XS. On XS you need to set a fixed Preview height, like it did for Setings module.

Note:

You will need to correct the collapse Editor animation so that it does not overlap the Preview module
See if everything is ok with example Playgroup inside Playground

It was decided not to do anything with Playground height (and the Settings height as well). Let's leave it to the user


Tracking issue: Move to Chalk IR

We want to get rid of our own representations of types and related data in hir_ty (like Ty, TraitRef and so on), and instead use the versions from chalk_ir everywhere. This is a rather large refactoring, and to achieve it without doing a gigantic PR that takes months, we're gradually aligning our own representations to look like the Chalk versions, with the goal that in the end we can mostly just define type Ty = chalk_ir::Ty<Interner> and be done.
We're actually getting close to this goal; I tried it to see where we are currently and we're at about 400 errors when doing the switch. Going through those errors, I found the following remaining tasks:

[ ] remove interner parameter from Substitution::interned (I added it wrongly)
[ ] move Ty inherent methods to free-standing, extension trait or builder (except those that exist on Chalk's Ty as well)

this is a big task that can be split further

[ ] same for TraitRef, ProjectionTy, Substitution ...
[ ] TyKind::Unknown -> Error
[ ] TyKind::ForeignType -> Foreign
[ ] adjust Lifetime to be more like Chalk (i.e. there's an interned method that contains the actual enum)
[ ] self_type_parameter takes Interner (on TraitRef and ProjectionTy)
[ ] remove all Binders value accesses, use VariableKinds instead of just a usize

make helper function for construction of VariableKinds with a number of type vars, since that's all we ever do right now

[ ] align Binders::subst with Chalk's substitute
[ ] hide Substitution internals, which are still getting accessed in lots of places
[ ] add lifetimes/consts to TyKind::Ref, TyKind::Array, DynTy
[ ] FnPointer: remove num_args, add num_binders (those are not the same), add FnSubst, substs -> substitution
[ ] remove SolutionVariables

I will start moving out the Ty creation methods to a dedicated builder next.

I'll go for Binders next.

I'll tackle add lifetimes/consts to TyKind::Ref, TyKind::Array, DynTy

@Veykril Great, keep in mind that for now we'll just always want to make them 'static :slightly_smiling_face:

I'll do FnPointer.

I'll move the accessors to the extension trait

I'm doing the remaining substs, substs_mut and equals_ctor.


chore: bdd-spring-1576240724 to 0.0.1

chore: Promote bdd-spring-1576240724 to version 0.0.1

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


Use alternate worker import

I'm hoping this fixes sourcemaps in workers based on some hints from this issue https://github.com/vitejs/vite/issues/11860

This didn't have the intended effect. Still needs to investigate it further.


Cannot write into markdown editor

Version
1.0.0
Test Environment
React latest
Current Behavior
Cannot write into markdown editor.
I used the plain example provided. I cannot type unless a select something and overwrite that. After that I can only press Enter, but other keys working. WYSIWYG mode is working just fine.
Expected Behavior
Able to write.

Quickly found the issue. We're using * {font-size: 0} to reset all white-spaces. Seems like it's causing issues for CodeMirror.


Remove old references to Liikennevirasto / Finnish Transport Agency

The Finnish Transport Agency is now the Finnish Transport Infrastructure Agency (in Finnish, Väylä)

Thanks @lapintom, good to hear from you!


tests: add check for bug 6149 - v1

Test for Bug #6149
Ticket
Redmine ticket:
https://redmine.openinfosecfoundation.org/issues/6149

Force-pushed tests for ips with auto, too, just in case.

Merged via https://github.com/OISF/suricata-verify/pull/1263. Thanks!


error: failed to receive status: rpc error: code = Unavailable desc = error reading from server: EOF When building docker

Rasa Open Source version
ToT
Rasa SDK version
No response
Python version
3.7
What operating system are you using?
Linux
What happened?
While trying out rasa with docker from repo's ToT. When I am trying to build the image I am getting rpc error: code = Unavailable desc.
Steps to reproduce

clone rasa github repo
run command to build docker

$ make build-docker
export IMAGE_NAME=rasa && \
docker buildx use default && \
docker buildx bake -f docker/docker-bake.hcl base && \
docker buildx bake -f docker/docker-bake.hcl base-poetry && \
docker buildx bake -f docker/docker-bake.hcl base-builder && \
docker buildx bake -f docker/docker-bake.hcl default
[+] Building 2.1s (9/10)
 => [internal] load .dockerignore                                                                                0.0s
 => => transferring context: 139B                                                                                0.0s
 => [internal] load build definition from Dockerfile.base                                                                   0.0s
 => => transferring dockerfile: 792B                                                                                0.0s
 => [internal] load metadata for docker.io/library/ubuntu:20.04                                                             2.0s
 => [auth] library/ubuntu:pull token for registry-1.docker.io                                                               0.0s
 => [1/5] FROM docker.io/library/ubuntu:20.04@sha256:fd92c36d3cb9b1d027c4d2a72c6bf0125da82425fc2ca37c414d4f010180dc19       0.0s
 => CACHED [2/5] RUN apt-get update -qq &&   apt-get install -y --no-install-recommends   python3   python3-venv   python3  0.0s
 => CACHED [3/5] RUN apt-get update && apt-get dist-upgrade -y --no-install-recommends                                      0.0s
 => CACHED [4/5] RUN update-alternatives --install /usr/bin/python python /usr/bin/python3 100    && update-alternatives -  0.0s
 => CACHED [5/5] RUN useradd -rm -d /app -s /sbin/nologin -g root -u 1001 rasa && groupadd -g 1001 rasa                     0.0s
 => preparing layers for inline cache                                                                                0.0s
error: failed to receive status: rpc error: code = Unavailable desc = error reading from server: EOF
Makefile:253: recipe for target 'build-docker' failed
make: *** [build-docker] Error 1

My Docker details
$ docker version
Client: Docker Engine - Community
 Version:           20.10.8
 API version:       1.41
 Go version:        go1.16.6
 Git commit:        3967b7d
 Built:             Fri Jul 30 19:54:08 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          22.06.0-beta.0
  API version:      1.42 (minimum version 1.12)
  Go version:       go1.18.3
  Git commit:       38633e7
  Built:            Fri Jun  3 17:55:46 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.6
  GitCommit:        10c12954828e7c7c9b6e0ea9b0c02b01407d3ae1
 runc:
  Version:          1.1.2
  GitCommit:        v1.1.2-0-ga916309
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Platform
Linux xyz-ubuntu 5.4.0-117-generic #132~18.04.1-Ubuntu SMP Thu Jun x 2x:3x:48 UTC 202x x86_64 x86_64 x86_64 GNU/Linux

Command / Request
No response
Relevant log output
No response

Any update on this. I face same issue when image size is big.

same here....


Matrix3_TEST.py and  StopWatch_TEST.py tests fail on Windows with conda-forge dependencies

Environment

OS Version: Windows 10
Source or binary build? Source, https://github.com/ignitionrobotics/ign-math/commit/ceab8c7f32f43c3ebf576146e88677ada17edf23, 6.10.0 release

This problem was detected while enabling Python support in the libignition-math6 package on conda-forge, see https://github.com/conda-forge/libignition-math4-feedstock/pull/25/ .

Matrix3_TEST.py:
2022-05-01T11:43:26.6555205Z 115/142 Test #115: Matrix3_TEST.py ........................***Failed    0.17 sec
2022-05-01T11:43:26.6570710Z ..........F..
2022-05-01T11:43:26.6571776Z ======================================================================
2022-05-01T11:43:26.6572650Z FAIL: test_to_quaternion (__main__.TestMatrix3)
2022-05-01T11:43:26.6573622Z ----------------------------------------------------------------------
2022-05-01T11:43:26.6574494Z Traceback (most recent call last):
2022-05-01T11:43:26.6575464Z   File "%SRC_DIR%\src\python_pybind11\test\Matrix3_TEST.py", line 310, in test_to_quaternion
2022-05-01T11:43:26.6576958Z     self.assertTrue(q == quatFromMat)
2022-05-01T11:43:26.6577872Z AssertionError: False is not true
2022-05-01T11:43:26.6578264Z
2022-05-01T11:43:26.6578931Z ----------------------------------------------------------------------
2022-05-01T11:43:26.6579667Z Ran 13 tests in 0.004s
2022-05-01T11:43:26.6580156Z
2022-05-01T11:43:26.6580693Z FAILED (failures=1)
2022-05-01T11:43:26.6581035Z

StopWatch_TEST.py:
2022-05-01T11:43:40.4890605Z 131/142 Test #131: StopWatch_TEST.py ......................***Failed   11.17 sec
2022-05-01T11:43:40.4892371Z .F.F
2022-05-01T11:43:40.4893342Z ======================================================================
2022-05-01T11:43:40.4894340Z FAIL: test_equal_operator (__main__.TestBox)
2022-05-01T11:43:40.4895380Z ----------------------------------------------------------------------
2022-05-01T11:43:40.4896347Z Traceback (most recent call last):
2022-05-01T11:43:40.4897542Z   File "%SRC_DIR%\src\python_pybind11\test\StopWatch_TEST.py", line 101, in test_equal_operator
2022-05-01T11:43:40.4898597Z     self.runTimer(watch2)
2022-05-01T11:43:40.4899537Z   File "%SRC_DIR%\src\python_pybind11\test\StopWatch_TEST.py", line 39, in runTimer
2022-05-01T11:43:40.4900831Z     self.assertGreaterEqual(_time.elapsed_run_time() + handleSteadyClock, timedelta(seconds=1))
2022-05-01T11:43:40.4902387Z AssertionError: datetime.timedelta(microseconds=998771) not greater than or equal to datetime.timedelta(seconds=1)
2022-05-01T11:43:40.4903290Z
2022-05-01T11:43:40.4904003Z ======================================================================
2022-05-01T11:43:40.4905073Z FAIL: test_start_stop_reset (__main__.TestBox)
2022-05-01T11:43:40.4906067Z ----------------------------------------------------------------------
2022-05-01T11:43:40.4907083Z Traceback (most recent call last):
2022-05-01T11:43:40.4908205Z   File "%SRC_DIR%\src\python_pybind11\test\StopWatch_TEST.py", line 125, in test_start_stop_reset
2022-05-01T11:43:40.4909302Z     self.runTimer(watch)
2022-05-01T11:43:40.4910333Z   File "%SRC_DIR%\src\python_pybind11\test\StopWatch_TEST.py", line 39, in runTimer
2022-05-01T11:43:40.4911701Z     self.assertGreaterEqual(_time.elapsed_run_time() + handleSteadyClock, timedelta(seconds=1))
2022-05-01T11:43:40.4913207Z AssertionError: datetime.timedelta(microseconds=999953) not greater than or equal to datetime.timedelta(seconds=1)
2022-05-01T11:43:40.4915090Z
2022-05-01T11:43:40.4916809Z ----------------------------------------------------------------------
2022-05-01T11:43:40.4917673Z Ran 4 tests in 11.013s
2022-05-01T11:43:40.4918078Z
2022-05-01T11:43:40.4918640Z FAILED (failures=2)
2022-05-01T11:43:40.4918985Z

Description

Expected behavior: The test suite works fine,
Actual behavior: The aforementioned test fail.

Steps to reproduce

Create a conda-forge environment like mamba create -n ignmathdev libignition-cmake2 eigen3 python pybind11 pybind11-abi vs2019_win-64 cmake ninja pkg-config
Compile ign-cmake and run tests
Check the output of the tests

Output
See log.

I stumbled back on this while adding tests in https://github.com/conda-forge/gz-math-feedstock/pull/11 .
The Matrix3_TEST on Windows seems to be a subtle bug in test_to_quaternion test in https://github.com/gazebosim/gz-math/blob/416e559320126d1efd0f0e768dd1a8c8dbdc2ca5/src/python_pybind11/test/Matrix3_TEST.py#L307 . Basically, the initial quaternion generated by Quaterniond(math.pi/2.0, math.pi/2.0, 0) is [x,y,z,w] = [0.5,0.5,-0.5,0.5], while the one created by Quaterniond(Matrix3d(q)) is (only on Windows, for some reason) [x,y,z,w] = [-0.5,-0.5,0.5,-0.5]. However, both answers are mathematically correct: the mapping between unit quaternions and rotation matrix is not bijective, but for each rotation matrix there are two corresponding unit quaternion, one the opposite of each other (this is what is usually called "Double-Cover", if you search Google for "unit quaternion double cover" there are many resources on this).
So the possible solutions are one the following:

Clarify in the documentation of Quaternion(Matrix3) constructor which unit quaternion is returned, and fix Windows to comply with the docs
Change the test to check if q == qFromMat or q == -qFromMat

Change the test to check if q == qFromMat or q == -qFromMat

I did this in https://github.com/gazebosim/gz-math/pull/561 .


The tcp_connect6_link_local test case failed in 1.49.2.

Version:  1.49.2
Platform: aarch64    openEuler 22.03 (LTS-SP3）
Hi. When I test the latest version 1.49.2, I encounter the same error as #4211 The error content is as follows. Can you provide some help?

2: not ok 313 - tcp_connect6_link_local
2: # exit code 134
2: # Output from process `tcp_connect6_link_local`:
2: # Assertion failed in /home/stage/root/spack-stage-libuv-1.49.2-aclfalclg5xrmbc33b4kxwtluzkbcgyn/spack-src/test/test-tcp-connect6-error.c on line 119: `uv_tcp_connect(&req, &server, (struct sockaddr*) &addr, connect_cb) == 0` (-99 == 0)

Error 99 is EADDRNOTAVAIL. Does your system have an IPv6 link local address?

Thank you very much. The test passed after ipv6 was disabled.


Add NewWebhookService func

Signed-off-by: jtcheng<EMAIL_ADDRESS>

merging to start using first


narrow [nfc]: Small tweaks to DmNarrow constructors

The list of these constructors has been growing. This tweaks a couple of aspects that I happened to notice when rereading these as part of merging #304 just now.

narrow [nfc]: Factor out DmNarrow.withOtherUsers; doc constructors more

narrow [nfc]: Strength-reduce implementation of DmNarrow.withUser
No need to allocate a Set for this, as it's always either
just one or two elements.  This version should be a bit more
efficient if we find ourselves constructing many of these.

Thanks, LGTM! Merged.


363: Remove unused CI jobs

Fixes #363
To test:

Check the CI on this isn't as broken

Still seems pretty broken

Yes, sorry, was still looking at it


K8SPG-559: fix .status.backupType

https://perconadev.atlassian.net/browse/K8SPG-559
DESCRIPTION
Problem:
The initial replica-create backup is incorrectly identified as incremental in the .status.backupType field.
Cause:
The backup type is currently determined by parsing the --flag option in the backup job's COMMAND_OPTS environment. If this flag is not found, the backup type defaults to incremental. However, the replica-create backup job does not include this flag.
Solution:
The operator should obtain the backup type from the pgbackrest info command.
CHECKLIST
Jira

[x] Is the Jira ticket created and referenced properly?
[x] Does the Jira ticket have the proper statuses for documentation (Needs Doc) and QA (Needs QA)?
[x] Does the Jira ticket link to the proper milestone (Fix Version field)?

Tests

[x] Is an E2E test/test case added for the new feature/change?
[x] Are unit tests added where appropriate?

Config/Logging/Testability

[x] Are all needed new/changed options added to default YAML files?
[x] Are the manifests (crd/bundle) regenerated if needed?
[x] Did we add proper logging messages for operator actions?
[x] Did we ensure compatibility with the previous version or cluster upgrade process?
[x] Does the change support oldest and newest supported PG version?
[x] Does the change support oldest and newest supported Kubernetes version?

Test name
Status

custom-extensions
passed

demand-backup
passed

init-deploy
passed

monitoring
passed

operator-self-healing
passed

scaling
passed

scheduled-backup
failure

self-healing
passed

start-from-backup
passed

tablespaces
passed

telemetry-transfer
passed

upgrade-minor
passed

users
passed

We run 13 out of 13

commit: https://github.com/percona/percona-postgresql-operator/pull/762/commits/2bed83480411b1820f987ba4c35ed93aac8bfcca
image: perconalab/percona-postgresql-operator:PR-762-2bed83480

Test name
Status

custom-extensions
passed

demand-backup
passed

init-deploy
passed

monitoring
passed

operator-self-healing
passed

scaling
passed

scheduled-backup
failure

self-healing
passed

start-from-backup
passed

tablespaces
passed

telemetry-transfer
passed

upgrade-minor
passed

users
passed

We run 13 out of 13

commit: https://github.com/percona/percona-postgresql-operator/pull/762/commits/98364baf4e3a34c668a055f34cafebb60483dc05
image: perconalab/percona-postgresql-operator:PR-762-98364baf4

Test name
Status

custom-extensions
passed

demand-backup
passed

init-deploy
passed

monitoring
passed

operator-self-healing
passed

scaling
passed

scheduled-backup
passed

self-healing
passed

start-from-backup
passed

tablespaces
passed

telemetry-transfer
passed

upgrade-minor
passed

users
passed

We run 13 out of 13

commit: https://github.com/percona/percona-postgresql-operator/pull/762/commits/4247e984fff9ba3b9ec44855a6f89e4d20a37d80
image: perconalab/percona-postgresql-operator:PR-762-4247e984f

Test name
Status

custom-extensions
passed

demand-backup
passed

init-deploy
passed

monitoring
passed

operator-self-healing
passed

scaling
passed

scheduled-backup
passed

self-healing
passed

start-from-backup
passed

tablespaces
passed

telemetry-transfer
passed

upgrade-minor
passed

users
passed

We run 13 out of 13

commit: https://github.com/percona/percona-postgresql-operator/pull/762/commits/3a52d276b933263d0e07f217632299f1c614e738
image: perconalab/percona-postgresql-operator:PR-762-3a52d276b

Test name
Status

custom-extensions
passed

demand-backup
passed

init-deploy
passed

monitoring
passed

operator-self-healing
failure

scaling
passed

scheduled-backup
passed

self-healing
passed

start-from-backup
passed

tablespaces
passed

telemetry-transfer
passed

upgrade-minor
passed

users
passed

We run 13 out of 13

commit: https://github.com/percona/percona-postgresql-operator/pull/762/commits/d738f5c5291b38b9b44da1a4222b27b55646c7c6
image: perconalab/percona-postgresql-operator:PR-762-d738f5c52

Test name
Status

custom-extensions
passed

demand-backup
passed

init-deploy
passed

monitoring
passed

operator-self-healing
passed

scaling
passed

scheduled-backup
passed

self-healing
passed

start-from-backup
passed

tablespaces
passed

telemetry-transfer
passed

upgrade-minor
passed

users
passed

We run 13 out of 13

commit: https://github.com/percona/percona-postgresql-operator/pull/762/commits/d738f5c5291b38b9b44da1a4222b27b55646c7c6
image: perconalab/percona-postgresql-operator:PR-762-d738f5c52


Add IE Banner

Sets up static content for #4579
We'll want to ensure some of the discussion in the issue is addressed. Ping me if there are questions!

@tristan-orourke We will need translations for Internet Explorer Banner and Dialog Language Lines.

@tristan-orourke We will need translations for Internet Explorer Banner and Dialog Language Lines.

I moved the script include, but its still not working. The banner does appear in IE, and not other browsers, but the "Learn more about this decision" button doesn't do anything, and the "Dismiss" button throws a console error: Object doesn't support property or method 'remove'

This works for me with a previous version of Hydrogen (v0.0.32). @bitliege: Is there anything that has changed with the Dialog component since v0.0.32?
When I switch out the version in the twig template file from the following, it works again:
https://cdn.jsdelivr.net/npm/<EMAIL_ADDRESS>to
https://cdn.jsdelivr.net/npm/<EMAIL_ADDRESS>For the route posted to on Dismiss, you may need to clear the route cache since /api/v1/browser-message is a new route: php artisan route:clear (macOS, not sure on Windows how to do this).

I moved the script include, but its still not working. The banner does appear in IE, and not other browsers, but the "Learn more about this decision" button doesn't do anything, and the "Dismiss" button throws a console error: Object doesn't support property or method 'remove'

This works for me with a previous version of Hydrogen (v0.0.32). @bitliege: Is there anything that has changed with the Dialog component since v0.0.32?
When I switch out the version in the twig template file from the following, it works again:
https://cdn.jsdelivr.net/npm/<EMAIL_ADDRESS>to
https://cdn.jsdelivr.net/npm/<EMAIL_ADDRESS>For the route posted to on Dismiss, you may need to clear the route cache since /api/v1/browser-message is a new route: php artisan route:clear (macOS, not sure on Windows how to do this).

Okay, with #e89645899ffbcda1934ad4587aeda2b7ec5b4fa9 the dismiss button now works.
I was going to suggest that we set the cookie directly in the browserMessage.js file, to avoid an extra request and fairly pointless api endpoint, but then I realized Laravel does some nice encryption for us that we may not want to skip.

Okay, with #e89645899ffbcda1934ad4587aeda2b7ec5b4fa9 the dismiss button now works.
I was going to suggest that we set the cookie directly in the browserMessage.js file, to avoid an extra request and fairly pointless api endpoint, but then I realized Laravel does some nice encryption for us that we may not want to skip.

I hate to suggest this now after so much work... but I now feel like we should just make the banner non-dismissible. We really don't expect the site to work in IE at this point. Thoughts? @mnigh @bitliege

I hate to suggest this now after so much work... but I now feel like we should just make the banner non-dismissible. We really don't expect the site to work in IE at this point. Thoughts? @mnigh @bitliege

I agree. The purpose of the banner is to alert the user that the site that they are currently visiting will not work using the browser they are currently using: dismissing the banner won't change that fact and they should not continue using the site.

Resolves #4579.


Action Refresh table fails when used on custom buttons

When action: Refresh table is used on the action configuration while the buttonType is custom, upon executing the action, an error gets thrown Validation failed: Action 'Refresh table' in the owner '_9Ya0XzWdoBAQpqjDkrH-S7pwUIPm5' not found.
Steps to reproduce the behavior:

Go to 'https://function-adminportal-test.shesha.dev/dynamic/TestModule/afterUpdates-details?id=828afbd7-6547-46d6-a508-0cd13df4b34c'
The above links has two buttons: Custom and Default. The custom one is the one throwing an error and the default one refresh accordingly
Click on 'Custom' button located inside panel2
Open developer console
Click on 'Custom' link on the dialog
Check the error getting displayed afterwards
Actual Results: Validation failed: Action 'Refresh table' in the owner '_9Ya0XzWdoBAQpqjDkrH-S7pwUIPm5' not found.`

Expected behavior
The action should be executed successfully, just how it is when using a default buttonTypes

Hi @hlayisanimazuze. It's fixed, please check

This will be included on release 0.37


[AGE Packaging] Bitrock windows installer XML script

This is a combined PR of @sarthak-19 and @AhmarZaidi  for making install builder XML script including pre, build, and post build actions.

looks good, ready to be merged.


CLOUD-932 always work with latest rc tag when scope is rc

@akanto please take a look
Instead of the latest RC tag, the latest pre release tag was used in most cases. But dev tags are also pre-release tags, and that caused some complications if dev tags appeared on the rc branch when rc patches were merged back to master.
The 1.1.0-rc.1 tag should be removed from the remote repo before merging this PR, because it will cause failures when the real 1.1.0-rc.1 should be released later

Jenkins build finished, all tests passed.
Refer to this link for build results: http://ci.sequenceiq.com/job/cloudbreak-pull-request/1645/


Resource bundles required to copy to the main app

As SumUpSDK has not supported Cocapods yet, we had to make a local pod as a workaround. However, SumUpSDK requires its bundles to be copied to the main app and that's not a good idea for distributing a framework which should be self-contained. Can you please use [NSBundle bundleForClass:] so those bundles can live inside SumupSDK? More here https://developer.apple.com/reference/foundation/nsbundle/1417717-bundleforclass?language=objc
Thanks a lot.

The SumUp SDK is currently a static library + headers + resources. In case of the manual integration, the static library is compiled into your application and will not be a separate bundle at runtime, which is why the shared resources have to be bundled with your app as well.
We are currently working on official CocoaPods support. By default, CocoaPods builds a dynamic framework of all your pods, so the SumUp resources will live inside the pods framework, as does SumUp's code. To bundle the shared resources with the pods framework, the podspec file needs this:
s.resources = ["SumupSDK.embeddedframework/Resources/SMPSharedResources.bundle"]

I am not yet sure (but we will try it out before releasing our pod) what this line does when you disable dynamic frameworks for CocoaPods, but I would assume CocoaPods then adds the resources to your app directly, as you would do manually.
Unfortunately, SumupSDK.embeddedframework/Resources/YTLibResources.bundle still needs to be bundled with your app directly, as third party code we cannot change requires those resources to be in the main bundle. We are investigating better solutions as part of our CocoaPods integration.
As we still support iOS 6, we cannot support dynamic frameworks (without making it even more complicated by supplying two variants).

Hi @shagedorn Thanks for the reply. We dont use SumupSDK directly from our main app, we wrap SumupSDK inside a dynamic framework for our main app to use, so podspec.resources won't help as it puts the bundle into our framework, not the main app. Anyway, it's great to know SumupSDK will be supported via cocoapods. That'd be helpful.  Thanks a lot.

For the shared resources bundle, that is ok: It doesn't (or shouldn't) live in the main app if the SDK does not. So podspec.resources should do the right thing.
The only issue we currently see and are trying to solve are the YTLibResources, because they do need to be bundled with the main app directly. We will update this issue as we go.

As of v3.0, we now officially support CocoaPods, and the above-mentioned YTLibResources bundle is not part of the SDK anymore. I assume this issue will be solved by those steps. Please re-open if it's still an issue.


eliminate the use of Mirror

This branch is not ready for merging. I'm creating this pull request only to make it available for discussion.
I found example uses of swift_getTypeByMangledNameInContext in the Reflect package and have made some progress toward using it to replace the use of Mirror.
In its current state, this branch has several failing tests, all related to associated value labels. I've outlined what's wrong and what needs to be done in the commit message and code comments.

The new version (7cab867) fixes the existential bug and passes all the tests (including some new ones).
Code coverage of EnumReflection.swift indicates the tests cover everything except an “impossible” case where some metadata is missing, and a mangled type name encoding that I don't know how to trigger.
It also runs faster.
Main branch (90e4e67e):
name               time       std        iterations
---------------------------------------------------
Success.Manual      34.000 ns ± 146.77 %    1000000
Success.Reflection 151.000 ns ± 121.01 %    1000000
Failure.Manual      35.000 ns ± 231.80 %    1000000
Failure.Reflection  65.000 ns ± 191.00 %    1000000

New version:
name               time       std        iterations
---------------------------------------------------
Success.Manual      33.000 ns ± 226.73 %    1000000
Success.Reflection 128.000 ns ±  85.30 %    1000000
Failure.Manual      37.000 ns ± 149.48 %    1000000
Failure.Reflection  73.000 ns ± 140.36 %    1000000

Surprisingly, unwrapping just the cached tag first to reject non-matching values (instead of unwrapping the whole cached extractor) consistently saves a few seconds in the Failure.Reflection benchmark.
name               time       std        iterations
---------------------------------------------------
Success.Manual      34.000 ns ± 261.66 %    1000000
Success.Reflection 135.000 ns ± 116.75 %    1000000
Failure.Manual      36.000 ns ± 221.68 %    1000000
Failure.Reflection  66.000 ns ± 174.90 %    1000000

I think I'm done dinking this.
I've tried a few variations to knock nanoseconds off the Failure.Reflection benchmark without success, like storing -1 for cachedTag instead of using an optional, but was unable to make it consistently faster. I suspect it's consistently slower than the main branch by ~3 nanos because the closure captures two words (the cached tag and the strategy) instead of just one (the cached tag).
I thought the whole private helpers section of the file was kind of jumbled so I reorganized it. The code is all the same, but hopefully it's in a more logical order now so it should be easier to read from top to bottom.

Really awesome work @mayoff! Thanks heaps for the informative comments and commits :) it's looking pretty close to optimal.


Reorganize and document where messages come from

Scorex's code organization and documentation would be greatly improved if we were more systematic about where we define messages and how we import them to be used elsewhere and about documenting from/to where they go.
I propose that we follow the style used in these actors: https://gitlab.com/AxiomFlow/OuroborosPraos/tree/master/src/main/scala/ouroboros/actor (GitLab Login Required)

@EzequielPostan , since you are already working on #162 , could you work on this one too?

It looks interesting. If I understand correctly, you would like to define on each actor companion's object a ReceivableMessages object that lists the messages that can be sent to the actor.
My concern here is about keeping an updated documentation (the comments that state which actors send the messages). Outdated documentation tends to be better than no documentation, but do you have any comments on this point @ceilican?
I searched for suggested documentation patterns for actor messages but found nothing interesting. Maybe we can ask on some irc channel for ideas.

It looks interesting. If I understand correctly, you would like to define on each actor companion's object a ReceivableMessages object that lists the messages that can be sent to the actor.

Yes. And also:

if an actor A sends messages M1, M2,... Mn to an actor B, we should have the following import statement in A:  import B.ReceivableMessages.{M1,M2,...Mn}.  (We should not use import B.ReceivableMessages._).

The point of this policy is to be able to see at a glance which messages A sends to B just by looking at the beginning of A's code, without having to skim hundreds of lines of code.

My concern here is about keeping an updated documentation (the comments that state which actors send the messages). Outdated documentation tends to be better than no documentation, but do you have any comments on this point @ceilican?

That's a good point. It is a very reasonable concern. Again, the rationale for this idea was to allow a reader of the code to quickly know, just by looking at a single line in B's code, which actors may send that particular message, without having to search throughout the code. This is not so important for a reader that uses a good IDE that can do a "find usages" search for him/her, though.  Therefore, I suggest that we address your concern by simply not adding those comments.
Another way of looking at this issue is that, if the documentation becomes outdated, then this is a sign that an actor C is now sending a message M to B that was originally intended to be sent from A only. This could be a bug in the code. And we should ensure that, when this happens, the code is thoroughly reviewed, to ensure that the semantics of sending M from C to B is the same as the semantics of sending it from A to B. Nevertheless, a better solution for this problem would be to have, instead of code comments that can become outdated without being noticed, actorRef "types" and then check (with a require statement), in the receive method of B that message M was sent by an actorRef of type A.actorRefType. Anyway, let's not pursue this "actorRefType" idea for now.

Yes. And also:

if an actor A sends messages M1, M2,... Mn to an actor B, we should have the following import statement in A: import B.ReceivableMessages.{M1,M2,...Mn}. (We should not use import B.ReceivableMessages._).

The point of this policy is to be able to see at a glance which messages A sends to B just by looking at the beginning of A's code, without having to skim hundreds of lines of code.

I didn't notice the above part. Thanks for the clarification.
The suggestions look good to me. We can avoid the comments in the code as mentioned and think about the most appropriate check for the correctness of the semantics of the actor system as a whole.
I will try to work on this ticket during the week.

I finally get some time for this.
I have a question about something I noticed, do we have any naming convention for actors? For example, to end their names with the word Actor (I know there may be better naming patterns). I am asking this because they have been hard to search/find. Sometimes they don't extend directly from the Actor trait.

@ceilican
I am applying this change and I can notice one thing about how to import the receivable messages.
A class can import the messages for two reasons.

To send a message
To receive a mesage

I think that using the import statement with the list of messages is good to find what messages are received by an actor. This means that we can add the convention of the import clause for case 2 described above.
However, for case 1 we may find statements in the code like actor ! message and the sender can never receive Message, making the import convention ambiguous. Maybe for these cases we could write actor ! SourceActor.ReceivableMessages.Message and avoid the import of the list of messages. This would avoid mixing the easy to find convention you propose.
This would add a convention we should keep along the evolution of the code, so I would like to know your comments on whether you think this distinction I mention useful at all.

@EzequielPostan , I didn't really understand the issue you described. Could you show a more concrete example illustrating the issue?

Or, perhaps more efficiently, submit a "work-in-progress" PR, showing your proposal in one case only. Then we can discuss that case, and decide whether to do it everywhere.

Thanks for the reply @ceilican
Let me give an example of each situation:
There are actors like NodeViewHolder that receive messages that are defined by other Actors,
https://github.com/ScorexFoundation/Scorex/blob/master/src/main/scala/scorex/core/NodeViewHolder.scala#L389
For example, it receives the ComparesViews message
https://github.com/ScorexFoundation/Scorex/blob/master/src/main/scala/scorex/core/NodeViewHolder.scala#L321
defined by NodeViewSynchronizer
https://github.com/ScorexFoundation/Scorex/blob/master/src/main/scala/scorex/core/network/NodeViewSynchronizer.scala#L332
We can add the import clause
import scorex.core.network.NodeViewSynchronizer.ReceivableMessages.CompareViews
I can then interpret the import clause as "if we read this import clause in a file, then the messages listed after ReceivableMessages are the ones received by the actor defined in the current file"
Now, if we go to another file like HLocalInterface.scala we can see that it sends a message StopForging
https://github.com/ScorexFoundation/Scorex/blob/master/examples/src/main/scala/examples/hybrid/HLocalInterface.scala#L44
defined in the PosForger receivable messages object.
In this case we could import as before
import packageName.PosForger.ReceivableMessages.StopForging
but in this case, HLocalInterface never actually receives the message StopForging, it only sends it.
Hence, reading the import clause wouldn't mean "the actor defined in this file receives this messages".
What I mentioned that could be done is for this last situation, is to avoid the import clause and use more explicit reference by adding the name of the receiver actor as follows
https://github.com/ScorexFoundation/Scorex/pull/180/files#diff-c49c91d2ae49bf5c7d1534b5775b3c81L44
Maybe I am misunderstanding the idea behind this issue.
Again, thanks for reviewing this in advance.

This is the WIP PR just in case: https://github.com/ScorexFoundation/Scorex/pull/180
The second commit shows the situation with the PosForger actor
I should review the case of the first commit (as I didn't notice this ambiguity at that point)

Ahh... You are misreading the "import" statements.
If you have:
class ActorA extends Actor {
  import ActorA.receivableMessages._
  import ActorB.receivableMessages.{Message1, Message2}

  ...
}

You should it read it as: "ActorA receives all its receivables messages and sends Message1 and Message2, which will be received by ActorB".

There are actors like NodeViewHolder that receive messages that are defined by other Actors.
For example, it receives the ComparesViews message defined by NodeViewSynchronizer.

That's another issue.  According to my proposal, the messages received by an ActorA should always be defined inside ActorA.ReceivableMessages. So, in the case above, CompareViews should be moved to NodeViewHolder.ReceivableMessages.
If some messages are receivable by more than one actor, we can put them in a separate trait and mix the trait into the ReceivableMessages objects of each actor.

In particular, if we have:
class ActorA extends Actor {
  import ActorA.receivableMessages._
  import ActorB.receivableMessages.{Message1, Message2}

  ...
}

the second import statement does not mean that Message1 and Message2 are received by ActorA.  They are, as their location implies, receivable by ActorB. The import just tells us that ActorA is going to use these messages (i.e. to send them to B).

Is my proposal clearer now? Is it clear that there is no ambiguity?

We may need to move some message classes (e.g. CompareViews) from one file to another, in order to adhere to this unambiguous naming/organization proposal.
I think that what made you confused about my proposal was the fact the location of Scorex's message classes is not adhering to this proposal yet.

Thanks for the clarifications @ceilican
Now I understand this better. I will review what I did and work it in this way.
As you mentioned, I will have to move messages from some objects to others and/or create common traits to locate them.
I guess Scorex's message classes not adhering to this proposal created my confusion as you mentioned.
Thanks again for your review and time.

@ceilican
Thanks for the review in the PR.
I am getting some failures in my tests (under examples project).
I have to investigate the cause. It may be related to the traits with shared messages (but I am not sure yet). I have the impression that if I have
trait A { case class X(i: Int)}
object B extends A
object C extends A

and if I send
import B.X
actor ! X(1)

then the code
import C.X
def receive: Receive = {
  case x: X =>
  //or
  case X(n) =>
}

will not receive the message as it will interpret B.X and C.X as different types. Intellij is telling me that I always have A.X but I am starting to distrust that.
I will try to check this during the weekend.
On a related topic, I have a few other comments:
There is a case class Message that is received by a few actors (e.g. NetworkController and PeerConnectionHandler). The class is this one
https://github.com/ScorexFoundation/Scorex/blob/master/src/main/scala/scorex/core/network/message/Message.scala#L10
Should I relocate it?
There is also a class PowBlock located at
https://github.com/ScorexFoundation/Scorex/blob/master/examples/src/main/scala/examples/hybrid/blocks/PowBlock.scala#L72
that is received by PowMiner and used in many places. The question again is, should I relocate the class?
One other thing I didn't document were messages from akka.io.Tcp
On a final note, there are messages with repeated names (in different scopes of course). For example, object BlackList in PeerConnectionHandler and a case class with the same name in NetworkController. It may be a good practice to avoid this kind of name repetitions.
Once again, thank you for your time.

I am getting some failures in my tests (under examples project).
I have to investigate the cause. It may be related to the traits with shared messages (but I am not sure yet).

I changed a trait that contained shared messages for an object and tests got fixed. So I guess that we can't factor out shared messages in traits.
In summary, after we decide what to do with the classes Message and PowBlock, I can change the traits I created for objects (unless you have another suggestion) and close this issue. Any name suggestions for the objects that allocate the shared messages is welcome (current traits names are: NetworkControllerSharedMessages, NodeViewHolderSharedMessages and LocallyGeneratedModifiersMessages), should we have a ReceivableMessages object inside these new objects that I propose to create? (note that these objects in a way violate the format we want to enforce if I understand correctly)

Interesting.
Yes, the format I suggested assumes that each message class is receivable by only one actor type. My suggestion to have a trait for shared messages was a temporary hack for cases where a message is receivable by more than one actor.
The question is: should we have messages that are receivable by more than one actor at all?  I think this makes the code unnecessarily more confusing. I would suggest to avoid shared messages by doing the following:
If we have a shared message class SomeMessage that is receivable by actors ActorA and ActorB, we could duplicate it into SomeMessageToA and SomeMessageToB and put them inside the respective ReceivableMessages objects of ActorA and ActorB.  On a case-by-case basis, we could give better names to these new message types.
From one point of view, it may be considered inelegant to duplicate code as I am suggesting above. However, within the conceptual framework that I am proposing, the messages SomeMessageToA and SomeMessageToB are actually distinct messages (because they are received by different actors and hence have a different meaning), and it is a mistake to conflate them into a single SomeMessage type as the current code does.

I suspect (maybe you could confirm this to me, @EzequielPostan) that such shared messages occur because of the following communication pattern:
ActorC sends SomeMessage to ActorA and it propagates the message to ActorB.
In other words, one of the actors that receives the message is just acting as a middleman. In such cases, maybe it would make sense to eliminate the middleman, and send the message directly from C to B.  But I guess this should be analyzed on a case-by-case basis.

To ensure that we move on quickly, I propose that we accept the PR with SharedMessages objects (as described by @EzequielPostan).
@EzequielPostan , could you push the commits where you changed the traits to objects? Please add comments saying who is sharing the messages in those objects.
@kushti, could you accept today (thursday) the PR after Ezequiel's commits?
Then tomorrow (friday), I will have a closer look on the shared messages and see what to do with them on a case-by-case basis.

I saw cases where a message is basically forwarded as you mention @ceilican . For example here: https://github.com/ScorexFoundation/Scorex/blob/master/src/main/scala/scorex/core/LocalInterface.scala#L95
But I would need to make a deeper review to check if all cases are like this.
I can refactor traits for objects and add the comments before going to bed, please wait for my confirmation before merging.

@kushti, what do you think about "killing" the middlemen (as discussed above), to simplify the architecture?

Just to confirm, I updated the PR to use objects instead of traits.
Thanks for the review @ceilican and let me know if I can help on something else.
I will move to a new issue now.


Don't let nil fall through dumpers for MSSQL

I'm having CI test out some changes to Ecto that I made and that are living on my Ecto fork.

:green_heart: :blue_heart: :purple_heart: :yellow_heart: :heart:


press Enter from Search Results page

If you are on the search results page, you cannot hit ENTER key when typing something in the global search bar. @andrejhristovski

This is fixed and pushed to dev, not visible on dev  due to problems with NPM and builds are failing.

pushed to prod. fixed


Update Candy Collection Cog for Persistent Storage

PR #273 introduced persistent storage for our cogs, allowing for things like leaderboards to remain in between bot restarts. The candy collection cog needs to be updated to utilize this functionality.
The utility code itself can be found here: https://github.com/python-discord/seasonalbot/blob/master/bot/utils/persist.py
With Hacktoberstats as an example:
https://github.com/python-discord/seasonalbot/blob/526dbc465c2d458c893c72b614282937d15190d7/bot/seasons/halloween/hacktoberstats.py#L24-L26

@sco1 Can I work on this? it seems we need to add a new class member like link_json to replace the json_location being used in this code https://github.com/python-discord/seasonalbot/blob/master/bot/seasons/halloween/candy_collection.py#L15 and use the utility for persistent storage. Let me know if my understanding is correct.

Sounds about right.
Are you a member of the discord server? If so, can you please let us know your in-server username. Thanks.

@scragly I have not signed up in discord till now , I will send you my username once done.

@Niladri24dutta Sure, just keep in mind it's a requirement for any of our community repositories (seasonalbot, bot and site), so this will have to be kept available for others until that time. Hope to see you in the server sometime soon!

@scragly  thanks in the mean time can you please assign this issue to me?

@Niladri24dutta Sure, just keep in mind it's a requirement for our seasonalbot repository, so this will have to be kept available for others until that time. Hope to see you in the server sometime soon!

I have joined the discord server and posted in the channel #event-hacktoberfest-2019. My username is nil2441139

@Niladri24dutta any luck?

Hi all, I found this looking for a 'good first issue' (this is the only one).
After some research, I found out that maybe this can be closed because of https://github.com/python-discord/sir-lancebot/commit/5e249bc2e0a944d0683ba35a2e1ee459d60668ce ?

Yeah, this no longer seems to be an issue. I'm sorry there wasn't an issue marked to work on, but if you are still interested, drop into the dev-contrib channel on the discord server, and we could look for issues you can work on 😄.


Python Pulsar Functions Debugging

Is your feature request related to a problem? Please describe.
It would be great to have out of the box Pulsar Debugging available for Python Functions

I have some more context I can add to this, as a TODO for myself I'll come back and fill out with a bit more depth latter


update layers console api

🚀 Deployed on https://65d6bce3b1e8ef4591808d86--cozy-fox-0defec.netlify.app


P256Verify setup  for testnet deployment

P256Verify
DISCLAIMER

To be able to compile the precompile to deploy with a random address, you need to have both zksolc and solc compiler binaries in the /usr/local/bin/ directory.
From now on, "verifier" refers to the precompile contract, and "validator" refers to the contract that calls the precompile.

Setup
make setup

Deploying the verifier on the in-memory node

Run the in-memory node:make run-node

Add a rich wallet private key in .env file.
Deploy the verifier on the in-memory node:make deploy-verifier-local

Deploying the verifier on testnet

Add a funded wallet wallet private key in .env file.
Deploy the verifier on the in-memory node:make deploy-verifier-testnet

Deploying the validator on the in-memory node

Run the in-memory node:make run-node

Add a rich wallet private key in .env file.
Deploy the verifier on testnet.make deploy-verifier-local

Hardcode its address in contracts/Validator.sol's P256_VERIFIER constant.
Deploy the validator on the in-memory node:make deploy-validator-local

Deploying the validator on the testnet

Add a funded wallet wallet private key in .env file.
Deploy the verifier on testnet.make deploy-verifier-testnet

Hardcode its address in contracts/Validator.sol's P256_VERIFIER constant.
Deploy the validator on testnet:make deploy-validator-testnet

Found some issues while testing out the validator. The system call for the precompile isn't successful when tested in the testnet. The system contract tries to do a normal call and it panics!
Call(Normal) 0xce89201ca3e036a324bbe7b4ed509430a6ac39cf           executeTransaction(bytes32, bytes32, tuple)   153342
 Call(Normal) 0xbd708e500b8f78d6f4e965cc515c40cd7b63d9c6                 0x9074726b   149436
 Call(Normal) 0x0000000000000000000000000000000000000002                 0x03e17259   146097
 Call(Normal) 0x000000000000000000000000000000000000fff5           0x  Error: Panic 145152

This mostly means, the precompile contract is not identified or not a part of the constants in the system contract thus when "precompileCall" opcode hits it panics.
Testnet Test :
Validator address : 0xbD708e500b8f78D6F4E965CC515C40cd7b63D9c6
Transaction hash : 0x0ad02ca386c5d9ff55c90e8bc6ea439f8b60b8b084ab8dd220c7fb2e8b366b7d
Please replay the transaction in-memory node by forking testnet to take a look at the call panics
i.e worst case even if the tested p256 signature is wrong, the call should be successful and should return 0x00/0.

Found some issues while testing out the validator. The system call for the precompile isn't successful when tested in the testnet. The system contract tries to do a normal call and it panics!
Call(Normal) 0xce89201ca3e036a324bbe7b4ed509430a6ac39cf           executeTransaction(bytes32, bytes32, tuple)   153342
 Call(Normal) 0xbd708e500b8f78d6f4e965cc515c40cd7b63d9c6                 0x9074726b   149436
 Call(Normal) 0x0000000000000000000000000000000000000002                 0x03e17259   146097
 Call(Normal) 0x000000000000000000000000000000000000fff5           0x  Error: Panic 145152

This mostly means, the precompile contract is not identified or not a part of the constants in the system contract thus when "precompileCall" opcode hits it panics.
Testnet Test : Validator address : 0xbD708e500b8f78D6F4E965CC515C40cd7b63D9c6 Transaction hash : 0x0ad02ca386c5d9ff55c90e8bc6ea439f8b60b8b084ab8dd220c7fb2e8b366b7d Please replay the transaction in-memory node by forking testnet to take a look at the call panics i.e worst case even if the tested p256 signature is wrong, the call should be successful and should return 0x00/0.

This is happening becase 0xfff5 (SystemContractCaller system contract) is not deployed in the era-test-node at the moment.


Is there a Python binding to libpointmatcher

I was wondering if there is a Python binding to libpointmatcher?

Clearly not, but I would highly appreciate it ;)

We starting to work on Python binding. If someone has some inputs, advices or needs. Now is the time to tell us!

There was the start of an implementation of python bindings. It supported registration, add numpy array as pointcloud input and support extracting the debugging information that are normally in the .vtk output. However, it needs to be clean up, there is a bunch of code that was made specifically for debugging penalized ICP. https://github.com/norlab-ulaval/test_penalized_icp

@PhiBabin How did you attempt the Python binding in your branch?

@YoshuaNava Thanks for the informations! I'll take a look at them! 🙂
@PhiBabin I don't have access to the repo you've mentionned. 🙁

@YoshuaNava  I used boost.python.
@AlexandreG87 Oups, my bad, it is now public.

Alright! So, after researching about all the Python bindings tools available, here are the ones I consider to have good potential to do the job. These solutions have been selected base on the following criteria :
Primary criteria:

Performance
Modern C++ and Python3 compatibility
Effective Numpy to Eigen conversion support (not 100% sure for cppyy)
Template compatibility
The ease of use and maintenance
The solution must still be actively maintained
No or minimal modification to the original code must be done

Secondary criteria:

Availability/Quality of documentation and tutorials
Control and flexibility over interface implementation
Does not require too much dependency

The fully automatic solutions :

Automatic Wrapper and Interface Generator (AutoWIG) :

Article on AutoWIG
YouTube video explaning how AutoWIG works
GitHub repo

cppyy :

Paper on cppyy
Bitbucket repo
Tutorial
Stack Overflow answer by the author

And the semi-automatic solutions :

Pybind11 :

GitHub repo
JaggedVerge tutorial
Real Python tutorial
YouTube playlist tutorial
Integrate Python and C++ with pybind11 - Robert Smallshire

Boost.Python :

GitHub repo
Official tutorial
Developpez.com tutorial (French tutorial)
Integrating Python and C++ with Boost Python part 1 | SciPy 2014 | Austin Bingham
Integrating Python and C++ with Boost Python part 2 | SciPy 2014 | Austin Bingham
Integrating Python and C++ with Boost Python part 3 | SciPy 2014 | Austin Bingham
Integrating Python and C++ with Boost Python part 4 | SciPy 2014 | Austin Bingham

Cython

Cython docs :
Cython GitHub repo
Cython Wiki
Real Python tutorial
Paper (IEEE Xplore access required)

List of projects and examples using these tools :

AutoWIG :

StatisKit
FP17

cppyy :

Example repos from the docs

Pybind11 :

Tensorflow (Use Eigen too)
Pytorch (Use Eigen too)
NumpyEigen
Open3D (Use Eigen too)
CMake examples
Python examples
Other examples
pclpy

Boost.Python :

EigenPy
MiniEigen
Examples

Cython :

Eigency
SageMath
Numpy
python-pcl

Some more stuff on the subject :

Real Python bindings tutorial
Blog - Which tool should you use? by Stephan Behnel (Cython core developper) (Not just about Cython!)
Examples for different Python bindings tools
Calling C/C++ from Python? - Stack Overflow

In short...
Well, all of the above tools have their strenghs and weaknesses, none of them are perfect. But, I think it's worth trying the fully automatic solutions first, since they are designed with the ease of use and maintenance in mind. On the other hand, AutoWIG generate Pybind11 or Boost.Python bindings, both of whom have extensive of tutorials, documentations and examples available. So, in the event AutoWIG doesn't  All of these tools are focused on performance, so I'm not worried about that too much. The main struggle being the Python call overhead, except for the PyPy interpreter. cppyy, Pybind11 and Cython are compatible with the PyPy interpreter, although cppy seems to get a bit more out of it than the other ones.
Finally
Every suggestions, comments or advices are welcome. Feel free to suggest anything that could make this project even better, the goal is to be able to provide good bindings for libpointmatcher. Thanks!

Thanks for the thorough analysis.  I guess we can start with AutoWIG.
@simonpierredeschenes any concerns?

Just adding a couple of remarks; feel free to ignore. :) Full disclosure: I'm the author of cppyy, and saw some of my documents referenced above, which is how I landed here.
First: "all of the above tools have their strenghs and weaknesses". This is an extremely important point as you have "Performance" on the top of your list. Performance in CPython (and PyPy even more so), from an automatic binder, depends on specializing the paths that you care about. If that is already happening out-of-the-box, then you're golden. If not, you're in for some work. I'm afraid that testing and profiling is the only way to find out.
On that point, yes, automatic binders do help a lot with maintenance, but even though they get you 99% of the way, that last bit that they lack is also the hardest part. I.e. compared to a full manual solution, that 1% would be more like 20-30% of the manual effort. Still a gain for sure, but you may be forced to learn more about the tool than you hoped for.
For Eigen/NumPy: note that Eigen's layout may differ depending on compiler settings (e.g. if you enable vector instructions), so simply copies may be the only portable way in some cases. As for cppyy, one problem with Eigen is that template expressions are intended to be squashed by the C++ compiler. This does not naturally happen in cppyy in the way that wrappers are generated, so sometimes you end up with massive expressions escaping to Python. I'm working on Eigen-specific module, but this is a WIP.
Finally, for PyPy, cppyy there has less functionality than on CPython. It will trivially outperform the other solutions, but it does so because there is builtin support for cppyy in PyPy. The others have to go through the emulated C-API which can be very slow. This builtin cppyy module is being separately maintained and because of the higher demand for CPython support, effort has mainly been going to CPython, so cppyy in PyPy is now lagging.

Thanks a lot @wlav ! Your inputs are much appreciated. We will continue our analysis, but I guess we need to do some quick tests with those tools to understand a bit more what is going on.

Yes! Thank you very much @wlav! Your feedback is very much appreciated!


Automate refreshing the content at tlomodel.org

Currently the content published at tlomodel.org is generated from the tip of master when @tamuri runs a script. It would be useful to automate this regeneration of the content, either:

at the time of PR merge; or,
routinely as part of the overnight build; or,
exposing that routine through a GitHubAction.

Discussed this with @tamuri today. One thing which would facilitate this in terms of allowing us to review diffs of automated updates to the website to diagnose any issues that arise, or if we have a semi automated approach which opens a PR to update website automatically, would be to decrease the amount of 'noise' in commits which regenerate the website content. Currently we include a footer in all HTML files with the date last updated which means all pages will be touched on any update (this information is also included on homepage separately). Asif also mentioned that the index files underlying the search mechanism seem to use different IDs internally on refreshing content, so we may wish to look at ways to keep these consistent as far as possible.


Sorting by table column(s)

Great library!  Would it be possible to add a sorting by columns feature?

I'm afraid not. I want Rich to be concerned with rendering only, which is why internally the Table class stores strings or renderable. Ultimately I hope that other projects will implement the rich protocol and they can handle more data-oriented features, and leave the rendering to Rich.

Understood, thanks!


Up to 5 minute lag on binary_sensor.ohme_car_charging

Describe the bug
The binary sensor has in some instances a lag of over 5 minutes from when the car stops charging to when it changes state to off
To Reproduce
Observe during an active charge session where Ohme are using multiple 15 minute charge slots. Car stops charging on the 15 minute boundary, binary sensor can take over 5 minutes before it changes state to reflect this
Expected behavior
Binary sensor to update within a minute
Screenshots

Home Assistant Version
Core 2023.12.4
Supervisor 2023.12.0
OS 11.2
Frontend 20231308.2
Integration Version
0.1.8
Additional context

I can reproduce this. Attached screenshot shows power draw sensor vs CT clamp for garage. Once the actual draw drops to 0w, the Ohme reported wattage drops to ~800w for a few minutes before going to 0w. Looks like it could be more of a moving average than an instantaneous measurement.
The car charging sensor currently just checks if power draw > 0 as the API doesn't expose any indication the car is charging besides this.
I'm inclined to say it's an API limitation, but another approach would be to either:

Update the car charging sensor to interpret whether we are in a 'smart charge slot' rather than evaluating power draw
Add a separate sensor for option 1 and keep the car charging sensor as is

What are your thoughts around this, and what would work best for your current use case?

If the current draw drops to less than 1.4kW it isn't charging anyway. So on a change of state from >1400 to <1400 change the binary sensor to off.
For sensing off to on though, it would still need to be a change from 0 to >0.
I'm not sure what you mean by 'interpret if in a smart charge slot' though. You could still have a situation where if too much charge had been scheduled therefore in a planned charge slot, but the Ohme still delivering zero current as the car is no longer accepting charge.
Current draw is the only reliable measure.

That doesn't hold true for my car - if I charge to 100%, it tapers off and sits at ~950w for a good hour or so. I worry that with PHEVs which only charge at 3kW anyway, having an arbitrary 'cut-off' could introduce strange results.
By 'interpret if in a smart charge slot' I was suggesting an additional sensor to say that the Ohme is offering power, whether or not the car is drawing anything.

Maybe you are looking at the wrong API field then. Charger current would be a better indicator than an averaged power figure.
The actual current and voltage readings update every 10 secs I am led to believe :)

Not sure if it helps, but results from last night. I did a charge last night and got the same issue on v0.2.1. See screenshot.

The Octopus Mini and my IHD both showed the power drop to normal house usage bang on 8am and the CT clamp I have on the live on my EV supply showed it drop to 0kwh at 8am, but Ohme API via the new integration shows approx 4kWh for 5 mins after the charge ended and so Car Charging sensor was on for 5 mins after the charging stopped until Ohme was reporting 0kWh.

@brutus6 can you check the current draw (amps) sensor and see if that had the same lag please?

Yes. Same lag of 5 mins. The amps went from 32 to 17 for 5 mins before hitting 0.

How annoying. So I've got a couple of ideas to combat this, but the winning one for now is:
If current draw has dropped more than 40% since the least reading
AND
CT clamp reading is less than the current draw
Assume not charging until the current draw next hits 0

It seems if you increase the API polling rate to every 10 secs for the CT clamp and the session info this is negated somewhat. I need to do further testing. I'm not sure if it is because you are doing multiple API requests at different intervals?

Currently:

CT reading comes every minute
Current/power readings come every 30s
It looks like the CT reading reflects the charge stopping straight away.

Ideally I don't want to make these any more frequent as to not get on the wrong side of Ohme, but of course you are free to change the code or trigger the update routines through an automation.
A sub-60s delay on the charging state sensor seems acceptable to me, so I'll try and get something in today with CT based charge state detection.
My Ohme is installed on the garage, with the CT on the garage supply so will ordinarily be 0 when the car isn't charging. @andyvee69 @brutus6 if one/both of you have the CT on the whole home supply it'd be useful extra data after this goes in, with bonus points for additional night time loads :)

I don't think using the CT clamp reading is the correct approach for a number of reasons - older units do not have CT clamps, not everyone has the CT clamp fitted with Pro/ePod, if you have home solar or battery storage then the CT clamp measuring total grid import may not necessarily reflect what is happening with charging current.

It would be trivial to detect whether the clamp is present (eg. have we ever seen it go over 0), and I think it could be quite useful as an additional metric for the binary sensor if available.
I hadn't thought about the solar/battery storage aspect though. That complicates things, especially with no support for multiple clamps.
Perhaps I'll instead try to interpret that sudden drop as no longer charging?

@andyvee69 what did you do to get the Current draw to update quicker? (looks like it's updating every 5 seconds on you screenshot above)?
@dan-r This integration is brilliant! I've been wanting to create some Automations in HA to stop the OHME from draining my home battery setup when I get charge slots outside of the 23:30 - 05:30 period.
I've got it setup but as there is currently a 5 min delay on the updates it means potentially I'd be pulling from the grid until the integration updates and reports either 0amp/watts which then turns my batteries back on.  Hope that makes sense!

It might be a valid sudden power drop from 7kw to 1.4kW charging though.

@andyvee69 what did you do to get the Current draw to update quicker? (looks like it's updating every 5 seconds on you screenshot above)?

Altered the code. I'll let @dan-r explain :)

How annoying. So I've got a couple of ideas to combat this, but the winning one for now is:
If current draw has dropped more than 40% since the last reading AND CT clamp reading is less than the current draw THEN Assume not charging until the current draw next hits 0
@brutus6 did the Ohme CT reading sensor drop quickly?

Unfortunately, as @andyvee69 pointed out, I don't have a CT clamp fitted into my Ohme ePod for load balancing (so it just shows 0amps for me) as I don't have solar, battery or any other high load items in the house and a 100amp cut out, so a CT was not required.
The CT clamp I refer to is a custom built one that uses ESPhome and connects to HA to give me the load on the live wire for my charger.

It might be a valid sudden power drop from 7kw to 1.4kW charging though.

That's true - it seems inconsistent from my testing but seems to be some sort of average for the current and power draw.

what did you do to get the Current draw to update quicker? (looks like it's updating every 5 seconds on you screenshot above)?

You can update the intervals in your local copy of coordinator.py, but this is overwritten with every intergration update.

Unfortunately, as @andyvee69 pointed out, I don't have a CT clamp fitted into my Ohme ePod for load balancing (so it just shows 0amps for me) as I don't have solar, battery or any other high load items in the house and a 100amp cut out, so a CT was not required.

Yeah thats fair enough, and will especially be true for the portable models.
@andyvee69 it seems (but correct me if I'm wrong) as if your main reason for reducing the lag is to coordinate your external loads to only come on when the power is cheap. An alternative to polling more often could be to have a binary sensor to say 'we are in a smart charge slot, power is cheap!', and so just interpret if at the current time we are midway through an upward sloping line on the charge graph?

Thanks @dan-r  I did change it to this; (but it hasn't changed the amount it's logging)
class OhmeChargeSessionsCoordinator(DataUpdateCoordinator):
"""Coordinator to pull main charge state and power/current draw."""
def __init__(self, hass):
    """Initialise coordinator."""
    super().__init__(
        hass,
        _LOGGER,
        name="Ohme Charge Sessions",
        update_interval=timedelta(seconds=10),
    )

@JaseCat Did you restart home assistant afterwards? I find reloading the intergration doesn't always reload everything. Also worth noting that if those values haven't changed, the states won't show as updated in home assistant.

your main reason for reducing the lag is to coordinate your external loads to only come on when the power is cheap. An alternative to polling more often could be to have a binary sensor to say 'we are in a smart charge slot, power is cheap!', and so just interpret if at the current time we are midway through an upward sloping line on the charge graph?

No, it isn't. It is the exact opposite actually - it iis accurately controlling the turning off of high loads. I know how people are going to use this binary sensor for controlling other loads and in particular charging home battery storage. So if the binary sensor is incorrectly showing that the car is charging at 3 minutes past the hour, or 3 minutes past the half hour, but the charging has now stopped, people are going to incorrectly assume that the next half hour period is now at off peak so force charge batteries etc etc. Interpreting the graph is of little value either as the vehicle has to be activley charging. Knowing that some people set schedules to add 2 or 3 times the amount of charge they actually require, the graph is fairly meaningless as the car could have hit its charge limit or be full. So even though Ohme are trying to charge it, the current would be zero and therfore you are no longer at off-peak (unless during 23:30-05:30 obviously).
The only way of accurately knowing if the car is charging or not is with an accurate current (or power) reading.

but it hasn't changed the amount it's logging

Are you plugged in and actively charging?

Just did a restart and it's still only reporting every 5 mins.  @andyvee69 Yes, I did a test charge to see if it would update quicker. I've attached a screenshot.

Knowing that some people set schedules to add 2 or 3 times the amount of charge they actually require, the graph is fairly meaningless as the car could have hit its charge limit or be full. So even though Ohme are trying to charge it, the current would be zero and therfore you are no longer at off-peak (unless during 23:30-05:30 obviously).

I was under the impression that if Ohme are trying to charge it, that means power is cheap regardless of whether the car is pulling anything. It seems to be that way for me at least, although its hard to test because I have to wait for my next statement to see what I was billed at.

Just did a restart and it's still only reporting every 5 mins. @andyvee69 Yes, I did a test charge to see if it would update quicker. I've attached a screenshot.

I expect its just the values not changing in the API. It may be possible to trigger an update, but that risks abusing the service and causing a lot of data traffic.

@dan-r I'd be happy with it being 30 seconds as per your original settings. But I don't even get that....

I was under the impression that if Ohme are trying to charge it, that means power is cheap regardless of whether the car is pulling anything

Err, no. Otherwise I could set in the Ohme app that I have a car with a 120kWh battery, set it to Add 100% charge, plug in and get almost endless off-peak electricity as my 40kWh car that only needs 10kWh to charge to 100% would be full after an hour and a half. Ohme would have scheduled 18hrs of charging, and as my car had stopped accepting charge would keep extending the session.
The car has to be actively charging in a half hour billing period for it to be off-peak (outside of 23:30-05:30) regardless of if Ohme are still trying to force charge in to a an already full car or not.

@dan-r I'd be happy with it being 30 seconds as per your original settings. But I don't even get that....
It seems to be a limitation of the API. The integration is pulling every 30 seconds but the data its getting back just isn't changing.

The car has to be actively charging in a half hour billing period for it to be off-peak (outside of 23:30-05:30) regardless of if Ohme are still trying to force charge in to a an already full car or not.
Interesting, I always thought that was a possibility but just against the IOG terms.

but just against the IOG terms

??? Not sure what you mean. If Octopus/Ohme have scheduled the car to be charging, and it is charging, why is that in breach of Ts&C's?

??? Not sure what you mean. If Octopus/Ohme have scheduled the car to be charging, and it is charging, why is that in breach of Ts&C's?

I meant the case of stating a larger car battery and percentage effectively 'gaming' IOG to get more cheap energy

stating a larger car battery and percentage effectively 'gaming' IOG to get more cheap energy

But it doesn't work. Because the car has to be actively charging. Which was my point. So using a planned charge slot as an indicator has limited value, as if the car isn't charging because it is already full, it isn't off peak.
The only reliable indicator is if the Ohme is delivering power or not.
Maybe some logic can be built for the car is charging binary sensor based around charge current has suddenly dropped and no longer in a planned charge slot. This would have to be evaluated within a minute after the hour and half hour to line up with billing periods. The 15 minute boundary doesn't really matter as the off peak continues to the end of the 30 minute billing period anyway.

The latest version has a 'charge slot active' sensor. Although not particularly useful on its own, the idea is to keep an eye on it then see if its possible to add some logic like you say.
Current best approach is:
IF
Sudden current drop of 40%+ since last reading
AND
Not in a charge slot
AND
Charge slot ended within the past minute
THEN
Override car charging binary sensor for 5 minutes before resuming normal criteria of power > 0

Sounds good @dan-r

That should get the delay down to a maximum of 30 seconds. I'm sure it would be possible to do targeted refreshes on the time boundary to get it down further, but under 30s seems good enough to me.

Sounds good, but only needs to be checked twice an hour at maybe xx:00:45 and xx:30:45 as a few minutes delay at any other time makes no difference .

The charge graph and charge state (along with a load more) come from OhmeChargeSessionsCoordinator when we poll it every 30s, so charge state and charge slot active binary sensors get re-evaluated every 30s too. Not massively efficient but seems to be the way every other integration works and means we get them updated 'for free' without making any extra requests.
Worst case, with the new logic, if the 30s timer pings at xx:59:59, we'll have accurate 'not charging' state at xx:00:29 (+ network latency).

@andyvee69 @JaseCat @brutus6 I've just published an update that seems to get the charge state much quicker in my testing.
Mind testing it out overnight (If you're still awake)?
Now:

I charged to 100 last night so some very short slots and decreasing loads, but it seems to be working quite well from these couple of examples. Garage power is a Shelly EM.

Downloaded the latest version and the lag is now only about 20 seconds in reality. However, and really not sure how important it is for people with batteries, but there is still a lag of 5 mins at about just 97W. Lol.

Much better. Well done @dan-r, good work.

@brutus6 is that v0.2.3? When you say the lag is ~20 seconds is that on the charging binary sensor?

Updated to 0.2.3 and mine is still only reporting every 5 mins…

Updated to 0.2.3 and mine is still only reporting every 5 mins…

There's no way to get readings more often or more accurate, however often you poll. The API just returns (what appears to be) a moving average of the past 5 minutes.
0.2.3 adds some logic to the charging binary sensor to detect the sudden drop and report as not charging. The power/current sensors are unchanged.

Sorry, bit late to the party on this one. Having had several years poking around this, you may find some further joy by taking into account the battery SOC data fields (x[0].batterySoc.wh specifically) which do tend to keep moving (or not) depending on if it's charging or not. You get roughly 1Wh increments per sec even when trickle charging a 6A.
"batterySoc": {
"wh": 128,
"rangeKm": 0.0,
"percent": 0.0,
"timestamp":<PHONE_NUMBER>593,
"highPrecision": null,
"maxRangeKm": null
Notes: Don't confuse this batterySOC fields with the identically names one under the "Car"fields. That one only increments if car is API compatible AFAIK.
Beware also, vehicles have quirks. For instance, MG ZS Trickle charge will produce a constant 6A charge, however it will not intiate the power/voltage/current fields which will remain NULL.
I'll post a few other helpful tips on another thread.
Terravolt.

@brutus6 is that v0.2.3? When you say the lag is ~20 seconds is that on the charging binary sensor?

So for any confusion. So I am comparing the Ohme Power Draw and Ohme Current Draw in your integration vs, my own custom CT clamp that I have fitted to the live wire on the charger, which until know I have been using to tell me if the charger is charging the car or not.
So for me it is more than sorted. Thanks.
I was just pointing out the smallest of differences (in terms of the lag 20 seconds between the Ohme and my custom CT clamp and the 5 min lag at 97w - I mean I don't know if they are issues for those with batteries or not. @andyvee69 will be better to confirm on that I am sure.
As I said, as far as I am concerned, it is more than close enough to my custom CT clamp reading that I have been using.

Sorry, bit late to the party on this one. Having had several years poking around this, you may find some further joy by taking into account the battery SOC data fields (x[0].batterySoc.wh specifically) which do tend to keep moving (or not) depending on if it's charging or not. You get roughly 1Wh increments per sec even when trickle charging a 6A.
"batterySoc": {
        "wh": 128,

        "rangeKm": 0.0,

        "percent": 0.0,

        "timestamp":<PHONE_NUMBER>593,

        "highPrecision": null,

        "maxRangeKm": null

Notes: Don't confuse this batterySOC field with the identically named sub one under the "Car". That one only increments if car is API compatible AFAIK.
Beware also, vehicles have quirks. For instance, MG ZS Trickle charge will produce a constant 6A charge, however it will not initiate the power/voltage/current fields AT ALL which will remain NULL. MG ZS normal will work as expected.
I'll post a few other helpful tips on another thread.
Terravolt.

That's very interesting, I hadn't been looking at that at all, assuming it was only for API compatible cars.
Happy new year, and I look forward to hearing more tips from you :)

@andyvee69 I've re-written some of the charge state detection logic in v0.2.5 so it now uses a combination of:

Mainly the batterySoc.wh field - but on the boundary we ALWAYS see some 1+ delta, meaning it takes 2 ticks (1 minute) to get the state change
Charge slot data - a sudden drop on a boundary is interpreted as charge having stopped, which avoids the extra tick delay above
Power draw - if all else fails or we don't have the data we need, revert to power draw over 0 watts.

So in my testing I get the charge state updated in less than 30 seconds. I'll give it a proper test tonight and report back, but some extra eyes on it wouldn't hurt.
The logic is here and is extensively commented.
Thanks to @hoggyhoggy for pointing that field out!

Looks good, Just so it's clear:
_ # Failing that, we use the watt hours field to check charge state:
# - If Wh has positive delta and a nonzero power reading, we are charging
# This isn't ideal - eg. quirk of MG ZS in #13, so need to revisit_
It's not the Wh value that stays Null in this case, it's the Power values that stay Null (the Wh still increments).
This is only when using the MG ZS in Trickle mode (which I imagine is pretty rare) although there might be others. Normal MG ZS selected works as expected - power, voltage, amps...

@hoggyhoggy its badly written but what I meant was that one of the conditions for charging in progress is power draw (w) > 0, which we ideally don't want to avoid issues like you mentioned with the power values staying null.
The latest version works well to detect charge state in my testing, BUT gives some funky results if I charge my car to 100%. The power draw pattern is strange to say at the least. The drops to 0 at 4:46 and onwards should register as 'not charging' but the downward slope should still show as charging. More testing to do here!

I'm seeing some unexpected behaviour with the binary sensor for car is charging

@andyvee69 are any those 'dropouts' longer than 1 update cycle? (~30s).
If not, I'll put out an update this evening to fix that.

Seem to be exactly 30 seconds

Thought so - I've got a fix for that ready, so check back this evening.
It should get the delay down to under 30 seconds on the boundary of a charge slot and under 60 seconds if for example the car hits its limit and stops pulling.
Thanks for your patience!

@andyvee69 have you had a chance to test the latest changes? Seems to be reliable in my testing now.

I'll give it a test tonight

Seems to be working well

Good news, thanks for testing!
I'll close this for now.


bug discovering metrics

Let's try to add metrics that demonstrate how many bugs or vulnerabilities are visible by some tools in a Java class. We can use SpotBugs or PMD. Maybe, we can use Infer. Let's start with one simple tool and see how it works.

@yegor256
Hi, can I take care of this problem?

@Raleksan sure, go ahead!

@yegor256
I think you can close this issue.

@Raleksan thanks!


Nock sends multiple request to aws sqs

While running my test i got this error from aws sdk: "InvalidChecksum: sendMessage returned an invalid MD5 response. Got "undefined", expecting "af1a2418a805dfb024b9b974c0f358a7"."
I did some digging and i found out thet nock is sending 4 request to sqs server instead of one.
I debugged my code and it dosen't run more then once.
my nock code:
var nock = require('nock');
nock.recorder.rec({
dont_print: true
});
Thanks.

Nock itself is not making HTTP requests, it avoids them.
Perhaps your lib is making them for you?
Also, perhaps try the latest version where, by default, network requests are disabled.
If you find that this is the fault of nock, please reopen this or another issue.

This is old, so it probably doesn't help you, but I ran into this and what I think is happening is that SQS is looking for a specific XML response (including an MD5 of the message) and if it doesn't get that, it tries up to 4 times to send the message.
The XML format that worked for me is:
<SendMessageResponse>
  <SendMessageResult>
    <MD5OfMessageBody>${md5}</MD5OfMessageBody>
    <MessageId>${messageId}</MessageId>
  </SendMessageResult>
  <ResponseMetadata>
    <RequestId>${requestId}</RequestId>
  </ResponseMetadata>
</SendMessageResponse>

I found that for the requestId and messageId it didn't matter what it was.
I couldn't get nock to work with SQS because of the timing of the requests in my situation and how the md5 would be different for each request. But, if you can make the md5 within the test, you can get it to work.