Use the newest available installed mafia to bootstrap

This changes the bootstrap script used when building a version of mafia you don't have yet.
The bootstrap script will now find the latest available version of mafia, and use that to build instead of cabal, so that we can benefit from the global cache.
If for some reason this fails, it will do a git clean -xdf and try the old cabal method of building.
The contract we are assuming here, is that for all mafia versions, mafia upgrade && mafia build will cause an executable to be produced at dist/build/mafia/mafia.
/cc @markhibberd @charleso

@jystic curious, what situation are you left in if mafia upgrade && mafia build fails in the middle? Assume you can just run it again. Can it get properly broken at that point?

@tmcgilchrist the way the script works, it would do a git clean -xdf and then fall back to the old method of using cabal to install it.

:shaved_ice:

:rocket:

I'm not 100% sure about merging this one, haven't convinced @charleso and @markhibberd it's a good idea yet :)
The only reasonable alternative is OS/X build bots making binaries. @markhibberd any idea what it would take to hook jenkins up to that?

I'm not 100% sure about merging this one, haven't convinced @charleso and @markhibberd it's a good idea yet :)

My main hesitation is as mafia stabilises is this worth it? This currently costs maybe half an hour a week, but I feel like that will drop away pretty dramatically over time. Not sure.

The only reasonable alternative is OS/X build bots making binaries. @markhibberd any idea what it would take to hook jenkins up to that?

Probably not much, maybe a half day of stuffing around with infrastructure.

Decision? @jystic

@markhibberd I think this is a no for the moment - the right way is OS/X build bots.


Missing the SuppressReadReceipt-Method

Ok, I have realized with the help of the Exchange managed API that there is a "SuppressReadReceipt" -Method in the Emailmessage-Class to suppress the read-receipt of each single message before the isRead-property is set to true.
Is there a way to doe the same with the help of php-ews.
If not, is it possible to implement this feature?
Thanks in advance for any reply and/or help!

This issue is over four years old. If you are still having trouble with the latest version from master, please open a new issue.


PE-2309: fix: previous versions cannot be downloaded in sharing landing page mode

If we access a shared file from a private drive and paste it in a new tab, it is not possible to download any revision of the file from the 'Details' and 'Activity' panel.
Now we are using the FileRevision to download the file.
--- Releases ---
Android release: https://appdistribution.firebase.google.com/testerapps/1:305132849030:android:6cf0cd5ec064fad3ffce07/releases/78v1noikoe9bo
iOS release: https://appdistribution.firebase.google.com/testerapps/1:305132849030:ios:06ea33a95a2e0d42ffce07/releases/5bj7d4c3luegg

I tested on Safari. It does work 💯


AttributeError: 'int' object has no attribute 'parent'

Hello,
when I tried to train an agent with this command line
python3 scripts/train_agent.py "./runs/cartpole_checkpoints" SB3_ON CartPole-v1 cuda '{"ALGO": "PPO"}' --save_freq=10000

the following error occurred :
(base) cyl@WP:~/reward-surfaces-master$ python3 scripts/train_agent.py "./runs/cartpole_checkpoints" SB3_ON CartPole-v1 cuda '{"ALGO": "PPO"}' --save_freq=10000
/home/cyl/anaconda3/lib/python3.9/site-packages/stable_baselines3-1.6.0-py3.9.egg/stable_baselines3/__init__.py
False
Default hyperparameters for environment (ones being tuned will be overridden):
OrderedDict([('batch_size', 256),
             ('clip_range', 'lin_0.2'),
             ('ent_coef', 0.0),
             ('gae_lambda', 0.8),
             ('gamma', 0.98),
             ('learning_rate', 'lin_0.001'),
             ('n_envs', 8),
             ('n_epochs', 20),
             ('n_steps', 32),
             ('n_timesteps', 100000.0),
             ('policy', 'MlpPolicy')])
Using 8 environments
cuda
Log path: ./runs/cartpole_checkpoints/ppo/CartPole-v1_5
Traceback (most recent call last):
  File "/home/cyl/reward-surfaces-master/scripts/train_agent.py", line 71, in <module>
    main()
  File "/home/cyl/reward-surfaces-master/scripts/train_agent.py", line 67, in main
    agent.train(steps, args.save_dir, save_freq=args.save_freq)
  File "/home/cyl/anaconda3/lib/python3.9/site-packages/reward_surfaces-0.0.1-py3.9.egg/reward_surfaces/agents/SB3/sb3_on_policy_train.py", line 371, in train
    checkpoint_callback = self.create_callbacks(save_dir, save_freq=save_freq, prefix=save_prefix)
  File "/home/cyl/anaconda3/lib/python3.9/site-packages/reward_surfaces-0.0.1-py3.9.egg/reward_surfaces/agents/SB3/sb3_on_policy_train.py", line 353, in create_callbacks
    eval_callback = EvalParamCallback(
  File "/home/cyl/anaconda3/lib/python3.9/site-packages/reward_surfaces-0.0.1-py3.9.egg/reward_surfaces/agents/SB3/sb3_on_policy_train.py", line 142, in __init__
    super(EvalParamCallback, self).__init__(eval_env, callback_on_new_best, n_eval_episodes, eval_freq, log_path,
  File "/home/cyl/anaconda3/lib/python3.9/site-packages/stable_baselines3-1.6.0-py3.9.egg/stable_baselines3/common/callbacks.py", line 310, in __init__
    super().__init__(callback_after_eval, verbose=verbose)
  File "/home/cyl/anaconda3/lib/python3.9/site-packages/stable_baselines3-1.6.0-py3.9.egg/stable_baselines3/common/callbacks.py", line 134, in __init__
    self.callback.parent = self
AttributeError: 'int' object has no attribute 'parent'

I hope I can get your help,Thanks.

Did you solve this issue? I clone this code to my local computer. But I still suffer from this issue. I tried to solve this issue but I failed. :(

I got the same error

A late response here.
Change the super init function of class EvalParamCallback in sb3_on_policy_train.py from:
super(EvalParamCallback, self).__init__(eval_env, callback_on_new_best, n_eval_episodes, eval_freq, log_path,
                                                best_model_save_path, deterministic, render, verbose, warn)

to
super(EvalParamCallback, self).__init__(eval_env, callback_on_new_best,
                                                n_eval_episodes = n_eval_episodes,
                                                eval_freq = eval_freq,
                                                log_path = log_path,
                                                best_model_save_path = best_model_save_path,
                                                deterministic = deterministic,
                                                render = render,
                                                verbose = verbose,
                                                warn = warn)

The problem is due to the class EvalParamCallback and its parent class EvalCallback.
EvalCallback init() can pass an additional callback_after_eval function but not in EvalParamCallback. So it's getting n_eval_episodes as the callback function.


MSOutput Should not silently tolerate exceptions during rule creation

Impact of the bug
MSOutput
Describe the bug
A change in Rucio caused tape output rule creation to fail and we missed this for 3 weeks causing 7 PB of tape transfers to pile up.
How to reproduce it
Break Rucio
Expected behavior
If MSOutput fails to create a rule it, it should trigger an alarm, at least if it fails for multiple cycles.
Additional context and error message
None

And I just stumbled upon this issue, after resolving another 3-4 weeks outage of rule creation in MSOutput, addressed in this ticket: https://github.com/dmwm/WMCore/issues/12044
I am setting this ticket to Q4 such that we can at least implement an alarm and get notified when the whole MSOutputConsumer cycle is skipped.

After a private discussion with Alan, we decided that I can start working on this issue. so far, we agreed on a two-pronged approach:

[ ] make sure that if MSOutput has trouble processing a single workflow, it continues processing the next workflow and does not break from the loop.
[ ] Send monitoring data to a new AMQ topic (or re-use and existing topic with a new document type).

the idea is to have a single plot with a time series of the number of workflows that MSOutput needs to process
if MSOutput fails to process one or more workflows, then it will be visible as a non-zero baseline in this new plot. I like this approached because it worked reliably for CRAB's Publisher monitoring
we can setup a grafana alert on this plot (send a message if the number of workflows is above a threshold for longer than X days.)

We considered the idea of having MSOutput sending an alert if it fails to process a workflow N times, but it would require to implement some new logic to keep track of past attempts. Too much effort developing new code when we can achieve the same result exploiting existing monitoring infrastructure.


rbd: use go-ceph for getImageMirroringStatus

Currently, getImageMirroringStatus() is using RBD CLI.
This commit converts RBD CLI to go-ceph API.
Fixes: #2120
Signed-off-by: Yati Padia<EMAIL_ADDRESS>

It's good to capture some context about why this is Draft currently. Maybe use a task list to track any pending item's progress?

It's good to capture some context about why this is Draft currently. Maybe use a task list to track any pending item's progress?

Nothing much to list @pkalever .  I am stuck with something here. But will try to finish this soon.

/retest ci/centos/upgrade-tests-rbd

/retest ci/centos/mini-e2e-helm/k8s-1.19

@mergifyio rebase


Map is not working in Server-Client UI

Update: I just realized adding

Resolved, using new API key and appending & . API_KEY + &


iOS mDNS lookup fails with HotSpot enabled

Splitting off from https://github.com/flutter/flutter/issues/41085.
With Catalina and iOS 13, iOS mDNS lookup is reported to fail when HotSpot is enabled on the device.
This issue is to investigate and recommend a fix, or an error message from the flutter tool as appropriate.
@dnfield @christopherfujino @jmagman @tvolkert

Closing this as a duplicate of: https://github.com/flutter/flutter/issues/46698


Support for FCGIWrapper?

Is it possible to also support setups which use FCGIWrapper to invoke PHP? There is obviously no process like FPM running here which could receive Opcache reset requests.

I'm not sure if it is possible, but if you submit a PR, i see no reason not to accept it.

@gordalina I'd happily make a PR if I had an idea how to solve this. With this setup there is no way to cross the boundary of the CLI SAPI.
So I guess the only option here is the usual workaround where a random file is created in the web root with a opcache_reset() call and request this file from CLI via HTTP.
Unless you have a better idea I'll close this one.

what's your exact use case? apache2 with mod_fcgi?
maybe if you expose fcgiwrapper through a socket, you can pass the path to cachetool so it connects as a regular fcgi client

@gordalina Unfortunately I'm not sure about how this setup was made. It looks like mod_fcgid (mind the d) is currently used and I'm not sure where to find the socket in use. With lsof I can find this:
php5-cgi  32581       <username>    0u     unix 0xffff88023555a440       0t0    5271086 /var/lib/apache2/fcgid/sock/57741.60

However, the permissions of that socket file are rather strict:
srwx------ 1 www-data www-data 0 Jan 26 08:39 /var/lib/apache2/fcgid/sock/57741.60

It this setup messed up (not unlikely) or am I missing something?

if you run cachetool as www-data you'd be able to access the socket, can you try that?
php cachetool.phar opcache:status --fcgi=/var/lib/apache2/fcgid/sock/57741.60

This works indeed but requires sudo. And the socket file apparently changes from time to time so I'd have to detect it as mentioned with lsof.
All in all not really a useful solution from my POV for this setup. It'll most likely be switched to regular PHP-FPM soon so this shouldn't be an issue anymore.
But I'll keep the solution in my head in case I encounter it again, thanks a lot!


Any possibility of overriding of minimumkey size of particular algorithm in AsymmetricSignatureProvider

Referring to my post
https://stackoverflow.com/questions/60599339/any-way-to-create-a-jwt-token-with-key-size-512-and-changing-default-minimum-si
I need to change the minimum value of RSA256 key requirement from 2048 to 512.
Any possibility i can do it?
I am using .net core 3.1 with
Assembly Microsoft.IdentityModel.Tokens, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=31bf3856ad364e35
I have to consume existing system key sets

@KamranShahid you can do that, but we set 2048 for security reasons. Keys of length 512 will not be accepted by our runtime as they are not strong enough.
You would have to make use of extensibility.
You would need a custom AsymmetricSignatureProvider
https://github.com/AzureAD/azure-activedirectory-identitymodel-extensions-for-dotnet/blob/cc6d48f33df400362d261b6272ba5e4215f64e52/src/Microsoft.IdentityModel.Tokens/AsymmetricSignatureProvider.cs#L38
Then overload:
https://github.com/AzureAD/azure-activedirectory-identitymodel-extensions-for-dotnet/blob/cc6d48f33df400362d261b6272ba5e4215f64e52/src/Microsoft.IdentityModel.Tokens/AsymmetricSignatureProvider.cs#L330
You may also need to overload: Sign and Verify.

I am just using SecurityTokenDescriptor to generate the id token. So looks like there isn't any option in the class to override the settings.
For the hack i have used Reflection to change the Values DefaultMinimumAsymmetricKeySizeInBitsForVerifyingMap and DefaultMinimumAsymmetricKeySizeInBitsForSigningMap
Bit dirty code but effective
var mainclass = typeof(AsymmetricSignatureProvider)
                       .GetField(nameof(AsymmetricSignatureProvider.DefaultMinimumAsymmetricKeySizeInBitsForSigningMap), BindingFlags.Public | BindingFlags.Static);
                var field = mainclass.GetValue(null) as Dictionary<string, int>;
                if (field != null)
                {
                    field["RS256"] = 512;

                }

                var mainclass2 = typeof(AsymmetricSignatureProvider).GetField(nameof(AsymmetricSignatureProvider.DefaultMinimumAsymmetricKeySizeInBitsForVerifyingMap), BindingFlags.Public | BindingFlags.Static);
                var field2 = mainclass2.GetValue(null) as Dictionary<string, int>;
                if (field2 != null)
                {
                    field2["RS256"] = 512;
                }

@KamranShahid you can always use reflection. Development teams do not consider changing internals as breaking. So this approach may not work in the future. When we mark items internal or private, we make assumptions that may not hold if changed externally.

@KamranShahid you can always use reflection. Development teams do not consider changing internals as breaking. So this approach may not work in the future. When we mark items internal or private, we make assumptions that may not hold if changed externally.

Then atleast put some option to set it as per the requirement.

@KamranShahid by design we make it hard to do things that we do not consider secure.

@KamranShahid by design we make it hard to do things that we do not consider secure.

Totally Agreed. But My Case is regarding support of some legacy system which might be running in production

@KamranShahid the solution I outlined above, although complicated (by design :-) ), is durable as it uses public API's. Microsoft is committed to maintaining backward compatibility. There are times when this is has to be broken, but this is very rare as we know how much trouble breaking backward compatibility causes.


.gitignore needs to ignore vi swap files

Hi guys,
I noticed that the .gitignore file does not contain .swp files (generated by vi/vim). This could lead to useless swap files being commited.

netbox/static/ could also be added


Support -fno-exceptions

This is a request to support -fno-exceptions with exceptionless builds (via macro switch). In certain cases this is desirable behavior, namely for embedded and other environments that might use this library.
I did a quick sweep of the code and most cases where exception-related code appear can be switched out with an std::abort() call similar to the standard library's handling.
However, I did see some questionable usage: throw operation_cancelled{}. I've not used those facilities quite yet, but am I correct in assuming the exception here is being used for a non-exceptional control flow purpose? In which case, is there a better way to handle interruption events? Or does it also make sense to std::abort() here, too?

Exploring this issue a bit, it seems that my initial assumption about operation_cancelled was correct; in a private fork I've added some CMake configuration to disable that set of functionality if exceptions are configured to be disabled.
I also had to add some macros in config.hpp that ultimately replace all usages of try, throw and catch with versions that either pass-through if exceptions are enabled or stub out the code if they are not (or std::abort() in the case of throw).
The combination of these two things seemed to work fine, but doctest turned into being a problem - firstly, you cannot test based on excepted signals (SIGABRT in our case), and it appears that there's some access protection violation in the signal handler when doctest attempts to construct an error message for the failed test (a cursory web search seems to suggest it's a clang-specific issue; I honestly have no idea how to debug what's going on there).
For now, I just added an #if !CPPCORO_NO_EXCEPTIONS check around the two tests (in task_tests.cpp and shared_task_tests.cpp) for the broken_promise test and all was well.
Otherwise, it seems to work OK if you don't need the cancellable facilities. We haven't tested it thorougly in the super-project but at least the tests run and pass.


MN - MDH:CDC Prime ELR Errors 06/03/2022

Name:  Srujana Japa
Email<EMAIL_ADDRESS>MDH has received 41 messages on 06/03/2022 with incorrect values in MSH.5 and MSH.6, would you please be able to correct and re-send them?
Please find the accession IDs in the attachment.

@jbiskie the user sent another email - Following-up on the below email. Are you able to fix and re-send these messages? Please let us know.

@shy2cdc Thanks for the list, will review today.

6/13 3:27pm
Confirming receipt of all but one– the only one I didn’t see resent was the following:
221240005558^ProPhase Diagnostics^33D2215033^CLIA

6/13 3:35pm
Sorry for the double email, but it looks like these are still utilizing incorrect MSH.5, but the MSH.6 was corrected. Can you fix this for all future transmissions please?
Error MSH.5 incorrect value
Sent MEDSS-ELR^2.16.840.1.114<IP_ADDRESS>.1^ISO
Needed MEDSS-ELR^2.16.840.1.114<IP_ADDRESS>.6.2.1^ISO


add eviction on cpu pressure

This adds eviction of "no CPU pods" once the shared CPU is depleted
cc @derekwaynecarr

@ConnorDoyle ok, moved the eviction code to the manager

@ConnorDoyle added test

@ConnorDoyle updated

@ConnorDoyle updated and added the 2 container test pod I forgot before.

💯


hammerd wrapper fixes for some loops

Depends-on: 267

@mkarg75 can you check the conflict with hammerdb/hammerd_wrapper.py ? Thanks!

/rerun all

Results for SNAFU CI Test

Test
Result
Runtime

hammerd
FAIL
00:17:14

Results for SNAFU CI Test

Test
Result
Runtime

hammerd
FAIL
00:16:58

/rerun all

Results for SNAFU CI Test

Test
Result
Runtime

hammerd
FAIL
00:00:31

/rerun all

Results for SNAFU CI Test

Test
Result
Runtime

hammerd
FAIL
00:16:39

/rerun all

Results for SNAFU CI Test

Test
Result
Runtime

hammerd
FAIL
00:15:25

/rerun all

Results for SNAFU CI Test

Test
Result
Runtime

hammerd
FAIL
00:15:31

/rerun all

Results for SNAFU CI Test

Test
Result
Runtime

hammerd
FAIL
00:15:30

/rerun all

Results for SNAFU CI Test

Test
Result
Runtime

hammerd
PASS
00:09:36


[BEAM-8645] Create a py test case for Re-iteration on GBK result.

Create a python test case to iterate on GBK result for multiple times.
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ X] Choose reviewer(s) and mention them in a comment (R: @username).
[ X] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

R: @ananvay

CC: @pabloem

Thanks for reviewing.
Updated.


libvirt: fix hardcoded modules path

Description of changes
This implements what's in #285929, which is a better implementation then what's currently in #286330.
Replaces /lib/modules with /run/current-system/kernel-modules/lib/modules if it's built on linux.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Maintainer pings: @fpletz @globin @lovesegfault

Is there anything left to do in this PR to get this merged?

I've been using it for the past couple of days, works perfectly. :+1:

:+1: this needs to be merged. I've been running this patch locally and haven't noticed any odd side-effects.

Anyone, please? This is a critical issue.

Does not this suffer from the same issue initial version of the other PR did – not working on non-NixOS Linux?

Probably better revert this and merge the other PR.

This change is incompatible with that – it would indiscriminately patch out the NixOS path from the other PR.
That’s one of the reasons why explicit patches are better than substitute functions.


If Opp Naming date format is MMYY then an Opp on last day of year not in 2016 names in following year

thread: https://powerofus.force.com/0D58000002qsrEk
Only happens with opportunities not in this year. 12/31/2016 works fine but 12/31/2015 shows as 1216 and 12/31/2017 shows as 1218.
Tested and this only happens with MMYY and MMYYYY. All the same date formats work as expected.

**lurch: add

Tracking W-010937

Resolution is to use lowercase y for year: http://stackoverflow.com/a/8686347


How to test Web Component with @skatejs/bore?

The skatejs/bore README says

Enzyme-like testing utility built for the DOM and Web Components, that works both on the server and in browsers.

But, when I try making a test like this:
import { mount } from '@skatejs/bore'
import { h } from '@skatejs/val'
import VueWebComponent from '.' // uses `customElements.define()`

test('testing works', () => {
    expect(VueWebComponent).toBe(VueWebComponent)
})

the test fails with
> jest --config ./config/jest.config.js
 FAIL  src/test.js
  ● Test suite failed to run

    ReferenceError: customElements is not defined

      142 |     }
      143 | }
    > 144 | customElements.define('real-slot', RealSlot)
      145 |
      146 | function makeVuePropTypes( vueProps ) {
      147 |     if (! vueProps ) return {}

      at Object.<anonymous> (src/index.js:144:1)
      at Object.<anonymous> (src/test.js:3:9)

Test Suites: 1 failed, 1 total
Tests:       0 total
Snapshots:   0 total
Time:        0.969s, estimated 1s
Ran all test suites.

So it seems @skatejs/bore didn't set up the environment for testing Web Components!
It appears we have to make sure we have some sort of DOM polyfill loaded first, but based on what the README advertised, I was hoping we didn't have to do that!
How do you recommend setting this up?

Ah, I see further down the README says:

To test your web components in Node or Jest, you'll have to use @skatejs/ssr. More information there.

Importing only @skatejs/ssr/register in my test files wouldn't work, but when I configured Jest to use the provided testEnvironment, then it worked.
Why won't it work without the environment if I import @skatejs/ssr/register directly?

Oh! Because Jest's environment is by default jsdom, so using importing @skatejs/ssr/register in my test file was conflicting with that. So, using @skatejs/ssr/jest for testEnvironment worked!

Yeah, you have to explicitly use the test environment for that reason. Glad you figured it out!

I have something similar... I want to test with Jest my Web component developed in Angular Elements. My test.spec.js file has:
require('@skatejs/ssr/register');

describe("Embeded myElement", () => {
  beforeAll(() => {
    var script = document.createElement("script");
    script.src = "./scripts/my-element.js";
    document.body.appendChild(script);

    var elem = document.createElement("my-element");
    myElement.apptitle = "asdfasdf";
    document.body.appendChild(myElement);
  })

  it("create", () => {
    // some testing
  })
})

And jest.conf.js:
module.exports = {
  preset: "jest-preset-angular",
  testEnvironment: "@skatejs/ssr/jest",
  setupTestFrameworkScriptFile: "<rootDir>/jest.ts",
  ...
}

I'm getting the error with Zone.js:
  ● Test suite failed to run

    ReferenceError: Zone is not defined

      at node_modules/zone.js/dist/zone.js:646:1
      at Object.<anonymous>.FUNCTION (node_modules/zone.js/dist/zone.js:9:65)
      at Object.<anonymous> (node_modules/zone.js/dist/zone.js:12:2)
      at Object.<anonymous> (node_modules/jest-preset-angular/setupJest.js:5:1)

Any idea?

@mnicic it sounds like Zone might be doing something odd with with environment detection, or something. Happy to fix on this side if it's a problem with our test env.

@treshugart Actually, I'm not reporting an issue. I'm just asking about how to implement it. Any suggestion? Or example?


Real time visual editing issue with the React app

Hi there 👋 I am trying to setup some basic frontend app based on React and I have started playing with @storyblok/react
React playground which is available under https://github.com/storyblok/storyblok-react/tree/feature/v1/playground is not working with the Visual Editor properly, blocks are not highlighted and live changes are not applied in real time:

When I run the similar playground available for Vue https://github.com/storyblok/storyblok-vue/tree/main/playground it is all working properly by default:

Space was created from the scratch with the default space creator (home page with feature/grid/teaser)
I was trying to setup React app from the scratch by installing @storyblok/react (with one simple text block in the space) and I was also unable to get visual editing working as expected.

Hi @serrg! Sorry to hear that.
Could you take a look at this example repository, to see if it works properly?
https://github.com/storyblok/getting-started/tree/master/reactjs
Or this example in Stackblitz:
https://stackblitz.com/edit/getting-started-react
Or this other one, that is a Next.js project (Next.js is a React-based framework):
https://github.com/storyblok/getting-started/tree/master/nextjs
And let me know if the same is happening to you?

Hi @fgiuliani! Thanks for the fast answer,

Could you take a look at this example repository, to see if it works properly?
https://github.com/storyblok/getting-started/tree/master/reactjs

I can confirm that this example is working excellent:

BUT getting-started is using version 1.0.4  and storyblok-react/playground is using 1.1.6 so it looks like there is something incompatible in the latest version.
When I have tried to update the example from getting-started and upgrade @storyblok/react package to the latest version (1.1.6) is not working properly inside the visual editor (blocks are not highlighted and live changes are not applied):

I have checked all the versions down from 1.1.6 and version 1.1.0 of the @storyblok/react is the latest version which is working with storyblok-react/playground

Sorry it was a misclick ;)

Additionally I have checked the example with gettings-started/nextjs and it is working good with the latest version 1.1.6 so it looks like from version 1.1.2 there is some issue with pure React apps.

Thank you for taking the time to test @serrg! We'll be taking a look to see what can be causing this issue.

Hey @serrg thanks again for testing the SDK and sharing your feedback. I created a PR to fix the issue with the bridge not being initialized for React projects:
https://github.com/storyblok/storyblok-react/pull/223
Could you try updating to the latest version of the SDK?

Hi @fgiuliani I have updated package to the latest version (1.2.1) and now it works excellent, I confirm that PR fixed the issue 🚀


FireFox mesh issues (https://github.com/niivue/niivue/issues/1129)

List of fixed issues (if they exist):

#1129

Also resolves issue #1128


egressgw: test for conflicting IP rules in ENI mode

When the DestCIDR of a policy changes, we first add the new IP rules before
deleting the old ones. But if the new DestCIDR is <IP_ADDRESS>, attempting to
install the matching IP rules results in an -EEXIST error.
To deal with this, addMissingIpRulesAndRoutes() returns a shouldRetry
value. If it is set, we retry the installation after
removeUnusedIpRulesAndRoutes() has deleted the unused rules.
Add a test for this scenario.

/test

/test

(no-change rebase to get CI healthy)


optimize numerator()

hello deluan,
first thank you for this very useful library.
i rewrote the numerator() function. usually one would send a pull request, but i know norhing about git (mercurial user).
so i just paste my version here:
func numerator(img *imageBinaryChannel, template *imageBinaryChannel, offsetX int, offsetY int) float64 {
	imgArray := img.zeroMeanImage
	templateArray := template.zeroMeanImage

	iidx := offsetY*img.width + offsetX
	idelta := img.width - template.width
	tidx := 0
	var sum float64 = 0
	for y := 0; y < template.height; y++ {
		for x := 0; x < template.width; x++ {
			sum += imgArray[iidx] * templateArray[tidx]
			iidx++
			tidx++
		}
		iidx += idelta
	}
	return sum
}

two big changes:
1st: i pulled constanst computations out of the loop, as they don't change.
2nd:i swapped the loop order - y outer loop, x inner loop.
this way the loop is a lot more cache friendly
the new code is not only faster but a lot simpler and easier to understand.
on my machine i got a speed improvement of 20%. hope you find this useful!
regards,
dederon

Thanks for looking into this. I agree your version is easier to read, but I could not reproduce the performance gains in my environment. I modified the BenchmarkNumerator to test both numerator functions, and the results are basically the same (yours is numerator2):
BenchmarkNumerator
BenchmarkNumerator/numerator
BenchmarkNumerator/numerator-8         	 1251135	       938.0 ns/op
BenchmarkNumerator/numerator2
BenchmarkNumerator/numerator2-8        	 1283866	       935.1 ns/op

This obviously using the benchmark image, not your images. I also don't see how swapping the loops would make any difference, as it would depend on width being smaller than height, which we can't control and it is not always de case.
 My tests were done in a MacBook M2, Go 1.22.2.

hi deluan,
thanks four your quick reply. i could not answer earlier as there where issues with the github login.
first - as requested - your benchmarks on my machine:
goos: linux
goarch: amd64
pkg: github.com/deluan/lookup
cpu: AMD Ryzen 7 7735HS with Radeon Graphics
BenchmarkNewIntegralImage-16             7547535               164.9 ns/op           336 B/op          3 allocs/op
BenchmarkLookupAll-16<PHONE_NUMBER> ns/op              48 B/op          2 allocs/op
BenchmarkNumerator-16                    2062993               578.1 ns/op             0 B/op          0 allocs/op
BenchmarkOCR-16<PHONE_NUMBER> ns/op          128565 B/op        121 allocs/op
BenchmarkOCRParallel-16<PHONE_NUMBER> ns/op          129252 B/op        133 allocs/op
PASS
ok      github.com/deluan/lookup        7.714s

i have three patches lookup.zip, which are based on each other, so they have to be applied in order:

00_speedup_numerator.patch
this is the code i postet in my first message.

benchmark:
BenchmarkNewIntegralImage-16             7562330               165.6 ns/op           336 B/op          3 allocs/op
BenchmarkLookupAll-16<PHONE_NUMBER> ns/op              48 B/op          2 allocs/op
BenchmarkNumerator-16                    2132992               556.7 ns/op             0 B/op          0 allocs/op
BenchmarkOCR-16<PHONE_NUMBER> ns/op          128508 B/op        121 allocs/op
BenchmarkOCRParallel-16<PHONE_NUMBER> ns/op          129243 B/op        133 allocs/op

compared to current version on github:
benchmark                        old ns/op     new ns/op     delta
BenchmarkNewIntegralImage-16     165           166           +0.42%
BenchmarkLookupAll-16            3118168       3030824       -2.80%
BenchmarkNumerator-16            578           557           -3.70%
BenchmarkOCR-16                  4434511       4101976       -7.50%
BenchmarkOCRParallel-16          1336423       1165681       -12.78%

so my estimation of 20 % speed improvements where too optimistic.

01_pullup_channel_check.patch
i pulled the check for channel compatibility from lookup() up to lookupAll(), because lookup() would do this test for every pixel, which is not necessary. i don't expect a speedup, as the branch predictor will optimize this branch away. what's more important is that lookup() does not return an error anymore. this brings me to

02_concurrent_lookup.patch
lookup() will do the search concurrently now. i wrote a func lookupCol(), which runs in its own goroutine for every image column. as expected, this leads to a tremendous speedup (compared to github version):

BenchmarkNewIntegralImage-16             7221274               166.4 ns/op           336 B/op          3 allocs/op
BenchmarkLookupAll-16                       3542            352004 ns/op            5889 B/op         73 allocs/op
BenchmarkNumerator-16                    2136513               559.8 ns/op             0 B/op          0 allocs/op
BenchmarkOCR-16<PHONE_NUMBER> ns/op          211939 B/op       1140 allocs/op
BenchmarkOCRParallel-16                     2125            576892 ns/op          212741 B/op       1153 allocs/op

benchmark                        old ns/op     new ns/op     delta
BenchmarkNewIntegralImage-16     165           166           +0.91%
BenchmarkLookupAll-16            3118168       352004        -88.71%
BenchmarkNumerator-16            578           560           -3.17%
BenchmarkOCR-16                  4434511       1230779       -72.25%
BenchmarkOCRParallel-16          1336423       576892        -56.83%

the 8-fold speed improvement of LookupAll() reflects the fact that my machine has 8 cores, so it scales linearly. Even the ocr routines benefit from this improvement. what bothers my is that lookupCol() has 9 (!) input parameters, which is awful.
that's all for now.
one last thing: when running go test on the lookup package there seems to be an error, event though the test suite is flagged as PASS:
......
6 total assertions

..
8 total assertions

..
10 total assertions

........
18 total assertions

....incompatible channels 0 <> 1
.
23 total assertions

.
24 total assertions

........
32 total assertions

PASS
ok      github.com/deluan/lookup        0.032s

regards,
heiko

Thanks and sorry for the late reply. I'll take a look at the patches.

usually one would send a pull request, but i know norhing about git (mercurial user).

You could use GitHub's own interface to create a PR. Just browse to the file you want to change and edit it:

If you use VSCode or IntelliJ, that should be very easy to create PR's from their own UI.
Anyway, I'll check the patches and will see if I can make the lookupCol better (less parameters). Thanks!

hi deluan,
thanks for taking your time to educate me!
please ignore my patch 02, i have something better in mind. my patch makes the code slower when using a small image in a small search window (because of the goroutine/channel overhead). i will report back soon (tm) with a much better approach.
regards,
heiko


The type of property isEnabled for Managed Application Definition should be boolean not string

I am trying to get resource Managed Application Definition with azure go sdk but it failed and threw below error message after created this resource successfully. After investigated, the property type of isEnabled in local package is string, however, the property type of isEnabled in the response returned from azure side is boolean. As they are inconsistent, so it encountered property type conversion issue. So the property type of isEnabled for managed application definition in repos azure-rest-api-spec (https://github.com/Azure/azure-rest-api-specs/blob/06b9412319ada8678c0ce07016cac1bf66258b1d/specification/resources/resource-manager/Microsoft.Solutions/stable/2018-06-01/managedapplications.json#L1014) and azure-sdk-for-go (https://github.com/Azure/azure-sdk-for-go/blob/781d9fb592a1818cb7fb8e0855ea9c00b095a9e9/services/resources/mgmt/2018-06-01/managedapplications/models.go#L589) should be updated from string to bool, right? If yes, please help to fix it ASAP since we’re blocked on here. Thanks.
The error message:
The error message:
Error: Error retrieving Managed Application Definition (Managed Application Definition Name "neiltest2-managedapplicationdefinition" / Resource Group "neiltest2-resources"): managedapplications.ApplicationDefinitionsClient#Get: Failure responding to request: StatusCode=200 -- Original Error: Error occurred unmarshalling JSON - Error = 'json: cannot unmarshal bool into Go struct field ApplicationDefinitionProperties.isEnabled of type string' JSON = '{"properties":{"authorizations":[{"principalId":"xxxx","roleDefinitionId":"xxxxx"}],"isEnabled":true,"lockLevel":"ReadOnly","displayName":"TestManagedApplicationDefinition","description":"Test Managed Application Definition","artifacts":[{"name":"ApplicationResourceTemplate","type":"Template","uri":"https://xxxx/applicationdefinitions/xxxxx/xxx/applicationResourceTemplate.json"},{"name":"CreateUiDefinition","type":"Custom","uri":"https://xxxx/subscriptions/xxx/resourceGroups/neiltest2-resources/providers/Microsoft.Solutions/applicationDefinitions/neiltest2-managedapplicationdefinition/applicationArtifacts/CreateUiDefinition?api-version=2017-09-01"},{"name":"MainTemplateParameters","type":"Custom","uri":"https://management.azure.com/subscriptions/xxxxx/resourceGroups/neiltest2-resources/providers/Microsoft.Solutions/applicationDefinitions/neiltest2-managedapplicationdefinition/applicationArtifacts/MainTemplateParameters?api-version=2017-09-01"}]},"id":"/subscriptions/xxxxxx/resourceGroups/neiltest2-resources/providers/Microsoft.Solutions/applicationDefinitions/neiltest2-managedapplicationdefinition","name":"neiltest2-managedapplicationdefinition","type":"Microsoft.Solutions/applicationDefinitions","location":"eastus","tags":{}}'
on main.tf line 6, in resource "azurerm_managed_application_definition" "example":
6: resource "azurerm_managed_application_definition" "example" {
Repro steps:
Create a resource group
Create Resource Managed Application Definition
Immediately call "Get" go rest api to get resource Managed Application Definition, which returns below error message

Hi @armleads-azure could you please have a look at this issue?

This has been fixed in later versions of the SDK:
github.com/Azure/azure-sdk-for-go/services/resources/mgmt/2019-07-01/managedapplications
in v39.0.0

Hi @neil-yechenwei since this has been fixed in a higher api-version, maybe you will have to upgrade the api-version...
While in the meantime, @MSEvanhi is it possible to back port this fix to the old api-version as well? Despite this might be a breaking change for the old api-version, I suppose as a bug fix, it may be allowed


Erase functionality

I want to add an Erase button which erases any selection previously done.
I used an button which when clicked calls a function very similar to the function responsible for Brush, which is:
for (var y = -1 * brushSize; y <= brushSize; y++) {
    for (var x = -1 * brushSize; x <= brushSize; x++) {
      if (x * x + y * y > brushLimit) continue;
      var offset =
        4 *
        ((pos[1] + y) * this.layers.visualization.canvas.width + (pos[0] + x));
      offsets.push(offset);
      labels.push(label);
    }
  }
  annotator._updateAnnotation(offsets, labels);

with one difference:
 labels.push(label);

The last line inside the for loop I've hardcoded 0 being pushed into labels.
 labels.push(0);

0 represents the background white color so on clicking the Erase button and selecting any area within a selection, it looks like stuff is being erased since the color of the label gets replaced by the background.
But in truth this isn't actually erasing anything, just continues to add label with white background. So when I export it, the erased area expectedly still shows up.
Any pointers on how to delete previously selected area?

Why not just choose current label=0 (annotator.currentLabel = index;) and use any of the desired tool?

it looks like stuff is being erased since the color of the label gets replaced by the background.
But in truth this isn't actually erasing anything, just continues to add label with white background. So when I export it, the erased area expectedly still shows up.

What I need to do is unselect an area, using current label 0 is still drawing label with color white.

Do you mean undo functionality
or fill existing labels?
Default label for every pixel is 0.

Undo is not what I want, I may want to erase one particular segment of labelling that I've done, I wouldn't want to delete all the other subsequent labelling I've done after it.
And with fill(), there is
        annotationData = this.layers.annotation.imageData.data;

How am I to update imageData of the annotation layer?, so that the updated image data is assigned to annotationData here.
One solution I thought of was to store a list of all the [x,y] coordinates that he's selected clicked for and do a clearRect() of annotation layer for each of them. What do you think?

@Abdul-Ghani There is no notion of segment or piece in brush tool. Every stroke directly paints pixels.

Right, and I want to do an eraser which can delete any labelling selection done by either polygon or brush or superpixel selection.
Can you help me by explaining the offset line here? (https://github.com/kyamagu/js-segment-annotator/blob/3f6e4be4ef2822b9946673f34920157cc3f89914/js/app/edit.js#L349)
for (var y = -2; y <= 2; y++) {
      for (var x = -2; x <= 2; x++) {
        // it is circle bitches
        if ((x*x + y*y) > 7) continue;
        var offset = 4 * ((pos[1]+y) * this.layers.visualization.canvas.width + (pos[0]+x));
        offsets.push(offset);
        labels.push(label);
      }
    }

the line javascript         if ((x*x + y*y) > 7) continue; I think is used to check if the point is within the circle or outside. The line after that,
        var offset = 4 * ((pos[1]+y) * this.layers.visualization.canvas.width + (pos[0]+x));

Can you help explain this above line?

Offset is the location within the memory buffer.

okay, how is it being calculated here? Can you explain the equation on the right side?

a buffer is row-major order, where pixels are arranged in RGBA order. RGBARGBARGBA...

Alright thanks @kyamagu this helped.


Fixing plex http probe

Fixing the HTTP probe. I've tested it on my cluster and it works fine.
The current HTTP probe doesn't work and causes the incorrect pod start, thus the incorrect ingress readiness

thank you @amatacchieri for the fix and @InputObject2 for the review, merged!


Moving translation resources from XLFs to JSONs

The problem
Right now, if someone wants to translate Files, they have to clone the whole project just to work on one small file, not to mention having to work with XML, which has really bloated syntax, making finding the important stuff hard.
The solution
Replacing the XLF files with JSON files (preferably using a JSON schema to determine the structure), and removing all of the unnecessary stuff that doesn't apply to translators, then using a custom script to generate XLF files from the JSONs.
Scope

Capability
Priority

Set a structure for all translation files to follow
Must

Use a JSON schema to pre-determine that structure
Should

Move the translation files into a separate repo, reducing the clone size
Could

Use a crowd sourcing platform to manage translations
Could

Require the translators themselves to move their translations into JSONs
Won't

Additional comments
If needed, I can write the JSON schema and/or port the old XMLs myself.

@TheOnlyTails How would this prevent the need to clone the repository?

Plus the repo is 17MB if you don't build the app which is not that much :)

@TheOnlyTails How would this prevent the need to clone the repository?

@yaichenbaum I've updated the issue description.

The xlif files also do a lot more than hold the strings, it keeps track of which strings were updated, which ones need to be translated etc.

That could all be passed as optional fields with default values. The files are still way too big and complex.

@TheOnlyTails How would we update those fields? The multilingual toolkit extension automatically updates them for us.

@yaichenbaum I wasn't aware of that, but I don't think it's a really big issue.

@TheOnlyTails Unfortunately I don't think it's worth spending time on creating an alternative method since our current focus is on the Files app rather than the tooling around it.

OK then, I see what you mean.


Fixed video labelling after subset call for HMDB51 dataset (hmdb51.py)

In the current torchvision release, when we take a subset of a given VideoClips object X (by calling X.subset(indices)), as we would when creating Val and Train splits in HMDB51, a new VideoClips instance Y is returned, where Y.video_paths is a subset of X.video_paths.
As a result, after a non-trivial subset is selected from the HMDB51 dataset, the i-th video in self.samples does not correspond to the i-th video in self.video_clips.video_paths.
Because of that, the label retrieved from self.samples[video_idx][1] in the __getitem__() method of HMDB51 is not the correct label for the video at self.video_clips.video_paths[video_idx]. This leads to mislabeling in the dataset.
Our fix stores self.indices to allow __getitem__() to invoke self.samples[self.indices[video_idx]][1], which retrieves the label of the correct video.

Thanks a lot for the fix!
Can you apply the same fix to UCF101 as well? I believe it will also suffer from the same problem.

Certainly, no problem!
I'll should have a fix pushed tomorrow.
-------- Original message --------
From: Francisco Massa<EMAIL_ADDRESS>Date: 8/17/19 3:14 AM (GMT-07:00)
To: pytorch/vision<EMAIL_ADDRESS>Cc: John Mistele<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [pytorch/vision] Fixed video labelling after subset call for HMDB51 dataset (hmdb51.py) (#1240)
Thanks a lot for the fix!
Can you apply the same fix to UCF101 as well? I believe it will also suffer from the same problem.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/pytorch/vision/pull/1240?email_source=notifications&email_token=AJCES3XVR627YC5ZQTNASSDQE66PVA5CNFSM4ILQLRFKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4QHFZY#issuecomment-522220263, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AJCES3XCFU25AC74HKWGELDQE66PVANCNFSM4ILQLRFA.

Codecov Report

Merging #1240 into master will decrease coverage by 0.41%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master    #1240      +/-   ##
==========================================
- Coverage   65.64%   65.23%   -0.42%
==========================================
  Files          74       74
  Lines        5784     5784
  Branches      884      884
==========================================
- Hits         3797     3773      -24
- Misses       1723     1737      +14
- Partials      264      274      +10

Impacted Files
Coverage Δ

torchvision/datasets/hmdb51.py
27.65% <0%> (ø)
:arrow_up:

torchvision/datasets/cifar.py
71.26% <0%> (-6.9%)
:arrow_down:

torchvision/datasets/folder.py
79.48% <0%> (-2.57%)
:arrow_down:

torchvision/datasets/voc.py
20.87% <0%> (-2.2%)
:arrow_down:

torchvision/datasets/lsun.py
20.83% <0%> (-2.09%)
:arrow_down:

torchvision/transforms/transforms.py
79.96% <0%> (-1.58%)
:arrow_down:

torchvision/transforms/functional.py
70.23% <0%> (-1.16%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 19315e3...3a1b2a7. Read the comment docs.

Alright, the changes are made. UCF101 labeling is patched.

Looks good to me - can you please check why the tests are failing?

The failures seem to be some sort of certificate issue. The error message is:
IOError: [Errno socket error] [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:727)
which occurs when trying to run utils.download_url(...) at lines 47 and 58 in test/test_datasets_utils.py.
However, it only seems to be an issue for Python 2.7. This test is also run for Python 3.6, and there is no issue. I also don't see how an SSL certificate issue relates to the minute code changes I made. If you have any insights, I'd be happy to hear them!

As an additional note, there seems to be identical Travis CI failures in https://github.com/pytorch/vision/pull/1244

Yeah, not sure what that's about. Let's wait until Francisco is back from vacation. Usually my test were failing due to OOM, but this shouldn't be the case given the nature of your changes.

Back from holidays.
So the test failures seem to be due to a change in GitHub. A fix had to be sent to PyTorch to address this, see https://github.com/pytorch/pytorch/pull/25042
This is something that should be fixed in a separate PR, and is unrelated to your change.
I created an issue to track this in https://github.com/pytorch/vision/issues/1268
If you could fix that issue first (for example with option 2), it would be great!

I'm merging this as the test errors are unrelated, and this is a pretty important fix. Thanks a lot for the catch and the fix @JMistele !


Javascript

#JavaScript
##User Story
As a boot camp student
I want the prework notes to be structured on a webpage
So that I can easily find and read the information
##Acceptance Criteria
GIVEN a Prework Study Guide website
WHEN I view the study guide
THEN I can see the four topics I learned listed along with a suggestion on what I should study first

The Javascript code is ready!


Add Sticky Jelly entity

If the jelly has a horizontal speed of 0, it levitates in the air.

The jellyfish will continue to be moved by wind, if this change is brought up again we'll look into adding an option to let it not move with wind.


Also clone program data for stake pool test-validator

This is needed for the program to work

Thanks for the contribution, we'll eventually have to fix up the test validator to properly copy all accounts for upgradeable programs.


Alter plan parameters in plan repositories to be Sets where appropriate

Many plans currently take a list of devices, where actually the uniqueness of the devices is assumed and required.
These lists should be replaced with a set and handling of the plan should be updated to allow for metadata to pass an order of the devices if it is required.
This will make the returned schema for a plan easier to reflect in a generated UI.

Done during the move of the plans from dls-bluesky-core to dodal
https://github.com/DiamondLightSource/blueapi/pull/645


xds-failover: disabling flaky and potential flaky tests

Commit Message: xds-failover: disabling flaky and potential flaky tests
Additional Description:
Temporarily disabling some of the xds-failover integration tests, as they are flaky.
Risk Level: low
Testing: N/A
Docs Changes: N/A
Release Notes: N/A
Platform Specific Features: N/A

@htuch I'm disabling most of the integration tests as I'm not sure which one are flaky.

@adisuissa you'll need to add a coverage exception, coverage tests failing.


Improving UI of Login /Signup page

Is your feature request related to a problem? Please describe.
The UI of login/signup need to be improved . Text fields need to be designed properly, there should be camoflauge of colors that match the overall UI of website.
Describe the solution you'd like.
will aling the text field properly , add password visibility button. improved UI to Camoflauge with the website. will also add a back to home button on login/signup page to navigate to home page.
Describe alternatives you've considered.
No response
Additional context.
has added the screenshots below.
Show us the magic with screenshots
we can design it somewhat as :-

Checklist

[X] I have checked the Existing Issues
[X] I have read the Contributing Guidelines
[X] I want to work on this issue. (optional)

@aditya-bhaumik  please assign me this issue under gsooc-extended.

@aditya-bhaumik  ok. As you wish.

@aditya-bhaumik can  you assign me the work of creating a contact us page for the website , if you are allowing i will raise an isuue.

Already someone is working on the contact us page check the older issues


Existing mbtiles-File not found

I just tested this plugin using the sample. While verifying process an existing local mbtiles file was not fount. The app always downloaded the file from web.
I could solve this by changing following lines in file map.js:
function verifyingMap(localFileName, remoteFile){

// leaving out the first lines...

		mbTilesPlugin.init({type: type}, function(rinit) {
			mbTilesPlugin.getDirectoryWorking(function(r) {
				// alert("getDirectoryWorking Verify : " + r.directory_working);
				var absoluteLocalFileName = **"file://"+**r.directory_working + localFileName;

				// check to see if files already exists
			  var file = **window.resolveLocalFileSystemURL(absoluteLocalFileName, function** (fileEntry) {

// skipping rest of function

The seems to be a change in the cordova file plugin. Hope this will help for others...

Hi,
On which OS do you test it ? iOS , Android, etc.. ? and which version ?
On which version of cordova ?
Regards
Florian.

Hi,
first of all: quite impressive response time! Thanks a lot!
And sorry, I did not think about it...
I'm using an archos tablet with android 5.1.1
Development on an Mac using Cordova 6.4.0
I created a new project and added mbtiles-support via cordova plugin add  The files plugin was added automatically. (cordova-plugin-file 4.3.0)
If I forgot something, just ask.
Best regards,
Til

I can't debug now but if you have time and android studio maybe you can link process and debug it from source in android studio and watch what getDirectoryWorking return.

I set the directory in the constuctor of th plugin. Just watch this , i think the problem is here and an exception on reading file or database up.

I've checked all the returns (you already provided necessary debug output) and the path to file was absolute correct. Just one thought coming up this moment - I did not test - maybe you missed "file://" in the beginning of the absolute path?
For test purpose I skipped your test and the map was rendered correctly, so path must have been correct. Only the test whether the file is present failed.

I think it's just the permission maybe , i forget that but do you
application provide the permission of acces in SDCARD. Now on Android 5.0.1
you need to ask the permision to the user , or the user need to provide it
in the settings(system) of application.
On Thu, Dec 15, 2016 at 1:22 PM, stihl-sicher<EMAIL_ADDRESS>wrote:

I've checked all the returns (you already provided necessary debug output)
and the path to file was absolute correct. Just one thought coming up this
moment - I did not test - maybe you missed "file://" in the beginning of
the absolute path?
For test purpose I skipped your test and the map was rendered correctly,
so path must have been correct. Only the test whether the file is present
failed.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/ffournier/cordova-plugin-mbtiles/issues/34#issuecomment-267315187,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADMiILcYcRwUCFS2RpYlcwgyP7rMZ7HZks5rITD5gaJpZM4LN9Fg
.

Permission für SDCARD is required when installing. So permission should be okay.

For test purpose I skipped your test and the map was rendered correctly, so path must have been correct. Only the test whether the file is present failed.

So the problem is when we resolve the file in constructor, but i can't understand how you can open the file without reuse the directory present in the plugin, due to the using of this value to find the file.


Delete backup wallet from settings

I left the NOT_AVAILABLE_WATCH_ONLY translations in there, because we may still need them at one point. However we don't have this feature yet I believe. So should I keep it?
Fixes: #63

Nice. Yes, keep it!


Merge cmd testing

Coverage increased (+1.6%) to 82.438% when pulling fc714f4f99420d7b15fbf13dcebce3561f37ea71 on merge_cmd_testing into 9894659d9ffb5247c431ee97e2732fc78f064630 on master.


Add server tests

We should add a test that checks that the server runs OK. It should:

Start the server

Use the fake model

Create a connection with websocat
Send a task via websocat (can we pipe to stdin?)

Closing in favor of https://github.com/OpenDevin/OpenDevin/issues/341

Tried to run the regression tests viapython evaluation/regression/run_tests.py but running into this error ...
(OpenDevin) (base) suibhne@Suibhnes-MBP OpenDevin % python evaluation/regression/run_tests.py
Traceback (most recent call last):
  File "/Users/suibhne/OpenDevin/evaluation/regression/run_tests.py", line 4, in <module>
    from opendevin import config
ModuleNotFoundError: No module named 'opendevin

Anyone else running into this?

@SuibhneOFoighil try setting
PYTHONPATH=`pwd`


Support for Home Assistant Energy dashboard

Home Assistant added a new Energy dashboard in version 2021.8, but it seems the devices provided via the Tuya v2 integration are not supporting it, as they are not visible when trying to add a device to the dashboard.
More info about the issue here: https://www.home-assistant.io/more-info/statistics/ and here https://developers.home-assistant.io/docs/core/entity/sensor/#long-term-statistics
edit: it seems I accidently tagged this issue as a bug and can't change it. should be enhancement

https://developers.home-assistant.io/blog/2021/05/25/sensor_attributes
I tried to play with it in sandbox and it works well. Basically just 2 attributes has to be added to sensor.py - state_class and last_reset
The question is do we really need to mark all of currently supported sensors as measurement?
Another one - does any of currently supported sensors report the real reset timestamp or we can default it to None?

@molnarti & @skynetua do your Tuya power devices (entities) show the correct wattage and volts?
All my power devices (light switches and power points) show incorrectly.
For example Volts show as 2314V instead of 321.4V and watts show as 2341W instead of 234.1
Tuya engineer told me it's not a bug. However somewhat useless that way it is.

@mycrouch I don't have any but it seems bugged after https://github.com/tuya/tuya-home-assistant/commit/ececdc12cc1bed16c9e0a86a097538e350c1647d
`Description of scale parameter
The following is an example of the constraint on the present voltage value. scale value is 1.
If the value obtained by querying the device status is {"cur_voltage":2230}, then the user-readable value is 2230/(10^1 )= 223V (the voltage is 223 volts).`

@mycrouch i have no problem with my tuya devices, they show the correct voltage even with version 1.4.3

@mycrouch  Can you provide your device info and device specification from the logs?

This is the log for a power point
{
"active_time":<PHONE_NUMBER>,
"biz_type": 0,
"category": "cz",
"create_time":<PHONE_NUMBER>,
"icon": "smart/icon/1492863365_0.png",
"id": "83767607a4cf12a3ef07",
"ip": "xxx.xxx.xxx.xxx",
"lat": "-27.65952312573868",
"local_key": "xxxxxxxxxxxx",
"lon": "xxxxxxxxxx",
"model": "单色LED灯",
"name": "Bathroom - Power Point",
"online": true,
"owner_id": "2286669",
"product_id": "eeqdSYkbi9u2W3Kt",
"product_name": "Double Power Point",
"status": [
{
"code": "countdown_1",
"value": 0
},
{
"code": "add_ele",
"value": 23
},
{
"code": "cur_current",
"value": 0
},
{
"code": "cur_power",
"value": 0
},
{
"code": "cur_voltage",
"value": 2468
},
{
"code": "test_bit",
"value": 1
},
{
"code": "voltage_coe",
"value": 607
},
{
"code": "electric_coe",
"value": 26421
},
{
"code": "power_coe",
"value": 15407
},
{
"code": "electricity_coe",
"value": 1350
},
{
"code": "switch_1",
"value": false
},
{
"code": "switch_2",
"value": true
},
{
"code": "countdown_2",
"value": 0
}
],
"sub": false,
"time_zone": "+10:00",
"uid": "xxxxxxxxxx",
"update_time":<PHONE_NUMBER>,
"uuid": "xxxxxxxxxx"
},
Log for light switch
{
"active_time":<PHONE_NUMBER>,
"biz_type": 0,
"category": "kg",
"create_time":<PHONE_NUMBER>,
"icon": "smart/product_icon2/kg_1.png",
"id": "38042678840d8e9504db",
"ip": "xxx.xxx.xxx.xxx",
"lat": "-27.65952312573868",
"local_key": "xxxxxxxxxx",
"lon": "xxxxxxxxxxx",
"model": "",
"name": "Bathroom - Light Switch",
"online": true,
"owner_id": "2286669",
"product_id": "XsB7wAYuXZRNPMcG",
"product_name": "Three Switch",
"status": [
{
"code": "switch_1",
"value": false
},
{
"code": "switch_2",
"value": false
},
{
"code": "switch_3",
"value": false
},
{
"code": "countdown_1",
"value": 0
},
{
"code": "countdown_2",
"value": 0
},
{
"code": "countdown_3",
"value": 0
},
{
"code": "add_ele",
"value": 0
},
{
"code": "cur_current",
"value": 0
},
{
"code": "cur_power",
"value": 0
},
{
"code": "cur_voltage",
"value": 2474
}
],
"sub": false,
"time_zone": "+10:00",
"uid": "xxxxxxxxxx",
"update_time":<PHONE_NUMBER>,
"uuid": "xxxxxxxxxxx"
},

@mycrouch I don't have any but it seems bugged after ececdc1

Description of scale parameter
The following is an example of the constraint on the present voltage value. scale value is 1.
If the value obtained by querying the device status is {"cur_voltage":2230}, then the user-readable value is 2230/(10^1 )= 223V (the voltage is 223 volts).`

Digged a bit deeper and seems like not all devices provide scale factor for integer value.

I've been digging through the code and in the sensor.py there is the following, should the code __state = (__value) * 1.0 / (10 ** __value_range.get('scale')) divide the power data by 10 thus giving the correct output?
@property
def state(self):
    """Return the state of the sensor."""
    return str(self.tuya_device.status.get(self._code))
    __value = self.tuya_device.status.get(self._code)
    print(self.tuya_device.status_range.get(self._code).values)
    if self.tuya_device.status_range.get(self._code).type == 'Integer':
        __value_range = json.loads(self.tuya_device.status_range.get(self._code).values)
        __state = (__value) * 1.0 / (10 ** __value_range.get('scale'))
        if __value_range.get('scale') == 0:
            return int(__state)
        else:
            return f"%.{__value_range.get('scale')}f" % __state
    return ''

Be great to get power monitoring plugs available in HA Energy Dashboard - they are currently reading correctly within Tuya2 however not available to HA Energy.

@jstott1070 unfortunately HA devs limited it to work only with energy monitoring devices. Power is not supported right now. I haven't seen energy reporting plugs by Tuya so far, only energy meters and circuit breakers.

@jstott1070 unfortunately HA devs limited it to work only with energy monitoring devices. Power is not supported right now. I haven't seen energy reporting plugs by Tuya so far, only energy meters and circuit breakers.

I would love to see Tuya update the integration to support HA's new energy dashboard. All my Tuya light switches and PowerPoints give me power consumption (even if it is currently wrong data)

I've corrected the power data by creating a sensor template for each device. It was painful but the data is now showing with a decimal point. ie 36.2 w
  entry_lightswitch:
    friendly_name: 'Entry - Light_power_w'
    unit_of_measurement: 'w'
    value_template: '{{(states.sensor.ty57210084500291442b65_2.state | float / 10 | round(1) )}}'

@jstott1070 unfortunately HA devs limited it to work only with energy monitoring devices. Power is not supported right now. I haven't seen energy reporting plugs by Tuya so far, only energy meters and circuit breakers.

Not sure i really understand. Currently in Tuya i can see Watts, Current and Amps but also kwhs - in HA i dont get the kwh which is really the valuable part of the monitoring.

@jstott1070 yeah, kWh is needed in HA. Could you post your logs from ha? https://github.com/tuya/tuya-home-assistant/wiki/How-to-Get-the-Log?_source=f75a22f47cc75545545c239077c07e53

I wonder if you can track the kWh using the HA Utility Meter. https://www.home-assistant.io/integrations/utility_meter/
After all kWh is just power over time.

No, utility_meter just follows what you set in source and not convert it:
# Example configuration.yaml entry utility_meter:   energy:     source: sensor.energy_in_kwh
You will be able to do this with template sensor when HA devs add option to define last_reset attribute for it.

@jstott1070 yeah, kWh is needed in HA. Could you post your logs from ha? https://github.com/tuya/tuya-home-assistant/wiki/How-to-Get-the-Log?_source=f75a22f47cc75545545c239077c07e53

So in Tuya i see the following:

However in HA the kWh is not coming through.
 Do you know what is it?

@skynetua the 22490 value is the total power usage which is indicated next to the right of the voltage in the app. This is a sum of the previous months values. I see this as 22.50 kWhs

@jstott1070 that's nice, I'll add it in the next PR for energy monitoring.

Will this mean that this value will be calculated in Tuya but passed to HA or will HA be able to use the deltas between the opening value and the current/live value to get a measure of usage within a time period (controlled by HA)? Sorry if this is a silly question - just trying to get my head around how this works.
BIG Thank you for all your hard work on this!.

Yes, HA will handle all calculations based on sensor updates delta as soon as we add support for new energy statistics.

Great stuff - thank you!..

@jstott1070 I created a "dirty" fix there https://gist.github.com/skynetua/4bcb6640582a8a730aa48e228b8ad3f9
Can you replace it with your HA file /config/custom_components/tuya_v2/sensor.py  and check if everything is ok, including new energy dashboard? It works ok with my energy meter.

Installed and sensors now available to HA Energy - no readings showing as yet but it can take some time to update so will check back tomorrow. The sensors are pulling though the total KWhs however i expect the Energy module to interpret the data.
Oddly, i cant see anyway in the Tuya App to reset the log period but thats probably for another thread elsewhere.

I'm using a work around using the Rieman sum integration   and this in my sensors.yaml for each switch

platform: integration
source: sensor.washer_power
name: washer_energy_spent
unit_prefix: k
round: 2

@jstott1070 I created a "dirty" fix there https://gist.github.com/skynetua/4bcb6640582a8a730aa48e228b8ad3f9
Can you replace it with your HA file /config/custom_components/tuya_v2/sensor.py  and check if everything is ok, including new energy dashboard? It works ok with my energy meter.

I've added the updated sensor.py
Seeing new energy sensors now

Yeah, new sensors now available and pulling info into HA Energy dash👍🏻

@skynetua I'm seeing data in the energy dashboard now. However, the figures are nowhere near accurate. One PowerPoint is reporting 100kWh. I'm certain this is a result of the previous issue of the integration not reporting the correct power data.
Even if the figure is divided by 10% it's still way higher than it should be. 28kWh for a single PowerPoint is more than my daily household use. Even 2.8 would'nt be accurate. Any ideas?

@mycrouch post your log for wrong devices.

This is the one that's showing very high
"active_time":<PHONE_NUMBER>,
"biz_type": 0,
"category": "cz",
"create_time":<PHONE_NUMBER>,
"icon": "smart/icon/ay15266175008440Qseh/2e27613c13fba9c5f64d1a44f34aa155.png",
"id": "xxxxxxxxxxxxxxxxxxxxxxxxx",
"ip": "***",
"lat": "***",
"local_key": "***",
"lon": "***",
"model": "双色LED Double Power Point",
"name": "Kitchen - Stove Power Point",
"online": true,
"owner_id": "2286669",
"product_id": "nma7ep8nzegzwyjh",
"product_name": "Double Power Point",
"status": [
{
"code": "countdown_1",
"value": 0
},
{
"code": "add_ele",
"value": 77
},
{
"code": "cur_current",
"value": 0
},
{
"code": "cur_power",
"value": 0
},
{
"code": "cur_voltage",
"value": 2272
},
{
"code": "test_bit",
"value": 1
},
{
"code": "voltage_coe",
"value": 627
},
{
"code": "electric_coe",
"value": 27888
},
{
"code": "power_coe",
"value": 15952
},
{
"code": "electricity_coe",
"value": 1305
},
{
"code": "switch_1",
"value": true
},
{
"code": "switch_2",
"value": false
},
{
"code": "countdown_2",
"value": 0
}
],
"sub": false,
"time_zone": "+10:00",
"uid": "***",
"update_time":<PHONE_NUMBER>,
"uuid": "xxxxxxxxxxxxxxxxxxxxxxxxx"

This one negative value
"active_time":<PHONE_NUMBER>,
"biz_type": 0,
"category": "cz",
"create_time":<PHONE_NUMBER>,
"icon": "smart/icon/1492863365_0.png",
"id": "xxxxxxxxxxxxxxxxxxxxxxxxx",
"ip": "***",
"lat": "***",
"local_key": "***",
"lon": "***",
"model": "单色",
"name": "Bedroom - South Power Point",
"online": true,
"owner_id": "2286669",
"product_id": "fp8li5uzdsc1cbx9",
"product_name": "Single Power Point",
"status": [
{
"code": "countdown_1",
"value": 0
},
{
"code": "add_ele",
"value": 4
},
{
"code": "cur_current",
"value": 81
},
{
"code": "cur_power",
"value": 72
},
{
"code": "cur_voltage",
"value": 2314
}
],
"sub": false,
"time_zone": "+10:00",
"uid": "***",
"update_time":<PHONE_NUMBER>,
"uuid": "xxxxxxxxxxxxxxxxxxxxxxxxx"

And this one not showing any data
"active_time":<PHONE_NUMBER>,
"biz_type": 0,
"category": "cz",
"create_time":<PHONE_NUMBER>,
"icon": "smart/icon/1492863365_0.png",
"id": "xxxxxxxxxxxxxxxxxxxxxxxxx",
"ip": "***",
"lat": "***",
"local_key": "***",
"lon": "***",
"model": "单色LED灯",
"name": "Garage - Power Point",
"online": true,
"owner_id": "2286669",
"product_id": "eeqdSYkbi9u2W3Kt",
"product_name": "Double Power Point",
"status": [
{
"code": "countdown_1",
"value": 0
},
{
"code": "add_ele",
"value": 39
},
{
"code": "cur_current",
"value": 486
},
{
"code": "cur_power",
"value": 682
},
{
"code": "cur_voltage",
"value": 2298
},
{
"code": "test_bit",
"value": 1
},
{
"code": "voltage_coe",
"value": 609
},
{
"code": "electric_coe",
"value": 30606
},
{
"code": "power_coe",
"value": 17866
},
{
"code": "electricity_coe",
"value": 1165
},
{
"code": "switch_1",
"value": false
},
{
"code": "switch_2",
"value": true
},
{
"code": "countdown_2",
"value": 0
}
],
"sub": false,
"time_zone": "+10:00",
"uid": "***",
"update_time":<PHONE_NUMBER>,
"uuid": "xxxxxxxxxxxxxxxxxxxxxxxxx"

@mycrouch another part, where value metrics are shown, like this one:
"code": "add_ele", "type": "Integer", "values": "{\"unit\":\"kwh\",\"min\":0,\"max\":100000000,\"scale\":3,\"step\":100}"

I matched the device ID for Kitchen - Stove Power Point to another part in the logs. This this what you need?
2021-08-16 19:39:53 DEBUG (SyncWorker_1) [tuya iot] Request: method = GET, url = https://openapi.tuyaeu.com/v1.0/devices/XXXXXXXXXXXXXXXXXXXX/specifications, params = None, body = None, t =<PHONE_NUMBER>639
2021-08-16 19:39:53 DEBUG (Thread-13) [tuya iot] _on_log: Received PUBLISH (d0, q0, r0, m0), 'cloud/token/in/333f69e7c10b87e0b508659c0dceca69', ... (348 bytes)
2021-08-16 19:39:53 DEBUG (Thread-13) [tuya iot] payload-> b'{"data":"kZVTUv8ckfW1fJROEVXDmax1OPfnaqTEVpIvns0Rna9EShVtTh5sqMTrtFZ7PAzZraNif1cTrVyRL/0dBeo8bur/j9U3ATTyw7kjMNwQPevRDdn99sEGU1Rz7wwWVfAjtDF3okt3gR4Qgzf5n0jeSteQ/EaV/n+Hzil8VftMkOSkkJbtDdeOgsZcyQi+kyzShDjoIWEaN5MFF9q6/NYqkHFgg3zno0W6A6j02HOkJTCG8IuhsM6njC33X97ZfYET","protocol":4,"pv":"2.0","sign":"9bbd78bada7c68e617e211ae024cfc06","t":1629106792}'
2021-08-16 19:39:53 DEBUG (Thread-13) [tuya iot] on_message: {'data': {'dataId': 'b759442e-8a0e-49bf-9da6-cee5a59cb74d', 'devId': '38042678cc50e3cd1843', 'productKey': 'XdGCESvXTI5Mvfx2', 'status': [{'code': 'cur_voltage', 't': '1629106792', '104': 2304, 'value': 2304}]}, 'protocol': 4, 'pv': '2.0', 'sign': '9bbd78bada7c68e617e211ae024cfc06', 't':<PHONE_NUMBER>}
2021-08-16 19:39:53 DEBUG (Thread-13) [tuya iot] mq receive-> {'data': {'dataId': 'b759442e-8a0e-49bf-9da6-cee5a59cb74d', 'devId': 'XXXXXXXXXXXXXXXXXXXX', 'productKey': 'XdGCESvXTI5Mvfx2', 'status': [{'code': 'cur_voltage', 't': '1629106792', '104': 2304, 'value': 2304}]}, 'protocol': 4, 'pv': '2.0', 'sign': '9bbd78bada7c68e617e211ae024cfc06', 't':<PHONE_NUMBER>}
2021-08-16 19:39:53 DEBUG (Thread-13) [tuya iot] mq _on_device_report-> [{'code': 'cur_voltage', 't': '1629106792', '104': 2304, 'value': 2304}]
2021-08-16 19:39:53 DEBUG (Thread-13) [tuya iot] _on_log: Received PUBLISH (d0, q0, r0, m0), 'cloud/token/in/333f69e7c10b87e0b508659c0dceca69', ... (412 bytes)
2021-08-16 19:39:53 DEBUG (Thread-13) [tuya iot] payload-> b'{"data":"oDIRgFd79DqOY/j6YjYl8BGbOlQUzOkQ70XYQEIYhp0gSxrQdgdDBjvFlMS0iVlyqtHUeeza/wsHW5wqcqWyH4ui7/GzwItT9zSGZeLD1MOiyKG9TETW4JxXHiGXCtFzXX93DjA+esmhHQ/C9rkGtdeQ/EaV/n+Hzil8VftMkOQ/5z8WQt4ym652AdGP48e5eksKRFv+zqhHzjXLh1XhJK+Y8O5zpo7bNNsX2hprj2AKgJK0xZqtJufSgpHewsQpF7ayPRfwVEIRsC27ULh2Wkwbl3YiPuxfOR2oZwllcxiA+tsq6Ym7tZfiP83N4NVH","protocol":4,"pv":"2.0","sign":"4a73741bf0865d5ca906ede36fddea3a","t":1629106792}'
2021-08-16 19:39:53 DEBUG (Thread-13) [tuya iot] on_message: {'data': {'dataId': '9564aedc-748d-4f6a-ab2b-97ec64449781', 'devId': 'XXXXXXXXXXXXXXXXXXXX', 'productKey': 'eeqdSYkbi9u2W3Kt', 'status': [{'code': 'cur_current', '4': 21, 't': '1629106792', 'value': 21}, {'code': 'cur_power', 't': '1629106792', '5': 39, 'value': 39}]}, 'protocol': 4, 'pv': '2.0', 'sign': '4a73741bf0865d5ca906ede36fddea3a', 't':<PHONE_NUMBER>}
2021-08-16 19:39:53 DEBUG (Thread-13) [tuya iot] mq receive-> {'data': {'dataId': '9564aedc-748d-4f6a-ab2b-97ec64449781', 'devId': 'XXXXXXXXXXXXXXXXXXXX', 'productKey': 'eeqdSYkbi9u2W3Kt', 'status': [{'code': 'cur_current', '4': 21, 't': '1629106792', 'value': 21}, {'code': 'cur_power', 't': '1629106792', '5': 39, 'value': 39}]}, 'protocol': 4, 'pv': '2.0', 'sign': '4a73741bf0865d5ca906ede36fddea3a', 't':<PHONE_NUMBER>}
2021-08-16 19:39:53 DEBUG (Thread-13) [tuya iot] mq _on_device_report-> [{'code': 'cur_current', '4': 21, 't': '1629106792', 'value': 21}, {'code': 'cur_power', 't': '1629106792', '5': 39, 'value': 39}]
2021-08-16 19:39:53 DEBUG (SyncWorker_1) [tuya iot] Response: {
"result": {
"category": "cz",
"functions": [
{
"code": "switch_1",
"type": "Boolean",
"values": "{}"
},
{
"code": "switch_2",
"type": "Boolean",
"values": "{}"
},
{
"code": "countdown_1",
"type": "Integer",
"values": "{\"unit\":\"s\",\"min\":0,\"max\":86400,\"scale\":0,\"step\":1}"
},
{
"code": "countdown_2",
"type": "Integer",
"values": "{\"unit\":\"s\",\"min\":0,\"max\":86400,\"scale\":0,\"step\":1}"
}
],
"status": [
{
"code": "switch_1",
"type": "Boolean",
"values": "{}"
},
{
"code": "switch_2",
"type": "Boolean",
"values": "{}"
},
{
"code": "countdown_1",
"type": "Integer",
"values": "{\"unit\":\"s\",\"min\":0,\"max\":86400,\"scale\":0,\"step\":1}"
},
{
"code": "add_ele",
"type": "Integer",
"values": "{\"min\":0,\"unit\":\"度\",\"scale\":0,\"max\":50000,\"step\":100}"
},
{
"code": "cur_current",
"type": "Integer",
"values": "{\"min\":0,\"unit\":\"mA\",\"scale\":0,\"max\":30000,\"step\":1}"
},
{
"code": "cur_power",
"type": "Integer",
"values": "{\"min\":0,\"unit\":\"W\",\"scale\":0,\"max\":50000,\"step\":1}"
},
{
"code": "cur_voltage",
"type": "Integer",
"values": "{\"min\":0,\"scale\":0,\"unit\":\"V\",\"max\":2500,\"step\":1}"
},
{
"code": "countdown_2",
"type": "Integer",
"values": "{\"unit\":\"s\",\"min\":0,\"max\":86400,\"scale\":0,\"step\":1}"
},
{
"code": "test_bit",
"type": "Integer",
"values": "{\"unit\":\"\",\"min\":0,\"max\":5,\"scale\":0,\"step\":1}"
},
{
"code": "voltage_coe",
"type": "Integer",
"values": "{\"unit\":\"\",\"min\":0,\"max\":1000000,\"scale\":0,\"step\":1}"
},
{
"code": "electric_coe",
"type": "Integer",
"values": "{\"unit\":\"\",\"min\":0,\"max\":1000000,\"scale\":0,\"step\":1}"
},
{
"code": "power_coe",
"type": "Integer",
"values": "{\"unit\":\"\",\"min\":0,\"max\":1000000,\"scale\":0,\"step\":1}"
},
{
"code": "electricity_coe",
"type": "Integer",
"values": "{\"unit\":\"\",\"min\":0,\"max\":1000000,\"scale\":0,\"step\":1}"

@mycrouch Yes, as you can see scale is 0 for all integer sensors. This is the reason of overscaled values. I believe you should create separate issue to discuss correct solution for this.

I've logged tickets via Tuya cloud console and I've been told my power data is not a bug, it's how the device reports it.
As per my previous posts here I've had to create a template for each entity to divide by ten percent to show correct power figures in the dashboard.
  entry_lightswitch:
    friendly_name: 'Entry - Light_power_w'
    unit_of_measurement: 'w'
    value_template: '{{(states.sensor.ty57210084500291442b65_2.state | float / 10 | round(1) )}}'

Yet my devices show correctly in Tuya Smart. If Tuya engineers tell me I don't have a problem that I clearly do have, I'm sure what else I can do except find a hack or workaround.

For energy values scale: 3 is usually used, so you should divide it to 1000 like workaround. Anyway create issue for this addon so official tuya mainterners could check it.

There is upcoming changes in state classes https://github.com/home-assistant/core/pull/54523 needed for monitoring. Lets wait for the next ha release.

@skynetua I integrated your gist but energy is not cumulative

and / or HA energy dashboard is not able to make sum.
What should be done?

@lahayeg could you post logs for this device? Seems like there is some standardization issue with add_ele component,

@skynetua  yes of course:
{
"active_time":1619082108,
"biz_type":18,
"category":"cz",
"create_time":1619082108,
"icon":"smart/icon/15324342875tjvmpp9uw8_0.png",
"id":"bfd67ca60788a7f3b995ls",
"ip":"",
"lat":"",
"local_key":"",
"lon":"",
"name":"Smart socket 4",
"online":true,
"owner_id":"7772026",
"product_id":"vnya2spfopsh9lro",
"product_name":"Smart socket",
"status":[
{
"code":"switch_1",
"value":true
},
{
"code":"countdown_1",
"value":0
},
{
"code":"add_ele",
"value":23
},
{
"code":"cur_current",
"value":7
},
{
"code":"cur_power",
"value":7
},
{
"code":"cur_voltage",
"value":2370
}
],
"sub":false,
"time_zone":"+02:00",
"uid":"***",
"update_time":1629311369,
"uuid":"8c2abe7a62764c4f"
}

@lahayeg let's test it in the next HA release. Changes in calculation and reset time  have been already made.

Unfortunately I can see the sensors, but I cannot add them in the energy menu, they are not present in the drop-down menu. Any idea?

@ripperhack this hasn't been fixed yet, please wait for the next update.

@jstott1070 I created a "dirty" fix there https://gist.github.com/skynetua/4bcb6640582a8a730aa48e228b8ad3f9
Can you replace it with your HA file /config/custom_components/tuya_v2/sensor.py  and check if everything is ok, including new energy dashboard? It works ok with my energy meter.

nice !!! thanks a lot !!!!!

@msteingr assuming your running Tuya V2 then you should apply @skynetua's "dirty fix" for now until it's fully incorporated into an update.
"@jstott1070 I created a "dirty" fix there https://gist.github.com/skynetua/4bcb6640582a8a730aa48e228b8ad3f9
Can you replace it with your HA file /config/custom_components/tuya_v2/sensor.py and check if everything is ok, including new energy dashboard? It works ok with my energy meter."

Hello @jstott1070  i copied the dirtyfix and was able to add the tuya into my energy dasbard , but the value looks faulty, for example the tuya app shows me a consumption of 2 KWH for yesterday (30/8) and the intehgration in home assistante shows 0.03 KWH for the same date..
Any suggestion?

@msteingr assuming your running Tuya V2 then you should apply @skynetua's "dirty fix" for now until it's fully incorporated into an update.
"@jstott1070 I created a "dirty" fix there https://gist.github.com/skynetua/4bcb6640582a8a730aa48e228b8ad3f9
Can you replace it with your HA file /config/custom_components/tuya_v2/sensor.py and check if everything is ok, including new energy dashboard? It works ok with my energy meter."

Hello @jstott1070 i copied the dirtyfix and was able to add the tuya into my energy dasbard , but the value looks faulty, for example the tuya app shows me a consumption of 2 KWH for yesterday (30/8) and the intehgration in home assistante shows 0.03 KWH for the same date..
Any suggestion?

I have the same problem,

an other question dont know if ther is an other topic.. but i bought as well an sonoff PWR2R and i am not able to include the enegry to my energy dashboard any suggestion?

Hello @skynetua
Issue remain even after update 2021.9.0

I am having the same issue. Tho in Energy Config i am getting this error

Also how useful are these smart plugs? Do they record how much energy is used between readings? Currently im getting readings every hour(ish), does it know i turned a high drain heater on for 10 minutes during that time?

@lahayeg energy dashboard operates with total value sensors now. You can create utility_meter based on this sensor and add it to the dashboard. It will automatically calculate total usage.
https://www.home-assistant.io/integrations/utility_meter/

Am I missing something?

I have added the DirtyFix  but still can't see any of my plugs in Energy menu
{
  "result": {
    "biz_type": 0,
    "category": "cz",
    "product_id": "cET7nrXx83igNwGF",
    "product_name": "BW-SHP12",
    "status": [
      {
        "code": "switch",
        "value": true
      },
      {
        "code": "countdown_1",
        "value": 0
      },
      {
        "code": "cur_current",
        "value": 0
      },
      {
        "code": "cur_power",
        "value": 0
      },
      {
        "code": "cur_voltage",
        "value": 0
      }

}
I believe its missing the key `add_ele` in the JSON. Will that be added through Tuyas side eventually?

@lahayeg energy dashboard operates with total value sensors now. You can create utility_meter based on this sensor and add it to the dashboard. It will automatically calculate total usage.
https://www.home-assistant.io/integrations/utility_meter/

@skynetua tried utility_meter but does not report same value than smart life app

an other question dont know if ther is an other topic.. but i bought as well an sonoff PWR2R and i am not able to include the enegry to my energy dashboard any suggestion?

Hi @msteingr I just updated the discussion topic here -> https://github.com/AlexxIT/SonoffLAN/discussions/569 - hope it helps.

From what i observed about the add_ele DP Code, the device on a regular basis (from my graphs - every ten mins when on i think) reports the delta consumed energy (from previous reporting time) to be added to the total energy. I suppose the Tuya API adds this delta energy to the total energy that you see in the app (and also adds under daily and monthly counters).
To implement the same in HA, one can create a MQTT sensor (says Total_ele) and add an automation (on state change of add_ele) to the add the add_ele to the Total_ele and publish to Total_ele MQTT sensor. Then use a utility meter based on Total_ele sensor to create daily/weekly/monthly sensors as needed.
The only problem i see with the automation is that if the delta energy between two reporting period is same, HA will not see it as a state change, the automation will not be triggered.
While defining the Total_ele MQTT sensor, you should include the state class as "total_increasing", so it will be included for long term stats and can be added to energy dashboard.

@madtech360 The issue is that different mnufactures use add_ele datapoint for different values. For someone above it already reports total as desired.
Tuya documentation states:

Increase power | add_ele |  Value range: 0 to 50000, interval: 100, multiple: 3, unit: kwh | Required

Power statistics related functions

Increased power: It refers to the power consumption of the device. Each time the “incremental” is reported.

@tsutsuku do you have any thoughts about standartizing this?

@madtech360 : I concluded the same yesterday and implemented it using a slightly different approach to an MQTT sensor - it appears that the update interval for the add_ele entry in the device log is different - based on how much energy the devices connected to the plug are utilizing.
I can also confirm that the problem you highlighted of automation not always triggering - this was visible on 1 device with a more or less standard power draw did not trigger on 2 occasions and thus total values were different.
It did however work well / accurately on another plug where the connected device's energy consumption fluctuates.

@skynetua : It is possible there is non-standardization here. I see for the few of the users, the add_ele seems to be reporting the delta energy. It seems to be working fine for now for me. I am going to wait and observe for few more days to check if HA calculated consumption is same as the total reported in the app.
@nduzaone: Thanks for confirming. Curious about your different approach to MQTT sensor. My HA knowledge is limited.
BTW with the current code in the master branch, i had to tweak the sensor.py to be able to add all the sensors to HA ( see issue #449)

@JinksKid  i have the same issue -  the dirty fix makes the smart plugs (gosuna UP111 in my case) show up, but i get the same error "The following entities do not have the expected state class 'total_increasing" "sensor.ty16005218c4dd572f4c50_4 (measurement)"

@skynetua  utiliy_meter is not suming sensor value when it's decreasing.
Reading the STATE_CLASS_TOTAL_INCREASING documentation seems covering it. How could I modify the "dirty fixe" in order to test?

What I ended up doing is to follow what was done here:  2021.8.0: Feel the energy  https://community.home-assistant.io/t/2021-8-0-feel-the-energy/326853/407 ⚡ - Blog - Home Assistant Community (home-assistant.io)
So add this to your config yaml
sensor:

platform: integration
source: sensor.dryer_power
name: energy_dryer_kwh
unit_prefix: k
round: 2

Change the sensor to be the power sensor from tuya
Then I can add this into my energy monitor no issue
From: Guillaume Lahaye @.>
Sent: 13 September 2021 10:35
To: tuya/tuya-home-assistant @.>
Cc: aberkeley @.>; Comment @.>
Subject: Re: [tuya/tuya-home-assistant] Support for Home Assistant Energy dashboard (#307)
@skynetua https://github.com/skynetua  utiliy_meter is not suming sensor value when it's decreasing.
Reading the STATE_CLASS_TOTAL_INCREASING documentation https://developers.home-assistant.io/blog/2021/08/16/state_class_total/  seems covering it. How could I modify the "dirty fixe" in order to test?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/tuya/tuya-home-assistant/issues/307#issuecomment-918012276 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ARQGZX4GSM3OP476O3HRWFLUBXATPANCNFSM5BR75LMQ .
Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675  or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub .   https://github.com/notifications/beacon/ARQGZXYGDJRK45S42F7GKETUBXATPA5CNFSM5BR75LM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOG234C5A.gif

@lahayeg you can change line 523 to
return "total_increasing"

What I ended up doing is to follow what was done here: 2021.8.0: Feel the energy https://community.home-assistant.io/t/2021-8-0-feel-the-energy/326853/407 ⚡ - Blog - Home Assistant Community (home-assistant.io) So add this to your config yaml sensor: - platform: integration source: sensor.dryer_power name: energy_dryer_kwh unit_prefix: k round: 2 Change the sensor to be the power sensor from tuya Then I can add this into my energy monitor no issue From: Guillaume Lahaye @.> Sent: 13 September 2021 10:35 To: tuya/tuya-home-assistant @.> Cc: aberkeley @.>; Comment @.> Subject: Re: [tuya/tuya-home-assistant] Support for Home Assistant Energy dashboard (#307) @skynetua https://github.com/skynetua utiliy_meter is not suming sensor value when it's decreasing. Reading the STATE_CLASS_TOTAL_INCREASING documentation https://developers.home-assistant.io/blog/2021/08/16/state_class_total/ seems covering it. How could I modify the "dirty fixe" in order to test? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#307 (comment)> , or unsubscribe https://github.com/notifications/unsubscribe-auth/ARQGZX4GSM3OP476O3HRWFLUBXATPANCNFSM5BR75LMQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . https://github.com/notifications/beacon/ARQGZXYGDJRK45S42F7GKETUBXATPA5CNFSM5BR75LM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOG234C5A.gif

Already tried (with the 3 methods) and does not report same result that app

@lahayeg you can change line 523 to
return "total_increasing"
Probably it will not work as you want anyway.
The better solution in this case is Statistics integration
It can calculate total for source sensor.

I will try but seems that already implement it in your PR !!

Already tried (with the 3 methods) and does not report same result that app

@lahayeg : Did you try to approach i have mentioned here?
https://github.com/tuya/tuya-home-assistant/issues/307#issuecomment-915938640
Works without issues for me and for @nduzaone   https://github.com/tuya/tuya-home-assistant/issues/307#issuecomment-915960900
Some screenshots here:
https://github.com/tuya/tuya-home-assistant/pull/464#issuecomment-919156202
If you need yaml of my setup, I can share.

Am I missing something?

Anyone knows why my Tuya breaker is not showing as a source in the Energy config please? Thanks,

Already tried (with the 3 methods) and does not report same result that app

@lahayeg : Did you try to approach i have mentioned here?
#307 (comment)
Works without issues for me and for @nduzaone #307 (comment)
Some screenshots here:
#464 (comment)
If you need yaml of my setup, I can share.

yaml gist -> Yes please!

Thanks for confirming. Curious about your different approach to MQTT sensor. My HA knowledge is limited.

@madtech360 Apologies for the late reply, my approach was probably more roundabout and likely not the most efficient but I get results.
Just 2 things I have noted - which can help some people -

Time Zone Differences - I am in Africa / Harare - my Tuya application is linked to the Central Europe Data Center - looking at the logs there's a 2 Hour difference between readings in HA vs the Tuya IOT Platform
Misalignment of add_ele readings - I extracted full readings from both platforms to compare and noted that in some cases HA had 2 extra entries and the Tuya IOT Platform duplicated 1 of the entries.  Refer to the attachment below:

I have concluded that my readings may not match up completely because of the 2 items above - but they're still close enough for my information tracking purposes.
Approach
After realizing that add_ele was reporting the usage since the last update, like you, I figured the only way was to take each of those entries add them up and get the final total per day.
Step 1
I created 3 placeholders using the Input Number integration, (1) To hold the value of the current reading (2) To hold the Previous Total (3) To hold the Total for the day.
Step 2
Each time the usage value was picked from the device logs,  an automation performs the following actions in succession:
—> copy the value from (3) into (2)
—> then copy the current add_ele value from the sensor created by the fix into (1)
—> add the value in (1) to (2) and move that value into (3)
Step 3
I also created a sensor that takes the value of (3) and displays it in kWh
Step 4
Then created the daily/monthly/yearly sensors using utility_meter
Step 5
Finally created an automation that runs every day at 00:00:00 to set all placeholders in (1) above to 0

@lahayeg
Here is my YAML config from Senor, Automation and Utility Meter:
MQTT Sensor to keep track of Total Consumed Energy -> Equivalent to Total Ele(kWh) in the Smart Life App
- platform: mqtt
  name: "Fridge Consumed Energy"
  state_topic: "/home/energy/fridge"
  value_template: "{{ value_json.energy | float }}"
  unit_of_measurement: "kWh"
  device_class: energy
  state_class: total_increasing

Note: I have set the state class of this MQTT sensor to total_increasing. This makes the sensor eligible for long term statistics and also makes it available in the energy dashboard.
Automation to add the Add Ele to Total Consumed Energy
- alias: Tuya Fridge Add Ele
  trigger:
  - platform: state
    entity_id: sensor.Fridge_Add_Ele
  condition: []
  action:
  - service: mqtt.publish
    data:
      topic: /home/energy/fridge
      retain: true
      payload: >-
        {"energy":{{
        states('sensor.fridge_consumed_energy') | float +
        states('sensor.Fridge_Add_Ele') | float }}
        }
  mode: single

Note: Replace sensor.Fridge_Add_Ele with the Add Ele sensor created from the dirty fix. I am not using the dirty fix, i created my own fix which i have included in the PR #452
Known Issues:

If the Delta consumed energy between to reporting period is same, HA will suppress the state change and automation will not trigger. You will loose this delta
If the Device goes offline (power failure or wifi loss) and comes back after sometime, the Automation is triggered as the state of Add_Ele goes from unknown to some numeric value. This will result in duplicate delta being added. The automation should be enhanced to check from trigger.from_state to make sure it is not unknown.

Utility Meter to keep track of Daily and Montly Consumed Energy -> Equivalent to the Monthly Counter and Daily Counter(graph) in the Samrt Life App
utility_meter:
  fridge_daily_energy:
    source: sensor.fridge_consumed_energy
    name: Fridge Daily Energy
    cycle: daily
  fridge_monthly_energy:
    source: sensor.fridge_consumed_energy
    name: Fridge Monthly Energy
    cycle: Monthly

Please look at https://github.com/tuya/tuya-home-assistant/issues/495

since the commits regarding energy haven't made it to a new release yet, I overrode sensors.py from master to HA but it breaks everything. Before the plugs were reporting W, A, V whereas with new sensor.py those sensors become completely unavailable and no new sensors for kWh are created

@drthanwho you can replace it with this till the next update

thanks a lot @skynetua that seems to work perfectly! Are we ever going to see that in the Core version that was just added in HA? That one seems to be even more behind since my plugs don't even show power, voltage and current when I tried to migrate to that one

I believe it will be merged to the core before the next release.

I am guessing this wasn't merged to the core (official) version? Migrated from V2 to official and lost all power data on the plugs, only showing a On/Off toggle now

It has, will be released with Home Assistant 2021.11

Looks like there's voltage/current/power information in 2021.11, but not not energy consumption information (ie, kWh).
I thought the new Tuya HA integration was just pulling in this Tuya v2, but I'm guessing it's not quite that simple since https://github.com/tuya/tuya-home-assistant/pull/464 looks like it added energy consumption back in September to this repo but it's nowhere to be seen in the HA/core repo.

Ditto, now got the addition sensors but nothing that works with Energy. Oddly each plug does have a sensor that can be added the Energy but they don’t work and their naming is odd.
Get Outlook for iOShttps://aka.ms/o0ukef

From: Tiernan @.>
Sent: Sunday, November 7, 2021 12:32:50 PM
To: tuya/tuya-home-assistant @.>
Cc: jstott1070 @.>; Mention @.>
Subject: Re: [tuya/tuya-home-assistant] Support for Home Assistant Energy dashboard (#307)
I am guessing this wasn't merged to the core (official) version? Migrated from V2 to official and lost all power data on the plugs, only showing a On/Off toggle now
It has, will be released with Home Assistant 2021.11
Looks like there's voltage/current/power information in 2021.11, but not not energy consumption information (ie, kWh).
I thought the new Tuya HA integration was just pulling in this Tuya v2, but I'm guessing it's not quite that simple since #464https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftuya%2Ftuya-home-assistant%2Fpull%2F464&data=04|01||c51f17a6d4974c7b5ba208d9a1eab6ba|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637718851719138640|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=3pf1%2BZmD7rh8bFk7ndVnIfCN431Z%2FY5Gkwx3Om%2B2fmI%3D&reserved=0 looks like it added energy consumption back in September to this repo but it's nowhere to be seen in the HA/core repo.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftuya%2Ftuya-home-assistant%2Fissues%2F307%23issuecomment-962602632&data=04|01||c51f17a6d4974c7b5ba208d9a1eab6ba|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637718851719148583|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=L785Thns8pJJDrIuPa6fjPj0de4bq1kz5BRkUR4Cqlk%3D&reserved=0, or unsubscribehttps://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FASP2VIHGQTTK6U2YPIWC6HLUKZWXFANCNFSM5BR75LMQ&data=04|01||c51f17a6d4974c7b5ba208d9a1eab6ba|84df9e7fe9f640afb435aaaaaaaaaaaa|1|0|637718851719148583|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|1000&sdata=0CJXtTcRXsaLQlmjmc9EZAr8ADxqGgkCNbnsMIANSXM%3D&reserved=0.