fix: [#8576] Field loses focus after keystroke

#minor
Description
This PR fixes the issue with the lost focus on the Prompt action's text fields after the first keystroke.
The NodeWrapper component has a condition that evaluates the selected nodes, and tabs to decide if the focus() function should be called. In the re-render of the component, this condition was evaluated true when it shouldn't, causing a unnecesary re-focus.
We updated the flags used in this condition to call the focus() function only when the tab and the node are selected.
Task Item
Fixes # 8576
Screenshots
Here we can see the focus working as expected in the Prompt action after the changes.
https://user-images.githubusercontent.com/44245136/218124309-aed8437e-f722-4a24-adf5-21d98cb24ec8.mp4

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

1 of 1 (100.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.001%) to 54.554%

Totals

Change from base Build<PHONE_NUMBER>:
-0.001%

Covered Lines:
19675

Relevant Lines:
31698

💛  - Coveralls

/promoted 9509


Document that the GitHub Actions shell examples assume what you set in run are saved to a temporary file

What article on docs.github.com is affected?
https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#using-a-specific-shell
What part(s) of the article would you like to see updated?
The table of shells gives examples of how they are used, but they are all from the perspective of passing in a file. I think clarifying that the text in the run key is saved to a temp file and the example command is used on that temp file would help make the examples make more sense.
Additional information
For instance, the Python example is:
steps:
  - name: Display the path
    run: |
      import os
      print(os.environ['PATH'])
    shell: python

And the shell docs say that Python uses python {0}. That would only work if what is specified in run was saved to a temp file.

Thanks for opening an issue @brettcannon! I'll triage this for the team to take a look 👀

Thanks @brettcannon! You or anyone else is welcome to open a PR clarifying this.

Hi, I added what I hope is an appropriate change to clarify that the scripts are run from a temporary file. Not 100% sure on the wording, so I am open to comments on #7824 before anything else.


KeyError when installing Qt 5.12.3 linux desktop

I'm getting the following error, which just started happening today, on multiple machines and using the last 3 versions of aqtinstall.
$ aqt install -O ~/Qt 5.12.3 linux desktop
Traceback (most recent call last):
  File "/home/sam/tempenv/lib/python3.7/site-packages/importlib_metadata/__init__.py", line 170, in __getitem__
    return next(iter(self.select(name=name)))
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/sam/tempenv/bin/aqt", line 10, in <module>
    sys.exit(main())
  File "/home/sam/tempenv/lib/python3.7/site-packages/aqt/__init__.py", line 38, in main
    return cli.run()
  File "/home/sam/tempenv/lib/python3.7/site-packages/aqt/installer.py", line 475, in run
    return args.func(args)
  File "/home/sam/tempenv/lib/python3.7/site-packages/aqt/installer.py", line 214, in run_install
    self._run_common_part(output_dir, base)
  File "/home/sam/tempenv/lib/python3.7/site-packages/aqt/installer.py", line 157, in _run_common_part
    self.show_aqt_version()
  File "/home/sam/tempenv/lib/python3.7/site-packages/aqt/installer.py", line 362, in show_aqt_version
    module_name = dist.entry_points[0].name
  File "/home/sam/tempenv/lib/python3.7/site-packages/importlib_metadata/__init__.py", line 172, in __getitem__
    raise KeyError(name)
KeyError: 0

Any help much appreciated.

Issue #222 is the same. Sorry for opening a dupe. I literally noticed this at about the same time (several hours ago), and I checked for open bugs, found none. Then I was researching it on and off for a couple hours before opening my issue.  I didn't refresh the issues list during that time, so I missed the fact that your ticket appeared.
Anyway we're in the (proverbial) same boat.
I wrote a "hack-around" that you might find useful:
sudo sed -i "s/        module_name = dist.entry_points\[0\]\.name/        module_name = 'HACKED'/" /usr/local/lib/python3.6/dist-packages/aqt/installer.py

Although your path would be /home/sam/tempenv/lib/python3.7/site-packages/aqt/installer.py

I'm going to paste my other content here and close my duplicate issue.

Fatal uncaught exception "aqt/installer.py", line 362, in show_aqt_version. Related: python3.6/dist-packages/importlib_metadata
https://github.com/miurahr/aqtinstall/blob/2f17a7d42c5d3c0c0f0d61bae36a2c721d4d25e1/aqt/installer.py#L362
I'm not confident to say whether or not this is a bug in aqtinstall or not.
Other possibilities:

something wrong with py 3.6 importlib_metadata
something wrong with the packaging or dist-info or "entry_points.txt" that pip downloads

This started failing for me "out of nowhere."  It was working in one CI job, then the next time I ran the same job (with no changes to my code or build scripts), this failure happened.
I have a way to replicate it using Docker (see below).
This is the error:
        + python3 -m aqt install --base http://mirrors.ocf.berkeley.edu/qt/ --outputdir /opt/atlassian/pipelines/agent/build/dl_third_party/Qt_desktop 5.15.0 linux desktop --archives icu qtbase qtconnectivity qtdeclarative qtgraphicaleffects qtimageformats qtquickcontrols qtquickcontrols2 qtsvg qttools
        Traceback (most recent call last):
          File "/usr/local/lib/python3.6/dist-packages/importlib_metadata/__init__.py", line 170, in __getitem__
            return next(iter(self.select(name=name)))
        StopIteration
        During handling of the above exception, another exception occurred:
        Traceback (most recent call last):
          File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
            "__main__", mod_spec)
          File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
            exec(code, run_globals)
          File "/usr/local/lib/python3.6/dist-packages/aqt/__main__.py", line 27, in <module>
            sys.exit(main())
          File "/usr/local/lib/python3.6/dist-packages/aqt/__init__.py", line 38, in main
            return cli.run()
          File "/usr/local/lib/python3.6/dist-packages/aqt/installer.py", line 475, in run
            return args.func(args)
          File "/usr/local/lib/python3.6/dist-packages/aqt/installer.py", line 214, in run_install
            self._run_common_part(output_dir, base)
          File "/usr/local/lib/python3.6/dist-packages/aqt/installer.py", line 157, in _run_common_part
            self.show_aqt_version()
          File "/usr/local/lib/python3.6/dist-packages/aqt/installer.py", line 362, in show_aqt_version
            module_name = dist.entry_points[0].name
          File "/usr/local/lib/python3.6/dist-packages/importlib_metadata/__init__.py", line 172, in __getitem__
            raise KeyError(name)
        KeyError: 0

Here is a docker file that will reproduce the failure:
FROM ubuntu:18.04

RUN apt-get update

RUN apt-get --assume-yes install  python3
RUN apt-get --assume-yes install  python3-pip

RUN pip3 install -Iv aqtinstall==1.1.2

RUN python3 -m aqt install --base  --outputdir /tmp/Qt 5.15.0 linux desktop --archives icu qtbase qtconnectivity qtdeclarative qtgraphicaleffects qtimageformats qtquickcontrols qtquickcontrols2 qtsvg qttools

Versioning clue??   If you simply replace this:
FROM ubuntu:18.04

With this:
FROM ubuntu:20.04

And re-run docker build, then you can see it succeeds on Ubuntu 20.

Possibly related: https://github.com/python/importlib_metadata/pull/278

If I do this, then I can once again run aqt as before:
pip3 uninstall importlib_metadata
pip3 install -Iv importlib_metadata==3.5.0

The entry_points uncaught exception reappears if I use  importlib_metadata==3.6.0 or  importlib_metadata==3.7.0


Cannot install on Node 8.9.4

Hi i ran  these commands
apt update
apt install nodejs
npm i -g web-code

attached the error.

The readme in this repo is not complete. Read this article instead:
https://medium.com/samsung-internet-dev/writing-software-using-a-phone-e71976f1f18d
You should do something like:
termux-fix-shebang `which npm`

hi i tried
termux-fix-shebang `which npm`

and

termux-fix-shebang `which web-code`

both gave the same error output (See the above screenshot)

Try to re-do everything step by step based on the article link which I posted above. It works for me.

hey @tanatospl thanks for the tips. i've uninstalled and cleared the old files/folders, went through all the steps but i'm still getting the same errors. Can i check which node version are you on?

npm -v
5.5.1
node -v
v8.9.3

thanks. i can confirm that this is not working on 9.5.0 too.

In your cli it should be 'apt-get' rather than 'apt get'.
I'll test it with recent node versions. Thanks for bringing this to mu attention.
@tanatospl if you want to add anything to the readme please make any changes you like I'll merge the PRs :).

@AdaRoseCannon just curious, have you tested with the latest node versions?

I haven't had a chance yet. I have been really busy with other projects.
On Fri, 23 Feb 2018, 10:20 Irvin Lin<EMAIL_ADDRESS>wrote:

@AdaRoseCannon https://github.com/adarosecannon just curious, have you
tested with the latest node versions?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/SamsungInternet/web-code/issues/22#issuecomment-367969110,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEB5MtPLPK7tVl5CiBqVT98O-Q8Dc8uVks5tXpDagaJpZM4R6fj1
.

This should now be fixed, let me know if it is still not working and I will re open this ticket. :)


Changing the runtime user to "root"

On both a Raspberry pi 4, and a Raspberry pi Zero W, the application will not start at boot unless the User=root is set in the service file.
pi@raspberrypi:~ $ cat /etc/os-release
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"

Service output:
pi@raspberrypi:~ $ sudo systemctl status busylight.service
● busylight.service - BusyLight
   Loaded: loaded (/etc/systemd/system/busylight.service; enabled; vendor preset: enabled)
   Active: failed (Result: signal) since Mon 2020-09-28 19:33:13 BST; 13s ago
  Process: 673 ExecStart=/usr/bin/python3 server.py (code=killed, signal=SEGV)
 Main PID: 673 (code=killed, signal=SEGV)

Sep 28 19:33:10 raspberrypi systemd[1]: Started BusyLight.
Sep 28 19:33:11 raspberrypi python3[662]: Can't open /dev/mem: Permission denied
Sep 28 19:33:13 raspberrypi systemd[1]: busylight.service: Service RestartSec=100ms expired, scheduling restart.
Sep 28 19:33:13 raspberrypi systemd[1]: busylight.service: Scheduled restart job, restart counter is at 5.
Sep 28 19:33:13 raspberrypi systemd[1]: Stopped BusyLight.
Sep 28 19:33:13 raspberrypi systemd[1]: busylight.service: Start request repeated too quickly.
Sep 28 19:33:13 raspberrypi systemd[1]: busylight.service: Failed with result 'signal'.
Sep 28 19:33:13 raspberrypi systemd[1]: Failed to start BusyLight.

Starting as pi

pi@raspberrypi:~/unicorn-busy-server $ ./start.sh
Can't open /dev/mem: Permission denied
Traceback (most recent call last):
  File "./server.py", line 12, in <module>
    from lib.unicorn_wrapper import UnicornWrapper
  File "/home/pi/unicorn-busy-server/lib/unicorn_wrapper.py", line 6, in <module>
    import unicornhat
  File "/usr/local/lib/python3.7/dist-packages/unicornhat.py", line 35, in <module>
    ws2812.begin()
  File "/usr/local/lib/python3.7/dist-packages/rpi_ws281x/rpi_ws281x.py", line 131, in begin
    raise RuntimeError('ws2811_init failed with code {0} ({1})'.format(resp, str_resp))
RuntimeError: ws2811_init failed with code -5 (mmap() failed)
./start.sh: line 1:   692 Segmentation fault      python3 ./server.py

As user root:
pi@raspberrypi:~/unicorn-busy-server $ sudo vi /etc/systemd/system/busylight.service
pi@raspberrypi:~/unicorn-busy-server $ grep User /etc/systemd/system/busylight.service
User=root
pi@raspberrypi:~/unicorn-busy-server $ sudo systemctl daemon-reload
pi@raspberrypi:~/unicorn-busy-server $ sudo systemctl start busylight
pi@raspberrypi:~/unicorn-busy-server $ sudo systemctl status busylight.service
● busylight.service - BusyLight
   Loaded: loaded (/etc/systemd/system/busylight.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2020-09-28 19:35:19 BST; 4s ago
 Main PID: 747 (python3)
   CGroup: /system.slice/busylight.service
           └─747 /usr/bin/python3 server.py

Sep 28 19:35:19 raspberrypi systemd[1]: Started BusyLight.
Sep 28 19:35:23 raspberrypi python3[747]:  * Serving Flask app "server" (lazy loading)
Sep 28 19:35:23 raspberrypi python3[747]:  * Environment: production
Sep 28 19:35:23 raspberrypi python3[747]:    WARNING: This is a development server. Do not use it in a production deployment.
Sep 28 19:35:23 raspberrypi python3[747]:    Use a production WSGI server instead.
Sep 28 19:35:23 raspberrypi python3[747]:  * Debug mode: off
Sep 28 19:35:23 raspberrypi python3[747]:  * Running on http://<IP_ADDRESS>:5000/ (Press CTRL+C to quit)

Thanks @maglub haven't experienced any problems with it, but good that you found a solution.


Identifier 'load' has already been declared

I am trying to setup the library in my project. I have added it into my +layout.server.js file like this:
export { loadFlash  } from 'sveltekit-flash-message/server';
export const load = loadFlash(async ({ locals,fetch}) => {
...

However i get the following error:
Internal server error: Parse failure: Identifier 'load' has already been declared (15:13)
At file: /src/routes/+layout.server.js
Contents of line 15: export const load = loadFlash(async (

I also have a +layout.js file with a load function. could it be because of that?
i am using  v2.2.1 and "@sveltejs/kit": "^1.26.0",

my bad - i was  using:
export { loadFlash } from 'sveltekit-flash-message/server';
instead of
import { loadFlash } from 'sveltekit-flash-message/server';
unfortunately i still have the error ;)

i tried to comment everything out of my hooks file and also the +layout.js file, but no help. really really strange. any idea what could be the problem?

Not really, try starting over and follow the instructions closely to see if you can get a minimal example working.

i found the problem. it is houdini. once i add https://houdinigraphql.com/ and adapt the vite.config file to use it the problem occurs. i think it is because houdini also is intercepting the load function already somehow. damn, that is too bad. especially as i don't know if this is something that you or the houdini team can or will fix :P

i created a PR for houdini with a fix :)

Just got hit by this, after so many weeks pushing off using using this library, the moment I do someone sends a PR to fix this :)
In the meantime, you could however simply recreate the behavior of loadEvent by doing exactly what the library does :
https://github.com/ciscoheat/sveltekit-flash-message/blob/662156fe9b3a04d197538b2e5cf61284c1db6d63/src/lib/server.ts#L30-L36
In other words, simply do this :
import { _loadFlash } from 'sveltekit-flash-message/server';

export const load = (async (event) => {
	const flash = _loadFlash(event).flash;

	// [whatever you need to do]

	return {
		...{other: 'data'},
		flash,
	};
}

and use this library like usual.
Looking out for the PR to be merged if it fixes this use case, the method call is fancier looking and I'd rather switch to it myself!

oh nice! great that is also works with this workaround. i can confirm that my PR makes it work as well. just waiting on the merge. you can theoretically also just npm install my fork if you want to directly go ahead, but if your workaround works, then there is no hurry i guess :)

I saw that the PR is merged, so I'm closing this!


TypeScript error when using hashHistory

TypeScript catch an error when using hashHistory in routerMiddleware.
error TS2346: Supplied parameters do not match any signature of call target.
And from the IDE:
Argument of type 'History' is not assignable to parameter of type 'History'. Property 'length' is missing in type 'History'
/* ROUTER */
import { hashHistory } from 'react-router'
import { routerMiddleware, syncHistoryWithStore } from 'react-router-redux';

/* MIDDLE WARES */
const middleWares = [thunkMiddleware, routerMiddleware(hashHistory)];

What should I do ?
Thanks for all the work here.

This isn't a bug in this library. If you're having problems with history typings, you should go to the project wherever those came from (Definitively Typed or somewhere else).


longitudinal MPC: add MAX_T in T_IDXS_LST

MAX_T (10.0) is missing in T_IDXS_LST, which is
[0.0, 0.059171597633136105, 0.23668639053254442, 0.5325443786982249, 0.9467455621301777, 1.4792899408284024, 2.1301775147928996, 2.8994082840236683, 3.7869822485207107, 4.792899408284023, 5.9171597633136095, 7.159763313609467, 8.520710059171599]

This PR adds MAX_T in T_IDXS_LST:
[0.0, 0.06944444444444445, 0.2777777777777778, 0.625, 1.1111111111111112, 1.7361111111111114, 2.5, 3.4027777777777786, 4.444444444444445, 5.625, 6.9444444444444455, 8.402777777777777, 10.0]

LGTM

Moved to #23960 since I wasn't able to push the updated refs to this branch.


massConvert command is not working for table names with lower case letters

Hi @jung-thomas ,
I am not able to make "massConvert" command work for tables with case sensitive names or lowercase letters.
Default option for tables which is "*" also is not working in this scenario. It is working perfectly fine for table names with upper case letters only.
I have hundreds of tables in different schemas with lowercase and case sensitive naming which are created by .hdbdd files.
Can you help me to figure out the solution?
Command used:
hana-cli massConvert -a -s "SCHEMANAME" -o "hdbtable"
Error is:
Schema: SCHEMANAME, Table: *
{ code: 397,
  message:
   'invalid object name: SCHEMANAME.COM.MYCOMPANY.DB::VENDORS has an unknown type.',
  sqlState: 'HY000' }
TypeError: Cannot read property 'stop' of undefined
    at process.errorHandler (C:\Users\SUCHEN\Documents\Suchen Workspaces\vscode files\hana-developer-cli-tool-example\bin\cli.js:14:22)
    at process.emit (events.js:198:13)
    at emitPromiseRejectionWarnings (internal/process/promises.js:119:20)
    at process._tickCallback (internal/process/next_tick.js:69:34)
C:\Users\SUCHEN\Documents\Suchen Workspaces\vscode files\hana-developer-cli-tool-example\bin\cli.js:14
    global.__spinner.stop()

Thanks.

@sucheno - thanks for reporting this issue. I've just committed a fix for this issue. I've pushed to NPM. Version 1.202009.1 and higher will contain this fix.


Sanity checks for uploads

Should we define what counts as a minimum valid upload ("at least one 3d file")
Should we disallow file types we don't expect
File size limits? Limits on number of files?
Extra checks on images - it's a cheap way to sneak in arbitrary image uploads to get free hosting. Can we add some rules to ensure they actually are intended as textures?

(4) is the trickiest

(4) is also format dependent.

Some formats never or rarely have external images. (not sure but I think fbx images are always embedded in the fbx)
Some formats allow embedded images so maybe we should insist upon it
Some formats don't support embedded images but are easy to check (obj)

Still fairly easy for people to find loopholes so maybe not worth overthinking this.


Search by multi channel audio

Would be nice to search by multi channel audio like DTS/DD51/DD71. Sure I can do it via RegEx, but a select would be cool.

Idea is welcome. Not an issue, though.

Its on my "if easy, implement" list. Else RegEx is your friend

@rix1337 if re.match(r'.*\.(DTS|DD\+*51|DD\+*71|AC3\.5\.*1)\..*', title):


KONFLUX-3386 emit the commit timestamp

Emit the commit timestamp as a result, this allows for subsequent tasks to consume it to get better reproducability.

/retest

Note, after some discussion #1040 with the same intent was closed.

This is needed to unblock RHEL AI

Rebased

Rebased

/retest


Speed up script

I've found your script on https://www.haegi.org/2018/07/; thanks for sharing it! 😃
I noticed that it loops over the entire file, even though its job is already finished once the MP4 start is found. This pull requests therefore adds a break statement after the mp4 file extraction, which makes the script ~3× faster for my test image. (In addition, it improves the way the filename parameter is passed to the extract function. 😉)

Thank you for your contribution :-)

Thanks for the quick merge! 😉


Fix vehicle_types.json wrongly required #85

Fix #85

A station_id field on vehicles in free_bike_status.json will not mark vehicle_types.json anymore as required, as the spec doesn't require it.

Code cleaning:

freeBikeStatusHasVehicleId var is renamed freeBikeStatusHasVehicleTypeId to better reflect what it check.

https://gbfs.nextbike.net/maps/gbfs/v2/nextbike_le/gbfs.json is not marked invalid anymore.

Hi @PierrickP,
I've checked https://deploy-preview-86--gbfs-validator.netlify.app/?url=https%3A%2F%2Fgbfs.nextbike.net%2Fmaps%2Fgbfs%2Fv2%2Fnextbike_le%2Fgbfs.json
and can confirm that vehicle_types.json is not marked as required anymore and consequently the feed is considered valid.
Thanks for the good work!


Rename digraph functions (related #8580)

Changed the function name to have a "digraph_" prefix.

New name
Old name

digraph_get()
getdigraph()

digraph_get_list()
getdigraphlist()

digraph_set()
setdigraph()

digraph_set_list()
setdigraphlist()

--
Best regards,
Hirohito Higashi (h_east)

No sure if get_list is a good suffix, and there are prop_type_list(), popup_list(), etc.
On the other hand there are also bunch of getmarklist(), getjumplist(), getchangelist(), etc.

Just for reference:
Functions with "set" in their name: 39
Functions starting with "set": 23 (59 %)
Functions that do not start with "set": 16 (41 %)
Functions with "get" in their name: 81
Functions starting with "get": 54 (67 %)
Functions that do not start with "get": 27 (33 %)
There is however a lot of functions for newer features that begin with the name of that feature, such as "term", "popup".
I guess it could go either way.

I guess this can be closed since the change seems to have been merged in patch 8.2.3226.


Throws error if I edit the manifest file

Information

Apktool Version (apktool -version) -2.3.1
Operating System (Mac, Linux, Windows) -Windows 10
APK From? (Playstore, ROM, Other) - Playstore

What I am trying to do

I need to MITM https requests through fiddler/burp but due to android N limitation, I need to specify the app to honor user installed certificates. So, I need to add a file in res/xml and change the manifest file to point to the file. (https://blog.ropnop.com/configuring-burp-suite-with-android-nougat/)
I decoded and rebuilt without changing anything and it worked but as soon as I changed the manifest, it threw errors.

APK
https://drive.google.com/open?id=1bAU35Z6uRDX603aueTCfNOazMVsQgVXS
Questions to ask before submission

Have you tried apktool d, apktool b without changing anything? Yes. If I don't change anything, the build process is successful

sorry. syntax mistake. I am trying to check if any more errors are there from my end. I will reopen it later if I need help. closing now.

Also @souravndp, Apktool is now at v2.3.4 and your report reflects v2.3.1...
~Ibuprophen


post renderes don't work when helm template writes to a file

Output of helm version: v3.3.0-rc.1
In the function renderResources the post renderer is run after the files have been written to disk and does not affect the files.

/assign

@donggangcj You may want to hold off on this. @technosophos is going to rewrite this section of code. It's mixing concerns. He's the one who put the comment on needing to refactor renderResources at the top of it. His rewrite will affect any work on this and may correct the issue.

ok！I am very in favor of rewriting this part of the code . 👍

Just stumbled upon this myself. Any chance we can get a quick fix before doing a bigger rewrite in this area? @technosophos @mattfarina

This is the fix so far: #8573 It is non-trivial.

Any progress on this?


Thread identification

Splits threads into separate boogie output files. Currently only works for pthreads (support for other thread libraries can be added later)
-t flag enables this functionality and requires the --analyse flag to be used alongside it.

Can you check in the C source for the two examples please

It already exists for basic_assign_pthread and never existed for the 'pthread' example (someone else committed it but without the source) which is why I removed it.

Oh ok my mistake, I thought it was a newly added example

This obviously needs to wait for #204 to be merged in.


Validation of authorization mechanism integration

[X] Authorization policies - add default configuration
[X] Fix - load configuration in Auth component
[X] Fix - authentication process doesn't allow "user" creation
[X] Define system behavior when the authentication fails
[X] Implementation for authentication failure
[X] Validation for authentication failure
[x] Authorization mechanism validation - Backend
[x] Fix - Keypass fails during access authorization verification
[X] Deployment environment update for pepkong inclusion

Implemented in dojot/pep-kong and dojot/pdp-ws


fix to save the state for survey status block

Issue #:
https://github.com/bcgov/met-public/issues/397
Description of changes:
Fix to save the state for each of the text that will appear in the survey block for each status.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the met-public license (Apache 2.0).

Codecov Report

Merging #1062 (d869dee) into main (6d70e5c) will decrease coverage by 0.17%.
The diff coverage is 25.00%.

@@            Coverage Diff             @@
##             main    #1062      +/-   ##
==========================================
- Coverage   74.39%   74.22%   -0.18%
==========================================
  Files         223      223
  Lines        6335     6355      +20
  Branches      441      448       +7
==========================================
+ Hits         4713     4717       +4
- Misses       1556     1568      +12
- Partials       66       70       +4

Flag
Coverage Δ

metweb
68.52% <25.00%> (-0.32%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

.../components/engagement/form/AddSurveyBlockTabs.tsx
58.33% <25.00%> (-9.24%)
:arrow_down:

met-web/src/utils/index.ts
43.47% <0.00%> (-20.81%)
:arrow_down:

met-web/src/apiManager/endpoints/index.ts
100.00% <0.00%> (ø)

met-web/src/apiManager/httpRequestHandler/index.ts
58.82% <0.00%> (ø)

...idget/PhasesWidgetMobile/EngagementPhaseMobile.tsx
100.00% <0.00%> (ø)


Redirect refactoring

What kind of change does this PR introduce?
Refactor
What is the new behaviour?
Refactor makePage so that we can release before #57 gets merged.
Does this PR introduce a breaking change?
No
Other information:
Please check if the PR fulfills these requirements:

[ ] Tests for the changes have been added
[X] Docs have been added / updated

Coverage remained the same at 100.0% when pulling 71016ec4975b809b68d8926d4f7e09fc64087efc on redirect-refactoring into eac323953b01ca64079b6a3900ea177b9ff88978 on master.


asko_import: Import owner data from apartment.txt etc. (ASU-1430)

Tune the AsKo importing so that we get correct owner (= reserver or buyer)
for every apartment.  Some of the status changes have been done in AsKo
without any reservations or lottery events so we have to create separate
reservations for those status update from the apartment.txt.
Also improve the Django Admin side so that it's easier to inspect
the import logs.  Namely, make customer names and apartment/project
names visible in more places.

Codecov Report

Merging #368 (32d46bf) into develop (6858637) will decrease coverage by 0.58%.
Report is 2 commits behind head on develop.
The diff coverage is 9.58%.

@@             Coverage Diff             @@
##           develop     #368      +/-   ##
===========================================
- Coverage    90.94%   90.37%   -0.58%
===========================================
  Files          336      336
  Lines        11845    11932      +87
  Branches       834      847      +13
===========================================
+ Hits         10773    10784      +11
- Misses         937     1013      +76
  Partials       135      135

Files
Coverage Δ

asko_import/admin.py
83.33% <ø> (ø)

asko_import/describer.py
21.81% <0.00%> (ø)

asko_import/issues.py
49.46% <50.00%> (+0.57%)
:arrow_up:

asko_import/models.py
48.69% <18.75%> (-4.77%)
:arrow_down:

asko_import/importer.py
13.92% <7.25%> (-1.34%)
:arrow_down:

... and 1 file with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


DTM: Issues observed in edit property window in subassets page.

Once the asset is created, later if user try to modify the custom property value, and if there is validation error, then the message is not displayed. It displays an empty validation.

The cancel option for properties is not working as expected. Edit the custom property and change the custom value and click cancel. Value does not get saved but when clicked on edit again, it shows the modified unsaved value which is incorrect.

Fix is available in the latest uploaded DTM version


Proof of concept: Webpack to Vite migration

Ticket #
Description
This PR is meant as a functioning demo of a migration to Vite as our frontend dev server and bundler for production. (It's not meant to be merged. If we agree on a path forward, this will get split up into smaller, more manageable PRs to land individually.)
I've verified the grants and arpa sites are basically functioning, but I haven't gone through in detail yet to make sure every detail is in place. (If you find anything that isn't working yet in vite, please flag it for me!)
Architecture and product considerations:

Vite is generally more modern and easier to understand and work with than webpack
Vite dev server is much quicker to start up and update, including support for hot module replacement
We'd drop support for non-evergreen browsers (notably, IE11) — this was discussed live, and decided this is fine
Migration to vite is not required but will make a migration to vue3 a bit simpler

The main tasks required for the migration (essentially, a more organized list of what this PR is doing):

Preparation prior to the Vite switchover

Migrate stray CommonJS (require and module.exports) to ES6 (import and export) syntax
Upgrade vuelidate package to v2 (our current v0.7 is not compatible with vite or vue3)
Remove lodash-checkit dependency (relies on global reference, which is only available in CJS/node context; checkit does not appear to be safe to run in a browser context... but we're only using it for email format checking, and we should just migrate this usage to vuelidate)
Switch our test runner (more research needed here)

Vite switchover

Update our package dependencies (mostly removing @vue/cli* and adding vite)
Update config (replace vue.config.js with vite.config.js, updates to .eslintrc.js)
Update package.json script references to run dev server and build via vite instead of vue-cli-service
Reconfigure entrypoints for grants and arpa reporter (vite expects index.html in a particular location to indicate separate entries for the two builds in dev)

Plenty of QA in staging

Screenshots / Demo Video
Testing
Automated and Unit Tests

[ ] Added Unit tests

Manual tests for Reviewer

[ ] Added steps to test feature/functionality manually

Checklist

[ ] Provided ticket and description
[ ] Provided screenshots/demo
[ ] Provided testing information
[ ] Provided adequate test coverage for all new code
[ ] Added PR reviewers

I'm in favor of the proposed move to Vite, and this PoC makes sense/looks good to me.
I do have one concern, which is that migrating the packages/client code to ES6 while leaving the packages/server code in CJS feels a bit messy and likely confusing to newcomers (honestly I'd probably get a bit flummoxed if I was going back and forth between client/server source code). If we decide to head in this direction, it would be ideal to migrate the entire repository to ES6 so that coding conventions remain consistent, assuming that doesn't unreasonably blow up the scope of this effort.

Thanks for reviewing, @TylerHendrickson!

If we decide to head in this direction, it would be ideal to migrate the entire repository to ES6 so that coding conventions remain consistent, assuming that doesn't unreasonably blow up the scope of this effort.

Completely agree on all counts! The problem is unfortunately I do think it'd be a big enough scope that it deserves a separate effort...
As a quick snapshot of the scale, while the client package uses require(...) 12 times, server uses require(...) 581 times across 126 files. Once we're at that scale, it deserves an incremental approach rather than a single PR. (Good news is we can use the same "eslint rule with ignored files as punch list" strategy we did with vue/recommended lint rules here. eslint-plugin-import has a no-commonjs rule that we can use as a ratchet and eventual enforcement mechanism.)
The other difference is that imports are all exercised on the frontend when we build for production, which is a nice check that they're all valid, whereas we don't get that for server code and our test coverage isn't stellar. (That said, we can likely achieve a similar result with eslint-plugin-import's static analysis using no-unresolved to verify imports exist.)

One other thing to mention: when I run yarn workspace client build locally, the resulting dist directory includes a deploy-config.js file. I realize this is just a PoC at the moment, but just dropping a note here that we'll want to figure out how to exclude that from builds.

One other thing to mention: when I run yarn workspace client build locally, the resulting dist directory includes a deploy-config.js file. I realize this is just a PoC at the moment, but just dropping a note here that we'll want to figure out how to exclude that from builds.

@TylerHendrickson This is a good callout, thanks! There's actually no 1:1 functionality to conditionally exclude a specific file from the public folder with vite. There are a few ways we could handle this:

Do away with deploy-config.js and rely on environment variables instead. (Is there background/context on why we have the deploy-config.js setup?) This I think aligns us better with standard web dev practices and minimizes confusion by minimizing the surface area of differences between environments. (Unless there's some need for our environments to have different code, rather than just different data/config?)
Bit hacky, but very easy: we can update the build command in package.json to vite build && rm dist/deploy-config.js.
Move the deploy-config.js into our src/ instead of our public/ directory, and then we can use rollup's configuration settings to manage splitting it out, naming it correctly, and excluding it. (I think this is possible, but the config would be a bit messy, and I haven't verified it would work.)
Do nothing. My understanding is the resource "aws_s3_object" "deploy-config" terraform block will overwrite the deploy-config.js file in the build even if one is present.


use compose bundle to replace the swarm catalog

Rancher Version:
v1.2.1
Docker Version:
1.12.3
OS and where are the hosts located? (cloud, bare metal, etc):
bare metal
Setup Details: (single node rancher vs. HA rancher, internal DB vs. external DB)
HA rancher
external DB
Environment Type: (Cattle/Kubernetes/Swarm/Mesos)
Swarm
Steps to Reproduce:

create swarm mode cluster
swarm catalog is missing

Results:
swarm catalog was removed in rancher v1.2.1
Expected:
Could you please support the compose bundle to deploy services on swarm mode cluster?
Or swarm mode is of no use in rancher any more.
Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.

To deploy your application across the swarm, use the bundle feature of the Docker experimental build.

More info:
https://docs.docker.com/compose/bundles

@HackerWilson This is really just too much a moving target.  As such we've really had to decrease the scope of what we support with anything related to Swarm.  We just don't really know where things are going and they change with each Docker release.  On this topic specifically, I do not think "bundles" or "dabs" will be a user level concept if not dropped completely.  In Docker 1.13 compose files are now supported natively https://github.com/docker/docker/pull/27998.  Those compose files are also "v3" which a new format with 1.13.  We will let something stick for at least one minor release before we support it :)  Any attempt at guessing where Docker is going has largely backfired.


Small improvements to dec_p_derivbound

Removed some intermediate arrays that are not really needed.

Before:
Operator: Boundary
Variant: New Form-1, [0.169584 ms, 0.382176 MB]
Variant: New Form-2, [0.2345205 ms, 0.43472 MB]
Operator: Dual Derivative
Variant: New Dual-Form-0, [0.2308955 ms, 0.434064 MB]
Variant: New Dual-Form-1, [0.179959 ms, 0.38152 MB]
Operator: Exterior Derivative
Variant: New Form-0, [0.218458 ms, 0.361184 MB]
Variant: New Form-1, [0.2857295 ms, 0.444304 MB]
After:
Operator: Boundary
Variant: New Form-1, [0.147083 ms, 0.365456 MB]
Variant: New Form-2, [0.247791 ms, 0.365376 MB]
Operator: Dual Derivative
Variant: New Dual-Form-0, [0.250625 ms, 0.36472 MB]
Variant: New Dual-Form-1, [0.158458 ms, 0.3648 MB]
Operator: Exterior Derivative
Variant: New Form-0, [0.194375 ms, 0.344464 MB]
Variant: New Form-1, [0.282 ms, 0.37496 MB]

Nice. Are there any changes here on the output of @code_warntype?

Good catch, the `numeric_sign' was falling into the generic case caused by the issue you linked (attr_types can't be used for inference). I've type-asserted them to be bools and the problem seems to be resolved now.

Latest:
Variant: New Form-1, [0.142125 ms, 0.365456 MB]
Variant: New Form-2, [0.2445835 ms, 0.365376 MB]
Operator: Dual Derivative
Variant: New Dual-Form-0, [0.249375 ms, 0.36472 MB]
Variant: New Dual-Form-1, [0.154042 ms, 0.3648 MB]
Operator: Exterior Derivative
Variant: New Form-0, [0.193291 ms, 0.344464 MB]
Variant: New Form-1, [0.28525 ms, 0.37496 MB]

Good catch, the numeric_sign was falling into the generic case caused by the issue you linked (attr_types can't be used for inference). I've type-asserted them to be bools and the problem seems to be resolved now.

Yeah type-asserting here is a good decision because the FastDEC module is allowed to make those sorts of assumptions.


+25 Icons, +3 Links

Description

Icons addition information

Added 25 icons:
8Bit Photo Lab (com.ilixa.ebp)
Ticket Restaurant (com.edenred.mobiletr)
Apk Extractor (com.ext.ui)
Automatic Clicker (simplehat.clicker)
Ballz (com.ketchapp.ballz)
Dunk Shot (com.ketchapp.dunkshot)
Vectorify Da Home! (com.iven.iconify)
Mr Gun (com.ketchapp.mrgun)
Flipa Clip (com.vblast.flipaclip)
Bubble UPnP (com.bubblesoft.android.bubbleupnp)
Character Pad (com.husseinelfeky.characterpad)
Glitch Lab (com.ilixa.glitch)
Bluk (com.pixelapestudios.bluk)
Screen Stream (info.dvkr.screenstream)
Super Arcade Racing (com.outofthebit.superarcaderacing)
Photo & Picture Resizer (com.simplemobilephotoresizer)
Explorer (org.amnh.explorer)
Rider (com.ketchapp.rider)
Audio Converter (com.bdroid.audiomediaconverter)
Buildings For Minecraft PE (com.electricfoal.buildingsformcpe)
Crop And Trim Video (fnzstudios.com.videocrop)
Pocket Mathematics (gecko.droid.mathematicshelper)
Quick Cursor (com.quickcursor)
Word Sorting Machine (wordsortingmachine.sbytex.com.wordsortingmachine)
Guarda! (it.yoomee.zanichelliapp)
Linked 3 icons:
Frequency Generator (com.boedec.hoel.frequencygenerator → frequency_generator.svg)
Voice Tools (com.DevExtras.VoiceTools → voice_recorder.svg)
Polette (com.polette.shop.app → lenskart.svg)
Contributor's checklist

[x] I followed the Lawnicons Guidelines (upd. Jan 2024) and will make changes if someone suggests. I will also make sure that Lawnicons builds correctly.

There are a couple of little things with alignment that I accepted as is so as not to waste our time. However, it remains to change the size of pocket_mathematics.svg and correct a couple of names in appfilter.xml.

Not sure why but Inkscape keeps telling me the width and height of the pocket_mathematics.svg is 160px * 160px. I redrew it from scratch though I can't guarantee it'll appear correctly. I have used svgo to optimize the icons beforehand in this pr, this time I'm simply exporting it as a plain.svg file.

Not sure why but Inkscape keeps telling me the width and height of the pocket_mathematics.svg is 160px * 160px.

I have a hypothesis that this may be due to the peculiarities of creating ellipses and the fact that Inkscape works strangely with integer coordinates. I will add another option so that the size becomes as needed. If you are interested, then compare it with your own.


Test error

mvn test gets following error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-shade-plugin:3.2.1:shade (spark-shade-jar) on project accumulo-spark-datasource: Error creating shaded jar: /data/home/jumin/git/masc/connector/zipfs/target/classes (Is a directory) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :accumulo-spark-datasource

from which directory did you execute this?

connector, same as the build command.


Issue with bullet physics vehicle

Operating system or device, Godot version, GPU Model and driver (if graphics related):
Fedora 26, master
Issue description:
I am having issues getting the vehicle implementation working on the bullet fork, here is a link to my test project
The only tweaking I have done to the vehicle is set the suspension to 10, but there is no way to get the vehicle to stay reasonably still. In other C++ projects where i used the vehicle it stays relatively still while the engine hums. This vehicle goes all over the place and is not quite usable.
Steps to reproduce:
Create a default vehicle, then set the suspension to 10 for each wheel (default vehicle sits there slumped and uneven, despite it being symmetrical)
Link to minimal example project:
https://my.mixtape.moe/utbpoy.zip
@AndreaCatania

Hi @Zireael07 I need to check if your project with cars, that you sent me some time ago, has this problem. Can you please send me a copy of your test project that you sent me?

As far as I can tell this was the project: https://www.dropbox.com/s/8bn3lxhtm8latrx/FreeRoamRacer.zip?dl=0

@Zireael07 Thanks you!

@Zireael07 In this archive the project.godot file is missing, are you sure that it is the right one?

Not quite sure - I will try to look tomorrow for the correct folder :)

This is the correct folder - https://www.dropbox.com/s/wtlehmskd4uzv7y/exportproto - Kopia.zip?dl=0

The project does not run for me, I get a
Invalid call. Nonexistent function 'get_point_pos' in base 'Curve3D'.

at script in scripts/curved_road_segment.gd:143

Wouldn't it make more sense to have the default vehicle be more sane? Perhaps the default vehicle can behave more like the vehicles in @Zireael07's project

Also I am not sure what you mean by the maximum force, I don't see that as a field for the vehicle in the editor.

I applied the settings you mentioned on my test project I made, the car still vibrates and moves all over the place even with the brakes set to 1

@bkeys: Are you 100% sure that you had bullet selected as the physics engine? Vibrations sounds like #9254 which is fixed by both bullet and by the Godot physics joints fix that went in just before bullet.

I have the same issue when I use a vehicle from https://github.com/Zireael07/FreeRoamRoguelikeRacerPrototype and put it on a ground plane without a position3d node. But it only appears for me when I use Godot physics, bullet physics is smooth.

Now I have the same issue with both godot and bullet physics when I use another car mesh.
Demo.zip
Here is the scene based on Zireael's demo.

Sorry all if I didn't respond but in this moment I'm a bit busy. I'll look
on this ASAP
On Dec 20, 2017 11:02 PM, "LinuxUserGD"<EMAIL_ADDRESS>wrote:

Now I have the same issue with both godot and bullet physics when I use
another car mesh.
Demo.zip https://github.com/godotengine/godot/files/1577139/Demo.zip
Here is the scene based on Zireael's demo.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/issues/12521#issuecomment-353195262,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AH9MR-S74x29f8q6k2fRVbvFyRJzMydhks5tCYPbgaJpZM4QLuQT
.

I can also reproduce the car vibrations with the new Truck Town demo with bullet.

I didn't check it yet, but strange thing is that vehicle code is exactly the same for two physics engines

Ok, just an update. I'm working on a new demo that has a complete race track and a car setup properly.
While it has been a frustrating week last night I finally got it to a point where it's become a really solid solution. I'm really impressed with it. One thing that really helped was when I found some documentation on the bullet raycast vehicle implementation.
From what I understand, we're using the underlying bullet physics engine but part of the vehicle implementation is still the same as the original Godot implementation, it's just working off of the back of the collision detections and such in bullet. As a result there might be a difference in how the values are applied in the full Godot physics engine and the Bullet physics engine.
I'll be adding stuff into the documentation soon based on my findings and the range of values was surprising. Below is a quick summary of a few things I remember of the top of my head:
Just to react directly on the OP mentioning setting suspension to 10:
suspension_travel is the amount the suspension can move from resting point (controlled by wheel_rest_length). The bullet documentation has this in cm but Godot assumes meters (it really depends on the size of your car in game off course), so if you set this to 10, thats not going to do you much good. In my test I have this set to 0.1 which is a 10cm travel, which is pretty normal for a car, probably even high for a race car.
suspension_stiffness is how stiff the springs are, I am not sure what the unit is here but the lower this number, the bouncier your car. Bullet suggest a value less then 50 for a offroad car, 50 - 100 for a road car, 200 for something like an Formula one car. For me having a very low racecar I originally had this on 70 or 80 which worked well except when going over a bump, it would compress the spring so much the main collission shape would hit the ground and that would send my car flying. I've got it set to 100 which works well for my racecar but I would set it lower if you have a car with a more normal ride height. But definately not as low as 10 :)
suspension_max_force, again I don't know the units but this was the setting I had the most problems understanding. It caps the maximum force the springs can counter, setting this to something as low as 10 makes the spring completely non-functional. I've found the minimum value that doesn't bottom out the car is a quarter of the weight of the car which makes sense as each spring holds a quarter of the weight of the car, but it is far too low in practice, the moment the springs start to work they get easily overpowered as the forces working on the springs are a multiple of this.
I found a value of 1000 worked well for my racecar.
damping_compression and damping_relaxation effect the damping on respectively compressing the spring and decompressing the springs. A value of 0.0 means no damping and your car will keep bouncing around. The documentation suggested putting relaxation slightly higher then compression.
For a normal car values of respectively 0.3 and 0.5 seem to work well, for my racecar I've got it on 0.5 and 0.8 which seems to work well but I'm still playing around with it
Finally the last two settings that are important to note:
wheel_roll_influence is a value that counters rolling the car. 0.0 and your car will easily roll, 1.0 and your car won't have body roll, but from what I've read its mostly trickery and not real physics. I've set mine at 0.5 and it works fine.
wheel_friction_slip is a value combined with the friction of the road and basically determine how grippy your tires are. 0.0 means no grip at all, 1.0 should be full grip but I think there is more going on here with multiplying the friction of the surface and the wheels being able to loose grip. For a F1 car you'd thus probably put this even higher.
If you set the rear wheels grip lower then your front wheels, you can have fun making donuts :)
I've found a value around 0.9 - 1.0 on all tires works well for driving on tarmac. This can be a fun value to play around with to simulate wear on tires :)
The only problem I have left is that when my car was in rest it would still slowly slide away even when the brake is on. The car was on a slope and the fiction on the road was lowered so that might explain that but its something that I think needs more investigation (@AndreaCatania already has a copy of my setup).
The other thing here I think that could use a bit more work is that when the wheels loose grip, they stop turning instead of spinning freely if engine_force is still set. That would be a neat thing to implement. Also I think we should be able to get feedback of the speed at which the wheels are rotating but thats not relative to this discussion :)

Wheel roll influence is indeed trickery. Godot's car implementation is basically a 1:1 copy of the Bullet physics raycast vehicle, so a couple months back I added the roll influence hack the Bullet raycast vehicle has to be able to avoid rolling over all of the time. I admit I don't really understand how it works, but I went with a value of 0.5 in my project too :)

Basically the same issue as mentioned here. Just confirmed the shaking behaviour with the truck demo project.

The shaking / vibration doesn't happen in bullet with every car setup.
See https://github.com/Zireael07/FreeRoamRoguelikeRacerPrototype and https://github.com/HugeGameArtGD/Solus-Stunts

@LinuxUserGD Can confirm... what a weird behavior.

Just wanted to add that it doesn't happen on every car setup.

But only for the TruckTownDemo?

I don't have it as bad as the truck town demo but when I ported the truck town demo to Godot 3 I didn't spend any time tweaking/improving settings.
The new demo project I'm working on which uses the settings I described above results in a pretty stable car when moving. Only when standing still on sloped ground it still gets moved ever so slightly.
So there is definately still an issue somewhere but with the right setup almost all issues are fine.
@LinuxUserGD keep in mind that the bullet examples you are referring to probably have cars that are way better set up then the ones in truck demo.

Well its hard for users of the engine to determine what settings work.
Trial and error seems to be the only option right now. Its rightfully still
a bug.
On 22 Feb 2018 11:48 PM, "Bastiaan Olij"<EMAIL_ADDRESS>wrote:

I don't have it as bad as the truck town demo but when I ported the truck
town demo to Godot 3 I didn't spend any time tweaking/improving settings.
The new demo project I'm working on which uses the settings I described
above results in a pretty stable car when moving. Only when standing still
on sloped ground it still gets moved ever so slightly.
So there is definately still an issue somewhere but with the right setup
almost all issues are fine.
@LinuxUserGD https://github.com/linuxusergd keep in mind that the
bullet examples you are referring to probably have cars that are way better
set up then the ones in truck demo.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/godotengine/godot/issues/12521#issuecomment-367849482,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIVbam-rm76wEqpeKWy8js5TcHNScCPuks5tXe7RgaJpZM4QLuQT
.

@MrMinimal yes and no.
Not using the right settings isn't a bug. It is however a problem in lack of documentation (which is why I've submitted a PR with that information to be added to the documentation) and a lack of tutorials (which is why I'll be making a video this weekend on the subject and will likely eventually do a write-up for the tutorial section in the Godot docs).
The car not standing completely still when the settings are within sensible ranges is however a bug and I've send my test files to @AndreaCatania so he can play around with them. I still haven't ruled out however that the fault lies in my code :)

Yes and strangely the extent setting of the collision shape influences how the car behaves on ground:
bug.zip

@LinuxUserGD I'll try out your test when I'm home tonight.
I have had problems with the collision shape of the car hitting the ground when the suspension is set to soft and the car bottoms out. The collision results in a big impulse (Newtons third law, For every action, there is an equal and opposite reaction) often sending the car flying.
It's hard to say at this point in time whether that is a bug in the physics engine or whether that is a consequence of setting up a car with to much suspension travel. I do think the impulse provides far to much force to the car for the behaviour though so there may very well be an error in those calculations.

Ok guys, I was preoccupied with my own demo and completing a video about setting up a Vehicle Body based car, took way more time then I should have spend on it.
Researching things I came across this document:
http://blender3d.org.ua/forum/game/iwe/upload/Vehicle_Simulation_With_Bullet.pdf
This is a writeup by someone who as far as I can tell has added the same functionality in the blender game engine and discusses the types of problems he's run into, a number of them sound very familiar to us :)
Thanks to him I finally found out how Max Force is supposed to be taken into account so I'll update the documentation on that soon as well. In a nutshell, it is the counter force the spring provided at full compression. Setting it extremely high amplifies the spring so that might not be a good idea.
Anyway, the chapter on friction in that document describes how forces enacting on wheels are implemented and how each wheel being individually checked leads to the jittering issues.
I'm just putting this info here because I am at work and I don't have time to test any of this out and see if implementing this solution in Godot actually resolves anything. I'll be experimenting more on this later in the week.

Finally got a chance to do some debugging specifically looking at the issue of the car jittering and turning around its axis when its standing still.
I think our problem lies in the application of _resolve_single_bilateral:
https://github.com/godotengine/godot/blob/master/scene/3d/vehicle_body.cpp#L526
I don't think the code itself is wrong, it actually looks like its doing the right thing. "body2" for us is always the ground which is never in motion so the code in theory could be greatly simplified. One thing that was interesting in comparing the code with the original bullet code is that at the end the contactDamping value for us is 0.4 while it's 0.2 in bullet. Changing it to 0.2 does help a lot but you can see the problem is still there, just limited.
The issue seems to be in the strength of the impulses. One thing that for my example exaggerated things was that my wheels where not exactly mirrored, they were slightly off its enough that you quickly get a small imbalance in counter forces growing into an angular momentum that starts a feedback loop.
The other thing that I think is related, and maybe where the contactDamping is hiding the problem, is that we calculate this impulse every frame and apply it every frame. There doesn't seem to be anything that takes into account our time delta. That would be fine if it was a single impulse but not one applied every frame. Unlike our forward impulse which gets s->get_step() applied.
I'm wondering if our contactDamping should be replaced by s->get_step() ?

The other thing that I think is related, and maybe where the contactDamping is hiding the problem, is that we calculate this impulse every frame and apply it every frame. There doesn't seem to be anything that takes into account our time delta. That would be fine if it was a single impulse but not one applied every frame. Unlike our forward impulse which gets s->get_step() applied.

This make thing more consistent when you change the fixed delta time from 60 to 120 or 30. It prevent to have different behaviours but doesn't fix the problem since the physics delta time in godot is fixed.
I think that we need to implement an extra algorithm that simulate the anti roll bar of car: https://en.wikipedia.org/wiki/Anti-roll_bar in this way we will be able to connect the two wheels that will result more stable

Hmmm, then I'm guessing the value of 0.4 was decided upon after experimenting what would work, but that is heavily dependent on positioning of the wheels, roll influence etc, I wonder if dividing it by roll influence together with the much lower step value was the main ingredient in atleast getting it to be stable.
I do agree, we need something better...

Ugh.. any other ideas?

@drarem #238

What's the status of this issue?

I think it's fixed

@LinuxUserGD In which version was it fixed?

@DVLP It was fixed in the truck town demo and bullet seems to be ok (godot physics is deprecated).


Remove (external)secrets for long-lived AWS creds.

We don't want to be using these any more; they're a hold-over from years ago when GOV.UK was hosted outside AWS. We've already configured IAM policies in Terraform to authorise Asset Manager to its S3 buckets, so let's remove these keys in order to check that the IAM stuff is working as intended.
Rollout: need to remember to delete the corresponding secrets from AWS Secrets Manager in the integration account after merging.

Ah, turns out the IAM stuff doesn't work yet (owing to probable bug in the app). Can't merge this just yet then.

We're now on an up-to-date version of the AWS client libs and this now works. RFAL.


Dropping support for 32bit iOS devices

The last devices from the Apple's portfolio are iPhone 5, iPhone 5C and iPad. Those devices are no longer supported by the latest versions of the - iOS11 and newer. On iossupportmatrix.com you can check the compatibility between the iOS and different devices and models.
With that said, our suggestion is to discontinue the support for 32bit devices with the next major release. This will reduce the sizes of the ipa file produced by {N} and the iOS runtime package, since we will no longer have to distribute 32bit binaries. However, this will not affect the end size of your application, because the Apple store automatically takes care to remove unneeded binaries from you app package. This also means less time for testing and building of the iOS runtime itself.
Our data shows that the share of the 32 bit devices is no more than 10% of all Apple devices. We are still in the process of assessing what would be the impact for the community, so your thoughts are most welcome.

I'm all for it if Apple/iOS is no longer supporting.

How do other frameworks (React Native, PhoneGap) treat 32bit iOS support today? I think the decision is even easier if we're all moving forward together.

I caution you on this because a ton of large businesses which are not on the ios store (using a private enterprise store); have a large number of the iPads which are only 32bit and virtually all iPods are 32bit.
I can also answer part of @toddanglin's question; Flutter just added 32bit support a few months ago; because this was considered a major blocker for multiple customers.    I'm not sure this is the right time to drop support of it yet.

Thanks, @NathanaelA. That's good context.
Any idea where ReactNative stands on this? Did a quick search and didn't find any definitive statements.

@toddanglin I just checked a RN project and it does support the three ARM architectures: armv7 armv7s arm64.
As it seems, it's really still too early to drop 32-bit support.


RubySrc2Cpg: kick-off project with just the MetaData pass

Could you add a very simple unit test for the metadata pass. To see the frontend running once at least during tests.


Rule: YAML alias from title

The rule should transform
# My title

into
---
aliases:
  - My title
---

# My title

This would be helpful as Quick Switcher (Ctrl + O) supports aliases, so notes could be found not only by filenames but also with titles

However, I am not sure how to handle edits
---
aliases:
  - Old title
---

# Old title

edited to
---
aliases:
  - Old title
---

# New title

How can we possibly detect that Old title was the previous title on modify event?

@mnaoumov , thank you for your request. Is my understanding correct that you would want the alias to keep one of its values the same as the name of the file?
For example Test File.md would have an alias value of
alias:
  - Test File

Then when the name is changed via Obsidain to something else like Test File 2.md, the YAML would be updated to
alias:
  - Test File 2

@pjkaufman No, I mean similar to https://github.com/platers/obsidian-linter/blob/master/docs/rules.md#yaml-title but for aliases, that could be multiple values, not just single


feat: add comtrya/comtrya

comtrya/comtrya: Configuration Management for Localhost / dotfiles
$ aqua g -i comtrya/comtrya

How to confirm if this package works well
Reviewers aren't necessarily familiar with this package, so please describe how to confirm if this package works well.
Please confirm if this package works well yourself as much as possible.
Command and output
$ comtrya --help
comtrya 0.8.2

USAGE:
    comtrya-x86_64-apple-darwin [FLAGS] [OPTIONS] <SUBCOMMAND>

FLAGS:
    -h, --help        Prints help information
        --no-color    Disable color printing
    -V, --version     Prints version information
    -v, --verbose     Debug & tracing mode (-v, -vv)

OPTIONS:
    -d, --manifest-directory <manifest-directory>    Directory

SUBCOMMANDS:
    apply
    help       Prints this message or the help of the given subcommand(s)
    version

Thanks!

Released. https://github.com/aquaproj/aqua-registry/releases/tag/v3.122.0


Add a fetch-only option for the git mirror experiment

The git mirror experiment currently calls git remote update as the primary update command. When the repository has a large number of branches (n > 10,000) this operation significantly slows down the server (especially with many parallel builds).
This PR adds an option which will optionally (must opt-in) call git fetch origin refspec instead of git remote update. This is essentially the same call except that git fetch only operates on one branch. This makes much less work for the server in the case of many branches.
And in case anyone asks, yes this is actually a concern for my team on the project we work with. We're running in to these limitations daily.

Thanks very much for the PR!
I wonder if we could make this behaviour the default? Looking at the docs for git remote update, it seems like git fetch --prune origin refspec would be totally adequate.

I like the idea of making it default and having fewer config options.
Do we anticipate downsides to always using git fetch?

Any updates on this?

We would like to use this option.


How to integrate resource owner password grant with the spring-security login flow?

Hi, I have a question about the integration with spring-security.
Background
I configured an authorization server which support password grants and I can retrieve access tokens and refresh tokens without any issue. Now I need to protect another service and since both are part of the same domain the password grant flow seems to be the most appropriate one.
The problem
The @EnableOAuth2Client annotation says

Apps that us password grants need to set the authentication properties in the OAuth2ProtectedResourceDetails before using the RestOperations, and this means the resource details themselves also have to be per session (assuming there are multiple users in the system).

This makes complete sense but I do not have any clue about how to use it together with WebSecurityConfigurerAdapter. Do I need to write a custom AuthenticationManager that calls oauth2RestTemplate.getAccessToken()?
Could you please shed some light?
Thanks.

I don't really know what your use case is. In your "background" you seem to say you want to implement a new resource server, which ought to be easy (you @EnableResourceServer). In the "problem" it sounds like a client app. Why wouldn't auth code grant be appropriate if it's a client app you want to add? Spring Boot 1.3 has @EnableOAuth2Sso that takes the pain out of it but all it does is configure an OAuth2ClientAuthenticationProcessingFilter which you can do yourself.

Hi Dave,
Thanks for answering. You are correct, I am working on a client app with SSO. Since both my authorization server and my client app are managed by me I do not need the auth code flow with redirections and an approval form. By my basic understanding of the OAuth grants, the password flow should fit in this case as I trust the client app to handle the user credentials.
In this scenario, I could configure a OAuth2ClientAuthenticationProcessingFilter to use the current user credential to instantiate a new OAuth2RestTemplate with ResourceOwnerPasswordResourceDetails, correct?
I am looking forward to Spring Boot 1.3. Does it supports all the current features implemented by spring-security-oauth?

I think you have the wrong idea about password grant, and I believe you should be using auth code flow for SSO: the benefit is that only the auth server has to collect user credentials. You don't have to obtain explicit approval if the apps are all part of the same system (that's why we have the autoApprove feature in Spring OAuth).

Hi Dave, I got the behavior I wanted by combining @EnableOAuth2Sso and the auto approval feature in the auth server. Sorry for not replying earlier. Thank you for the help.

I need the same solution as well. Is that possible any of you show me some sample code?
Thanks

@dsyer https://oauth.net/2/grant-types/password/
@jrglee if, by any chance, you still have the implementation, I would love to see that!)

My 2 cents.  Sometimes your grant type is mandated by the vendor you are connecting to.
I have machine to machine trust relationship that works only on password grant type.  It does give
me back an access and refresh token which I would like to use with webclient so that it will manage
the token and refresh for me.
It feels like Spring fights me every step of the way because it is not supposed to be a valid use-case.


Add a ord wallet runics commmand.

Works similiar to ord wallet cardinals. Should return

output
rune name
rune id
rune balance
btc balance

@raphjaph this has been resolved.

Thanks!


add Backend::from_usize

This method allows you to write strategies that are generic over the backend, which can be useful

not required, just use TryInto bounds


Ha historical sensor

Fix for #6 (historical sensor), and also #10 (mfa)

@LeighCurran I'm still letting this soak on my HA instance (see #6), but I think this is no pretty much good to go.

Hum, no... I think my Throttle handling has broken something...

Im not sure if its my system, or if its the last commit to show only available tariffs - But I seem to have lost my tariff sensors this morning, and no amount of removing and adding the integration seems to be able to get them back.

Dammit. Likely my changes, sorry.
I noticed that the API tends to return no data for a while for the previous day once it ticks over to the new day. That includes the tariffs.
When that's the case, the integration should refuse to start, and get retried later by HA.
Do you have something in your logs that says as much?
Olivier Mehani @.***>
Sent from my mobile, please excuse my brevity.

No need to be sorry  - thats the joys of testing new things!
I cant see anything related to the component in the logs anywhere - Did I need to change the logging level somewhere? I tried in the config.yaml file, but that hasnt seemed to have helped - But I might have the code wrong

I rolled back to the previous version, then tried to roll forward once the data for yesterday was available. Same thing happened where it marked all sensors apart from kilowatt  hour, estimated balance and dollar value usage.

Ok, the data's in now. It looks like it takes most of the next day before the API has the next-day data.
Try restarting your instance now, it should now have the some data, including tariffs.
I have an idea to make this more resilient. That'll require adding a feature to the AuroraPlus Python module. I'll have a go at this later today.
Olivier Mehani @.***>
Sent from my mobile, please excuse my brevity.

@slothking87 Don't try the latest commits. They don't work just yet 😉

@slothking87 try this one.
Job for later q:

Oh well... After midnight counts as “tomorrow”, right? ToU tariffs should be there even when starting on weekends now.

Hmmm - Initially it shows all my sensors as unavailable again.. Ill try and reset it a bit later once yesterdays data come in to see if that helps. Just to confirm to update Im just downloading the latest files from your fork and replacing the ones already in HA? Just to make sure that its not something that I am doing that is breaking it

Yeah, copying all the custom_components/auroraplus files to the same path in your config directory should do the trick.
It's strange that nothing is reported now (I haven't touched the logic for the daily summaries, so they should still be happy). I suspect you should see errors in your logs.
One thing to note: this is now temporarily using a fork of LeighCurran/AuroraPlus. I updated the manifest.json, but I don't know how HA handles this sort of updates. I use a docker container for my tests, so I can trash the whole thing (I did notice discrepancies like using old dep code if I wasn't doing it), but if you're on a more permanent setup, you might have to find where HA install those deps, and how to update them.
My money is on getoken() not getting the arguments it expects (which would indicate an old version of either upstream AuroraPlus or my fork branch (where I now fixed that issue).

Ok, after removing the component, removing the config, rebooting, removing any orphaned sensors and statistics, rebooting, then adding the files back in, rebooting, adding the config back in, rebooting Now I have three sensors.

No hourly sensors... There should be something in the logs talking about the tariffs it discovered (likely: none).
Olivier Mehani @.***>
Sent from my mobile, please excuse my brevity.

The logs show these (If these are the correct logs)
2023-08-14 21:56:30.406 DEBUG (SyncWorker_5) [custom_components.auroraplus.sensor] Error: None 2023-08-14 21:56:30.407 DEBUG (SyncWorker_5) [custom_components.auroraplus.sensor] Adding sensor: Estimated Balance 2023-08-14 21:56:30.451 DEBUG (SyncWorker_5) [custom_components.auroraplus.sensor] Adding sensor: Dollar Value Usage 2023-08-14 21:56:31.319 DEBUG (SyncWorker_5) [custom_components.auroraplus.sensor] Adding sensor: Kilowatt Hour Usage 2023-08-14 21:56:31.715 DEBUG (SyncWorker_3) [custom_components.auroraplus.sensor] Updating sensor: Estimated Balance 2023-08-14 21:56:31.962 DEBUG (SyncWorker_9) [custom_components.auroraplus.sensor] Updating sensor: Dollar Value Usage 2023-08-14 21:56:35.492 DEBUG (SyncWorker_0) [custom_components.auroraplus.sensor] Updating sensor: Kilowatt Hour Usage
EDIT: When I changed the password to be deliberately incorrect - it did give me an error in the first line so it is authenticating correctly by the looks

I loaded the files again, and the logs look a bit more normal now. Its throwing lots of messages about data not being available which would be expected for this hour of the morning I guess.
There is also this error that started to come up (in bold below) - Not sure if it was anything to worry about
2023-08-15 08:20:42.369 DEBUG (MainThread) [custom_components.auroraplus.sensor] AuroraApi ready with <auroraplus.api object at 0x7f822eaf90>
2023-08-15 08:20:42.372 WARNING (SyncWorker_12) [custom_components.auroraplus.sensor] Error updating data: api.gettoken() missing 2 required positional arguments: 'username' and 'password'
2023-08-15 08:20:42.443 DEBUG (MainThread) [custom_components.auroraplus.sensor] Estimated Balance created
2023-08-15 08:20:42.444 DEBUG (MainThread) [custom_components.auroraplus.sensor] Dollar Value Usage created
2023-08-15 08:20:42.444 DEBUG (MainThread) [custom_components.auroraplus.sensor] Kilowatt Hour Usage created
2023-08-15 08:20:42.444 DEBUG (MainThread) [custom_components.auroraplus.sensor] Kilowatt Hour Usage Tariff T41 created (historical)
2023-08-15 08:20:42.444 DEBUG (MainThread) [custom_components.auroraplus.sensor] Kilowatt Hour Usage Tariff T31 created (historical)
2023-08-15 08:20:42.444 DEBUG (MainThread) [custom_components.auroraplus.sensor] Dollar Value Usage Tariff T41 created (historical)
2023-08-15 08:20:42.444 DEBUG (MainThread) [custom_components.auroraplus.sensor] Dollar Value Usage Tariff T31 created (historical)
2023-08-15 08:20:42.444 INFO (MainThread) [custom_components.auroraplus.sensor] Aurora+ platform ready with tariffs ['T41', 'T31']
2023-08-15 08:20:42.483 DEBUG (MainThread) [custom_components.auroraplus.sensor] Accessing data for EstimatedBalance
2023-08-15 08:20:42.483 DEBUG (MainThread) [custom_components.auroraplus.sensor] Data for EstimatedBalance not yet available
2023-08-15 08:20:42.483 DEBUG (MainThread) [custom_components.auroraplus.sensor] Accessing data for DollarValueUsage
2023-08-15 08:20:42.483 DEBUG (MainThread) [custom_components.auroraplus.sensor] Data for DollarValueUsage not yet available
2023-08-15 08:20:42.484 DEBUG (MainThread) [custom_components.auroraplus.sensor] Accessing data for DollarValueUsage
2023-08-15 08:20:42.484 DEBUG (MainThread) [custom_components.auroraplus.sensor] Data for DollarValueUsage not yet available
2023-08-15 08:20:42.484 DEBUG (MainThread) [custom_components.auroraplus.sensor] Accessing data for KilowattHourUsage
2023-08-15 08:20:42.484 DEBUG (MainThread) [custom_components.auroraplus.sensor] Data for KilowattHourUsage not yet available
2023-08-15 08:20:42.485 DEBUG (MainThread) [custom_components.auroraplus.sensor] Accessing data for day
2023-08-15 08:20:42.485 DEBUG (MainThread) [custom_components.auroraplus.sensor] Data for day not yet available
2023-08-15 08:20:42.485 WARNING (MainThread) [custom_components.auroraplus.sensor] Kilowatt Hour Usage Tariff T41: no metered records, can't obtain hourly data
2023-08-15 08:20:42.510 DEBUG (MainThread) [custom_components.auroraplus.sensor] Accessing data for day
2023-08-15 08:20:42.511 DEBUG (MainThread) [custom_components.auroraplus.sensor] Data for day not yet available
2023-08-15 08:20:42.511 WARNING (MainThread) [custom_components.auroraplus.sensor] Kilowatt Hour Usage Tariff T31: no metered records, can't obtain hourly data
2023-08-15 08:20:42.513 DEBUG (MainThread) [custom_components.auroraplus.sensor] Accessing data for day
2023-08-15 08:20:42.513 DEBUG (MainThread) [custom_components.auroraplus.sensor] Data for day not yet available
2023-08-15 08:20:42.513 WARNING (MainThread) [custom_components.auroraplus.sensor] Dollar Value Usage Tariff T41: no metered records, can't obtain hourly data
2023-08-15 08:20:42.516 DEBUG (MainThread) [custom_components.auroraplus.sensor] Accessing data for day
2023-08-15 08:20:42.516 DEBUG (MainThread) [custom_components.auroraplus.sensor] Data for day not yet available
2023-08-15 08:20:42.516 WARNING (MainThread) [custom_components.auroraplus.sensor] Dollar Value Usage Tariff T31: no metered records, can't obtain hourly data

Cool, so now you are on my branch of the integration, but the gettoken error is a tell-tale sign that you still have the upstream AproraPlus Python lib, rather than my fork.
I have the same issue on my actual server. I need to work out how to make HA install a new version, after installing the original one from HACS.
Olivier Mehani @.***>
Sent from my mobile, please excuse my brevity.

HA! This might work https://developers.home-assistant.io/docs/creating_integration_manifest/#custom-requirements-during-development--testing
pip install auroraplus@git+https://github.com/shtrom/AuroraPlus@more-than-1-day --target ~/.homeassistant/deps
hass --skip-pip-packages pychromecast

(I'm not certain what ~/.homeassistant/deps should actually be in your case)

That sounds plausible. I'll give it a go tomorrow morning. This annoying thing called work keeps getting in the way of all the fun things

Ok, my error is different (I think my machine was overloaded due to weekly backups), but the stacktrace helps:
  File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 320, in _async_setup_platform
    await asyncio.shield(task)
  File "/config/custom_components/auroraplus/sensor.py", line 98, in async_setup_platform
    AuroraPlus = await hass.async_add_executor_job(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/auroraplus/sensor.py", line 96, in aurora_init
    session.getmonth()
  File "/usr/local/lib/python3.11/site-packages/auroraplus/__init__.py", line 89, in getmonth
    self.month = self.request("month", index)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/auroraplus/__init__.py", line 63, in request
    + self.serviceAgreementID
      ^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'api' object has no attribute 'serviceAgreementID'

I have checked that path, and I can see that my fork is properly installed.
If you check your gettoken error, you should be able to click on the message, and see a stacktrace that lists the path

That should give you an indication of what path to use with the pip install, in my case, that would have been /usr/local/lib/python3.11/site-packages/. That said, I didn't have to do it, as the correct lib was installed after a restart, so maybe you can just get away with removing the auroraplus directory and restart.

Hmmm - I seem to have broken mine completely now. It knows that the files are there in custom_ components because I get the warning in the logs, but it doesnt attempt to get any data which kind of points back to square one on the package problem. I wasnt able to find a way to locate that directory to remove the file (I run everything on a RPI running HAOS core). Ill have another attempt tonight hopefully!

Nope. No luck again. I thought maybe if I removed the integration via hacs that it might remove everything and then reinstall manually - no luck there either. Need to do some more reading

It seems to work fine on my real instance now.
I think there are two things you are running into:

Obviously, HACS reinstalls the upstream version, so you need to copy the files from the branch again
HA seems to install Python deps on start-up if missing. After installing the integration, I think you need to stop your server, find where the auroraplus Python lib (not this integration) is, likely in config/deps, or in .../lib/python.../site-packages somewhere.

I think if you

remove everything,
restart,
install the upstream integration with HACS,
stop the server
overwrite the files (mainly sensor.py and manifest.json)
looks for the auroraplus python lib, and remove it
restart the server

At this point, it should try to setup the new (patched) integration, realise that a dep is missing, and install it based on the manifest, which should point to my fork.

Ok, new error this time. Unsure if it is related to the same thing. Im still getting the username and password error so I guess that indicates that I have the wrong thing somewhere still.
2023-08-17 18:31:30.696 DEBUG (MainThread) [custom_components.auroraplus.sensor] AuroraApi ready with <auroraplus.api object at 0x7f6b5755d0>
2023-08-17 18:31:30.698 WARNING (SyncWorker_9) [custom_components.auroraplus.sensor] Error updating data: api.gettoken() missing 2 required positional arguments: 'username' and 'password'
.........
2023-08-17 18:31:31.017 ERROR (MainThread) [homeassistant.components.sensor] auroraplus: Error on device update!
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 576, in _async_add_entity
await entity.async_device_update(warning=False)
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 938, in async_device_update
await self.async_update()
File "/config/custom_components/auroraplus/sensor.py", line 265, in async_update
float(self._api.EstimatedBalance), self._rounding)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: float() argument must be a string or a real number, not 'dict'

Hum, so you still have the gettoken error, but also a dict-instead-of-number that happens in one of the error cases when data is not available (which it isn't because of the gettoken).
The former should no longer happen if on my fork. Are you sure you have the latest version of the manifest.json, and it's in the right place?
As a workaround, you could continue the hacking and manually replace the files froc he AuroraPlus Python lib with my fork, too.
Olivier Mehani @.***>
Sent from my mobile, please excuse my brevity.

Getting somewhere slowly. Manually updated the sensor.py file with the latest one and get the same error you got before. Now just to find how to access      /usr/local/lib/python3.11/site-packages/auroraplus
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 362, in _async_setup_platform
await asyncio.shield(task)
File "/config/custom_components/auroraplus/sensor.py", line 98, in async_setup_platform
AuroraPlus = await hass.async_add_executor_job(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/auroraplus/sensor.py", line 96, in aurora_init
session.getmonth()
File "/usr/local/lib/python3.11/site-packages/auroraplus/init.py", line 72, in getmonth
self.month = self.request("month")
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/auroraplus/init.py", line 52, in request
request = self.session.get(self.url + '/usage/' + timespan +'?serviceAgreementID=' + self.serviceAgreementID + '&customerId=' + self.customerId + '&index=-1', headers={'Authorization': self.token})
^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'api' object has no attribute 'serviceAgreementID'

Did you do the same with the manifest.json?
Olivier Mehani @.***>
Sent from my mobile, please excuse my brevity.

On another note, Aurora seems to be stuck at the 14th. I don't have any data since then, even on their web dashboard.
The latest version of the integration should at least be able to get back to the latest day with some data.
Olivier Mehani @.***>
Sent from my mobile, please excuse my brevity.

Did you do the same with the manifest.json?

I do - it references the new link.   There is a file I found with the haacs data in there. I might try removing that in case it keeps downloading the original files each reboot. I manually tried to change it to your repository and the new fork but that kept reverting back.

What's the path and contents of that file?

\config.storage\hacs.repositories
"647237192": {
"repository_manifest": {
"name": "Aurora+",
"render_readme": true,
"country": [
"AU"
]
},
"full_name": "shtrom/AuroraPlusHA",
"authors": [
"@LeighCurran"
],
"category": "integration",
"description": "Home Assistant integration for Aurora+  ",
"domain": "auroraplus",
"etag_repository": "W/"7402aaa2037ecd065622bffdd49b92024fa389f2e84c40c1f2dd64a7802d5c45"",
"last_updated": "2023-08-15T12:51:56Z",
"default_branch": "main",
"installed_commit": "fe4b60f",
"installed": true,
"last_commit": "fe4b60f",
"manifest_name": "Aurora+",
"last_fetched":<PHONE_NUMBER>.467871
},

Ok, I started again and now I get a new error. Just for something different
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 576, in _async_add_entity
await entity.async_device_update(warning=False)
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 938, in async_device_update
await self.async_update()
File "/config/custom_components/auroraplus/sensor.py", line 265, in async_update
float(self._api.EstimatedBalance), self._rounding)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: float() argument must be a string or a real number, not 'dict'
Still getting the missing arguments message also, so that could be the problem still.

Yeah, I think that's the same cause.
Have you found where HA installs the Python deps? I think it might be worth trying to upgrade your install of AuroraPlus with my branch (see previous message about using pip). I doubt HA will try to replace it, which I suspect is also what's preventing the update.
Olivier Mehani @.***>
Sent from my mobile, please excuse my brevity.

Yeah - I've been trying to find a way to use that. I run home assistant supervised on a RPI which seems to make it  more challenging

Do you have console/ssh access to it?
Olivier Mehani @.***>
Sent from my mobile, please excuse my brevity.

I can SSH into the core. But the commands dont seem to work in that section. There doenst seem to be much info out there on how all that works.

Ok moving somewhere. I found an add on that lets me install custom packages  (https://github.com/hassio-addons/addon-appdaemon).
Down to the single error now

The logs from the install below if that means anything much:
Collecting https://github.com/shtrom/AuroraPlus/archive/refs/heads/more-than-1-day.zip
Downloading https://github.com/shtrom/AuroraPlus/archive/refs/heads/more-than-1-day.zip (6.7 kB)
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: requests in /usr/lib/python3.11/site-packages (from auroraplus==1.1.6) (2.28.2)
Requirement already satisfied: charset-normalizer<4,>=2 in /usr/lib/python3.11/site-packages (from requests->auroraplus==1.1.6) (3.2.0)
Requirement already satisfied: idna<4,>=2.5 in /usr/lib/python3.11/site-packages (from requests->auroraplus==1.1.6) (3.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.11/site-packages (from requests->auroraplus==1.1.6) (1.26.16)
Requirement already satisfied: certifi>=2017.4.17 in /usr/lib/python3.11/site-packages (from requests->auroraplus==1.1.6) (2023.7.22)
Building wheels for collected packages: auroraplus
Building wheel for auroraplus (setup.py): started
Building wheel for auroraplus (setup.py): finished with status 'done'
Created wheel for auroraplus: filename=auroraplus-1.1.6-py3-none-any.whl size=4376 sha256=8e78b487be1c07af179ab15202aeb9440073369bb880c3de645da421155d4ada
Stored in directory: /tmp/pip-ephem-wheel-cache-zdr8traz/wheels/c3/4c/f9/fbb5d2c86727f1c5c19439e058ea9389c1efee140b57b6c3a3
Successfully built auroraplus
Installing collected packages: auroraplus
Successfully installed auroraplus-1.1.6

slothking87

Nevermind - Read the logs again and the authentication error was there - Just hidden a bit deeper

SUCCESS!! If anyone else as noob as me is reading this - The  way to run the pip command is to install the Advanced SSH & Web Terminal add on. The other SSH add on I was using didnt allow the correct access to run the commands.
After running the command and rebooting it seems to have grabbed the backdated data fairly instantly.

Woohoo! You would just have gotten one day worth of data, right? -- Olivier Mehani @.***> Sent from my mobile, please excuse my brevity.

It seems to have pulled everything back to the 19th when I check the sensors and the logs.
It does show this under the dev tools/statistics section. Previously if I deleted them and rebooted it seemed to work ok, but not this time. Are you seeing this in yours also? I might just be impatient as it has only just started to pull down data.

Ah, you must have been lucky and data for the 20th wasn't available when you first started it, so it went back one more day.
The Unsupported state class is, AFAIK, expected, as this is how ha-historical-sensor works: it doesn't have a state class (for immediate data), and instead calculates stats itself and injects them into the recorder.
Judging by the comments in the example, setting a state class would break stats, so I think we'll have to live with this warning. But I think it's harmless.
Olivier Mehani @.***>
Sent from my mobile, please excuse my brevity.

That makes sense. So far nothing has imported into the energy dashboard. It has recorded my cakes up to the 13 when I started trying to make changes, then nothing. I guess I'll see if it imports tomorrow morning with the new data. At least I can see it in the sensor information

They should show up as soon as recorded.
Make sure you are using the right entities. You might have to remove them from the Energy config, delete the unavailable one, again, and re-do the config.
Olivier Mehani @.***>
Sent from my mobile, please excuse my brevity.

That did the trick. I had to remove the config, reboot, remove the old statistics and sensors, reboot, put the config back in and reboot once more and that gave home the correct things to put in the energy dashboard. It cleared any history I had in there - but it at least works now!

Cool. I think you're good to go now!
I've had it running for about a week with no new changes required, so I think it's pretty solid now. The only issue seems to be data in Aurora being slow to show up (up to multiple days).
Let's see how you fare. Barring install issues that you had due to running forked code, the rest should be smooth now.
Olivier Mehani @.***>
Sent from my mobile, please excuse my brevity.

Yep. All looking good so far - Thanks heaps for your help (and patience). Out of interest how far back should it be grabbing historical data?

tl;dr: it only looks as far as the most recent day with data.
The Aurora+ API lags a bit before it has data, so it tells you whether data is available or not for each day.
If data is not available, the AuroraPlusHA historical sensors will try the day before, all the way to the latest day that has some data available. If the AuroraPlus API makes multiple days available at once (or within an hour, which is the refresh period), the integration will only look at the latest, and not go backwards any further.
The HistoricalState then only insert data that is newer than the last recorded data, and ignores older data, even if it was previously missing.
There might be a way to play further with recorder to 1) find gaps in statistics, 2) query those from the Aurora+ API and 3) backfill the data in he recorder. However, I don't think it's natively supported by the HistoricalState, so would be tricky to implement.
I concluded I could live with some occasional gaps (;
Olivier Mehani @.***>
Sent from my mobile, please excuse my brevity.

@slothking87 I just pushed an interim fix to support MFA https://github.com/shtrom/AuroraPlus/tree/oauth-mfa-token to this branch.
Instructions are in this commit https://github.com/LeighCurran/AuroraPlusHA/pull/9/commits/7e277e5db346cba05effbb02f2d0cd5151773c8c
You might have to redo the dance to trash your previous state and pull a new version of the dependency library

I feel like Im missing something very obvious - But I cant get it to work.
When loading the URL mentioned in a web browser, I do get a generic error page, but with this message which makes me think Im missing something somewhere
"Sorry, but we're having trouble signing you in.
We track these errors automatically, but if the problem persists feel free to contact us. In the meantime, please try again.
Correlation ID: 2ba71a8f-221c-4b05-8d0d-1e1d6ade01c9
Timestamp: 2023-11-29 06:14:55Z
AADB2C99059: The supplied request must present a code_challenge"

Yeah, the url should have been generated with a code challenge...
Can you show me the code you used, and the url (this one is safe to share as it's protected by the code_challenge so it's locked to your code, but feel free to edit parts out if you want)
Olivier Mehani @.***>
Sent from my mobile, please excuse my brevity.

Excuse the noob questions - So the process is to run the commands as they are listed and that opens the correct link?
The code I got was identical to yours so guessing I missed something or something didnt want to work.

Ok, so this URL should be pasted back into the api.redirect_uri(), and you'll get an access_token.
Then, yes, update your HA secrets to have username='', and password=<access_token>, and restart HA. If you're in luck, you'll start getting data again.

PS: be careful about posting redirect_uris (I was talking about posting the first URL that is returned by oauth_authorize), as they do contain secrets.
In this case, the secret is protected by the code_challenge that only your client api instance has, but best be prudent.

If you're in luck, you'll start getting data again.

I am!

so close! I got the extra information, but getting the usual authentication error inside home assistant. How much of that final string is whats needed for the access token?
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 359, in _async_setup_platform
await asyncio.shield(task)
File "/config/custom_components/auroraplus/sensor.py", line 98, in async_setup_platform
AuroraPlus = await hass.async_add_executor_job(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/auroraplus/sensor.py", line 96, in aurora_init
session.getmonth()
File "/usr/local/lib/python3.11/site-packages/auroraplus/init.py", line 89, in getmonth
self.month = self.request("month", index)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/auroraplus/init.py", line 63, in request
+ self.serviceAgreementID
^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'api' object has no attribute 'serviceAgreementID'

Ok, same thing here on my non-test HA deployment. Hold tight.

You need to redo what you did there https://github.com/LeighCurran/AuroraPlusHA/pull/9#issuecomment-1685250663 to get the new version of the auroraplus lib.

Ok, that didn't work. But I realised existing instances do need an update in the config/custom_components/auroraplus/manifest.json to point to the new branch: https://github.com/shtrom/AuroraPlusHA/commit/7e277e5db346cba05effbb02f2d0cd5151773c8c, then a restart.
I also had to delete and remove and reinstall the previous install of the auroraplus:
rm -rf /usr/local/lib/python3.11/site-packages/auroraplus*
pip install --upgrade --no-cache-dir   git+https://github.com/shtrom/AuroraPlus@oauth-mfa-token

You can check with
cat /usr/local/lib/python3.11/site-packages/auroraplus/__init__.py

that it corresponds to the version at https://github.com/shtrom/AuroraPlus/blob/oauth-mfa-token/auroraplus/init.py

Hmmm, Im still getting the same error as before with the no service agreement ID. Ill have a better play over the weekend - Im probably doing something wrong somewhere

As usual, Aurora had a wobble just when I was testing, but it came good today, so it works on mine.
If you still have the serviceAgreementID, it's likely you're not on the updated branch yet. I'm not sure how you managed to update it last time, but if you can ssh into the system that runs it, check the __init__.py and compare it to the branch. If it still is different, this means that the lib is not updated.
Worse case scenario, just delete the auroraplus directory (the Python lib, not the HA integration), and it should get reinstalled, hopefully to the correct version, on restart.

Bingo - That worked!!!!!!
IF anyone uses supervised HA on a RPI like I do, You can run the pip commands via an add on (https://github.com/home-assistant/addons-development).  Just add the command (pip install git+https://github.com/shtrom/AuroraPlus@oauth-mfa-token in this case) under the pypi option, save, then start the add on and it does the rest for you. keep an eye on the logs tab  to see when its done.

Awesome!
I still need to work out how to make HA fetch and record the token, rather than having to put it in manually, but that's a good functional workaround in the meantime.

Ok, it looks like the access_token has a three-week expiry. Data stopped coming on the 20th, I just had to put a new one in my secrets.yaml and restart. I got data for yesterday straight away, as expected. Let's see if that expires around the 17th.
If so, that'll make the need for a way to refresh the token more pressing... If that's indeed something that the Aurora+ Web UI supports. Otherwise, we'll have to at least streamline the process of obtaining a new token.

@slothking87 re: https://github.com/LeighCurran/AuroraPlus/pull/3#issuecomment-1884213483
I just updated the ha-historical-sensor as per https://github.com/ldotlopez/ha-historical-sensor/issues/12, and pushed it.
This should fix your issue on 2024.1.2, but you'll have to do the usual dance of trashing the old python dep to and make sure the new one is installed.

Ok, it looks like the access_token has a three-week expiry. Data stopped coming on the 20th, I just had to put a new one in my secrets.yaml and restart. I got data for yesterday straight away, as expected. Let's see if that expires around the 17th.

It lasted until the 25th! Sound like a... 29 day lifetime? I'm renewing it today, so I guess we'll see on the 26th of Feb?

(psst, have a try at #11 for a UI-based configuration of the access token).