Change $layout.svelte to @layout.svelte and [param].svelte to {param}.svelte?

Ok hear me out.
I think we closed https://github.com/sveltejs/kit/issues/894 prematurely; it does make sense to use a character that doesn't require special handling, if possible. It's awkward to type src/routes/$layout.svelte on the command line — you get to the $ and either have to reach for the backslash, or curse your lack of forethought in not typing 'src (single quote only, no doubles!) instead of src.
Even if you don't find yourself typing/tab-completing route filenames on the command line frequently, you might find yourself attempting to cmd-click on the filename in your editor's integrated terminal (e.g. in the output from git diff or tsc or whatever) in order to edit the file. At least in VSCode, that doesn't work for filenames containing $layout, probably because something somewhere is replacing it with the layout environment variable.
These problems go away with @.
Related to the cmd-click problem: VSCode doesn't recognise this as a file — if you hover it, it will only highlight the file up to the [ character:

This, on the other hand...

...works perfectly:

I should note that this behaviour appears to be specific to VSCode; iTerm reverses the treatment of [bar] and {bar}. But I think it's probably more important that cmd-click works in integrated editor terminals, and VSCode is the most popular one of the bunch.
Aesthetically, @layout.svelte and @error.svelte seem fine, and I actually prefer {param} to [param], not least because it's more aligned with the syntax of Svelte itself.

It doesn't need to, since you don't have a reason to type $app/paths etc on the command line (and it wouldn't mean anything in that context anyway). But we could, I guess

Yeah there's no techical reason, more an alignment question

@GrygrFlzr points out that @ is a nuisance character in a lot of chat contexts (discord, github, twitter). Maybe +layout.svelte?

As a vscode user myself I think this change would be great and I'd avoid a considerate amount of URL copy/pasting around. I remember NextJS RFC about this but I don't think the {} option was proposed or discarded at any point.

Can someone remind me what is the downside of having layout.svelte without any prefix? Or why do we need to prefix the file in the first place?

Ha yes, Next and Nuxt explicitly based their dynamic parameter syntax on Sapper — I guess past me accidentally sabotaged them.
The prefix is there because xyz.svelte creates the /xyz route, but layout.svelte doesn't create the /layout route. You need some way to indicate that it's a 'special' file (simply saying 'layout and error are special names' doesn't work, because we might need to add to that list of names over time — as we did recently — and it would be a breaking change if we didn't have a prefix because someone might already be using the name we want to add). In Sapper that was done with the _ prefix, but that's also used for private modules/components, and suffers from the same problem of not being future-proof.

+ is somewhat less ergonomic on a QWERTY keyboard and requires finagling your right hand or the combination of left hand shift plus right hand =, versus the current $ being reachable by the left hand. This isn't a common use case if we only apply it to the files, but if we align $lib to +lib, that's a much more frequently-typed thing that would quite literally be more painful to type.
Also I think it +app/env looks funny, but that's hardly a strong argument.

Thanks for the clarification @Rich-Harris and I see your point. Looks like we have 2 "special" files (layout, error) that need to be excluded from route creation. It looks like we're saying that the overhead of adding new files to an exclusion list is greater than having a special character to signal this is a 'special' file but at the same time, any special files need to have some code changes associated that would handle those use-cases.
When you refer to a breaking change, is this in regards to SvelteKit, sapper or something else? If we're talking about SvelteKit this current stage might be the right time to have breaking changes (if there even a right time for that 😀)
Is it an option to have a new folder, something like kit (same level as lib) where all these special files would live, this way we'd avoid collisions and we would be able to keep the file name simple?

Some findings about the VS Code Ctrl+Click behavior on Windows:
src/routes/[dynamic]/[nested]/test.svelte goes to the folder
src/routes/[dynamic]/[nested]/test.svelte shows files inside the dynamic folder
src/routes/[dynamic]/[nested]/test.svelte goes to the file
src/routes/[dynamic]/[nested]/test.svelte does a search for test.svelte files if there's more than one
This works the same for {}.

I would lean to either # or ~ in that order as a replacement for $.

# has the same issue as @ in terms of being annoying on chat/social media platforms (channel names, hashtags), also I think Vite uses it internally so it might break? Not sure about potential issues with ~.

Ha yes, Next and Nuxt explicitly based their dynamic parameter syntax on Sapper — I guess past me accidentally sabotaged them.
The prefix is there because xyz.svelte creates the /xyz route, but layout.svelte doesn't create the /layout route. You need some way to indicate that it's a 'special' file (simply saying 'layout and error are special names' doesn't work, because we might need to add to that list of names over time — as we did recently — and it would be a breaking change if we didn't have a prefix because someone might already be using the name we want to add). In Sapper that was done with the _ prefix, but that's also used for private modules/components, and suffers from the same problem of not being future-proof.

Almost all the other solutions mentioned above give me the gut feeling that things won't work elsewhere. I think using _ for layout/error and - for private routes makes the most sense. _ and - are almost always allowed on files and in the average command line because _ often gets used as a alphabetic sorting hack and - as a alternative to a space  .

@Rich-Harris yeah I agree, while I was typing I got to the same conclusion.
So ATM it looks like we have a few options (@, _, $, +) for file special file prefixing, what are the criteria that we'd use to decide the best option?

For dynamic parameters, it seems that the only plausible characters that don't behave differently between VSCode and iTerm (iTerm makes filenames with [] and () interactive but VSCode doesn't; VSCode makes filenames with {} and <> interactive but iTerm doesn't) are pipes:
src/routes/|dynamic|/|nested|/test.svelte

That becomes interactive (i.e. cmd-clickable) in iTerm and VSCode; I can't guarantee the same is true everywhere, but it seems like the surest bet.
I still think @ is the best option for layout/error files. In github/discord it's very easy to wrap in backticks, and I'm not too concerned about the twitter case. There is good precedent for its use in scoped packages (though incidentally, this is a reason not to use it for @app/* etc, but rather to keep those as $).

\ / : * ? " < > | are all disallowed characters in filenames on Windows.

windows: pissing in the punchbowl since 1985

ok, new proposal!
src/routes/%dynamic%/%nested%/index.svelte

Works without backticks, interactive in both iTerm and VSCode, allowed in windows

One more.
src/routes/~dynamic~/~nested~/index.svelte
src/routes/~dynamic~/~nested~/~layout.svelte

Tilde has a nice calming wavy effect. Ergonomically easy to reach.

Update:
src/routes/~dynamic/~nested/index.svelte
src/routes/~dynamic/~nested/~layout.svelte

Just start the param w/ it. Even easier.

With the latter variant, layout, error, and any future files we want to implement becomes a special reserved keyword.

@GrygrFlzr yeah, I immediately realized, and deleted my comment—which is why you're yelling into the void.

Here's an argument for % that I think is solid: you can't use it in a URL pathname (unless it's followed by characters that are being %-encoded). The other characters we've been discussing don't have that property (source). ([] is also safe on that basis, incidentally.) If we used ~ or =, for example, it would make it harder to use them as literals.
We can also reduce the ugliness by 50%, by having %param.svelte instead of %param%.svelte. (We'd need to use both in more complex cases like %param%-foo.svelte or %param%-%bar%.svelte, but those are very much the minority.)

One major downside to % — Vite barfs on it:

If you change the import to be URL-encoded, you get this:

Haven't delved into it too much yet but I think this is a bug in Vite — I suspect it should be encoding import URLs when it transforms modules

Yeah, I think this is something that Vite could address. It's totally fine to have modules with % in the filename:

I'm indifferent about changing prefixes for stuff like layout files, but I for sure don't like trading [] or {} for another pair of characters. The 80% use case is reading the file names or creating them, not clicking on the file names in some terminal that doesn't know how to read it. So it feels totally wrong for me to optimize for the 20% use case and trade the readability and convention (Sapper, Next etc use []) for something that happens to work on VS Code.

@dummdidumm I'm coming around to this opinion.

Have we discarded the underscore (_) option?
src/routes/_dynamic_/_nested_/index.svelte // dynamic path, non-dynamic file
src/routes/_dynamic_/_nested_/file_name.svelte // dynamic path, non-dynamic file
src/routes/_dynamic_/_nested_/file-name.svelte // dynamic path, non-dynamic file
src/routes/_dynamic_/_nested_/_name_.svelte  // dynamic path, dynamic file
src/routes/_dynamic_/_nested_/_var1___var2_.svelte // dynamic path, dynamic file
src/routes/_dynamic_/_nested_/_var1_-_var2_.svelte  // dynamic path, dynamic file
src/routes/_dynamic_/_nested_/_private.svelte  // dynamic path, non-dynamic and private file
<EMAIL_ADDRESS>// dynamic path, reserved file

_ is a safe character so no encoding required and it seems to work fine vscode/term/browser(devtools)

I do like how expressive [ ] and { } are but at the same time not being able to click-through files and I need to manually find the file causes some frustration. The fact that I need to manually find/goto file it slows down the flow and doesn't feel like a great developer experience (not saying is bad, but not great).
I'm not saying we should change this convention, but if it slightly improves the development experience for everyone, wouldn't we want that @dummdidumm ?

_ already has meaning (private modules/components), so either we have to find a new solution for that, or we can't use it for this

I've added an example that covers that:
src/routes/_dynamic_/_nested_/_private.svelte  // dynamic path, non-dynamic and private file
dynamic routing is wrapped by _ while private is prefixed. Do you reckon this wouldn't be feasible?

feasible, but deeply confusing

Honestly, I'm leaning more toward this being a "bug" with IDEs / terminals that provide clickable links. I no longer favor ~ tilde.

How about using . for private modules/components and _ for layouts / errors?

. files are sometimes entirely hidden in .gitignore
node_modules
.*
# etc.

. files are sometimes entirely hidden in .gitignore
node_modules
.*
# etc.

Any freshly created nuxt project always came with an preconfigured .gitignore based on your projects configuration. SvelteKit could offer a preconfigured .gitignore when creating a fresh svelte project too. If we add all the major . prefixed files that people generally try to target with .* we may be on to something.

afraid that's a non-starter

What about using * for those files? like src/*layout.svelte, will it work with vscode and iTerm?

What about using * for those files? like src/routes/*layout.svelte, will it work with vscode and iTerm?

https://github.com/sveltejs/kit/issues/1149#issuecomment-823319922

missed that

Here's an issue for the VS Code bug where I posted some details showing the line number where the bug is in case anyone wants to try fixing VS Code instead of changing SvelteKit: https://github.com/microsoft/vscode/issues/118687

^ would be another option. = as well
.I still don't think that _ should be out of the running. I'm not sure why it's an issues that it's used for private modules. We don't want _layout.svelte or _error.svelte to be routable so it seems in keeping with the private aspect of it

It's an issue because if we later want to add _foo.svelte with some other special meaning, it's a breaking change because anyone could have already been using it with no special meaning as a component they can import from other pages, and upgrading would then likely break their apps.

Core team has been bikeshedding this, and we think the best option is to use two underscores for special files:

__layout.svelte
__layout.reset.svelte
__error.svelte
_private.js

We don't like any of the alternatives to [params], so we're just going to cross our fingers and hope for a VSCode fix

We need to make sure to change this for the manifest files:
https://github.com/sveltejs/kit/blob/3b21b7e558e33885f40b7e09befb4c6c3d86b958/packages/kit/src/core/dev/index.js#L58-L63


HTML5 History API

I were told to ask if java-express supports the HTML5 history API? If not, would it be possible to add it?

I'm not actively working on it anymore (as you can see on the last-commit date), but I may take a look at it in the next days.

Sure thing! Sounds good. Thanks a lot

You could just provide your own fallback, this way you could control how and in which case a fallback should be given:
// Put this line to the very end of you request-handlers to catch all 404 requests
app.all((req, res) -> {

   // Fallback file
   res.send(Paths.get("the/path/to/your/index.html"));
});

Sorry for late response. It was resolved. Thanks!


Enable fast reclaimation of working_dir

Experiment to see if a new parameter --upcycle could be used to keep around and reuse 4.x specific working directories for jobs like scan-sources. This also avoids the long rm -rf runs associated with these workspaces.

Codecov Report

Merging #337 (25e6aad) into master (670fccd) will decrease coverage by 0.06%.
The diff coverage is 3.44%.

@@            Coverage Diff             @@
##           master     #337      +/-   ##
==========================================
- Coverage   27.51%   27.45%   -0.07%
==========================================
  Files          33       33
  Lines        7462     7482      +20
  Branches     1495     1501       +6
==========================================
+ Hits         2053     2054       +1
- Misses       5320     5338      +18
- Partials       89       90       +1

Impacted Files
Coverage Δ

doozerlib/cli/images_streams.py
0.00% <0.00%> (ø)

doozerlib/distgit.py
31.59% <0.00%> (-0.14%)
:arrow_down:

doozerlib/gitdata.py
17.85% <0.00%> (-0.33%)
:arrow_down:

doozerlib/runtime.py
27.24% <0.00%> (-0.39%)
:arrow_down:

doozerlib/cli/__init__.py
59.74% <100.00%> (+0.52%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 670fccd...25e6aad. Read the comment docs.

@jupierce seems this has something related with https://github.com/openshift/doozer/pull/323/, which just cleans up the cloned repo, fetches , and resets to the latest remote branches. I could be wrong. Any thoughts on integrating them together?

/lgtm

/lgtm


Notifications

Have a way to notify users (maybe groups as well?) for comments on requests, or when certain statuses are done. Maybe add a button for reviewers?

Closing in favor of tracking this in #88.


Can't see Variables in python (conda 3.7.10)

Issue Type: Bug

variables "button" in interactive window doesn't work (dont show the script variables after cell run)
Jupiter:Variables table doesn't show any variables after running cell

VS Code version: Code 1.59.0 (379476f0e13988d90fab105c5c19e7abc8b1dea8, 2021-08-04T23:13:12.822Z)
OS version: Windows_NT x64 10.0.19043
Restricted Mode: No

System Info

Item
Value

Extensions (6)

Extension
Author (truncated)
Version

anaconda-extension-pack
ms-
1.0.1

python
ms-
2021.8.1105858891

vscode-pylance
ms-
2021.8.0

jupyter
ms-
2021.8.1195043623

Dupe of #6993


KAFKA-12503: cache size set by the thread

Make is so threads do not directly resize other threads caches
Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

@guozhangwang @ableegoldman Can you take a look?
@vvcephei I think this needs to be picked to 2.8 as well

One unrelated test failure: kafka.api.SaslGssapiSslEndToEndAuthorizationTest.testNoConsumeWithDescribeAclViaAssign

I will don't know about testing this via an integration test. I worry about introducing just another flaky test, these types of issue get caught mostly in soak. Even with the optimal conditions for it to go wrong it took a couple days.
I will think about how to improve the tests but that will happen in a different PR for sure

I don't think we should aim to catch things in soak, although I'm not on the side of "we should never catch anything in soak/everything must have an integration test" either. This particular issue does seem appropriate for an integration test -- as I said concurrency bugs are hard to actually reproduce, but we should absolutely be flexing the cache resize codepath in an integration test. I'm happy to leave that as followup work as long as we do get around to it.
This looks good and the build passed so I'll merge this to unblock the release

Merged and cherrypicked to 2.8 cc @vvcephei


Raidboss: infoText overrides tts for spoken alerts

Description
Since the latest version of cactbot, any triggers that have both an infoText and tts property now have the tts value ignored when doing a spoken alert.
Ex:
Options.SpokenAlertsEnabled = true;
Options.Triggers = [
	{
		zoneRegex: /.*/,
		triggers: [
			{
				regex: /test/,
				infoText: 'Foo',
				tts: 'Bar'
			}
		]
	}
]

If you do /e test in game, this trigger will play "Foo" over TTS instead of "Bar".

Should be fixed with 45fc915f.


aws-sdk-cpp: include dev output as header path hint

Description of changes
This includes the dev output in the hints to find the aws/core/Aws.h header.
Before this change, you would likely need to pass AWS_CORE_HEADER_FILE explicitly.
This is especially useful when using the find_package(AWSSDK) workflow.
Usage of individual targets is still pretty painless as install(TARGETS respects the $out + $dev split we do.
Related upstream issue: https://github.com/aws/aws-sdk-cpp/issues/2009
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Merging once green

Ping, merge conflict but still LGTM otherwise

rebased on master

Wrong indentation in the cmake file but indentation in those are just visual
94-        "C:/AWSSDK/${AWSSDK_INSTALL_INCLUDEDIR}/aws/core"
95:    "/nix/store/ha5w42s778vvmr7liq5z4azjmwlx5c1h-aws-sdk-cpp-1.11.37-dev/include/aws/core"


Added tests for CardStackAnimationOptions

Codecov Report

Merging #16 into master will increase coverage by 0.38%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master      #16      +/-   ##
==========================================
+ Coverage   79.04%   79.43%   +0.38%
==========================================
  Files          17       17
  Lines         778      778
==========================================
+ Hits          615      618       +3
+ Misses        163      160       -3

Impacted Files
Coverage Δ

...fle/SwipeCardStack/CardStackAnimationOptions.swift
100% <0%> (+18.75%)
:arrow_up:

This should close #6


Some new XWAU2024 reticles not displayed in VR

Multiple reports:
https://forums.xwaupgrade.com/viewtopic.php?t=14390#p193535
https://forums.xwaupgrade.com/viewtopic.php?t=14389
https://discord.com/channels/851918539496620062/1104824817519181878/1261477319005048906
https://discord.com/channels/851918539496620062/1104824817519181878/1261583875449032744
It seems the impacted ships are:
The X-Wing has no laser sight, but does have a missile sight
The Y-Wing has no laser sight, and no missile sight, but missile ammo counts do appear floating above the dash when switched to missiles.
The B-Wing Fighter didn't have an aiming reticle for lasers/ions. It did for torpedoes, thankfully.
According to Ace, the reticles were probably updated in XWAU2024_beta3.

changes are to Y-wing, B-wing, Z-95 and X-wing I believe. Some partially, the Y-wing completely
They had new laser or missile sights. Or adjustments to exciting.
So they used new dat numbers

@Prof-Butts can you document here your findings? I will try to debug on my side and it will make coordination in different timezones easier 😄

It seems this function is actually the culprit:
https://github.com/Prof-Butts/xwa_ddraw_d3d11/blob/master/impl11/ddraw/Reticle.cpp#L56
Instead of that "stupid numbering system" 😄 it should be just iterating the DatFile and retrieving the metadata for the element in the position/slot/index specified by <FlightModel>.ini
The question is, should it be implemented in JeremyAnsel.Xwa.Dat or in DATReader?

Fixed in 7a028ef057b7b06c8ef4e06ba18a52ae15720ace


Better error messages

Description
The logic in place for error parsing was not logging the important parts of the error to the console log in the browser. For some reason, Chrome does not print the cause field of the error object, as it should.
So, I've added a new log statement that always prints out the error object from the response. Also, I attempt to parse for Neptune errors a bit more generously. So there is a better chance of extracting the message details if the error object contains it.
Validation

Verified by schema sync with my SSH tunnel off so I receive connection refused errors

Related Issues

Check List

[x] I confirm that my contribution is made under the terms of the Apache 2.0
license.
[x] I have run pnpm checks to ensure code compiles and meets standards.
[x] I have run pnpm test to check if all tests are passing.
[ ] I have covered new added functionality with unit tests if necessary.
[x] I have added an entry in the Changelog.md.

LGTM just a small nit to discuss but not blocking the PR


Styling/coding standards

Which coding standards/PEP guides are you planning to adhere to.
As this is a python library aimed at python users(presumably) the PEPs  would be a good place to start.
Also when implementing the classes there are a couple options regarding the OOP interface as I see it:

literal translations so getPos player  becomes player.getPos(). or player.get_pos()
@property decorator to wrap the calls to the DLL to the user so that getPos player  becomes player.pos , and  player setPos [a,b,c] becomes player.pos = [a,b,c].
Both of the above, having one approach call the other for example. This illustrates both approaches, and is therefor less pythonic ("one way to do something").

     class RV_Object(object):
        def __init__(self):
           # todo
           pass

        @property
        def pos(self)->tuple:pass

        @pos.setter
        def pos(self, pos:tuple)->None:pass

        def get_pos(self)->tuple: pass # todo

        def set_pos(self, pos:tuple)->None: pass # todo

Using PEP8 would be a good start ;).
I'd suggest doing as "everyone else does" and increasing the line size limit to 120 chars (with 1080p and even 4k monitors nowadays)
I'm sitting on a fence regarding the camelCase vs snake_notation. I'm used to snake but I also see the benefit of sticking to the original SQF function names. Deep inside me I'd prefer to stick with the original naming (less confusion) but I haven't done enough Arma modding (nor even used Intercept much) to speak authoritatively about this matter.

I have some boilerplate code here: https://github.com/Tirpitz93/intercept-python/tree/python_classes.


Irradiance forecasts no longer available

Hi there,
I have recently upgraded to HA 2023.9.3 and I have upgraded dwd_weather through the GUI to the latest version (should be 2.0.1).
I have been using my own template sensors to calculate total rest of day or rolling 24 hour irradiance forecasts and the like based on dwd_weather to have an idea how much heat to expect from my solar panels in the near future and to make decisions off of that.
Here is one example that used to work for several months prior to the two upgrades (HA and dwd_weather) I made:
- platform: template
  sensors:
    sun_irradiance_staaken_rest_of_day_forecast:
      friendly_name: "Rest of day forecast of irradiance"
      unit_of_measurement: 'W/m^2'
      value_template: >-
        {% set irrTotal = namespace(value=0) %}
        {% set current_hour = now().hour %}
        {% set hours_left = 24 - current_hour %}
        {% for irr in state_attr('sensor.sun_irradiance_staaken', 'data') -%}
          {% if loop.index <= hours_left %}
            {% set irrTotal.value = irrTotal.value + irr.value %}
          {% else %}
            {% break %}
          {% endif %}
        {%- endfor %}
        {{ irrTotal.value }}

After the upgrades I found the following error in the logs:

TemplateError('TypeError: unsupported operand type(s) for +: 'int' and 'NoneType'') while processing template 'Template<template=({% set irrTotal = namespace(value=0) %} {% set current_hour = now().hour %} {% set hours_left = 24 - current_hour %} {% for irr in state_attr('sensor.sun_irradiance_staaken', 'data') -%} {% if loop.index <= hours_left %} {% set irrTotal.value = irrTotal.value + irr.value %} {% else %} {% break %} {% endif %} {%- endfor %} {{ irrTotal.value }}) renders=4>' for attribute '_attr_native_value' in entity 'sensor.sun_irradiance_staaken_rest_of_day_forecast'

I checked the sensor sensor.sun_irradiance_staaken in the developer tools and it showed the sensor's state as "unknown" and the attribute values looked like this:

state_class: measurement
data:

datetime: '2023-10-04T19:00:00.000Z'
value: null
datetime: '2023-10-04T20:00:00.000Z'
value: null
datetime: '2023-10-04T21:00:00.000Z'
value: null
datetime: '2023-10-04T22:00:00.000Z'
value: null
datetime: '2023-10-04T23:00:00.000Z'
value: null
(...)

Before the upgrades I would see numeric values instead of the null values
Since I head read the notes on the new trigger/action in templates and the breaking change in dwd_weather in the latest release I thought it might have to do with that and I updated my sensor definition to:
- template:
  - trigger:
      - platform: time_pattern
        hours: "/1"
      - platform: homeassistant
        event: start
      - platform: event
        event_type: event_template_reloaded
    action:
      - service: weather.get_forecast
        data:
          type: hourly
        target:
          entity_id: weather.staaken_staaken
        response_variable: irradiance_data
    sensor:
      - name: "Rest of day forecast of irradiance"
        unique_id: sun_irradiance_staaken_rest_of_day_forecast
        state: >-
          {% set irrTotal = namespace(value=0) %}
          {% set current_hour = now().hour %}
          {% set hours_left = 24 - current_hour %}
          {% for irr in irradiance_data.data -%}
            {% if loop.index <= hours_left %}
              {% set irrTotal.value = irrTotal.value + irr.value %}
            {% else %}
              {% break %}
            {% endif %}
          {%- endfor %}
          {{ irrTotal.value }}
        unit_of_measurement: "W/m^2"

after that the error changed and it now is:

Template variable warning: 'dict object' has no attribute 'data' when rendering '{% set irrTotal = namespace(value=0) %} {% set current_hour = now().hour %} {% set hours_left = 24 - current_hour %} {% for irr in irradiance_data.data -%} {% if loop.index <= hours_left %} {% set irrTotal.value = irrTotal.value + irr.value %} {% else %} {% break %} {% endif %} {%- endfor %} {{ irrTotal.value }}'

so I manually tested the new get_forecast service like:
service: weather.get_forecast
data:
  type: hourly
target:
  entity_id: weather.staaken_staaken

and I got this:

forecast:

datetime: "2023-10-04T19:00:00Z"
condition: cloudy
wind_bearing: SW
wind_gusts: 5.14
precipitation_probability: 11
temperature: 14
templow: 14
wind_speed: 2.57
precipitation: 0
(...)

This tells me two things:

I will slightly have to adjust my code as the structure has changed a bit
there is no point in fixing the issue above as the attribute I have always been using (irradiance) is no longer part of the forecast

Obviously I am missing something but I can't figure out what it is. Any hints how to solve this and get the hourly forecasts for irradiance back would be highly appreciated!
Thanks,
Alex

i have a similar issue (station „X104“). the sensors sensor.precipitation_probability_berlin_tegel and sensor.berlin_tegel_niederschlagsdauer (i renamed the first entity id, btw) are unavailable and i get this error:
Dieser Fehler wurde von einer benutzerdefinierten Integration verursacht

Logger: homeassistant
Source: custom_components/dwd_weather/connector.py:215
Integration: Deutscher Wetterdienst (documentation, issues)
First occurred: 19:39:04 (10 occurrences)
Last logged: 19:48:07

Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/home/homeassistant/homeassistant_p3.11/lib/python3.11/site-packages/homeassistant/helpers/update_coordinator.py", line 233, in _handle_refresh_interval
    await self._async_refresh(log_failures=True, scheduled=True)
  File "/home/homeassistant/homeassistant_p3.11/lib/python3.11/site-packages/homeassistant/helpers/update_coordinator.py", line 389, in _async_refresh
    self.async_update_listeners()
  File "/home/homeassistant/homeassistant_p3.11/lib/python3.11/site-packages/homeassistant/helpers/update_coordinator.py", line 172, in async_update_listeners
    update_callback()
  File "/home/homeassistant/homeassistant_p3.11/lib/python3.11/site-packages/homeassistant/helpers/entity.py", line 779, in async_write_ha_state
    self._async_write_ha_state()
  File "/home/homeassistant/homeassistant_p3.11/lib/python3.11/site-packages/homeassistant/helpers/entity.py", line 879, in _async_write_ha_state
    state, attr = self._async_generate_attributes()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/homeassistant/homeassistant_p3.11/lib/python3.11/site-packages/homeassistant/helpers/entity.py", line 820, in _async_generate_attributes
    state = self._stringify_state(available)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/homeassistant/homeassistant_p3.11/lib/python3.11/site-packages/homeassistant/helpers/entity.py", line 785, in _stringify_state
    if (state := self.state) is None:
                 ^^^^^^^^^^
  File "/home/homeassistant/.homeassistant/custom_components/dwd_weather/sensor.py", line 288, in state
    result = self._connector.get_precipitation_probability()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/homeassistant/.homeassistant/custom_components/dwd_weather/connector.py", line 289, in get_precipitation_probability
    return self.get_weather_value(WeatherDataType.PRECIPITATION_PROBABILITY)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/homeassistant/.homeassistant/custom_components/dwd_weather/connector.py", line 215, in get_weather_value
    value = self.dwd_weather.get_reported_weather(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/homeassistant/homeassistant_p3.11/lib/python3.11/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 350, in get_reported_weather
    return self.report_data[weatherDataType.value[0]]
           ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'wwP'

the attribute values of both sensors are null, too.

so it seems that either the station does not offer those particular values or that theres a bug in the simple_dwd_weatherforecast library that occurs while processing these values. either way, it would be better to handle both cases more gracefully, i.e. indicating that the sensor values are not available  at this particular station by disabling the sensors or indicate that there’s been an error handling the values by logging a more accurate error.

@almiho:

I will slightly have to adjust my code as the structure has changed a bit
there is no point in fixing the issue above as the attribute

There shouln't be changes to the sensor values. The problem is somewhere else.

I have always been using (irradiance) is no longer part of the forecast

The forecast breaking change is only related to the weather entity so you should be able to use them if we can get it working for you again.
Can you please upload the debug log, so I can troubleshoot?

@diplix the station X104 you have mentioned should have the sensor values you are looking for. The error looks like some kind of network error, as this specific case should not happen. Can you file a new issue for this and upload your debug log as well? Thanks.

@almiho:

I will slightly have to adjust my code as the structure has changed a bit
there is no point in fixing the issue above as the attribute

There shouln't be changes to the sensor values. The problem is somewhere else.
Got it, I think I was assuming that I would have to use the weather.get_forecast service rather than the ssensor sensor.sun_irradiance_staaken which I had used before. More than happy to keep things the same if we can make it work again.

I have always been using (irradiance) is no longer part of the forecast

The forecast breaking change is only related to the weather entity so you should be able to use them if we can get it working for you again.

The reason why I was assuming that it might have to do with the upgrade is that everything was working all the way up until I upgraded both HA and dwd_weather.

Can you please update to the latest version and upload the debug log, so I can troubleshoot?

Sure thing, here you go.
home-assistant_dwd_weather_2023-10-05T15-38-24.286Z.log
Thanks for your help!
Alex

@diplix the station X104 you have mentioned should have the sensor values you are looking for. The error looks like some kind of network error, as this specific case should not happen. Can you file a new issue for this and upload your debug log as well? Thanks.

i looked at it more closly in #87 and there actually is no error but rather it’s expected behavior if you enable force_hourly like i did …

This is now fixed in v2.0.3. Please let me know if not and otherwise please close this issue.

This is now fixed in v2.0.3. Please let me know if not and otherwise please close this issue.

I have upgraded HA 10.0 -> 10.1 and dwd_weather 2.0.2 -> 2.0.3 (and restarted HA, of course)
Unfortunately it looks like the symptoms are still the same for me:

is there anything else I should be doing?

Can you please give me the station id, so I can reproduce this error?

sure, it is Station N7651
this is really interesting, looks like pretty much everything but irradiance works

Unfortunately I have to confirm, that DWD has removed the data from this station :(

I have a similar problem with a particular station
Traceback (most recent call last):   File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 233, in _handle_refresh_interval     await self._async_refresh(log_failures=True, scheduled=True)   File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 389, in _async_refresh     self.async_update_listeners()   File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 172, in async_update_listeners     update_callback()   File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 779, in async_write_ha_state     self._async_write_ha_state()   File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 879, in _async_write_ha_state     state, attr = self._async_generate_attributes()                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^   File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 820, in _async_generate_attributes     state = self._stringify_state(available)             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^   File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 785, in _stringify_state     if (state := self.state) is None:                  ^^^^^^^^^^   File "/config/custom_components/dwd_weather/sensor.py", line 296, in state     result = self._connector.get_sun_duration()              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^   File "/config/custom_components/dwd_weather/connector.py", line 306, in get_sun_duration     return self.get_weather_value(WeatherDataType.SUN_DURATION)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^   File "/config/custom_components/dwd_weather/connector.py", line 220, in get_weather_value     value = self.dwd_weather.get_reported_weather(             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^   File "/usr/local/lib/python3.11/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 326, in get_reported_weather     return self.report_data[weatherDataType.value[0]]            ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^ KeyError: 'SunD1'
However, this key does exist in the dataset.
https://opendata.dwd.de/weather/local_forecasts/mos/MOSMIX_L/single_stations/H744/kml/MOSMIX_L_LATEST_H744.kmz

Unfortunately I have to confirm, that DWD has removed the data from this station :(
See here: https://opendata.dwd.de/weather/local_forecasts/mos/MOSMIX_L/single_stations/N7651/kml/MOSMIX_L_LATEST_N7651.kmz at the key Rad1h

That's almost funny, this means Murphy must have decided that the station in Staaken removes that exact attribute which I am using in that exact night in which I run my update on HA, well .. what should I say.
Anyways, I chose a different station and I got the sensors to work again but I still see error messages and exception from dwd_weather in the protocol.
I am still running HA 2023.10.1 and I have upgraded to dwd_weather 2.0.4 now.
I have removed the old weather station "Staaken" and I added a new one from FU Berlin

After starting up HA I see two error message in the protocol, the first one is

Logger: homeassistant.components.sensor
Source: helpers/entity_platform.py:507
Integration: Sensor (documentation, issues)
First occurred: 23:23:55 (1 occurrences)
Last logged: 23:23:55
Error adding entities for domain sensor with platform dwd_weather
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 507, in async_add_entities
await asyncio.gather(*tasks)
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 752, in _async_add_entity
await entity.add_to_platform_finish()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1057, in add_to_platform_finish
self.async_write_ha_state()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 779, in async_write_ha_state
self._async_write_ha_state()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 879, in _async_write_ha_state
state, attr = self._async_generate_attributes()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 820, in _async_generate_attributes
state = self._stringify_state(available)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 785, in _stringify_state
if (state := self.state) is None:
^^^^^^^^^^
File "/config/custom_components/dwd_weather/sensor.py", line 288, in state
result = self._connector.get_precipitation_probability()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/dwd_weather/connector.py", line 294, in get_precipitation_probability
return self.get_weather_value(WeatherDataType.PRECIPITATION_PROBABILITY)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/dwd_weather/connector.py", line 220, in get_weather_value
value = self.dwd_weather.get_reported_weather(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 326, in get_reported_weather
return self.report_data[weatherDataType.value[0]]
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'wwP'

and the second one is:

Logger: homeassistant.components.sensor
Source: helpers/entity_platform.py:367
Integration: Sensor (documentation, issues)
First occurred: 23:23:55 (1 occurrences)
Last logged: 23:23:55
Error while setting up dwd_weather platform for sensor
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 367, in _async_setup_platform
await asyncio.gather(*pending)
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 507, in async_add_entities
await asyncio.gather(*tasks)
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 752, in _async_add_entity
await entity.add_to_platform_finish()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1057, in add_to_platform_finish
self.async_write_ha_state()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 779, in async_write_ha_state
self._async_write_ha_state()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 879, in _async_write_ha_state
state, attr = self._async_generate_attributes()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 820, in _async_generate_attributes
state = self._stringify_state(available)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 785, in _stringify_state
if (state := self.state) is None:
^^^^^^^^^^
File "/config/custom_components/dwd_weather/sensor.py", line 288, in state
result = self._connector.get_precipitation_probability()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/dwd_weather/connector.py", line 294, in get_precipitation_probability
return self.get_weather_value(WeatherDataType.PRECIPITATION_PROBABILITY)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/dwd_weather/connector.py", line 220, in get_weather_value
value = self.dwd_weather.get_reported_weather(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 326, in get_reported_weather
return self.report_data[weatherDataType.value[0]]
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'wwP'

if I wait for 3 minutes another error message shows up:

Dieser Fehler wurde von einer benutzerdefinierten Integration verursacht
Logger: homeassistant
Source: custom_components/dwd_weather/connector.py:220
Integration: Deutscher Wetterdienst (documentation, issues)
First occurred: 23:24:54 (3 occurrences)
Last logged: 23:26:54
Error doing job: Task exception was never retrieved
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 233, in _handle_refresh_interval
await self._async_refresh(log_failures=True, scheduled=True)
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 389, in _async_refresh
self.async_update_listeners()
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 172, in async_update_listeners
update_callback()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 779, in async_write_ha_state
self._async_write_ha_state()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 879, in _async_write_ha_state
state, attr = self._async_generate_attributes()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 820, in _async_generate_attributes
state = self._stringify_state(available)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 785, in _stringify_state
if (state := self.state) is None:
^^^^^^^^^^
File "/config/custom_components/dwd_weather/sensor.py", line 288, in state
result = self._connector.get_precipitation_probability()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/dwd_weather/connector.py", line 294, in get_precipitation_probability
return self.get_weather_value(WeatherDataType.PRECIPITATION_PROBABILITY)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/dwd_weather/connector.py", line 220, in get_weather_value
value = self.dwd_weather.get_reported_weather(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 326, in get_reported_weather
return self.report_data[weatherDataType.value[0]]
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'wwP'

any idea what causes these errors and how I could fix them?
Thanks,
Alex

Hi all,
I think I could trace down the problem. It is based on the "mixed" mode for data retrieval, that's the default mode you are presented in a pop-up when configuring the integration:
"mixed_data": "Use report data if available and forecasted data if not (recommended)",
Thus, sensors are created that correspond to a certain selection of attributes of the MOSMIX_L KML files ("forecast_data"). When retrieving the "report_data", a different data source is used (CSV files), which provides only a smaller set of attributes. And here lies the problem in "simple_dwd_weatherforecast/dwdforecast.py".
class WeatherDataType(Enum) defines e.g.  PRECIPITATION_PROBABILITY = ["wwP", ""]  # Unit: % (0..100), where wwP is the key of the attribute of the MOSMIX data, but there is simply no equivalent in the CSV data. Resulting in the KeyError: 'wwP' error as seen above.
For now, a workaround is to set up new stations and explicitly selecting report or forecast data.

Hi, thanks for helping troubleshooting.

I think I could trace down the problem. It is based on the "mixed" mode for data retrieval, that's the default mode you are presented in a pop-up when configuring the integration:

No, this is not the cause. It lies within the library simple_dwd_weatherforecast

class WeatherDataType(Enum) defines e.g. PRECIPITATION_PROBABILITY = ["wwP", ""]  # Unit: % (0..100), where wwP is the key of the attribute of the MOSMIX data, but there is simply no equivalent in the CSV data. Resulting in the KeyError: 'wwP' error as seen above.

This is correct. I fixed the code in 2.0.5 so this error should not happen again.

I just updated to 2.0.5, rebooted HA and removed & re-added the weather station, but still:

Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 233, in _handle_refresh_interval
await self._async_refresh(log_failures=True, scheduled=True)
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 389, in _async_refresh
self.async_update_listeners()
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 172, in async_update_listeners
update_callback()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 779, in async_write_ha_state
self._async_write_ha_state()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 879, in _async_write_ha_state
state, attr = self._async_generate_attributes()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 820, in _async_generate_attributes
state = self._stringify_state(available)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 785, in _stringify_state
if (state := self.state) is None:
^^^^^^^^^^
File "/config/custom_components/dwd_weather/sensor.py", line 314, in state
result = self._connector.get_precipitation_probability()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/dwd_weather/connector.py", line 294, in get_precipitation_probability
return self.get_weather_value(WeatherDataType.PRECIPITATION_PROBABILITY)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/config/custom_components/dwd_weather/connector.py", line 220, in get_weather_value
value = self.dwd_weather.get_reported_weather(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 326, in get_reported_weather
return self.report_data[weatherDataType.value[0]]
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'wwP'

Can you please provide me the line with the configuration from the debug logs as well?

Hi, thanks for helping troubleshooting.
You're welcome

No, this is not the cause. It lies within the library simple_dwd_weatherforecast
Exactly, that's what I wrote one line below 😉

This is correct. I fixed the code in 2.0.5 so this error should not happen again.
Error still occurs. Attached debug log:
home-assistant_dwd_weather_2023-10-16T14-26-34.331Z.log

I can reproduce the error on calling  dwd_weather.get_reported_weather(dwdforecast.WeatherDataType.PRECIPITATION_PROBABILITY, True) directly from the library. That is what happens within the HA integration.
Whereas the corresponding call for the forecast data works dwd_weather.get_forecast_data(dwdforecast.WeatherDataType.PRECIPITATION_PROBABILITY, time_now)

Some new errors:
Logger: homeassistant.util.async_
Source: util/async_.py:137
First occurred: 17:00:14 (20 occurrences)
Last logged: 17:09:44
Detected blocking call to putrequest inside the event loop. This is causing stability issues.
Please report issue to the custom integration author for dwd_weather
doing blocking calls at custom_components/dwd_weather/connector.py, line 213:
return markdownify(self.dwd_weather.get_weather_report(), strip=["br"])

and
Logger: homeassistant
Source: custom_components/dwd_weather/connector.py:213
Integration: Deutscher Wetterdienst
First occurred: 13:56:13 (357 occurrences)
Last logged: 16:59:43

Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 233, in _handle_refresh_interval
    await self._async_refresh(log_failures=True, scheduled=True)
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 389, in _async_refresh
    self.async_update_listeners()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 172, in async_update_listeners
    update_callback()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 779, in async_write_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 879, in _async_write_ha_state
    state, attr = self._async_generate_attributes()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 820, in _async_generate_attributes
    state = self._stringify_state(available)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 785, in _stringify_state
    if (state := self.state) is None:
                 ^^^^^^^^^^
  File "/config/custom_components/dwd_weather/sensor.py", line 297, in state
    self._connector.get_weather_report(),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/dwd_weather/connector.py", line 213, in get_weather_report
    return markdownify(self.dwd_weather.get_weather_report(), strip=["br"])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/markdownify/__init__.py", line 242, in markdownify
    return MarkdownConverter(**options).convert(html)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/markdownify/__init__.py", line 64, in convert
    soup = BeautifulSoup(html, 'html.parser')
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/bs4/__init__.py", line 315, in __init__
    elif len(markup) <= 256 and (
         ^^^^^^^^^^^
TypeError: object of type 'NoneType' has no len()

and
Logger: homeassistant
Source: util/async_.py:151
First occurred: 17:00:14 (20 occurrences)
Last logged: 17:09:44
Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 233, in _handle_refresh_interval
    await self._async_refresh(log_failures=True, scheduled=True)
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 389, in _async_refresh
    self.async_update_listeners()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 172, in async_update_listeners
    update_callback()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 779, in async_write_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 879, in _async_write_ha_state
    state, attr = self._async_generate_attributes()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 820, in _async_generate_attributes
    state = self._stringify_state(available)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 785, in _stringify_state
    if (state := self.state) is None:
                 ^^^^^^^^^^
  File "/config/custom_components/dwd_weather/sensor.py", line 297, in state
    self._connector.get_weather_report(),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/dwd_weather/connector.py", line 213, in get_weather_report
    return markdownify(self.dwd_weather.get_weather_report(), strip=["br"])
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 764, in get_weather_report
    self.update(with_report=True)
  File "/usr/local/lib/python3.11/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 549, in update
    self.download_latest_kml(self.station_id, force_hourly)
  File "/usr/local/lib/python3.11/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 790, in download_latest_kml
    request = requests.get(url, headers=headers)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 714, in urlopen
    httplib_response = self._make_request(
                       ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 415, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 244, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/local/lib/python3.11/http/client.py", line 1286, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/lib/python3.11/http/client.py", line 1297, in _send_request
    self.putrequest(method, url, **skips)
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 219, in putrequest
    return _HTTPConnection.putrequest(self, method, url, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/util/async_.py", line 164, in protected_loop_func
    check_loop(func, strict=strict)
  File "/usr/src/homeassistant/homeassistant/util/async_.py", line 151, in check_loop
    raise RuntimeError(
RuntimeError: Blocking calls must be done in the executor or a separate thread; Use `await hass.async_add_executor_job()`; at custom_components/dwd_weather/connector.py, line 213: return markdownify(self.dwd_weather.get_weather_report(), strip=["br"]

I have updated to 2.0.5 and I have switched again the weather station, this time to 10381: Berlin-Dahlem. I did stick to the recommended "data fetching" approach. The result is, everything seems to work fine for me now.
Thanks very much for your help and thanks very much developing this module in the first place!
As far as I am concerned the case can be closed. Should we keep it open because there is still a parallel discussion going on?
Thanks,
Alex
Just in case someone is curios here is one of the sensors I did to control a few things subject to expected performance of my PV system. If you spot anything that isn't used as intended or that doesn't make sense. Again, as per my experience it works fine like this:
    berlin_irradiance_forecast_rest_of_day:
      friendly_name: "Berlin irradiance forecast rest of day"
      unit_of_measurement: 'W/m^2'
      value_template: >-
        {% set irrTotal = namespace(value=0) %}
        {% set current_hour = now().hour %}
        {% set hours_left = 24 - current_hour %}
        {% set irradiance_data = states.sensor.berlin_dahlem_sonneneinstrahlung.attributes.data | default([]) %}
        {% if irradiance_data is not none %}
          {% for irr in irradiance_data %}
            {% if loop.index <= hours_left %}
              {% set irrTotal.value = irrTotal.value + irr.value %}
            {% else %}
              {% break %}
            {% endif %}
          {% endfor %}
        {{ irrTotal.value }}
        {% else %}
          unavailable
        {% endif %}

@almiho I'm glad to hear this!
Yes, leave this open for a moment longer until all issues mentioned here are resolved.
@Failure404 This error arises because you have enabled the weather report (the text one) without an available report. This is now fixed in the new version 2.0.7


Improvements to easyrule cli script

add method to "unblock" a former block rule
derive name of block list alias from script, allowing management of multiple individial lists by using symlinks

In order to get patches applied you must sign Individual or Corporate CLA as described at https://www.pfsense.org/about-pfsense/#legal

CLA signed

@rbgarga I signed the CLA 4 weeks ago and remerged the current master to solve the conflicst. Might you consider to merge this or are there any objections?

@phil-davis @rbgarga
Guys, is there any reason why this isnt processed? Can I assist in anything? I would really appreciated to see this in the official release

done


Proxy Not Renewing Token

I am having an issue where I can get a token and the application functions properly. But after the 1 hour expiry elapses the token is not renewed by the resource proxy.
We have been able to determine the issue (with the help of ESRI support) in the code at line 213 in proxy.jsp, where the code is checking for specific error codes.
This string  used to check the error code is not being decoded and appears in a  "wing-ding"  format and the error codes are not found. If the error code is not found the renew token function is not triggered.
I have not been able to reproduce this error on other systems that use the resource proxy and it is only occurring with this application.
Here is a (modified) copy of the proxy.config we are using:
<?xml version="1.0" encoding="utf-8" ?>
<ProxyConfig allowedReferers="*"
                logFile="e:\WEB-INF\classes\proxy_log.log"
                logLevel="FINEST"
                mustMatch="true">
  <serverUrls>
        <serverUrl url="https://<URL>/arcgis/rest/services"
        matchAll="true"
             tokenServiceUri="https://<URL>/portal/sharing/rest/generateToken/"
               username="username"
               password="password"
        />

  </serverUrls>
</ProxyConfig>

I've been working with @juicetiger1 on this issue.
Here are my testing notes if it helps address the issue:
Issue: Resource Proxy (Java) fails to Renew Token after it Expires
Notes:

Client is running resource proxy 1.1.2
https://github.com/Esri/resource-proxy/releases/tag/v1.1.2
The proxy is configured to allow access to allow access to a secured Geocode service via tokenServiceUri, Username, Password combination

<serverUrl url="https://citymaps.peterborough.ca/arcgis/rest/services/PtboLocatorLocateApp/GeocodeServer"
matchAll="true"
tokenServiceUri="https://citymaps.peterborough.ca/portal/sharing/rest/generateToken"
username="XXXXX"
password="XXXXX"
/>

ArcGIS Server is federated with Portal. Both Server and Portal are at 10.6.1

The resource proxy is hosted on Tomcat web server.

The proxy is able to successfully generate a token and use it without a problem.

After an hour passes, the token expires, and the resource proxy fails to recognize that the token is expired, and doesn't attempt to generate a new token.

The proxy log shows no errors

We inspected the Java code responsible for renewing the token

After adding a line to print log of the response from Server that includes any error messages, we found that the proxy prints some incoherent text instead of a proper string response.

Following the logic of the code in line 212, this string won't match any of the conditions of the If statement and won't be able to recognize the error returned from the server/portal
https://github.com/Esri/resource-proxy/blob/master/Java/proxy.jsp#L212

The issue seems limited to this one installation of the resource proxy

Client has other applications setup with their own resource proxy (also, Java on Tomcat) and they all work fine with similar configuration.

I've attempted to reproduce the issue, but couldn't. The proxy works fine.

We're looking for help identifying what's causing the resource proxy to behave this way.

@juicetiger1, @AmrEldib,
I've run into what may be the same issue. If so, I've managed to fix/workaround it. I was getting the same "wing-ding" response body when one particular service sent along an authentication error that looked like this when written to the log:

I actually found two issues when handling the authentication error, but the root cause is that the authentication error is gzip encoded. Now, I didn't dig deep enough to determine if this encoding issue is with the proxy code, something else in the call stack (like a tomcat, or connection issue), or if the problem was with the server itself sending an ill-formed response (it would be nice to know why other responses aren't gzipped, or if they are, why they get properly decoded and this one doesn't). But either way I was able to put in a work-around to gzip decode the response if other means didn't catch the authentication error, like so:
    //proxy gets the response back from server
    private boolean fetchAndPassBackToClient(HttpURLConnection con, HttpServletResponse clientResponse, boolean ignoreAuthenticationErrors) throws IOException{
        if (con!=null){
            //copy the response content to the response to the client
            InputStream byteStream;
            if (con.getResponseCode() >= 400 && con.getErrorStream() != null){
                if (ignoreAuthenticationErrors && (con.getResponseCode() == 498 || con.getResponseCode() == 499)){
                    log.info("Authentication error found in response code.");
                    return true;
                }
                byteStream = con.getErrorStream();
            }else{
                byteStream = con.getInputStream();
            }

            ByteArrayOutputStream buffer = new ByteArrayOutputStream();
            int length = 5000;
            byte[] bytes = new byte[length];
            int bytesRead;

            while ((bytesRead = byteStream.read(bytes, 0, length)) > 0) {
                buffer.write(bytes, 0, bytesRead);
            }

            //if the content of the HttpURLConnection contains error message, it means the token expired, so let proxy try again
            //only check small responses (< 1KB). It's probably expensive to convert and parse large responses every time.
            log.info("RESPONSE BUFFER SIZE: " + buffer.size());
            if(buffer.size() <= 1024 && !ignoreAuthenticationErrors){
                String strResponse = buffer.toString();
                log.info("RESPONSE: " + strResponse);
                if (hasResponseBodyAuthenticationError(strResponse)) {
                    log.info("Authentication error found in response body.");
                    return true;
                }

                //Do a gzip decode and check those results as well
                if("gzip".equals(con.getHeaderField("Content-Encoding"))) {
                    GZIPInputStream gzin = new GZIPInputStream(new ByteArrayInputStream(buffer.toByteArray()));
                    ByteArrayOutputStream outputBytes = new ByteArrayOutputStream();
                    int read = 0;
                    byte gzBytes[] = new byte[1024];

                    while ((read = gzin.read(gzBytes, 0, gzBytes.length)) > 0) {
                        outputBytes.write(gzBytes, 0, read);
                    }

                    String strGzipResponse = outputBytes.toString();
                    log.info("CONTAINS GZIP HEADER. UN-GZIPPED RESPONSE: " + strGzipResponse);
                    if (hasResponseBodyAuthenticationError(strGzipResponse)) {
                        log.info("Authentication error found in gzipped response body.");
                        return true;
                    }
                }
            }

            copyHeadersToResponse(con, clientResponse);

            clientResponse.setStatus(con.getResponseCode());

            byte[] byteResponse = buffer.toByteArray();
            OutputStream ostream = clientResponse.getOutputStream();
            ostream.write(byteResponse);
        }
        return false;
    }

Since I'm unable to determine exactly what conditions cause the decoding issue, I just check all responses for errors that aren't caught via the first check by gzip decoding them, and optimize to only check small-ish responses to save some cycles.
You also may notice that I moved the code that sets the response headers to occur after the authentication error check. I found that when it's before the check, it adds duplicate headers that cause issues at the client since that section of code would be called twice if an authentication error occurred.
Finally, you may notice I removed all the flush() and close() calls, as those are unnecessary in Java 8, so be sure to leave those in if using something lower.
Anywho, I hope this helps you (or anybody) out. Apologies, I can't submit a pull request for a fix because I've refactored this project into a Spring Boot project to make it more easily deployable in a Kubernetes cluster.

On another, slightly related note, I've also found that some error responses have been missed because they contain a space before and after the colon and should be checked, like so:
private boolean hasResponseBodyAuthenticationError(String responseBody){
        return responseBody.contains("error")
               && (responseBody.contains("\"code\": 498")
                   || responseBody.contains("\"code\": 499")
                   || responseBody.contains("\"code\":498")
                   || responseBody.contains("\"code\":499")
                   || responseBody.contains("\"code\" : 498") //add this check
                   || responseBody.contains("\"code\" : 499")); // and this check
    }

Are you able to send the copyHeadersToResponse function as well?
Thanks

@juicetiger1

Are you able to send the copyHeadersToResponse function as well?
Thanks

I've made no changes here, other than re-arranging the code. I simply extracted all of the header-setting code at the top of the function into its own function and moved it below the authentication checking code.

copyHeadersToResponse

Hi @cpore,  can you please help to share the function copyHeadersToResponse.

Are you able to send the copyHeadersToResponse function as well?
Thanks

`	private void copyHeadersToResponse(HttpURLConnection con, HttpServletResponse clientResponse){
Map<String, List> headerFields = con.getHeaderFields();
Set headerFieldsSet = headerFields.keySet();
	//copy the response header to the response to the client
	for (String headerFieldKey : headerFieldsSet){
		//prevent request for partial content
		if (headerFieldKey != null && headerFieldKey.toLowerCase().equals("accept-ranges")){
			continue;
		}

		List<String> headerFieldValue = headerFields.get(headerFieldKey);
		StringBuilder sb = new StringBuilder();
		for (String value : headerFieldValue) {
			// Reset the content-type for OGC WMS - issue #367
			// Note: this might not be what everyone expects, but it helps some users
			// TODO: make this configurable
			if (headerFieldKey != null && headerFieldKey.toLowerCase().equals("content-type")){
				if (value != null && value.toLowerCase().contains("application/vnd.ogc.wms_xml")){
					_log(Level.FINE, "Adjusting Content-Type for WMS OGC: " + value);
					value = "text/xml";
				}
			}

			// remove Transfer-Encoding/chunked to the client
			// StackOverflow http://stackoverflow.com/questions/31452074/how-to-proxy-http-requests-in-spring-mvc
			if (headerFieldKey != null && headerFieldKey.toLowerCase().equals("transfer-encoding")) {
				if (value != null && value.toLowerCase().equals("chunked")) {
					continue;
				}
			}

			sb.append(value);
			sb.append("");
		}
		if (headerFieldKey != null){
			clientResponse.addHeader(headerFieldKey, DataValidUtil.removeCRLF(sb.toString()));
		}
	}
}`

Are you able to send the copyHeadersToResponse function as well?
Thanks

private void copyHeadersToResponse(HttpURLConnection con, HttpServletResponse clientResponse){
	Map<String, List<String>> headerFields = con.getHeaderFields();
	Set<String> headerFieldsSet = headerFields.keySet();

	//copy the response header to the response to the client
	for (String headerFieldKey : headerFieldsSet){
		//prevent request for partial content
		if (headerFieldKey != null && headerFieldKey.toLowerCase().equals("accept-ranges")){
			continue;
		}

		List<String> headerFieldValue = headerFields.get(headerFieldKey);
		StringBuilder sb = new StringBuilder();
		for (String value : headerFieldValue) {
			// Reset the content-type for OGC WMS - issue #367
			// Note: this might not be what everyone expects, but it helps some users
			// TODO: make this configurable
			if (headerFieldKey != null && headerFieldKey.toLowerCase().equals("content-type")){
				if (value != null && value.toLowerCase().contains("application/vnd.ogc.wms_xml")){
					_log(Level.FINE, "Adjusting Content-Type for WMS OGC: " + value);
					value = "text/xml";
				}
			}

			// remove Transfer-Encoding/chunked to the client
			// StackOverflow http://stackoverflow.com/questions/31452074/how-to-proxy-http-requests-in-spring-mvc
			if (headerFieldKey != null && headerFieldKey.toLowerCase().equals("transfer-encoding")) {
				if (value != null && value.toLowerCase().equals("chunked")) {
					continue;
				}
			}

			sb.append(value);
			sb.append("");
		}
		if (headerFieldKey != null){
			clientResponse.addHeader(headerFieldKey, DataValidUtil.removeCRLF(sb.toString()));
		}
	}
}


support for UE4 console commands

I'd like to create a QT application that executes some UE4 console commands by clicking some buttons.
Is there a way of doing that currently?

Currently it is not available, but it is an easy addition:
'''
UWorld->Exec(context, string)
'''

Oh thanks for pointing me in the right direction.
I,ll create a pull request if you don't mind

Yes, no problem for me, thanks

I think that I'm not doing it correctly. I'm probably not getting the uworld I need because neither Play() nor execute_command() do anything. The log part works just fine and is only for debugging
PyObject *py_ue_execute_console(ue_PyUObject *self, PyObject *args)
{
	ue_py_check(self);

	UWorld *world = ue_get_uworld(self);
	if (!world)
		return PyErr_Format(PyExc_Exception, "unable to retrieve UWorld from uobject");

	PyObject *py_message;
	if (!PyArg_ParseTuple(args, "O:log", &py_message)) {
		return NULL;
	}

	PyObject *stringified = PyObject_Str(py_message);
	if (!stringified)
		return PyErr_Format(PyExc_Exception, "argument cannot be casted to string");

	char *message = PyUnicode_AsUTF8(stringified);
	FString Command = FString::Printf(TEXT("%s"), UTF8_TO_TCHAR(message));
	world->Exec(world, *Command);
	UE_LOG(LogPython, Log, TEXT("%s"), *Command);
	Py_DECREF(stringified);

	Py_INCREF(Py_None);
	return Py_None;
}

import unreal_engine as ue

world = ue.get_editor_world()
world.play()
world.execute_command('showdebug')

When you play() the running world will be the PIE. I strongly suggest you to not use the editor_world as the context but an actor in the level.

any tips on how to get a proper world context?

Hi, any actor can access its world with get_world()

Hi, would you like to make a pull request or you want me to manually include the patch ?

I really like to but I never got it working.
None of my attempts were successful.

Ok, i'll work on it, do not worry

the console variable wasn't really the problem. I failed to retrieve the UWorld for the editor

the console_exec(string) command has been added:
import unreal_engine as ue

ue.get_editor_world().console_exec('your command')


NPE in dart::FinalizablePersistentHandle::Finalize

Dart SDK Version 2.0.0-dev.65.0
Mac OSX Sierra

I've been working on an async native extension, but have been running into a specific error for a few days. I've been trying hard to debug it, and eventually was able to attach lldb to Dart, so that I could find where my crash was happening.
In short, I have a native extension that:

Binds to a socket
Creates a new native port that listens for commands from the Dart world
Starts a separate worker thread that:

Listens for connections on said socket
Parses HTTP requests using Joyent's http-parser from Node.js
http-parser sends callbacks through a SendPort; my Dart code captures these and then sends them to more Dart code.
Everything else is handled in Dart, except for that once it's time to send the response, a Uint8List is sent back along the SendPort and the native write is called.

This all works perfectly. I have about a bajillion print statements, one at each stage of the request lifecycle, and each is called in the expected order on successful requests, so it seems like this code is good-to-go.
Or so it seems...
In reality, this sequence executes perfectly for a variable number requests (ranges from 1 to about 5), and then out of the blue explodes with this error:
===== CRASH =====
version=2.0.0-dev.65.0 (Tue Jun 26 14:17:21 2018 +0200) on "macos_x64"
si_signo=Segmentation fault: 11(11), si_code=1, si_addr=0x0
Dumping native stack trace for thread f03
Stack dump aborted because InitialRegisterCheck failed.
Abort trap: 6

Here's what the backtrace is in LLDB:
* thread #10, stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
    frame #0: 0x0000000000000000
error: memory read failed for 0x0
Target 0: (dart) stopped.

(lldb) bt
* thread #10, stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
  * frame #0: 0x0000000000000000
    frame #1: 0x00000001005b52f3 dart`dart::FinalizablePersistentHandle::Finalize(dart::Isolate*, dart::FinalizablePersistentHandle*) + 35
    frame #2: 0x0000000100261c5c dart`dart::Isolate::VisitWeakPersistentHandles(dart::HandleVisitor*) + 156
    frame #3: 0x00000001005a219d dart`dart::Scavenger::Scavenge() + 621
    frame #4: 0x0000000100597b45 dart`dart::Heap::CollectNewSpaceGarbage(dart::Thread*, dart::Heap::GCReason) + 613
    frame #5: 0x000000010059698d dart`dart::Heap::AllocateNew(long) + 109
    frame #6: 0x0000000100290a44 dart`dart::Object::Allocate(long, long, dart::Heap::Space) + 100
    frame #7: 0x00000001002ef994 dart`dart::Instance::New(dart::Class const&, dart::Heap::Space) + 100
    frame #8: 0x00000001003d211d dart`dart::DRT_AllocateObject(dart::NativeArguments) + 413
    frame #9: 0x000000010304085b
    frame #10: 0x000000010622f4a9
    frame #11: 0x00000001159edf14
    frame #12: 0x00000001159d9bfa
    frame #13: 0x00000001159d1da3
    frame #14: 0x0000000106229864
    frame #15: 0x000000010620af15
    frame #16: 0x000000010622d5dc
    frame #17: 0x0000000103040ce3
    frame #18: 0x000000010022e048 dart`dart::DartEntry::InvokeFunction(dart::Function const&, dart::Array const&, dart::Array const&, unsigned long) + 744
    frame #19: 0x00000001002315cd dart`dart::DartLibraryCalls::HandleMessage(dart::Object const&, dart::Instance const&) + 861
    frame #20: 0x000000010025bc70 dart`dart::IsolateMessageHandler::HandleMessage(dart::Message*) + 1296
    frame #21: 0x000000010028a11a dart`dart::MessageHandler::HandleMessages(dart::MonitorLocker*, bool, bool) + 234
    frame #22: 0x000000010028a8cc dart`dart::MessageHandler::TaskCallback() + 860
    frame #23: 0x000000010042023d dart`dart::ThreadPool::Worker::Loop() + 61
    frame #24: 0x00000001004200c8 dart`dart::ThreadPool::Worker::Main(unsigned long) + 136
    frame #25: 0x0000000100329ca7 dart`___lldb_unnamed_symbol3098$$dart + 71
    frame #26: 0x00007fffc82c693b libsystem_pthread.dylib`_pthread_body + 180
    frame #27: 0x00007fffc82c6887 libsystem_pthread.dylib`_pthread_start + 286
    frame #28: 0x00007fffc82c608d libsystem_pthread.dylib`thread_start + 13

At first, I was convinced this was something in the code I had written, so I continued to add print statements throughout, trying to track down the source.
What actually happened was that the error occurred at different stages in the request's lifecycle each time, leading me to believe that the source is not within the code I had written.
So, I went looking for dart::FinalizablePersistentHandle::Finalize, and found its declaration:
https://github.com/dart-lang/sdk/blob/master/runtime/vm/dart_api_impl.cc#L665-L679
void FinalizablePersistentHandle::Finalize(
    Isolate* isolate,
    FinalizablePersistentHandle* handle) {
  if (!handle->raw()->IsHeapObject()) {
    return;  // Free handle.
  }
  Dart_WeakPersistentHandleFinalizer callback = handle->callback();
  ASSERT(callback != NULL);
  void* peer = handle->peer();
  Dart_WeakPersistentHandle object = handle->apiHandle();
  (*callback)(isolate->init_callback_data(), object, peer);
  ApiState* state = isolate->api_state();
  ASSERT(state != NULL);
  state->weak_persistent_handles().FreeHandle(handle);
}

Conclusion
Based on the fact that the trace says the error originated with a call to a null pointer, and that the above code contains assertion that callback != NULL, followed by a call to said callback, I think the source of the error is that callback at some point becomes set to NULL.
If I'm reading this right, then the ASSERT macro effectively does nothing, since it's an empty do with a while condition that always evaluates to false, so it makes that a NULL callback would escape and be able to called by code, and thereby crash the VM.
https://github.com/dart-lang/sdk/blob/master/runtime/platform/assert.h#L266-L268
Thanks in advance for your help. Didn't expect to find this in this specific way, but it was pretty fun tracking it down.

Interesting, it looks to me like all the paths that allocate weak handles check for a NULL callback. Does your native extension use Dart_NewExternalTypedData, Dart_NewExteralLatin1String, Dart_NewExternalUTF16String, or Dart_NewWeakPersistentHandle? Do you have a publicly accessible reproduction?
(ASSERT only has an effect in a debug build of the VM. We run our test suite on such a VM.)

My code is here: https://github.com/angel-dart/wings
And a quick grep:
Tobes-MacBook-Air:net_driver thosakwe$ grep -n -i typeddata -r .
Binary file ./.dart_tool/build/generated/angel_wings/lib/src/libwings.dylib matches
Binary file ./.dart_tool/pub/bin/build_native/build_native.dart.snapshot.dart2 matches
Binary file ./.dart_tool/pub/bin/build_runner/build_runner.dart.snapshot.dart2 matches
Binary file ./.dart_tool/pub/bin/build_runner/graph_inspector.dart.snapshot.dart2 matches
./lib/src/bind_socket.cc:17:    Dart_TypedData_Type addressType;
./lib/src/bind_socket.cc:25:    HandleError(Dart_TypedDataAcquireData(addressHandle, &addressType, &addressData, &addressLength));
./lib/src/bind_socket.cc:26:    HandleError(Dart_TypedDataReleaseData(addressHandle));
Binary file ./lib/src/bind_socket.o matches
./lib/src/http_listener.cc:70:                if (msg->type == Dart_CObject_kExternalTypedData)
./lib/src/http_listener.cc:77:                else if (msg->type == Dart_CObject_kTypedData)
Binary file ./lib/src/http_listener.o matches
Binary file ./lib/src/libwings.dylib matches
./lib/src/send.cc:19:    Dart_TypedData_Type dataType;
./lib/src/send.cc:27:    HandleError(Dart_TypedDataAcquireData(dataHandle, &dataType, &dataBytes, &dataLength));
./lib/src/send.cc:29:    HandleError(Dart_TypedDataReleaseData(dataHandle));
Binary file ./lib/src/send.o matches
./lib/src/util.cc:9:    Dart_TypedData_Type type;
./lib/src/util.cc:24:    HandleError(Dart_TypedDataAcquireData(address_handle, &type, &data, &length));
./lib/src/util.cc:26:    HandleError(Dart_TypedDataReleaseData(address_handle));
Binary file ./lib/src/util.o matches
./lib/src/worker_thread.cc:89:        third.type = Dart_CObject_kExternalTypedData;
./lib/src/worker_thread.cc:90:        third.type = Dart_CObject_kExternalTypedData;
./lib/src/worker_thread.cc:91:        third.value.as_external_typed_data.type = Dart_TypedData_kUint8;
./lib/src/worker_thread.cc:121:    addr.type = Dart_CObject_kExternalTypedData;
./lib/src/worker_thread.cc:127:    addr.value.as_external_typed_data.type = Dart_TypedData_kUint8;
Binary file ./lib/src/worker_thread.o matches

I'm not directly calling any of the functions you mentioned, BUT I am using TypedDataAcquireData and TypedDataReleaseData.

I figured out what my error was; it's not a VM error.
While I wasn't calling Dart_NewExternalTypedData, some C objects I sent back were marked as ExternalTypedData, rather than just TypedData.
I didn't realize the distinction that ExternalTypedData must provide its own finalizer.


Prevent error if using paginator :none

The PR fixes a bug if you use paginator :none and have config.top_level_meta_include_page_count set to true.

+1 this is a current issue for my team and we have to use a custom paginator to get around it

@fsmanuel Thanks!


Multiple accounts (again)

I'm sorry but i can't figure it out. Lets say i have user with password (it's been created before any social attempts). I can get its token using curl -X POST -d "client_id=<id>&client_secret=<secret>&grant_type=password&username=user1&password=<pass>" http://localhost:8100/auth/token. Then i can use this token to get data from server.
Now, suppose i want to attach Google OAuth2 to this user. I was trying like this:
curl -H "Authorization: Bearer <user1-token>" -X POST -d "grant_type=convert_token&client_id=<id>&client_secret=<secret>&backend=google-oauth2&token=<google_token>" http://localhost:8100/auth/convert-token
And it doesn't work - i get new user not associated with user1. I was trying to extend auth pipeline but i can't get user1 there. How should i do it?

This is not related to my package but to python-social-auth.
You should check this link regarding your issue.

OK, thanks.

I was able to get around this by adding a step to my pipeline that picked up the strategy.request.user to not create new accounts like so:
def apiUserPipeline(strategy, details, backend, user=None, *args, **kwargs):
    if  user is None and strategy.request.user:
        return {'user': strategy.request.user}

It is odd that convert_token does not seem to "see" the authenticated users the same way that using vanilla social_app_django does when you hit their begin URL.  But using something like the the above early in your pipeline (I have it as the 5th step before the get_username part of the default pipeline.


files-cli: 2.13.50 -> 2.13.53

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://github.com/files-com/files-cli/releases.
meta.description for files-cli is: Files.com Command Line App for Windows, Linux, and macOS.
meta.homepage for files-cli is: https://developers.files.com
Updates performed

Golang update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 2.13.53 with grep in /nix/store/qbvgfshidqn5y5im5l3wpvznsjcssay0-files-cli-2.13.53
found 2.13.53 in filename of file in /nix/store/qbvgfshidqn5y5im5l3wpvznsjcssay0-files-cli-2.13.53

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

2 package rebuild(s)

First fifty rebuilds by attrpath

files-cli

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/qbvgfshidqn5y5im5l3wpvznsjcssay0-files-cli-2.13.53 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A files-cli https://github.com/r-ryantm/nixpkgs/archive/7b95e19a1b96baacc5ddd7e2aae95052a5417a82.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/7b95e19a1b96baacc5ddd7e2aae95052a5417a82#files-cli

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/qbvgfshidqn5y5im5l3wpvznsjcssay0-files-cli-2.13.53
ls -la /nix/store/qbvgfshidqn5y5im5l3wpvznsjcssay0-files-cli-2.13.53/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review --extra-nixpkgs-config '{ allowInsecurePredicate = x: true; }' run on x86_64-linux 1

1 package built:

files-cli

Maintainer pings
cc @kashw2 for testing.

@NixOS/nixpkgs-merge-bot merge


Version 1.5.11 changelog

CHANGELOG.md mentions version 1.5.11 but based on the history of package.json, this version was never released

Yeah the changelog is manually written of the summary of changes in the app (with respect to the API) and the releases here are auto generated typings from the main app code which don't happen at the same pace. If the changelog mentions a version then it likely means this was added in the app version. The API typings sometimes don't get updated until we send everything to public.

@lishid thanks for the explanation. Those questions wouldn't be asked if #168 is done

The changelog.md file has been updated to clarify this distinction.


Adding Authentication & Authorization Middleware

this is an awesome boilerplate
but I think it should also include API Authentication & Authorization Backend
can be integrated using the passport
supported Authentications can be:

JWT

for Authorization, there could be a base Gaurd to create roles and permission for Authorization

for Authorization, there could be a base Gaurd to create roles and permission for Authorization
agreed 100 % - a middleware to tackle to dynamic permissions assigned to the auth user.  i had work on one such feature using laravel. but i have no idea how to do it in ts on nestjs

I like the idea, but this discussion is a bit old,
Is someone planning to work on this feature?
cc: @vinayak25


Report layout orientation on preview

Hi,
we are facing issue with report layout orientation on preview.
On version 16.x. reports with RDLC layout and landscape orientation renders now as portrait orientation on preview. Same reports works fine on 15.x.
Has there been any change from version 15.x to 16.x how does the preview work?
testing on version:
W1 16.0 (Platform 15.0.39761.0 + Application 16.0.10855.0)
BR,
Bojan

This forum is for extensibility issues. You can report bug using following resouces:
•	On premises - https://support.microsoft.com/en-us/supportforbusiness/productselection?sapId=93d37907-ad94-d591-22e9-593cfa09dd3f
•	On line – contact your partner, or if you are a partner open a support incident through Partner Center - https://partner.microsoft.com/en-us/pcv/dashboard/overview


Fatal error: UnCSS & Typekit

After updating to the latest version 0.4.2 it seems there is a new behaviour which results in a fatal error, with UnCSS somehow trying to parse a Typekit URL. I haven't seen this issue with the previous version.
~/Sites/repos/missed-in-hkg (master) grunt uncss
Running “uncss:dist” (uncss) task
Fatal error: uncss/node_modules/css: unable to parse http://use.typekit.net/c/1a04cf/1w;proxima-nova,2,b5t:P:n3,b5w:P:n4/k?3bb2a6e53c9684ffdc9a9bf41d5b2a62183a574751fd848f787439ef36ae423e0294bf618b0faa6759749a087087087 :
missing ‘{‘ near line 13:1
	8:
	9:
	10:
403 - Forbidden

	11:
	12:
	13:     ->

Should this be an expected behaviour?
A current workaround to fix this for now is to include Typekit in the ignoreSheets option:
options: {
  ignoreSheets : [/use.typekit/]
}

This seems like an upstream issue. Please ask there.


Bump version to 2.6

To release a new version we need to bump it first.

Coverage remained the same at 91.546% when pulling ef2d93111193c6f8493752319372c89755a93d82 on version-26 into 54694fd8fd7a1df66c1bbd77707d01162b0aa275 on master.

👍


Move to official org

Hey @alexherbo2,
This is Zeno, creator of Dracula. I just this found repo, good job on that!
I'm wondering if you would like to have this become an official theme in our Dracula organization?
Let me know what you think.

Hi @zenorocha,
I think @Delapouite would be more suited, as he is the original creator of the color scheme for Kakoune.
His implementation lives in kakoune-colors.

Got it! Thanks for the info @alexherbo2 ;)

@zenorocha Can you check my work?
I adapted to dracula/template, and took the time to ensure everything was ok/clean.

That's perfect @alexherbo2! You can see it live now: https://draculatheme.com/kakoune 🎉 🎉 🎉
Thanks!!

@zenorocha Can I suggest a cat with some green for the logo?
 ___
(__ \
  / /
 .' '·.
'      ”
╰       /\_/|
 | .         \
 ╰_J`    | | |
     ' \__- _/
     \_\   \_\

Done :) https://draculatheme.com/kakoune


fix no longer valid gist-based carbon url example in about page.

[x] Prettier lint-ed
[x] Integration tests (if applicable)

The main changes is in this commit: https://github.com/carbon-app/carbon/commit/64cef8b37c41c60489196387deeee2032c46a231 .

It fixes: I found on about page that the gist-based url example that seems to be no longer valid url (the original gist get deleted maybe?), which result in the url redirecting to home. So I replace it with the example gist url found in the readme.md
Also added example of the original gist url VS the result carbon url, to make it easy to understand which ID from the url to copy, and not to include username from the url.

Also in spirit of Hacktoberfest, if you think this PR is valid and accepted, please kindly add this label to this PR: hacktoberfest-accepted. Thank you 🙂
Btw Carbon rocks! Useful tool indeed!
Preview of Changes

Cool 👍 Also if it's not too much to ask, would you please add hacktoberfest-accepted label on this PR @mfix22 ? thanks!

in spirit of Hacktoberfest, if you think this PR is valid and accepted, please kindly add this label to this PR: hacktoberfest-accepted. Thank you 🙂

Of course!


Add description for LPAR inventory

For me it's ready to merge

After a new rebase from the master, it's ready to merge for me

Backporting to najdorf requires the Gemfile.lock for ibm_power_hmc to be updated from 0.8.3 -> 0.8.4
https://github.com/ManageIQ/manageiq/blob/najdorf/Gemfile.lock.release#L705
Should I submit a PR or is this comment enough?

@jaywcarman if there was a PR that bumped the gem version, then that should probably be marked with najdorf/yes? as well

@chessbyte this PR does bump the minium ibm_power_hmc gem version in our plugin gemspec:
https://github.com/ManageIQ/manageiq-providers-ibm_power_hmc/blame/master/manageiq-providers-ibm_power_hmc.gemspec#L22
If this gets backported, does that somehow automatically propagate up to the core manageiq/Gemfile.lock.release in najdorf, or do I need to propose the change through a PR?

The Gemfile.lock.release is not automatically updated, tor example here is a PR to bump a gem in a release branch https://github.com/ManageIQ/manageiq/pull/21720
Not sure if @Fryguy wants to do the bundle install himself or if he wants you to put a PR in

@CallisteH A conflict occurred during the backport of this pull request to najdorf.
If this pull request is based on another pull request that has not been marked for backport, add the appropriate labels to the other pull request. Otherwise, please create a new pull request direct to the najdorf branch in order to resolve this.
Conflict details:
diff --cc manageiq-providers-ibm_power_hmc.gemspec
index dd7b427,77bed7f..0000000
--- a/manageiq-providers-ibm_power_hmc.gemspec
+++ b/manageiq-providers-ibm_power_hmc.gemspec
@@@ -19,7 -19,7 +19,11 @@@ Gem::Specification.new do |spec
    spec.executables   = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
    spec.require_paths = ["lib"]

++<<<<<<< HEAD
 +  spec.add_dependency "ibm_power_hmc", "~> 0.8.1"
++=======
+   spec.add_dependency "ibm_power_hmc", "~> 0.8.4"
++>>>>>>> 76bb863 (Merge pull request #39 from CallisteH/fix/lpar_description)

    spec.add_development_dependency "manageiq-style"
    spec.add_development_dependency "simplecov", ">= 0.21.2"

I'll manually backport this

najdorf backport details:
commit b7d0286a8fe44f2b228087715b9e716248b85e13 (HEAD -> najdorf, upstream/najdorf)
Author: Adam Grare<EMAIL_ADDRESS>Date:   Wed Feb 16 11:40:16 2022 -0500

    Merge pull request #39 from CallisteH/fix/lpar_description

    Add description for LPAR inventory

    (cherry picked from commit 76bb863c6aafaf0ca32dd690ceb22259598eebbf)


Corrected import org.gradle.util.OperatingSystem to org.gradle.os.Operati

Corrected import org.gradle.util.OperatingSystem to org.gradle.os.OperatingSystem
org.gradle.util.OperatingSystem is no longer found in Gradle 1.0-milestone-5,
so we change it to org.gradle.os.OperatingSystem
Signed-off-by: Ealden Esto E. Escanan<EMAIL_ADDRESS>

This also seems to build correctly in 1.0-milestone-6


TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.
If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.
If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/51210

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/71125

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/101137

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/101320


HACS Integration Improvements

Good work so far on the component, I understand it's still a work in progress, but I have noticed a few small items when it's installed via HACS (which are not really functional issues) but would be small improvements, if you were not already aware of these.

From the "information" screen (ie, which you open the component in HACS) and you select "For more information visit the repository.", it redirects you here instead of to the repository
When updating in HACS, there's no release notes to show what has changed since the previous version
Version are git commit reverences instead of actual version numbers
Ultimately, it would be nice if the integration were shown in the HA Integrations menu

Hi,
Glad you are liking the integration!
Thanks for you multiple requests / issues. Il pick this issue likely tomorrow and have a look at your requests the coming weekend.

Can you elaborate on what you mean by:
"Ultimately, it would be nice if the integration were shown in the HA Integrations menu"
Do you mean this page?

No, I mean in HA. If you go to Configuration -> Integrations, most of the integrations are shown here (not all, most of the integrations are still being updated to be included here), with all of the related entities etc. It doesn't make things work any better, just that things are then more polished off ;)
An example of a HACS integration which does this is that of UniFi Protect

Hmm, weird. When I manually installed the integration it did show up. But when installing through HACS it does not show up as you say. I did not find a quick fix for this, so I'll have a look on this the comming weekend.


WAF on RE + AppConnect automation (VM,K8s) and workflow guide changes

Hi Team,
In this PR, I have submitted the changes that I made wrt to WAF on RE + AppConnect automation cases both for VM and K8s scenarios and also, I have made changes to workflow guides as per requirement.
Please review the changes and let me know if any further modifications has to be made.
Thank you,
Salini K

Looks good for me so merging it.