Mercedes not showing SoC

Describe the bug
Login works incl. redirect but SoC of car cannot be retrieved and gives a token expired and refresh token is not set
Steps to reproduce

Use the current config
Start evcc
Login car b-klasse

Configuration details
network:
  # schema is the HTTP schema
  # setting to `https` does not enable https, it only changes the way URLs are generated
  schema: http
  # host is the hostname or IP address
  # if the host name contains a `.local` suffix, the name will be announced on MDNS
  # docker: MDNS announcements don't work. host must be set to the docker host's name.
  host: <IP_ADDRESS>
  # port is the listening port for UI and api
  # evcc will listen on all available interfaces
  port: 7070

interval: 30s # control cycle interval

# sponsor token enables optional features (request at https://cloud.evcc.io)
sponsortoken: XXX

# telemetry enables aggregated statistics
#
# Telemetry allows collecting usage data (grid and green energy, charge power).
# Data is aggregated, no individual charging sessions are tracked. The collected,
# anonymous data can be retrieved using https://api.evcc.io.
#
# See https://github.com/evcc-io/evcc/pull/4343 or details.
#
# For time being, this is only available to sponsors, hence data is associated with
# the sponsor token's identity.
#
# telemetry: true

# log settings
log: debug
levels:
  site: debug
  lp-1: debug
  lp-2: debug
  cache: error
  db: error

# modbus proxy for allowing external programs to reuse the evcc modbus connection
# each entry will start a proxy instance at the given port speaking Modbus TCP and
# relaying to the given modbus downstream device (either TCP or RTU, RS485 or TCP)
modbusproxy:
  #  - port: 5200
  #    uri: solar-edge:502
  #    # rtu: true
  #    # readonly: true

# meter definitions
# name can be freely chosen and is used as reference when assigning meters to site and loadpoints
# for documentation see https://docs.evcc.io/docs/devices/meters
meters:
- type: template
  template: fronius-gen24
  usage: grid
  host: <IP_ADDRESS>
  port: 502
  name: grid1
- type: template
  template: fronius-gen24
  usage: pv
  host: <IP_ADDRESS>
  port: 502
  name: pv1
- type: template
  template: fronius-gen24
  usage: battery
  host: <IP_ADDRESS>
  port: 502
  name: battery1

# charger definitions
# name can be freely chosen and is used as reference when assigning charger to vehicle
# for documentation see https://docs.evcc.io/docs/devices/chargers
chargers:
  - name: charger1
    type: template
    template: easee
    user: XXX
    password: XXX
    charger: XXX

# vehicle definitions
# name can be freely chosen and is used as reference when assigning vehicle to loadpoint
# for documentation see https://docs.evcc.io/docs/devices/vehicles
vehicles:
  - name: car1
    type: mercedes
    title: b-klasse
    capacity: 12 # kWh
    clientId: XXX
    clientSecret: XXX
    vin: XXX
    #onIdentify: # set defaults when vehicle is identified
    #  mode: pv # enable PV-charging when vehicle is identified
    #  minSoC: 0 # immediately charge to 0% regardless of mode unless "off" (disabled)
    #  targetSoC: 100 # limit charge to 90%

# site describes the EVU connection, PV and home battery
site:
  title: Home # display name for UI
  meters:
    grid: grid1 # grid meter
    pv: pv1 # list of pv inverters/ meters
    battery: battery1 # battery meter
  prioritySoC: 50 # give home battery priority up to this soc (empty to disable)
  bufferSoC: 80 # ignore home battery discharge above soc (empty to disable)
  residualPower: 100

# loadpoint describes the charger, charge meter and connected vehicle
loadpoints:
  - title: Hof # display name for UI
    charger: charger1 # charger
    #meter: pv1 # charge meter
    mode: "off" # set default charge mode, use "off" to disable by default if charger is publicly available
    vehicle: car1 # set default vehicle (disables vehicle detection)
    resetOnDisconnect: true # set defaults when vehicle disconnects
    soc:
      # polling defines usage of the vehicle APIs
      # Modifying the default settings it NOT recommended. It MAY deplete your vehicle's battery
      # or lead to vehicle manufacturer banning you from API use. USE AT YOUR OWN RISK.
      poll:
        # poll mode defines under which condition the vehicle API is called:
        #   charging: update vehicle ONLY when charging (this is the recommended default)
        #   connected: update vehicle when connected (not only charging), interval defines how often
        #   always: always update vehicle regardless of connection state, interval defines how often (only supported for single vehicle)
        mode: charging
        # poll interval defines how often the vehicle API may be polled if NOT charging
        interval: 60m
      estimate: true # set false to disable interpolating between api updates (not recommended)
    phases: 0 # electrical connection (normal charger: default 3 for 3 phase, 1p3p charger: 0 for "auto" or 1/3 for fixed phases)
    enable: # pv mode enable behavior
      delay: 1m # threshold must be exceeded for this long
      threshold: 0 # grid power threshold (in Watts, negative=export). If zero, export must exceed minimum charge power to enable
    disable: # pv mode disable behavior
      delay: 3m # threshold must be exceeded for this long
      threshold: 0 # maximum import power (W)
    guardDuration: 5m # switch charger contactor not more often than this (default 5m)
    minCurrent: 6 # minimum charge current (default 6A)
    maxCurrent: 16 # maximum charge current (default 16A)

# tariffs are the fixed or variable tariffs
# cheap (tibber/awattar) can be used to define a tariff rate considered cheap enough for charging
tariffs:
  currency: EUR # three letter ISO-4217 currency code (default EUR)
  grid:
    # either static grid price
    type: fixed
    price: 0.294 # EUR/kWh

    # # or variable via tibber
    # type: tibber
    # cheap: 0.2 # EUR/kWh
    # token: "476c477d8a039529478ebd690d35ddd80e3308ffc49b59c65b142321aee963a4" # access token
    # homeid: "cc83e83e-8cbf-4595-9bf7-c3cf192f7d9c" # optional if multiple homes associated to account

    # # or variable via awattar
    # type: awattar
    # cheap: 0.2 # EUR/kWh
    # region: de # optional, choose at for Austria
  feedin:
    # rate for feeding excess (pv) energy to the grid
    type: fixed
    price: 0.08 # EUR/kWh

# mqtt message broker
mqtt:
  # broker: localhost:1883
  # topic: evcc # root topic for publishing, set empty to disable
  # user:
  # password:

# influx database
influx:
  # url: http://localhost:8086
  # database: evcc
  # user:
  # password:

# eebus credentials
eebus:
  # uri: # :4712
  # interfaces: # limit eebus to specific network interfaces
  # - en0
  # certificate: # local signed certificate, required, can be generated via `evcc eebus-cert`
  #   public: # public key
  #   private: # private key

# push messages
messaging:
  events:
    start: # charge start event
      title: Charge started
      msg: Started charging in "${mode}" mode
    stop: # charge stop event
      title: Charge finished
      msg: Finished charging ${chargedEnergy:%.1fk}kWh in ${chargeDuration}.
    connect: # vehicle connect event
      title: Car connected
      msg: "Car connected at ${pvPower:%.1fk}kW PV"
    disconnect: # vehicle connected event
      title: Car disconnected
      msg: Car disconnected after ${connectedDuration}
    soc: # vehicle soc update event
      title: SoC updated
      msg: Battery charged to ${vehicleSoC:%.0f}%
    guest: # vehicle could not be identified
      title: Unknown vehicle
      msg: Unknown vehicle, guest connected?
  services:
  # - type: pushover
  #   app: # app id
  #   recipients:
  #   - # list of recipient ids
  # - type: telegram
  #   token: # bot id
  #   chats:
  #   - # list of chat ids
  # - type: email
  #   uri: smtp://<user>:<password>@<host>:<port>/?fromAddress=<from>&toAddresses=<to>
  # - type: ntfy
  #   uri: https://<host>/<topics>
  #   priority: <priority>
  #   tags: <tags>

Log details
Using config file: /config/evcc.yaml
starting evcc: 'EVCC_DATABASE_DSN=/data/evcc.db evcc --config /config/evcc.yaml'
[main  ] INFO 2023/08/01 20:32:06 evcc 0.118.11
[main  ] INFO 2023/08/01 20:32:06 using config file: /config/evcc.yaml
[main  ] INFO 2023/08/01 20:32:06 starting ui and api at :7070
[site  ] INFO 2023/08/01 20:32:17 site config:
[site  ] INFO 2023/08/01 20:32:17   meters:      grid ✓ pv ✓ battery ✓
[site  ] INFO 2023/08/01 20:32:17     grid:      power ✓ energy ✓ currents ✓
[site  ] INFO 2023/08/01 20:32:17     pv 1:      power ✓ energy ✓ currents ✗
[site  ] INFO 2023/08/01 20:32:17     battery 1: power ✓ energy ✗ currents ✗ soc ✓ capacity ✗
[site  ] INFO 2023/08/01 20:32:17   vehicles:
[site  ] INFO 2023/08/01 20:32:17     vehicle 1: range ✓ finish ✗ status ✗ climate ✗ wakeup ✗
[lp-1  ] INFO 2023/08/01 20:32:17 loadpoint 1:
[lp-1  ] INFO 2023/08/01 20:32:17   mode:        off
[lp-1  ] INFO 2023/08/01 20:32:17   charger:     power ✓ energy ✓ currents ✓ phases ✓ wakeup ✗
[lp-1  ] INFO 2023/08/01 20:32:17   meters:      charge ✓
[lp-1  ] INFO 2023/08/01 20:32:17     charge:    power ✓ energy ✓ currents ✓
[lp-1  ] DEBUG 2023/08/01 20:32:17 phase timer inactive
[lp-1  ] DEBUG 2023/08/01 20:32:17 pv timer inactive
[lp-1  ] DEBUG 2023/08/01 20:32:17 guard timer inactive
[lp-1  ] INFO 2023/08/01 20:32:17 vehicle updated: unknown -> b-klasse
[main  ] INFO 2023/08/01 20:32:17 ensure the oauth client redirect/callback is configured for b-klasse: http://<IP_ADDRESS>:7070/oauth/vehicles/1/callback
[site  ] DEBUG 2023/08/01 20:32:17 ----
[lp-1  ] DEBUG 2023/08/01 20:32:17 charge power: 0W
[site  ] DEBUG 2023/08/01 20:32:17 pv power: 113W
[site  ] DEBUG 2023/08/01 20:32:17 battery soc: 97%
[site  ] DEBUG 2023/08/01 20:32:17 battery power: 308W
[site  ] DEBUG 2023/08/01 20:32:17 grid power: -12W
[site  ] DEBUG 2023/08/01 20:32:17 grid currents: [-0.601 2.11 -0.562]A
[site  ] DEBUG 2023/08/01 20:32:17 site power: 396W
[lp-1  ] DEBUG 2023/08/01 20:32:17 charge currents: [0.009 0.01 0.013]A
[lp-1  ] DEBUG 2023/08/01 20:32:17 charge total import: 508.743kWh
[lp-1  ] DEBUG 2023/08/01 20:32:17 charger status: B
[lp-1  ] INFO 2023/08/01 20:32:17 car connected
[lp-1  ] DEBUG 2023/08/01 20:32:17 pv timer elapse
[lp-1  ] DEBUG 2023/08/01 20:32:17 pv timer inactive
[lp-1  ] DEBUG 2023/08/01 20:32:17 charger: guard elapse
[lp-1  ] DEBUG 2023/08/01 20:32:17 guard timer inactive
[lp-1  ] DEBUG 2023/08/01 20:32:17 charge total import: 508.743kWh
[lp-1  ] DEBUG 2023/08/01 20:32:17 charger vehicle id: 9EE933F9
[lp-1  ] ERROR 2023/08/01 20:32:17 vehicle soc: Get "https://api.mercedes-benz.com/vehicledata/v2/vehicles/XXX/containers/electricvehicle": oauth2: token expired and refresh token is not set
[mercedes] DEBUG 2023/08/01 20:32:27 callback request received
[site  ] DEBUG 2023/08/01 20:32:47 ----
[lp-1  ] DEBUG 2023/08/01 20:32:47 charge power: 0W
[site  ] DEBUG 2023/08/01 20:32:47 pv power: 111W
[site  ] DEBUG 2023/08/01 20:32:47 battery soc: 97%
[site  ] DEBUG 2023/08/01 20:32:47 battery power: 297W
[site  ] DEBUG 2023/08/01 20:32:47 grid power: 14W
[site  ] DEBUG 2023/08/01 20:32:47 grid currents: [-0.594 2.15 -0.551]A
[site  ] DEBUG 2023/08/01 20:32:47 site power: 411W
[lp-1  ] DEBUG 2023/08/01 20:32:47 charge currents: [0.009 0.01 0.013]A
[lp-1  ] DEBUG 2023/08/01 20:32:47 charge total import: 508.743kWh
[lp-1  ] DEBUG 2023/08/01 20:32:47 charger status: B

What type of operating system are you running?
HomeAssistant Add-on
Version
0.118.11

The log tells me that the initial login is performed. From that moment on it should work. If not please use the ui login.

UI Login works but SoC is still not shown:

@andig can you please reopen

We need a vehicle trace log.

How to create one? I put
log: debug
levels:
  car1: trace

Restarted evcc but result is the same.

mercedes: trace

Nothing with Mercedes debug or trace apart of that after I use the login of „b-klasse“ in evcc menu:
[mercedes] DEBUG 2023/08/02 06:39:35 callback request received
[mercedes] TRACE 2023/08/02 06:39:36 sending login update...

@andig it works now. replugging the charger cable solved it. :D


feat: add global user properties plugin

Summary
This PR adds a simple enrichment plugin that transforms the user properties field into global_user_properties - a beta feature used internally first
Checklist

[ ] Does your PR title have the correct title format?
Does your PR have a breaking change?:

@justin-fiedler should we point this pr to v1.x?

for other events which are not belong to amplitude special events, if there is no user_properties, do we want to set it to null?

I think returning the event unmodified would work in these cases

isNotSpecialAmplitudeEvent means no event_type is '$identify', isAmplitudeIdentifyEvent means the event_type = '$identify'.

The type narrowing is not what i'm expecting it to do, but ideally I'm only modifying events of type IdentifyEvent and BaseEvent - really, I'm trying to avoid mutating group identify and revenue events, and other special event types that may come along the way

Thanks everyone for the comments (and sorry for the amount of copy-pasta I left on the ground). I've updated the readme and the tests to include a modules.d.ts that fixes the types.
Unfortunately, ts-jest is having some issues picking up this file, so the tests use a couple of any's. I'll make this a follow up ticket if this isn't a blocker to this pr.

@justin-fiedler @yuhao900914 I've limited my changes to the plugin package - I'll be merging this if there are no concerns


Geometric invalidity in exported CityJSON files

Hello,
I imported a CityGML file using the 3DCityDB Importer-Exporter, which I then exported as a CityJSON file. While the CityGML file is valid (checked with FME, 3DCityDB, val3dity), the output CityJSON file seems to be invalid when it comes to the geometries. There are consecutive duplicate points in boundaries, and val3dity reported other errors as well (CONSECUTIVE_POINTS_SAME, RING_SELF_INTERSECTION, INTERSECTION_RINGS, etc).
Since the input CityGML file is valid, I thought maybe this problem would be related to the 3DCityDB Exporter.
Thanks!

Duplicate of https://github.com/citygml4j/citygml-tools/issues/27. As described there, increase the number of significant digits. This can be set under "Preferences -> Export -> CityJSON options -> Precision Coordinates".
Check the chapter <IP_ADDRESS> of the online user manual and especially the "Caution" box.


Only invoke the install of qubes-windows-tools.iso if it does not already exist in dom0 // Create qubes.Filecopy if missing and populate with defaults // wrap sed in sudo due to changed permissions in R4.1

[EK - Might want to hold off on reviewing this for a bit - I am now thinking the QWT check should be earlier,  and dependency/install perhaps controlled by a flag. Also, a qubes version check based on /etc/qubes-release might be helpful for the rpc policy file changes to avoid some security issues in most cases.]
First two commits (for allowing the script to skip downloading the QWT ISO if it already exists)

This helps the script run faster if QWT is already installed.
This can avoid an error for some users if there are issues with the qubes repos. E.g. the current (as of Oct 18 2021) Qubes v4.1rc1 issues with QWT missing from the repos. Particularly helpful if the user has a manual workaround for getting QWT installed outside of the script.

Second two commits (to allow the script to copy the post-QWT customizing scripts to the target VM for execution, without "ask" intervention):
3. Under R4.1, the rpc policy directory has changed, leading to a missing qubes.Filecopy file from the script perspective. The easy temporary fix (because R4.1 will read files from the now-deprecated directory) is to create the file with the default values. This may not be the best from a security perspective, esp. if the user has utilize the new directory structure to restrict the policy even more than the default.
4. In addition the deprecated target directory of /etc/qubes-rpc/policy no longer allows members of the qubes groups to create files there, so sed -i throws an error. Quick fix is to prefix with sudo, but that is also not the best approach.
An alternative for 3 (and removing the need for 4) would be for the script to add a policy file in the new directory if running R4.1 or later, then remove it when done. I may update the PR to do this instead, as it is less likely to have security side-effects.
For changing of policy directory/format, see https://github.com/QubesOS/qubes-issues/issues/6989

I am going to resubmit piecemeal.


Add rand, ror, rsub operators in dict_items

Added rand, ror, rsub functions for dict_items object

ref #3248

Added rand, ror, rsub functions for dict_items object and removed test annotation


Review README and website content

@efvicario Hello, the last part before releasing the package is to review the content of README and the website. Please let me know if you would like to change any writing parts, add more links, or have any questions.
The citation information will be fixed after the release.
Like dictionary, I also encourage you to make changes on README.Rmd by yourself. I am happy to provide assistance here.

Hi @mianzg
Sorry for the delay, I missed this post.
There are some edits here for the introduction for the readme file, I have reuploaded it. Please let me know if I can look over anything else!
Elizabeth

Hi, anything I can do to move publishing along? I want to link to this dataset in a paper that I'm about to submit. Thanks

@efvicario Thanks for reaching out. I will wrap up and make a release for the data package by this Sunday. Does it work for you?

Hi @mianzg , thank you! Much appreciated.

Hi @mianzg, I can cite the package now I see, was it Götschmann et al before and now I am the first author? Let me know if it's correct so I can cite the placeholder! Thanks

Hi @efvicario, I will take over from @mianzg. I will work on fixing the citation information

Citation is fixed and website updated. All covered in the following Pull Request which I have already merged into main: https://github.com/openwashdata/watercostaccra/pull/7


refactor: delete env file

:tada: This PR is included in version 1.1.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Add support for classes blocking mocking.

Certain classes should not be mocked. They may not be able to support being mocked
because of implementation details that conflict with how OCMock is implemented.
This allows library developers to prevent those classes from being mocked.

Merged this with three noteworthy changes:

The method to block mocking is a class method and we don't want to check for presence of the protocol. Therefore I have move the declaration from a formal protocol to a category.
I was uncomfortable with the double use of nil/reason to signal whether mocking should be block and to return a reason. For that reason I changed the signature to return a boolean and the reason in a separate argument. I've also changed the to the "positive", i.e. returning NO means that mocking is not supported. Of course, not implementing the method still implied that mocking is supported.
I reverted the removal of the nil object check in the partial mock object class. That wasn't covered anymore as far as I understand.


Do not finalize the value of the properties of a task that is skipped

https://github.com/gradle/gradle/pull/7371 added support to finalize the value of certain task input and output properties when the task starts execution. However, this finalization can be expensive and is pointless for many properties of a task that is skipped.
For example, finalizing the compile classpath for a compile task may involve a bunch of network access to resolve dependencies, however the value will be ignored if the task has no source files to compile and is skipped.
Expected Behavior
Finalize the value of only those properties of a task that are required to decide whether a task should be skipped. Only if the task will proceed should the remaining properties be finalized.
We should probably do the same with the "get from cache or run task actions" decision, and only finalize properties such as local state properties after we've decided to run the task actions.
The implementation might simply be to set all task properties to "finalize on next query" early in the execution lifecycle, so they are finalized only as required by whatever needs the value.
Current Behavior
Gradle does too much work when skipping tasks.

We already don't finalize inputs for tasks that are skipped early via onlyIf() or -x. I think this is only about primary inputs (@SkipWhenEmpty) that should be finalized first, checked if we can skip the task, and only then we should continue finalizing the rest of the inputs.
Is there another case we need to care about here?

That's a good point: we should always validate any property that has been finalized.

I want to look at task property validation as part of https://github.com/gradle/gradle/issues/4468, so that validation is triggered in some way by finalizing the value.


How to assign own name to directory ?

Hello guys ! I am new to Chat Script, and am trying to make my bot assign specified name of dir by user. Seems like it is possible but I am far away from being familiar to the syntax.
The example below will explain what i mean:
t: (create [file directory] _*) ^keep() ^repeat() popen( “mkdir _0” ‘^myfunc)
When i say create file or directory fileName123 i would like it to create dir with specified name.
I am not sure how to apply Chat Script syntax to the system command so it would be readable, for now every directory is called “_0”.
Anyone would be able to tell me if it is possible to do it ?
I am using Linux Fedora.
Thanks in Advance !
Bart

Solved ! It was the matter of using ^system instead of popen.
t: (create [file directory] _*) ^keep() ^repeat() ^system(mkdir '_0)


v.5.9.1 animated blocks cannot be created

with the latest version of the plugin (v5.9.1) you simply cannot create animated blocks. You can press plus in the workspace and enter a name for the block, but no matter what you enter, as soon as you confirm, the window simply disappears again and the animated block is not created, please fix this

I need your mcreator.log. Find it in user/your user/.mcreator/logs

Nevermind, found the issue. They decided to rename a line of code for no reason in the latest patch.


Fix resize logic (partly)

Extract refactored code from #1427, that will likely to conflict with others
mitigate negative flicker bug (#1443)

@dwelle Would you check the behavior if it works for you in terms of #1443?
There's another approach you might like, which is to change like the following
  const offsetPointer = 0;
  const minSize = 0;

in resizeElements.ts.

This behaves weirdly --- it skips resizing below ~10px width/height from both sides:

Well, that's logically correct with offsetPointer.
I made it 0 in the latest commit. How do you like this one?
(No idea yet to solve both issues.)

(No idea yet to solve both issues.)

I have one. How do you like this? A breaking change in UI.

OK, let me commit it, because it make a lot of sense.

Works great, except one problem: resizing to the negative moves the shape by one pixel before expanding further.

I have one. How do you like this? A breaking change in UI.

I don't know. I wouldn't do this in this PR (let's create a discussion issue for it).
Also, this PR worsens a case we fixed one (https://github.com/excalidraw/excalidraw/pull/341): when you click on a handle to resize and move the mouse slightly, the element jumps and resizes to cursor position (the position of the handle you clicked on). This behavior is present even in prod, but in this PR it looks like it was made worse :/.
(Note that all the above bugs and the bug this PR fixes were all fixed and working ~month ago, so I'm just trying to make us fix these regressions to move us back to where we once were.)

I guess #341 doesn't take account the case of #1296 and negative flip.

This behavior is present even in prod

Are you sure? That's the purpose of offsetPointer.

I actually don't understand why flicker happens, so maybe there's a better way to fix it. It should technically be possible.
Reverting a commit for now.

I actually don't understand why flicker happens

It think I got it and fixed.
@dwelle Would you please check it again?

resizing to the negative moves the shape by one pixel before expanding further.

I can't reproduce this issue. Maybe this only occurs in a low-resolution screen.
The possible hack I come up with is the following:
diff --git a/src/element/resizeElements.ts b/src/element/resizeElements.ts
index e05e463..95ff974 100644
--- a/src/element/resizeElements.ts
+++ b/src/element/resizeElements.ts
@@ -140,7 +140,7 @@ export const resizeElements = (
   const handleOffset = 4 / appState.zoom; // XXX import constant
   const dashedLinePadding = 4 / appState.zoom; // XXX import constant
   const offsetPointer = handleOffset + dashedLinePadding;
-  const minSize = 0;
+  const minSize = 0.1; // a magic number for low resolution screen
   if (selectedElements.length === 1) {
     const [element] = selectedElements;
     if (resizeHandle === "rotation") {

Does this mitigate your issue?

Still suboptimal in that it freezes resizing below the given threshold. When I get a chance, I'll try to fix this since you're probably stabbing in the dark, being on high DPI screen yourself.

When I get a chance, I'll try to fix this

That would be nice.

this PR worsens a case we fixed once (#341)

I assume it is fixed with the last commit. Any chance this would be merged soon? I'd like to work on a follow-up PR.

Still suboptimal in that it freezes resizing below the given threshold.

Hm, you mean it's suboptimal as is? To my understanding, we need to freeze it while switching handles going negative (ex. "e" -> "w"), unless we introduce a new UI (https://github.com/excalidraw/excalidraw/pull/1475#issuecomment-618709800).

Hm, you mean it's suboptimal as is? To my understanding, we need to freeze it while switching handles going negative (ex. "e" -> "w"), unless we introduce a new UI (#1475 (comment)).

No. It was working fine once, as I said. Either way, there's nothing fundamentally preventing it, unless there was a major change to the way we handle it that I'm not aware of (possible).

It was working fine once

I don't know how it's working at the point of #341, but as it's using only delta, it will loose the position when going negative size. Same as #1296

there was a major change to the way we handle it

So, #1330 rewrote the whole logic to base on the pointer position without delta.
But, because it has offsetPointer, it deals with #341 case.
678c750 is probably providing a better behavior than before. It freezes resizing while changing handles, but otherwise it will loose the position (= make gap between the pointer and the handle).

I guess I'm a bit confused with discussion points. Let me try listing them.

Ideal UI
The current resizing logic (position based, no delta)
What prevents merging this PR

Let's merge it, and we'll make this work more optimally later.


docs:modify the one-line comment example

docs:modify the one-line comment example ( #20 )

docs:modify a typo in the xss attack example  ( #14 )


Add optional module telemetry

Overview/Summary
PR to add optional module telemetry using the ARM template method.
Testing Evidence
Please provide any testing evidence to show that your Pull Request works/fixes as described and planned (include screenshots, if appropriate).
As part of this Pull Request I have

[x] Checked for duplicate Pull Requests
[x] Associated it with relevant issues, for tracking and closure.
[x] Ensured my code/branch is up-to-date with the latest changes in the main branch
[ ] Performed testing and provided evidence.
[x] Updated relevant and associated documentation.
[ ] Updated the "What's New?" wiki page (located in the Enterprise-Scale repo in the directory: /docs/wiki/whats-new.md)

/azp run unit

/azp run unit

/azp run unit

/azp run unit

/alz run unit

/azp run unit

Looking good... a couple of feedback items added inline.
Also, do we still want to go with the bitwise in the deployment ID, or have a couple of Subscription scoped deployments for connectivity and management? This might give us greater flexibility in the future, and make it easier to identify things like hub and spoke vs. vwan deployments, etc.

Changed now, we have four deployments but identity is disabled for now until we add the provider declaration.

/azp run e2e

/azp run unit


bug: small  typo

what the problem is:
pyinstaller -F --add-data 'roots.pem:grpc/_cython/_credentials/" scanner.py should be pyinstaller -F --add-data 'roots.pem:grpc/_cython/_credentials/' scanner.py
what we should do:
- pyinstaller -F --add-data 'roots.pem:grpc/_cython/_credentials/" scanner.py
+ pyinstaller -F --add-data 'roots.pem:grpc/_cython/_credentials/' scanner.py

where the problem is: https://github.com/google/gcp_scanner/blob/103cbd0cbad74ceff8aa04b891351cbf41f71886/README.md#L92

Hi @ZhengLin-Li  , there is a pull request https://github.com/google/gcp_scanner/pull/50 which is for the readme.md file opened  by me. I have corrected this typo.


Consolidated Link/Backlink views for both canvases and regular notes

Hey! This plugin is something that I think should be a core feature of Obsidian. However, for me it isn't necesary to differentiate between links to/from canvases and links to/from regular notes. I'd love to have these together in two views instead of four. I don't know if it would be possible to add a toggle in settings to change this behaviour, but I'd just love to get rid of the built in "linked mentions" and "links" views, and replace them wih views that combine that with the canvas link views you've made here.

Hi @belacmu
First time I wanted to combine these two view with the core plugin view: "backlinks" and "outgoing link", but not long after, I realized that it hard to implemented(maybe impossible, I am not sure), because the core plugin is built-in the Obsidian and Obsidian is not open-source. So I turn to created two new view to show what I wanna.

That makes sense! And I'm not actually familiar with any of this stuff, so you would know much better than me. I wonder if it would be possible for your two new views to also include "regular links" within them. Basically remaking those two built in views, and including them in your own custom views.

Good idea I think, it will take some time to implement


Code of Conduct at Event Page - add a template

In the Known Implementations Resources section at the bottom of https://chaoss.community/metric-code-of-conduct-at-event/ add an additional resource that helps Events add code of conducts.
Montréal's Women in Tech developed a really helpful template to implement at events that could be a helpful in this page.
https://manifestedesfemmesentech.com/en/code-of-conduct-template

Thanks!


Is it possible to use a wider data type than float with expr_eval()?

Looks like the return type of expr_eval() is float which is 32 bit wide on my 64 bit system. Is it possible to use double which is of 64 bits while evaluating an expression?

Well, expr.h is a single-header library, so you can probably copy it and find/replace all float with double to make it work.
Alternatively, you may replace float with some expr_num_t where appropriate. It would allow you to redefine expr_num_t as double if needed, or any other implementation like fixed-point C++ class with overloaded operators. I haven't thought that far, but I think it might be convenient in some cases.
If you decide to go with expr_num_t way - may I ask you to do a pull request then?


Support explicit type compatibility assertion

Sometimes we may want to explicitly assert whether the type of a expression is compatible with a certain type, but type assertion does not satisfy the type safety we want:
interface SomeEventData {
  foo: number;
  bar: string;
}

this.emit('some-event', {foo: 123} as SomeEventData);

The behavior is desired but it would be nice if we have an elegant way for more secure type checking instead of:
let data: SomeEventData = {foo: 123};

E.g.:
this.emit('some-event', {foo: 123}: SomeEventData);

Even better: make sure that incompatible types will raise a type error
class MyEvent<A> {
  readonly _A: A
  constructor(readonly name: string) {}
}

type Handler<A> = (a: A) => void

declare class MyEventEmitter {
  on<A>(event: MyEvent<A>): (handler: Handler<A>) => void
  emit<A>(event: MyEvent<A>): (a: A) => void
}

interface SomeEventData {
  foo: number
  bar: string
}

const emitter = new MyEventEmitter()
const someEvent = new MyEvent<SomeEventData>('some-event')

emitter.emit(someEvent)({}) // error
emitter.emit(someEvent)({ foo: 'a' }) // error
emitter.emit(someEvent)({ foo: 1, bar: 's' }) // ok

emitter.on(someEvent)(() => {}) // ok
emitter.on(someEvent)((foo: number) => {}) // error
emitter.on(someEvent)(foo => {}) // ok
emitter.on(someEvent)((foo: SomeEventData) => {}) // ok

@gcanti Looks nice (sharing data type between emit and on)! Though creating a wrapper that persists at runtime does not sound ideal to me, especially when we have a lot of libs adapting to implementations like EventEmitter.

Yeah, maybe a branded type instead of a class?
type MyEvent<A> = string & { _A: A } // branded type, the runtime representation is still a string

function createEvent<A>(name: string): MyEvent<A> {
  return name as any
}

const someEvent = createEvent<SomeEventData>('some-event')

I cannot find other discussion on this, but I have also thought it would be useful to have a type-assertion like operator that doesn't actually coerce the type of the expression.
This operator:

Still provides a contextual type.
Can be used on any expression.
Has the type of the contained expression (rather than the type being ensured/checked against).

For example:

interface Foo {
    bar(x: string): void;
}

// 'x' has type
//
//    {
//        bar(a: string): void,
//        baz(): number
//    }
//
const x = {
    bar(a) {
      console.log(a.toLowerCase());
    }
    baz() {
      return 100;
    }
} compatibleWith Foo;

@gcanti Nice try, but it would still be weird working with existing event emitters. :(

migrated from #20188
in my code i often use, let's call them, type invariants assertions:
function mustBe<T>(_: () => T): void {}
interface A {
    isThat: true;
    text: string;
}
interface B {
    isThat: false;
    value: number;
}
type C = A | B;
declare const C: C;
mustBe<true>(() => C.isThat); // <-- type error as expected

now i wish i could do type invariant assertions without emitting any useless code, making it a zero cost abstraction:
makesure C.isThat sameas true;

where makesure is a new TS syntax/construct that participates in type checking, but doesn't get emitted
few more examples
makesure typeof c.isThat subtypeof boolean
makesure MyClass subtypeof BaseClass
// ...

Tracking at #7481 / #26064


Support Deeper fishfinder.

I have a deeper fishfinder and it collects GPS and depth data into a Sqlite database. It is already building a topographic chart of any area where I use it. I believe data like this would be very useful for openseamap.
It costs around $250 so I would expect this to be a popular solution in the future.

Thanks!
Please send some info about the system by mail, so we can add it to the Wiki.
In which area do you collect data?
Please upload some test data to depth.openseamap.org for investigating.

Hi
The product is called deeper. Here is a video about the solution.
https://www.youtube.com/watch?v=HyzQdEGLw40
There is also info about it on Wikipedia
https://en.wikipedia.org/wiki/Deeper_Fishfinder
Worth to note is that the application has an backup to gdrive functionality. This will dump an sqlite database file on you google drive.
My version is a pro+ and that version has a builtin gps. You can probably also use the cheaper standard version and use the phone gps if the phone is at the same location.
My plan is to measure natural harbors around Stockholm in sweden.
I already uploaded a demo file I created by putting the sonar on a fishing rood.
/Johan
Date: Wed, 11 May 2016 21:56:36 -0700
From<EMAIL_ADDRESS>To<EMAIL_ADDRESS>CC<EMAIL_ADDRESS><EMAIL_ADDRESS>Subject: Re: [OpenSeaMap/depth_webfrontend] Support Deeper fishfinder. (#176)
Thanks!
Please send some info about the system by mail, so we can add it to the Wiki.
In which area do you collect data?
Please upload some test data to depth.openseamap.org for investigating.
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub

Hi Johan, sounds good!

sqlite database file

Please can you send me a test file by mail?
Do you have a DB/data description?

HiThe mails fails. The file is around 4 mb and might be to large.I believe the table "Measurement" would be best to use. This contains the columns "longitude", "latitude", "depth"You can open the file using this toolhttp://www.snapfiles.com/get/databasebrowser.html
Date: Wed, 25 May 2016 02:34:54 -0700
From<EMAIL_ADDRESS>To<EMAIL_ADDRESS>CC<EMAIL_ADDRESS><EMAIL_ADDRESS>Subject: Re: [OpenSeaMap/depth_webfrontend] Support Deeper fishfinder. (#176)
Hi Johan, sounds good!

sqlite database file

Please can you send me a test file by mail?
Do you have a DB/data description?
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub

Here are some sample data of the deeper backup files.
https://darkstoneathome.de/owncloud/s/BrqJzRjH6qTcQQP
They are sqlite databases. I think the table "Measurement" is the most important.

Welcome to OpenSeaMap :-)
Please write by email (adress on the website
then I will send you more informations

Hello, You might be interested in the Deeper Chirp+ as it supports NMEA0183 over UDP / WiFI https://support.deeper.eu/284760-Extracting-NMEA0183-data-over-UDP-port-Deeper-CHIRP-model-only.
Johannes


SLIM-675-rename-configurationobjects

 You should sign our Contributor License Agreement in order to get your pull request merged.

All is well.

All is well.

All is well.

All is well.

All is well.

All is well.

All is well.

All is well.

All is well.


Incorrect cursor location when going from split to single view

When
When reducing one of the panel to go from a split view of the code and rendered document to a single panel (or the other way around), the cursor is not at the correct position.
Outcome
Clicking in the text to position the cursor will end up writing on the line below / above
This only appears until some text is typed in.
The cursor can then be moved to a correct location by clicking in the text again.
Environment

Linux Debian stretch
Chromium  53.0.2785.143
Monod version ff0b2d9
Template: no template

Aloha! Thanks for the bug report.
I tried to fix this issue in #248, but the fix is not good enough..

Sorry just checked the issues not the PR list :(

Well, let's keep the issue open for now.


feat(scripts/install.sh): don't pass install token via command line

I'm leaving the actual option in, as it's currently still displayed in the UI, but I'm removing it from documentation and manifests for Chef, Puppet and Ansible.

Modified the tests as well.


Remove dependency on @bigtest/performance

Motivation
https://github.com/thefrontside/bigtest/pull/1006#issuecomment-916783304
Approach
Remove the polyfill, since interactors are browser-only anyway.
After merge
Update sample app dependencies. The dev process for that part of this repo doesn't feel great, since with its own lock files it doesn't fit in the monorepo workflows.

@cowboyd you sure we can just switch to window.performance? Am I missing something obvious here re: environments?

@cherewaty Yup, it's just ponyfills for each environment: https://github.com/thefrontside/bigtest/blob/v0/packages/performance/package.json#L6
You can see the browser ponyfill just pulls them off globalThis https://github.com/thefrontside/bigtest/blob/v0/packages/performance/browser-ponyfill.js
You may want to use globalThis instead of window as that seems to be the preferred nomenclature these days.

Replaced by https://github.com/thefrontside/interactors/pull/73


Fixes issues with reload because of output emit

This has two changes:
1020ea2 ensure that if there is no change in folder structure, only files are added or removed the directory watcher is invoked with those fileNames and other wise its invoked with directory name being watched since the watches on child are updated at later time, we wouldn't know comprehensive list of changes. This change helps in optimizing in not having to refresh program when we get better paths for file or directories added
a37585e unifies the logic to ignore wild card directory watch call back invoke.  It checks if the file is excluded by config or output file next to the input file so that those emits can be ignored.
Note :
2cf6d4f  shows testcase where rootFile thats being renamed invokes file watcher first and update happens... after which the actual directory watcher is invoked.. so there is intermittent program update that still includes file to be renamed but that file is deleted from the disk. So the error is that file not found. After wild card directory update (because of async nature of the directory watching update)  the correct error is shown (module not found).
I did consider having to update the filenames if fileWatcher was invoked with file created with deleted but given that the update eventually happens i think it would be extra cost to bet the new files names from the disk on such events for the config and hence decided to keep it as is. Note that before this fix the correct error was never shown till you invoke tsc again.
Fixes #37994, #38684

Ping.. I want to get this in and there is another PR I am working on that depends on this.


Caching stages

What happened? What did you expect to happen?
I have the following main.py file and I would like to understand how I utilize the caching functionality in Dagger.
I am using the following version:
dagger-io==0.5.1
Python 3.11.3
import sys
import anyio
import dagger

async def main():
    config = dagger.Config(log_output=sys.stdout)
    async with dagger.Connection(config) as client:
        first = (
            client.container()
            .from_("python:3.11-slim")
            .with_directory("/app", client.host().directory("."))
            .with_exec(["pip", "install", "-r", "/app/requirements.txt"])
        )

        second = first.with_exec(["echo", "hello 1"])

        await second.exit_code()

if __name__ == "__main__":
    anyio.run(main)

When I run the pipeline the first time, nothing is cached and all command are executed. The second time I run the pipeline all stages are cached and none are returned. However when I change for the first stage from echo hello 1 to echo hello 2 the python libraries are installed yet again in the first stage.
How can I obtain a caching functionality for the pipeline to ensure that I do not run all stages prior to the modified stage?

I don't know the structure of your repository, but from what I'm seeing it's likely that the change you're making to this main.py file, which would be loaded as a part of client.host().directory("."), is causing the cache to be invalidated.
I'd recommend changing client.host().directory(".") to client.host().directory(".", include: ["requirements.txt"]) or perhaps client.host().directory(".", exclude: ["main.py"]), so that the change of echo hello 1 to echo hello 2 in the main.py doesn't invalidate the cache.

@SebastianBalle can you confirm if @jlongtine's suggestion worked for you?

@mircubed yes the suggestion worked for me. My actual use case was actually to run a Python script and automatically include relevant files. I have added a small example on how I did it.
import sys
import anyio
import dagger

async def main():
    config = dagger.Config(log_output=sys.stdout)
    async with dagger.Connection(config) as client:
        python_file_path = "path/to/python_script.py"
        include = find_dependencies(python_file_path)
        first = (
            client.container()
            .from_("python:3.11-slim")
            .with_directory("/app", client.host().directory(".", include=include)
            .with_exec(["python", f"/app/{python_file_path}")
        )

        python_file_path = "path/to/python_script_1.py"
        include = find_dependencies(python_file_path)
        second = (
            client.container()
            .from_("python:3.11-slim")
            .with_directory("/app", client.host().directory(".", include=include)
            .with_exec(["python", f"/app/{python_file_path}")
        )

        await second.exit_code()

if __name__ == "__main__":
    anyio.run(main)

I used the functionfind_dependencies is used to find the .py files that contained the imported functions in the executed python file. From this I can ensure that when the primary Python file changes or any files in which the related functions are changed the cache is invalidated and the new code is used.
def find_dependencies(file_path: str) -> List[str]:
    """
    Find dependencies of a Python file.

    :param file_path: The path of the Python file to analyze.
    :type file_path: str
    :return: A list of Python files that are needed to run the input Python file, including the input file itself but excluding __init__.py files, with relative paths.
    :rtype: List[str]
    """
    # Read the source code from the file
    with open(file_path, "r") as file:
        source_code = file.read()

    # Parse the source code to an AST (Abstract Syntax Tree)
    node = ast.parse(source_code)

    # Extract import statements (both 'import' and 'from ... import')
    imports = [n for n in node.body if isinstance(n, (ast.Import, ast.ImportFrom))]
    dependencies = []

    input_file_dir = os.path.dirname(os.path.abspath(file_path))
    current_working_dir = os.getcwd()

    # Temporarily add the parent directory of the input file to sys.path
    sys.path.insert(0, os.path.dirname(input_file_dir))

    for import_node in imports:
        if isinstance(import_node, ast.Import):
            for alias in import_node.names:
                module_name = alias.name
                try:
                    # Find the module specification using importlib
                    spec = importlib.util.find_spec(module_name)
                    if spec is not None and spec.origin is not None:
                        # Check if the module's origin is within the same directory as the input file
                        if (
                            os.path.dirname(spec.origin) == input_file_dir
                            and not os.path.basename(spec.origin) == "__init__.py"
                        ):
                            dependencies.append(
                                os.path.relpath(spec.origin, current_working_dir)
                            )
                except ImportError:
                    pass
        elif isinstance(import_node, ast.ImportFrom):
            module_name = import_node.module
            if module_name:
                parent_package = os.path.dirname(file_path).replace(os.sep, ".")
                relative_module_name = f"{parent_package}.{module_name}"
                try:
                    # Find the module specification using importlib
                    spec = importlib.util.find_spec(relative_module_name)
                    if spec is not None and spec.origin is not None:
                        # Check if the module's origin is within the same directory as the input file
                        if (
                            os.path.dirname(spec.origin) == input_file_dir
                            and not os.path.basename(spec.origin) == "__init__.py"
                        ):
                            dependencies.append(
                                os.path.relpath(spec.origin, current_working_dir)
                            )
                except ImportError:
                    pass

    # Remove the temporarily added directory from sys.path
    sys.path.pop(0)

    # Return the unique set of dependencies, including the input file itself but excluding __init__.py files, with relative paths
    dependencies = list(set(dependencies))
    if not os.path.basename(file_path) == "__init__.py":
        dependencies.append(os.path.relpath(file_path, current_working_dir))
    return  #dependencies


Make CMake project load all .cmake files from the module folder

This makes it so you can add a new module without having to modify the torque3d.cmake
Also updated the current modules by moving option code to each one.

Updated for fix merge problems on #1145
Thx a lot :)


Add React Hooks for customization (part 14)

Related to #2539.

Changelog Entry

Resolves #2539, added React hooks for customization, by @compulim, in the following PRs:

PR #2652: Update samples to use hooks

Description
This work will modify samples.

We will move some sample to functional components
We will use hooks instead of HOC
We will remove glamor from our samples to reduce emissions.

Specific Changes

08.customziation-user-highlighting: Remove glamor
09.customization-reaction-button: Move to functional component, using Web Chat hooks, remove glamor
10.b.customization-password-input: Move to functional component, using Web Chat hooks, remove glamor
Remove 22.change-locale which is a draft of 22.customization-change-locale

[ ] Testing Added

Coverage remained the same at 63.679% when pulling b1f436dc8e952f4205e119d815d19e4f5915c907 on compulim:feat-redux-hooks-14 into 95ab4e61b1ef4de8d326dcdefa98911efbcff7ac on microsoft:master.


Add license slides draft

Checklist

[x] I made sure that the markdown files are formatted properly.
[ ] I made sure that all hyperlinks are working.

[ ] I added the new content to the README.md inside the topics subdirectory, e.g., 01_version_control/README.md.
[ ] I added the new content to the timetable.md in the root of this repository.
[ ] I make sure to update the content/links on the homepage.

I will add it to the timetable etc. in https://github.com/Simulation-Software-Engineering/Lecture-Material/pull/94


Added constants file, removing constants from php/utils.php

Adding the get-your-refresh-token URL to the utils file was heinous (It took 8 lines!) and so inspired me to get the default constants moved out of the code.

Can we embed the environment variables dynamically into the YAML so we just have a single "constants" object that defines all of the sources?
http://symfony.com/legacy/doc/reference/1_2/en/02-yaml#chapter_02_dynamic_yaml_files

Just to expand, I don't like the idea of having a constants file that we then dynamically parse and munge with other variables.
... though looking at it more, I see there is a very clear 1-to-1 mapping between how the ENV variables are constructed and constants. If we added an explanation to the top of the constants.yml file that explained how these can be modified via ENV (and where to go look for that code), that would satisfy me.

It doesn't seem that Symfony has the ability to process inline PHP in 3.x, and in my attempt all PHP added was passed through as a string. I looked through the code and couldn't find any way to make it discern inline PHP, and so suppose that the was lost somewhere between 1.2 (as linked) and 3.0.
I added the note at the top of the constants file and changed the names of the vars to match the constants' names for clarity.

This looks good. Thanks for digging through the interpolation; I guess it's a lot cleaner in Ruby's yml parser (which I'm familiar with) to interpolate values
I'm glad you were able to pull them out into a single location. Thank you!


Set Py_GIL_DISABLED=1 for free threaded Python on Windows

When free threaded CPython is installed from the official Windows installer it doesn't have the macro Py_GIL_DISABLED properly set becuase its pyconfig.h file is shared across the co-installed default build.
Define the macro when building free threaded Python extensions on Windows so that each individual C API extension doesn't have to work around this limitation.
See https://github.com/pypa/setuptools/issues/4662

BTW I think we should release a new version that includes this PR ASAP. I think this issue have already been a block reason for the people who want to build free threading extension on windows

The test failures seem unrelated. Unfortunately, they are also failing on main, but they don't look the same. I'll rebase this PR on main and manually compare the PR tests with the ones from main.

The tests seem similar enough, I'll merge this and then see if I have time to fix the CI :upside_down_face:

This specific issue is a symptom of Python not being designed to have paths shared between multiple builds. I think it makes sense to at least discuss in more depth how this could be avoided.

I think it makes sense to at least discuss in more depth how this could be avoided.

Supporting the same set of headers for all platforms (or at minimum, all Windows architectures) and relying on preprocessor switches to choose the build options is a pretty standard way of doing this on Windows.
I certainly wouldn't want to switch wholesale to a configure/make model for all Windows users just for the sake of a temporary flag. If there's a second feature that deserves this, we can consider it.

Hey @FFY00! Are you going to merge this in pypa/setuptools as well? Having a released version of setuptools with this included would help a lot in unblocking CI for a lot of projects.

Hey, sorry for the late reply, I am not a setuptools maintainer.
cc @jaraco

My new job has been keeping me busy, so I'm just getting caught up on this. Let's see...
For future reference, I would have done the merge differently:

Merges in distutils need to have project-absolute references (e.g. pypa/distutils#310 instead of #310) because these commits are merged into Setuptools and when they're merged, they'll become Setuptools references.
Also, to help maintain fidelity, this project prefers to use merge commits instead of squash commits, as it provides the ability to summarize changes by looking at merge commits or dive into the details.

I'll get those things repaired and then get the latest changes merged into Setuptools.


Add Logos

Feature Description
Add Logo section from figma design to design system for consistent use.
https://www.figma.com/file/bMhG3KyrJF8vIAODgmbeqT/Design-System?node-id=1906%3A0## Alternatives Considered

Additional Context
if resources available, sub this new logo out in the dashboard while here!

Do not alter or remove anything below. The following sections will be managed by moderators only.
Acceptance Criteria

Adds logo and assets to design system

Implementation Brief

@csossi this is a really easy one, just a regression test that the logo on the side nav of the dashboard is still rendering - updated the source.

Verified in QA


[Console] fix console deinit: add ability to unblock linenoise, to fix deadlock (try 2) (IDFGH-9188)

Previous PR: https://github.com/espressif/esp-idf/pull/9983
I am re-opening this PR to get more traction on it.
This solves: https://github.com/espressif/esp-idf/issues/9974
Motivation:

currently, console deinit causes a deadlock
without deinit, we can't create consoles "as needed"
for example, USB Serial JTAG console should be destroyed when entering USB host mode
for example, "web" consoles should be destroyed when the browser window is closed

Hi @chipweinberger, I started looking into your PR. Can I ask you to rebase and fix the build issues and conflict with upstream?

sure, one moment.

@SoucheSouche updated. was just a trivial collison

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@chipweinberger, there has been some changes in the ring buffer so I am now looking at your PR again since the semaphores for rx and tx were removed.

since the semaphores for rx and tx were removed.

I updated the PR and removed the semaphores. But as I was saying I think we should move this bit: #define rbBUFFER_UNBLOCK_RX_FLAG    ( ( UBaseType_t ) 32 ) to its own variable, to simplify race condition worries.

The other issue is that this comment is still valid and your current solution won't be accepted.

@SoucheSouche xTaskAbortDelay is going to cause a lot of issues IMO, because it does not give a chance for clenup. Memory leaks at minimum, bugs at worse.

@SoucheSouche , did you understand this comment? "Thinking about this again, select on a pair of file descriptors (stdin and eventfd) is probably the cleanest solution." I'm not sure what that solution would look like.
If you use xTaskAbortDelay, make sure the thread is not blocked in user code.

Hi @chipweinberger,
Sorry that I left this PR not updated for so long. I will get back at it this week. It's been a while so it might take a bit of time to get back into it but I will update you as I go through it.
Hopefully we will finally be able to merge it this time.

Hi @chipweinberger, a little update on the PR.
Since it was fairly old, I first had to rebase it on the latest master and realized that your solution was no longer working.
The concept of adding 2 states to the console repl is great and works just fine but the solution that you were using to force linenoise to return didn't seem to work.
I experimented with select as @igrr suggested and ended up finding a solution.
The very drafty logic implemented in linenoiseDumb() and linenoiseRaw() is the following:
[...]
        if (select(in_fd + 1, &read_fds, NULL, &except_fds, NULL) < 0) {
            // select returned with -1, handle the error
        }

        // std_in is either ready to read or an exceptional condition occurred
        if(FD_ISSET(in_fd, &except_fds)) {
            // exceptional condition occurred, return from linenoise back into the while loop
            // of `esp_console_repl_task` where the repl state will be reevaluated
            break;
        }
        else if(FD_ISSET(in_fd, &read_fds)) {
            // do the reading since std_in is ready for reading
        }
        else {
            continue;
        }

I kept uart_unblock_reads and usb_serial_jtag_unblock_reads and I make them fire an "exceptional condition" event that will make the select return and if(FD_ISSET(in_fd, &except_fds)) evaluated to true.

seems like a reasonable solution!
impressive :) fun :)

I had to implement the select for the usb serial jtag driver so I will have to first merge this added feature before proceeding with the actual deadlock fix.
I will keep you posted.

related: https://github.com/espressif/esp-idf/issues/9964

Hi @chipweinberger,
a little update on the fix.
I am still waiting to merge the select() feature for the usb cdc and usb serial jtag drivers before I can merge the fix for the deadlock.
I will let you know when the select() features have been merged but it might take a little while.

hi @chipweinberger,
Sorry I didn't update you for so long. It was quite a journey to implement and merge the select functionality in the usb serial jtag driver and the usb cdc driver.
But great news, I just merged the last one today so I can finally update the bugfix and submit it for review.
I will try to submit it before the end of the week, so I hope it will be merged before Christmas.

Yes I can imagine it was a difficult fix. Congrats!
My product is now shipping: jamcorder.com

Congratulation to you!
The project looks really good! I wish you the best.
And I couldn't help but notice that you are missing the mouse hover on the FAQ section for the desktop version of the website.


When creating a new slot, navigate to it

Right now we don't

Please do, sometimes user might start editing already created ones as it does not navigate to new slot. I have tested that and sometimes seems to be annoying 👯♂  👍

Closed by https://github.com/samtecspg/articulate/commit/8754e8dedad49035de5d6eccd39dced4437f4756


encoding/json: custom unmarshaler is ignored for map

If i use a custom type for string implementing Unmarshaler interface and use it as key type in a map, the unmarshaling seems not working as expected.
package main

import (
	"encoding/json"
	"fmt"
	"strings"
)

type MyString string

func (m *MyString) UnmarshalJSON(data []byte) error {
	var v string
	if err := json.Unmarshal(data, &v); err != nil {
		return err
	}
	*m = MyString(strings.ToLower(v))
	return nil
}

func main() {
	var m MyString
	_ = json.Unmarshal([]byte(`"HELLO WORLD"`), &m)
	fmt.Println(m) // CORRECT output: hello world

	var p map[MyString]string
	_ = json.Unmarshal([]byte(`
{
	"KEY1": "1",
	"KEY2": "2"
}
`), &p)
	fmt.Println(p) // INCORRECT output: map[KEY1:1 KEY2:2] but expected: map[key1:1 key2:2]
}

This sounds related to @mvdan's last comment on a similar issue: https://github.com/golang/go/issues/28189#issuecomment-442566259
In this case, MyString is a string as far as reflect.ValueOf.Kind is concerned:
        var m MyString
	v := reflect.ValueOf(m)
	fmt.Println(v.Kind()) // string

        // Even though MyString implements a custom TextUnmarshaler, the ValueOf.Kind check is used first.
	var textUnmarshalerType = reflect.TypeOf((*encoding.TextUnmarshaler)(nil)).Elem()
	p1 := make(map[MyString]string)
	v2 := reflect.ValueOf(p1)
	fmt.Println(reflect.PtrTo(v2.Type().Key()).Implements(textUnmarshalerType)) // true

See related code here: https://github.com/golang/go/blob/dfbc9c83a910c79cb3cc34dbfaed3c436e1b6ecb/src/encoding/json/decode.go#L633-L637
Because the string key case is checked first, your custom UnmarshalText or UnmarshalJSON on your map key type will not be used.
As a workaround, you can define a custom map type that defines UnmarshalJSON and does the work you want, but it will not be as clean as you expect in your example. I'll let the domain experts take over from here.
/cc @rsc @mvdan @dsnet @bradfitz

Playing around with reflect package. I found:
var m MyString
var x string
fmt.Println(reflect.ValueOf(m).Type().Name()) // returns "MyString"
fmt.Println(reflect.ValueOf(x).Type().Name()) // returns "string"

It could be a bit hacky ...

I'm pretty sure this is working as intended:

To unmarshal a JSON object into a map, Unmarshal first establishes a map to use. If the map is nil, Unmarshal allocates a new map. Otherwise Unmarshal reuses the existing map, keeping existing entries. Unmarshal then stores key-value pairs from the JSON object into the map. The map's key type must either be a string, an integer, or implement encoding.TextUnmarshaler.

Note that it mentions encoding.TextUnmarshaler for map keys, but not json.Unmarshaler. Why do you think your code should be doing something different?

@mvdan thanks for pointing out the doc.
However, it doesn't seem working per the below code.
package main

import (
	"encoding/json"
	"fmt"
	"strings"
)

type MyString string

func (m *MyString) UnmarshalText(text []byte) error {
	fmt.Println("not invoked")
	*m = MyString(strings.ToLower(string(text)))
	return nil
}

func main() {

	var p map[MyString]string
	_ = json.Unmarshal([]byte(`
{
	"KEY1": "1",
	"KEY2": "2"
}
`), &p)
	fmt.Println(p) // INCORRECT output: map[KEY1:1 KEY2:2] but expected: map[key1:1 key2:2]
}

Am i missing anything?

The problem is that Kind can not distinguish between static type and underlying type, in:
var x MyString = "x"
t := reflect.TypeOf(x)

Even x has type MyString, t.Kind() will return string.


Don't include the _BackUpThisFolder_ButDontShipItWithYourGame folder in the artifacts

We have no way to get core dumps from users in any case, so there's no need to save this. Until github's upload-artifacts fixes their slow performance, this should save about 5 minutes in upload/download time.

This works, but I'd like feedback from the more experienced Unity people if you think this is a risk that's not worth taking.

Wait, no, it didn't work. Moving to draft.

there, that's what I wanted to do!


bug: after deleting last video from playlist, playlist should get removed from UI

Steps for producing bug

Create a new playlist
Add any one video to it
Delete that video from the playlists page or remove it from the video details playlist modal
Then playlists page still has grid space for the playlist having that deleted video

This issue is resolved after this commit refactor: playlist delete & update


[BUG] : gif is too large for readme

Describe the bug
I have attached below the screenshot that the gif is too large to fit inside readme. I saw that it's showing good in the website but if it's also inside the readme it'd be good
Expected behavior
The gif should be inside readme.
Screenshots

You have a Problem


Use ::Settings instead of deprecated VMDB::Config.new in MiqServer

Removed deprecated calls to VMDB::Config.new in MiqServer.
@miq-bot add-label wip, core

@miq-bot remove-label wip

Euwe Backport details:
$ git log
commit a1b52a0404b00f8109f6f51ec12b2a197c53029b
Author: Jason Frey<EMAIL_ADDRESS>Date:   Fri Sep 30 22:05:09 2016 -0400

    Merge pull request #11337 from yrudman/use-settings-directly-in-miq-server

    Use ::Settings instead of deprecated VMDB::Config.new in MiqServer
    (cherry picked from commit 77c9b3d7f446cfc1af33741d23c65a9310f0fc11)


One Dimension of Input Layer as None

Hi, I am asking this question again. Please help me out.
I am trying to implement a sentence level attention model for a set of Documents. So the idea is that I generate a sentence embedding using a CNN and then merge the sentence embeddings for a particular document by learning a sentence level attention.
The problem is that each document might have different number of sentences. I have tried padding each document with dummy sentences, but that only increases the training time. Is there a way by which I can have one of the dimensions in the Input Layer as None.
For example,
input_layer = Input(shape=(None,img_h,), dtype='int32', name='input_layer')
embedding_layer = TimeDistributed(Embedding(len(W2V), img_w, input_length=img_h, weights=[W2V], trainable=True, mask_zero=True, name='embedding_1'))(input_layer)
#Apply CNN
#Apply Pooling
#Get Sentence Level Attention Values
#Sum sentence embeddings based on attention
#Apply Dense Layer

So I implemented this and my model summary looks something like this
Layer (type)                     Output Shape          Param #     Connected to
====================================================================================================
input_layer (InputLayer)         (None, None, 82)      0
____________________________________________________________________________________________________
timedistributed_1 (TimeDistribut (None, None, 82, 50)  8034900     input_layer[0][0]
____________________________________________________________________________________________________________
timedistributed_4 (TimeDistribut (None, None, 230, 80, 41630     timedistributed_1[0][0]
____________________________________________________________________________________________________________________________________________
maxpooling2d_1                   (None, None, 690)     0           timedistributed_4[0][0]
____________________________________________________________________________________________________________________________
timedistributed_5 (TimeDistribut (None, None, 1)       477480      maxpooling2d_1[0][0]
____________________________________________________________________________________________________
flattenwithmasksentence_1 (Flatt (None, None)          0           timedistributed_5[0][0]
____________________________________________________________________________________________________
softmaxwithmask_1 (SoftmaxWithMa (None, None)          0           flattenwithmasksentence_1[0][0]
____________________________________________________________________________________________________
repeatvector_1 (RepeatVector)    (None, 690, None)     0           softmaxwithmask_1[0][0]
____________________________________________________________________________________________________
permute_1 (Permute)              (None, None, 690)     0           repeatvector_1[0][0]
____________________________________________________________________________________________________
merge_1 (Merge)                  (None, None, 690)     0           createsentencemask_1[0][0]
                                                                   permute_1[0][0]
____________________________________________________________________________________________________
documentembedding_1 (DocumentEmb (None, 690)           0           merge_1[0][0]
____________________________________________________________________________________________________
dropout_1 (Dropout)              (None, 690)           0           documentembedding_1[0][0]
____________________________________________________________________________________________________
dense_1 (Dense)                  (None, 27)            18657       dropout_1[0][0]
====================================================================================================

But this does not seem to perform as well as the case where I pad the documents with dummy sentences and fix the number of sentences for each document.
Am I doing something wrong?
Please make sure that the boxes below are checked before you submit your issue. If your issue is an implementation question, please ask your question on StackOverflow or join the Keras Slack channel and ask there instead of filing a GitHub issue.
Thank you!

[x] Check that you are up-to-date with the master branch of Keras. You can update with:
pip install git+git://github.com/fchollet/keras.git --upgrade --no-deps

[ ] If running on TensorFlow, check that you are up-to-date with the latest version. The installation instructions can be found here.

[x] If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:
pip install git+git://github.com/Theano/Theano.git --upgrade --no-deps

[x] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

Hello,
I read it a few times but I'm not sure to understand what exactly is your problem.
Your code runs fine :
img_w = 100
img_h = 50

input_layer = Input(shape=(None,img_h,), dtype='int32', name='input_layer')
embedding_layer = TimeDistributed( Embedding(100, img_w, input_length=img_h,name='embedding_1'))(input_layer)
m = Model(input_layer,embedding_layer)
m.predict(np.zeros((40,80,50))).shape
# (40, 80, 50, 100)

Usually if it's possible to compute with None, it will.
Is this just a speed performance issue? Is this a can't make loss converge issue? Is this a I can't get a model to compile issue ?
The half-truncated summary is not very informative but it seems to reveal that you are using some custom layers ("softmaxwithmask","flattenwithmasksentence"). Maybe you have in bug in those.
"If your issue is an implementation question, please ask your question on StackOverflow or join the Keras Slack channel and ask there instead of filing a GitHub issue."


failed to download MPD manifest

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like this: [x])
Use the Preview tab to see what your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2018.12.17. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

[ x] I've verified and I assure that I'm running youtube-dl 2018.12.17

Before submitting an issue make sure you have:

[ x] At least skimmed through the README, most notably the FAQ and BUGS sections
[ x] Searched the bugtracker for similar issues including closed ones
[x ] Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

[x ] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):
youtube-dl  --all-subs --verbose https://www.youtube.com/watch?v=yw23Ks_SEoo
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--all-subs', u'--verbose', u'https://www.youtube.com/watch?v=yw23Ks_SEoo']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.12.17
[debug] Python version 2.7.15rc1 (CPython) - Linux-4.15.0-43-generic-x86_64-with-Ubuntu-18.04-bionic
[debug] exe versions: ffmpeg 2.8.14-0ubuntu<IP_ADDRESS>, ffprobe 2.8.14-0ubuntu<IP_ADDRESS>
[debug] Proxy map: {}
[youtube] yw23Ks_SEoo: Downloading webpage
[youtube] yw23Ks_SEoo: Downloading embed webpage
[youtube] yw23Ks_SEoo: Refetching age-gated info webpage
[youtube] yw23Ks_SEoo: Downloading MPD manifest
WARNING: [youtube] yw23Ks_SEoo: Skipping DASH manifest: ExtractorError(u'Failed to download MPD manifest: HTTP Error 404: Not Found (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.',)
[debug] Default format spec: bestvideo+bestaudio/best
[info] Writing video subtitles to: Mystery Road-yw23Ks_SEoo.en.vtt
[debug] Invoking downloader on u'https://r2---sn-nx5s7n7s.googlevideo.com/videoplayback?ei=n2kkXM2_M42qkwaqxZvgBw&gcr=us&itag=227&keepalive=yes&requiressl=yes&signature=70F1E65D37B226F6976BE7E989DA86E69C61F117.3C224658CA27E68E8017092028E9A1F6D6E5122D&expire=1545911807&key=yt6&initcwndbps=1923750&source=yt_media&mv=m&dur=7281.114&clen=4381113098&c=WEB&lmt=1539631331379704&aid=cb0db72acfd2128a&ipbits=0&gir=yes&mn=sn-nx5s7n7s%2Csn-o097znl6&ip=<IP_ADDRESS>&mm=31%2C26&fvip=5&ms=au%2Conr&asource=youtube&mt=1545890010&sparams=aid%2Caitags%2Casource%2Cclen%2Cdur%2Cei%2Cgcr%2Cgir%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Ckeepalive%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Crequiressl%2Csource%2Cexpire&aitags=142%2C143%2C144%2C145%2C146%2C161%2C222%2C223%2C224%2C225%2C226%2C227%2C273%2C274%2C275%2C279%2C280%2C317%2C318%2C357%2C358%2C359%2C360&pl=16&id=6487bd0d88704312&mime=video%2Fmp4&ratebypass=yes'
[download] Destination: Mystery Road-yw23Ks_SEoo.f227.mp4
[download] 100% of 4.08GiB in 08:28
[debug] Invoking downloader on u'https://r2---sn-nx5s7n7s.googlevideo.com/videoplayback?ei=n2kkXM2_M42qkwaqxZvgBw&gcr=us&itag=329&keepalive=yes&requiressl=yes&signature=11D91CED3B5E9C327C742FB86FD68ED8889C5A60.40D18F1CF17C784ACFCB30299CF2760447F9F4B4&expire=1545911807&key=yt6&initcwndbps=1923750&source=yt_media&mv=m&dur=7281.184&clen=351430577&c=WEB&lmt=1539630860419029&ipbits=0&gir=yes&mn=sn-nx5s7n7s%2Csn-o097znl6&ip=<IP_ADDRESS>&mm=31%2C26&fvip=5&ms=au%2Conr&asource=youtube&mt=1545890010&sparams=aid%2Casource%2Cclen%2Cdur%2Cei%2Cgcr%2Cgir%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Ckeepalive%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Crequiressl%2Csource%2Cexpire&aid=cb0db72acfd2128a&pl=16&id=6487bd0d88704312&mime=audio%2Fmp4&ratebypass=yes'
[download] Destination: Mystery Road-yw23Ks_SEoo.f329.m4a
[download] 100% of 335.15MiB in 00:40
[ffmpeg] Merging formats into "Mystery Road-yw23Ks_SEoo.mp4"
[debug] ffmpeg command line: ffmpeg -y -i 'file:Mystery Road-yw23Ks_SEoo.f227.mp4' -i 'file:Mystery Road-yw23Ks_SEoo.f329.m4a' -c copy -map '0:v:0' -map '1:a:0' 'file:Mystery Road-yw23Ks_SEoo.temp.mp4'
ERROR: Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2048, in post_process
    files_to_delete, info = pp.run(info)
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 479, in run
    self.run_ffmpeg_multiple_files(info['__files_to_merge'], temp_filename, args)
  File "/usr/local/bin/youtube-dl/youtube_dl/postprocessor/ffmpeg.py", line 204, in run_ffmpeg_multiple_files
    raise FFmpegPostProcessorError(msg)
FFmpegPostProcessorError: Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument

...
<end of log>

If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):

Single video: https://www.youtube.com/watch?v=BaW_jenozKc
Single video: https://youtu.be/BaW_jenozKc
Playlist: https://www.youtube.com/playlist?list=PL4lCao7KL_QFVb7Iudeipvc2BCavECqzc

Note that youtube-dl does not support sites dedicated to copyright infringement. In order for site support request to be accepted all provided example URLs should not violate any copyrights.

Description of your issue, suggested solution and other information
This is the first time I have seen this happen.  youtube-dl is an awesome tool and I was surprised to see this.

Outdated ffmpeg.


Refactor stacks to use prototypes

About the PR
Previously stacks used an object to identity their StackType (which was always an enum), now it uses a string id of a stack prototype which is validated on startup and resolved when used.
Also changes the Metal stack to Steel as requested by Zumorica.
Pasta prototype not included.

Now this is epic. Can I get 50 merges

@DrSmugleaf error-reporting with this PR seems to be a little broken.

@DrSmugleaf error-reporting with this PR seems to be a little broken.

That happens when the stack type property is left empty in YAML. I'll add the entity id to the message.


Incomplete info in config causes uncaught exceptions in autoconnect

When opening a fresh instance of the app and inserting connection information in the settings the autoconnect feature tries to already connect and thus causes uncaught exceptions. Should be fixed/tested!

Duplicate of #35


Import error for python 3

On isort==4.2.11, I get this:
File "/Users/weskendall/.pyenv/versions/3.5.3/envs/pgmock/lib/python3.5/site-packages/isort/isort.py", line 43, in <module>
        from .pie_slice import OrderedDict, OrderedSet, input, itemsview
    ImportError: cannot import name 'input'
    error in setup command: Error parsing /Users/weskendall/code/pgmock/setup.cfg: ImportError: cannot import name 'input'

Bumping down to isort==4.2.9 fixes this. It looks like input is not defined in the pie_slice.py file for python3

Anyway this could be patched soon? I can open up a PR if necessary. It's causing so much of our stuff to break

Somehow the wheel file had an outdated version of pie_slice, while the source version had the correct one, I've pushed out 4.2.12, after removing all cached build files, and verified it has the correct version of the file, can you try again?
Thanks!
~Timothy


Reduce chattiness of event dispatching

Moves the event description into the debug level span, omitting debug output at trace level. As a result, we no longer allocate a string each time we process an event, at the cost of getting slightly worse output when an event takes too long to dispatch at info or higher levels.
Note that this changes logging output, causing every event to be fully shown in logs at debug level, unless disabled.
Closes #1838

bors r+


Create OverviewPage for system users

The first version of the OverviewPage now closely approximates the visual appearance of the draft found in Issue #2.

This page is interactive at http://localhost:5173/authfront/ui/auth/overview
given that the app, by way of Vite server, is running in the frontend sub-repo:
See README for instructions on running the app:
https://github.com/Altinn/altinn-authentication-frontend/blob/main/frontend/README.md
### Tasks
- [x] OverviewPage should look similar to draft found in issue-2
- [x] OverviewPage should for demonstration in Week 42 have a link to draft page number 2 (still missing)

The first iteration of OverviewPage now works well visually, and is linked to other pages.


ci: adding testing with M1

What does this PR do?
bump the min mac-12 and adding mac-14 which is M1 machine
ref: https://github.blog/changelog/2024-01-30-github-actions-macos-14-sonoma-is-now-available
if test are passing, we need:

[ ] add new entry to group-check

Before submitting

Was this discussed/agreed via a GitHub issue? (not for typos and docs)
[x] Did you read the contributor guideline, Pull Request section?
[x] Did you make sure your PR does only one thing, instead of bundling different changes together?
Did you make sure to update the documentation with your changes? (if necessary)
Did you write any new necessary tests? (not for typos and docs)
[x] Did you verify new and existing tests pass locally with your changes?
Did you list all the breaking changes introduced by this pull request?
Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review
Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

Reviewer checklist

[x] Is this pull request ready for review? (if not, please submit in draft mode)
[x] Check that all items from Before submitting are resolved
[x] Make sure the title is self-explanatory and the description concisely explains the PR
[x] Add labels and milestones (and optionally projects) to the PR so it can be classified

cc @carmocca @borda

@awaelchli shall we skip these MPS tests for not enough memory or make it smaller?
https://github.com/Lightning-AI/pytorch-lightning/actions/runs/8054227242/job/21998318755?pr=19477#step:14:2668

Those tests don't seem too big. Maybe there's a memory leak making later tests fail. It will need some research


Simulation should fall back to decomposition when unitary isn't available

A user should be able to implement a gate with a _decompose_ into other known gates, and be able to simulate it.
...I actually thought it was working this way already (oops).

I'm not sure I agree with this.
Already we have had a regression because xmon simulator was using KAK and causing it to be very slow.  This means for a complicated gate it will fall back on whatever decompose is when it is faster and better to just implement the unitary.
Decompose itself is also very confusing.  Do we allow passing in a different compose?
It also feels like we are sicking everything in the capsule: why are we putting transformations on the gates into the simulator?

@dabacon even if it's faster and better to actually implement the unitary, I don't think we should require it of the user. Making this change won't make anything slower; it'll just give the user more options for getting a gate to be simulated.
Also, XmonSimulator already does this! I.e., the following code works, but not if you replace XmonSimulator with Simulator
import cirq

class MyGate(cirq.Gate):

    def _decompose_(self, qubits):
        a = qubits[0]
        yield cirq.H(a)
        yield cirq.X(a)

circuit = cirq.Circuit.from_ops(MyGate()(cirq.NamedQubit('a')))

simulator = cirq.google.XmonSimulator()
simulator.simulate(circuit)

why are we putting transformations on the gates into the simulator?

The user said simulate, and it is our job to do our best to get that done. More generally, we want it to be easy to define the gates you want to use and having the simulator natively understand "oh I should try to decompose" gives users an additional implementation option.
That all being said, I think that it does make sense to have a strict "give me what I understand and nothing else" simulator. That would be the xmon simulator.

I agree the xmon simulator shouldn't do it.

I think the problem here is in decompose.  Decompose is  not well defined.  There are infinite number of decompositions, why did you chose one over the other?  It seems completely arbitrary.  Already we see people getting factors of 10 worse because they are using the default decompose on methods in a particular gate setting.  This is completely unexpected for most users.  There are places where decompose is NOT the most efficient decomposition.  People ask: "should I implement unitary or decompose or both?"
I strongly agree simulate(circuit) should just work.  I just worry that it needs to work in a way that doesn't bork users simulation speed by a factor of 10 because they didn't understand that the gate called decompose a bunch of times to actually figure out what was being done.
And this isn't just about simulation: it's about fundamentally not hiding the native gate set for different architectures from the users.  When you play lose with decompose, you encourage people to write code that ends up being impractical on the NISQ architectures.

The 10x slowdown in XmonSimulator is because we forced unnecessary decompositions. In sparse simulator we stop as soon as there's a unitary or apply unitary method. I think this is the right thing to do, with the possible caveat that operations on large numbers of qubits should be further decomposed.
In my mind the intent of decompose is clear, but perhaps not clearly documented. The documentation does say what the intent is:

For example, a Toffoli could be decomposed into Hadamards, CNOTs, and Ts.

These operations are simpler because they act on fewer qubits. In general, a
decomposition should move closer towards the basic 1-qubit and 2-qubit gates
included by default in cirq. It is not necessary to get all the way there,
just closer (callers will iteratively decompose if needed).

i.e. all decompositions must lead to the common two qubit operations, and code doing decomposition is supposed to stop as soon as they see something they can use. This was made easier by the introduction of cirq.decompose with its keep predicate.

So this should really go over in  #930 where I bring up decompose being ill defined.
But a question does arise here about how to fix this "bug".  If we are going to rely on decompose as a fallback for missing unitary, why shouldn't that be part of the unitary contract (i.e. fallback on decompose)

That isn't at all clear, and not mathematically well defined (closer in what metric?)

The distance metric I had in mind was "number of decompositions remaining until you're within the set".

If we are going to rely on decompose as a fallback for missing unitary, why shouldn't that be part of the unitary contract (i.e. fallback to decompose)

I like that idea. We'll need some way to indicate the qubit order...
I would open that as a separate issue to combine these various workarounds into a top-level support decomposing when asking for unitary, has_unitary, and apply_unitary.


Connectivity to external endpoints is lost from nodes with a network interface configuration error

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?
My team deploys Cilium in EKS clusters with IPSec enabled and using the ENI datapath. Occasionally, when many nodes are restarted in succession, we are observe continuous timeouts between pods on a random nodes and endpoints external to the cluster, specifically the control plane (kube-apiserver) and the Route 53 Resolver.
We have made a strong correlation between these timeouts to an error log from the cilium-agent on that node: level=error msg="Skipping invalid ENI device config" error="failed to parse eni subnet cidr \"\": invalid CIDR address: " resource=eni-06f8eb5cfc1c69457 subsys=ipam.
When we exec onto a cilium-agent on an impacted node, we see some network interfaces have an IP CIDR, but one does not. In the screenshot below, you'll see that eth0 and eth1 are setup properly, but eth2 is not. We have confirmed only the pods associated with the bad network interface experience the control plane timeouts.
When we restart the cilium-agent, the network interface is configured correctly and the timeouts are resolved.
Cilium Version
v1.11.14
Kernel Version
5.15.117-73.143.amzn2.x86_64
Kubernetes Version
v1.24.14-eks-c12679a
Sysdump
No response
Relevant log output
2023-06-20 17:57:54.640 ip-10-46-5-75.ap-south-1.compute.internal cilium-pk8qp level=info msg="Received own node information from API server" ipAddr.ipv4=<IP_ADDRESS> ipAddr.ipv6="<nil>" k8sNodeIP=<IP_ADDRESS> labels="map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:m5.2xlarge beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-south-1 failure-domain.beta.kubernetes.io/zone:ap-south-1c k8s.appian.com/lifecycle:on-demand k8s.io/cloud-provider-aws:b6c74e5459d960a7879231e8c47281cf kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-10-46-5-75.ap-south-1.compute.internal kubernetes.io/os:linux node.kubernetes.io/instance-type:m5.2xlarge topology.kubernetes.io/region:ap-south-1 topology.kubernetes.io/zone:ap-south-1c]" nodeName=ip-10-46-5-75.ap-south-1.compute.internal subsys=k8s v4Prefix="<nil>" v6Prefix="<nil>"
2023-06-20 17:57:54.667	ip-10-46-6-152.ap-south-1.compute.internal cilium-operator-6d66cbdd98-skzb2 level=info msg="Resolving IP deficit of node" available=0 availableForAllocation=0 availableInterfaces=3 instanceID=i-0e8e3829a594d658d maxIPsToAllocate=8 name=ip-10-46-5-75.ap-south-1.compute.internal neededIPs=8 remainingInterfaces=3 selectedInterface= selectedPoolID= subsys=ipam used=0
2023-06-20 17:57:54.667	ip-10-46-6-152.ap-south-1.compute.internal cilium-operator-6d66cbdd98-skzb2 level=info msg="No more IPs available, creating new ENI" addresses=8 instanceID=i-0e8e3829a594d658d name=ip-10-46-5-75.ap-south-1.compute.internal securityGroupIDs="[sg-03fdc915a467849cf]" subnetID=subnet-01df502e3f5e527ea subsys=ipam
2023-06-20 17:57:54.767 ip-10-46-5-75.ap-south-1.compute.internal cilium-pk8qp level=info msg="Waiting for IPs to become available in CRD-backed allocation pool" available=0 helpMessage="Check if cilium-operator pod is running and does not have any warnings or error messages." name=ip-10-46-5-75.ap-south-1.compute.internal required=1 subsys=ipam
2023-06-20 17:57:55.065	ip-10-46-6-152.ap-south-1.compute.internal cilium-operator-6d66cbdd98-skzb2 level=info msg="Created new ENI" addresses=8 eniID=eni-007b185d134f21a2d instanceID=i-0e8e3829a594d658d name=ip-10-46-5-75.ap-south-1.compute.internal securityGroupIDs="[sg-03fdc915a467849cf]" subnetID=subnet-01df502e3f5e527ea subsys=ipam
2023-06-20 17:57:55.874	ip-10-46-6-152.ap-south-1.compute.internal cilium-operator-6d66cbdd98-skzb2 level=info msg="Attached ENI to instance" addresses=8 attachmentID=eni-attach-064898490eaf52ad0 eniID=eni-007b185d134f21a2d index=1 instanceID=i-0e8e3829a594d658d name=ip-10-46-5-75.ap-south-1.compute.internal securityGroupIDs="[sg-03fdc915a467849cf]" subnetID=subnet-01df502e3f5e527ea subsys=ipam
2023-06-20 17:57:59.769 ip-10-46-5-75.ap-south-1.compute.internal cilium-pk8qp level=info msg="Endpoints restored" failed=0 restored=0 subsys=daemon
2023-06-20 17:57:59.769 ip-10-46-5-75.ap-south-1.compute.internal cilium-pk8qp level=info msg="Adding local node to cluster" node="{ip-10-46-5-75.ap-south-1.compute.internal default [{InternalIP <IP_ADDRESS>} {CiliumInternalIP <IP_ADDRESS>}] <IP_ADDRESS>/16 <nil> <nil> <nil> 0 local 4 map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:m5.2xlarge beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:ap-south-1 failure-domain.beta.kubernetes.io/zone:ap-south-1c k8s.appian.com/lifecycle:on-demand k8s.io/cloud-provider-aws:b6c74e5459d960a7879231e8c47281cf kubernetes.io/arch:amd64 kubernetes.io/hostname:ip-10-46-5-75.ap-south-1.compute.internal kubernetes.io/os:linux node.kubernetes.io/instance-type:m5.2xlarge topology.kubernetes.io/region:ap-south-1 topology.kubernetes.io/zone:ap-south-1c] 1 }" subsys=nodediscovery
2023-06-20 17:57:59.814	ip-10-46-6-152.ap-south-1.compute.internal cilium-operator-6d66cbdd98-skzb2 level=info msg="Resolving IP deficit of node" available=8 availableForAllocation=6 availableInterfaces=3 instanceID=i-0e8e3829a594d658d maxIPsToAllocate=1 name=ip-10-46-5-75.ap-south-1.compute.internal neededIPs=1 remainingInterfaces=3 selectedInterface=eni-007b185d134f21a2d selectedPoolID=subnet-01df502e3f5e527ea subsys=ipam used=1
2023-06-20 17:58:00.554 ip-10-46-5-75.ap-south-1.compute.internal cilium-pk8qp level=info msg="Migration of ENI datapath successful, 0 endpoints were migrated and none failed." subsys=daemon
2023-06-20 17:58:14.811	ip-10-46-4-171.ap-south-1.compute.internal cilium-operator-6d66cbdd98-wph2h level=info msg="Resolving IP deficit of node" available=9 availableForAllocation=5 availableInterfaces=3 instanceID=i-0e8e3829a594d658d maxIPsToAllocate=8 name=ip-10-46-5-75.ap-south-1.compute.internal neededIPs=8 remainingInterfaces=3 selectedInterface=eni-007b185d134f21a2d selectedPoolID=subnet-01df502e3f5e527ea subsys=ipam used=9
2023-06-20 17:58:14.846 ip-10-46-4-171.ap-south-1.compute.internal cilium-operator-6d66cbdd98-wph2h level=warning msg="Failed to update CiliumNode" attempt=0 error="Operation cannot be fulfilled on ciliumnodes.cilium.io \"ip-10-46-5-75.ap-south-1.compute.internal\": the object has been modified; please apply your changes to the latest version and try again" instanceID=i-0e8e3829a594d658d name=ip-10-46-5-75.ap-south-1.compute.internal subsys=ipam updateStatus=true
2023-06-20 17:58:15.623	ip-10-46-4-171.ap-south-1.compute.internal cilium-operator-6d66cbdd98-wph2h level=info msg="Resolving IP deficit of node" available=14 availableForAllocation=0 availableInterfaces=2 instanceID=i-0e8e3829a594d658d maxIPsToAllocate=3 name=ip-10-46-5-75.ap-south-1.compute.internal neededIPs=3 remainingInterfaces=2 selectedInterface= selectedPoolID= subsys=ipam used=9
2023-06-20 17:58:15.623	ip-10-46-4-171.ap-south-1.compute.internal cilium-operator-6d66cbdd98-wph2h level=info msg="No more IPs available, creating new ENI" addresses=3 instanceID=i-0e8e3829a594d658d name=ip-10-46-5-75.ap-south-1.compute.internal securityGroupIDs="[sg-03fdc915a467849cf]" subnetID=subnet-01df502e3f5e527ea subsys=ipam
2023-06-20 17:58:15.648	ip-10-46-4-171.ap-south-1.compute.internal cilium-operator-6d66cbdd98-wph2h level=warning msg="Failed to update CiliumNode" attempt=0 error="Operation cannot be fulfilled on ciliumnodes.cilium.io \"ip-10-46-5-75.ap-south-1.compute.internal\": the object has been modified; please apply your changes to the latest version and try again" instanceID=i-0e8e3829a594d658d name=ip-10-46-5-75.ap-south-1.compute.internal subsys=ipam updateStatus=true
2023-06-20 17:58:15.981	ip-10-46-4-171.ap-south-1.compute.internal cilium-operator-6d66cbdd98-wph2h level=info msg="Created new ENI" addresses=3 eniID=eni-06f8eb5cfc1c69457 instanceID=i-0e8e3829a594d658d name=ip-10-46-5-75.ap-south-1.compute.internal securityGroupIDs="[sg-03fdc915a467849cf]" subnetID=subnet-01df502e3f5e527ea subsys=ipam
2023-06-20 17:58:16.667	ip-10-46-4-171.ap-south-1.compute.internal cilium-operator-6d66cbdd98-wph2h level=info msg="Attached ENI to instance" addresses=3 attachmentID=eni-attach-024dfa55623394ce5 eniID=eni-06f8eb5cfc1c69457 index=2 instanceID=i-0e8e3829a594d658d name=ip-10-46-5-75.ap-south-1.compute.internal securityGroupIDs="[sg-03fdc915a467849cf]" subnetID=subnet-01df502e3f5e527ea subsys=ipam
2023-06-20 17:58:16.853	ip-10-46-4-171.ap-south-1.compute.internal cilium-operator-6d66cbdd98-wph2h level=warning msg="Failed to update CiliumNode" attempt=0 error="Operation cannot be fulfilled on ciliumnodes.cilium.io \"ip-10-46-5-75.ap-south-1.compute.internal\": the object has been modified; please apply your changes to the latest version and try again" instanceID=i-0e8e3829a594d658d name=ip-10-46-5-75.ap-south-1.compute.internal subsys=ipam updateStatus=true
2023-06-20 17:58:17.253 ip-10-46-5-75.ap-south-1.compute.internal cilium-pk8qp level=error msg="Skipping invalid ENI device config" error="failed to parse eni subnet cidr \"\": invalid CIDR address: " resource=eni-06f8eb5cfc1c69457 subsys=ipam

Anything else?

This issue seems related to #25518, however we have confirmed that our nodes have net.ipv4.conf.all.rp_filter = 0 set.
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Thanks for the report! Definitely looks like something we should look in to.
Would you be able to capture a sysdump on a cluster affected by this? Thanks!

Sure, I'll work on getting a sysdump.

@squeed I can work on getting a sysdump, but there is a lot of sensitive information that needs to be redacted so it will take some time. Is there any particular information you're looking for?

Attached is the sysdump.
cilium-sysdump-20230621-102241-redacted.zip


OCM-2645: Recreate cluster if it was deleted from other client

This PR adds the ability to reconcile a cluster that was created by TF, but then was removed via a different method, Not via terraform.
In this case, And if the user still has the cluster resource in his .tf files. Upon an apply, the provider will re-create the cluster.
Before this PR, the provider would have error out, since it could not find the cluster defined in the state.
Note: This PR also modifies the cluster_waiter. This is because we now do need to update the resource if the cluster id has changed. Also moved the timeout validation into a validator this will now throw an error if the user has set the timeout to a negative number. Unlike it worked before, printing a warning and using the default value.
The polling (cluster_waiter) was also moved to a new function, so both Create and Update can call.

/test managed-oidc-config-e2e-presubmit

/approve
/retest

/lgtm