Fix ESLint issues (Warnings) - react-hooks/exhaustive-deps

Running npm run lint in the command line shows various ESLint issues that we should fix. The issues are split into warnings (yellow-coded) and errors (red-coded). Within warnings and errors, the issues are split into categories by ESLint rule name.
Example:

Fix all warnings under the category react-hooks/exhaustive-deps. Please make sure to regression test so that we know nothing has broken through these changes.
List of issues (May not be exhaustive so double check):

[ ] wtt_front/client/src/components/TreeUtil/TreeNameAndSize.js

[ ]     49:34  warning  Assignments to the 'scientific' variable from inside React Hook useEffect will be lost after each render. To preserve the value over time, store it in a useRef Hook and keep the mutable value in the '.current' property. Otherwise, you can move this variable directly inside useEffect  react-hooks/exhaustive-deps
[ ]     49:34  warning  Assignments to the 'genus' variable from inside React Hook useEffect will be lost after each render. To preserve the value over time, store it in a useRef Hook and keep the mutable value in the '.current' property. Otherwise, you can move this variable directly inside useEffect       react-hooks/exhaustive-deps

[ ] wtt_front/client/src/pages/Map/Map.js

[ ]     100:6  warning  React Hook useEffect has a missing dependency: 'map'. Either include it or remove the dependency array                                                                                react-hooks/exhaustive-deps
[ ]     245:6  warning  React Hook useEffect has missing dependencies: 'currentTreeDb', 'onLoad', 'queryClient', 'setCurrentTreeDataVector', and 'setCurrentTreeId'. Either include them or remove the dependency array. If 'onLoad' changes too often, find the parent component that defines it and wrap that definition in useCallback  react-hooks/exhaustive-deps

[ ] wtt_front/client/src/pages/Map/MapLayers.js

[ ]     44:6  warning  React Hook useEffect has a missing dependency: 'map'. Either include it or remove the dependency array  react-hooks/exhaustive-deps
[ ] wtt_front/client/src/pages/Map/MapLayout.js
[ ]     146:6  warning  React Hook useEffect has a missing dependency: 'currentTreeDataVector'. Either include it or remove the dependency array                                                                                react-hooks/exhaustive-deps
[ ]     155:6  warning  React Hook useEffect has a missing dependency: 'currentTreeDataVector'. Either include it or remove the dependency array. You can also replace multiple useState variables with useReducer if 'setCurrentTreeData' needs the current value of 'currentTreeDataVector'  react-hooks/exhaustive-deps

[ ] wtt_front/client/src/pages/Map/MapboxControlPortal.js

[ ]     49:6  warning  React Hook useEffect has missing dependencies: 'container' and 'position'. Either include them or remove the dependency array  react-hooks/exhaustive-deps

[ ] wtt_front/client/src/pages/Map/MapboxMarkerPortal.js

[ ]     63:6  warning  React Hook useEffect has missing dependencies: 'className', 'eventHandlers', 'map', and 'options'. Either include them or remove the dependency array  react-hooks/exhaustive-deps
[ ]     81:6  warning  React Hook useEffect has missing dependencies: 'addedToMap' and 'map'. Either include them or remove the dependency array                              react-hooks/exhaustive-deps

[ ] wtt_front/client/src/pages/Map/TreeCountLayer.js

[ ]     69:6  warning  React Hook useEffect has missing dependencies: 'flyToZoom', 'id', 'maxzoom', and 'minzoom'. Either include them or remove the dependency array  react-hooks/exhaustive-deps

[ ] wtt_front/client/src/pages/Map/TreeLayer.js

[ ]     33:6  warning  React Hook useEffect has missing dependencies: 'id', 'layer', and 'on'. Either include them or remove the dependency array  react-hooks/exhaustive-deps
[ ]     40:6  warning  React Hook useEffect has a missing dependency: 'id'. Either include it or remove the dependency array                       react-hooks/exhaustive-deps

[ ] wtt_front/client/src/pages/Map/TreeLayerLegend.js

[ ]     111:6  warning  React Hook useEffect has a missing dependency: 'layers'. Either include it or remove the dependency array              react-hooks/exhaustive-deps
[ ]     129:6  warning  React Hook useEffect has missing dependencies: 'layers' and 'map'. Either include them or remove the dependency array  react-hooks/exhaustive-deps

[ ] wtt_front/client/src/pages/NewTree/NewTreeButton.js

[ ]     60:6  warning  React Hook useEffect has missing dependencies: 'endPlanting' and 'mutateTreeData'. Either include them or remove the dependency array  react-hooks/exhaustive-deps

[ ] wtt_front/client/src/pages/NewTree/NewTreePlantingMarker.js

[ ]     55:6   warning  React Hook useEffect has missing dependencies: 'coords', 'isFollowingUser', and 'setCoords'. Either include them or remove the dependency array  react-hooks/exhaustive-deps
[ ]     64:6   warning  React Hook useEffect has missing dependencies: 'isPlanting' and 'setCoords'. Either include them or remove the dependency array                  react-hooks/exhaustive-deps

[ ] wtt_front/client/src/pages/Tree/TreeNotes.js

[ ]     43:6  warning  React Hook useEffect has a missing dependency: 'saveNotes'. Either include it or remove the dependency array. Mutable values like 'notesRef.current' aren't valid dependencies because mutating them doesn't re-render the component  react-hooks/exhaustive-deps

[ ] wtt_front/client/src/pages/UserLocation/useGeolocation.js

[ ]     68:6  warning  React Hook useEffect has missing dependencies: 'available' and 'geoOptions'. Either include them or remove the dependency array  react-hooks/exhaustive-deps

[ ] wtt_front/client/src/pages/UserLocation/useUserLocation.js

[ ]     73:6  warning  React Hook useEffect has missing dependencies: 'context' and 'state.coords'. Either include them or remove the dependency array  react-hooks/exhaustive-deps

Hi @jazhen, assign me this issue, please.


Difficult to use the presence of an optional dependency to use an optional dependency in a non-optional dependency

Phew!  That issue title was a mouthful.  So for context see this PR: https://github.com/kvark/collision-rs/pull/38
We want to export a feature called "serde" and only turn on the "serde" feature in cgmath if our "serde" feature is enabled.  We were able to accomplish this with a few workarounds, but it was fairly non-obvious how to do what might become a common task.
In addition to this it was also discovered that proc-macro crate dependencies are silently ignored if placed under a dependency section using target.'cfg(...)'.dependencies I imagine this is due to needing the macro system to evaluate the cfg statement, but it's still annoying and behaves unexpectedly.

$ cargo --version
cargo 0.21.0 (5b4b8b2ae 2017-08-12)

Something that might help this situation would be having the ability to specify additional feature lists like ["serde", "serde_derive", "cgmath/serde"] with an optional dependency, and not just as part of a feature not attached to a dependency.
Alternatively: Permit a feature named serde to exist alongside an optional dependency called serde if and only if that feature also has "serde" in its feature list so that the optional serde crate is used.  (That's kind of a complicated ruleset so I'm not sure if I like this idea)

proc-macro crate dependencies are silently ignored

I wanted to provide a few more details on this.  So with this project structure
collision-rs.zip. The following commands give the following output (rust stable as default with rust nighty installed as well)
$ cargo check
error: failed to learn about crate-type `proc-macro` early on

$ cargo +nightly check
    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs

$ cargo +nightly check --features="serde"
   Compiling collision v0.11.0 (file:///C:/Users/JKiesel.CISDEV/collision-rs)
error[E0463]: can't find crate for `serde_derive`
  --> src\lib.rs:46:1
   |
46 | extern crate serde_derive;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^ can't find crate

error: aborting due to previous error

error: Could not compile `collision`.

To learn more, run the command again with --verbose.

Thanks for the report! @Xaeroxe would this perhaps be solved by https://github.com/rust-lang/cargo/issues/1286?

@alexcrichton Yes, that RFC sounds great, but doesn't it break backwards compatibility?

@Xaeroxe ok! In that case I'm going to close this in favor of that issue.
Strictly interpreted it does indeed break backwards compatibility, but there's a number of possible solutions we could employ to avoid the breaking change.


MVC TOC

Clean up MVC TOC
Review URL

@tdykstra should I schedule a Skype meeting? Sorry about my sloppy work on this on.


AEUX plugin doesn't work on After effect cc 2022

Hello, AEUX doesn't work on After effect cc 2022. I'm using the latest version of the plugin on Windows 10.
Does anyone solve this?

Same issue here

Same here. I click "Send selection to Ae" in Sketch and nothing happens.

I have the same problem
MacBook M1 Pro MacOS 12.4
AE 22.1.1
AEUX 0.8.1-B

I'm having the same problem... any updates?

Same issue!! Help! When to update?

For me it works again with the most recent AE update.

Still not working for me

I had the same issue. I tried multiple ways to get rid of this but nothing worked. Finally, it started working when I stop syncing the dropbox app on my MacBook. If you guys have a dropbox app, quit the app and then try opening AEUX in after effects. Hope this will work.


feat: sort comments in explorer by selected location

Solves GH-121
PR Checklist
Please check if your PR fulfills the following requirements:

[X] Tests for the changes have been added (for bug fixes / features)
[X] Docs have been added / updated (for bug fixes / features)

PR Type
What kind of change does this PR introduce?

[ ] Bugfix
[X] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Other... Please describe:

What is the current behavior?
See GH-121
What is the new behavior?
Sorts comments in the comment explorer ascending by location information.
Does this PR introduce a breaking change?

[ ] Yes
[X] No

Other information

Ouch. This could have been a bit easier if I would have seen #38 before 🙈
My changes so only bring this feature to the comment explorer. My underlying believe, however, still holds true: the necessary to split the CSV model from the internal data model (cf. #123).


EXPLAIN IMPUTATION PLAN

From @axch: Crosscat should be able to explain why it made an imputation choice it did, phrased as:

What are the columns highly dependent on this column (i.e., in the same view)?
What are the rows that are similar to this row with respect to those columns (i.e., in the same cluster)?
What are the sufficient statistics of the column over the similar rows (i.e., cluster suffstats)?

This can be done exactly for a single model; explaining an integral over all models might be a little more difficult to visualize.  No amputations, please.

This echoes #77, and I'd love to have this soon. It might be a candidate for implementation in recipes because those questions are answerable generically, not only in a crosscat-specific way.
Also, should this particular version of the bug be on crosscat's tracker? If so, perhaps the issue here is to plan an interface by which particular models can tell bayeslite about these imputation choices?

Because of #194 , we might want to accelerate this or something like it into a polish release if it's possible.


incorrect number of columns for @ key

this code for example: chain[chain.data_key][chain.filter][downbreak]['@'][v]
returns a df with multiple columns instead of one

Resolved by #248.


Show timeline of called_process

In the while zone, tasks repeat several times, however, in the timeline it only shows the time for the last iteration of the task. As show below, there is some missing info between add1 and add2.

This PR adds the called_process option. Switching to this option, it will show the timeline for all the process that called by the WorkGraph:

Codecov Report
Attention: Patch coverage is 77.27273% with 5 lines in your changes missing coverage. Please review.

Project coverage is 79.24%. Comparing base (5937b88) to head (15069d6).
Report is 34 commits behind head on main.

Files
Patch %
Lines

aiida_workgraph/utils/__init__.py
77.27%
5 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #238      +/-   ##
==========================================
+ Coverage   75.75%   79.24%   +3.48%
==========================================
  Files          70       62       -8
  Lines        4615     4567      -48
==========================================
+ Hits         3496     3619     +123
+ Misses       1119      948     -171

Flag
Coverage Δ

python-3.11
79.24% <77.27%> (+3.57%)
:arrow_up:

python-3.9
?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Hi @agoscinski , the frontend test caught a bug in this PR. https://github.com/aiidateam/aiida-workgraph/actions/runs/10441999297/job/28914107364
Very helpful!


Add files via upload

https://github.com/JetBrains/swot/pull/25149

@Couri3r There are several files in the request, along with file.zip. It must not be there. Please remove file.zip and make pull request again.  I have to close this request without merging. Thank you for your understanding.


Added pages to footer section

Hi @Priyaaa1
This is to close the issue #154. I have added vision and mission, blogs, Terms and Conditions and Privacy policy page with and routed each of it appropriately. Please review it and let me know if any additional changes are required.

@Devaki01 kindly resolve conflicts

@Devaki01 kindly resolve conflicts

Ok ..will try and sort it out


Set IsAdmin using LDAP

The IsAdmin flag is set based on whether the admin filter
returned any result. The admin filter is applied with the user dn
as the search root.
In the future, we should update IsAdmin as well on each login.
Alternately, we can have a periodic sync operation.

Thanks your PR!

In the future, we should update IsAdmin as well on each login.

It might not be ideal solution, because I think Gogs does not talk to LDAP server always, only when user session is expired, or preform operations that need password validation.

Looks all good.


TEST

Codecov Report

Merging #6 (582c5ad) into master (3d12ef0) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master       #6   +/-   ##
=======================================
  Coverage   41.37%   41.37%
=======================================
  Files          11       11
  Lines         261      261
=======================================
  Hits          108      108
  Misses        153      153

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


Alert rules provided by the cos-agent relation generate false positives

On my microk8s cluster consisting of 3 units, after relating microk8s charm to grafana-agent using both juju-info and cos-agent relation I ended up with 4 active alerts: KubeAPIDown, KubeControllerManagerDown, KubeletDown, KubeSchedulerDown.
The root cause seems to be that these alerts rely on the presence of juju_charm label which is missing in my env:
One of the alert rules:
absent(up{job="apiserver",juju_application="microk8s",juju_charm="grafana-agent",juju_model="microk8s",juju_model_uuid="57280f89-7c62-4703-8622-02de020641d2"} == 1)

count(up{job="apiserver",juju_application="microk8s",juju_charm="grafana-agent",juju_model="microk8s",juju_model_uuid="57280f89-7c62-4703-8622-02de020641d2"})

Empty query result

versus (without juju_charm label in the query the result is 3 as expected)
count(up{job="apiserver",juju_application="microk8s",juju_model="microk8s",juju_model_uuid="57280f89-7c62-4703-8622-02de020641d2"})

{} 3

The microk8s cluster itself is healthy, and all services are running.
Another alert-related problem I discovered is that client cert expiration alerts fire a bit too close to the actual expiration date. Entering the critical state only 24h before the expiration might be a bit challenging in real-life scenarios for the cluster administrator.
Versions:

juju 2.9.32
microk8s charm: latest/edge, rev 115
microk8s snap: v1.28.0 5788 1.28/stable
grafana-agent charm: rev 12, latest/candidate

Hi @przemeklal, thank you for raising the issue. This is well-known and happens because of https://github.com/canonical/grafana-agent-k8s-operator/issues/190
This was also noted in the PR https://github.com/canonical/charm-microk8s/pull/88#issuecomment-1639512955
Unfortunately, the bug occurs inside grafana-agent and therefore there is nothing we can do from the side of the microk8s charm to deal with. In fact, for our testing, we had to resolve to custom builds of the grafana-agent charm.
Not sure what the timeline would be to get a fix for this, cc @dstathis should know more about this.

We had a few things pull our attention away. But that issue is now being actively worked on.


adds styled text molds to ++dill-blit and ++sole-effect

This PR implements the molds and transformations discussed in #212...
It's not ready to be merged -- right now it styled dill-blits are slogged, and then output with the styles stripped. I opened it now so the molds and overall approach could be reviewed early.
questions:

should the core in drum:ta-klr be moved somewhere else? A "klr-engine" in zuse?
right now, lines are hard-wrapped at the terminal width -- should I be wrapping at the previous word boundary? (I haven't taken the time to deconstruct ++re yet ...)

I'll get started on the actual color printing in term.c now ...
Examples:
=/  klr/styx  ~[[```%r "foo"] [```%g "bar"] [```%b "baz"]]
[[ost %diff %sole-effect %klr klr]~ +>.$]

dill slogs
[[p={} q=[p=~ q=%r]] "foo"]
[[p={} q=[p=~ q=%g]] "bar"]
[[p={} q=[p=~ q=%b]] "baz"]

and then outputs foobarbaz.
And, with inheritance:
=/  klr/styx
      :~  'foo bar baz bud boogity '
          [```%r "woogity "]
          [```%g ~['doogity ' [[~ `%k ~] "foogity "]]]
          [[`%un ~ ~] "zooooooooooooooooooooooooooogity dobity-dap "]
          [[`%br `%w `%m] ~['beepbopboop']]
      ==

slogs:
[[p={} q=[p=~ q=~]] "foo bar baz bud boogity "]
[[p={} q=[p=~ q=%r]] "woogity "]
[[p={} q=[p=~ q=%g]] "doogity "]
[[p={} q=[p=%k q=%g]] "foogity "]
[[p={%un} q=[p=~ q=~]] "zooooooooooooooooooooooooooogity dobity-dap "]
[[p={%br} q=[p=%w q=%m]] "beepbopboop"]

and outputs
foo bar baz bud boogity woogity doogity foogity zooooooooooooooooooooooooooogity dobity-dap beepbopboop

Wrapping at word boundaries kind of pleases nobody -- it's hard to define
"right."  I would just send long lines all the way down to the terminal,
and let the terminal wrap.
You might farm it out into a library, but wait until it works and is
stable.  Also, you haven't addressed colored text in the prompt, right?
On Tue, Jul 26, 2016 at 2:15 PM, Joe Bryan<EMAIL_ADDRESS>wrote:

This PR implements the molds and transformations discussed in #212
https://github.com/urbit/arvo/issues/212...
It's not ready to be merged -- right now it styled dill-blits are slogged,
and then output with the styles stripped. I opened it now so the molds and
overall approach could be reviewed early.
questions:

should the core in drum:ta-klr be moved somewhere else? A
"klr-engine" in zuse?
right now, lines are hard-wrapped at the terminal width -- should I
be wrapping at the previous word boundary? (I haven't taken the time to
deconstruct ++re yet ...)

I'll get started on the actual color printing in term.c now ...
Examples:
=/  klr/styx  ~[[```%r "foo"] [```%g "bar"] [```%b "baz"]]
[[ost %diff %sole-effect %klr klr]~ +>.$]
dill slogs
[[p={} q=[p=~ q=%r]] "foo"]
[[p={} q=[p=~ q=%g]] "bar"]
[[p={} q=[p=~ q=%b]] "baz"]
and then outputs foobarbaz.
And, with inheritance:
=/  klr/styx
:~  'foo bar baz bud boogity '
[```%r "woogity "]
[```%g ~['doogity ' [[~ `%k ~] "foogity "]]]
[[`%un ~ ~] "zooooooooooooooooooooooooooogity dobity-dap "]
[[`%br `%w `%m] ~['beepbopboop']]
==
slogs:
[[p={} q=[p=~ q=~]] "foo bar baz bud boogity "]
[[p={} q=[p=~ q=%r]] "woogity "]
[[p={} q=[p=~ q=%g]] "doogity "]
[[p={} q=[p=%k q=%g]] "foogity "]
[[p={%un} q=[p=~ q=~]] "zooooooooooooooooooooooooooogity dobity-dap "]
[[p={%br} q=[p=%w q=%m]] "beepbopboop"]
and outputs
foo bar baz bud boogity woogity doogity foogity zooooooooooooooooooooooooooogity dobity-dap beepbopboop

You can view, comment on, or merge this pull request online at:
https://github.com/urbit/arvo/pull/216
Commit Summary

adds styled text molds to ++dill-blit and ++sole-effect

File Changes

M arvo/dill.hoon
https://github.com/urbit/arvo/pull/216/files#diff-0 (13)
M arvo/zuse.hoon
https://github.com/urbit/arvo/pull/216/files#diff-1 (8)
M lib/drum.hoon
https://github.com/urbit/arvo/pull/216/files#diff-2 (76)
M sur/sole.hoon
https://github.com/urbit/arvo/pull/216/files#diff-3 (1)

Patch Links:

https://github.com/urbit/arvo/pull/216.patch
https://github.com/urbit/arvo/pull/216.diff

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/urbit/arvo/pull/216, or mute the thread
https://github.com/notifications/unsubscribe-auth/AALyAR1PuF67YPzq73J1-QNHjc8IDvAmks5qZnkNgaJpZM4JVl_E
.

I don't see any line-wrapping in term.c -- I see it in drum (++ta-tan and ++se-dump) and dill (++crud). Something like  (~(win re a) [0 edg]) ...
That's just for sole-effect and dill-blit, maybe ++slog is wrapped in the interpreter? (u3t_slog() calls u3_lo_tank(), but I don't know what that does ...)

And yet, I just tested, and lines are being wrapped in term.c ... @should test these things first

Kids these days!  Put a long line in a file and cat it -- Unix' s ancient technology will wrap the line itself. The 80s called, they want their vt100 back...
Sent from my iPhone

On Jul 26, 2016, at 3:58 PM, Joe Bryan<EMAIL_ADDRESS>wrote:
And yet, I just tested, and lines are being wrapped in term.c ... @should test these things first
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

Yeah, but / inverted roof notifications / are all us...

On Jul 26, 2016, at 4:25 PM, cgyarvin<EMAIL_ADDRESS>wrote:
Kids these days! Put a long line in a file and cat it -- Unix' s ancient technology will wrap the line itself. The 80s called, they want their vt100 back...
Sent from my iPhone

On Jul 26, 2016, at 3:58 PM, Joe Bryan<EMAIL_ADDRESS>wrote:
And yet, I just tested, and lines are being wrapped in term.c ... @should test these things first
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://github.com/urbit/arvo/pull/216#issuecomment-235436732, or mute the thread https://github.com/notifications/unsubscribe-auth/AAHENUQ1jIenztvuF9TiuDYMzlHHYTl7ks5qZpeHgaJpZM4JVl_E.

I could try to offer an excuse, but ... let's just agree to never speak of this moment again.

That's true.  I see the Jedi mind trick here...
On Tue, Jul 26, 2016 at 4:27 PM, Raymond Pasco<EMAIL_ADDRESS>wrote:

Yeah, but / inverted roof notifications / are all us...

On Jul 26, 2016, at 4:25 PM, cgyarvin<EMAIL_ADDRESS>wrote:
Kids these days! Put a long line in a file and cat it -- Unix' s ancient
technology will wrap the line itself. The 80s called, they want their
vt100 back...
Sent from my iPhone

On Jul 26, 2016, at 3:58 PM, Joe Bryan<EMAIL_ADDRESS>wrote:
And yet, I just tested, and lines are being wrapped in term.c ...
@should test these things first
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub <
https://github.com/urbit/arvo/pull/216#issuecomment-235436732>, or mute
the thread <
https://github.com/notifications/unsubscribe-auth/AAHENUQ1jIenztvuF9TiuDYMzlHHYTl7ks5qZpeHgaJpZM4JVl_E
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/urbit/arvo/pull/216#issuecomment-235436943, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AALyAVuDPNzgRBZFrkrmnJVkMC0FtckHks5qZpfNgaJpZM4JVl_E
.

I tried including the ANSI escape sequences directly in the blit'd (list @c), but it only works for decorations. Foreground/background colors aren't rendered at all, and the %br decoration (\033[49m) sticks and can't be reset.
I'm not yet sure what's causing this, but we'll clearly need changes in term.c one way or the other...

I've done some more thorough testing:
All the decorations work: %bl blinks, %un underscores, %br renders as red foreground text. Decorations also compose (although I had to un/re-focus my terminal window to get %un and %bl to compose, which made me briefly question my sanity. Again.)
Foreground colors have no effect -- on the marked-up text or any subsequent output. Background colors have no immediate effect, but all set subsequent output to be bright (ie, %br, rendering as red foreground).
This is enough to satisfy the basic requirements (blinking, red, underlined syntax errors). That could be done by either a) keeping the current types, but ignoring all foreground/background colors, or b) limiting the types to only include decorations. I think a makes the most sense as a first step -- at least until I can figure out what's happening in term.c ...
Thoughts?

I've implemented option a -- text effect ANSI codes are applied in dill, but foreground/background colors are ignored.
I've got styled prompts working too, but only if they're hardcoded in drum... When I change cad.sole-prompt from tape to styx, I get find.styx with the following trace:
/~bilrex-rontel-macbep-rovdun--paddes-wattuc-hinbyl-wollyn/home/0/sur/sole:<[3 3].[118 3]>
/~bilrex-rontel-macbep-rovdun--paddes-wattuc-hinbyl-wollyn/home/0/sur/sole:<[4 1].[118 3]>
/~bilrex-rontel-macbep-rovdun--paddes-wattuc-hinbyl-wollyn/home/0/sur/sole:<[99 3].[100 38]>
/~bilrex-rontel-macbep-rovdun--paddes-wattuc-hinbyl-wollyn/home/0/sur/sole:<[100 3].[100 38]>
/~bilrex-rontel-macbep-rovdun--paddes-wattuc-hinbyl-wollyn/home/0/sur/sole:<[100 4].[100 18]>
/~bilrex-rontel-macbep-rovdun--paddes-wattuc-hinbyl-wollyn/home/0/sur/sole:<[100 6].[100 18]>
/~bilrex-rontel-macbep-rovdun--paddes-wattuc-hinbyl-wollyn/home/0/sur/sole:<[49 7].[49 15]>
/~bilrex-rontel-macbep-rovdun--paddes-wattuc-hinbyl-wollyn/home/0/sur/sole:<[49 11].[49 15]>
-find.styx

That's a pretty weird error -- ++styx is in zuse, and most definitely in scope. It's also used in ++sole-effect without any issues.
That stack trace routes through ++sole-lo (an %ask generator helper), which is a funky mold/gate that I don't quite understand... Any idea what's going wrong there?

You probably haven't reloaded zuse?
Sent from my iPhone

On Jul 27, 2016, at 4:02 PM, Joe Bryan<EMAIL_ADDRESS>wrote:
I've implemented option a -- text effect ANSI codes are applied in dill, but foreground/background colors are ignored.
I've got styled prompts working too, but only if they're hardcoded in drum... When I change cad.sole-prompt from tape to styx, I get find.styx with the following trace:
/~bilrex-rontel-macbep-rovdun--paddes-wattuc-hinbyl-wollyn/home/0/sur/sole:<[3 3].[118 3]>
/~bilrex-rontel-macbep-rovdun--paddes-wattuc-hinbyl-wollyn/home/0/sur/sole:<[4 1].[118 3]>
/~bilrex-rontel-macbep-rovdun--paddes-wattuc-hinbyl-wollyn/home/0/sur/sole:<[99 3].[100 38]>
/~bilrex-rontel-macbep-rovdun--paddes-wattuc-hinbyl-wollyn/home/0/sur/sole:<[100 3].[100 38]>
/~bilrex-rontel-macbep-rovdun--paddes-wattuc-hinbyl-wollyn/home/0/sur/sole:<[100 4].[100 18]>
/~bilrex-rontel-macbep-rovdun--paddes-wattuc-hinbyl-wollyn/home/0/sur/sole:<[100 6].[100 18]>
/~bilrex-rontel-macbep-rovdun--paddes-wattuc-hinbyl-wollyn/home/0/sur/sole:<[49 7].[49 15]>
/~bilrex-rontel-macbep-rovdun--paddes-wattuc-hinbyl-wollyn/home/0/sur/sole:<[49 11].[49 15]>
-find.styx
That's a pretty weird error -- ++styx is in zuse, and most definitely in scope. It's also used in ++sole-effect without any issues.
That stack trace routes through ++sole-lo (an %ask generator helper), which is a funky mold/gate that I don't quite understand... Any idea what's going wrong there?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

It's weird, I can make the change now, on top of this latest commit. I must've got myself into some kind of inconsistent state...
I was doing this after ++styx had been defined and used in previous commits -- including adding it to and using it in ++sole-effect. I tried |reload %zuse, |reset, starting a fake zod from src, etc. Eventually, I rolled back sur/sole/hoon, and copied in all changes in the same transaction by first syncing a tmp desk, then merging to home ... I thought I'd done those in two steps, guess I did 'em at the same time...
More changes incoming.

Ok, styled ++sole-prompt works now.
I still have to implement left-truncation for long buffers (++se-just in drum), then this "initial version" should be done.

... and another state transition for drum.

Left/right truncation for long buffers (++se-just) is done -- as far as I can tell, that's everything but the drum state transition (for plan a, that is ...).
Questions:

should the stub/styx engine (++klr) go somewhere else?
same question for ++ansi in dill ...
since I've changed $pro in ++dill-blit, should I just change $out instead of adding $klr?

I want to review all this and we'll probably have to redesign it a little
bit, which if true is my fault.  But probably won't get to it till the
afternoon.
(My experience is that generally the best quality subsystems are built by
implementing it first, then designing it, then making the implementation
conform to the design.  So we/you have done 1/3 of the work.  Ok, I guess
what I'm thinking is that ++styx should probably be folded into ++tanq in
some sense, and evolve into a new text output path, which will have to live
side by side with the old path for a little while. Feel free to cogitate.
I also may try to move or at least crosspost the conversation to
urbit.org/fora...)
On Wed, Jul 27, 2016 at 11:35 PM, Joe Bryan<EMAIL_ADDRESS>wrote:

Left/right truncation for long buffers (++se-just) is done -- as far as I
can tell, that's everything but the drum state transition (for plan a,
that is ...).
Questions:

should the stub/styx engine (++klr) go somewhere else?
same question for ++ansi in dill ...
since I've changed $pro in ++dill-blit, should I just change $out
instead of adding $klr?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/urbit/arvo/pull/216#issuecomment-235811310, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AALyAdOpgBf4Q6mrVdM_VJ3cmg2oK1yAks5qaE2igaJpZM4JVl_E
.

That makes sense -- I'll count it as a victory that your mind has been changed re: ++tanq ;)

Sorry, I still haven't gotten to this.  The packets must flow!
If you have a cycle or two, try thinking about how styled text might fit
into ++tanq, and whether the / windowing in ++re is righteous or
pointless.  It certainly makes me feel more in control of an output wall.
But does it matter?
On Thu, Jul 28, 2016 at 2:17 PM, Joe Bryan<EMAIL_ADDRESS>wrote:

That makes sense -- I'll count it as a victory that your mind has been
changed re: ++tanq ;)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/urbit/arvo/pull/216#issuecomment-236028068, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AALyAQSw2cG3CxZrHtcg5PHbvoa0ZpEIks5qaRxjgaJpZM4JVl_E
.

I've posted my comments on fora: http://urbit.org/fora/posts/~2016.8.2..21.19.29..2ab8~/. I started explaining, and couldn't stop myself, so it's kind of a sole/tank tutorial as well. You did say "longform" ...

Urbit, now with Premium Content!
Sent from my iPhone

On Aug 2, 2016, at 2:21 PM, Joe Bryan<EMAIL_ADDRESS>wrote:
I've posted my comments on fora: http://urbit.org/fora/posts/~2016.8.2..21.19.29..2ab8~/. I started explaining, and couldn't stop myself, so it's kind of a sole/tank tutorial as well. You did say "longform" ...
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

Still in progress, I presume.

Nothing new from me since that forum post

Yeah, I'm the slacker here.  The good news is, %jael is code complete, so I
can actually get to this stuff...
On Wed, Oct 5, 2016 at 7:47 AM, Joe Bryan<EMAIL_ADDRESS>wrote:

Nothing new from me since that forum post
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/urbit/arvo/pull/216#issuecomment-251696530, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AALyAfmoBrc4GcKzn91oUCcqxaJv4zFdks5qw7hlgaJpZM4JVl_E
.

I posted a follow-up comment to that forum post, with a short-term proposal to advance this feature.

Thanks again!  It's the first thing on my agenda tomorrow, I swear...
Sent from my iPhone

On Oct 5, 2016, at 8:22 PM, Joe Bryan<EMAIL_ADDRESS>wrote:
I posted a follow-up comment to that forum post, with a short-term proposal to advance this feature.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

I finally got to this!  I'm so proud of myself!
See forum comment at http://urbit.org/fora/posts/~2016.8.2..21.19.29..2ab8~/

I've rebased this onto master, splitting out the kernel changes (first 5 commits).
For posterity's sake, the original branch is here: https://github.com/joemfb/arvo/tree/kuler

foreground/background colors work now. My mistake is stunningly stupid: I was passing the color codes as atoms, not "strings". I somehow did it correctly for effects, not for colors, and promptly failed to notice the difference ...

And the cursor position is now accurate ...
This PR is now a functionally complete proof-of-concept. There are lots of potential targets for the next step: mold-definition review, dill interface, ++tanq, ++qe, ANSI conversion in term.c, etc. I'm not sure how those should be targeted, or in what order. This probably calls for another forum post ...

I've posted a comment on the form post, laying out my questions and potential next steps.

I'm going to re-rebase this back to where it once belonged. Other than that, looks alright.

@juped, have you already re-rebased? I'd like to do some minor cleanup. If you have, I'll just make a separate PR once this is merged.

If you have things to change, do it quickly before it's too late :)

I've added some cleanup commits. I was gonna squash and reorder, but I'm out of time, and it's not important ;)
I'll probably have more cleanup in subsequent changes. I think there'll be a bit of churn in this area ...

I looked at this again over the weekend and realized there's a problem with ++se-just -- it's adjusting the buffer far too early. This is because the off and edg indices are style agnostic, but the length calculations in ++scag:klr/++slag:klr are (incorrectly!) not. This bug would've exists in the old branch as well (https://github.com/joemfb/arvo/tree/kuler).
Also, not only is my existing cursor-position solution a deplorable hack, it's sometimes incorrect, depending on the style of the prompt.
I've got a commit that fixes both of these problems, but it introduces a bizarre new one. Typing a buffer long enough to trigger the wrapping in ++se-just causes the terminal to "eat" previous lines of output! (The input buffer moves up in the window by one line with each adjustment, overwriting the lines above.)
See https://github.com/joemfb/arvo/commit/461bcbab49c7fb708e0aef5b501811f22c615161 (in https://github.com/joemfb/arvo/tree/color-buffer-fix)
Does that line-eating behavior trigger any bells? I don't understand how dill or term.c could be causing that.
Note: I could fix those bugs individually, but I'm kind of attached to that not-quite-yet-working solution: it isolates the ANSI-encoding length adjustments to ++se-show, whereas my previous "hack" spread them around more...

Does anything in the vicinity emit \r characters? We're not in normal line discipline. That behavior sounds really weird, though.

Nothing that I know if is emitting \r. My description was wrong though, the line-eating behavior exists on this branch as is. Not sure how I confused that ...
My guess is that the ANSI escape codes are causing non-visible, naive line wrapping, which is interacting badly with whatever mechanism pins the prompt to the bottom of the window. IIRC, removing buffer adjustment from ++se-just causes something similar.
I'll keep digging.

Wow.  I really have no idea what could be causing the line-eating behavior.
On Mon, Oct 17, 2016 at 12:24 PM, Joe Bryan<EMAIL_ADDRESS>wrote:

I looked at this again over the weekend and realized there's a problem
with ++se-just -- it's adjusting the buffer far too early. This is
because the off and edg indices are style agnostic, but the length
calculations in ++scag:klr/++slag:klr are (incorrectly!) not. This bug
would've exists in the old branch as well (https://github.com/joemfb/
arvo/tree/kuler).
Also, not only is my existing cursor-position solution a deplorable hack,
it's sometimes incorrect, depending on the style of the prompt.
I've got a commit that fixes both of these problems, but it introduces a
bizarre new one. Typing a buffer long enough to trigger the wrapping in
++se-just causes the terminal to "eat" previous lines of output! (The input
buffer moves up in the window by one line with each adjustment, overwriting
the lines above.)
See joemfb@461bcba
https://github.com/joemfb/arvo/commit/461bcbab49c7fb708e0aef5b501811f22c615161
(in https://github.com/joemfb/arvo/tree/color-buffer-fix)
Does that line-eating behavior trigger any bells? I don't understand how
dill or term.c could be causing that.
Note: I could fix those bugs individually, but I'm kind of attached to
that not-quite-yet-working solution: it isolates the ANSI-encoding length
adjustments to ++se-show, whereas my previous "hack" spread them around
more...
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/urbit/arvo/pull/216#issuecomment-254306994, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AALyARv9_uCOGGP6DrCKAM8w5ZGZNa2vks5q08tjgaJpZM4JVl_E
.

I just confirmed that removing ++scag/++slag from ++se-just (and just outputting long lines) causes similar behavior on master

And I just confirmed by hypothesis: the terminal is wrapping, invisibly, due to longer-than expected lines (containing ANSII escape codes). The line-eating behavior is just the current, naive line-wrapping behavior.
Anybody know off-the-top how the input buffer is pinned to the bottom of the terminal window? My guess is that's our culprit.

I pushed my fix to this branch, since it doesn't introduce any new bugs ...

I don't know for sure, but I would guess it's the normal way to erase the current line, \r followed by \033K

And ESC[K doesn't move your cursor position. (I think.)

What you're describing is in u3_term_io_hija(), but I think we're actually looking at ESC[J in _term_it_show_clear().
I ask about pinning the input buffer to the bottom because I think that mechanism is what's being thrown off by naive line-wrapping. It seems like there's some mechanism in place -- for instance, command-k in Terminal.app clears the screen, but the first couple lines of subsequent output are vertically separated by an entire window-height.
Urbit has been my only experience with low-level terminal I/O, so I'm not sure what to look for ... ;)

You don't do anything special, you just fail to emit \r\ns. We redraw the last line on every terminal event, I think. Do you mean on a resize?

I can tell you that the Unix tty subsystem is well-known as the haunt of
shoggoths.
Yes, pinning the input buffer to the bottom involved throwing a lot of
terminfo crap at the terminal.    But I do remember that it felt
abusive.
The upside is that at least it's a stable interface, so when you fix it it
tends to stay fixed...
On Mon, Oct 17, 2016 at 1:06 PM, Joe Bryan<EMAIL_ADDRESS>wrote:

What you're describing is in u3_term_io_hija(), but I think we're
actually looking at ESC[J in _term_it_show_clear().
I ask about pinning the input buffer to the bottom because I think that
mechanism is what's being thrown off by naive line-wrapping. It seems like
there's some mechanism in place -- for instance, command-k in Terminal.app
clears the screen, but the first couple lines of subsequent output are
vertically separated by an entire window-height.
Urbit has been my only experience with low-level terminal I/O, so I'm not
sure what to look for ... ;)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/urbit/arvo/pull/216#issuecomment-254317883, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AALyARlAop3l_b7Z769N6gOJBuvX9eVyks5q09VEgaJpZM4JVl_E
.

(If we do something more complicated than this, as I think you're saying, reducing it to the simplest thing might isolate the problem.)

I'm basically saying "here be dragons."  :-)  But once you've vanquished
the dragons, they stay vanquished.
Yes, re-outputting the line is how we keep it steady...
On Mon, Oct 17, 2016 at 1:10 PM, Raymond Pasco<EMAIL_ADDRESS>wrote:

(If we do something more complicated than this, as I think you're saying,
reducing it to the simplest thing might isolate the problem.)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/urbit/arvo/pull/216#issuecomment-254318832, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AALyAdz7hdBbiQEBVTNgHbvcXe3n69IOks5q09YqgaJpZM4JVl_E
.

I've added a temporary solution -- it just truncates the buffer early. That works fine for avoiding the line-wrapping/eating, but can make for pretty narrow buffers on comets, for instance. I'll look further into term.c when I can.
This PR is now in a stable state, with no bugs that I'm aware of.

That's not a problem, since there's another kernel patch going in alongside this that truncates the comet name in the prompt.
Are things reasonable in 80-width terminals? That's all I care about.

No, it's really not reasonable with only 80 columns. On a fake-zod, in :dojo, I have to truncate 40 characters early. Anything longer, and it "wraps".

That sounds like a game-breaking bug! (Since 80 characters is basically the officially endorsed terminal width.)

makes his terminal wider

@agrees, 80 is standard.
Us oldies use this to put lots of terminals on our tiny screens...
Sent from my iPhone

On Oct 17, 2016, at 2:16 PM, Raymond Pasco<EMAIL_ADDRESS>wrote:
That sounds like a game-breaking bug! (Since 80 characters is basically the officially endorsed terminal width.)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

Sounds like we need an ansi-skipping "display length" calculator
On Monday, 17 October 2016, Joe Bryan<EMAIL_ADDRESS>wrote:

makes his terminal wider
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/urbit/arvo/pull/216#issuecomment-254336985, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABxXhhxDYO8yx2vMi5DrOiOAuVT5Ntxdks5q0-YRgaJpZM4JVl_E
.

I'm just putting it up here... Everything in this PR works and is "usable" without anything breaking, eating output, etc. However, if you run with 80 columns, your prompt will start scrolling left pretty quickly, which is at the very least irritating.
As I understand it, there's no line wrapping in term.c  -- see months earlier in this comment thread ;). A "ansi-skipping length calculator" is an accurate, functional description of what we need, there's no obvious place to put it. It probably needs to happen in termios... Maybe there's a configuration option?

Well, in the prompt scrolling code then. Iirc that also had issues with
unicode
On Monday, 17 October 2016, Joe Bryan<EMAIL_ADDRESS>wrote:

I'm just putting it up here... Everything in this PR works and is "usable"
without anything breaking, eating output, etc. However, if you run with 80
columns, your prompt will start scrolling left pretty quickly, which is at
the very least irritating.
As I understand it, there's no line wrapping in term.c -- see months
earlier in this comment thread ;). A "ansi-skipping length calculator" is
an accurate, functional description of what we need, there's no obvious
place to put it. It probably needs to happen in termios... Maybe there's
a configuration option?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/urbit/arvo/pull/216#issuecomment-254338713, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABxXhsPh7Z7cBqKl2HZuylJWnQYnRrIOks5q0-eogaJpZM4JVl_E
.

I'll postpone this patch for now and look at the subsystem a bit later. I've been down terminal rabbit holes before. It's going to end up being something we're actively doing, not something we're failing to do.

(If you can find it faster than me, you're welcome to, of course.)

 Probably using ++lent or sth to determine how long
the buffer is.
On Monday, 17 October 2016, Raymond Pasco<EMAIL_ADDRESS><javascript:_e(%7B%7D,'cvml','notifications@github.com');>> wrote:

(If you can find it faster than me, you're welcome to, of course.)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/urbit/arvo/pull/216#issuecomment-254339281, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABxXhhzWDsPLzlrAAJcNORCnEdsU1b97ks5q0-gsgaJpZM4JVl_E
.

Yeah, that's in ++se-just. To see the bug in question, replace
(se-show (sub p.lin off) (scag edg (slag off q.lin)))
with (se show lin). Then type an input buffer longer than your terminal width, and navigate back and forth. The input buffer will eat previous lines of output.
In this PR, I've updated ++se-just, to work correctly with ++stub, but the non-printing ANSI escape codes wrap off the end of the line (invisibly), and trigger the line-eating. So I'm truncating early to account for the ANSI escape codes.

What a sh*tshow!  I do believe I've hated Unix tty handling longer than
anyone else alive on this thread.  Thanks everyone for wrestling this
alligator.
Keep in mind again that even if it's not the most architecturally elegant
thing, it's way easier if we can coerce it into working with the existing
term.c...
On Mon, Oct 17, 2016 at 3:25 PM, Joe Bryan<EMAIL_ADDRESS>wrote:

Yeah, that's in ++se-just. To see the bug in question, replace
(se-show (sub p.lin off) (scag edg (slag off q.lin)))
https://github.com/urbit/arvo/blob/8275335a355306f7a80d41d4f5a6e83c37dfe17a/lib/drum.hoon#L401
with (se show lin). Then type an input buffer longer than your terminal
width, and navigate back and forth. The input buffer will eat previous
lines of output.
In this PR, I've updated ++se-just, to work correctly with ++stub, but
the non-printing ANSI escape codes wrap off the end of the line
(invisibly), and trigger the line-eating. So I'm truncating early to
account for the ANSI escape codes.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/urbit/arvo/pull/216#issuecomment-254352863, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AALyAeNy3zc9We6u4ZRXNx4hoii86fWvks5q0_XngaJpZM4JVl_E
.

I found a fix in term.c; comment this out: https://github.com/urbit/urbit/blob/166431ed99e9497aa0ac6642e2ab85e219f46e4e/vere/term.c#L584-L586.
uty_u->ufo_u.out.cuu1_y is set to \033[A ("cursor up") here: https://github.com/urbit/urbit/blob/166431ed99e9497aa0ac6642e2ab85e219f46e4e/vere/term.c#L309.
Any idea why that was in there? Or how I might work around it in %dill?

the relevant commit: https://github.com/urbit/urbit/commit/712968d37e652aef0103064cf42b35d3c0886466 ;)

while I'm at it, how can this second branch ever be false? https://github.com/urbit/urbit/blob/166431ed99e9497aa0ac6642e2ab85e219f46e4e/vere/term.c#L579

In any case, it goes all the way back: https://github.com/urbit/archaeology-factor/blob/5deb547259158f389e90cd6faf6438041130003f/urbit/v/term.c#L352

Nice work.

Good catch!  No, great catch!
The bottom line is, if we have to patch vere, we have to patch vere.  Let's
just make sure we get galaxy IP timeouts in the same fix...
On Mon, Oct 17, 2016 at 7:21 PM, Joe Bryan<EMAIL_ADDRESS>wrote:

I found a fix in term.c; comment this out: https://github.com/urbit/
urbit/blob/166431ed99e9497aa0ac6642e2ab85e219f46e4e/vere/term.c#L584-L586.
uty_u->ufo_u.out.cuu1_y is set to \033[A ("cursor up") here:
https://github.com/urbit/urbit/blob/166431ed99e9497aa0ac6642e2ab85
e219f46e4e/vere/term.c#L309.
Any idea why that was in there? Or how I might work around it in %dill?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/urbit/arvo/pull/216#issuecomment-254388599, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AALyAQmqORa89Hv8yBJETrI-l98xvwyYks5q1C1CgaJpZM4JVl_E
.

It's possible this could be worked around by manipulating the cursor position before and after $lin blits (ie, setting the cursor +40 past the column width)... That's the only thing I can think of that might work, and I haven't tried it yet.

I've done a little refactoring to clean up some rough edges, and reordered my recent commits. d5142f4 (the HEAD of this branch) is the temporary hack to fix line-eating; it's made obsolete by urbit/urbit#780. Leaving it here for now in the interest of flexibility ...

@juped, is the plan to release this at the same time as urbit/urbit#780? (and is that vere 0.4?)
If so, I'll drop the early-truncation hack in 8b2494b and this should be good-to-go.

Yes, and don't change anything now, it's taken into account.

Sounds good, I'll restrain myself from making further changes ;)

Whoops, forgot to hit send last night. Either the state adapter for drum got lost somehow or it never existed, not sure. However, this still works fine as-is without the state adapter, and the unlinks it forces are actually desirable to me just to avoid possible inconsistent dojo states. Shame about the stack traces and errors it prints, though.

I never wrote the state adapter. The last couple times I "installed" these changes, everything worked fine. I don't recall seeing a stack-trace either...
The only state change is mir.source -- want me to work up an adapter that just clears it every time? The only side-effect should be an extra card every once awhile ...

That would be good if you can - though it seems to work, I don't like the possibility of resetting kiln state.

Merging this PR into a fake ~zod blows up in sur/sole/hoon:
/~nec/home/0/sur/sole:<[49 11].[49 15]>
-find.styx

That's after an automatic reload of %zuse and the vanes. Seems like sur/sole/hoon is being compiled without those updated cores.  Do I need to bump a version number somewhere?
(To apply the changes in one transaction, I'm copying this PR into a %scratch desk, then merging %scratch into %home)

No, we have a longstanding issue here - don't do it in one event, do the kernel first and then the rest.

I can't reproduce any errors caused by the absence of a drum state-transition. I also don't know how to release these changes without errors...
Merging in only the kernel changes in this PR breaks the console. Any input produces:
%hard
exit
<stack>
[%swim-take-vane %d %unto ~]
<stack>

Merging in the drum and sole changes after that fixes everything again, without any stack trace.
I wrote a simple transition in ++drum-port, but its presence or absence has no effect:
$1  %_(old bin (~(run by bin.old) |=(a/source %_(a mir [0 ~]))))

What's the error that you were seeing?

That sounds a lot more like an entirely failed update to me. Do a git merge and copy into your mount, not a clay merge.

For the sake of clarity:
$ git checkout master
$ git checkout -b scratch
$ cd ..
$ bin/urbit -F -I zod -A ./arvo/ -c fake-zod

~zod:dojo> |mount %

$ git merge colors
$ cp arvo/* ../fake-zod/home/arvo/

(%swim-take-vane as above)
$ cp sur/sole.hoon ../fake-zod/home/sur/ && cp lib/drum.hoon ../fake-zod/home/lib/

... and then everything's hunky-dory.

Can you do the following:

fetch git://the.ug/~ray/arvo.git staging to somewhere
use urbit v0.4, arvo origin/master
make a fakezod using the origin/master arvo directory
|mount /=base=
check out my staging
rsync -rC --delete arvo/arvo/ zod/base/arvo/ (important to keep those trailing slashes on!)
wait for kernel compile, which should be error-free
rsync -rC --delete arvo/ zod/base/
wait for it to sync, should have a lot of crashes but ultimately be harmless and relink your apps once you press a key

Ha ha, I'll give you something else to think about!  (...puts finishing
touches...)
On Mon, Oct 24, 2016 at 1:53 PM, Joe Bryan<EMAIL_ADDRESS>wrote:

Sounds good, I'll restrain myself from making further changes ;)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/urbit/arvo/pull/216#issuecomment-255862892, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AALyAc8d7AZ-dYGdLXKILacYHxUg7aXUks5q3RrGgaJpZM4JVl_E
.

Ok, I was able to get a stack-trace (nest-fail) followed by %ap-slam-mismatch and the ++drum-part mold.
I still got the same errors on keyboard input after syncing the kernel changes, before syncing everything else:
exit
<stack>
[%swim-take-vane %d %unto ~]
<stack>
%hard

I also did a second run through, with my pseudo state-adapter from earlier in this thread, but that produced the exact same stack trace and error messages. It'll need to be a full, mold-based state transition.
I have to step out for awhile, but I can come back to this in a couple hours.

On keyboard input makes sense. In theory the two update events go out without room for anyone to press a key; they should be one but that hits the bug you saw. Does everything work afterwards?

Everything works after completing both syncs. The only problem is the
stack/nest-fail/prep-mismatch and subsequent relink ...
It sounds like you might want that outcome anyway
On Tuesday, October 25, 2016, Raymond Pasco<EMAIL_ADDRESS>wrote:

On keyboard input makes sense. In theory the two update events go out
without room for anyone to press a key; they should be one but that hits
the bug you saw. Does everything work afterwards?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/urbit/arvo/pull/216#issuecomment-256132175, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAq3YCLI4hIpnmHXTwJsOiPb5Y_Pbmvpks5q3kxrgaJpZM4JVl_E
.

 I'm suspicious of how it affects other hood parts, though. They should have their state reset, according to my mental model, but aren't?

Yeah, prep-mismatch being a /recoverable/ error sounds deeply suspicious
(At least like, automatically recoverable, iirc as an app it should keep
the old version until you sync in something compatible)
On Tuesday, 25 October 2016, Raymond Pasco<EMAIL_ADDRESS>wrote:

I hate the error spamming but like the forced relinks. I'm suspicious of
how it affects other hood parts, though. They should have their state
reset, according to my mental model, but aren't?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/urbit/arvo/pull/216#issuecomment-256134839, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABxXhosXfLPMg-lrZcnf6UnZbwQ1xfYoks5q3k4CgaJpZM4JVl_E
.

There you go, then. Can wait for the true hood state adapter. A real one would be preferable, as much as I want the simplicity of unlinking everyone.

eagerly awaits a shell model that doesn't depend on subscriptions staying
open to not lose history
On Tuesday, 25 October 2016, Raymond Pasco<EMAIL_ADDRESS>wrote:

Whoops, forgot to hit send last night. Either the state adapter for drum
got lost somehow or it never existed, not sure. However, this still works
fine as-is without the state adapter, and the unlinks it forces are
actually desirable to me just to avoid possible inconsistent dojo states.
Shame about the stack traces and errors it prints, though.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/urbit/arvo/pull/216#issuecomment-256071536, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABxXhs6egJucQ3_-wyyY_XRCTpwvpvudks5q3iHBgaJpZM4JVl_E
.

I'll wait until the evening (since you said "hours") and if it's not there by then I'll push it back to the next kernel since this one is otherwise ready. No rush if you can't.

i'm working on it now.  I'll update you soon

😨

I've added a transition; it seems like it should work, but it crashes on ++drum-pith-0 (which is _!!). I'm not sure what I can use in its place ...
Note: my other guess is this naive fork in :hood:
++  hood-part-old  ?(hood-part drum-part-old)

Here's the stack:
onto
/~zod/home/0/app/hood:<[69 28].[72 42]>
/~zod/home/0/app/hood:<[70 3].[72 42]>
/~zod/home/0/app/hood:<[71 3].[72 42]>
/~zod/home/0/app/hood:<[72 3].[72 42]>
/~zod/home/0/app/hood:<[72 10].[72 41]>
/~zod/home/0/app/hood:<[72 31].[72 40]>
/~zod/home/0/app/hood:<[37 7].[40 9]>
/~zod/home/0/app/hood:<[37 11].[37 28]>
/~zod/home/0/app/hood:<[37 15].[37 28]>
/~zod/home/0/app/hood:<[35 36].[35 49]>
/~zod/home/0/lib/drum:<[15 7].[17 5]>
/~zod/home/0/lib/drum:<[16 15].[16 26]>
/~zod/home/0/lib/drum:<[18 19].[18 21]>

/cc @ohAitch

This is mostly working now, for some value of "mostly" and "working".
The syncs come in, and no relinking happens, but there's still a stacktrace:
onto
/~zod/home/~2016.10.25..20.48.02..61da/arvo/ford:<[1.210 24].[1.210 52]>
/~zod/home/0/lib/drum:<[111 1].[894 3]>
/~zod/home/0/lib/drum:<[114 1].[894 3]>
/~zod/home/0/lib/drum:<[115 1].[894 3]>
/~zod/home/0/lib/drum:<[116 1].[894 3]>
/~zod/home/0/lib/drum:<[117 1].[894 3]>
/~zod/home/0/lib/drum:<[134 1].[894 3]>
/~zod/home/0/lib/drum:<[390 3].[395 14]>
/~zod/home/0/lib/drum:<[391 3].[395 14]>
/~zod/home/0/lib/drum:<[392 3].[395 14]>
/~zod/home/0/lib/drum:<[393 3].[395 14]>
/~zod/home/0/lib/drum:<[393 11].[393 53]>
/~zod/home/0/lib/drum:<[393 32].[393 52]>
nest-fail
ford: core /~zod/home/~2016.10.25..20.50.31..26d1/app/hood

This is dumped near the beginning of the second sync. The line numbers refer to drum on master, specifically (se-blit %pro q.lin). In other words, while syncing in updates, the old version of drum tries to blit to the new version of %dill. Not sure if there's any workaround for this, outside of splitting things into 3 syncs ...

My state transition commit (5cb0c03) is a child of the irrelevant hack (8b2494b). Do you want me to rebase or reparent in some way?

I'll zap it if you don't, it's no big deal

zapped it. the pr can be accurate for posterity's sake.
One last thing (and I really hope I mean it this time): should the state transition be squashed onto 5423cfa ("adds support for styled prompt") so that arvo is "safe" on a per-commit basis?

You can. I'm not losing sleep either way, but it's better.

here's to shipping!

Would it be worth rotating all the marks to %blit2 etc? Then you can have
new dill just ignore old blits
On Tuesday, 25 October 2016, Joe Bryan<EMAIL_ADDRESS>wrote:

My state transition commit (5cb0c03
https://github.com/urbit/arvo/commit/5cb0c03a96bc9be55e7c27d46cddb5088c0add82)
is a child of the irrelevant hack (8b2494b
https://github.com/urbit/arvo/commit/8b2494b435d9b166c2ed591ab9b70ff8f2d70daf).
Do you want me to rebase or reparent in some way?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/urbit/arvo/pull/216#issuecomment-256174546, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABxXhuSLzytWHiAEvNl2gdJznZLClenOks5q3m4fgaJpZM4JVl_E
.

On the one hand, that seems like the correct approach. On the other, changing the %dill interface to avoid a one-time stack trace seems pretty drastic.
awaits further instruction

That's already why blit and dill-blit are different things!  I'd say, do the ugly blit2 thing and clean it up in cc-release.
We need a better boilerplate design for upgradable vanes...
Sent from my iPhone

On Oct 25, 2016, at 3:38 PM, Joe Bryan<EMAIL_ADDRESS>wrote:
On the one hand, that seems like the correct approach. On the other, changing the %dill interface to avoid a one-time stack trace seems pretty drastic.
awaits further instruction
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

Kernel changes are now all purely additive: ++dill-blit has been extended with $klr and $pom. The state transition now runs without errors.

Ideal.
On Tue, Oct 25, 2016 at 5:03 PM, Joe Bryan<EMAIL_ADDRESS>wrote:

Kernel changes are now all purely additive: ++dill-blit has been extended
with $klr and $pom. The state transition now runs without errors.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/urbit/arvo/pull/216#issuecomment-256213901, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AALyAbZFgkOT3tA2r-sWw_7gRl3y9PgSks5q3pjSgaJpZM4JVl_E
.


Unifi: offline device shown as "home"

Home Assistant release with the issue:
0.97.2
Last working Home Assistant release (if known):
0.96.x
Operating environment (Hass.io/Docker/Windows/etc.):
Hassio on docker
Component/platform:
Unifi
Description of problem:
Multiple clients which are offline in the Unifi Controller shown as "home" in HA.
Unifi Controller 5.10.26 (via Hassio)
I've used the dev tools to manually set these devices to not_home, however they are promptly changed back to home by the Unifi integration.
Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):
<config added through integrations>

Traceback (if applicable):
Log line from when Unifi integration is added. This iPhone had been off wifi for >30mins when the integration was added to HA
2019-08-13 12:07:58 DEBUG (MainThread) [aiounifi.api] [ {'_id': '5bd48655ddb88d01ef574448',
  '_is_guest_by_ugw': False,
  '_last_seen_by_ugw':<PHONE_NUMBER>,
  '_uptime_by_ugw': 7887,
  'assoc_time':<PHONE_NUMBER>,
  'authorized': True,
  'bytes-r': 0,
  'first_seen':<PHONE_NUMBER>,
  'gw_mac': 'redacted',
  'hostname': 'redactediPhone',
  'ip': 'redacted',
  'is_guest': False,
  'is_wired': True,
  'last_seen':<PHONE_NUMBER>,
  'latest_assoc_time':<PHONE_NUMBER>,
  'mac': 'redacted',
  'network': 'HOME',
  'network_id': '5b639e8d4c80068775855ae8',
  'oui': 'Apple',
  'qos_policy_applied': True,
  'rx_bytes': 16766995,
  'rx_bytes-r': 0,
  'rx_packets': 15715,
  'site_id': '5b638cf84c800687758559a7',
  'tx_bytes': 2163798,
  'tx_bytes-r': 0,
  'tx_packets': 13958,
  'tx_retries': 0,
  'uptime': 15098,
  'user_id': '5bd48655ddb88d01ef574448',
  'wifi_tx_attempts': 0},

Additional information:
Maybe related to #25876?

Have removed and re-added the Unifi integration with the above device remaining offline.
Interestingly, the last_seen value has updated. In this instance, latest_assoc_time represents the correct time the device was connect to wifi.
 2019-08-13 12:30:06 DEBUG (MainThread) [aiounifi.api] [ {'_id': '5bd48655ddb88d01ef574448',
  '_is_guest_by_ugw': False,
  '_last_seen_by_ugw':<PHONE_NUMBER>,
  '_uptime_by_ugw': 7887,
  'assoc_time':<PHONE_NUMBER>,
  'authorized': True,
  'bytes-r': 0,
  'first_seen':<PHONE_NUMBER>,
  'gw_mac': 'redacted',
  'hostname': 'redactediPhone',
  'ip': 'redacted',
  'is_guest': False,
  'is_wired': True,
  'last_seen':<PHONE_NUMBER>,
  'latest_assoc_time':<PHONE_NUMBER>,
  'mac': 'redacted',
  'network': 'HOME',
  'network_id': '5b639e8d4c80068775855ae8',
  'oui': 'Apple',
  'qos_policy_applied': True,
  'rx_bytes': 16766995,
  'rx_bytes-r': 0,
  'rx_packets': 15715,
  'site_id': '5b638cf84c800687758559a7',
  'tx_bytes': 2163798,
  'tx_bytes-r': 0,
  'tx_packets': 13958,
  'tx_retries': 0,
  'uptime': 16419,
  'user_id': '5bd48655ddb88d01ef574448',
  'wifi_tx_attempts': 0},

Got the same issue.

A couple of further observations. Note, the referenced iphone was not connected to wifi at all during these tests.

When adding a Unifi integration, the last_seen attribute reported by aiounifi.api changes dramatically<PHONE_NUMBER> ><PHONE_NUMBER>).

  2019-08-13 13:48:06 DEBUG (MainThread) [aiounifi.api] [{'_id': '5b64c2d15ab20c063cc27ab5',
  '_is_guest_by_ugw': True,
  '_last_seen_by_ugw':<PHONE_NUMBER>,
  '_uptime_by_ugw': 1273,
  'assoc_time':<PHONE_NUMBER>,
  'authorized': True,
  'bytes-r': 0,
  'first_seen':<PHONE_NUMBER>,
  'gw_mac': 'redacted',
  'hostname': 'redacted-iPhone',
  'ip': 'redacted',
  'is_guest': True,
  'is_wired': True,
  'last_seen':<PHONE_NUMBER>,
  'latest_assoc_time':<PHONE_NUMBER>,
  'mac': 'redacted',
  'network': 'GUEST',
  'network_id': '5b64aa654c80068775855b33',
  'oui': 'Apple',
  'qos_policy_applied': True,
  'rx_bytes': 64084,
  'rx_bytes-r': 0,
  'rx_packets': 0,
  'site_id': '5b638cf84c800687758559a7',
  'tx_bytes': 11768,
  'tx_bytes-r': 0,
  'tx_packets': 83,
  'tx_retries': 0,
  'uptime': 7082,
  'user_id': '5b64c2d15ab20c063cc27ab5',
  'wifi_tx_attempts': 0},

2019-08-13 13:48:08 DEBUG (MainThread) [aiounifi.api] [{'_id': '5b64c2d15ab20c063cc27ab5',
  'first_seen':<PHONE_NUMBER>,
  'hostname': 'redacted-iPhone',
  'is_guest': True,
  'is_wired': True,
  'last_seen':<PHONE_NUMBER>,
  'mac': 'redacted',
  'oui': 'Apple',
  'site_id': '5b638cf84c800687758559a7'},

Sometimes after adding the Unifi integration, the entity state flaps from home > not_home > `home.

2019-08-13 14:07:19 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=device_tracker.redacted_iphone, old_state=<state device_tracker.redacted_iphone=home; (SNIP) @ 2019-08-13T14:07:03.574839+01:00>, new_state=<state device_tracker.redacted_iphone=not_home; (SNIP) @ 2019-08-13T14:07:19.418263+01:00>>
2019-08-13 14:07:19 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=device_tracker.redacted_iphone, old_state=<state device_tracker.redacted_iphone=not_home; (SNIP) @ 2019-08-13T14:07:19.418263+01:00>, new_state=<state device_tracker.redacted_iphone=home; (SNIP) @ 2019-08-13T14:07:19.541697+01:00>>

I had this happen once for a device in my testing for my reported unifi issue where devices are being reported as invalid entity at times.  Curious if these status pulls are related at all from pulling from the entity domain.  https://github.com/home-assistant/home-assistant/issues/25885

I've also got a repro; homeassistant docker, controller on a cloud-key.
I'm testing with specifically reverting lines 297 and 342 from this commit.
https://github.com/home-assistant/home-assistant/commit/f03538f8666b021818b771ae950f08c849156460#diff-324cad01d3995e0340760ccf1892aaab
Had to do it real quick over ssh at lunch -- will see if that shakes out and stabilizes my device states over the day.

That change is for devices e.g. Network infrastructure. Clients are for all normal network devices

I'd need to go read through the code (and I do know you're the owner here, so I appreciate it), but it does appear to have stabilized my flapping state behavior on my client devices.

Been stable for 8 hours after that revert. I don't really have a solid integration environment for this, but I can work my way through the code, specifically the state property to better understand why.

@swestcott your iPhone is reported as being wired, that's a bit troublesome in it self.
@jeremydk still working for you?
@mkmer can you share logs?

Ok, just upgraded to .98 and it now appears to be working.  I have tested several devices disconnecting from wifi and they are staying "not_home" where previously the would go "not_home" for ~1 minute then switch back to "home" (aka flap as described above).
I'll update if it stops working.

well... I take that back.  One of my 4 device trackers was working for ~1 day, the other 3 continued to work in the "flap" mode where they go "not_home" for ~1 minute then return to the "home" state indefinitely.

Following up on this issue further: I'm currently on Unifi Controller 5.10.26.  Noticed that the "last seen" times under "insights" continues to show current time even when devices are disconnected (and not showing on the clients list.  I suspect this is the root cause of the devices reappearing as "home".  Anyone else seeing this?

Wifi devices reporting as LAN is a long-standing bug in Unifis software
https://community.ui.com/questions/Wifi-devices-showing-as-LAN/0771fb9f-84f0-4111-a417-3eada0d88c9d
https://community.ui.com/questions/Wireless-clients-shown-as-wired-clients/49d49818-4dab-473a-ba7f-d51bc4c067d1
Since this issue has persisted for 3 years(!) in Unifi is there anything we can do to fix it inside hass?
This unfortunate bug makes device_tracker useless, as I run into this bug all of the time, and none of my automations trigger
I had hoped dont_track_wired_clients: true would solve the issue, but as far as I can tell that option is non-functioning in both the yaml config and the UI config, as setting it doesn't prevent wired clients from appearing.

Thanks! I have been thinking about how to properly identify a device that goes between wired and wireless in a deterministic way

@mkmer - yes, I'm experiencing the same issue. It's caused by a longstanding bug in Unifi's controller software. There are a number of threads on their forum describing the issue, like this one or this this one.
The pattern is consistent for me:

a wireless client disconnects from the wifi network
the controller correctly reports the client has disconnected
about a minute later the controller incorrectly reports the client has re-connected, this time as a wired device
about 10–15 minutes later, the controller reports the phantom wired device has disconnected

Here's what it looks like in the controller's events feed:

(It doesn't show when the iPhone's connected to LAN as wired devices but does indicate when phantom wired devices disconnected.)
I contacted Unifi about this but was told they couldn't help because my controller is running in docker on a Synology NAS, rather than a supported device from Ubiquiti.
It's a frustrating problem since it causes false positives with presence detection in HASS. Previously, I was able to workaround the issue by enabling the Unifi's components SSID filtering which would ignore updates from devices with a matching essid attribute. This worked perfectly because when the controller erroneously reports a wireless client is connected as a wired device, it no longer contains the essid or ap_mac attributes (as seen in the logs posted by @swestcott).
Unfortunately, with the recent overhaul of the Unifi integration, this workaround no longer works. I had hopes that enabling the new dont_track_wired_clients option might be the answer but no luck there either. This is as of v0.98.4.
BTW, No complaints here—@Kane610's updates have been fantastic and are much appreciated. The ability to configure integration options directly in the frontend is awesome.

As I have basically the same problem, let me purpose an Idea:
We have an ssid_filter but don't use it obviously.
So: Why don't we use this? If a disconnected client gets assigned to LAN, it is obviously2.0 NOT in this SSID, so not_home

The only successful workaround I've found is to switch to the Direct AP component. Definitely not ideal, it lacks many of the niceties provided by the main Unifi integration.  However, presence detection has been rock solid since I made the switch.

I have exactly the same issue. @aaronwolen, the problem with that solution is that most people will probably have multiple APs.
The ssid_filter is working for me, but since this new integration form I can no longer whitelist devices. homeassistant.components.unifi.device_tracker will happily report every client's state.

@ruimarinho, the direct AP component can be used with multiple APs.
When you say the ssid_filter option is working for you, do you mean it prevents recently disconnected wireless clients from spuriously reporting they're connected to the LAN as a wired device?

@aaronwolen that's news to me. I might give it a shot then! No, sorry for the confusion - I meant it filters out the correct entities from the provided ssid but I still experience the wired device issue. I was referring to @LukasQ's comment.

So, @ruimarinho you're saying that the ssid_filter filters out clients, once they switch from e.g. "SuperSSID" WiFi to "WLAN"?
Because my disconnected clients get dumped into "WLAN" Network and I wonder why HA still "sees" them, even if the ssid_filter for "SuperSSID" is active.

Hey! It would be great if one of you who has got this problem could try out my PR to verify if it solves this issue

Are there docker containers built from PR branches that can be used to more easily test?

@chrishoage don't think so. If you fork the gitrepo you can start it through dev container with visual studio code

Bummer, I'm not sure I'll have time to set up a new environment to test this change.
Hopefully someone else in this thread will be able to!

I'm testing this out already so will post my updates as soon as I have enough data.

@ruimarinho how's the evaluation going?

If someone could point me to a documentation how to setup the build pipeline, I'd test it on my hypervisor as well

Do a fork of my home-assistant GitHub project check out this PRs branch and then follow dev instructions on how to set it up. You can run it on whatever machine you want

So far, so good! No false positives yet. :)

Great! Working on tests right now. Will get it merged in time for 0.100 beta

So far, so good! No false positives yet. :)

Can you confirm that the Unifi controller is showing the device as wired? I'll note that this issue doesn't always happen, so it might be a good idea to ensure that the unifi controller shows the wireless device as wired and hass shows the device as away.

I've gotten one situation like that with one of our iPads, in that case I had a extra print tell me when the expected wired state diverged from what Unifi reported

Mine are shown as "WLAN", not wired aka "LAN"

See my screenshot here: https://github.com/home-assistant/home-assistant/issues/25915#issuecomment-529047903
When the issue with devices showing as "home" in hass when in fact they are "away" the device will show in the state shown in the screenshot in the unifi controller.
Note a wireless device like a cellphone is shown connected to the wired network, not an access point (or disconnected, as it should show)

I haven’t been able to check if the is wired property is being properly set because every time I remember to check away from home it’s already past a few hours in 😅
For example, almost two hours in and the property is set to false. However, the false positives are gone since I applied the patch. My wife’s going to be very happy 😃

Thanks a lot for your work on this @Kane610!

@Kane610, the beta fix is working beautifully so far!

Awesome! Thanks for the feedback @aaronwolen

@Kane610 Thanks a bunch for the great work!

Thanks!

Could it be the case we have the same problem again?
I face after the recent update to 0.13.2 more or less the same problems. Clients tend to "flicker" in their home state.

No the specific issue of wireless devices becoming wired shouldn't come back. What is 0.13.2?


Redis container exiting

Describe the bug
I keep finding redis exited.

Reproduction steps
No specific actions preceed this, but it seems to keep happening.
Expected behavior
Redis does not exit?
Additional context
I've disabled watchtower for Kaizoku in case updates/restarts were the culprit. Here is my compose:

version: '3'

volumes:
  db:
  redis:

services:
  app:
    container_name: kaizoku
    image: ghcr.io/oae/kaizoku:latest
    environment:
      - DATABASE_URL=postgresql://kaizoku:kaizoku@db:5432/kaizoku
      - KAIZOKU_PORT=3000
      - REDIS_HOST=redis
      - REDIS_PORT=6379
    env_file:
      - /opt/appdata/.id.env
      - /opt/appdata/.timezone.env
    volumes:
      - /mnt/media/media/manga:/data
      - /opt/appdata/kaizoku/config:/config
      - /opt/appdata/kaizoku/logs:/logs
    depends_on:
      db:
        condition: service_healthy
    ports:
      - '8089:3000'
    restart: always
    security_opt:
      - apparmor:unconfined
    networks:
      - homelab
    labels:
      com.centurylinklabs.watchtower.enable: false
  redis:
    image: redis:7-alpine
    volumes:
      - redis:/data
    security_opt:
      - apparmor:unconfined
    networks:
      - homelab
    labels:
      com.centurylinklabs.watchtower.enable: false
  db:
    image: postgres:15-alpine
    restart: unless-stopped
    healthcheck:
      test: ['CMD-SHELL', 'pg_isready -U kaizoku']
      interval: 5s
      timeout: 5s
      retries: 5
    environment:
      - POSTGRES_USER=kaizoku
      - POSTGRES_DB=kaizoku
      - POSTGRES_PASSWORD=kaizoku
    volumes:
      - db:/var/lib/postgresql/data
    security_opt:
      - apparmor:unconfined
    networks:
      - homelab
    labels:
      com.centurylinklabs.watchtower.enable: false

networks:
  homelab:
    driver: bridge
    external: true

Note that I included the fix from my other issue where the database becoming unreachable due to incompatible Postgres version changes from v15 to v16. So my current compose has a slight change from the default db, where image: postgres:15-alpine rather than :alpine.

is 7-alpine now pulling in 7.2 ?   mine is using 7.0 reddis with postgres 15.3

Appears to exit at least daily

is 7-alpine now pulling in 7.2 ? mine is using 7.0 reddis with postgres 15.3

Yes, it looks like it grabbed 7.2.1 - do you think that's the issue?

Appears to exit at least daily

is 7-alpine now pulling in 7.2 ? mine is using 7.0 reddis with postgres 15.3

Yes, it looks like it grabbed 7.2.1 - do you think that's the issue?

Ok, it looks like you'd have to use the tag 7.0-alpine to keep it on a 7.0.x version, as 7-alpine seems to function more as a 'latest'. Here are the tags: https://hub.docker.com/_/redis

using 7.0-alpine generates the following error
kaizoku-redis-1  | 1:C 24 Sep 2023 17:28:13.444 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
kaizoku-redis-1  | 1:C 24 Sep 2023 17:28:13.444 # Redis version=7.0.13, bits=64, commit=00000000, modified=0, pid=1, just started
kaizoku-redis-1  | 1:C 24 Sep 2023 17:28:13.444 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
kaizoku-redis-1  | 1:M 24 Sep 2023 17:28:13.444 * monotonic clock: POSIX clock_gettime
kaizoku-redis-1  | 1:M 24 Sep 2023 17:28:13.445 * Running mode=standalone, port=6379.
kaizoku-redis-1  | 1:M 24 Sep 2023 17:28:13.445 # Server initialized
kaizoku-redis-1  | 1:M 24 Sep 2023 17:28:13.445 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
kaizoku-redis-1  | 1:M 24 Sep 2023 17:28:13.445 # Can't handle RDB format version 11
kaizoku-redis-1  | 1:M 24 Sep 2023 17:28:13.445 # Fatal error loading the DB: Invalid argument. Exiting.
kaizoku-redis-1 exited with code 1

Ok, I may have resolved this on my end - the line I was using for watchtower was not being acknowledged/recognized for some reason. I switched the labels to the following and it appears to no longer be stopping:
    labels:
      - "com.centurylinklabs.watchtower.enable=false"

I'll continue testing, and close in the meantime


Nightly compiler required for: "use { super::{ super::symbol, }, };

This urlo thread urged me to post here. As stated in the thread, I gave rust-analyzer a try and found it to be unstable. Just as one should be expecting from that it is only available through +nightly. Version used: rust-analyzer 8e9ccbf 2022-01-09 dev.
If attempting to navigate the following code with ALEGoToDefinition in vim-ale, one only seems to be able to get as deep as into other_func(). Any attempt to find unknown_func() fails.
src/main.rs
use my_crate::some_func;
fn main() { some_func(); }

src/lib.rs
mod some_module {
    pub fn unknown_func() {
        println!("This function runs successfully, but is invisible to rust-analyzer!");
    }
}

mod other_module {
    pub mod some_submodule {
        use { super::{ super::unknown_func, }, };

        pub fn other_func() { unknown_func() }
    }

    pub use some_submodule::*;
}

use { some_module::unknown_func, other_module::other_func, };
pub fn some_func() { other_func(); }

Running ALEInfo shows the following error message: (no attempts to study nor to understand the logs in full have been done)
error: the option 'Z' is only accepted on the nightly compiler

Removing braces makes rust-analyzer able to resolve the symbol, but that is easier to do in this reduced example than the real codebase I encountered the problem in.

Running ALEInfo shows the following error message: (no attempts to study nor to understand the logs in full have been done)
error: the option 'Z' is only accepted on the nightly compiler

Does it show anything before that line? Do you have any configuration that passes an unstable flag to rustc? AFAIK rust-analyzer doesn't pass any -Z flags to rustc ever unless you tell it to. The configuration could come from for example a .cargo/config.toml file. Also does cargo check --workspace --all-targets work?

Removing braces makes rust-analyzer able to resolve the symbol, but that is easier to do in this reduced example than the real codebase I encountered the problem in.

I am honestly surprised that that code is valid rust. Guess I learnt something new about rust. It should still be fixed in rust-analyzer though.

Ooops, sorry! It seems I was a bit quick jumping to conclusions. That actual error message was unrelated, and caused by me failing to disable other linters. Once running in a more clean environment, the only thing showing up in the log is rust-analyzer launching. Finding unknown_func() still fails.
% cargo clean; cargo check --workspace --all-targets; echo $?
    Checking my_crate v0.0.1 (/my_crate)
    Finished dev [unoptimized + debuginfo] target(s) in 1.20s
0

I take it you are able to reproduce?

Can reproduce that we fail to resolve if the use tree looks like given in your snippet
use super::super::unknown_func; // makes resolution work in the snippet
use { super::{ super::unknown_func, }, }; // fails resolution in the snippet


declare class table split

Pull Request type
Please check the type of change your PR introduces:

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no API changes)
[ ] Build-related changes
[ ] Documentation content changes
[ ] Other (please describe):

What is the current behavior?

Issue Number: N/A
What is the new behavior?

Does this introduce a breaking change?

[ ] Yes
[ ] No

Other information

This change is

Current dependencies on/for this PR:

main

PR #668   👈

This comment was auto-generated by Graphite.


Resigning message with cloned cert removes signature and certificate completely

Hi,
Love this extension! So here's the scenario: I've got a SAMLResponse that's being forwarded to target server, and I want to resign this response using my cloned certificate to see if the server actually validates the response. I use SAMLRaider to clone the certificate and that works well.
When using the "(Re-)Sign Message" the signature and certificate is removed from the request instead of replacing the existing signature, leaving me with a SAML response that has neither a certificate nor a signature. Running burp 1.7.23 on windows 10 x64, using the extension from bapp store.
See illustrations:
Before using the "(Re-)Sign Message" functionality:

After using it (sensitive fields in SAMLResponse grayed out):

Hi @egilas
Do you still have this problem? Do you have this problem with all SAML Messages? Is the whole SAML Message or only the SAML Assertion signed in it's original state? Can you provide me such a SAML Message so that I can reproduce it (you can replace the sensitive content with something else)?
Best regards,
Emanuel

I did not get an answer for further investigations. I'll close the issue.


[Doc enhancement] [Update calls meetings bots] [2372138]

added a ink to Teams recording policy

Acrolinx Scorecards
We currently enforce a minimum score of 80.
Click the scorecard links for each article to review the Acrolinx feedback on grammar, spelling, punctuation, writing style, and terminology:

Article
Score
Issues
Scorecard
Processed

msteams-platform/bots/calls-and-meetings/calls-meetings-bots-overview.md
86
27
link
:white_check_mark:

More info about Acrolinx
Use the Acrolinx extension, or sidebar, in Visual Studio Code to check spelling, grammar, style, tone, clarity, and key terminology when you're creating or updating content. For more information, see Use the Visual Studio Code extension to run Acrolinx locally.

Docs Build status updates of commit 5537ce0:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/bots/calls-and-meetings/calls-meetings-bots-overview.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit b30d75a:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/bots/calls-and-meetings/calls-meetings-bots-overview.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Acrolinx Scorecards
We currently enforce a minimum score of 80.
Click the scorecard links for each article to review the Acrolinx feedback on grammar, spelling, punctuation, writing style, and terminology:

Article
Score
Issues
Scorecard
Processed

msteams-platform/bots/calls-and-meetings/calls-meetings-bots-overview.md
86
27
link
:white_check_mark:

More info about Acrolinx
Use the Acrolinx extension, or sidebar, in Visual Studio Code to check spelling, grammar, style, tone, clarity, and key terminology when you're creating or updating content. For more information, see Use the Visual Studio Code extension to run Acrolinx locally.


[implementing-a-choice-exception-strategy] Issues

Description:
https://github.com/wso2/integration-studio-examples/tree/master/migration/mule/implementing-a-choice-exception-strategy
README.md:
Set up and run the example

The URLs should be changed to http://localhost:8290 instead of 8090.
Once the following URL is called > http://localhost:8290/file?filePath=invalidPath
getting the following response.

[2020-05-04 20:29:20,316] ERROR {org.wso2.carbon.connector.FileRead} - Invalid absolute URI "file://invalidPath".
[2020-05-04 20:29:20,317] ERROR {API_LOGGER.ReadCSVFileAndResponseJSON} - Invalid absolute URI "file://invalidPath".
[2020-05-04 20:29:20,330]  WARN {API_LOGGER.ReadCSVFileAndResponseJSON} - Executing fault handler due to exception encountered
[2020-05-04 20:29:20,331]  WARN {API_LOGGER.ReadCSVFileAndResponseJSON} - ERROR_CODE : 0 ERROR_MESSAGE : Error occured in the mediation of the class mediator
[2020-05-04 20:29:20,331]  WARN {API_LOGGER.ReadCSVFileAndResponseJSON} - Executing fault sequence mediator : org.apache.synapse.mediators.base.SequenceMediator

When it is invoked http://localhost:8290/file?filePath=C:/input.csv, getting the following response.
Actual:

[{"orderId":null,"name":"name","pricePerUnit":null,"units":null},{"orderId":1,"name":"T-shirt","pricePerUnit":25,"units":2},{"orderId":2,"name":"Jacket","pricePerUnit":40.5,"units":3}]

Expected:
	[
		  {
		    "orderId": "1",
		    "name": "T-shirt",
		    "pricePerUnit": "25.0",
		    "units": "2"
		  },
		  {
		    "orderId": "2",
		    "name": "Jacket",
		    "pricePerUnit": "40.5",
		    "units": "3"
		  }
		]

we can fix this issue by enabling long path in windows,
https://superuser.com/questions/1119883/windows-10-enable-ntfs-long-paths-policy-option-missing


Feature request: hide group if it doesn't contain any mappings

Hi, first of all, thanks for this awesome plugin, it rocks!
It would be awesome if groups that have no mapping associated to it in the current buffer didn't show up. My use case is mappings for LSP functions, which are only set when an LSP server is attached, so the +lsp group would only show up in buffers that have attached an LSP server, and not in others.

If you only register those group on those buffers, then the +lsp menu won't show up in any other buffers. Did you do it that way? or did you register that group globally?
I do it the same way for my LSP config.

What I was doing was to declare all group names beforehand and then modularly registering mappings. I thought the group would only show if there were at least one mapping attached to it.
Indeed, registering the groups on demand works, but once the buffer which uses LSP is loaded, the group is always there, even for buffers that make no use of LSP. I don't know if it's even possible to show per-buffer groups, but that was what I was trying to achieve.
What do you think?

pass buffer=BUFNR as an option to register. Below you can find the code I'm using in on_attach
function on_attach(client, bufnr)
   local keymap = {
    c = {
      name = "+code",
      r = { "<cmd>lua require('lspsaga.rename').rename()<CR>", "Rename" },
      a = { "<cmd>lua require('lspsaga.codeaction').code_action()<CR>", "Code Action" },
      d = { "<cmd>lua require'lspsaga.diagnostic'.show_line_diagnostics()<CR>", "Line Diagnostics" },
      l = {
        name = "+lsp",
        i = { "<cmd>LspInfo<cr>", "Lsp Info" },
        a = { "<cmd>lua vim.lsp.buf.add_workspace_folder()<CR>", "Add Folder" },
        r = { "<cmd>lua vim.lsp.buf.remove_workspace_folder()<CR>", "Remove Folder" },
        l = {
          "<cmd>lua print(vim.inspect(vim.lsp.buf.list_workspace_folders()))<CR>",
          "List Folders"
        }
      }
    },
    x = {
      s = { "<cmd>Telescope lsp_document_diagnostics<cr>", "Search Document Diagnostics" },
      w = { "<cmd>Telescope lsp_workspace_diagnostics<cr>", "Workspace Diagnostics" }
    }
  }

  wk.register(keymap, { buffer = bufnr, prefix = "<leader>" })
end

This should only show the +lsp group for buffers with connected clients

Thank you @folke, that way it works as I was expecting, without introducing complexity to the plugin's side.


Update builtins.md

Preview
What are you changing in this pull request and why?
builtins is a variable rather than a function. But it does contain references to the builtin dbt context methods:

ref
source
config

Checklist

[x] Review the Content style guide and About versioning so my content adheres to these guidelines.

nonblocker quesiton @dbeatty10 - does builtins.ref() function  need to say variable or function? prob function because ref in involved? added it as a comment in the deploy preview

does builtins.ref() function need to say variable or function? prob function because ref in involved?

yep, function 👍
Thanks for reviewing and deploying this 🙌


Memoize text width

Summary
The formatter processes strings a couple of times:

propagate_expands: Test if the string contains a newline and, if so, set mode: Propagated on all enclosing groups
print_text: Measures the width of the text to compute the line_width
fits_text (only if the text is inside of a group): Measures the width of the text to determine if the content fits. Runs n times where n is the number of parent groups.

This PR adds a TextWidth field to all text FormatElements that either stores the computed width or that it is a multiline string.
This reduces the traversal for non-multiline strings to exactly once. Multiline strings still get traversed multiple times and you could argue this PR makes it worse because the implementation now computes the text width only to throw it away when realizing it is a multiline string. However, multiline strings are rare.
Memoizing the text width improves performance by 2-12%.
The downsides of this change are:

It will make it harder to add more fields to Text FormatElements because we are now very close to the 24 bytes
The same logic is now spread in multiple places, increasing the risk of one-off bugs

Test Plan
The tests are currently failing because I need to update them to pass TextWidth

Current dependencies on/for this PR:

main

PR #6547

PR #6548

PR #6549

PR #6550

PR #6552   👈
PR #6551

This comment was auto-generated by Graphite.

Sorry i had missed this PR.
I'll try to trigger codspeed for perf numbers.

Sorry i had missed this PR.
I'll try to trigger codspeed for perf numbers.

Code speed only comments on regressions but you can see the results in the run summary https://github.com/astral-sh/ruff/pull/6552/checks?check_run_id=16438240046

3-12%, that's huge!

 3-12%, that's huge!

The win is bigger for formats that
a) Use more groups or best fitting elements because the printer has to measure fits for every group (there's some optimisation but the worst case scenario is that it computes it for every group)
b) Code that has many lines exceeding the line width (with nested groups), because the printer than has to measure fits for every inner group


Best practice of using nexrender concurrently?

Hi All
I am trying to render tens of thousands videos(15-30s) by using nexrender-cli concurrently .
But i am facing these problem:

After Effects often crashes in my macbook pro
Always meet this error
[Bu7XfSmOub2Ou-7_qp3EG] [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fbb0ca04200] moov atom not found [Bu7XfSmOub2Ou-7_qp3EG] /Users/edisonchow/ae/Bu7XfSmOub2Ou-7_qp3EG/result.mov: Invalid data found when processing input

Is there anyone else using nexrender like me ? Or is there a best practice of using nexrender concurrently?
Thanks for ur help

Depends on what you are trying to achieve.
Rendering tens of thousands of videos at the same time on a macbook pro, not likely to happen as there will not be enough resources to run so many instances of After Effects at the same time.
Rendering tens of thousands of videos one after another, possible with nexrender-server + nexrender-worker, just need a little bit more setup.


Better support for Java Optional and List

Mergify complains: "GitHub error message: Merge commits are not allowed on this repository."


add pyi_generator script

All Submissions:

[x] Have you followed the guidelines stated in CONTRIBUTING.md file?
[x] Have you checked to ensure there aren't any other open Pull Requests for the desired changed?

Type of change

[x] New feature (non-breaking change which adds functionality)

New Feature Submission:

[ ] Does your submission pass the tests?
[x] Have you linted your code locally prior to submission?

Description
Add a script to generate .pyi files which can be used by editor like VSCode, PyCharm, etc..
To suggest all the known arguments when initializing a component.

This is awesome! BTW are there any existing "pyi" generators out there already?  If there are, where do they fall short of our needs?

This is awesome! BTW are there any existing "pyi" generators out there already? If there are, where do they fall short of our needs?

mypy provide stubgen which generate a stub based on a existing file.
The reason we need our own script is because we want to enrich the signature of the create method for all components.
The script scan all the components, collect the attributes and add them.
(I've actually used stubgen to generate reflex.vars.pyi, then edited it manually after it to make it work.)


when receiving a GOAWAY, allow earlier streams to still process

This is a first step to better GOAWAY support (cc #83). This patch will close all streams greater than the last stream ID in the GOAWAY frame, but allow streams less than to still finish.

Looks like this broke h2spec, expecting the connection to have closed, so I'll need to add the second step into this PR as well: detecting when the streams are finished and return the goaway + close.

LGTM so far (minus h2spec failure)

Updated to now look for when all active streams have finished, and responds with a new GOAWAY and shuts down.