Double byte characters are printed twice

Like this:
$ console echo いろは123
いいろろはは123

I guess this is caused by a recent commit on this year. Didn't happen previously.

I think you mean double-width characters rather than double-byte characters.  The first three characters are three bytes in UTF-8.  (Note to myself, mostly: The first three characters are: U+3044 U+308D U+306F.)
The problem originally didn't reproduce for me, but I managed to reproduce it by switching my system locale to "Japanese (Japan)."  To do this, open the Control Panel, search for "Region" or "Region and Language" and open the Administrative tab.  Then, in the "Language for non-Unicode programs" box, change the system locale to Japanese.
I noticed that after doing this, the available fonts for the console change.  With Windows 7 English, the options were:

Consolas
Lucida Console
Raster Fonts (aka System?)

With a Japanese locale, the options are:

MS ゴｼツヶ (I think I got that right?  I used shapecatcher.com to recognize the Katakana...)
"Raster Fonts".

However, winpty still successfully changes the font to Lucida Font, even though it isn't an available option.  I tested using WINPTY_SHOW_CONSOLE=1 and manually changed the font to one of the other options, and winpty still duplicated the double-width characters.
With my original system locale of English, the Japanese characters were single-width in the Windows Console.  After changing the system locale, the characters become double-width, even with Lucida Console, where they're rendered as a featureless box.
The obvious guess is that double-width characters take up two console cells, so Windows reports the same Unicode codepoint for each cell.  That could create a problem (i.e. how does winpty detect two Japanese characters in the English locale versus a single Japanese character in the Japanese locale?).  I'll need to do some more investigating.

The TrueType Japanese font is known as MS Gothic.

I think you mean double-width characters rather than double-byte characters.

Nevermind.  According to https://msdn.microsoft.com/en-us/library/cc194788.aspx, these ideas are equivalent in a double-byte character set, such as Shift-JIS.
The commit that broke things was probably https://github.com/rprichard/winpty/commit/72557cb8c45ddec7a4bbf15a8d32689eda156c47.
The fix will probably involve examining the COMMON_LVB_LEADING_BYTE and COMMON_LVB_TRAILING_BYTE console attributes.

Thanks, it looks your guess is correct. The following patch did it here (Win10 and CP932 codepage):
diff --git a/agent/Terminal.cc b/agent/Terminal.cc
index d1cdfd4..5b0622a 100644
--- a/agent/Terminal.cc
+++ b/agent/Terminal.cc
@@ -143,6 +143,8 @@ void Terminal::sendLine(int line, CHAR_INFO *lineData, int width)
             length = termLine.size();
             m_remoteColor = color;
         }
+        if (lineData[i].Attributes & COMMON_LVB_TRAILING_BYTE)
+            continue;
         // TODO: Is it inefficient to call WideCharToMultiByte once per
         // character?
         char mbstr[16];

Hmm, it seems that this patch is not enough.
This patch indeed fixes THIS issue (kanji characters aren't repeated anymore), but I can observe subtle out of sync in console screen.
One apparent issue is a character "ー" (U+30FC) .
This character is a Japanese punctuation mark represented in two bytes in Shift_JIS (CP932), and occupies two character cells in ordinary CP932 console environment (with MS Gothic font).
However, it seems that this character takes up only one character cell when Lucida Console is used as console font.
This completely breaks the layout estimation of win32 console application being run and also mintty rendering, where this character is assumed to occupy two character cells.
(Naive non-Unicode programs just assume double-byte characters in DBCS code page should occupy two cells. More sophisticated Unicode based programs like mintty usually use some kind of wcwidth(), which basically returns East_Asian_Width property of a Unicode character. As for U+30FC, it is "Wide").
I could temporary fix this by disabling the call to m_console->setSmallFont() in Agent constructor and let it use MS Gothic in my environment, but apparently it is not a satisfactory thing to do...

I'll probably make the choice of font depend upon the code page.  There's a table in the registry at HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Console\TrueTypeFont.  It's described here.
The code page can change while the console is open (e.g. mode con: cp select=932), so perhaps winpty should regularly check for this and change the font.
I'm currently documenting everything I've learned from studying how the console works with code pages and fonts.  It's full of complications.  For example, ReadConsoleOutputW reads double-width characters as two CHAR_INFO values if you're using a TrueType font, but if you instead use a raster font, then the double-width character is only a single CHAR_INFO value, and the trailing CHAR_INFO values are zeroed out.

One apparent issue is a character "ー" (U+30FC) .
This character is a Japanese punctuation mark represented in two bytes in Shift_JIS (CP932), and occupies two character cells in ordinary CP932 console environment (with MS Gothic font).
However, it seems that this character takes up only one character cell when Lucida Console is used as console font.

Apparently the situation is more complicated than this.
In Raster Font and MS Gothic, U+30FC is always drawn/rendered as full-width.  In Lucida Console and Consolas, the character is always drawn/rendered as half-width.
In terms of the data model (e.g. ReadConsoleOutputW, selection), the character is half-width or full-width depending upon the font size and OS version.  I tested Lucida Console, Consolas, and MS Gothic, with all font sizes between 1px tall and 50px tall.  Results:

Win7: full-width except for these cases, which are half-width:

Lucida Console: 1px, 2px, 3px, 5px, 6px, 7px, 8px, 10px, 11px, 13px, 15px, 16px, 18px, 21px, 23px, 26px
MS Gothic: 3px

Win8, Win8.1, and Win10 legacy mode: full-width except for these cases, which are half-width:

Lucida Console: 1px, 2px, 3px, 5px, 6px, 7px, 8px, 10px, 11px, 13px, 15px, 16px, 18px, 21px, 23px, 26px
MS Gothic: 1px, 2px, 4px, 6px, 8px, 10px, 12px, 14px, 16px, 18px

Win10 new mode: The character is half-width with Consolas and Lucida Console.  It is full-width with  MS Gothic.  This is the only fully correct mode.
(I didn't test any other OS versions.)

winpty currently picks Lucida Console 6px as its first choice, which is in the broken list for all the OS versions I tested.  If I'm careful to use 5px or 7px for MS Gothic, then winpty should handle U+30FC correctly on all OS versions.  If I instead choose 6px MS Gothic, it will fail on Win8 up to Win10 new mode.
In the Windows console properties dialog, when I choose MS Gothic, there is a box listing possible font sizes.  It's interesting that the first font sizes in that box overlap with the sizes that don't work: 6, 8, 10, 12, 14, 16, 18, 20, 24.
This Windows behavior makes no sense to me.  At least the new console in Win10 works.  I wonder if there are other characters with this problem.

@nu774 What version of Windows are you using?

I'm using Windows 10.
And I wasn't even aware of THAT complicated and seemingly buggy situation of OS native console.
I noticed the issue on this character (U+30FC) because it appears quite frequently in Japanese text (katakana) when transliterating foreign words. Every long vowels are transliterated into this character  (for example, console becomes コンソール)

https://github.com/rprichard/winpty/commit/cce293dd40824bb41473e866efcfe75feab99bd3 fixes this issue, but https://github.com/rprichard/winpty/commit/c3999b5043b7d6c2d1b2dd5ac95fc37e778e4647 rewrites the console font configuration to use appropriate TrueType fonts for the CJK code pages (932, 936, 949, 950).


kernel: 4.1.33 -> 4.1.35

Motivation for this change
#19884
Things done

[X] Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)

[X] NixOS
[ ] OS X
[ ] Linux

[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[X] Fits CONTRIBUTING.md.

The other stable kernel PRs don't have the hashes correctly updated.


[WIP] KAFKA-18082: Add 0.11, 1.0, 1.1 and 2.0 to consumer upgrade e2e

related to https://issues.apache.org/jira/browse/KAFKA-18082
Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

client e2e all passed.

Thanks for the PR. My take is that we do not want to do this. We want to take advantage of KIP-896 and have 2.1 as the new baseline. The value over benefit for these is not worth it. cc @chia7712.

Thanks for the PR. My take is that we do not want to do this. We want to take advantage of KIP-896 and have 2.1 as the new baseline. The value over benefit for these is not worth it

The benefit of this change is to ensure that rolling upgrades of the consumer work when the server is running a compatible version (e.g., 3.9). Since streams_upgrade_test.py already includes similar tests, adding tests for the consumer makes sense.
However, I agree that the benefit is minimal, as the use case is rare (e.g., upgrading from 0.11 to 0.4 is highly unusual). Therefore, I prefer to add a new test case (same file) that runs the 3.9 server to cover these rare scenarios. This approach would eliminate the numerous if-else conditions in the environment setup.

Thank you for the comments, and a special thanks to Chia-Ping for helping with the answer!

However, I agree that the benefit is minimal, as the use case is rare (e.g., upgrading from 0.11 to 0.4 is highly unusual). Therefore, I prefer to add a new test case (same file) that runs the 3.9 server to cover these rare scenarios. This approach would eliminate the numerous if-else conditions in the environment setup.

Thanks for the suggestion! I’ll go ahead and extract those conditions into new tests, and if either of you have any further concerns, please feel free to let me know.

@chia7712 Kafka Streams is different - it has to handle its own state during upgrades (kind of like the broker). The same is not true for regular clients. Maintaining tests like this one is pretty painful. Things will eventually break in odd ways (since the 0.11 code is so old). Also, it's so risky to go from 0.11 to 4.x+ in one go given multiple cycles of deprecations, removals, etc.
It's simpler to say that such users should first upgrade to 3.9 and then do a second upgrade to 4.x+. This way you avoid maintaining (and running) these tests forever.

On a related note, I do think Kafka Streams should also trim their upgrade matrix, but I think they'll have to wait to 5.0 for that.

@chia7712 Kafka Streams is different - it has to handle its own state during upgrades (kind of like the broker). The same is not true for regular clients. Maintaining tests like this one is pretty painful. Things will eventually break in odd ways (since the 0.11 code is so old). Also, it's too risky to go from 0.11 to 4.x+ in one go given multiple cycles of deprecations, removals, etc. (including non intersecting supported Java versions).

Sorry to interrupt the discussion and thanks for the insightful comment, I 100% agree we don't encourage user to do major bumps like 0.11 to 4.x.
 If we can't guarantee Kafka Consumer can successfully upgrade from 0.11 to 4.x, how can Kafka Streams guarantee that? Especially this kind of upgrade is still valid until Kafka Streams 5.0.0 comes out?

@brandboat The Kafka Streams upgrade test ensures the end to end behavior is fine (which includes the underlying clients used by Kafka Streams).

which includes the underlying clients used by Kafka Streams

Users might be confused when they see that "Kafka Streams compatibility ranges includes 0.11 to 4.0 and Kafka Consumer requires 0.11 -> 3.9 and 3.9 -> 4.0". Although we can explain that Kafka Streams internally handles consumer compatibility. However, if there is indeed no known compatibility issue - it seems to me providing a clear and straightforward compatibility matrix #18091 along with comprehensive compatibility tests would better encourage users to upgrade to version 4.0

@chia7712 the number of users looking to upgrade from 0.11 to 4.0 is likely 0. Such users are going to take years before they upgrade to 4.0. So, I think we'd be optimizing for the wrong thing.

the number of users looking to upgrade from 0.11 to 4.0 is likely 0. Such users are going to take years before they upgrade to 4.0. So, I think we'd be optimizing for the wrong thing.

Yes, I agree. Should we ensure that all clients (Consumer, Streams, Connect) follow a consistent upgrade path? It might be difficult for users to determine the correct sequence. For example, users with applications utilizing both Streams and Consumers may be unsure whether they should first upgrade both jars to version 3.9 and then proceed to 4.x+, or follow the Streams compatibility matrix to upgrade directly to 4.x+

Yes, that would be simplest. That said, we should be clear that Connect is more of a server component (i.e. it doesn't run alongside customer applications) even though it communicates with the brokers as regular clients do.
Perhaps we should start that discussion with the Streams folks again. I want to check the specifics of the Streams upgrade test as it seems pretty complicated (https://github.com/apache/kafka/pull/17876/files).

@mjsax Apologies for bringing up this issue again. I believe that having clear and consistent rolling-upgrade rules can encourage users to adopt version 4.0, which is why I am persistent about this issue.


clarify that front matter does not support template syntax

Attempt at clarifying documentation so that it is explicit that:

front matter does not support template syntax
permalink as a front matter key is the sole exception

see https://github.com/11ty/eleventy/issues/135#issuecomment-391937032
see https://github.com/11ty/eleventy/issues/315

Love it! Thank you!


[BUG] Management status is NotReady after power failure

Describe the bug
Harvester 1.2.0 was functioning correctly until a power outage occurred. Following the server's recovery, the management status displays as "NotReady," while the node status remains in the "Ready" state. Attempting to access the management URL results in an "ERR_ADDRESS_UNREACHABLE" error.
To Reproduce
Steps to reproduce the behaviour:

Install Harvester 1.2.0 on a server and ensure it's functioning correctly.
Simulate a power failure or sudden shutdown of the server.
After the power failure, restart the server or bring it back online.
Check the management status, which should now display as "NotReady."
Verify that the node status remains in the "Ready" state.
Attempt to access the management URL, and observe the "ERR_ADDRESS_UNREACHABLE" error.

Expected behaviour
In the event of a power failure or sudden server shutdown, the system should gracefully handle the interruption. After the server is restarted or brought back online, the management status should automatically return to "Ready."
Support bundle

Environment

Harvester ISO version: 1.2.0
Underlying Infrastructure (e.g. Baremetal with Dell PowerEdge R630): Baremetal custom build running on Asus B760M-Plus and Intel i5 13500.

Additional context
Running kubectl get pods -n kube-system, the following pod statuses are observed:
- harvester-whereabouts-l9fwf is in a CrashLoopBackOff state.
- rke2-canal-sfzdz is in an Init:CrashLoopBackOff state.

When examining the logs with kubectl logs -n kube-system harvester-whereabouts-l9fwf, the following error is encountered:
Error from server: Get "htts://<IP_ADDRESS>:10250/containerLogs/kube-system/harvester-whereabouts-l9fwf/whereabouts": proxy error from <IP_ADDRESS>:9345 while dialing <IP_ADDRESS>:10250, code 503: 503 Service Unavailable
Thank you!

Please add more details:
1. Is this single-node cluster or multi-node cluster
2. The CPU core, memory and disk size of each NODE is this cluster
3. kubectl get pods -A

Please add more details:
1. Is this single-node cluster or multi-node cluster
2. The CPU core, memory and disk size of each NODE is this cluster
3. kubectl get pods -A

It is a single-node cluster.
The i5 13500 is a 13500 core, it has 64 GB of memory and Harvester is installed in a 256 GB NVME SSD and data is using a 1TB SSD.

I'll post kubectl get pods -A when I get the chance because I can't ssh into the server.

Here it is

Did you use dhcp or static for the NODE IP ? Both NODE IP and VIP can't change, you need to make sure the DHCP server can offer the same IP in case it is used, normally via IP/MAC binding.

Please check this file:  /oem/90_custom.yaml, there will be some contents as below, get the values of BOOTPROTO.
    - path: /etc/sysconfig/network/ifcfg-mgmt-bo
      permissions: 384
      owner: 0
      group: 0
      content: |+
        STARTMODE='onboot'
        BONDING_MASTER='yes'
        BOOTPROTO='none'
        POST_UP_SCRIPT="wicked:setup_bond.sh"

        BONDING_SLAVE_0='ens3'

        BONDING_MODULE_OPTS='miimon=100 mode=active-backup '

        DHCLIENT_SET_DEFAULT_ROUTE='no'

    - path: /etc/sysconfig/network/ifcfg-ens3
      permissions: 384
      owner: 0
      group: 0
      content: |
        STARTMODE='hotplug'
        BOOTPROTO='none'
      encoding: ""
      ownerstring: ""

I just booted the server and it was able to pick up the same IP as before. So it's working. Here's configuration:
cat /oem/90_custom.yaml
    - path: /etc/sysconfig/network/ifcfg-mgmt-bo
      permissions: 384
      owner: 0
      group: 0
      content: |+
        STARTMODE='onboot'
        BONDING_MASTER='yes'
        BOOTPROTO='none'
        POST_UP_SCRIPT="wicked:setup_bond.sh"

        BONDING_SLAVE_0='eno1'

        BONDING_MODULE_OPTS='miimon=100 mode=active-backup '

        DHCLIENT_SET_DEFAULT_ROUTE='no'

      encoding: ""
      ownerstring: ""
    - path: /etc/sysconfig/network/ifcfg-eno1
      permissions: 384
      owner: 0
      group: 0
      content: |
        STARTMODE='hotplug'
        BOOTPROTO='none'
      encoding: ""
      ownerstring: ""
    - path: /etc/sysconfig/network/ifcfg-mgmt-br
      permissions: 384
      owner: 0
      group: 0
      content: |+
        STARTMODE='onboot'
        BOOTPROTO='dhcp'
        BRIDGE='yes'
        BRIDGE_STP='off'
        BRIDGE_FORWARDDELAY='0'
        BRIDGE_PORTS='mgmt-bo'
        PRE_UP_SCRIPT="wicked:setup_bridge.sh"
        POST_UP_SCRIPT="wicked:setup_bridge.sh"

        DHCLIENT_SET_DEFAULT_ROUTE='yes'

      encoding: ""
      ownerstring: ""

sudo cat /oem/harvester.config
install:
  automatic: false
  mode: create
  managementinterface:
    interfaces:
    - name: eno1
      hwaddr: 08:bf:b8:35:04:40
    method: dhcp
    ip: ""
    subnetmask: ""
    gateway: ""
    defaultroute: true
    bondoptions:
      miimon: "100"
      mode: active-backup
    mtu: 0
    vlanid: 0
  vip: <IP_ADDRESS>

Also, this is a brand new installation so I'm ok with wiping it and installing it again if it's just easier to fix it

@andrewdafonseca What do you mean, the cluster is in running state now ?
If not, please get more information: https://github.com/harvester/harvester/issues/4593#issuecomment-1757649291, thanks.

Good, then you may close this issue, thanks.

Thanks for all the help


Feature Request: Hotkey to pause/stop video

Description of issue / Feature request
I'm really enjoying Aerial so far - I have it set up on my Late 2013 MBPr hooked up to an external 4k monitor in clamshell mode. Unfortunately, any time Aerial is running, GPU usage goes up and so the fans spin up. As soon as I log in and start just using a browser/regular apps, the fans turn off.
The feature I'd like to request is a hotkey toggle that pauses the video on the screensaver to stop using the GPU. Or, if there is a way to detect if the monitor is off via DDC/CI (ddcctl) that would be cool to integrate too!

Hey @klamike
A few things :

So regarding hotkey, sadly that's not possible, screensavers are no longer allowed (post Catalina) to intercept key presses. That may come back some day, hopefully

Regarding monitor off, are you saying you are manually turning it off, and Aerial keeps running ? In general I would suggest you use the power saving preferences where you can set for how long your screensaver should run. This way you can at least limit it a bit (that slider) :

I do believe that Aerial gets notified if a screen gets connected/disconnected, though I haven't played much with that, there may be something I can do around that, I'll make a note to investigate.

Regarding CPU/GPU usage, on a 2013 MBP, off the top of my head, you don't have hardware decoding on the Intel IGP for HEVC decoding, so 4K HEVC (and even 1080p HEVC) will be a bit taxing on your CPU, more than the GPU. If you use 4K HDR however, you're adding some GPU load because of the Dolby Vision color processing. 1080p H.264will not spin your fan but that's not really a satisfactory solution, that's just a limitation of those Mac generations.

Let me know if that clears things up.

Thanks for the quick reply! I have it set to 4K HEVC and this is what the usage graphs look like through Activity Monitor:

You can clearly see when the screensaver is on/off from the GPU graph. Looks like the CPU usage also goes up above background levels (a few apps open, not streaming any video) when Aerial is on.
Unfortunate that screensavers can't intercept key presses. Maybe a triple mouse click or something can work?

As far as I know the decoding is always done on the CPU, so that's probably what you are hearing, the GPU usage is activity monitor is not always representative (it's not doing 3D rendering) of a "tough" load. What it does here is just rendering the video (with some 10 bit processing but not very hard).
You can double check for yourself playing one of the videos from the cache, directly in Quicktime. Aerial uses macOS's system decoding APIs so what you get in Aerial and Quicktime should be 100% similar. I would guess CPU temperature is what makes the fan kicks in but I can be wrong.
Mouse is clamped down as much as keyboard, there's really no way to interact with a running screensaver nowadays.
You didn't answer my question regarding the monitor thing btw.

Oh sorry about that - Yes it does appear that Aerial is still running since the fans don't turn down when the monitor is off.
I just verified by looking at the usage graphs as well which tell the same story - turning off monitor has no affect on GPU or CPU load.

Ok then.
I'll keep this open and try to have a look if I can detect a monitor off event and do something about it. In the meantime I suggest you clamp down the running time with the setting above.
I'll try to have a look this week but no promise :)

Awesome, thanks again for the help and quick response!

@klamike
Ok so, had some time this morning to look at this, and unfortunately it's not really great news :

First, I did a quick check and here at least, I don't get any event from macOS's display APIs when I power off/standby my external monitor. What I get is events for plugged/unplugged monitor. A standbyed monitor is still seen as connected by macOS, and thus it doesn't stop rendering stuff there. So, no easy fix this way
I did look at ddcctl, and it flat out doesn't work with my external monitor on my old Intel iMac :

$ ./ddcctl -d 1 -p ?
D: CGDisplay CD30F1A3-6617-FB04-0D8C-5C0EA5A6B477 dispID(#441231492) (3008x1692 0°) HiDPI
I: found 1 external display
I: polling EDID for #1 (ID 441231492 => IOService:/AppleACPIPlatformExpert/PCI0@0/AppleACPIPCI/PEG0@1/IOPP/GFX0@0/ATY,Labrador@4/AMDFramebufferVI)
E: Failed to poll display!

Now I can't totally blame the project, this is a weird screen I'm using (an Asus PQ321, one of the first 4K DP display that only used one cable, doing MST). It's very likely that the firmware on this thing is borked. While it's connected straight with a mini DP to DP cable (no dongle which is the issue most of the time for that error), it still errors on me which makes it hard to even implement something like this.

I also did try many ioreg commands and I don't get anything meaningful regarding power states. Maybe I'm looking at the wrong one but running :

ioreg -c IODisplayConnect -b -f -r -l -i -d 2
I always get state 3, whether it's on or off :
"IOPowerManagement" = {"DevicePowerState"=3,"CurrentPowerState"=3,"CapabilityFlags"=49152,"MaxPowerState"=3}

Could you try the command above, with your screen on and off, and see if it reads the power state ?

But to me the biggest issue is this will not work on M1, at least for the time being : https://github.com/kfix/ddcctl/issues/86#issuecomment-734523004

So I'm very reluctant to work on something that will only work for some, and won't work going forward unless Apple reverts their decision on not implementing I2C on arm. This is not really satisfying, I would have loved to also use this to control brightness on external displays but if it doesn't work on arm, there's little point wasting efforts on it.

Totally understand that it's not worth the time if Apple is going to make it obsolete soon anyway, especially since this is a pretty niche feature too.
I tried the commands and it looks like ddcctl does recognize the power state for my monitor:
Power on:
$ ddcctl -d 1 -p ?
D: NSScreen #1007443977 (1920x1080 0°) HiDPI
I: found 1 external display
I: polling display 1's EDID
I: got edid.name: HP Z27s
I: got edid.serial: redacted
D: action: p: ?
D: querying VCP control: #214 =?
I: VCP control #214 (0xd6) = current: 5, max: 4

Power off:
$ ddcctl -d 1 -p ?
D: NSScreen #1007443977 (1920x1080 0°) HiDPI
I: found 1 external display
I: polling display 1's EDID
I: got edid.name: HP Z27s
I: got edid.serial: redacted
I: got edid.name:
I: got edid.serial: äþ
D: action: p: ?
D: querying VCP control: #214 =?
E: No data after 10 tries!
E: DDC send command failed!
E: VCP control #214 (0xd6) = current: 0, max: 0

My output for ioreg is the same as yours - the power state is 3 regardless.
Do you have a link for me to buy you coffee? Really appreciate the time you've been putting in on this issue and I'm loving Aerial in general.

Ha interesting, so ioreg wasn't what I was looking for.
The way ddcctl works is a bit weird here though, basically the command fails when you power off, you don't get a real reply (that may happen in other situations too, so maybe not the best thing to poll regularly...). The return code for off should be 2 or 1 apparently (depending on what they call standby and poweroff)
I'm wondering though if all monitors work the same way here, of if this depends on the type of power off. It's not fully off since apparently it replies to the EDID polling, but not the DCC commands. Do you happen to have two power off buttons (like a "soft" power off that puts it on standby, and maybe another one at the back or something ? Just curious on how this is supposed to work.
If you feel adventurous, maybe try powering it off with :
$ ddcctl -d 1 -p 2
or maybe
$ ddcctl -d 1 -p 1
Does it look "off" the same way that it does when you do it by pressing whatever button ?
Still if there's a reliable way to do this in the future (and also control brightness for external monitors) I'd love to add it, as niche as it is. Aerial is already full of niche stuff ;)
Regarding coffee, no need but if you really want to, there's a link in the about box of Aerial, it goes here : https://ko-fi.com/glouel

There are two buttons - one on the back to cut power and one on the front to standby.
$ ddcctl -d 1 -p 1 makes it blink off then back on
$ ddcctl -d 1 -p 2 makes it turn off like pressing the front button (front led is goes from white to orange), can't wake with mouse/keyboard input (need to press the front button twice to turn the monitor off fully then back on again)
$ ddcctl -d 1 -p 3 same as 2 as far as I can tell
$ ddcctl -d 1 -p 4 makes it turn off completely (front led goes off too), to turn back on just press the front button once.
$ ddcctl -d 1 -p 5 same as 4 as far as I can tell
Over 6 does nothing. Hope this helps, I realize it's super weird behavior...

Ok great. According to your previous test, 5 should be on, so I guess maybe your monitor doesn't handle that command in a write fashion correctly (it kinda reads in reverse, like 1 is on and it goes "more" off as you go down, but that goes against your previous test.
I did try powering up my PC but there doesn't seem to be an equivalent for windows and I don't have a linux box with a monitor connected at the moment so I can't double check, I'll see if I can try something else, this is mostly out of curiosity at this point.
But just to confirm, when you powered it off on your test in the previous message, did you press the front button, or the back button ?  I would assume that you did front ?

Yep just the front.


[MonorepoBuilder] Fix autoloading for overlapping namespaces

There is a problem when two packages are using namespaces like App and App\Foo.
Currently only the namespace itself is checked to be contained within a composer.json, so when you have a namespace App it will be found in each composer.json using App\<whatever> and the path of the last matching composer.json will be taken.
I tried to fix this by adding quotation marks for the lookup, so partial matches won't be successful anymore.

Hey, thanks for nice and clear PR.
Please add new test method instead of overriding old test, e.g. testOverlapingPaths to make clear they both work.

@TomasVotruba Thanks for your quick response. I reverted the previous commit and made a new one.
I modified the getRelativeSourcePath helper method, so I don't have to duplicate this part for another source directory.
Should I squash the commits?

I see your comment now.
I'm such a case a new method with overlapping source

You can squash them in the end. Thanks for such a high quality approach :+1:

Now I am using a dedicated method for the overlapping source path.
I'm not sure if I like it since now there is duplicated logic but I can't remove this without touching the method. ;-)

Thank you :+1:
Well, not everything duplicated have to be in separated method. I prefer readability and code that is chilly to read over purism and complicated constructions.

You can squash it and it's ready to merge

Btw, have you tried that command on this monorepo package? Is the result the same?

Ok, I squashed the commits now.
To be honest, I haven't tried to run the merge command before, but now I have and there are no changes.

Let's :ship:  it !

Thank you very much and many thanks for this great tool. :bouquet:

Thanks for nice word. Glad you enjoy it :+1:
Do you have it in some open-sourced packages? I'd like to see it in action if you do

Not yet, just some private experimental stuff where I used to have many single repositories and I got overwhelmed by the management overhead. I will let you know when or if it will ever go public. ;-)

Awesome :+1:  Look forward to it


Checkpoint, using vocabulary identifiers instead of raw labels

This isn't done, opening PR for diffing purposes.

Going to merge this to deploy it out to Henry to test against the iSB instances there.


Overriding variadic function to remove all extra parameters now fails to size parameters correctly

Describe the bug
Since commit 7891d2611, correcting parameters to variadic functions is not interpreting them correctly under apps using x86-16.cspec.
Eg. on opening my project and looking at a representative function I saw this:
void __cdecl16far Lm_DecayAdjust(long param_1)
{
   int unaff_BP;
   int iVar1;
   DgnLanguageModel_0xc2_t *pDVar2;

   iVar1 = unaff_BP + 1;
   pDVar2 = GetAllowableLangModel(*(IDX_ONE_BASED *)(param_1 + 6));
   if ((pDVar2 == NULL) || (pDVar2->pLmAdjust_0xbe == NULL)) {
      ReportFailure_0e38("sdlm.cpp",0x3b9,"No valid LM Adjustment module\n",(int)pDVar2,
                         (int)((ulong)pDVar2 >> 0x10),iVar1);
   }
   LmAdjust::RecalcDecayAndTieScale(pDVar2->pLmAdjust_0xbe);
   return;
}

after an attempt to fix this by overriding the signature (changing from void ReportFailure_0e38(LPCSTR pszCategory, int iErrorLoc?, LPCSTR pszErrorReason, int, int, int) to void ReportFailure_0e38(LPCSTR pszCategory, int iErrorLoc?, LPCSTR pszErrorReason)) using the same calling convention (__cdecl16far) Ghidra now produces:
void __cdecl16far Lm_DecayAdjust(long param_1)
{
   DgnLanguageModel_0xc2_t *pDVar1;

   pDVar1 = GetAllowableLangModel(*(IDX_ONE_BASED *)(param_1 + 6));
   if ((pDVar1 == NULL) || (pDVar1->pLmAdjust_0xbe == NULL)) {
      ReportFailure_0e38((LPCSTR)0x34f81038,(int)"User_Iterate" + 10,(LPCSTR)0x35da03b9);
   }
   LmAdjust::RecalcDecayAndTieScale(pDVar1->pLmAdjust_0xbe);
   return;
}

It is looking like Ghidra is now treating all 2-byte values as offsets to addresses (relative to the data segment DS)!
To Reproduce
Steps to reproduce the behavior:

Load the unzipped attached dump (from the Decompile menu Debug Function Decompilation
Override the call (as above)
See error

Expected behavior
Recognition of numbers vs pointers and correct interpretation of parameters.
Screenshots
N/A
Attachments
dragon_Lm_DecayAdjust.zip
Environment (please complete the following information):

OS: Windows 11
Java Version: <IP_ADDRESS>
Ghidra Version: [e.g. 10.3-DEV
Ghidra Origin: locally built - commit 7891d2611

Additional context
Add any other context about the problem here.

Found the issue, in class FunctionPrototype.java replace the line:
https://github.com/NationalSecurityAgency/ghidra/blob/8dd584c9bca0fadae91ee0eecbcade178e2c2e1b/Ghidra/Framework/SoftwareModeling/src/main/java/ghidra/program/model/pcode/FunctionPrototype.java#L91
with:
modelname = proto.getCallingConventionName();

Also, I can only assume that maybe the upgrade process isn't quiet correct, as all the function prototypes under auto_proto folder (and elsewhere) have been given the default calling convention __stdcall!

@Wall-AF the zip you supplied only contains an XML file which is inadequate for import.  Could you please provide a GZF export instead.

I rolled back to commit 8242c31c4a8 and also had to zip the gzf!  DRAGON_8242c31c4a8.zip

Could you please confirm that the GZF was produced while in a good state prior to the upgrade.

It appears your gzf has already been subjected to the upgrade.  Could I get a copy prior to the upgrade if at all possible so I can verufy the upgrade did not cause any issue.

Found the issue, in class FunctionPrototype.java

Good find!  Thanks.  Will put fix in.

Mmm, sorry, I don't seem to be able to go back despite taking a copy of the *.rep folder beforehand!

Is there any chance that this would have affected all indirect function calls (to a typed function pointer) as well?

Also, I can only assume that maybe the upgrade process isn't quiet correct, as all the function prototypes under auto_proto folder (and elsewhere) have been given the default calling convention __stdcall!

I would need to see this occur in a sample during upgrade.  I will attempt to reproduce.

Is there any chance that this would have affected all indirect function calls (to a typed function pointer) as well?

This was indeed the cause and 6799c19e19fa28f43fe2e31480a232edc0f74887 did fix it. The function pointer in question had a user defined __gocall convention for go.


Tabular Editor 2.25  complaining missing DLL files

!!! Please make sure you're using the latest version of Tabular Editor 2.X before submitting a bug !!!
!!! For Tabular Editor 3 issues, go here -> https://github.com/TabularEditor/TabularEditor3/issues !!!
Describe the bug
I installed Tabular editor v2.25 from github and  when i am opening the Tabular editor post install it throws error about AMO version 15+ missing DLL . i did check and i have the latest AMO packaged installed which are 16 and above.
I used to have old version of Tabular  that was working before /
To Reproduce
Steps to reproduce the behavior:

Download tabular 2.25 from github

open the report

See error

Expected behavior
A clear and concise description of what you expected to happen.
Observed behavior
A clear and concise description of what happened. Include any error messages or stack traces if possible. Also, feel free to add screenshots or recordings showing the problem.
Application specifics

Tabular Editor 2.X version: [e.g. 2.12.0]
Semantic Engine used: [e.g. Power BI Desktop, Azure Analysis Services, SSAS 2019, etc.]
Model Compatibility Level: [e.g. 1400, 1500, 1520, etc.]

Additional context
Add any other context about the problem here.

Did you use the Portable version or the Installer version?


🐛 [firebase_auth] Can not login twitter in web

Bug report
Describe the bug
Open the popup with FirebaseAuth.instance.signInWithPopup(TwitterAuthProvider()) and after a few seconds [firebase_auth/popup-closed-by-user] The popup has been closed by the user before finalizing the operation. error occue
Steps to reproduce

run flutter run -d chrome --web-port=5000 --web-renderer=canvaskit
press login button
wait few second

Expected behavior
E-mail address can be entered without error
Sample project
import 'package:firebase_auth/firebase_auth.dart';
import 'package:flutter/material.dart';

class SamplePage extends StatefulWidget {

  SamplePage(): super();

  @override
  SamplePageState createState() => SamplePageState();
}

class SamplePageState extends State<SamplePage> with WidgetsBindingObserver {

  @override
  void initState() {
    super.initState();
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: Container(
        padding: EdgeInsets.only(top:50,left:50),
        child: ElevatedButton(
          child   : Text("twitter login"),
          onPressed: (){
            print("000");
            FirebaseAuth.instance.signInWithPopup(TwitterAuthProvider()).then((_user){
              print("111 $_user");
            }).onError((error, stackTrace){
              print("222 $error");
            }).catchError((_error){
              print("3333 $_error");
            });
          },
        ),
      ),
    );
  }
}

Additional context
I took a video, but it was too big, so I put it on GoogleDrive
https://drive.google.com/file/d/1GFoTtaBmzBBu-gDPoPrzKC5Blh4MWKTL/view?usp=share_link

Flutter doctor
Run flutter doctor and paste the output below:
Click To Expand
[√] Flutter (Channel stable, 3.3.10, on Microsoft Windows [Version 10.0.22000.1335], locale ja-JP)
Checking Android licenses is taking an unexpectedly long time...[√] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
[√] Chrome - develop for the web
[√] Visual Studio - develop for Windows (Visual Studio Community 2022 17.1.1)
[√] Android Studio (version 2020.3)
[√] VS Code (version 1.74.2)
[√] Connected device (3 available)
[√] HTTP Host Availability

Flutter dependencies
Run flutter pub deps -- --style=compact and paste the output below:
Click To Expand
PASTE OUTPUT INSIDE HERE

Hello @hatano0x06,
It seems that it is only happening in Chrome Debug, if you are opening the URL in your web browser, everything works fine.
Do you see the same thing?

@Lyokone
It's happen everytime
I create url that you can try it
https://storyplotter-cdddb--twitterlogin-report1-4jjs4wcs.web.app/

Thanks it helps! 🙏

After investigation, I've opened an issue in the native JS repository.

@hatano0x06 do you still face this issue?

@Lyokone
No problem occurred!
Thank you very much!
Flutter 3.7.8 • channel stable
firebase_auth: ^4.4.0


Why the mappings in rails is replaced by default setting

I use elasticsearch in my rails application. The following are my settings and mappings:
settings index: { number_of_shards: 5 }, analysis: { analyzer: 'ik' } do
    mappings dynamic: 'strict' do
      indexes :title, analyzer: 'ik', type: :string
      indexes :content, analyzer: 'ik', type: :string
      indexes :likes_count, type: :integer
      indexes :tags do
        indexes :name, analyzer: 'ik', type: :string
      end
    end
  end

When I use Note.import to build index for my  data, I just found the settings and mappings in elasticsearch using the default configuration.
curl -XGET localhost:9200/collections/collection/_mapping?pretty
{
  "collections" : {
    "mappings" : {
      "collection" : {
        "properties" : {
          "content" : {
            "type" : "string"
          },
          "created_at" : {
            "type" : "date",
            "format" : "dateOptionalTime"
          },
          "id" : {
            "type" : "long"
          },
          "image_content_type" : {
            "type" : "string"
          },
          "image_file_name" : {
            "type" : "string"
          },
          "image_file_size" : {
            "type" : "long"
          },
          "image_updated_at" : {
            "type" : "date",
            "format" : "dateOptionalTime"
          },
          "likes_count" : {
            "type" : "long"
          },
          "newly_added_at" : {
            "type" : "long"
          },
          "priority" : {
            "type" : "long"
          },
          "recommended" : {
            "type" : "boolean"
          },
          "slug" : {
            "type" : "string"
          },
          "tags" : {
            "properties" : {
              "name" : {
                "type" : "string"
              }
            }
          },
          "title" : {
            "type" : "string"
          },
          "updated_at" : {
            "type" : "date",
            "format" : "dateOptionalTime"
          },
          "user_id" : {
            "type" : "long"
          },
          "visible" : {
            "type" : "boolean"
          }
        }
      }
    }
  }
}

These properties all use a default setting and mapping configuration. I have tried a lot but fail to deal with it. Could somebody shed some light on this problem?

It's hard to find what is actually to do in this situation. I found in source code, that create_index! without force: true doesn't detelte an existing index, checks if it exists, and then creates new one: https://github.com/elastic/elasticsearch-rails/blob/master/elasticsearch-model/lib/elasticsearch/model/indexing.rb#L227
So, my solution to the problem is to use custom callback, where create_index! being called before every call to index_document:
module Searchable
  extend ActiveSupport::Concern

  included do
    after_commit on: :create do
      self.class.__elasticsearch__.create_index!
      __elasticsearch__.index_document
    end
  end
end

It'll be nice to have some more information about create_index! in README.


Set example links to replace current history item

A very minor improvement which occurred to me mere seconds after merging #130
This makes it so clicking the examples replaces the current history item, which should always be the empty URL with no parameters (and therefore a relatively uninteresting history item). This makes it so clicking the various examples multiple times leads to a much more useful history/back button listing.

Would seem to work a treat. Ship it.


fork() should use cmd.Run() rather than cmd.Start()

func (c *Cmd) Run() error {
if err := c.Start(); err != nil {
return err
}
return c.Wait() // if execute the command fails，return error
}
I use endless in my project. In a production environment, we need to modify the configuration file frequently, and then restart the program for the new configuration to take effect.
But the configuration file often has syntax errors, making the program unable to restart successfully. We need to modify the configuration again and restart again, but endless output "Fork err: Another process already forked. Ignoring this one."

In my scenario, I use'./app -s restart' to restart server, when app get the arg of '-s restart', it's will do

check config file(include format)
get PIDFile in cofig file
When the process pointed to by pid is running, send SIGHUP to it

so I think u should check config file before send restart signal
about fork() use the cmd.Start(), I use it to run a deamon process, so the cmd.Run() nerver return after noew process exit

This is just my suggestion, I hope it will be useful to you


Need Help - Conditional Formatting and Delete Column

Hi, I have an excel with duplicate column header. Import excel doesn't work with duplicate header.
Hence trying to delete the duplicate column and then import excel. How do I achieve this?
Secondly, Conditional Text Format.
have a column with True and False Values. Need Cell Background Green and Red.
We need better example set in help which is not enough self explanatory since I could not understand $worksheet.Names["FinishPosition"].Address .
Also, Export-Excel -ExcelPackage $excel -ConditionalTest $ct -show  ??? what does $excel contain? and what does excelpackage mean. I did google search, still doesn't make sense.
"FYI, Check the typo in example -ConditionalTEST which should be -ConditionalText if I am not wrong"
Can you post me a Simple Example with Get-process | export-excel  -ConditionalText "" -range "C:C"? Where I believe C:C is column C per example in the code.

First part
You can use -HeaderName for that   so
-headerName  firstCol , secondCol , thirdCol, RenamedDuplicate, fourthCol, Fifthcol.
I'll review the help for the -ExcelPackage parameter. The Excel Package is what you get if you Open an XLSx file using the EPPlus library which does all the hard work for the module. So you can do
$excel  = Open-ExcelPackage <<path>>
or
$excel = export-excel -PassThru  xxxx
If  you call Add-ConditionalFormatting you need to give it something to add the formatting to - that's the package
To explain $worksheet.Names["FinishPosition"].Address
$worksheet has a collection of named ranges; these live in .Names, the data here is a list of racing car results, so the $worksheet.Names["FinishPosition"] range named "finish postion" - the car's result in the race $worksheet.Names["FinishPosition"].Address   is the cells of that range.
I'll try to make that example clearer and will check for typos (one more time!)
To Set false red and true green

$ruleForFalse = New-ConditionalText -Text $false -BackgroundColor Red   -ConditionalType Equal  -Range "C:C"
$ruleforTrue  = New-ConditionalText -Text $true  -BackgroundColor Green -ConditionalType Equal  -Range "C:C"

export-excel C:\temp\test.xlsx -ConditionalFormat $ruleForFalse,$ruleforTrue -Show

(That assumes your data is on sheet1 - otherwise you need to specify the worksheet name

Great works perfect. Helps a lot in my daily reporting.
But now having issues with Setting up column Width. Couldnt get the difference between worksheet, worksheetname, workbook , why so many names :( . Not sure how/what to pass with worksheet parameter.
Get-Process | select 'Responding','SessionId','ExitCode','Name' | Export-Excel $ENV:Userprofile\Desktop\Newfolder\prox4.xlsx
$excel = Open-ExcelPackage "$ENV:Userprofile\Desktop\Newfolder\prox4.xlsx"
$ct = New-ConditionalText -Text  'TRUE' -BackgroundColor yellow -Range "A:A"
$ct2 = New-ConditionalText -Text 'chrome' -BackgroundColor Brown -Range "D:D"
Export-Excel -ExcelPackage $excel -ConditionalText $ct,$ct2
Set-ExcelColumn -ExcelPackage $excel  -Column 4 -Width 100

Ok a workbook is an XLSx file; a workbook has 1 or more page(s) in it worksheet(s).
An Excel package is container which holds one workbook and has methods like "save". You get a workbook with Open-ExcelPackage or Export-Excel -passthru
$excel = get-process | export-excel path.xlsx -passthru
does the same as get-process | export-excel path; $excel = Open-ExcelPackage $path
You can use $ws = $Excel.workbook.worksheets["Sheet1"] and pass $ws as a worksheet (which makes sense if you are doing something else with the sheet) or pass $Excel and "Sheet1" (which makes sense in other cases)
Set-ExcelColumn -ExcelPackage $excel -Column 4 -Width 100 doesn't say which worksheet you want to set the column in.  Just add the worksheet name.

Get-Process | select 'Responding','SessionId','ExitCode','Name' | Export-Excel $ENV:Userprofile\Desktop\Newfolder\prox4.xlsx
$excel = Open-ExcelPackage "$env:userprofile\Desktop\Newfolder\prox4.xlsx"
Set-ExcelColumn -ExcelPackage $excel  -Column 4 -Width 100 -Worksheetname "Sheet 1"
This throw error:
The variable cannot be validated because the value 0 is not a valid value for the StartRow variable.
At C:\Program Files\WindowsPowerShell\Modules\ImportExcel\5.3.4\Set-Column.ps1:118 char:32

... (-not $StartRow)   {$startRow   = $Worksheet.Dimension.Start.Row    }

                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

CategoryInfo          : MetadataError: (:) [], ValidationMetadataException
FullyQualifiedErrorId : ValidateSetFailure

WARNING: The range pararameter you have specified also needs a worksheet parameter.
WARNING: Can set the width of a column or a range but not a String object

"Sheet 1"
No space in the default sheet names. Set-ExcelColumn doesn't check that the worksheet name is valid. I'll fix that at some point.

Never Mind :)
I just tried $excel.Save() after set-ExcelColumn and it worked. Not deleting\updating the comments so that it may help others as well.

Thank you very much once again. That is all I needed for now. You have been a great help.
Will open new requests if I need anything else. I am closing this request.
As everyone, I appreciate all the good work. Wish you good luck.


implement adjoint ODE solver

Description
For more scalable ODE solvers in Stan, the adjoint ODE method as described in the design doc
https://github.com/stan-dev/design-docs/blob/master/designs/0027-adjoint-ode.md
should be implemented in stan-math.
Example
Please refer to the design doc for details.
Expected Output
Faster ODE results and their gradients for the case of many parameters and few ODE states.
Current Version:
v4.0.1

implemented with #1905


Fix unicode logging errors

Change Summary

Added utf-8 encoding to the file log configuration.
Related issue number

fix #294
Checklist
I ran some tests locally but did not create a unit test.

[ ] Tests for the changes exist and pass on CI
[X] Documentation reflects the changes where applicable
[X] Change is documented in CHANGELOG.md (if applicable)
[X] My PR is ready to review

Thank you!


Layout: Label gets put below menu

Not every Screen behaves this way I noticed this with a ScrollPanel inside of a Panel inside a morphpanel with a big html label inside. When I resized the browser the layout didn't resize itself and it got below the menu

Not only did the label get put under the menu, the label didn't report it's new size when the text wrapped, so the label changed its size while the layout didn't re-layout. fixed in dc342877842c1d8cf6cb6ee37d7ed3abc3b28253


Fix autocorrect

Trying to use this method to fix the problem.

@galligan you'll want to remove the changes to components/WalletProvider.tsx that got copied over from the demo app

@galligan you'll want to remove the changes to components/WalletProvider.tsx that got copied over from the demo app

@saulmc I'm not 100% certain but it seems like those changes are what are needed for the demo. Maybe @jazzz knows.

 This PR was supposed to go into the demo.

This is the demo.xmtp.com as its in the Private Org.... isn't it


seed_r7_ros_pkg: 0.3.3-1 in 'kinetic/distribution.yaml' [bloom]

Increasing version of package(s) in repository seed_r7_ros_pkg to 0.3.3-1:

upstream repository: https://github.com/seed-solutions/noid-ros-pkg.git
release repository: https://github.com/seed-solutions/seed_r7_ros_pkg-release.git
distro file: kinetic/distribution.yaml
bloom version: 0.9.0
previous version for package: 0.3.2-1

seed_r7_navigation
* Merge branch 'master' into add_test

seed_r7_ros_pkg
* Merge branch 'master' into add_test
* Update CMakeLists.txt
* toplevel added
* Contributors: hi-kondo

seed_r7_samples
* Merge pull request #62 <https://github.com/hi-kondo/seed_r7_ros_pkg/issues/62> from hi-kondo/ori-master
  rostest directory of seed_r7_samples changed
* depend package added
* hand constructor added
* rostest directory of seed_r7_samples changed
* Merge pull request #61 <https://github.com/hi-kondo/seed_r7_ros_pkg/issues/61> from hi-kondo/add_rostest/roslaunch_and_demo.test
  Add rostest/roslaunch and demo.test
* rostest timeout changed
* update test

moved to #23220


Uniswap v3

We need to add uniswap v3 liquidity pools for our products - across all queries.

Initially, there is just one pool that has significant volume on Uniswap v3: https://info.uniswap.org/#/pools/0x151ccb92bc1ed5c6d0f9adb5cec4763ceb66ac7f
Going to get this pool added to the ETH2x-FLI Supply Breakdown and Index Coop Supply Breakdown queries - will provide a roadmap for future queries that need to incorporate Uniswap v3.

Also need to add individual liquidity providing positions on uniswap v3 to correctly assess exposure to our assets. We can start with ETH2x-FLI, given that it is the only product that has significant volume currently.

Picking this up!


YAML and JSON are scanned like structures

Description
Please include a summary of the change and which is is fixed.

Added StructContentProvider to separate structure storage
YAML and JSON might be scanned like structures
The case in yaml with multiline credential is found:

key: |
  5UpEr5eCrEt

How has this been tested?
Please describe the tests that you ran to verify your changes.

[x] UnitTests
[x] do benchmark with --dept 33

@babenek
Okay I understood what you mean except about whatthepatch case(xDizzix)..
Is it related with recursive scan with proper file type(extension)?
And regarding propagation of config, we can make a config as a classmethod and make it access from all over source code.
But I love singleton style more as we using.


Analysis gets stuck in pinning step

After few successful apps analysis the process either fails in pinning step or gets stuck in pinning step.

When the process is stuck in pinning step the emulator displays a blank/white screen

I think this is a limitation of the emulator itself and may not be solvable from project end. I am wondering if you faced similar issue and if yes, how did you deal with it?
Right now I am restarting the emulator and then re-execute run.py. For a large number of app analysis this is not a suitable solution.
To overcome this, I am planning to execute multiple emulators and parallel the analysis process. Multiple emulators cannot be run from same source I think, I got the following error,
emulator: ERROR: There's another emulator instance running with the current AVD '<build>'. Exiting...

How about I copy crylogger source into multiple location and execute emulators from each of them but pointing work_dir to the same location. Do you foresee any potential issues with this idea? Thanks!

Hi,
I remember that when I was using the emulator I was able to successfully execute several applications without the need of rebooting the emulator too frequently. From time to time, a reboot of the emulator might be necessary as you mentioned.
Does this issue always occur with the same application? Maybe you can try to exclude the applications for which the pinning fails. Some applications might not work properly on the emulator for a variety of reasons.
Regarding the question about multiple emulators, I think you can clone the repository in multiple locations and run them in parallel, if needed. When you run them, it should associate a different ID, which should be passed to run.py. One problem I can see is that the emulator is heavy to run so this solution might not be feasible on some machines.
Thanks


Rate Adjustment Frontend Logic

Added rate adjustment logic. This functionality is untested from a backend perspective. The rate is adjusted to be a number from 0-1 in the send data function in App.jsx.

Additional Note: This implementation introduces a good amount of dead code. Considering its last minute I will leave it for now, after interim demo I will go back through the codebase and remove dead code and refactor where needed.


When the Godot editor is fullscreened on a Mac, Context Menus don't show up until you move the mouse

Godot version
v4.1.1.stable.mono.official [bd6af8e0e]
System information
Macbook Pro 14-inch - M2 Max - MacOS Ventura 13.5.1
Issue description
When Godot is fullscreened/maximized and I right-click to open a context menu, the context menu doesnt immediately show until I move the mouse. I notice a similar issue when opening dropdown boxes or dialog boxes.
I notice that the issue goes away when one of the following happens:

Godot is un-fullscreened/unmaximized
A screen recording is happening (Cmd+shift+5 and Record the screen/selected portion)

Steps to reproduce
Open Godot on a Mac, leave it non-full-screened, and open a project.
While non-full-screened:

Right-click somewhere to open a context menu (Do not move the mouse after.)
Notice that the context-menu immediately appears

Maximize/Fullscreen the editor:

Right-click somewhere to open a context menu. (Do not move the mouse after.)
Notice that the context-menu does not appear regardless of how long you wait.
Move the mouse.
Notice that the context-menu appears.

Minimal reproduction project
N/A. Issue is reproducible on a new empty project.

Confirmed reproducible in these Godot Engine builds:

v4.2.1.stable.official [b09f793f5]
v4.2.1.stable.mono.official [b09f793f5]
v4.3.dev1.official [9d1cbab1c]
v4.3.dev1.mono.official [9d1cbab1c]

Reproduced on:

Vulkan API 1.2.231 - Forward+ - Using Vulkan Device #0: Apple - Apple M3 Max
MacOS Sonoma 14.2.1

Moving the mouse will trigger the context menu to open as described. However, I noticed doing so also causes the close/min/max buttons and system menus for Godot to disappear until focus is moved to another app. For example, moving to Chrome and clicking inside the window and then moving back to fullscreen Godot restores the close/min/max buttons.
I created a second issue for the system menus and controls, but they may have the same root cause as the failing editor menus: #86480

I can confirm that updating MacOS to 14.6.1 seems to have resolved this issue for me and #86480


Query by example should support array properties

Currently array properties are not supported in a useful way by example. This will require changes not only to this project but also for TypeScript generation to provide different options to with methods for array properties.

Fixed


fix(ESSNTL-4820): Populate the host group column

Requests the groups data from the backend
Populates the Group column from the data
Adjusts some column widths so all the data better in a row

Before commit:

After commit:

I couldn't get sorting on the Group column working though so I gave up :/  That can be for another PR.

@RedHatInsights/team-interact or @dkuc can we get 1 more approving review on this?
Thank you!

Looks good!


FRR: OSPF influence interface cost via carp

Problem description
FRR natively doesn’t seem to support any interaction with carp status as the variant in openbsd does (carp note in “depend on” keyword https://man.openbsd.org/ospfd.conf.5).
In OPNsense we do have events available if CARP status changes (https://docs.opnsense.org/development/backend/autorun.html),  which could help facilitate the feature, but there are still some challenges to overcome.
One of the challenges is FRR doesn’t have an easy reload option, there is an option to implement something like a clean reload which compares running verses new configuration
but it’s not 100% clear how well it performs  (https://github.com/FRRouting/frr/blob/master/tools/frr-reload.py).
We can control the running instance using vtysh, but this means after gui changes our event changes are being removed.
If there was a reload available, there would still be a challenge in generating the right configuration since in
our architecture there’s a clear separation of concerns tying data in the configuration to the output generated
using our template engine (the configuration doesn’t know about the carp status, when writing these to the config it would clutter history).
Solution design
With vtysh we can update the running configuration, which is quite convenience in terms of staying in service while changing settings. For this we can use carp events to update the active service.
Since the configuration has no knowledge about the changes made after startup, we should make sure to signal the ospf daemon after new settings are applied (frr is [re]started).

For flexibility reasons, we should add an additional field in Routing -> OSPF -> Interfaces [new] containing the demoted cost for the interface as well as the VHID to track.
(We could also follow an interface instead of a vhid, but when different protocols are used it might come in practical to bind to a vhid)
The event handler should be generic for our FRR plugin, so other protocols can use the handler in the future. The one included in the proposal on the next page implements the following workflow:

Based on the current interface vhid status and configured settings we calculate a proposed cost (either the normal one or the demoted one), using vtysh we ask the running service for the active cost.
When these differ (on a per interface/vhid base), we push new costs to ospfd using vtysh.
The mapping between interface and vhid doesn’t have to be the configured alias mapping, so we can influence interface costs on other interfaces as well.
We do have to explain this properly, the new field type to display the vhid’s should probably add an interface description to ease usage.
sponsored by : Modirum (https://www.modirum.com/)

Until OPNsense 20.7.5 is release, will need the new field type from master.
opnsense-patch  668d916


Pull and fetch

SonarQube analysis reported 1 issue

 1 major

Watch the comments in this conversation to review them.


Azure.Identity Improvements for .NET (June - September 2021)

Azure.Identity June - September 2021 Releases

[ ] Feature: Support Tenant Id Challenges / Hints tenant-hint.md

Support Key Vaults accross multiple tenants
Address common issues when customers use VS/VSCode credentials with multiple credentials signed in

[ ] Feature: Add On-Behalf-Of (OBO) Auth Flow for the Microsoft Graph Team tracking issue

The OAuth 2.0 On-Behalf-Of flow (OBO) serves the use case where an application invokes a service/web API, which in turn needs to call another service/web API. The idea is to propagate the delegated user identity and permissions through the request chain. For the middle-tier service to make authenticated requests to the downstream service, it needs to secure an access token from the Microsoft identity platform, on behalf of the user.
Prototype: https://github.com/jongio/azidext/pull/41

[ ] Feature: Support overriding MSI_ENDPOINT for dev-time debugging for the Azure Kubernetes Service team #670

The Bridge to Kubernetes enables a user to natively debug one microservice on their local machine when "bridged" to other microservices running in Kubernetes.  AKS is looking for an environment variable that can be overridden to specify a custom managed identity endpoint. This is required so that when the user's locally running code tries to call the managed identity endpoint for a token, they are able to intercept it and redirect the call to the cluster so that the token can be fetched from the endpoint on the cluster.

[ ] Feature: Add support for Managed Identity regional AAD authentication endpoints #20027

The [guidance] from the Azure IAM wiki for service teams using MI is to authenticate using a regional endpoint (e.g. https://eastus2euap.login.microsoft.com). However, the MSAL example given in the wiki uses APIs that are not currently exposed/used by [MsalConfidentialClient], namely WithAuthority(Uri, bool) and WithInstanceDicoveryMetadata(string).
Today, when using the regional AAD endpoint with Azure.Identity (using a [ClientCertificateCredential]), we see an error Application error - the login request was malformed and could not be matched with an existing authentication endpoint or instance. The error goes away when using a global endpoint (https://login.microsoftonline.com/).

[ ] Review Backlog of Customer-Reported feature requests related to token convenience methods

[ ] Request: Add support for fetching an access token from a refresh token
[ ] Request: provide the functionality of building a token credential from (a: existing credential, b: tenant id) for refresh token based credentials: InteractiveBrowserCredential and DeviceCodeCredential, VisualStudioCodeCredential (request)
[ ] Request: provide the functionality of setting tenant id for AzureCliCredential (request)
[ ] Request: provide a valid token in VisualStudioCodeCredentialBuilder without tenant id, use this token we can list the tenants (request)
[ ] Request: provide the functionality of listing cached account(azure environment, tenant id, user name, client id) for SharedTokenCacheCredential (request)
[ ] Feature: Expose Credential type for DefaultAzureCredential and ChainedTokenCredential
Enables users know which credential type is being used. #8948
[ ] Feature: Add new StaticTokenCredential type (prototype PR)
Encapsulate an AAD credential with a prefetched token for an AAD application.

Backlog Stretch Goals

[ ] Feature: Improve DefaultAzureCredential Errors (proposal)

The current DAC error messages are difficult to understand and do not focus the user on actions to take.
Remove text that doesn't help the user solve the issue; and do a better job of directing the user towards the action the need to perform to resolve the issue.

[ ] Stretch Goal: Improve JavaScript Dev-time Credential Experience

Feature: Add new WebStormCredential for JavaScript which authenticates as the Azure user signed in to WebStorm

[ ] Stretch Goal: Improve Python Dev-time Credential Experience

Feature: Add new PyCharmCredential for Python which authenticates as the Azure user signed in to PyCharm

Final Review of README.md / Getting Started Guide

Update "Next Steps" section which lists libraries that support azure-identity (should probably point to a different list rather than repeating the list here)
Update sections on new features / champion scenarios

Regarding the Tenant ID Challenge feature, are there tickets needed for for services that need to send the tenant ID in the challenge header in order to light up this feature? Like SQL Server, Postgres, Service Bus, etc


Make connection for new Notebook

Fix #4455 and #4501
Make connection after Model is ready based on the profile and disconnect the connection when close notebook.

Make Connection With Notebook And SQL1


Heuristic for picking the chunk/batch size?

ForwardDiff has a heuristic for picking chunk size, with a default threshold of 12 dictated by memory bandwidth:
https://github.com/JuliaDiff/ForwardDiff.jl/blob/ff56092ed2960717ce45f53a90584898c232e74b/src/prelude.jl#L24-L34
https://github.com/JuliaDiff/ForwardDiff.jl/blob/ff56092ed2960717ce45f53a90584898c232e74b/src/prelude.jl#L8
Does Enzyme have something similar I could use? I seem to remember a graph showing performance as a function of chunk size, with a maximum around 8-12 as well, but it disappeared in the Slackhole

Not presently, but contributions welcome!

Do you know which graph I'm talking about? Is it in some publication online?

I don't think it's in a publication, but around 10mins in @tgymnich has some in his talk at EnzymeCon https://youtu.be/nPN_Z5j6JDM?feature=shared

@tgymnich do you remember what machine you used for these measurements?

This will also now depend a lot more on the program in Julia.
for example, batching something with a linear solve will almost always be faster since we now do one linear solve to be reused for all chunks

@vchuravy this must have been a bare metal AWS machine provided by @wsmoses. I believe it was with AVX512.

I'm asking because I'm including vector mode in DI, so it would be nice to have a function in Enzyme I can call to pick a decent chunk size if the user doesn't provide it. Even if the function is dumb at the moment, I feel like that's definitely something I don't want to decide myself

8/16 should be a safe bet.

I'm asking because I'm including vector mode in DI, so it would be nice to have a function in Enzyme I can call to pick a decent chunk size if the user doesn't provide it. Even if the function is dumb at the moment, I feel like that's definitely something I don't want to decide myself

Sure, open a PR to enzyme to add a function which returns 16 for now and we can add more complex analysis later.


Can not reproduce demo

Hello,
I feel a little silly posting this but I really can not figure out what I am doing wrong. I am trying to set up a simple example such as what you have on the project homepage. I have a fiddle here: https://jsfiddle.net/4zwhwfyy/5/
The header does not scroll. I have tried asking around IRC and trying many combinations of options but I would think something this simple should not need this effort. In the likely possibility that it is something simple and stupid, at least you get another free issue closed. :)
Thanks for taking a look.

you dont have enough stuff in the table to scroll. how about this:
https://jsfiddle.net/4zwhwfyy/6/

Unfortunately this is what the fiddle looks like for me, which you have linked. My OS is ubuntu linkx 14.04 LTS , Browser is google chrome 48

Fortunately I can reproduce on my phone, but it's 1am

where did you copy the lib code from? Its not correct. Specifically the line
var $floatContainer = $('<div>').css('overflow', '').attr('aria-hidden', 'true');
should be
var $floatContainer = $('<div>').css('overflow', 'hidden').attr('aria-hidden', 'true');

it works much better if you use the code from github:
https://jsfiddle.net/4zwhwfyy/7/

Well, I can't complain! I cant imagine how that specific line was changed, but using the hosted source has fixed the problem. I greatly appreciate your time, even though I see it could have been avoided entirely.
Have a great weekend.


added weighted binary cross entropy and weighted MSE objectives

weighted objective functions for binary cross-entropy and mean-squared error objective functions.
w0 and w1 are lists of weights for the negative & positive class, respectively, with w0_i and w1_i corresponding to task i in a multi-tasked model.
The benefit of the weighted objective functions is the ability to handle cases when some tasks have a much higher degree of class imbalance then others. This functionality is different from the existing weights argument to model.train because the weights argument is assigned with w_j for datapoint j, whereas the proposed objectives would assign weights at the task rather than the datapoint level.

This functionality is different from the existing weights argument to model.train because the weights argument is assigned with w_j for datapoint j, whereas the proposed objectives would assign weights at the task rather than the datapoint level.

I don't get it. You can set weights for tasks (in compile), but also for classes (class_weight argument  in fit) and for samples, what you call data points (sample_weight argument in fit).

Ok, I think the functionality I was trying to achieve is a combination of the loss_weights in compile() and the class_weight argument in fit().
It seems the loss weights assign a weight for each task. And the class_weights assign a weight for each class. How are those two combined together to determine weight of class 1 for task i vs. weight of class 1 for task j?
I would like to handle a class like this:
w1=[1,1,5,3]
w0=[3,4,2,9]
So the relative w1 & w0 for the 4 tasks of interest are different.

It looks like @fchollet is using "task" to refer to a particular output layer, but in this context @annashcherbina is using "task" to refer to a specific index WITHIN a given output layer. The loss_weights argument appears to specify one weight per output layer, because loss_weights has the same length as self.outputs (https://github.com/fchollet/keras/blob/master/keras/engine/training.py#L494) and for a sequential model, the length of self.outputs is 1 (https://github.com/fchollet/keras/blob/master/keras/models.py#L240). It doesn't look like there is currently a way to weight different indices within a particular output layer.

I think this would be too niche a use case for the time being.

Hi @fchollet. The current alternative to having index-specific weights would be to create multiple Dense layers, each containing a single neuron. Not only would this force the user to use a functional model rather than a Sequential model, but when working like a large number of tasks (~200-400, which is not uncommon when doing multitasking), it would clutter the architecture of the model and make it unwieldy to read and manipulate. All major deep learning for genomics papers till now have used a single Dense layer for the output (see: Basset, DeepSEA), and all had a different class imbalance for different output indices. Adding this functionality would go a long way towards making Keras more usable for genomics. Hopefully you reconsider :)

@akundaje we avoid including support for specialized use case directly into the codebase, because there is a very long tail of such use cases and it would be too much to maintain. This functionality is self-contained and only spans a few lines of code, so it would not be an issue for you to keep it as part of your own codebase. If it's useful for others, you can make it available as part of a genomics toolbox. Then if there is enough demand, we'll consider including it.

@fchollet Sure. Makes sense. Thanks for clarifying.


Where is the next.js image optimization?

What is the purpose of using next/image with "srcset" when nothing is optimized? img duplicates same digitalocean image URL 8 times.
https://starter-uploads.nyc3.digitaloceanspaces.com/user_ed78b0e88f.png
Same 1.2Mb image loads on retina macbook and iphone 4 over 3g.

I removed loader with hardcoded URL to make use of responsive images.
https://github.com/strapi/strapi-starter-next-corporate/blob/10002a289f91e5f87e6eb745ec69be1dcbea0d44/starter/components/elements/image.js#L23
Next.JS figured out the rest and loaded optimized srcset images.


Change default query to load most-resent completed month of data

Currently the site fails to load data on first view at the first of each month, since the new month's data is not yet available from NYC

Noting from Slack conversation today that the simplest solution is to load the previous month's data by default so that a full month is always shown.


RDK-51313 [RDKV] Refactoring the usersetting comrpc related test case

[RDK-51313] Refactoring the usersetting comrpc related test cases
[RDK-51419] Fix Segmentation fault issue when enabling hdcp profile
[RDK-51088] RDKV- Enable Verify contracts test in L2 Test with thunder R4.4.1 version
Reason for change: Create common functions for com rpc connection creation, Enabling Verify contracts test in L2 Test with thunder R4.4.1 version
Test Procedure: NA
Risks: Medium.
Priority: P0

Closing this PR, replaced by new PR #5561 with the same changes


prediction

Hi, i used the test.py with the pretrained model that you posted in your git but i got an error, can you please help me fix it

I updated the code by adding norm_layer as group and width as 48. Can you check now?

Thanks alot, the previous error was fixed but i got another error

That code line is unwanted. I updated the script.


feat: add generateOnlyEnums option to typescript

generateOnlyEnums was added to typescript plugin options.
This allows typescript plugin to generate only enums.
Our team use relay package.  relay-compiler generates Query, Mutation and fragment types based on schema.graphql file. So other types this library generates are not useful to us, except enums.
The problem is whenever we have a type that uses an enum it generates that type again, I mean there could be definition of an enum in 20 different files. We always liked to have a single source of truth for enums. We even wrote a small package relay-enum-generator.  So we would really appreciate it if there were an option in your package to make this possible. I suppose other teams using relay probably have this issue.

Thank you @c0m1t !
We recently merged this: https://github.com/dotansimha/graphql-code-generator/pull/3760 this also does that, and generates only enums ;)


Rename the Infos Button in Check-in Page

Avoid duplicates

[ ] This enhancement request has not already been raised before
[ ] Enhancement request is specific for iOS only, for general issues / questions that apply to iOS and Android please raise them in CWA-Wishlist
[ ] If you are proposing a new feature, please do so in CWA-Wishlist

Current Implementation

Press the 3 dot button, it says Infos instead of Informationen, as it is in the Tagebuch page.
Suggested Enhancement

Rename to "Informationen"
Expected Benefits

See #2549 in cwa-app-android.

PR #2544

#2544 has been merged into the Release/2.2.x branch.

Fixed in Version 2.2.

@DerVogel2020 Thanks.


Add local search

There's currently a mix of server-side and client-side filtering. If we can move all filtering to server-side, we can use pagination instead of loading every single extension in the front-end. flagrow.io results are always fetched in a single request.
I don't know if I should work on the above right away or if we first release this before doing more work on the filtering feature.

Codecov Report

Merging #73 into master will decrease coverage by 0.41%.
The diff coverage is 0%.

@@            Coverage Diff             @@
##           master      #73      +/-   ##
==========================================
- Coverage   29.28%   28.87%   -0.42%
==========================================
  Files          43       43
  Lines        1055     1070      +15
==========================================
  Hits          309      309
- Misses        746      761      +15

Impacted Files
Coverage Δ

src/Repositories/ExtensionRepository.php
0% <0%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 27ad2a1...799968f. Read the comment docs.

I think there's an issue with the js view though

I also copied a simple debounce implementation from underscore. I'm not sure how we are supposed to include js dependencies.
Ideally we should use the lodash method, but it's not available globally and it's not a very good idea to import lodash via an extension...

a) release often and iterate. So implement full server side filtering later on; this has to be done one way or another.
b) yes not sure about dependencies either, have been thinking the same now that I have a better grasp of typescript, something for the internals channel?


Use buckets in DVC

I've shared the project role with you, you should be able to use the same bucket I use.
made also some cleanup in live.yaml, updated requirements since they were broken (by time)
Seldon does not work atm, still under investigation, everything else should work.

Please, try also to perform one iteration over the project, check whether it works for you.

Thank you!


CUDA 12.0 build on windows depends on cuda-cudart >=12.1.105

This dependency then forces cuda-version >=12.1,<12.2.0a0, which is not compatible with regular CUDA 12.0 compilation.
This then leads to failure in https://github.com/conda-forge/kaldi-feedstock/pull/47 (for linux it works because there are older libmagma_sparse builds that pull in cuda-cudart 12.0)
There was some discussion about this in #39, but nothing that seemed like explicitly choosing 12.1 components in the 12.0 build.
@conda-forge/magma CC @jakirkham

Think cuda-version {{ cuda_compiler_version }} needs to be added to requirements/host of all packages to ensure the same CUDA version is used in requirements/build as requirements/host
It was added here:
https://github.com/conda-forge/magma-feedstock/blob/509c0dcf2b395f27f255f465b183af5fe2656b5f/recipe/meta.yaml#L126
However sections like this are missing it:
https://github.com/conda-forge/magma-feedstock/blob/509c0dcf2b395f27f255f465b183af5fe2656b5f/recipe/meta.yaml#L149-L155
This affects things like run_exports that packages add, which is likely what is happening with cuda-cudart in this case


 Ideally it should put something sensible in the error callback instead of crashing.
Environment Setup
parse-server v2.1.3
Steps to reproduce
          var promise = Parse.Cloud.httpRequest({
            method: "GET",
            url: "https://www.blah blahblah",
          });

Logs/Trace
TypeError: Cannot read property 'statusCode' of undefined
    at Request._callback (cloudcode/dist/node_modules/parse-server/lib/httpRequest.js:46:35)
    at self.callback (cloudcode/dist/node_modules/parse-server/node_modules/request/request.js:199:22)
    at emitOne (events.js:77:13)
    at Request.emit (events.js:169:7)

I'll check it out!


Remove deprecated aiohttp _copy_cookies

The _copy_cookies function has been deprecated.
http://aiohttp.readthedocs.io/en/v0.14.4/_modules/aiohttp/web.html
vs
http://aiohttp.readthedocs.io/en/v1.2.0/_modules/aiohttp/web.html
This means this module no longer runs against the latest aiohttp

Hi! Thank you for this PR, but this project is not actively maintained I have number of other duties (projects in https://github.com/aio-libs org).
If you have interest in aiohttp_sse I can give you commit rights.

This change was added with https://github.com/jettify/aiohttp_sse/pull/3


Remove CacheLookupScheduler

I created CacheLookupScheduler because Goma wasn't getting cache hits.  Turns out this was actually due to #176.  This is now an entirely unused scheduler type.
I thought I'd raise an issue to determine if we want to simply delete it to avoid the maintenance cost.  If I understand the RBE protocol it should actually never be required...

Actually, the protocol does not require it and I think for more simple clients this is a good feature to have. If anything we may want to document that most clients make the cache lookup pointless.
For example, it would be faster to just skip cache lookups all together on the clients and just execute commands, because the scheduler is almost certainly closer to the cache than the clients. For example tools like recc would make more sense to work this way to be able to dispatch commands faster.

So long as you're happy with it being there, I'll leave it in...


[Telegraf] Keep getting:  Error: unable to parse out as JSON, unexpected end of JSON input

I am using the example from json from the readme on data formats: https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
[
{
"a": 5,
"b": {
"c": 6
},
"ignored": "I'm a string"
}
]
Here is my config file using the tail input:
# Telegraf Configuration
#
# Telegraf is entirely plugin driven. All metrics are gathered from the
# declared inputs, and sent to the declared outputs.
#
# Plugins must be declared in here to be active.
# To deactivate a plugin, comment out the name and any variables.
#
# Use 'telegraf -config telegraf.conf -test' to see what metrics a config
# file would generate.
#
# Environment variables can be used anywhere in this config file, simply prepend
# them with $. For strings the variable must be within quotes (ie, "$STR_VAR"),
# for numbers and booleans they should be plain (ie, $INT_VAR, $BOOL_VAR)
# Global tags can be specified here in key="value" format.
[global_tags]
# dc = "us-east-1" # will tag all metrics with dc=us-east-1
# rack = "1a"
## Environment variables can be used as tags, and throughout the config file
# user = "$USER"
# Configuration for telegraf agent
[agent]
## Default data collection interval for all inputs
interval = "10s"
## Rounds collection interval to 'interval'
## ie, if interval="10s" then always collect on :00, :10, :20, etc.
round_interval = true
## Telegraf will send metrics to outputs in batches of at most
## metric_batch_size metrics.
## This controls the size of writes that Telegraf sends to output plugins.
metric_batch_size = 1000
## For failed writes, telegraf will cache metric_buffer_limit metrics for each
## output, and will flush this buffer on a successful write. Oldest metrics
## are dropped first when this buffer fills.
## This buffer only fills when writes fail to output plugin(s).
metric_buffer_limit = 10000
## Collection jitter is used to jitter the collection by a random amount.
## Each plugin will sleep for a random time within jitter before collecting.
## This can be used to avoid many plugins querying things like sysfs at the
## same time, which can have a measurable effect on the system.
collection_jitter = "0s"
## Default flushing interval for all outputs. You shouldn't set this below
## interval. Maximum flush_interval will be flush_interval + flush_jitter
flush_interval = "10s"
## Jitter the flush interval by a random amount. This is primarily to avoid
## large write spikes for users running a large number of telegraf instances.
## ie, a jitter of 5s and interval 10s means flushes will happen every 10-15s
flush_jitter = "0s"
## By default, precision will be set to the same timestamp order as the
## collection interval, with the maximum being 1s.
## Precision will NOT be used for service inputs, such as logparser and statsd.
## Valid values are "ns", "us" (or "µs"), "ms", "s".
precision = ""
## Logging configuration:
## Run telegraf with debug log messages.
debug = false
## Run telegraf in quiet mode (error log messages only).
quiet = false
## Specify the log file name. The empty string means to log to stderr.
logfile = ""
## Override default hostname, if empty use os.Hostname()
hostname = ""
## If set to true, do no set the "host" tag in the telegraf agent.
omit_hostname = false
###############################################################################
#                            OUTPUT PLUGINS                                   #
###############################################################################
# Send telegraf metrics to file(s)
[[outputs.file]]
## Files to write to, "stdout" is a specially handled file.
files = ["stdout", "metrics.out"]
## Data format to output.
## Each data format has it's own unique set of configuration options, read
## more about them here:
## https:##github.com#influxdata#telegraf#blob#master#docs#DATA_FORMATS_OUTPUT.md
data_format = "json"
###############################################################################
#                            PROCESSOR PLUGINS                                #
###############################################################################
# # Print all metrics that pass through this filter.
# [[processors.printer]]
###############################################################################
#                            AGGREGATOR PLUGINS                               #
###############################################################################
# # Keep the aggregate min#max of each metric passing through.
# [[aggregators.minmax]]
#   ## General Aggregator Arguments:
#   ## The period on which to flush & clear the aggregator.
#   period = "30s"
#   ## If true, the original metric will be dropped by the
#   ## aggregator and will not get sent to the output plugins.
#   drop_original = false
###############################################################################
#                            INPUT PLUGINS                                    #
###############################################################################
###############################################################################
#                            SERVICE INPUT PLUGINS                            #
###############################################################################
# Stream a log file, like the tail -f command
[[inputs.tail]]
## files to tail.
## These accept standard unix glob matching rules, but with the addition of
## ** as a "super asterisk". ie:
##   "#var#log#**.log"  -> recursively find all .log files in #var#log
##   "#var#log##.log" -> find all .log files with a parent dir in #var#log
##   "#var#log#apache.log" -> just tail the apache log file
##
## See https:##github.com#gobwas#glob for more examples
##
files = ["test.json"]
## Read file from beginning.
from_beginning = true
## Data format to consume.
## Each data format has it's own unique set of configuration options, read
## more about them here:
## https:##github.com#influxdata#telegraf#blob#master#docs#DATA_FORMATS_INPUT.md
data_format = "json"
##tag_keys = [
##]
Here is my output:
./telegraf.exe -config telegraf.conf
2017/01/20 15:39:00 I! Starting Telegraf (version 1.1.2)
2017/01/20 15:39:00 I! Loaded outputs: file
2017/01/20 15:39:00 I! Loaded inputs: inputs.tail
2017/01/20 15:39:00 I! Tags enabled: host=patapl-BLMRQ72
2017/01/20 15:39:00 I! Agent Config: Interval:10s, Quiet:false, Hostname:"patapl-BLMRQ72", Flush Interval:10s
2017/01/20 15:39:00 Seeked test.json - &{Offset:0 Whence:0}
2017/01/20 15:39:00 E! Malformed log line in test.json: [[], Error: unable to parse out as JSON, unexpected end of JSON input
2017/01/20 15:39:00 E! Malformed log line in test.json: [  {], Error: unable to parse out as JSON, unexpected end of JSON input
2017/01/20 15:39:00 E! Malformed log line in test.json: [      "a": 5,], Error: unable to parse out as JSON, invalid character ':' after top-level value
2017/01/20 15:39:00 E! Malformed log line in test.json: [      "b": {], Error: unable to parse out as JSON, invalid character ':' after top-level value
2017/01/20 15:39:00 E! Malformed log line in test.json: [          "c": 6], Error: unable to parse out as JSON, invalid character ':' after top-level value
2017/01/20 15:39:00 E! Malformed log line in test.json: [      },], Error: unable to parse out as JSON, invalid character '}' looking for beginning of value
2017/01/20 15:39:00 E! Malformed log line in test.json: [      "ignored": "I'm a string"], Error: unable to parse out as JSON, invalid character ':' after top-level value
2017/01/20 15:39:00 E! Malformed log line in test.json: [  }], Error: unable to parse out as JSON, invalid character '}' looking for beginning of value
It is valid json though so I'm not sure why it can't parse it.

@kostasb - could you take a look at this when you get a chance?

Hi Anish,
It looks like that JSON string is pretty-printed, meaning that there are newlines in it.
The JSON string should be a one-liner, have you tried the following format?
{"a": 5,"b": {"c": 6},"ignored": "I'm a string"}
Also, please direct your future questions to this repo: https://github.com/influxdata/telegraf/issues or to our mailing list.
Thanks,
Kostas

2017-12-17T18:27:30Z E! Error in plugin [inputs.httpjson]: unable to parse out as JSON, invalid character '<' looking for beginning of value

@kostasb its been a long time since this issue was addressed, but its still an issue.
can the error message be updated to be helpful?
2017-12-17T18:27:30Z E! Error in plugin [inputs.httpjson]: unable to parse out as JSON, invalid character '<' looking for beginning of value

Why is there a requirement for pretty printed JSON, aka one-line JSON?  Most integration tools such as Mulesoft will create the new line.  Pretty ridiculous requirement.

@dubnemo can you open a new issue on telegraf with your config and use case. The above issue (multi-line json when using the tail plugin) is only an issue because the tail plugin reads and parses files one line at a time, hence the need for full and parseable (non-prettyprinted) data, regardless of the format.


create-withdrawal-change.sh falsely assumes that just because the ping fails, that the machine is safely offline.

https://github.com/eth-educators/eth-docker/blob/9d6aa2471c98bc4f8663530b17db4b3396e3ae8d/.eth/ethdo/create-withdrawal-change.sh#L34-L35C33
A machine might actually be online but this test will think it's not if the user in which runs this script is a non-root user without permission to ping which results in an error like:
ping -v <IP_ADDRESS>
ping: socket: Operation not permitted

Which would make the script think that the machine is offline, when in-fact it is online.

Hmm. I haven't seen a Live USB where ping is disallowed for the default user. Which one did you use, and do you have a suggestion on how to strengthen this test?

Offline detection is now slightly less naive. It still assumes that the user can do ping to be successful, which is true on Ubuntu Live and Tails Live.
wget and curl can't be assumed to be available. Still curious which Live USB actually disallows ping.