Separating tests meta-data from the testing engine code

We have concerns about the size of dist/tests.json. Currently it is a 200+KB file. Add this to the 250+KB of quail.jquery.js and Quail is starting to become a half-mega monster.
The point is that just a small part of this file is required to run Quail tests. For instance, properties like “testability”, “title”, “description”, “guidelines” and “tags” are just meta-data attached to the test and are not relevant to run the test.
This means that, if you run 200 tests in a document and just one issue is found, still you have to download the meta-data for all 200 tests, even if you application may be interested about that single issue information only.
To make it worst, even localization information is available for all languages and all tests. It’s clear that this will not work as soon as the number of languages supported start to grow.
There should be a clear separation on the way Quail is distributed. At one side it has a engine to run tests. At the other end it provides information about tests (or issues) that is relevant to applications using Quail, to show it to the end user.
And additional drawback is that the “custom tests” scripts are included inside quail.jquery.js, which need to be loaded on page load, instead of being loaded on demand inside tests.json, which could be called tests.js and have the custom code included there as well.
So we have two proposals:

Move custom tests from quail.jquery.js to tests.js (former tests.json). This will make quail.jquery.js much smaller and tests.js much bigger.
Move the meta-data about tests out of the above tests.js, optionally. This will make this file much smaller.

The idea for point “2” is having configurations for grunt build that would define whether or not to have the meta-data outside. Actually, it could even define what we want inside and what outside.
The result would be directory with the following (e.g.):
/dist/
	/quail.jquery.js
	/tests.js
	/meta/
		/testid/
			/en/
				meta.json
			/nl/ (…)

So, basically, every test would have it’s own directly inside “meta”, with a dedicated meta.json available for each supported language.
Inside tests.js, a test could look like this:
"aAdjacentWithSameResourceShouldBeCombined”: {
    "type": "custom",
    "callback": "aAdjacentWithSameResourceShouldBeCombined"
}

While in meta.json, we would have this:
{
    "testability": 1,
    "title": "Adjacent links that point to the same location should be merged",
    "description": "Because many users of screen-readers use links to navigate the page, providing two links right next to eachother that point to the same location can be confusing. Try combining the links.",
    "guidelines": {
      "wcag": {
        "2.4.4": {
          "techniques": [
            "H2",
            "F89"
          ]
        },
        "2.4.9": {
          "techniques": [
            "F89"
          ]
        },
        "4.1.2": {
          "techniques": [
            "F89"
          ]
        }
      }
    },
    "tags": [
      "link",
      "content"
    ]
}

As said, we could even make this configurable, so one could decide, for example, to have “testability” and “tags” available inside tests.js, so filtering could be applied when running tests.
Finally, one could also decide to have everything together, just like it happens today.
I hope all this makes sense.

As discussed during last week's call, here is my proposal for this approach:

Guidelines: Section 508 has moved to WCAG 2, so we can probably remove this for a next version of Quail. As for WCAG2, a different approach to testing for WCAG 2 conformance has been build for Quail. This supersedes the wcag2 value, so this too can be removed. This makes the entire 'guidelines' property redundant.
Title/description should be moved to their own localisation files. We should use URIs for identifying the different assessments, and have the localisation files define the title/description that belongs to that URI.
Testability is an important property of the assessments, so I'd suggest keeping that in.
Tags should get their own mechanism. Rather then be build into the core, we should have a component for tags that defines a subset of the assessments to be tested. We'd give that it's own class and have each 'tag' defined in a separate configuration file

I pulled the metadata into each assessment file.
https://github.com/quailjs/quail/pull/387


Grunt not detecting changes in .jshintrc

tl;dr - my the grunt task runs fine, it just doesn't change when I modify the .jshintrc
I changed my globals in the .jshintrc (in the root of my project folder) from:
"globals": {
"console": true,
},
 TO
"globals": {
"console": true,
"describe": true,
"it": true,
"before": true,
"after": true,
"beforeEach": true,
"afterEach": true
},
Now my Sublime JSHint plugin will accept the newly changed globals but my the Grunt task hasn't changed.

Just found the answer on Stack Overflow, I needed to set it in options:
options: {
jshintrc: '.jshintrc'
}


Add Logging

Issue
Debugging errors is difficult because there's little in the way of log output.
Solution
Add logging for the standard log levels. For bonus points, colorize by log level:

DEBUG -- Grey: #333333
INFO -- Cyan: #009ce0
WARNING -- Yellow: #fcdc00
ERROR -- Red: #f44e3b
CRITICAL -- White text with red background: #ffffff with #f44e3b

DEBUG -- Grey: #333333 is RGB 51, 51, 51
INFO -- Cyan: #009ce0 is RGB 0, 156, 224
WARNING -- Yellow: #fcdc00 is RGB 252, 220, 0
ERROR -- Red: #f44e3b is RGB 244, 78, 59
CRITICAL -- White text with red background: #ffffff with #f44e3b is RGB 255, 255, 255 and RGB 244, 78, 59


How to Read Deep Values

Apologies for the newbie question, however, Im struggling with this:
My state is like this:
import { List as list,
	     Map as map } from "immutable";

const initialTabState = list.of(
	map({
		tabClass : "tabPreview tab activeTab",
		tabName : "Preview"
	}),
	map({
		tabClass : "tabBody tab",
		tabName : "Body"
	}),
	map({
		tabClass : "tabSleeve tab",
		tabName : "Sleeve"
	})
);

In my reducer function, I get 'typeError' when I do this:
console.log( "Initial State is...", initialTabState.isList() );
And how do I 'read' the value of third Map obj nested within the 'initialTabState' if I was to read it in console?
Thanks for the help

If you want to read the third value you can do initialTabState.get(2), and if you want to read a nested value you can do initialTabState.getIn([2, 'tabName']), passing it an array, and same works for setting values.

Thanks for answering!
In the future, answer on http://stackoverflow.com/questions/tagged/immutable.js?sort=votes


Further GPU-aware MPI fixes

This is a draft PR. Further commits may be incoming.

Thanks @samhatfield for addressing all. Good to merge.


How do I fix FancyZones' mess with dropdowns?

Microsoft PowerToys version
0.74.1
Installation method
PowerToys auto-update
Running as admin
Yes
Area(s) with issue?
FancyZones
Steps to reproduce
It's not the problem with FancyZones, but the problem FancyZones has created.
After recent update dropdowns in various applications (Edge, Adobe Apps) are now stuck to the upper left corner. Annoying af. How do I restore it to normal behavior? It drives me nuts.
✔ Expected Behavior
Dropdowns appear where they supposed to appear
❌ Actual Behavior
Dropdowns appear at the upper left corner of the screen.

Other Software
No response

/bugreport

I was able to replicate the issue by opening the "Running as admin" menu in Edge. Other menus open normally, and the same "Running as admin" menu in other browsers (such as Chrome) also works fine.
No need for the bug report, as it's reproducible.
To help us verify the fix for as many scenarios as possible, please let us know which other applications are affected.
As a workaround, you can disable the Allow popup windows snapping or Move newly created windows to their last known zone option. The combination of these two options seems to be causing the issue.

It was quite a specific one. Transcribe options window in Adobe Premiere experienced the most unusable behavior because it didn't show a dropdown at all when clicked. That's when I came here to report an issue. As of now the problem is gone after your suggestions.

Hi @iamshilov, could you please verify if the issue is reproducible on PowerToys 0.76?
/needinfo


Initialize path with basename

When using useBasename, the path in the address bar (and state) should be initialized with the basename path when the app inits. This is the behavior you got with redux-router and is also the behavior you get from Ember's router (when using their router's rootURL). This makes the association between history and state consistent from the very beginning. As it is now, the basename is only introduced into the browser history and state when the path is changed, e.g. with pushPath. (I'm using history 1.17.0 and redux-simple-router 1.0.2.

Unfortunately, because of this, and the fact that the configured basename isn't available to redux-simple-router (because it's just closed over), this would require a change to rackt/history.

Unfortunately, because of this, and the fact that the configured basename isn't available to redux-simple-router (because it's just closed over), this would require a change to rackt/history.

This is not the correct behavior from the POV of history/router, per https://github.com/rackt/history/pull/197. If the URL doesn't have the basename for some reason, it's not correct for the history to lie to you about what the basename of the current location is. You'll have to handle this redirect in userspace.

This is not the correct behavior from the POV of history/router, per https://github.com/rackt/history/pull/197. If the URL doesn't have the basename for some reason, it's not correct for the history to lie to you about what the basename of the current location is. You'll have to handle this redirect in userspace.


Create a Stack layout

A stack layout is when the children stack vertically, and one of the elements (the main) will expand as needed.
Example:

PR: https://github.com/patternfly/patternfly-next/pull/78


Feature request: Ability to customize scroll bar visibility

With a RichTextBox, you can change HorizontalScrollBarVisibility/VerticalScrollBarVisibility to customize the scroll bar's visibility. It doesn't appear that there's an equivalent on MarkdownViewer.

The default theme for MardownViewer has a control template with FlowDocumentScrollViewer setting the vertical scrollbar visibility to Auto. You can override it by providing your own template.
Taking by example the MainWindow.xaml from the sample project (some lines omitted for clarity):
<Window x:Class="Markdig.Wpf.SampleApp.MainWindow"
        xmlns:markdig="clr-namespace:Markdig.Wpf;assembly=Markdig.Wpf">
  <FrameworkElement.Resources>
    <Style TargetType="markdig:MarkdownViewer">
      <Setter Property="Template">
        <Setter.Value>
          <ControlTemplate TargetType="markdig:MarkdownViewer">
            <FlowDocumentScrollViewer Document="{TemplateBinding Document}"
                                      ScrollViewer.HorizontalScrollBarVisibility="Visible"
                                      ScrollViewer.VerticalScrollBarVisibility="Visible" />
          </ControlTemplate>
        </Setter.Value>
      </Setter>
    </Style>
  </FrameworkElement.Resources>
  <DockPanel>
    <markdig:MarkdownViewer x:Name="Viewer"/>
  </DockPanel>
</Window>

That worked, thanks!


Skip "find_split_transpose" if split has more than one outputs

The current matcher is doing an assert on the number of outputs but instead, it should skip it if the number of outputs is greater than 1.
This matcher is doing
input --> slice --> transpose
    |
     ---->slice --> transpose
    |
    ---->slice --> transpose

to
input --> transpose --> slice
                |
                 ----> slice
                |
                ----> slice

This is only useful if slice in original input graph has one output node (transpose in this case). If it has multiple outputs then doing transpose before the slice would change the layout for the slice's outputs (other than transpose) in original graph.
Partially solves #2116

I think we need to add a unit test. I've seen this happen in distilgpt2 as well.

I think we need to add a unit test. I've seen this happen in distilgpt2 as well.

Yea a unit test would be good.


Does it support vue 2 / nuxt 2?

No support right now. Maybe will be supported in the future.

Hi @xiaoluoboding, is the vue-sonner mount for Vue 2 (maybe 2.7?) still being considered? That would be great, works like a charm on my Vue 3 projects !


Ads on page : fmovies.se

I am using Unified hosts + fakenews + gambling + porn list updated lat on 29 Dec , 2016. Any workaround?

It is off-topic as this repo only collects hosts file from other resources. Sorry for this trouble.


Calendar is not displayed on API 7.0 in RTL

i tried to use the library on devices with API 7.0 , it worked on some of them and a blank pager is displayed on others since weekdays and days are not shown (no decorators) but clickable.
Anyone could help me solving this please?
issue appears on these devices:
Huawei P9 Lite (7.0)
Huawei Y7 Pro (7.0)

Same problem to me.
I'm working on Android 6.0.1

https://github.com/prolificinteractive/material-calendarview/pull/573/files

Better RTL support should be coming in the next release, stay tuned


Misc fixes to v0.4.0

These changes should (?) fix the package on Julia nightly and make the test output slightly nicer.

I see now why the tests are organized as they are... Will remove commit 5d56624

Codecov Report

Merging #75 into master will not change coverage.
The diff coverage is 0%.

@@           Coverage Diff           @@
##           master      #75   +/-   ##
=======================================
  Coverage   81.36%   81.36%
=======================================
  Files           4        4
  Lines         676      676
=======================================
  Hits          550      550
  Misses        126      126

Impacted Files
Coverage Δ

src/AmplNLWriter.jl
74.76% <0%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6a81ccb...4d5640d. Read the comment docs.

Thanks! I'm not too worried about getting this to work on v0.7 just yet.


:bug:  fix failed update roleRef in clusterrolebinding in upgrade case

Summary
the roleRef is changed in work-execution clusterrolebinding, so failed to update the clusterrolebinding during upgrade scenario.
Related issue(s)
Fixes #

/hold

/lgtm

/approve

/unhold


[WIP] Fix case sensitive issues and input validation for Geo values.

Why is this PR needed?

SPATIAL_INDEX property, POLYGON, LINESTRING, and RANGELIST UDF's are case sensitive.
SPATIAL_INDEX.xx.gridSize and SPATIAL_INDEX.xxx.conversionRatio is accepting negative values.
IN_POLYGON_LIST udf, buffer size is accepting negative/character values.

What changes were proposed in this PR?

converted properties to lower case and made UDF's case insensitive.
added validation for negative/character values.

Does this PR introduce any user interface change?

No

Is any new testcase added?

Yes

Build Success with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbonPRBuilder2.3/5140/

Build Success with Spark 2.4.5, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_2.4.5/3389/

Build Success with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbonPRBuilder2.3/5211/

Build Success with Spark 2.4.5, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_2.4.5/3463/

LGTM

Build Success with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbonPRBuilder2.3/5254/

Build Success with Spark 2.4.5, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_2.4.5/3507/

Build Failed  with Spark 2.4.5, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_2.4.5/3511/

Build Failed  with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbonPRBuilder2.3/5258/

LGTM

retest this please

Build Success with Spark 2.4.5, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_2.4.5/3518/

Build Success with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbonPRBuilder2.3/5264/

Build Success with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbonPRBuilder2.3/5270/

Build Success with Spark 2.4.5, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_2.4.5/3525/

Build Success with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbonPRBuilder2.3/5273/

Build Failed  with Spark 2.4.5, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_2.4.5/3528/

LGTM

retest this please

Build Success with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbonPRBuilder2.3/5278/

Build Failed  with Spark 2.4.5, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_2.4.5/3533/

retest this please

Build Success with Spark 2.4.5, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbon_PR_Builder_2.4.5/3534/

Build Success with Spark 2.3.4, Please check CI http://<IP_ADDRESS>:12602/job/ApacheCarbonPRBuilder2.3/5279/


docker_image - handling movement of the 'latest' tag in an image repository

Cookbook version
4.9.2
Chef-client version
Chef: 14.12.9
Platform Details
CentOS Linux release 7.6.1810 (Core)
Scenario:
I have a docker_image resource, as shown below, and what I'm not sure I understand is if this resource will recognize that the latest tag has been moved by the maintainer of  quay.io/newrelic/synthetics-minion and that it now needs to download the new image that the latest tag is referencing.
Based on what I am seeing in the code of this cookbook, here, I do not think it will.  It looks like it is only considering the name of the tag, not if the image ID has changed.
docker_image 'newrelic-minion' do
  tag 'latest'
  repo 'quay.io/newrelic/synthetics-minion'
  action :pull_if_missing
end

Steps to Reproduce:
Use the code example from the Scenario section above.
The code will download a docker image.
Move the latest tag in the repo to a newer image.
Run chef-client again and observe that even though the latest tag is pointing to a newer image, the newer image is not downloaded.
Expected Result:
Can you confirm if the latest tag has been moved in a repo to a new image, if the code I pasted above should download it automatically on the next chef run?
Actual Result:
It does not appear that the code example I've provided downloads the newer image when the latest tag is moved in a repo.

@mlcooper if this resource will recognize that the latest tag has been moved by the maintainer of quay.io/newrelic/synthetics-minion and that it now needs to download the new image that the latest tag is referencing
In this case you should use pull action instead of pull_if_missing.  pull action identify the new image with latest tag if the image ID has changed and download it automatically on the next chef run.

@mlcooper Please check ^^ and confirm.

@mlcooper I'm closing this ticket based on comment.


Social Links Formatting

"i found this slight style bug on mobile, I'm guessing you could fix this by removing the height in CSS on that box"

From Maple

Feedback from @itsMapleLeaf

Interesting! I suspect it comes from the footer component's tailwind:
relative grid grid-cols-2 gap-4 sm:grid-cols-4
Specifically, it says that on small screens, each row of icons should have 2 elements, or columns.
Maybe trying grid-cols-3 might make a good workaround?


Roll Flutter Engine from cff1e751f853 to bc2791d5ad23 (1 revision)

https://github.com/flutter/engine/compare/cff1e751f853...bc2791d5ad23
2024-11-19<EMAIL_ADDRESS>EmbedderTest: Eliminate unused include (flutter/engine#56698)
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC aaclarke<EMAIL_ADDRESS>on the revert to ensure that a human
is aware of the problem.
To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose
To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

Commit queue failed; closing this roll.


Game crash occurs when a minecart structure with a chest is materialized (can be reliably reproduced)

Describe the Bug
(This is my very first time to open an issue, so make me noticed if there's anything I didn't explain clearly.)
I was playing on minecraft 1.20, with fabric 0.14.21, fabric api-0.86.1 and create-0.5.1d(which is the only mod loaded, as shown in the mc crash report below), and discovered a way to get game crashed.
The crash-report, minecart mashine blueprint and a world save are attached.
Reproduction Steps

build the machine, either by placing blocks on your own or directly print it in creative mode.
right click the clockwork bearing, so the shaft on it becomes an entity.
turn on the lever, the minecart machine becomes an entity.
fill the chest on the machine with the item specified by the brass funnel(I used cogwheel in the blueprint).
close the chest, and turn down the lever. The whole machine turns into blocks again.
turn on the lever again, and take items out of the chest(either only take a few or clear the chest is OK).
finally, turn down the lever - boom!

Expected Result
The game shouldn't crash. I guess the new feature allowing chests on moving structure being opened should be blamed...
Screenshots and Videos
No response
Crash Report or Log
https://github.com/eyja42/create_crash_report_2023_9
Operating System
Windows 11
Mod Version
0.5.1d
Minecraft Version
1.20.1
Other Mods
no other mods
Additional Context
No response

I am experiencing the same crash it seems:
crash-2023-09-09_21.17.49-server.txt

duplicate of #1080


Could we compose switch%platform with other variables?

Could we have a composable switch%platform with other variables?
Instead of having:
switch%platform (Runtime.platform) {
  | Client =>
    switch (foo) {
    | (true) =>  // ...
    | (false) =>  // ...
    }
  | Server =>  // ...
}

switch%platform (Runtime.platform, foo) {
  | (Client, _) =>  // ...
  | (Server, false) =>  // ...
  // ...
}

Does it make sense?

We chatted offline and decided to won't complicate the ppx


Inserter texture miscoordinate on low-res graphics

Good day!
When yellow inserters pick up anything texture of hand flies in air near him. Did not check with any other inserters except burning (all is ok with him).
This happen only on low-res texture graphics and with every version of mod from 0.0.22 to last. Use with bob's inserters mod (1.0.1).

That is bizarre. The bug only happens for hand-closed, not hand-open, but the graphics are specified by the exact same code, the only difference is that the closed-state uses the string "closed" instead of "open" in the file reference, and the files themselves are properly positioned. I'm investigating.

This may be a Factorio bug:

Every single hand picture is defined using the exact same function, just pointing to different images. I'm remaking the images on the off chance something is wrong with the png file.

Yep, I've tried to investigate himself, but can't find any error in mod code. Also came to the conclusion that this may be the problem of the Factory itself and finaly report here.

Ah, nope, found it. I was looking at the inserter-closed-mask file, when I needed to be looking at the long-handed-inserter-closed-mask file. It was not properly rescaled.

I will fix this as part of the regular update cycle, but if you would like to fix it in your game now you can download the corrected image and replace your local copy.

Yep, thank you very much!

All is super now! Thank you again!


Centered barcode inside fixed size canvas?

Hey, awesome library!
I am trying to draw centered barcodes in a canvas of a fixed size, say: 1920x1080
I have experimented with the config parameters, but it doesn't seem there isn't a straight-forward way
to get a particular sized canvas. JsBarcode will set the height/width based upon it's results.
I was almost able to get there by using margins, but it's not straightforward to compute the font width, etc.
I suppose I could render each image twice, and use the results from the first render to algebraically solve
for the margin numbers I need to get a 1920x1080 canvas, but that's a bit rough, and I am in
a performance sensitive application.
Any suggestions on how to get the barcode rendered into a pre-sized canvas without resizing?
Thanks
Cameron

JsBarcode does not support fixed with fixed margins since that would require squashing of pixels and make the barcode unreadable.
What you could do it follow this: https://stackoverflow.com/questions/11625767/canvas-inside-canvas-of-html5
Generate a barcode that you know will fit in the 1920 width. Then center that in the 1920x1080 canvas.

JsBarcode does not support fixed with fixed margins since that would require squashing of pixels and make the barcode unreadable.
What you could do it follow this: https://stackoverflow.com/questions/11625767/canvas-inside-canvas-of-html5
Generate a barcode that you know will fit in the 1920 width. Then center that in the 1920x1080 canvas.


Add docker support

I really like the simplicity and all but it really needs docker support

Well, I'm not very familiar with Docker, but it's actually a good idea to add support for it.
Thanks for checking out my project!

Okay, it kind of works but binding mounts is broken, so Status cannot list disks and count processess properly. It is definitely a problem with docker-compose.yml file.
Moreover, noticed that SELinux set to 1 causes PermissionDenied when trying to read /proc/1/status (but this can be fixed easily I think)
So, leaving this open as I don't know how to set up volumes properly

Here is the example from the tool I'm using right now.
Basically it just mounts everything it needs
'''
docker container run -it
-p 80:3001
-v /etc/os-release:/etc/os-release:ro
-v /proc/1/ns/net:/mnt/host_ns_net:ro
--privileged
mauricenino/dashdot
'''

Okay, I think I see the problem now; Running
docker container run -it -p 9090:9090
-v /dev:/dev:ro
-v /etc/os-release:/etc/os-release:ro
-v /etc/hostname:/etc/hostname:ro
--privileged status-app

throws me this
docker: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/proc/net" to rootfs at "/proc/net": "/var/lib/docker/overlay2/f272ca448e278361ad962fe41002bb0e413a42fa9b2923cf3953f1e2d2088f30/merged/proc/1/net" cannot be mounted because it is inside /proc: unknown.

and, after a bit of searching I found this and what I was trying to do was actually impossible.
Dashdot you are using seems to check if it's running in Docker, so I think the solution would be to simply mount all required stuff into a container subdirectory (like /machine_root) and tell Status this is the right root. Or, if it's possible, detect everything automatically without setting any additional variables.
I will try to check this solution tomorrow.

It somehow works, but still many things are broken (cpu freq, cpu temp, storage & network). The weird behaviour is probably caused by symlinks (seems Docker volumes are not designed to work with them) and for now, I don't know how to solve it. Maybe there is a way getting the path symlink is pointing to.
Here are some examples from directories Status is taking information from:

CPU frequencies (look at cpufreq)

$ ls -l /sys/devices/system/cpu/cpu0/
total 0
drwxr-xr-x. 2 root root    0 May  6 10:48 acpi_cppc
drwxr-xr-x. 6 root root    0 May  5 19:13 cache
lrwxrwxrwx. 1 root root    0 May  5 19:13 cpufreq -> ../cpufreq/policy0
drwxr-xr-x. 6 root root    0 May  6 10:48 cpuidle
-r--------. 1 root root 4096 May  6 10:48 crash_notes
-r--------. 1 root root 4096 May  6 10:48 crash_notes_size
lrwxrwxrwx. 1 root root    0 May  6 10:48 driver -> ../../../../bus/cpu/drivers/processor
lrwxrwxrwx. 1 root root    0 May  6 10:48 firmware_node -> ../../../LNXSYSTM:00/LNXSYBUS:00/LNXCPU:00
drwxr-xr-x. 2 root root    0 May  6 10:48 hotplug
drwxr-xr-x. 2 root root    0 May  6 10:48 microcode
lrwxrwxrwx. 1 root root    0 May  6 10:48 node0 -> ../../node/node0
drwxr-xr-x. 2 root root    0 May  5 19:13 power
lrwxrwxrwx. 1 root root    0 May  5 22:06 subsystem -> ../../../../bus/cpu
drwxr-xr-x. 2 root root    0 May  6 10:48 thermal_throttle
drwxr-xr-x. 2 root root    0 May  5 19:13 topology
-rw-r--r--. 1 root root 4096 May  5 22:06 uevent

Temperature sensors

$ ls -l /sys/class/hwmon/
total 0
lrwxrwxrwx. 1 root root 0 May  5 19:13 hwmon0 -> ../../devices/virtual/thermal/thermal_zone0/hwmon0
lrwxrwxrwx. 1 root root 0 May  5 23:29 hwmon1 -> ../../devices/pci0000:00/0000:00:06.0/0000:02:00.0/nvme/nvme0/hwmon1
lrwxrwxrwx. 1 root root 0 May  5 19:13 hwmon2 -> ../../devices/platform/coretemp.0/hwmon/hwmon2
lrwxrwxrwx. 1 root root 0 May  5 19:13 hwmon3 -> ../../devices/platform/eeepc-wmi/hwmon/hwmon3

As we can see, those symlinks are pointing to kinda weird paths, and those will definitely vary between different distributions and hardware.
For now, leaving this issue open. I have very little experience with Docker so somebody else will be needed to help fixing this problem.

I tried the following DOCKERFILE to host this using Portainer:
FROM ubuntu
RUN apt-get update && apt-get install -y git python3 python3-pip
WORKDIR /app
RUN git clone https://github.com/dani3l0/Status .
RUN pip3 install --no-cache-dir --disable-pip-version-check -r requirements.txt
ENV STATUS_CUSTOM_ROOT=/mnt/host
EXPOSE 9090
CMD ["python3", "status.py", "--no-config"]

This does work and shows correct system details when accessing it via <ip>:9090 or <hostname>:9090.
However I was not able to make it work when trying to put it behind my NGINX reverse proxy.
Not sure if this is of any help in this issue, I just figured I'd post my (somewhat) successful container deployment.

Thanks for your reply, your solution works way better. However, still some features are broken due to container isolation (storage, network, etc), but I believe with little tweaks in docker-compose we can get a fully working Docker instance.
When it comes to proxy, I just tried running it under Caddy with /status root path, all worked as expected. Might be a problem in ur config?
Anyways, I will play around Docker configuration today and push changes shortly. Thanks for help, much appreciated.

Okay, thanks to your typo in environment variable and little stackoverflow help I was able to find a decent solution. Old Dockerfile was ok, but docker-compose needed some tweaks.
Docker support status:

CPU - all seems to be ok
Memory - fixed process count, section looks good for now
Storage - should work in most cases, but some custom configurations may cause problems
Network - fixed, using host network
Host - fixed hostname and OS infomation, all info here seems to be ok

Lemme guys know if there are still any remaining problems, hope it's much better now


bg image wasn't wide enough

on a macbook 12"

You gotta set the bg color yourself. I guess I could detect screen size and fill the image to size

oh ok

See note at bottom of usage :point_right: https://github.com/ngoldman/himawari-bg#usage
Think it's worth doing the screen size detection and fill? For now it's just taking whatever image is handed off from himawari.js and passing that to wallpaper.

Ooh turns out when I changed this before setting it then it did what I
wanted
[image: Inline image 1]
Before I had it on 'Center'
On Tue, Feb 9, 2016 at 5:39 PM, ngoldman<EMAIL_ADDRESS>wrote:

See note at bottom of usage [image: :point_right:]
https://github.com/ngoldman/himawari-bg#usage
Think it's worth doing the screen size detection and fill? For now it's
just taking whatever image is handed off from himawari.js and passing
that to wallpaper.
—
Reply to this email directly or view it on GitHub
https://github.com/ngoldman/himawari-bg/issues/4#issuecomment-182162266.

Marking this as resolved for now :sparkles:


limited to 10,000 members

I tried to use this command, but it failed;
The change that you're trying to make contains 13975 distinguished name values. This exceeds the limit of 10000 DN
values.
+ CategoryInfo          : NotSpecified: (:) [Update-DistributionGroupMember], ADOperationException
+ FullyQualifiedErrorId : [Server=DB3PR0302MB3388,RequestId=c9fc8033-97d4-43d6-95aa-5be5e2072fa8,TimeStamp=18/05/2
021 11:08:48] [FailureCategory=Cmdlet-ADOperationException] 7AD20693,Microsoft.Exchange.Management.RecipientTasks.
UpdateDistributionGroupMember
Please add this limit.
[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 18125af8-f8f7-a031-2a8f-270907d34b54
Version Independent ID: 3e873980-c99b-0cfb-1e8a-0af4314d2d97
Content: Update-DistributionGroupMember (ExchangePowerShell)
Content Source: exchange/exchange-ps/exchange/Update-DistributionGroupMember.md
Service: exchange-powershell
GitHub Login: @chrisda
Microsoft Alias: chrisda

@anwarmahmood1, OK. Done.
Thanks for the feedback.
Closing


cypress initial setup

Initial PR for Cypress Tests.
This initial commit includes one sample test where we just login to ACM.

So all the files under test/e2e/cypress-ui/cypress/sample are for a simple test or utilities to be used by Cypress? Just seems like a lot of files so wanted to be suer this wasn't for a generic cypress example...

Tests will be under cypress/tests - you are correct that the ones under /sample are generic cypress examples. I'll remove them.

/lgtm


Added code_line_description_json_task

@googlebot I signed it!

Thank you for submitting your task! We are in the process of pairing submissions with task reviewers and will be in touch soon.

I am one of the two reviewers assigned to review this task.
Thank you for your submission! I like the idea of a task testing whether a model can demonstrate understanding of code, by describing what the code does. However, I do not feel that this tasks meets the bar for inclusion in its current state. I do think this task can be improved, and then included. Some specific concerns follow:

Note that the model will propose natural continuations of the input string, and doesn't know anything explicit about distinctions between inputs and targets. For that reason it might be more natural to have an input that looked like: "\nPython code:\n[Python code example]\nEnglish language description:\n" and to override the example_output_prefix and example_input_prefix. Currently, the language model will be asked to generate strings like "Q: print(type(x)A: prints data type of the variable x" which may be unnatural.
Some of the Python code has inconsistent preceding or trailing whitespace, or is malformed (eg print(type(x) is missing a closing parenthesis, if(j<0): print('negative')  is missing a new-line and has a trailing whitespace)
Some of the descriptions of code output seem confusing -- for instance print('Hello World'[1:5]) does not print "Hello World" despite being described that way in the description, and the description of the pieces of the try/except/finally block seem confusing in isolation.
I'm concerned that bleu or rouge are bad choices for measuring performance here. There are many equally good ways to describe the code snippets, and most of those would probably generate bleu or rouge scores that are similar to what you would get from for instance just giving the code snippet again as an output.

I think a task like this might be a better match for multiple choice, where different multiple choice responses could correspond to descriptions of the task that were high quality, subtly wrong, and obviously wrong.

I wonder if it might make more sense to describe short blocks of code, rather than restricting the code to only be single lines. This would allow the code to more naturally do more complex things, and would make it easier to describe things like a try/except/finally block.
Closely related, and in the interest of having graded task difficulty -- I think all the code here is fairly straightforward. It would be good to have a wider range of code complexity, and see if the model can understand what more complex or obfuscated code examples are doing.
The README.md appears incomplete.
I would encourage you to check out the review guidelines at https://github.com/google/BIG-bench/blob/main/docs/doc.md#submission-review-process ."

Thank you for the submission!  Overall, I'm excited for tasks that test a models ability to understand and execute code. This task is a step in the right direction. I believe it can be significantly improved and should be before becoming a benchmark

correctness

The task is valid & passes the tests
There are several typos in the description & tasks which need to be addressed

examples

print(type(x) without closing bracket
"mutiples elements"

inconsistencies across answers

some strings have '' around them while others do not 'hello world' vs helloworld in 1st and 3rd examples
the same action has different comments (sometimes assigning is in the comment, while other times just what is being assigned is mentioned)

sometimes is specific about content, other times not (example: "stores multiple elements in a tuple" does not mention what is stored, while several other lines do.

inconsistency on spacing throughout

specificity

This does not appear to be a zero shot task. The natural completion given the input is additional code, not a description of what the code does, as such simple code examples also do not typically appear with comments. Adding comment syntax could help here.

example: x = y = z = "hello world"  # This line

The task mixes asking the model for rough understanding of python semantics (like x = 5 assigns to the variable x), with actually "executing" to get the result, such as in print(5+4) ⇒ prints 9

thoroughness

The task tests a variety of python methods.
Does not test how a model performs under mild changes to the same code. It is possible that the model knows what hello world does because it is everywhere, but will fail with a less common string.

difficulty

This task is not trivially solvable by existing language models in its current form (I tried). I suspect that additional prompting / few shot examples would significantly improve results.

not solvable by memorizing the internet

I believe this task (once additional contextual information is provided) could be solved by a model which has memorized python coding tutorials

novelty

I am not aware of other python statement to code datasets
There exist much larger  derived python function —> docstring datasets which may provide a more natural testbed for code understanding

https://github.com/EdinburghNLP/code-docstring-corpus
Tricky thing is that these are inherently solvable by memorizing github.

justification

Code understanding is an important problem for language models. The description explains the task but does not go into much detail as to why this is important.
The description contains examples that are from C (or some language with explicit int types), but says it is testing Python.

size

The task is the minimal acceptable size and the interactions are very short. The task should be expanded to include additional examples.

compute resources

Task is straightforward to run, requiring a single inference on each input.

Reading Jascha's comment, I agree on all counts.
Echoing a few specific points

There are many valid ways to describe code (as shown by the task itself including very different descriptions for similar tasks), so BLEU/ROUGE my not be the right choice
Multiple choice could be a good solution to this issue. One has to be careful when creating the choices.
Restricting to statements is a nice constraint, but also limits the range of what the task can test.
The task will benefit from a larger variety of test cases for different aspects of Python, and many variations within a single or set of language constructs

Thanks for the reviews, will update addressing the feedback!

@Sohl-Dickstein and @dmrd thank you for the feedback. I was wondering if I could limit the task to predict the output of a python code blocks that are fairly easy to human beings who are familiar with python limiting output to 'print' statements only. For this particular task, creating multiple choices for natural language explanation of code block turns out to be pretty complex. The true answer for a code block can incorporate both syntactical notion of the code as well as logical. For example: for i in range(10):\nprint(i) can have explanation as 'prints values from 0 to 10', 'prints 0,1,2,3,4,5,6,7,8,9'.

Limiting to output of print statements would be fine, hopefully with some range of complexity over the code.
Note though that if you do want to use multiple choice, it's OK if there are multiple ways a correct answer could be phrased. For instance for your example you could have:
prompt text: for i in range(10):\nprint(i)
possible English language descriptions: ["prints values from 0 to 10", "prints values from 1 to 10", "computes the first ten prime numbers", "prints 'hello world' to the terminal"]
here only one of the multiple choice examples is correct, despite their being multiple ways to correctly describe the code. See https://github.com/google/BIG-bench/blob/main/bigbench/benchmark_tasks/simple_arithmetic_json_multiple_choice/task.json for an example using multiple choice.

A related framing which which removes the ambiguity / possible multiple ways to answer is to focus on having the LM "execute" the code and predict what would be printed to stdout. This is particularly interesting if it requires binding multiple variables, like x = 1; y = 2; z = x + y; print(-z) --> -3
This may be sufficiently different to warrant its own problem.
Example:
for i in range(10):
  print(i)
# The code above prints the following to stdout:
1
2
3
4
5
6
7
8
9

A related framing which which removes the ambiguity / possible multiple ways to answer is to focus on having the LM "execute" the code and predict what would be printed to stdout. This is particularly interesting if it requires binding multiple variables, like x = 1; y = 2; z = x + y; print(-z) --> -3
This may be sufficiently different to warrant its own problem.
Example:
for i in range(10):
  print(i)
# The code above prints the following to stdout:
1
2
3
4
5
6
7
8
9

Thanks for the suggestion, limiting to output actually helps in constraining the task to a specific outcome. If possible I will be submitting another pull request with output centric task as well. For now, the task I will be submitting will look something like 'which of the following about this code snippet true?'

@dmrd Please signal to "accept" or "reject" by Friday, March 26th.

@chiafullo accept
This is a great improvement over the original version - thank you for submitting!

@lewkowycz for meta review and merge (if acceptable)

I am metareviewing this task and I will merge it.


PRM sample

Add a new WDF driver sample to demonstrate the usage of the PRM direct-call interface.

Unless you expect to have more PRM samples, you should move the contents of prm/samples to simply prm to simplify folder structure.
If possible, please add support for ARM64.

@NeoAdonis : Sample still does not build!!  See build failure.

@NeoAdonis : Sample still does not build!! See build failure.

@JakobL-MSFT What is the WDK version that you are building with? This is a new sample reliant on Ge WDK headers (prminterface.h).

@JakobL-MSFT What is the WDK version that you are building with? This is a new sample reliant on Ge WDK headers (prminterface.h).

Now I understand!!!
First. You need to target our "develop" branch.  Not "main".
Second.  The samples in the develop branch needs to build with (1) "in market" EWDK and (2) also GE EWDK.  Exceptions can happen (as in your case) and we add them to .\exclusions.csv file in relevant branch.  The entry would look as follows:
prm\prmsample,*,,22621,Only NI: Not supported on NI.
(Please note the .\exclusion.csv file differs between main and develop branch.  You need to edit based on the develop branch version)
I can make the edit above (or you can add it - please keep the entries sorted)
Cheers.
Jakob


api: record update fix

FIX Loads all recordext functions registered before
updating a record via Record.commit().

Signed-off-by: Jan Aage Lavik<EMAIL_ADDRESS>

@jalavik JSONAlchemy is going to be removed. You can copy some parts of the registry to invenio_records/registry.py, however new hook/signal loader is in preparations. We need to finish with package release sprint first.

@jirikuncar Aight, cool. I'll wait for the new hook setup and then see.


Adds directories

I don't think the test will pass. Now each directory has its own .travis.yml, but there should be one in the top level.

@tmatsuo I will remove those.

I suppose since each step is supposed to be taken as a standalone app, it's ok to have travis files in each directory. What do you think?

Travis doesn't look at them, so those are nothing to do with our test run. I would remove them.

closing in favor of #30 so we can run tests


checkTarget not working while user just lost connection.

This problem never happed to me yet, but by looking at the code I can imagine this happening sometimes.
I hope you can take a look at it!
Lets say we have 2 clients (client 1 and client 2).
If client 1 sends a message to client 2, normally the GCM plugin will check if the user is offline. If the user is offline it wil send the message to the GCM.
Now I have got the following problem.
If client 2 loses it's connection because the wifi stopped working for example, Openfire doesn't immediately detect that the user is offline. Openfire pings the client every X seconds and if the client doesn't respond then Openfire sais the client is offline.
So this is what happens between the connection loss and the Openfire ping.

Client 1 sends a message to client 2.
Client 2 just lost connection but Openfire does not know it yet because it didn't ping client 2 yet.
Client 1's message triggered the gcm plugin
The GCM plugin thinks the user is online because Openfire still didn't ping client 2.
The GCM plugin does nothing
Then eventually Openfire will ping client 2 and detect it is offline.

So because of the gap between connection loss and ping sometimes clients will not receive any notification.

We have developed the plugin to ensure that no messages are lost (in the context of Openfire).
Using Openfire, in the presence of a weak network, messages/packets are lost.
Hence, to ensure deliveries (at-least-once) and no message/packet losses we have developed the plugin.
The plugin ensures 'Stanza Acknowledgements' (the ability to know if messages have been received by one's peers) with no losses (at-least-once deliveries).
For more information, please see-
https://www.atklique.com/bridge/blog/?q=node/5
For more information (and integration of the plugin with Openfire), please drop an email to<EMAIL_ADDRESS>with you requirements.

https://www.atklique.com/bridge/blog/?q=node/5
This link seen dead.
Please Provide jar file of plugin or another Link.
I am very thankful to you for this.


dynamic configuration of Traefik with Consul Template

This will make it way easier to remove Ansible from edge nodes

We'll probably want to wait on https://github.com/hashicorp/consul-template/pull/675 to move forward on this, as it'll make it a lot easier.

Now we're just waiting on Consul-Template v0.17.0: https://github.com/hashicorp/consul-template/releases

@siddharthist Is this available? Looking for a consul template to dynamically update traefik.

@gvenka008c No, and I don't expect it will be soon. Sorry!


Question: Pivoting

I've been wondering what the best way to accomplish pivoting with commando vm would be.
Was hoping someone could bounce some ideas around, I've seen this https://github.com/shunf4/proxychains-windows but haven't heard too much about it.  Initially I suppose we could just do all this from within a docker container but it would be cool to find out if this is something we can accomplish through the WSL2 or Windows OS level.  I'm sure we can create a single tunnel, but what about performing something akin to: proxychains nmap -p- targetip

@digitalohm Sorry I did not respond earlier. I would recommend using a C2 framework for this. C2's such as Covenant (or Cobalt Strike) have built in port scanners within their implants.

This could also be a whole conversation because it also might depend on what your goals are and what situation you are in. I would invite you to join the BloodHound Gang Slack channel if you would like to chat about it. You can join the #commando-vm channel or DM me directly (day1player).

Thanks @day1player the herokuapp links for the invites do not seem to be working :(

Uh oh.. Slack is kind of a pain. Are you getting an error?

Yeah the link  will time-out With he following error

Application error
An error occurred in the application and your page could not be served. If you are the application owner, check your logs for details. You can do this from the Heroku CLI with the command
heroku logs --tail

@digitalohm it looks like they disabled that invite link for the time being, can you send me your email to<EMAIL_ADDRESS>I can submit a request in the bloodhound slack to get you an invite.

@day1player
Your message to<EMAIL_ADDRESS>couldn't be delivered.
The group commandovm only accepts messages from people in its organization =
or on its allowed senders list, and your email address isn't on the list.

:(

@digitalohm 🤦
You can email me directly<EMAIL_ADDRESS>


Slow AGA playback performance

I am seeing very slow AGA animation playback ob Amiga A1200. When compared to same Amiga with 68060 accelerator. Considering how fast the PiStorm32 is, this seems weird. I know others ave shown certain AGA games and even non AGA games running slow. This maybe similar issue?
Here are links to the two test files I was using. They are extreme examples but the idea was to push the hardware. Either real 68060 or devices like PiStorms.
HAM6 ANIM: https://drive.google.com/file/d/1RP-bD8ktdwqTNmUBqyE6z1aS05qCAhJ9/view?usp=share_link
HAM8 ANIM: https://drive.google.com/file/d/1RWZ4pzAwK7EIeeuaI8MJYQgJKRSbSyna/view?usp=share_link

Issue has been moved to gateware.

The issue is partially identified and covers both FPGA firmware and Emu68. In short words - in FPGA we add one or two unneeded wait states when setting up control lines, also, the clock generated from A1200 14.28MHz is less than optimal and will be fixed too.
On emu68 side there is no read or write buffer. Therefore all transfers to CHIP occur at the bit width given by the opcode:

As you may see on the sampled data above the HAM8 animation (same applies to HAM6 version), RLE-compressed, iswritten to CHIP memory byte by byte. Had Emu68 a write buffer, these could be combined to 32bit writes and had more chance to utilise the memory bandwidth.

Great to see this! If you need anymore examples/testing let me know. Thanks!

For clearance, updated diagram with 68020 states shown. You might notice that we are slightly out of sync. You may also see there that Pistorm protocol occupies small amout of time compared to CHIP transfer itself. The short period between "PS proto" and "S0" is the time where we are syncing to bus clock cycle - the same happens on accelerators working with any other clock than 14.28MHz

Please give following firmware a try:
PS32-lite.hex.bin.gz
To use it you must put it as it is (in .gz format!) on your boot partition. Then, modify your initramfs line from e.g.
initramfs kick.rom

to
initramfs PS32-lite.hex.bin.gz,kick.rom

This version of firmware reports write completion earlier, so that in case of subsequent writes the Emu68 can start pushing new data slightly before the previous transfer completed. This allows faster bulk writes to chip memory. Also, the timing is slightly improved.

Another thing which yoiu might be interested in, will show how the performance can increase when write buffer in Emu68 is ready. Please use e.g. MainActor to convert any of the animations you linked from ANIM7 to ANIM8. MainActor will store the animation compressed using LONG words, instead of bytes, so that, during playback, the data will be pushed to Amiga bus in 32 bit packets, instead of 8 bit packets (effectively giving a boost ratio of more than 4x).
For my local test with above linked firmware and ANIM8 format, the hi-res ham8 played at something around 15 fps

New PiStorm32-lite Emu68 image:
Emu68-pistorm32lite.zip

This is promising. It’s good to know of optimized settings to use going forward. Obviously converting all prior ANIMS one may encounter for smooth playback is not practical. Still, progress is always welcome!

I'm not a techie at all. I am not sure I understand of the two files you've posted, what exactly to do with them. I did read the procedure for the hex.bin file. Then later I see you've posted that image. Understand, I am a child with this stuff. I don't really understand what I need to do. I generally wait for, "run this updater" Heh. So, I apologize if I am slow to act. I'm hoping others much smarter than me are looking at this too. Thank you for your work and time.

 I had ZERO luck getting MainActor to convert the file as you asked. It kept corrupting the palette. HOWEVER. I used Art Department Pro and used its easy CONVERT ANIM option. This made a successful ANIM8 format file from the ANIM5. I also set to LONG word. I can confirm, when playing back in MainActors player, not only is HAM8 mode working it's around 10-12fps. Which is a big improvement over the 1-2fps I was getting before the update. Still not as fast as the 68060 A4000, but MUCH closer.


nerdctl run argument parsing order potentially broken?

From nerdctl run --help:
Usage:
  nerdctl run IMAGE [COMMAND] [ARG...] [flags]

From docker run --help:
Usage:  docker run [OPTIONS] IMAGE [COMMAND] [ARG...]

We are trying to run a container in a script with a complex set of options passed to the command but because of this ordering it's not clear that nerdctl can safely disambiguate between arguments it should process and arguments for the spawned container.
The docker equivalent command looks like this:
docker run --rm --mount type=bind,source=A_PATH,target=A_PATH --env PGHOST=host.lima.internal --env PGPORT=5432 --env PGUSER=postgres --env PGPASSWORD=password -it postgres:12.3 pg_dump --schema-only --no-privileges --no-owner --file A_FILE db_name

If I attempt to use the argument order in the help, it of course tries to pass run arguments like --rm and --env to the pg_dump binary in the container and fails. If someone can confirm this is unexpected, I may be able to take a look at resolving it.

Just the --help text is broken.
The actual implementation is correct AFAIC, but let us know if something is actually broken.

@AkihiroSuda Ahh, I see. I think this actually stems from the fact that I am trying to use --mount which is (it seems) unsupported by nerdctl and thus it stops argument parsing and gives me this error:
FATA[0000] requires at least 1 arg(s), only received 0

FATA[0000] requires at least 1 arg(s), only received 0

cc @robberphex
Can we improve this error message?

ok, I will fix this later.


eventbridge-kafka-connector v1.3.1+ breaks authentication with kube2iam (InstanceProfileCredentialsProvider)

Describe the bug
Commit 1bc23b83f65659ba2cac3c637974cddc04b63dc9 introduced a change in v1.3.1 to Leverage DefaultSupplier to automatically reload credentials on file refresh. This change causes authentication to fail when using kube2iam which in turn uses InstanceProfileCredentialsProvider.
To Reproduce
Steps to reproduce the behavior:

Deploy kube2iam in a k8s cluster
Deploy a KafkaConnect deployment with kube2iam annotations for a role:

iam.amazonaws.com/role: <role>

Deploy an eventbridge-kafka-connector sink to the KafkaConnect cluster in Step 2.
Watch authentication fail with the following message:

{
  "@timestamp": "2024-09-16T19:40:38.334Z",
  "source_host": "cxp-connect-eventbridge-connect-0",
  "file": "EventBridgeSinkTask.java",
  "method": "handleFailedEntries",
  "level": "WARN",
  "line_number": "145",
  "thread_name": "task-thread-eventbridge-sink-0",
  "@version": 1,
  "logger_name": "software.amazon.event.kafkaconnector.EventBridgeSinkTask",
  "message": "Retrying failed putItems call: attempts=1 maxRetries=2 errorMessage=software.amazon.awssdk.core.exception.SdkClientException: Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[SystemPropertyCredentialsProvider(), EnvironmentVariableCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(profileName=default, profileFile=ProfileFile(profilesAndSectionsMap=[])), ContainerCredentialsProvider(), InstanceProfileCredentialsProvider()]) : [SystemPropertyCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., EnvironmentVariableCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., WebIdentityTokenCredentialsProvider(): Either the environment variable AWS_WEB_IDENTITY_TOKEN_FILE or the javaproperty aws.webIdentityTokenFile must be set., ProfileCredentialsProvider(profileName=default, profileFile=ProfileFile(profilesAndSectionsMap=[])): Profile file contained no credentials for profile 'default': ProfileFile(profilesAndSectionsMap=[]), ContainerCredentialsProvider(): Cannot fetch credentials from container - neither AWS_CONTAINER_CREDENTIALS_FULL_URI or AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variables are set., InstanceProfileCredentialsProvider(): Failed to load credentials from IMDS.]",
  "class": "software.amazon.event.kafkaconnector.EventBridgeSinkTask",
  "mdc": {
    "connector.context": "[eventbridge-sink|task-0] "
  }
}

Expected behavior
The connector should be able to authenticate using the InstancProfileCredentialsProvider. Versions prior to v1.3.1 would have an authentication log like the following:
cxp-connect-eventbridge-connect-0 cxp-connect-eventbridge-connect {"@timestamp":"2024-09-18T18:12:32.669Z","source_host":"cxp-connect-eventbridge-connect-0","file":"EventBridgeWriter.java","method":"<init>","level":"INFO","line_number":"137","thread_name":"task-thread-eventbridge-sink-0","@version":1,"logger_name":"software.amazon.event.kafkaconnector.EventBridgeWriter","message":"[@9d66194-dirty] Resolving iam credentials","class":"software.amazon.event.kafkaconnector.EventBridgeWriter","mdc":{"connector.context":"[eventbridge-sink|task-0] "}}
cxp-connect-eventbridge-connect-0 cxp-connect-eventbridge-connect {"@timestamp":"2024-09-18T18:12:32.791Z","source_host":"cxp-connect-eventbridge-connect-0","file":"Logger.java","method":"debug","level":"DEBUG","line_number":"85","thread_name":"task-thread-eventbridge-sink-0","@version":1,"logger_name":"software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider","message":"Loaded credentials from IMDS with expiration time of 2024-09-18T18:42:32Z","class":"software.amazon.awssdk.utils.Logger","mdc":{"connector.context":"[eventbridge-sink|task-0] "}}

Environment:
Client Version: v1.29.6
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.30.3
Java 11

Additional context
I reverted commit 1bc23b83f65659ba2cac3c637974cddc04b63dc9 and redeployed the connector and the expected behavior was observed:
cxp-connect-eventbridge-connect-0 cxp-connect-eventbridge-connect {"@timestamp":"2024-09-18T18:12:32.669Z","source_host":"cxp-connect-eventbridge-connect-0","file":"EventBridgeWriter.java","method":"<init>","level":"INFO","line_number":"137","thread_name":"task-thread-eventbridge-sink-0","@version":1,"logger_name":"software.amazon.event.kafkaconnector.EventBridgeWriter","message":"[@9d66194-dirty] Resolving iam credentials","class":"software.amazon.event.kafkaconnector.EventBridgeWriter","mdc":{"connector.context":"[eventbridge-sink|task-0] "}}
cxp-connect-eventbridge-connect-0 cxp-connect-eventbridge-connect {"@timestamp":"2024-09-18T18:12:32.791Z","source_host":"cxp-connect-eventbridge-connect-0","file":"Logger.java","method":"debug","level":"DEBUG","line_number":"85","thread_name":"task-thread-eventbridge-sink-0","@version":1,"logger_name":"software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider","message":"Loaded credentials from IMDS with expiration time of 2024-09-18T18:42:32Z","class":"software.amazon.awssdk.utils.Logger","mdc":{"connector.context":"[eventbridge-sink|task-0] "}}

@sugarcrm-jgminder thx for flagging this and glad you could revert to the previous version for now. I'll discuss this with the team and see how it can be fixed.
cc/ @maschnetwork

@sugarcrm-jgminder it sounds though that the initial credential load that we do here succeeds and the exception is thrown after a token refresh?

@embano1 Below is a full message after a Kafka Connect pod starts. The credential load fails and it fails to find any suitable credential provider within the default chain:
{
  "exception": {
    "stacktrace": "org.apache.kafka.connect.errors.ConnectException: software.amazon.awssdk.core.exception.SdkClientException: Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[SystemPropertyCredentialsProvider(), EnvironmentVariableCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(profileName=default), ContainerCredentialsProvider(), InstanceProfileCredentialsProvider()]) : [SystemPropertyCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., EnvironmentVariableCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., WebIdentityTokenCredentialsProvider(): Either the environment variable AWS_WEB_IDENTITY_TOKEN_FILE or the javaproperty aws.webIdentityTokenFile must be set., ProfileCredentialsProvider(profileName=default): Cannot invoke \"java.nio.file.Path.getFileSystem()\" because \"path\" is null, ContainerCredentialsProvider(): Cannot fetch credentials from container - neither AWS_CONTAINER_CREDENTIALS_FULL_URI or AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variables are set., InstanceProfileCredentialsProvider(): Failed to load credentials from IMDS.]\n\tat software.amazon.event.kafkaconnector.EventBridgeWriter.<init>(EventBridgeWriter.java:141)\n\tat software.amazon.event.kafkaconnector.EventBridgeSinkTask.start(EventBridgeSinkTask.java:43)\n\tat org.apache.kafka.connect.runtime.WorkerSinkTask.initializeAndStart(WorkerSinkTask.java:329)\n\tat org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:202)\n\tat org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:259)\n\tat org.apache.kafka.connect.runtime.isolation.Plugins.lambda$withClassLoader$1(Plugins.java:236)\n\tat java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)\n\tat java.base/java.lang.Thread.run(Thread.java:840)\nCaused by: software.amazon.awssdk.core.exception.SdkClientException: Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[SystemPropertyCredentialsProvider(), EnvironmentVariableCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(profileName=default), ContainerCredentialsProvider(), InstanceProfileCredentialsProvider()]) : [SystemPropertyCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., EnvironmentVariableCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., WebIdentityTokenCredentialsProvider(): Either the environment variable AWS_WEB_IDENTITY_TOKEN_FILE or the javaproperty aws.webIdentityTokenFile must be set., ProfileCredentialsProvider(profileName=default): Cannot invoke \"java.nio.file.Path.getFileSystem()\" because \"path\" is null, ContainerCredentialsProvider(): Cannot fetch credentials from container - neither AWS_CONTAINER_CREDENTIALS_FULL_URI or AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variables are set., InstanceProfileCredentialsProvider(): Failed to load credentials from IMDS.]\n\tat software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:111)\n\tat software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain.resolveCredentials(AwsCredentialsProviderChain.java:130)\n\tat software.amazon.awssdk.auth.credentials.internal.LazyAwsCredentialsProvider.resolveCredentials(LazyAwsCredentialsProvider.java:45)\n\tat software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider.resolveCredentials(DefaultCredentialsProvider.java:129)\n\tat software.amazon.event.kafkaconnector.EventBridgeWriter.<init>(EventBridgeWriter.java:139)\n\t... 10 more",
    "exception_class": "org.apache.kafka.connect.errors.ConnectException",
    "exception_message": "software.amazon.awssdk.core.exception.SdkClientException: Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[SystemPropertyCredentialsProvider(), EnvironmentVariableCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(profileName=default), ContainerCredentialsProvider(), InstanceProfileCredentialsProvider()]) : [SystemPropertyCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., EnvironmentVariableCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., WebIdentityTokenCredentialsProvider(): Either the environment variable AWS_WEB_IDENTITY_TOKEN_FILE or the javaproperty aws.webIdentityTokenFile must be set., ProfileCredentialsProvider(profileName=default): Cannot invoke \"java.nio.file.Path.getFileSystem()\" because \"path\" is null, ContainerCredentialsProvider(): Cannot fetch credentials from container - neither AWS_CONTAINER_CREDENTIALS_FULL_URI or AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variables are set., InstanceProfileCredentialsProvider(): Failed to load credentials from IMDS.]"
  },
  "source_host": "cxp-connect-eventbridge-connect-0",
  "method": "doRun",
  "level": "ERROR",
  "message": "WorkerSinkTask{id=eventbridge-sink-0} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted",
  "mdc": {
    "connector.context": "[eventbridge-sink|task-0] "
  },
  "@timestamp": "2024-09-17T13:33:44.789Z",
  "file": "WorkerTask.java",
  "line_number": "212",
  "thread_name": "task-thread-eventbridge-sink-0",
  "@version": 1,
  "logger_name": "org.apache.kafka.connect.runtime.WorkerTask",
  "class": "org.apache.kafka.connect.runtime.WorkerTask"
}

Hi @embano1, @maschnetwork,
I had a look how the code behaves if the DefaultCredentialsProvider is build w/ or w/o providing ProfileFileSupplier.defaultSupplier() 1bc23b83f65659ba2cac3c637974cddc04b63dc9. As far as I can see, the build of the DefaultCrednetialsProvider propagates the ProfileFileSupplier to the InstanceProfileCredentialsProvider:
https://github.com/aws/aws-sdk-java-v2/blob/832c1b93a2c2712e65d03f687d11fde8a8a6faa2/core/auth/src/main/java/software/amazon/awssdk/auth/credentials/DefaultCredentialsProvider.java#L106-L110
The InstanceProfileCredentialsProvider propagates it further:
https://github.com/aws/aws-sdk-java-v2/blob/832c1b93a2c2712e65d03f687d11fde8a8a6faa2/core/auth/src/main/java/software/amazon/awssdk/auth/credentials/InstanceProfileCredentialsProvider.java#L107-L112
Finally it's used in
https://github.com/aws/aws-sdk-java-v2/blob/2.28.6/core/auth/src/main/java/software/amazon/awssdk/auth/credentials/internal/Ec2MetadataConfigProvider.java#L86-L105
and
https://github.com/aws/aws-sdk-java-v2/blob/2.28.6/core/auth/src/main/java/software/amazon/awssdk/auth/credentials/internal/Ec2MetadataDisableV1Resolver.java#L47-L51

I tried the following locally:
package software.amazon.event.kafkaconnector;

import software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider;
import software.amazon.awssdk.profiles.ProfileFile;

public class Test {

    public static void main(String[] args) {
        var provider = InstanceProfileCredentialsProvider.builder()
                .asyncCredentialUpdateEnabled(false)
                .profileFile(ProfileFile.defaultProfileFile())
                .profileName(null)
                .build();
        var creds = provider.resolveCredentials();

        System.out.println(creds);
    }

}

It fails with the exception:
Exception in thread "main" software.amazon.awssdk.core.exception.SdkClientException: Failed to load credentials from IMDS.
	at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:111)
	at software.amazon.awssdk.core.exception.SdkClientException.create(SdkClientException.java:47)
	at software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider.refreshCredentials(InstanceProfileCredentialsProvider.java:167)
	at software.amazon.awssdk.utils.cache.CachedSupplier.lambda$jitteredPrefetchValueSupplier$8(CachedSupplier.java:300)
	at software.amazon.awssdk.utils.cache.CachedSupplier$PrefetchStrategy.fetch(CachedSupplier.java:448)
	at software.amazon.awssdk.utils.cache.CachedSupplier.refreshCache(CachedSupplier.java:208)
	at software.amazon.awssdk.utils.cache.CachedSupplier.get(CachedSupplier.java:135)
	at software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider.resolveCredentials(InstanceProfileCredentialsProvider.java:149)
	at software.amazon.event.kafkaconnector.Test.main(Test.java:14)
Caused by: java.io.UncheckedIOException: java.net.SocketTimeoutException: connect timed out
	at software.amazon.awssdk.utils.FunctionalUtils.asRuntimeException(FunctionalUtils.java:180)
	at software.amazon.awssdk.utils.FunctionalUtils.lambda$safeSupplier$4(FunctionalUtils.java:110)
	at software.amazon.awssdk.utils.FunctionalUtils.invokeSafely(FunctionalUtils.java:136)
	at software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider.getSecurityCredentials(InstanceProfileCredentialsProvider.java:283)
	at software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider.createEndpointProvider(InstanceProfileCredentialsProvider.java:212)
	at software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider.refreshCredentials(InstanceProfileCredentialsProvider.java:158)
	... 6 more
Caused by: java.net.SocketTimeoutException: connect timed out
	at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
	at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
	at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
	at java.base/java.net.Socket.connect(Socket.java:609)
	at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:177)
	at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:509)
	at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:604)
	at java.base/sun.net.www.http.HttpClient.<init>(HttpClient.java:277)
	at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:376)
	at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:397)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1252)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1231)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1080)
	at java.base/sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:1014)
	at software.amazon.awssdk.regions.internal.util.ConnectionUtils.connectToEndpoint(ConnectionUtils.java:45)
	at software.amazon.awssdk.regions.util.HttpResourcesUtils.readResource(HttpResourcesUtils.java:112)
	at software.amazon.awssdk.regions.util.HttpResourcesUtils.readResource(HttpResourcesUtils.java:91)
	at software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider.lambda$getSecurityCredentials$3(InstanceProfileCredentialsProvider.java:283)
	at software.amazon.awssdk.utils.FunctionalUtils.lambda$safeSupplier$4(FunctionalUtils.java:108)
	... 10 more

The cause of SdkClientException: Failed to load credentials from IMDS. is in the socket timeout to <IP_ADDRESS>.
Unfortunately we only wrap and propagate the SdkClientException to Kafka Connect:
https://github.com/awslabs/eventbridge-kafka-connector/blob/6e9992493c904a7825ea4f8d9b554c0e2866eecc/src/main/java/software/amazon/event/kafkaconnector/EventBridgeWriter.java#L135-L141
so we are not able to see the root cause.
@embano1, @maschnetwork we should add a detailed log error message before raise the ConnectException to see the root cause.

@sugarcrm-jgminder would you be open to try a test build to debug this further as per @agebhar1's recommendation?
@agebhar1 thx a ton for jumping in here!

@embano1 Sure no problem. I'd be happy to trying test builds out for debugging this issue.

The different behavior of InstanceProfileCredentialsProvider is due to the (default) profile.
package software.amazon.event.kafkaconnector;

import software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider;
import software.amazon.awssdk.profiles.ProfileFile;
import software.amazon.awssdk.profiles.ProfileFileSupplier;

public class Test {

    public static void main(String[] args) {
        var provider = InstanceProfileCredentialsProvider.builder()
                //.profileFile(ProfileFile.defaultProfileFile()) // w/ < v1.3.1
                .profileFile(ProfileFileSupplier.defaultSupplier()) // w >= v1.3.1
                .profileName(null)
                .build();
        var creds = provider.resolveCredentials();

        System.out.println(creds);
    }

}

This yields to the following error, if no AWS profile exists in the filesystem:
Exception in thread "main" software.amazon.awssdk.core.exception.SdkClientException: Failed to load credentials from IMDS.
	at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:111)
	at software.amazon.awssdk.core.exception.SdkClientException.create(SdkClientException.java:47)
	at software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider.refreshCredentials(InstanceProfileCredentialsProvider.java:167)
	at software.amazon.awssdk.utils.cache.CachedSupplier.lambda$jitteredPrefetchValueSupplier$8(CachedSupplier.java:300)
	at software.amazon.awssdk.utils.cache.CachedSupplier$PrefetchStrategy.fetch(CachedSupplier.java:448)
	at software.amazon.awssdk.utils.cache.CachedSupplier.refreshCache(CachedSupplier.java:208)
	at software.amazon.awssdk.utils.cache.CachedSupplier.get(CachedSupplier.java:135)
	at software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider.resolveCredentials(InstanceProfileCredentialsProvider.java:149)
	at software.amazon.event.kafkaconnector.Test.main(Test.java:15)
Caused by: java.lang.NullPointerException
	at java.base/java.nio.file.Files.provider(Files.java:101)
	at java.base/java.nio.file.Files.newInputStream(Files.java:156)
	at software.amazon.awssdk.profiles.ProfileFile$BuilderImpl.lambda$build$0(ProfileFile.java:314)
	at software.amazon.awssdk.utils.FunctionalUtils.lambda$safeSupplier$4(FunctionalUtils.java:108)
	at software.amazon.awssdk.utils.FunctionalUtils.invokeSafely(FunctionalUtils.java:136)
	at software.amazon.awssdk.profiles.ProfileFile$BuilderImpl.build(ProfileFile.java:314)
	at software.amazon.awssdk.profiles.ProfileFileSupplier.lambda$defaultSupplier$2(ProfileFileSupplier.java:58)
	at software.amazon.awssdk.auth.credentials.internal.Ec2MetadataConfigProvider.resolveProfileFile(Ec2MetadataConfigProvider.java:129)
	at software.amazon.awssdk.auth.credentials.internal.Ec2MetadataConfigProvider.resolveProfile(Ec2MetadataConfigProvider.java:121)
	at software.amazon.awssdk.auth.credentials.internal.Ec2MetadataConfigProvider.configFileEndpointOverride(Ec2MetadataConfigProvider.java:117)
	at software.amazon.awssdk.auth.credentials.internal.Ec2MetadataConfigProvider.getEndpointOverride(Ec2MetadataConfigProvider.java:102)
	at software.amazon.awssdk.auth.credentials.internal.Ec2MetadataConfigProvider.getEndpoint(Ec2MetadataConfigProvider.java:70)
	at software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider.getImdsEndpoint(InstanceProfileCredentialsProvider.java:224)
	at software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider.createEndpointProvider(InstanceProfileCredentialsProvider.java:210)
	at software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider.refreshCredentials(InstanceProfileCredentialsProvider.java:158)
	... 6 more

@sugarcrm-jgminder forgot to ask whether IRSA would be an option for you? https://aws.github.io/aws-eks-best-practices/security/docs/iam/#iam-roles-for-service-accounts-irsa

Regarding your question in your test PR whether it can be considered a bug on the SDK behavior, I'm inclined to agree here

You can configure any credentials provider that has a profileFile() method on its builder to reload profile credentials. These credentials profile classes are: ProfileCredentialsProvider, DefaultCredentialsProvider, InstanceProfileCredentialsProvider, and ProfileTokenProvider.

When ProfileCredentialsProvider.resolveCredentials() is called, the SDK for Java reloads the settings. ProfileFileSupplier.defaultSupplier() is one of several convenience implementations of ProfileFileSupplier provided by the SDK.

Reading this in the SDK docs I would expect the convenience method to work in any of the supported cases. @maschnetwork WDYT?

A temporary fix for us would be to implement a custom supplier using the default behavior but catching NPEs from InstanceProfileCredentialsProvider, right?

@embano1 @agebhar1 I'll take a look throughout the weekend

A workaround could be:
package software.amazon.event.kafkaconnector;

import software.amazon.awssdk.profiles.ProfileFile;
import software.amazon.awssdk.profiles.ProfileFileSupplier;

public class Test {

    static ProfileFileSupplier defaultSupplier() {
        return () -> {
            try {
                return ProfileFileSupplier.defaultSupplier().get();
            } catch (final NullPointerException ignored) { // ~/.aws/credentials and ~/.aws/config is missing
                return ProfileFile.defaultProfileFile();
            }
        };
    }

    public static void main(String[] args) {
        var supplier = defaultSupplier(); // instead of ProfileFileSupplier.defaultSupplier()
        var profileFile = supplier.get();
        System.out.println(profileFile);
    }

}

Which works in both cases, presence and absence of ~/.aws/credentials and ~/.aws/config.

Thanks. Indeed this looks like a bug in the SDK. Trying to summarize my debugging process below for reference and documentation.
ProfileFileSupplier.defaultSupplier() effectively returns ProfileFile.builder().build()since credentialsSupplierOptional and configurationSupplierOptional are empty and therefore none of the other branches will catch it. Later during resolveCredentials of the DefaultCredentialsProviderChain the contentLocation within ProfileFile Builder will be null and therefore lead to the Nullpointer:
@Override
public ProfileFile build() {
    InputStream stream = content != null ? content :
                         FunctionalUtils.invokeSafely(() -> Files.newInputStream(contentLocation));

Results in: java.lang.NullPointerException: Cannot invoke "java.nio.file.Path.getFileSystem()" because "path" is null
This will happen first in the ProfileCredentialsProvider and will be ignored to move to the next provider as outlined here. But as @agebhar1 nicely pointed out we'll run into it again for the InstanceProfileCredentialProvider, for example in Ec2MetaDataConfigProvider.
This should be ultimately fixed in the SDK and I'll cut a GIthub issue tomorrow outlining our findings.
As a quick workaround I agree with @agebhar1 last proposal and provide something like a WrappedDefaultProfileFileSupplier - maybe defaulting to ProfileFile.defaultProfileFile() and only when profiles are present delegate to the ProfileFileSupplier.defaultSupplier() - then we can prevent running into the Nullpointer in the first place. Wdyt?

Great work team! @sugarcrm-jgminder is this currently blocking for you? v1.3.x just had minor updates based on some user requests, which I'm not sure you need right now? If so, we can cut a temporary fix as per the above suggestion or wait until it's fixed in the AWS SDK (if they ack it's a bug)?

Created issue here: https://github.com/aws/aws-sdk-java-v2/issues/5635

@embano1 This is not a blocker and I can wait for a resolution on https://github.com/aws/aws-sdk-java-v2/issues/5635.

Perfect, thx a ton for your investigation here and raising the issue!


Help :For eddsa certificates in server mode

For EDDSA certificates I need call same method CertificateKeyPair  or different method
Certificate readCertificate = PemUtil.readCertificate(new FileInputStream("resources/ed25519crt.pem")); PrivateKey privateKey = PemUtil.readPrivateKey(new FileInputStream("resources/ed25519key.pem")); conf.setDefaultExplicitCertificateKeyPair(new CertificateKeyPair(readCertificate, privateKey));

this is the correct method, but it might be that x25519 certificates are not supported by our underlying api.


feat(ansible): Introduce role for Kepler installation

This commit adds the kepler_systemd role which handles the installation for Kepler on both Metal and VM environments. By separating the installation tasks into a dedicated role, the process is simplified and easier to manage for each environment individually
Addresses: #271

Sample CI runs:
Standalone Kepler: https://github.com/sustainable-computing-io/kepler-metal-ci/actions/runs/10992589882
With Model Server: https://github.com/sustainable-computing-io/kepler-metal-ci/actions/runs/11005724516/job/30558949044


⚠ Console Web has degraded performance

In fd86618, Console Web (https://console.inu-cafeteria.app) experienced degraded performance:

HTTP code: 200
Response time: 10792 ms

Resolved: Console Web performance has improved in f9de9a1.


Add support for aspnetcore#39837 aka : Bind get/set/after modifiers

Feature request type
Enhance component
Component name
MudTextField but probably other as well
Is your feature request related to a problem?
Mudblazor doesn't support this new way for binding yet.
Describe the solution you'd like
Add implementation for :

@bind:set
@bind:get
@bind:after

Have you seen this feature anywhere else?
https://github.com/dotnet/aspnetcore/issues/39837
Describe alternatives you've considered
Keep using the old binding
Pull Request

[ ] I would like to do a Pull Request

Code of Conduct

[X] I agree to follow this project's Code of Conduct

When 6.1.0 is released, you can use all net7 features. Please be sure your project also targets net7.
The latest 6.0.19-dev.3 preview also supports both net 6 and net7.

Just so you know in aspnetcore 7.0.0 bind:after is broken I presume it will be patched quite quickly.

Oh, I see.
Thanks for the information.


Use timed wait around get() calls in PinotInstanceRestletResourceTest

cc @jackjlli , @mcvsubbu

Codecov Report

Merging #4649 into master will increase coverage by 0.21%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #4649      +/-   ##
============================================
+ Coverage     64.36%   64.57%   +0.21%
  Complexity       32       32
============================================
  Files          1076     1076
  Lines         56085    56085
  Branches       8189     8189
============================================
+ Hits          36097    36219     +122
+ Misses        17343    17205     -138
- Partials       2645     2661      +16

Impacted Files
Coverage Δ
Complexity Δ

...elix/core/relocation/RealtimeSegmentRelocator.java
25% <0%> (-15%)
0% <0%> (ø)

...impl/dictionary/DoubleOnHeapMutableDictionary.java
63.04% <0%> (-13.05%)
0% <0%> (ø)

...ache/pinot/common/utils/TarGzCompressionUtils.java
68.23% <0%> (-12.95%)
0% <0%> (ø)

.../apache/pinot/common/config/RealtimeTagConfig.java
93.33% <0%> (-6.67%)
0% <0%> (ø)

...mpl/dictionary/DoubleOffHeapMutableDictionary.java
73.21% <0%> (-3.58%)
0% <0%> (ø)

.../broker/routing/HelixExternalViewBasedRouting.java
87.85% <0%> (-2.81%)
0% <0%> (ø)

...broker/broker/helix/LiveInstanceChangeHandler.java
57.77% <0%> (-2.23%)
0% <0%> (ø)

...not/broker/broker/helix/ClusterChangeMediator.java
76.92% <0%> (-2.2%)
0% <0%> (ø)

...inot/core/operator/docidsets/AndBlockDocIdSet.java
55.55% <0%> (-2.09%)
0% <0%> (ø)

...g/apache/pinot/common/utils/helix/HelixHelper.java
53.89% <0%> (-1.2%)
0% <0%> (ø)

... and 25 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ca01c86...bc916c2. Read the comment docs.


Gradle 4.5.x complete distro docs have broken links

For example, "User Manual/DSL" links are broken, links to plugins (Java plugin etc) are broken too. Gradle 4.4.1 has working links.

Thanks for reporting, I've tried several links and didn't met a broken one. Could you provide more details, e.g. which url on which documentation is broken? That would be great.

Paging @eriwen ☝, I think this is related to the new docs layout work.

Were one to serve the docs/ directory, links would resolve; otherwise links especially between different types of docs like the DSL reference and userguide will not resolve.
The only alternative I can think of right now is links to the live docs, which we do for guides (since those are not distributed yet). Do you all have other ideas?

It would be great to retain pre-4.5 option where docs are usable for offline use (from filesystem, not via webserver). Kinda old school, but I'd like it.

@GrafDiffusore Thank you very much!


New styles in Audio control component

Description
As part of design changes introduced in #2113, the Audio control component in Design Library has new styles.

Small size is removed due to its lack of use.
Tiny is renamed as Small.
Extra large is renamed as Large.
New sizes for each variant

Small: 32px, w-8 h-8
Medium: 48px, w-12 h-12
Large: 64px, w-16 h-16

New icon sizes for each variant

Small: 24px, w-6 h-6
Medium: 32px, w-8 h-8
Large: 40px, w-10 h-10

Hover and Focus styles follow the Button style
Default style has 2px for border radius, although this changes depending on the component context.

Context
The new styles are based on #415 implementation
Figma
Component in Design Library.

This issue is blocked because the design for single result page for Audio also needs to be updated to match the new Audio control component sizes.

Sorry, @fcoveram! I was wrong. Your list of what needs to be updated is right:

What I understood from the chat we had https://github.com/obulat, is that this request creates unintended changes in the following components:
Row (large, medium, and small)
Box (large, and medium)
Global


UIControl subclass

I'm curious why you didn't make it a UIControl subclass?

To be honest I think I was just lazy 🙈


Introduced ArrayReplergeRecursive for better config merging

Prior config merging had a flaw when merging lists (arrays which contain integer keys). The old version used the same behaviour as array_replace_recursive, so that values with the same key of array B overwrite values of array A (except if both values are arrays, then they are merged recuresively).
The new behaviour works with a mix of replace and merge (hence the name):

If the key to merge is an integer, the value from array B is appended to array A
If both keys are arrays, they are replerged
Else the value of array B overwrites value of array A

The replurge function is now used in both Zend\Config\Config::merge() as well as in Zend\Config\Factory::fromFiles(). A short benchmark shows that the performance of it is rather good (only about 5 times slower than array_merge_recursive or array_replace_recursive).
See the change for testing the new behavior in Zend\Config\Config::merge() here
As I don't want to steal credits, thanks @EvanDotPro for the name :)

A better name is required!

I'd suggest to call it:

merge

This operation is a merger of two structures. Duplicate keys are overwritten, structures are walked recursively. That's what array_merge_recursive was meant to do in the first place, but we decided it's lame because it creates new structures from duplicate string values.
Despite that, the operation is still a MERGE operation.

I can't speak for @DASPRiD, but let me just go on record saying the name suggestion was simply meant as a joke / placeholder pending a better suggestion. I simply saw no point in wasting time debating/brainstorming a name at the time of writing the function.
I do happen to think that merge and replace in this case are both misnomers and that we can do better than that. Do I have a particular suggestion at this point? Not really, but we should probably go with something descriptive like ArrayReplaceOrMerge rather than confuse the behavior with PHP's built in merge or replace functionality.

Updated benchmarks: https://github.com/zendframework/zf2/pull/880#issuecomment-4325511
Keep in mind these benchmarks include the $config->toArray() calls and are not meant to directly benchmark the performance of this method, but how it performs in the context of array merging.

@DASPRiD @EvanDotPro This function still recognises arrays with int keys as list, which is an incorrect assumption.
Here is an example, where "replege" will not work correctly:
<?php
// this is _not_ a numerical array, but an associative array with integer keys
$production = array(

//  priority => callback
    100      => array('\Application\Listener','foo'),
    0        => array('\Application\Listener','bar'),
    -10      => array($this, 'errorHandler')

);

// this is an array that is being merged
$development = array(
    // overwrite the last listener
    -10 => function($evt){ throw $e->getTarget(); }
);

// the following will produce an unexpected result,
// because keys are being removed during the merge (!?)
$listeners = ArrayReplergeRecursive::replerge($production, $development);

// attach listeners
foreach($listeners as $l) {  $this->events()->attach('dispatch', $l); }
// ...

@Thinkscape: This is completly correct, as we mimic the behaviour of array_merge_recursive here (except for string keys).

The goals look fine, and the benchmarks are acceptable. The name is not. :)
I agree with @Thinkscape -- this is a merge operation, regardless of where it differs from the internal PHP functionality. As such, I'd argue ArrayMergeRecursive, and make sure you document in the class docblock and/or method docblock how it differs from PHP's solution.

It's not bad, nor wrong, nor exotic. Keys are keys. Integers can be used for priorities, reference maps (i.e. "ids"), positional information etc. Historically, priorities have been used since early zf1 in various components.
I've updated my example (last 3 lines) so you might understand the use case better.

I'd argue that priority should not be in the keys, but instead a property/parameter, and it's up to the class receiving the values to then apply priority correctly. Such an approach also makes it much more explicit as to what the integer values represent -- used as keys, they could be seen as potentially typos. Basically, for configuration purposes, if the key is an important part of the configuration, it should not be a value, not a key; keys in configuration are to allow access to the value.

Ok. How about creating a common class for all array related functions?
Proposed names:

Stdlib\ArrayLib
Stdlib\Arrays
Stdlib\ArrayTools

A skeleton:
<?php
namespace Zend\Stdlib;

class ArrayTools
{
    public static function isList(array $array){ // ... }
    public static function isHash(array $array){ // ... }
    public static function hasNonNumericKeys(array $array){ // ... }
    public static function hasNumericKeys(array $array){ // ... }

    public static function iteratorToArray($iterator, $recursive = true){ // ... }
    public static function merge($arrayA, $arrayB){ // ... }
}

I think that idea has merit, @Thinkscape -- but as a separate PR all its own. :)

@weierophinney not possible, as it a dependency for Config::merge() and ::factory
@DASPRiD can I integrate your branch into mine and combine it altogether ?

it's up to the class receiving the values to then apply priority correctly.

@weierophinney @EvanDotPro Ref: my example above.
When working with lists that hold configuration arrays for future instances, how would one configuration file remove (or alter) elements from another configuration file. Examples include event manager config in production/development files, application config overriding default module's config, module B overriding module A. If we discard numeric keys and merge arrays, then there is no simple way to archive that, or I fail to see one.

@Thinkscape You can merge my branch over, yes.

@Thinkscape Using the logic in this PR, we would have no way to override/change the values in a config parameter that contains a "list" or really, any integer keys. With array_replace_recursive, we could do this, but at the cost of not cleanly being able to append values to the "list".
With either method, there's no way to remove or unset values strictly from the context of the config files.

I've merged all changes from this PR into PR 880.
Please close it.