Focus/Activate existing tab instead of creating new one from other program.

What feature or improvement do you think would benefit Files?
If a specific directory is invoked by another program (i.e Free Download Manager, Everything, AutoHotkey) it'll be opened in a new tab instead of switching to the already existing tab if present.
Please provide a setting to change this to prefer the existing tab instead.
Requirements
-Check the directory to open against directories already open.
-If identical to an already opened directory, switch to that tab instead.
Files Version
<IP_ADDRESS>
Windows Version
10.0.19045.2364
Comments
No response

Thanks for the feedback, I've added this to the project board.

Thanks for the feedback, we're tracking this in #9709.


Update json4s-jackson, json4s-scalap to 3.6.7

Updates

org.json4s:json4s-jackson
org.json4s:json4s-scalap

from 3.6.6 to 3.6.7.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention @scala-steward in the comments below.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [{ groupId = "org.json4s" }]

labels: semver-patch

https://github.com/json4s/json4s/compare/v3.6.6...v3.6.7


Display the URL of components while starting up docker

When using dock-cli up, we should display the DNS addresses of started components.
The idea is that, after up, a table such as this one would be displayed:
+-----------+---------------------+------+
| Component | DNS address         | Port |
+-----------+---------------------+------+
| web       | http://[...].docker | 80   |
| mysql     | tcp://[...].docker  | 3306 |
+-----------+---------------------+------+

The point is also that we should display logs, so many options:

Display the table then follow the logs of containers (which may hide the table quickly, as for instance MySQL logs at startup is ~50 lines)
Display the table and finish the command, we need to type dock-cli logs (to create) to follow logs.
Ask y/n to go to case 1 or 2.

I think it would be great to have a separate command to display both the table and the logs.
I see the dock-cli up as an environment bootstrap command only. This could include the table displayed right after the environment finishes bootstrapping (chained commands) but not the logs (which is something not related to the bootstrap action itself, but something "external" the bootstrapping mechanism).
Does that make sense?

Separate commands sounds good to me. The logs hiding the table when that may have been what you were after sounds like a potential ux issue.

:+1:


[WIP] Windows docker image

Implemented pretty much all of the same tests that are there for the Linux container image using Pester. Currently, the plugin-installation-manager-cli is being used to install plugins, there are some issues that need to be corrected before this works will all the same inputs as the install-plugins.sh script. This is a WIP since there are some outstanding items, but I wanted to get it out there for feedback.

The jdk 11 image is not currently feasible because awt will not load on windowsservercore

Doesn't really seem to be a lot of interest in this, so I will close it out.


Use Bolt as datastore

https://github.com/boltdb/bolt

Bolt is really only key/value and does not fit the bill


added top spacing and reduced bottom

fixes https://github.com/heptio/velero/issues/1572 by adding margin-top & improves readability/spacing a bit by reducing margin-bottom

Looks great!


Setup production for https

Redirect port 80 to 443. Use HTTPS for entire site... I'll need to purchase a certificate.

https://www.phusionpassenger.com/library/walkthroughs/deploy/ruby/ownserver/standalone/oss/deploy_app_main.html


Add Getting Started section to Readme.

Readme Update
Add 'Getting Started' section to the Readme so that folks know how to start with the project.

Added the Getting Started Section to the Readme.


Add option to Justify

It isn't that great that we can't justify a Paragraph of Text directly in Manim using MarkupText. Also, there is a Paragraph Mobject but it does things manually which Pango does for us. Also, we should make Paragraph object use these rather than implementing it by hand, as it is done currently.
these are the link to docs which we should implement
https://developer.gnome.org/pango/stable/pango-Layout-Objects.html#pango-layout-set-justify
https://developer.gnome.org/pango/stable/pango-Layout-Objects.html#pango-layout-set-indent
https://developer.gnome.org/pango/stable/pango-Layout-Objects.html#pango-layout-set-spacing
https://developer.gnome.org/pango/stable/pango-Layout-Objects.html#pango-layout-set-line-spacing
https://developer.gnome.org/pango/stable/pango-Layout-Objects.html#pango-layout-set-auto-dir
https://developer.gnome.org/pango/stable/pango-Layout-Objects.html#pango-layout-set-alignment
This should be done after https://github.com/ManimCommunity/ManimPango/issues/30
cc @PhilippImhof

I have done some in https://github.com/ManimCommunity/ManimPango/commit/942f2f68fe0cef448287b3359de207dd444ca1ea
where I could create things like

or like this

or with an indent in the start

or justified


Fix Logic for Removing Partition Columns from RecordBatch

Which issue does this PR close?
closes #9290
Rationale for this change
The current logic for removing partition_by columns assumes that the columns are at the end of the record batch. This is not always true when using COPY with the partition_by option.
What changes are included in this PR?
Removes columns from record batch based on name rather than position.
Are these changes tested?
Yes, modified copy.slt to cover
Are there any user-facing changes?
Yes, partitioned COPY statements will work correctly even if specifying columns that do not come at the end of the schema

Thanks @devinjdangelo -- I don't fully understand what problem this PR is fixing. Is this fixing wrong results? Or a panic? Or something else 🤔
Could you possibly add a new test (rather than modify the existing one) so it is clearer?

Apologies for not properly documenting the issue up front. I created #9290 to document the incorrect behavior that this PR is fixing. Essentially, ListingTables enforced a particular ordering on the columns which Copy alone does not. The logic for eliminating the partition columns from the RecordBatches to be written to disk needed to updated now that it cannot rely on the fixed column ordering.
I also restored the original test and added a new one instead of modifying.

Thanks @devinjdangelo -- I don't fully understand what problem this PR is fixing. Is this fixing wrong results? Or a panic? Or something else 🤔
Could you possibly add a new test (rather than modify the existing one) so it is clearer?

Apologies for not properly documenting the issue up front. I created #9290 to document the incorrect behavior that this PR is fixing. Essentially, ListingTables enforced a particular ordering on the columns which Copy alone does not. The logic for eliminating the partition columns from the RecordBatches to be written to disk needed to updated now that it cannot rely on the fixed column ordering.
I also restored the original test and added a new one instead of modifying.

Thank you so much -- I am sorry for not being able to see it immediately. I struggle these days to keep up with everthing going on and don't seem to have the ability to research things as much as I would like to


Add docstrings for set_adapter and keep frozen

Hello all,
In this PR I have added information to the docstring of set_adapter functions throughout the library. Specifically, I make the behavior where it sets the adapters to trainable explicit. I also provide example code to ensure that the adapters are frozen. Please see #1374.

I have updated the snippets, I hope that it is the correct format now?

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@EricLBuehler Could you please run make style on your changes?

I just ran make style and removed a merge conflict.

Thanks for updating, but there seems to have been some issues. There is still a merge conflict here:
https://github.com/huggingface/peft/pull/1447/files#diff-ea5314dbd5288bc9963787ca71c23a3e386a62fa9e1b3a36d0d68a7c96c9c0d4
Also, it seems that other parts of the code have been linted that are not affected by your change, which should not happen. I assume that you have pulled the latest state of main and are using a recent ruff version (0.2.1)?

Ok, the merge conflict at your link is gone now and I ran make style again. Did that fix it?

On my end, it looks like my fork is now only 6 commits ahead and my main is up-to-date. Is this correct?

Thank you! Glad to help.


Fix two small typos in gt4py branch

Changes qsfg variable back to qs as in original fortran code.
Changes filename in @ported_function from cloudsc2.f90 back to cloudsc.f90.

I don't think I have the rights to merge. 🤷🏽♂
It seems like you need to merge?

You're right, I had to add you as a collaborator. Now you should be able to merge yourself.

Thanks for adding me as a collaborator!
Before merging I ran the checks, and there are two issues:

The --with-loki runs fail during configuration with:

CMake Error at ecbuild/cmake/ecbuild_log.cmake:190 (message):
  CRITICAL - loki_transform_transpile() was removed!

  Please use

      loki_transform( COMMAND convert [...] )

  or

      loki_transform_target( COMMAND convert [...] ).

Call Stack (most recent call first):
  loki/cmake/loki_transform.cmake:455 (ecbuild_critical)
  cloudsc-dwarf/src/cloudsc_loki/CMakeLists.txt:662 (loki_transform_transpile)

The --single-precision runs fail during the build stage with

[Loki::Scheduler] Applied transformation <DrHookTransformation> in 0.00s
[Loki::Scheduler] Applied transformation <SanitiseTransformation> in 0.00s
Traceback (most recent call last):
  File "../../../loki/loki_env/bin/loki-transform.py", line 11, in <module>
    sys.exit(cli())
  File "/home/runner/work/dwarf-p-cloudsc/dwarf-p-cloudsc/build/loki/loki_env/lib/python3.8/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/home/runner/work/dwarf-p-cloudsc/dwarf-p-cloudsc/build/loki/loki_env/lib/python3.8/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/home/runner/work/dwarf-p-cloudsc/dwarf-p-cloudsc/build/loki/loki_env/lib/python3.8/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/runner/work/dwarf-p-cloudsc/dwarf-p-cloudsc/build/loki/loki_env/lib/python3.8/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/runner/work/dwarf-p-cloudsc/dwarf-p-cloudsc/build/loki/loki_env/lib/python3.8/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/home/runner/work/dwarf-p-cloudsc/dwarf-p-cloudsc/build/loki/loki_env/lib/python3.8/site-packages/scripts/loki_transform.py", line 206, in convert
    eliminate_dead_code=eliminate_dead_code, allowed_aliases=horizontal.index
AttributeError: 'NoneType' object has no attribute 'index'

Should we merge anyway, and solve these later? Or should I try to solve them now, and only merge when all checks are green?


CA Game Day Exercises

Backlog Task
To ensure we're ready to support unexpected issues in production, as a team we need to run through some sample scenarios.
Option to use a collaborative coding paradigm to ensure the whole team becomes familiar with the troubleshooting process.
Potential Exercises:

LIMS / EHR offline
TI or RS offline
Inter-service auth failure
Invalid message contents
Corrupt zip file via SFTP
Changed SFTP encryption key

Pre-conditions

Access to PagerDuty (or similar tool for notification) (PagerDuty Setup #177)
Access to RS metabase
Access to appropriate RS Slack channels
Access to RS Azure logs & environment (storage blob containers)

Completion Criteria

[ ] We've completed at least two exercises:

[ ] One demonstrating an unexpected failure within the TI service.
[ ] One demonstrating an unexpected failure within ReportStream or the receiving service.

Tasks

[x] Determine the scenarios
[x] Determine when this exercise will take place

[x] Get support from the RS team

[x] Document an intro to the session
[x] Submit sample data for scenarios
[x] Staging configuration change to support one of the scenarios
[ ] Conduct a blameless post-mortem for each exercise
[ ] Conduct a retro for the whole exercise

Other Notes

Any other notes to help clarify this task for the team

Our Game Day is slated for tomorrow. Two scenarios have been prepped. Test messages and staging config updates are planned for tomorrow.

Pushing this back as we've had to defer due to staging access issues.

The Game Day is now scheduled for this Friday at Noon ET.

Game Day Intro: The goal is to be better prepared when an incident takes place in production. Today, we’ll be treating a number of issues in staging as if they were real issues taking place in production. We should leverage our documented incident response process, as well as the troubleshooting document that shows how to detect ETOR errors in TI and ReportStream. Both are in Notion. Any questions?

We completed one of the exercises last Friday, with the next scheduled for this coming Friday.

Part 2 of the Game Day completed today. We're planning to host a retro for part 2 on Monday.

Retro started, but we'd like to continue this tomorrow (Tuesday)


Allow adding of "cover" as a door sensor (feature request)

I have a garage door opener, implemented in HA as "cover" and would like to add it to Alarmo as a Door sensor.
Of course it's state is not purely binary, as it can be "opening", "closing", etc as well as "open" and "closed" but my thinking is that Alarmo could use "closed" and anything else as "open"
I know I could derive a binary_sensor in HA using a template, but it certainly would be neater if Alarmo natively supported Garage Doors.
Thanks!

A cover  is an actuator and not a sensor.
You will have to create a template binary_sensor for mapping the state to a sensor.
I understand that it would be preferrable to have covers directly integrated in Alarmo, but you already gave a good reason for not wanting it.
Some users asked the same question regarding adding the support for sensor entities, it would create a lot of difficulties since the mapping to safe/unsafe conditions becomes non-trivial.
I prefer to limit Alarmo to binary sensors only to avoid such complexities. Hope you can live with this 👍

A cover  is an actuator and not a sensor.
You will have to create a template binary_sensor for mapping the state to a sensor.
I understand that it would be preferrable to have covers directly integrated in Alarmo, but you already gave a good reason for not wanting it.
Some users asked the same question regarding adding the support for sensor entities, it would create a lot of difficulties since the mapping to safe/unsafe conditions becomes non-trivial.
I prefer to limit Alarmo to binary sensors only to avoid such complexities. Hope you can live with this 👍

Fair enough! Easy enough to create the binary_sensor

Fair enough! Easy enough to create the binary_sensor


input position of LED resets to zero if you go back to LED layout page

[x] I confirm that this is an issue rather than a question.

Bug report
Steps to reproduce
What is expected?
What is actually happening?
System

So, after you set up led count etc in LED output and layout and save, it saves fine, but if you return to layout and look at settings in classic layout again, the input position resets to zero (I save on 345, but always resets to 0 if you open page again) all other setting remain as saved.

libreelec 9.2 hyperion unofficial install pi3

Can you do me favor and follow our issue rules and use our issue template...otherwise we will ignore it...and please stop multiposting...use the edit button..the whole team get a email for every post...thanks!


Header Being Removed

I am having a hard time figuring out why my header is being removed on screen resizing to small.
I am not using any of the responsive classes. Just the straight layout classes.
Here is the code I am using:

      <jdoc:include type="modules" name="tbar-right" />

<button id="demo-appmenu-lower-right"
        class="mdl-button mdl-js-button mdl-button--icon">
  apps
</button>

  <jdoc:include type="modules" name="f-nav" style="html"/>

More soon

<button id="demo-menu-lower-right"
        class="mdl-button mdl-js-button mdl-button--icon">
  more_vert
</button>

<a class="mdl-menu__item" rel="nofollow" href="<?=$script_url?>/login"><?=$word[51]?>

Yet Another Action

@findborg Would you mind creating a codepen that reproduces the issue? Also, what’s your OS version and browser version?

Ubuntu 14.04 LTS
Chrome Version 43.0.2357.134
here is an example with the issue. http://findborg.com/n
Also, forgot to mention that I am pulling in one of the theme style sheets as well.
https://storage.googleapis.com/code.getmdl.io/1.0.0/material.grey-indigo.min.css

Check the configuration options in the documentation. It provides the class needed to show the header on smaller screens.

Also for future reference, support questions should be asked on StackOverflow. This tracker is for issues with the code provided by the project itself.


add observability to zhipu chat model

the PR content:

add observability to ZhiPu chat model
add related unit tests

related issue:
https://github.com/spring-projects/spring-ai/issues/1432

thanks!  merged in 7bd8a17d330a746c161fd0ae57155e33ee38d787


Run the GP2 Notebooks to start getting fimiliar with the API

For this, you should clone your GitHub Forked Repos onto Chimera, and then from there; you will need to have followed the QUICKSTART steps to setup a tunnel to a Jupyter notebook which will allow you to run the notebooks from the cluster where all the data is stored.

GP2 original notebooks have been ran by team


Switch product placement in Products.tsx

In Products.tsx, the Big adopt a Hive product has to be in Small adopt a Hive's place, so their places need to be switched around. Because right now the bigger hive is first, and it would be better if the smaller would come first in the list and after that the bigger hive.

Can I work on this?

Can I work on this?

Sure, go ahead! Feel free to ask any questions here if you have some problems.


Manually merge pull request #1486, to make travis ci and codecov work…

… after apache incubator transition.
What is the purpose of the change
XXXXX
Brief changelog
XX
Verifying this change
XXXX
Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Make sure there is a GITHUB_issue filed for the change (usually before you start working on it). Trivial changes like typos do not require a GITHUB issue. Your pull request should address just this issue, without pulling in other changes - one PR resolves one issue.
[ ] Format the pull request title like [Dubbo-XXX] Fix UnknownException when host config not exist. Each commit in the pull request should have a meaningful subject line and body.
[ ] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
[ ] Write necessary unit-test to verify your logic correction, more mock a little better when cross module dependency exist. If the new feature or significant change is committed, please remember to add integration-test in test module.
[ ] Run mvn clean install -DskipITs to make sure unit-test pass. Run mvn clean test-compile failsafe:integration-test  to make sure integration-test pass.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

Hi, this is already merged, please keep your repo synced with lasted repo.

Codecov Report

Merging #1526 into master will decrease coverage by 1.22%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1526      +/-   ##
==========================================
- Coverage   31.59%   30.36%   -1.23%
==========================================
  Files         682      879     +197
  Lines       33035    45519   +12484
  Branches     6592     9451    +2859
==========================================
+ Hits        10438    13823    +3385
- Misses      20742    29411    +8669
- Partials     1855     2285     +430

Impacted Files
Coverage Δ

...om/alibaba/com/caucho/hessian/io/LocaleHandle.java
18.18% <0%> (-81.82%)
:arrow_down:

...aba/dubbo/remoting/http/jetty/JettyHttpServer.java
57.4% <0%> (-22.08%)
:arrow_down:

...src/main/java/com/alibaba/dubbo/rpc/RpcResult.java
29.26% <0%> (-6.03%)
:arrow_down:

...rpc/protocol/dubbo/telnet/InvokeTelnetHandler.java
51.89% <0%> (-3.8%)
:arrow_down:

...o/rpc/cluster/support/MergeableClusterInvoker.java
35.15% <0%> (-2.71%)
:arrow_down:

...ava/com/alibaba/dubbo/rpc/filter/tps/StatItem.java
77.77% <0%> (-1.39%)
:arrow_down:

...va/com/alibaba/dubbo/rpc/filter/ContextFilter.java
82.14% <0%> (-1.2%)
:arrow_down:

...ibaba/dubbo/remoting/telnet/codec/TelnetCodec.java
19.52% <0%> (-0.85%)
:arrow_down:

...onfig/spring/schema/DubboBeanDefinitionParser.java
20.9% <0%> (-0.17%)
:arrow_down:

...baba/dubbo/cache/support/AbstractCacheFactory.java
100% <0%> (ø)
:arrow_up:

... and 234 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 05da804...4f34540. Read the comment docs.


Empty tags content if not used

I would take benefits of an option that would reset the tags content, before any state change.
For example, I use open-graph meta in few states on my application. I expect them to be empty in other states ; by now to make it work I must add

in any state.

@rolintoucour — Thank you for your suggestion.
Can you provide a concrete example with different pages that contain specific tags and the desired outcome?
Thanks again!


xnet

Include a short description of your patch here
A more detailed description of your contribution can be added here.
Definition of Done

[x] I have read the Contribution guidelines and the Code of Conduct.
[x] I have built and run the extension with my changes using all of the following profiles:

[x] "Run bundled extension"
[x] "Run Web extension"

[x] I have written unit tests for my changes.

Optional: reference a GitHub issue linked to your contribution (e.g. "Closes #<ISSUE_NUMBER>").

Just Learning about my android

Hello and thank you for submitting a PR! Would you mind explaining what the goal of your proposed changes is?
Have a nice day!

Closing as there has not been any feedback.


Add a section about coloring

I made the coloring section more specifically about coloring, and also adjusted a few things around pattern detection. @adrhill I think this is good to go

Why did you removed my figures? :cry:

Well, you just removed all my text and figures...
I don't know why I worked on that if you remove all my contributions.

As for the illustrations, it might be better to save realistic matrices for the last section, where we will demonstrate the Julia pipeline. In the theoretical stuff beforehand, we can probably keep using the same kind of simple images that Adrian has been creating. For instance, we could have one showing various column colorings on a very small matrix, with different numbers of colors in total.

Maybe just open a branch and create a PR instead of hijacking mine?
You just removed all my modifications, Guillaume.
The repeating part was just about the seed, not the coloring and the figures. Am I wrong?
For the bicoloring, I’m not talking about our research and how we can approach it (relation with star and acyclic symmetric coloring); I’m just suggesting a nice comment that both AD methods can be combined.
I thought that this fit well into the workflow of the section and can easily illustrated.
I’m just disappointed because I felt that you removed my contribution, and I don’t feel legitimate putting my name on the blog.
I also spent time generating new figures one evening.

Maybe just open a branch and create a PR instead of hijacking mine? You just removed all my modifications, Guillaume.

My apologies for that, you are right that I should have opened a new branch. The reason I did this is because Adrian had left a comment yesterday asking for changes, and I didn't know if you planned to take it into account or not, so I thought I would make the changes myself.
We only have one week to get this blog post ready to the point of submission, so yes I was a bit assertive to ensure that we don't get stuck. I should have asked you first when you would have time to work on it.

The repeating part was just about the seed, not the coloring and the figures. Am I wrong?

I think you are wrong. Everything you wrote apart from the mention of bicoloring was strictly included in what was already written above.

For the bicoloring, I’m not talking about our research and how we can approach it (relation with star and acyclic symmetric coloring); I’m just suggesting a nice comment that both AD methods can be combined.

For me, not mentioning bicoloring meant no visualizations, perhaps a pointer to an article but that's about it.

I also spent time generating new figures one evening.

I thought I recognized the figures you had already made for our bicoloring paper, which is why I removed them, sorry. Ideally there should be no overlap with our own research, because these blog posts are forbidden to contain self-promotion. Besides, we don't have to reuse the exact same matrix from the SIAM Review, we can go with something simpler. The goal of this section is to give intuition about what the coloring means, whereas we can put realistic matrices in the final section.

I’m just disappointed because I felt that you removed my contribution, and I don’t feel legitimate putting my name on the blog.

I understand your disappointment. I want you to contribute to this blog post because you're an essential part of the ecosystem the three of us built together. But the paragraphs you wrote about coloring were redundant with the rest, so I took the liberty to remove them.

I have reverted my hijacking and opened #14 to propose alternative changes. Since @adrhill is taking the lead on this blog post, I'll let him decide how to combine the PRs

Yeah, I'll close this in favor of #14, since this is a duplicate of the "Leveraging structure" section.
Let's build from there!


Media queries according to div element

Feature
I have an one row in which three columns. When i change the with width of row manually to 360px then    each column width moved to 100% means (co-120) if i set width of row 1024px then i want  three column in an row means width 33.333% each. media queries work on screen size
Current behaviour
The row will show two, three columns based on screen size

There's currently no way to do this in CSS without a polyfill as far as I know.


[Not openssl program] Mailing list "openssl-users" not working…

I have problem with two ASN.1 sequences, and I cannot test certificate.
I need to post information on mailing list "openssl-users". Message sended, but I doesn't see it in archives. I using a temporary email (I doesn't have a real for now).

Please see https://mta.openssl.org and join the openssl-users mailing list.

Not working but I am subscribed. Temporary email hostings are blocked?

Please send a message to<EMAIL_ADDRESS>with the details, such as your email address and a few lines about the issue.


Add timeout to connect attempts to fix #19

Without connect timeouts, I get intermittent hangs on some tests I run. With this change, I ran a whole bunch of repeats and got no failures.
I don't have an exact reproducer for causing the connect call to hang. In my case, it happens when a Kubernetes Service is in the process of starting. According to some googling, hangs can happen for a variety of reasons, such as if the network simply drops packets for a destination that doesn't exist yet.

Also, some of the builds fail because recoveringWith now has a different signature, so that will also need to be addressed to get the CI to pass.

All right, done. I'm pretty sure nobody uses GHC 8.2 anymore though :P

Friendly ping on review @mbg

Hi @thomasjm,
Sorry for the delay in getting this reviewed. It's on my radar, but I just haven't been able to find the time yet.


Update deploy-cli.md

We're deprecating the GIS 'e2e' example— possibly temporarily— so how about swapping in the Pipelines variant instead.

This change is

Thanks @amygdala!
/lgtm
/approve


Code Commenting Huge Sections of Code out, which wastes tokens...This would be a 10 second fix

Describe the bug
At the moment, about 30%-40% of the time, bolt.new outputs commented code, breaking the app. This result in wasted tokens and time, and annoyance.
I imagine it would be the quickest/easiest fix in the world.
If bolt.new simply appended "Always output full uncommented, unnabreviated code." to the end of every user message, before passing to the model, it would basically solve that problem...
In practice when I do that manually it usually stops bolt.new making this mistake, but it is laborious and annoying to have to append that to the end of every single message.
Link to the Bolt URL that caused the error
www.bolt.new
Steps to reproduce
Just use bolt.new for more than half an hour.
Expected behavior
Not to comment out huge sections of my code base every time bolt.new works on it, essentially breaking the application over and over.
Screen Recording / Screenshot
No response
Platform

OS: [e.g. macOS, Windows, Linux]
Browser: [e.g. Chrome, Safari, Firefox]
Version: [e.g. 91.1]

Additional context
No response

Hey @bensdb01 this is driving me crazy too. no matter how many times I tell it to give me a full file.
Following to see any responses.

Have a look at .bolt/prompt, would adding your "Always output full uncommented, unnabreviated code." to that work?
you can read/edit it using cat in a terminal, but those edits will not persist on reloading.
I'd be interested to see if that works for you.

@bensdb01, @neillh, @jim80 we have found this issue to be tougher than expected to resolve but are working on it. You can track progress and see the tips and tricks for working around it in issue #124

here with the same issue


Allow body in custom HTTP methods

This should fix the issues expressed here: https://github.com/square/okhttp/issues/229

I don’t think I understand the utility of the boolean parameter. Shouldn’t we just relax our rules and permit the custom method?

Ideally, yes - that'd be more in line with the spec as I understand it, and that's how I've done it in a custom internal fork.
My thinking was that the boolean parameter would keep the current behaviour and error handling exactly the same, just in case anyone relied on this current behaviour (unlikely but not possible).
I'd be happy to update the PR to remove the permitsRequestBody method though.

Yeah, let's do that.

I updated the PR, I believe this is now better conforming to the HTTP spec.

Any news on this?

I’m getting ready to cut OkHttp 3.6. Wanna revert your unnecessary changes?

Please integrate this fix because I really need to use PROPFIND with OkHttp.

Any plan for this pull request? I need it.


Transition to "in_review" after levy calculation

We're now calculating the levy before we show the user the screen to
review the submission. Therefor we need to transition it to "in_review",
rather than completed, otherwise the user never gets to confirm their
submission is correct.

 API changes to transition to in_review once levy calculation completes

actually, this looks to be more complicated than this because the submissions update action is called by both the levy calculator when calculation is performed and also needs to be used by the frontend app to "complete" a submission. Closing and will re-open with a more complete approach

re-opening. we may not need to re-use this action to transition the state of the submission after all. merging this should allow us to upload submissions and get as far as the review screen.


feat: add precommit

Change(s):

add pre-commit client config and github workflow+azure-pipelines to run server side

Reason for Change(s):

add more early controls in "python standard" way

Version Updated:

No

Testing Completed:

Yes (github workflow only and private azure-pipelines)

Checked that the validations are passing and have addressed any issues that are present:

No change to code. Add validation check only.

This PR adds precommit config
Example run as is without fixing anything
https://github.com/juju4/Azure-Sentinel/actions/runs/3867376910/jobs/6592111638#step:4:52
Following fixed initial config, not sure if you prefer to be in this PR or a different one
https://github.com/juju4/Azure-Sentinel/actions/runs/3867831878
https://github.com/Azure/Azure-Sentinel/compare/master...juju4:Azure-Sentinel:devel-precommit2
Follow-up hooks here or new PR:

codespell - https://github.com/codespell-project/codespell
https://github.com/juju4/Azure-Sentinel/commit/b079d3657003453bfcb249c3f9ee9e1773fa89e9
a bit of remediation (there is a lot here...)
https://github.com/juju4/Azure-Sentinel/actions/runs/3867789949/jobs/6592801053
detect-secrets - https://github.com/Yelp/detect-secrets
PSScriptAnalyzer - https://github.com/PowerShell/PSScriptAnalyzer

@juju4 Could you please resolve the conflicts?
Also could you please fix the build errors.
Can you please provide more details on the pre-commit? Is it for validating the spelling mistakes in any of the file.

@juju4 Could you please take a look at previous comment?

I merged latest upstream to address conflict.
As for precommit pipelines, please confirm that I should merge remediation branches too.
On codespell remediations, only doing part of them as too many. Each content owner should address their own.
To avoid pipeline failing, I'm putting the non-remediated folders in exclude list
arm-ttk pipeline failing part is new and should not be related to PR as no functional change.

We will take a look. Thanks!

I applied remediation changes to branch to fix precommitValidation pipeline (but impacting tons of files as json prettify/standardize, end of files, UTF-8 BOM, duplicate key and so on)
Not sure about the failing check if related or not as can't see details for ADO tests.
codespell and the rest in future PR

Any input here?
Should I revert remediation and put test in continue-on-error like https://github.com/Azure/Azure-Sentinel/pull/7323 ?

News ?

@juju4 Why are we required to remove spaces in all the files in the repo? Is this really required?
@v-atulyadav Can you communicate the requirement of this change offline? I don't see any code changes in this PR. And in fact it is very difficult to even judge the required changes in vast set of 5500+ files. Please work with the author to remove the files which are not required for this PR before any evaluation is initiated.

I reverted the content changes, merged current master and added continue-to-error to not failing pipelines, but the goal of pre-commit is early validation and as such, it

checks validity of file, partly similar to existing controls
prettify/standardize formatting of some files like json. see

    - id: pretty-format-json
      args:
          - "--autofix"
          - "--indent=4"
          - "--no-sort-keys"

Indent with 4 spaces is what I observed the most, typically from Azure portal export template.
But else, current repository content has various format/indentation.
There are also content with utf8 bom, duplicate key and other issues. This may be without impact when used in Sentinel initially, now but may not always be the case.
With the continue-on-error, change should be painless but content owners need to review so at some point, it is not just pipeline warnings.
Applied successfully in my production environment but subset of this repo.

Hi @devikamehra, @juju4 has responded on your comments please check for the same. Thanks

@juju4 Thank you so much for clarifying.
@v-atulyadav Can you work with Prateek on this? Should we be making this change, what will be the impact, etc.

Hi @devikamehra, sure I will discuss this with Prateek. Thanks.

Hi @juju4, could you please give us a brief description of this PR. Thanks

Hi @devikamehra, mahesh will look into this. Thanks

Description of PR is in above comments but let's try to reword

this add a .pre-commit-config file
main goal is to use on developer client aka pre-commit
still also enabled "server" side aka ADO+Github as sometimes client side not done

The pre-commit validation includes default action from https://pre-commit.com/

ensure yaml and json file are valid
remove extra end-of-file
check for case conflict
check for private key
check for requirements.txt (python)
ensure not adding large file by mistakes (if limit too low, should be raised. likely with solutions zip inclusion)
prettify/normalize json file in order to make them easier to compare

More can be added later like codespell, powershell lint, other secrets check and so on.

Hi @mkchiliveri, could you please provide your response on this. Thanks

Hi @mkchiliveri, please provide your response on this. Thanks

Hi @mkchiliveri, could you please provide your response on this. Thanks

Hi @mkchiliveri, please provide your response on this. Thanks

Hi @mkchiliveri, could you please provide your response on this. Thanks

Hi @mkchiliveri, could you please provide your response on this. Thanks

Hi @mkchiliveri, please provide your response on this. Thanks

Hi @daxesht, please provide your feedback on this. Thanks

Hi @daxesht, could you please provide your feedback on this. Thanks

Hi @daxesht, please provide your feedback on this. Thanks

Hi @daxesht, please provide your feedback on this. Thanks

Hi @daxesht, please provide your feedback on this. Thanks

Any update on this or path forward?

Hi @juju4, I apologize for the inconvenience. We will provide you with an update by the 13th. Please update your branch from master in the meantime. Thanks

Hi @juju4, Having analyzed the contents, we are not able to determine the reason for this change. Could you please elaborate on this and provide screenshots of this functionality if possible? Also the changes you are requesting are for both Github workflows and Azure pipelines, can't we handle this in one place instead of both? It would also be great if we could connect over a call to discuss further. Thanks

Hi @juju4, waiting for your response over above comments and also please resolve branch conflicts. Thanks

Hi @juju4, please check comments above and act accordingly. Thanks

Hi @juju4, I am looking forward to your response to the above comments, as well as the resolution of branch conflicts.
Thanks

Hi @juju4, please respond over above comments and also please resolve branch conflicts. Thanks

Hi @juju4, please check comments above and act accordingly. Thanks

Hi @juju4, hope you are doing well. Just wanted to check if you got a chance to look at the feedback shared. Please feel free to reach out to us for any queries and/or support. Thanks

The content of the PR has been explained in https://github.com/Azure/Azure-Sentinel/pull/7056#issuecomment-1483832608
It applies to both Github Action and ADO because ultimately it should be available to whatever backend is used as pipeline with Sentinel Repository feature.
It adds no functionality outside of validating more content but goes further than existing validations. same than #7323

Noted @juju4,
We will check internally and provide you with an update as soon as possible.
Thanks

Hi @juju4,
We will provide you with an update on this PR by the middle of next week, and if we find any difficulty, we will reach out to you.
Thanks.

Hi @juju4 ,
As discussed, can you please check on if we can enforce this rule only on files committed as part of PR so that author can take an action immediately instead of the full review.

Per https://github.com/pre-commit-ci/issues/issues/90, this is not possible. at least, not with the tool.
As said, the normal way would be cleaning existing repo progressively. Part work done in following commits that were reverted to keep this PR small. Can push them back after but for separate PR.
https://github.com/Azure/Azure-Sentinel/pull/7056/commits/84ca86a4dd2dae03e22f21bb267e1f5726b03fe0
https://github.com/Azure/Azure-Sentinel/pull/7056/commits/f24f0417b86d895ef944ebe311be61c48661d4d8
https://github.com/Azure/Azure-Sentinel/pull/7056/commits/84221848608d4bb9c7ef2191eec56c898218c08e

Hi @juju4, as discussed we will take this as internal feature and try to develop this in future. Thanks


AWS provider fails validation for the new  il-central-1 AWS region

Describe the bug
Cluster secret stores using AWS provider fail validation when the region is il-central-1:
admission webhook "validate.clustersecretstore.external-secrets.io" denied the request: region not found: il-central-1

To Reproduce
Steps to reproduce the behavior:
Create a new  AWS SSM Parameter Store  backed  secret store with region: il-central-1 :
---
apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    # ARN of IAM role to use when querying SSM
    eks.amazonaws.com/role-arn: arn:aws:iam::12346789023:role/my-iam-role
  name: my-service-account
  namespace: external-secrets
---
apiVersion: external-secrets.io/v1beta1
kind: ClusterSecretStore
metadata:
  annotations:
  name: my-secret-store
spec:
  provider:
    aws:
      auth:
        jwt:
          serviceAccountRef:
            name: my-service-account
            namespace: external-secrets
      region: il-central-1
      service: ParameterStore

Kubernetes 1.27
ESO 0.9.2
Expected behavior
AWS region id il-central-1 should pass validation
Additional context
This is the (very) new AWS region in Tel Aviv.  It opened for business on 8/1/2023.  I suspect an update to the AWS SDK client would resolve the issue.

Hi @adrianrgould 👋 I briefly looked into this issue, and below is where the validation runs.
https://github.com/external-secrets/external-secrets/blob/5384954f46513a811425ce22c9186b2eb915f32e/pkg/provider/aws/provider.go#L95-L110
It seems https://github.com/external-secrets/external-secrets/pull/2568 updated the AWS SDK to v1.44.314, which includes the il-central-1 region. The change has been released in the v0.9.2 release, so IIUC if you use the latest version, it should work. I checked it locally just in case and confirmed the issue you described above happened in v0.9.1 but could not reproduce it with v0.9.2. Would you make sure that you really use the latest v0.9.2 ESO?

Hi @shuheiktgw, thank you for looking into it.  You're correct, I was using v0.9.0, and not v0.9.2.   I had erroneously committed a kustomize chart cache which caused the version mismatch.  Gitignore updated!   I confirmed that il-central-1 is validating with v0.9.2.  Thanks again!


Update facility departure datetime

Can you also do the same for the locationPeriodLow just in case it is different?


fix #1377 loading overlay juddering on scroll in Safari Mobile

This PR changes default overlay position from fixed to absolute to prevent an issue on Safari Mobile where fixed-positioned overlay is not repainted on screen during scrolling until the scrolling ends.
I've tested the fix on the requesting application and on the following sample, on ipad (Safari 8), galaxy tab with Chrome, and on desktop browsers to check for regressions (Firefox, Chrome, IE7-9)

http://ariatemplates.com/samples/?path=samples.templates.domInteractions.processingIndicator.Sample

I also tested those samples in major desktop browsers (Fx, Ch, IE7-9) for non-regression:

http://ariatemplates.com/samples/?path=samples.widgets.dialog.resizable.TemplateResizableDialog
http://ariatemplates.com/samples/?path=samples.widgets.dialog.movable.TemplateMovableDialog

This didn't land in 1.7.1 as it was found to cause regressions during release testing.


Remove call to "new Function()"

This module constructs an empty function for internal use, which causes content-security-policy issues in chrome extensions (see https://github.com/guyonroche/exceljs/issues/512). Replacing that with a normal function call seems to resolve the issue.
I also updated the dependencies, since they were fairly old.

hey @doug-martin, it would be great if we can get this PR merged.
It is causing unsafe eval errors when using excel.js (dependent on declare.js) package inside a chrome extension. Reference issue: https://github.com/guyonroche/exceljs/issues/512


eslint fix

fixed the linting errors from current version

Can you squash down your commits into one commit ? Will be simpler to follow the history once it's merged. Thanks

sorry i guess I messed up the rebase, I will close this till I fix it


Enhance Export Functionality: Add JSON and CSV Support (Fixes #27)

This pull request introduces a new feature to export results to either JSON or CSV format based on a newly added -o flag.

@TheYahya will this be considered or should I close?

Thank you @kartikeysemwal.
Sorry for the late response.


Add crio runtime support

Add crio runtime support

Thanks, a few small suggestions but otherwise it's good

Anything else I should do here ?

@jsalatiel did you close this PR on purpose?

It was a rookie mistake :D , but I have created another one that has already been merged

Awesome! Thanks and congrats! :)


Fix count_surface_iterator

This increments the pointer value, not the actual count.
References: https://github.com/Hjdskes/cage/issues/176

Doh. Computers are hard.


feat: add proposer boost reorg flag to enable it in block production

Motivation
Wire the existed proposer boost reorg functionality to block production.
Description

Call getProposerHead() in block production
Call predictProposerHead() in prepareForNextSlot()
Introduce proposerBoostReorgEnabled flag. Default to be false for now as this is seen as an experimental feature
Add proposerBoostReorg e2e test thanks to @tuyennhv
Add related metrics

Closes #5125

@ensi321 there is a conflicting file packages/fork-choice/src/forkChoice/interface.ts

:tada: This PR is included in v1.20.0 :tada:


Gitops tools keep reporting CRDs as out-of-sync due to status field

Describe the bug
CRDs contain status information that are subject to change once they are applied to a cluster. Gitops tools like argo-cd detect those changes and mark the entries as out-of-sync
Steps to reproduce the issue:

Install argo-cd e.g. as described here https://argoproj.github.io/argo-cd/getting_started/
Deploy the banzaicloud kafka-operator using helm:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: kafka-operator
  namespace: argocd
spec:
  destination:
    namespace: kafka
    server: https://kubernetes.default.svc
  project: default
  source:
    path: charts/kafka-operator
    repoURL: https://github.com/banzaicloud/kafka-operator.git
    targetRevision: v0.12.0
  syncPolicy:
    automated: {}

Check the argo-cd web-ui and check the kafka-operator application

Expected behavior
CRDs do not contain information that are subject to change, allowing gitops tools like argo-cd to successfully manage them and keep them in-sync
Additional context
argo-cd: v1.5.2
kafka-operator: v0.12.0

@abyth Those fields are set by Kubernetes and  used to store information about the state if CRD itself (e.g. what versions of the CRD  Kubernetes serves in case a resource has multiple versions).

Thanks for your reply. I do understand what the fields are used for - i was just wondering whether or not they have to be part of the definition. Other CRDs we are using feature that field but are not defining it in the CRD resource


Enhancement - Ability to Drill into a DT Dashboard when selecting a "Pinned" SLO on a dashboard

Request -->When you select the SLO (pinned on a dashboard) instead of directing you to the SLO dashboard page (filtered to that specific SLO), you are directed to another dashboard? Use case = We have a main "summary SLO" that is a summary of steps in a customer journey. We have smaller (children) SLOs for each of the individual sub-steps.  When you select the main SLO, we would like to see a dashboard that features a breakdown of the child SLOs.
Knowing that there is a PU for images, which enables you to "click-to-dashboard" (example of an internal dashboard below).
!PU(image):url=https://inspirewellness.sharepoint.com/sites/psm/SiteAssets/SRE%20Logo.jpg?raw=true;out=#dashboard;id=a2426c14-2261-4695-9fa0-7c368339ae7d
....Could we create a PU similar for SLOs?
!PU(SLO):url=https://dt.humana.com/e/d3610855-748f-4365-9bb1-f036451fac27/ui/slo?dashboardId=91a9cf8e-f851-4a95-a423-11e97ac0d810&id=38a8ed34-119f-3bb2-b3cb-55a0d439d4a2?raw=true;out=#dashboard;id=a2426c14-2261-4695-9fa0-7c368339ae7d

Have you considered using !PU(menu)? You could have multiple drilldowns that way. John has been using this a lot in his UserJourney dashboard to jump to other types of analysis:

@LucasHocker
What would the Syntax look like, if applied to an SLO?
Apdex !PU(menu):name=Application;url=#uemapplications/uemappmetrics;uemapplicationId=APPLICATION-008569CDB300AE03 !PU(menu):name=Easy_Travel;url=http://www.easytravel.com

Hmm... it doesn't look like there's a place to slot in markup for SLO tiles. I'll noodle how to make it cleaner, but for now you could put a markdown tile next to the SLO tile.

Ok, I came up with a solution for this one. I added a parameter to the menu powerup for title. You'll use this to match the title of your SLO tile. So create a markdown tile and add the menu powerup syntax there. It should add the menu items to your SLO tile and hide the markdown tile

Pushed enhancement to GitHub, adding to ChromeStore 1.58.
.


lighthouse: 4.3.0 -> 4.4.1

I checked and the contracts haven't changed

@zimbatm Oh, sorry, I can't merge this PR cuz your commit isn't signed. Can you add a sign pls?

I added lighthouse update to this PR https://github.com/nix-community/ethereum.nix/pull/363. Thanks for this PR!


Fix Chocolatey deploy

For context: choco push was failing for indeterminate reasons (see this Travis log, or the raw log here). Unfortunately due to COVID-19 I don't have access to a Windows dev machine to debug this on, so until then (or until someone else can dive in and debug) we can't deploy to Chocolatey.


Fix list command parsing

Some remainders of the code before the project app broke the list command parsing with extra options.

I will add more tests for the other functions in cmd-parsing in separate PR


Minor typo fix.

Filename should be "app/models/occupation.js" in stead of "app/models/occupation".

Duplicate: https://github.com/ember-data/active-model-adapter/pull/88


Relative paths in "evenBetterToml.schema.associations" not working in VS Code

Thank you for creating this extension!
Using VS Code 1.82.2 + Even Better TOML 0.19.2, with the following folder structure:
🗃 P:
 └─ 📂 Projects
     └─ 📁 App  <- Workspace root
         ├─ 📁 .vscode
         │   ├─ 📄 extensions.json
         │   └─ 📄 settings.json
         └─ 📁 src
             ├─ 📁 schema
             │   └─ 📄 app.schema.json
             └─ 📁 template
                 └─ 📄 template.app.toml

.vscode/settings.json contents:
{
    "evenBetterToml.schema.associations": {
        "^.*\\.app\\.toml$": "./src/schema/app.schema.json"
    }
}

Reading the LSP error in VS Code, it appears that the folder used as a base for the relative path is not the workspace, P:\Projects\App, but one level above, P:\Projects. Because of this, it tries to read the schema from P:\Projects\src\schema\app.schema.json which doesn't exist, instead of P:\Projects\App\src\schema\app.schema.json where it's actually located.
 WARN document_open:publish_diagnostics:collect_schema_errors:validate_root:validate:load_schema: failed to fetch schema error=JsValue(Error: ENOENT: no such file or directory, open 'p:\Projects\src\schema\app.schema.json'
Error: ENOENT: no such file or directory, open 'p:\Projects\src\schema\app.schema.json') document_url=file:///p%3A/Projects/App/src/template/template.app.toml schema_url=file:///p%3A/Projects/src/schema/app.schema.json schema_url=file:///p%3A/Projects/src/schema/app.schema.json schema_url=file:///p%3A/Projects/src/schema/app.schema.json
ERROR document_open:publish_diagnostics:collect_schema_errors: schema validation failed error=failed to load schema file:///p%3A/Projects/src/schema/app.schema.json

Caused by:
    JsValue(Error: ENOENT: no such file or directory, open 'p:\Projects\src\schema\app.schema.json'
    Error: ENOENT: no such file or directory, open 'p:\Projects\src\schema\app.schema.json') document_url=file:///p%3A/Projects/App/src/template/template.app.toml

Also experienced the same issue mentioned in https://github.com/tamasfe/taplo/issues/377#issuecomment-1657494623: I have to delete %APPDATA%\Code\User\globalStorage\tamasfe.even-better-toml and reload the VS Code on every schema-related path change. The JSON: Clear Schema Cache command has no effect.

Full project, for reproduction:
App.zip
Screenshot:

FWIW file://<relative path> seems to work for me

Looks like it works without the dot:

The hyperlink in .vscode/settings.json is not working, but I guess I can live with that.
Many thanks for your suggestion!

Looks like it works without the dot that indicates the current directory:

The hyperlink in .vscode/settings.json is not working, but I guess I can live with that. Unless this is considered a bug, the issue can be closed from.
Many thanks for your suggestion!

Sadly the workaround suggested by @rhenescu does not work for me. Maybe it only works on Windows by accident. Getting an Error: ENOENT: no such file or directory on macOS. Same when prefixed with ./. Without file:// it is relative to one folder above the workspace root. This really needs to be fixed!


[BUG] Bad padding on keygen QR pairing

Describe the bug
When initiating a keygen on macOS and having the QR presented, moving the window as narrow but as high as possible gives the buttons a bad formatting
To Reproduce
Steps to reproduce the behavior:

Open keygen QR on macOS
Narrow the window
increase hight of window
See error

Expected behavior
Keep padding of buttons properly
Screenshots

System:

Device: macOS
Version 1.8.

QA done.


Incorrect table name for ALTER TABLE statement

Description
Cnanges to table metadata can't be saved because the generated statement uses only the name of the table, but doesn't include schema name
DBeaver Version
23.3.3
Operating System
Win 11
Database and driver
Altibase <IP_ADDRESS>.8
Altibase JDBC driver <IP_ADDRESS>.0
Steps to reproduce

Open any table and do any changes to metadata (e.g. rename a column)
Click save and persist changes

Expected: Changes are saved
Actual: Query error is shown

Open Defect in TestQuality


Allow cosign save to reuse save directory to help dedupe shared layers.

Summary

When using cosign save with dozens, or in my case, hundreds of images, the storage consumption can really add up with each image requiring their own directory.  There's potentially a case where images can share layers so this change allows for the cosign save function to be able to reuse a directory from a previous save and just build on the index.json along with an additional annotation for each entry to aide with the corresponding cosign load.
From there, I've modified cosign load to have a new optional flag --registry where you can specify a registry to load all of the images to from a single directory.  In cases where you are only dealing with a single image, the load command would function exactly how it does today where you just provide the remote reference via the cli.
Release Note

Updated cosign save to append the index.json if pointed at a --dir used in a previous save.
Added the org.opencontainers.image.ref.name annotation to the index.json for clarity and to help with cosign load in cases where the directory is reused.
Updated cosign load to have an optional flag called --registry that can be used in cases where the index.json from a save contains multiple images.  This handles being able to control where the images get loaded to in the case where providing a single ref via the CLI no longer makes sense.  If you don't provide the --registry flag, the cosign load command will continue to function as it does today.

Documentation

Example usage:
cosign save image1:v1.0 --dir ~/saves
cosign save image2:v1.0 --dir ~/saves
cat ~/saves/index.json
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.oci.image.index.v1+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 1581,
         "digest": "sha256:c5b6da2ffbc9f63e8d250d5985f722830dfb5cc8958e8f64f7e14edc18b584fa",
         "annotations": {
            "kind": "dev.cosignproject.cosign/image",
            "org.opencontainers.image.ref.name": "image1:v1.0"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 558,
         "digest": "sha256:077bc8e8cfe1577f89959c749da1fbe941bde8b35b7ccdfe4f5832fb5ade2ec1",
         "annotations": {
            "kind": "dev.cosignproject.cosign/sigs",
            "org.opencontainers.image.ref.name": "image1:v1.0"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 468,
         "digest": "sha256:7e6fb78b8f35c859c9200567229a4350c17e057d4d13c80121a3f603748a1cb0",
         "annotations": {
            "kind": "dev.cosignproject.cosign/atts",
            "org.opencontainers.image.ref.name": "image1:v1.0"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
         "size": 967,
         "digest": "sha256:7e42b1a631859c6740835659a022ed96097955dde2707f32e7a089f5b5f64c06",
         "annotations": {
            "kind": "dev.cosignproject.cosign/imageIndex",
            "org.opencontainers.image.ref.name": "image2:v1.0"
         }
      },
      {
         "mediaType": "application/vnd.oci.image.manifest.v1+json",
         "size": 558,
         "digest": "sha256:88145843e5dcd7a0d515cf874997b1d4eaa742a1594dc75c6bd111f109cf4455",
         "annotations": {
            "kind": "dev.cosignproject.cosign/sigs",
            "org.opencontainers.image.ref.name": "image2:v1.0"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 469,
         "digest": "sha256:6fa5a413e06af7babaeae39bb06c92b563463080fd46003b078050787c3ef6f0",
         "annotations": {
            "kind": "dev.cosignproject.cosign/atts",
            "org.opencontainers.image.ref.name": "image2:v1.0"
         }
      }
   ]
}

cosign load  --dir ~/saves --registry your.registry.com

Hello @priyawadhwa.  Thanks for the feedback!

Instead of a --registry flag on load, maybe it would make sense to take a regex on the image name? That way you could say something like "load all the images" or "load all the images from a specific registry"

I'm looking at this from a scenario where I'm ...

saving images from "Registry A"
tar/compressing the directory used as the output
walking it over to an air-gapped environment
loading it all to "Registry B" and maybe even a subsequent call to load "Registry C".

So with that I kind of need to specify the registry when the load command is called.  Does that help?
I'll get to work on those e2e test a.s.a.p.

@amartin120 thanks for the explanation, I think I misunderstood before what the --registry flag was for!
We have this environment variable we use to signal pushing signatures to a different registry called COSIGN_REPOSITORY -- https://github.com/sigstore/cosign#specifying-registry
WDYT of reusing that instead of the flag?

I'll take a look at COSIGN_REPOSITORY .  Thanks for the info!

@priyawadhwa I apologize for the delay.  COSIGN_REPOSITORY seems like a good option for a single repository.  However, this PR allows for loading multiple images in a single cosign load execution which could load to multiple repositories, just in another registry.
We could still reuse COSIGN_REPOSITORY if we could just put a registry name in there instead of a full repository designation.  Would this still make sense over a separate registry flag?
Hopefully I'm understanding things correctly here.  Thanks for your patience.

@amartin120 no worries, I think we can just go ah we can keep the flag since it seems more accurately named and more intuitive. I can take a look at this once CI is green!

@priyawadhwa Thank you.  I may need some help with the CI.  Both of the failing stages look like timeouts as opposed to true failures.  Am I reading the logs right?

@amartin120 I think the linter is picking up some things that need to be fixed! If you go through the files on this PR it should be visible in the Github UI:
Error: S1017: should replace this `if` statement with an unconditional `strings.TrimPrefix` (gosimple)
  Error: ineffectual assignment to err (ineffassign)
  Error: ineffectual assignment to err (ineffassign)
  Error: ineffectual assignment to err (ineffassign)
  Error: SA[40](https://github.com/sigstore/cosign/actions/runs/6551101939/job/18094664470?pr=3239#step:4:42)06: this value of `err` is never used (staticcheck)
  Error: SA4006: this value of `err` is never used (staticcheck)
  Error: SA4006: this value of `err` is never used (staticcheck)

Strange.  Those were not there when I responded originally.  Maybe a Github hiccup at the time?  Maybe I was just blind?  The logs just showed timeouts.  Oh well.  I'll work on this today now that I can see them.  Thanks again for the patience.

Oh, hey! All checks are all passing. Any objections to merging this @priyawadhwa?

This PR is almost a year old with no activity.  Bumping for visibility to see if we can get some traction.  This is a much needed feature to help bloat when using multiple images.


Skill Declarations submit with ajax

This includes server-side verification, and ux works on profile AND on applications, which includes a save all button.

Every time you add a new skill on My Profile, the textarea gets another red asterisk.

This requires you to run npm install to function on IE


Bug: Ordinal lint confused

In the following document harper belives that "2" in the title should be changing the "the"
# 2

The

The lint warning is: This number needs a different suffix to sound right. and the suggested fix is: Replace with: “nd”

I'm unable to replicate this issue on the latest version of Harper. Would you mind confirming that it no longer plagues us so I may close this issue?

I cant replicate it either! Closing.


Sanitize/ escape user input

When a tech savvy user enters html code or scripts in the text input box it is output directly unto the page. Scripts fortunately do not run but the html is parsed on the page.  A fix is being implemented in #85

fixed in #85


[Feature][Config UI] Blueprint detail page - Improve error message tone and voice consistency

Search before asking

[X] I had searched in the issues and found no similar feature requirement.

Description
This is an improvement for user experience. We need to show helpful and consistent messages on the UI when a Blueprint fails, so that users can fix the problem more easily. For this feature, we need a list of all known errors that cause the failure of Blueprints, and then work on the copywriting for these messages.
Use case
No response
Related issues
#1447
Are you willing to submit a PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Bump

Closing this because it's an ongoing improvements and will be addressed along the way of Config UI iterations.


Opening up security - Vault post

Vault article, detailing a short overview of how it was implemented recently in a client, some of the problems the team came across and how they eventually solved them.
Requesting any feedback on content or grammar, happy to modify before merge.

@flock3 Are you wanting more reviews on this or awaiting a merge?

Sorry no, I need to make some final updates based on review feedback, I will try and get these updates done in the next day

@Rucknar ready for merge


feat: support starting S3rver with TLS

Add support for starting S3rver with TLS in a similar manner as done in serverless-offline

@AndreasSvendby
Thanks for your great contribution! When I merged this PR, I found that there ware some conflicts. So I modified this PR little to resolve these conflicts.

@AndreasSvendby
Thanks for your great contribution! When I merged this PR, I found that there ware some conflicts. So I modified this PR little to resolve these conflicts.

Great, thanks for the quick addition 😄 I needed this for a project 😁


Issue with certain mp3 files -> `read': negative length -1430667650 given (ArgumentError)

Hey all,
I am working with a very specific mp3 file trying to get all the metadata from it using format parser gem and I got the following error (StringIO throwing on read):
ruby/gems/2.7.0/gems/id3tag-0.14.1/lib/id3tag/audio_file.rb:53:in `read': negative length -1430667650 given (ArgumentError)

Unfortunately, I cannot share the file within this issue (as I am not the author) and I simply cannot recreate a file with the breaking scenario.
But the problem apparently is the following:
1 - the file has a v2_tag_present so it gets through should_and_could_read_v2_frames? - https://github.com/krists/id3tag/blob/master/lib/id3tag/tag.rb#L118
2 - but when tries gets the v2_tag_body it fails giving the error - here
2.1 - inside the v2_tag_body method, v2_tag_frame_and_paddding_size gives a negative value and stores in bytes_to_read
2.2 - the negative values comes from the v2_extended_header_size being bigger than the v2_tag_header.tag_size
2.2 -   then @file.read methods of course fails
I was trying to fix the issue myself but I am fairly new to Ruby and also to working with mp3 files and id3 v2_tags.
But nonetheless, here are some proposals, please let me know your thoughts and I will create a PR:
Proposal 1:
I could simply return BLANK_STRING in v2_tag_body if the bytes_to_read happen to be negative something like this:
    def v2_tag_body
      if v2_tag_size > 0
        @file.seek(v2_tag_frame_and_padding_position)
        bytes_to_read = v2_tag_frame_and_padding_size

       return BLANK_STRING if bytes_to_read < 0

        limit = ID3Tag.configuration.v2_tag_read_limit
        if (limit > 0) && (bytes_to_read > limit)
          bytes_to_read = limit
        end
        @file.read(bytes_to_read) || BLANK_STRING
      end
    end

Proposal 2
I wonder though if the fix would be better to be done somewhere in get_v2_extendend_header_size as it is commented that it could only be either 6 or 10 bytes, but in reality it is giving a much bigger result, but unfortunately I am not sure what value should return in this case or why this number is much bigger than the expected:
    def get_v2_extended_header_size
      if v2_tag_header.extended_header?
        if v2_tag_major_version_number == 3
          # ext. header size for 2.3.0 does not include size bytes.
          # There are only 2 possible sizes - 6 or 10 bytes, which means extended header can take 10 or 14 bytes.
          4 + NumberUtil.convert_string_to_32bit_integer(v2_extended_header_size_bytes)
        else
          SynchsafeInteger.decode(NumberUtil.convert_string_to_32bit_integer(v2_extended_header_size_bytes))
        end
      else
        0
      end
    end

Original error traceback:
Traceback (most recent call last):
        43: from exe/format_parser_inspect:22:in `<main>'
        42: from exe/format_parser_inspect:22:in `map'
        41: from exe/format_parser_inspect:24:in `block in <main>'
        40: from exe/format_parser_inspect:24:in `public_send'
        39: from /****/format_parser/lib/format_parser.rb:109:in `parse_file_at'
        38: from /****/format_parser/lib/format_parser.rb:109:in `open'
        37: from /****/format_parser/lib/format_parser.rb:110:in `block in parse_file_at'
        36: from /****/format_parser/lib/format_parser.rb:178:in `parse'
        35: from /****/format_parser/lib/format_parser.rb:178:in `to_a'
        34: from /****/format_parser/lib/format_parser.rb:178:in `each'
        33: from /****/format_parser/lib/format_parser.rb:178:in `each'
        32: from /****/format_parser/lib/format_parser.rb:178:in `each'
        31: from /****/format_parser/lib/format_parser.rb:178:in `each'
        30: from /****/format_parser/lib/format_parser.rb:174:in `block in parse'
        29: from /****/format_parser/lib/format_parser.rb:206:in `execute_parser_and_capture_expected_exceptions'
        28: from /*/.rbenv/versions/2.7.4/lib/ruby/gems/2.7.0/gems/measurometer-1.3.0/lib/measurometer.rb:48:in `instrument'
        27: from /****/format_parser/lib/format_parser.rb:207:in `block in execute_parser_and_capture_expected_exceptions'
        26: from /****/format_parser/lib/parsers/mp3_parser.rb:103:in `call'
        25: from /****/format_parser/lib/parsers/mp3_parser.rb:318:in `with_id3tag_local_configs'
        24: from /*/.rbenv/versions/2.7.4/lib/ruby/gems/2.7.0/gems/id3tag-0.14.1/lib/id3tag.rb:52:in `local_configuration'
        23: from /*/.rbenv/versions/2.7.4/lib/ruby/gems/2.7.0/gems/id3tag-0.14.1/lib/id3tag/configuration.rb:9:in `local_configuration'
        22: from /*/.rbenv/versions/2.7.4/lib/ruby/gems/2.7.0/gems/id3tag-0.14.1/lib/id3tag/configuration.rb:32:in `local_configuration'
        21: from /****/format_parser/lib/parsers/mp3_parser.rb:322:in `block in with_id3tag_local_configs'
        20: from /****/format_parser/lib/parsers/mp3_parser.rb:103:in `block in call'
        19: from /****/format_parser/lib/parsers/mp3_parser.rb:302:in `blend_id3_tags_into_hash'
        18: from /****/format_parser/lib/parsers/mp3_parser.rb:302:in `each_with_object'
        17: from /****/format_parser/lib/parsers/mp3_parser.rb:302:in `each'
        16: from /****/format_parser/lib/parsers/mp3_parser.rb:303:in `block in blend_id3_tags_into_hash'
        15: from /****/format_parser/lib/parsers/mp3_parser.rb:50:in `to_h'
        14: from /****/format_parser/lib/parsers/mp3_parser.rb:50:in `each_with_object'
        13: from /****/format_parser/lib/parsers/mp3_parser.rb:50:in `each'
        12: from /****/format_parser/lib/parsers/mp3_parser.rb:52:in `block in to_h'
        11: from /****/format_parser/lib/parsers/mp3_parser.rb:52:in `public_send'
        10: from /*/.rbenv/versions/2.7.4/lib/ruby/gems/2.7.0/gems/id3tag-0.14.1/lib/id3tag/tag.rb:19:in `block (2 levels) in <class:Tag>'
         9: from /*/.rbenv/versions/2.7.4/lib/ruby/gems/2.7.0/gems/id3tag-0.14.1/lib/id3tag/tag.rb:24:in `content_of_first_frame'
         8: from /*/.rbenv/versions/2.7.4/lib/ruby/gems/2.7.0/gems/id3tag-0.14.1/lib/id3tag/tag.rb:40:in `first_frame_by_id'
         7: from /*/.rbenv/versions/2.7.4/lib/ruby/gems/2.7.0/gems/id3tag-0.14.1/lib/id3tag/tag.rb:40:in `find'
         6: from /*/.rbenv/versions/2.7.4/lib/ruby/gems/2.7.0/gems/id3tag-0.14.1/lib/id3tag/tag.rb:40:in `each'
         5: from /*/.rbenv/versions/2.7.4/lib/ruby/gems/2.7.0/gems/id3tag-0.14.1/lib/id3tag/tag.rb:40:in `block in first_frame_by_id'
         4: from /*/.rbenv/versions/2.7.4/lib/ruby/gems/2.7.0/gems/id3tag-0.14.1/lib/id3tag/tag.rb:64:in `frame_ids'
         3: from /*/.rbenv/versions/2.7.4/lib/ruby/gems/2.7.0/gems/id3tag-0.14.1/lib/id3tag/tag.rb:68:in `frames'
         2: from /*/.rbenv/versions/2.7.4/lib/ruby/gems/2.7.0/gems/id3tag-0.14.1/lib/id3tag/tag.rb:73:in `v2_frames'
         1: from /*/.rbenv/versions/2.7.4/lib/ruby/gems/2.7.0/gems/id3tag-0.14.1/lib/id3tag/audio_file.rb:59:in `v2_tag_body'
/*/.rbenv/versions/2.7.4/lib/ruby/gems/2.7.0/gems/id3tag-0.14.1/lib/id3tag/audio_file.rb:59:in `read': negative length -1430667650 given (ArgumentError)

Hi, @lao!
This is a really strange case and thank you for this very detailed explanation. Your suggested changes in MR are reasonable.
I will merge that and release a new gem version.


[Feature]: Add a method to GenericContainer to expose a random container port with the same number as the host port

Module
Core
Problem

Currently, it is not possible to expose a random port in a container to the same random port on the host. To achieve this goal, you can only use the deprecated class org.testcontainers.containers.FixedHostPortGenericContainer. For HiveServer2 with Zookeeper service discovery enabled, there are similar operations as follows.

import org.apache.curator.test.InstanceSpec;
import org.junit.jupiter.api.Test;
import org.testcontainers.containers.FixedHostPortGenericContainer;
import org.testcontainers.containers.GenericContainer;
import org.testcontainers.containers.Network;
public class ExampleTest {
    @Test
    void test() {
        Network network = Network.newNetwork();
        int randomPort = InstanceSpec.getRandomPort();
        try (
                GenericContainer<?> zookeeper = new GenericContainer<>("zookeeper:3.9.3-jre-17")
                        .withNetwork(network)
                        .withNetworkAliases("foo")
                        .withExposedPorts(2181);
                GenericContainer<?> hiveServer2 = new FixedHostPortGenericContainer<>("apache/hive:4.0.1")
                        .withNetwork(network)
                        .withEnv("SERVICE_NAME", "hiveserver2")
                        .withFixedExposedPort(randomPort, randomPort)
                        .dependsOn(zookeeper)
        ) {
            zookeeper.start();
            hiveServer2.withEnv("SERVICE_OPTS", "-Dhive.server2.support.dynamic.service.discovery=true" + " "
                    + "-Dhive.zookeeper.quorum=" + zookeeper.getNetworkAliases().get(0) + ":2181" + " "
                    + "-Dhive.server2.thrift.bind.host=<IP_ADDRESS>" + " "
                    + "-Dhive.server2.thrift.port=" + randomPort);
            hiveServer2.start();
        }
    }
}

The only purpose of org.apache.curator.test.InstanceSpec#getRandomPort() is to get a random host port. This can sometimes conflict with the port in the container.
This is like a Docker Compose unit like this,

services:
  zookeeper:
    image: zookeeper:3.9.3-jre-17
    ports:
      - "12181:2181"
  apache-hive-1:
    image: apache/hive:4.0.1
    depends_on:
      - zookeeper
    environment:
      SERVICE_NAME: hiveserver2
      SERVICE_OPTS: >-
        -Dhive.server2.support.dynamic.service.discovery=true
        -Dhive.zookeeper.quorum=zookeeper:2181
        -Dhive.server2.thrift.bind.host=<IP_ADDRESS>
        -Dhive.server2.thrift.port=23593
    ports:
      - "23593:23593"

There is almost no way around org.testcontainers.containers.FixedHostPortGenericContainer to use a random numeric port both on the host, inside the container, and in the container's environment variables.

Solution

I was expecting there to be a method in GenericContainer to expose the same host port number on a random container port. If this method is called org.testcontainers.containers.GenericContainer#withRandomExposedPorts(), it can expose a random container port. And allow the host to obtain this port number through org.testcontainers.containers.GenericContainer#getFirstMappedPort(), then the use of org.testcontainers.containers.FixedHostPortGenericContainer can obviously be simplified to,

import org.junit.jupiter.api.Test;
import org.testcontainers.containers.GenericContainer;
import org.testcontainers.containers.Network;
public class ExampleTest {
    @Test
    void test() {
        Network network = Network.newNetwork();
        try (
                GenericContainer<?> zookeeper = new GenericContainer<>("zookeeper:3.9.3-jre-17")
                        .withNetwork(network)
                        .withNetworkAliases("foo")
                        .withExposedPorts(2181);
                GenericContainer<?> hiveServer2 = new GenericContainer<>("apache/hive:4.0.1")
                        .withNetwork(network)
                        .withEnv("SERVICE_NAME", "hiveserver2")
                        .withRandomExposedPorts()
                        .dependsOn(zookeeper)
        ) {
            zookeeper.start();
            hiveServer2.withEnv("SERVICE_OPTS", "-Dhive.server2.support.dynamic.service.discovery=true" + " "
                    + "-Dhive.zookeeper.quorum=" + zookeeper.getNetworkAliases().get(0) + ":2181" + " "
                    + "-Dhive.server2.thrift.bind.host=<IP_ADDRESS>" + " "
                    + "-Dhive.server2.thrift.port=" + hiveServer2.getFirstMappedPort());
            hiveServer2.start();
        }
    }
}

Benefit

This helps simplify the process of starting a HiveServer2 with Zookeeper service discovery enabled.

Alternatives

Using org.testcontainers.containers.FixedHostPortGenericContainer can indeed directly solve the current issue, which is what https://github.com/apache/shardingsphere/pull/33768 and https://github.com/apache/shardingsphere/issues/29052 are doing.
But the problem is that org.testcontainers.containers.FixedHostPortGenericContainer has been deprecated.

Would you like to help contributing this feature?
No

Hey @linghengqian, what you are trying to achieve there, should already work like this:
import org.apache.curator.test.InstanceSpec;
import org.junit.jupiter.api.Test;
import org.testcontainers.containers.FixedHostPortGenericContainer;
import org.testcontainers.containers.GenericContainer;
import org.testcontainers.containers.Network;
public class ExampleTest {
    @Test
    void test() {
        Network network = Network.newNetwork();
        try (
                GenericContainer<?> zookeeper = new GenericContainer<>("zookeeper:3.9.3-jre-17")
                        .withNetwork(network)
                        .withNetworkAliases("foo")
                        .withExposedPorts(2181);
                GenericContainer<?> hiveServer2 = new GenericContainer<>("apache/hive:4.0.1")
                        .withNetwork(network)
                        .withEnv("SERVICE_NAME", "hiveserver2")
                        .withExposedPort(4711)
                        .dependsOn(zookeeper)
        ) {
            zookeeper.start();
            hiveServer2.withEnv("SERVICE_OPTS", "-Dhive.server2.support.dynamic.service.discovery=true" + " "
                    + "-Dhive.zookeeper.quorum=" + zookeeper.getNetworkAliases().get(0) + ":2181" + " "
                    + "-Dhive.server2.thrift.bind.host=<IP_ADDRESS>" + " "
                    + "-Dhive.server2.thrift.port=" + hiveServer2.getFirstMappedPort());
            hiveServer2.start();
        }
    }
}

I don't see why the port within the container needs to be the same and why the internal port needs to be random (I understand it can be arbitrary though).

hiveServer2.withEnv("SERVICE_OPTS", "-Dhive.server2.support.dynamic.service.discovery=true" + " "
                    + "-Dhive.zookeeper.quorum=" + zookeeper.getNetworkAliases().get(0) + ":2181" + " "
                    + "-Dhive.server2.thrift.bind.host=<IP_ADDRESS>" + " "
                    + "-Dhive.server2.thrift.port=" + hiveServer2.getFirstMappedPort());
hiveServer2.start();

@kiview I've wanted to do something similar before, but the problem is that testcontainers simply don't allow it.  Mapped port can only be obtained after the container is started

java.lang.IllegalStateException: Mapped port can only be obtained after the container is started
	at org.testcontainers.shaded.com.google.common.base.Preconditions.checkState(Preconditions.java:513)
	at org.testcontainers.containers.ContainerState.getMappedPort(ContainerState.java:161)
	at io.github.linghengqian.hive.server2.jdbc.driver.thin.ZookeeperServiceDiscoveryTest.assertShardingInLocalTransactions(ZookeeperServiceDiscoveryTest.java:75)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1597)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1597)

I'm not sure if I should pull the unit tests I wrote in https://github.com/linghengqian/hive-server2-jdbc-driver/pull/14 into a separate git, since this exception seems to be thrown intentionally by testcontainers.

Hi, what you can try is create a custom script, see LocalStackContainer for reference.
https://github.com/testcontainers/testcontainers-java/blob/main/modules/localstack/src/main/java/org/testcontainers/containers/localstack/LocalStackContainer.java#L126-L132
The script must be copied to the container when it is starting
https://github.com/testcontainers/testcontainers-java/blob/main/modules/localstack/src/main/java/org/testcontainers/containers/localstack/LocalStackContainer.java#L204-L212
Hope that helps.

@eddumelendez I must admit I'm a testcontainers newbie, and I'm assuming /usr/local/bin/docker-entrypoint.sh is some kind of built-in script? I further tested org.testcontainers.containers.GenericContainer#containerIsStarting at https://github.com/linghengqian/hive-server2-jdbc-driver/pull/14 and concluded that the Docker Image of apache/hive:4.0.1 will not continue to monitor changes in environment variables after it is started. Therefore, modifying the environment variables in the mounted /testcontainers_start.sh has no effect.

I don't see why the port within the container needs to be the same and why the internal port needs to be random (I understand it can be arbitrary though).

@kiview Allow me to explain what the following Docker Compose file does.

services:
  zookeeper:
    image: zookeeper:3.9.3-jre-17
    ports:
      - "12181:2181"
  apache-hive-1:
    image: apache/hive:4.0.1
    depends_on:
      - zookeeper
    environment:
      SERVICE_NAME: hiveserver2
      SERVICE_OPTS: >-
        -Dhive.server2.support.dynamic.service.discovery=true
        -Dhive.zookeeper.quorum=zookeeper:2181
        -Dhive.server2.thrift.bind.host=<IP_ADDRESS>
        -Dhive.server2.thrift.port=23593
    ports:
      - "23593:23593"

At this time, for the znode /hiveserver2/serverUri=<IP_ADDRESS>:23593;version=4.0.1;sequence=0000000000 in service zookeeper, the value exists as hive.server2.instance.uri=<IP_ADDRESS>:23593;hive.server2.authentication=NONE;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=<IP_ADDRESS>;hive.server2.thrift.port=23593;hive.server2.use.SSL=false.
If I want to connect to this HiveServer2 instance apache-hive-1 in unit tests outside the Docker Network, I actually need to create a database connection with a JdbcUrl of jdbc:hive2://localhost:12181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2 through the JDBC Driver.
This database connection first connects to Zookeeper Server zookeeper, finds the host and port of HiveServer2 apache-hive-1 as <IP_ADDRESS>:23593 from /hiveserver2/ znode, and then reconnects to HiveServer2 using the <IP_ADDRESS>:23593 information.
This means that for apache-hive-1, the values of -Dhive.server2.thrift.port, hostPort, and containerPort all need to be the same random number, such as 23593... It looks like there is no way for me to completely eliminate the use of org.testcontainers.containers.FixedHostPortGenericContainer.

the entrypoint in apache/hive image is /entrypoint.sh. What I am suggesting is override the entrypoint to create a custom one with /testcontainers_start.sh in order to create new env vars and calling /entrypoint.sh as part of the custom script for Hive initialization.

The result of my test at https://github.com/linghengqian/hive-server2-jdbc-driver/pull/14 is that I cannot call GenericContainer#getMappedPort(int) in GenericContainer#containerIsStarting. This will cause the following exception,

[main] ERROR tc.apache/hive:4.0.1 - Could not start container
org.testcontainers.containers.ContainerLaunchException: Timed out waiting for container port to open (localhost ports: [32818] should be listening)
	at org.testcontainers.containers.wait.strategy.HostPortWaitStrategy.waitUntilReady(HostPortWaitStrategy.java:112)
	at org.testcontainers.containers.wait.strategy.AbstractWaitStrategy.waitUntilReady(AbstractWaitStrategy.java:52)
	at org.testcontainers.containers.GenericContainer.waitUntilContainerStarted(GenericContainer.java:909)
	at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:500)

This requires two class tests in JDK17+. I don't seem to notice any explanation from the testcontainers documentation.

import com.github.dockerjava.api.command.InspectContainerResponse;
import org.testcontainers.containers.GenericContainer;
import org.testcontainers.images.builder.Transferable;
import java.io.IOException;
import java.net.ServerSocket;
@SuppressWarnings({"resource"})
public class HS2Container extends GenericContainer<HS2Container> {
    String zookeeperConnectionString;
    private static final String STARTER_SCRIPT = "/testcontainers_start.sh";
    private final int randomPortFirst = getRandomPort();
    public HS2Container(final String dockerImageName) {
        super(dockerImageName);
        withEnv("SERVICE_NAME", "hiveserver2");
        withExposedPorts(randomPortFirst);
        withCreateContainerCmdModifier(cmd ->
                cmd.withEntrypoint("sh", "-c", "while [ ! -f " + STARTER_SCRIPT + " ]; do sleep 0.1; done; " + STARTER_SCRIPT)
        );
    }
    public HS2Container withZookeeperConnectionString(final String zookeeperConnectionString) {
        this.zookeeperConnectionString = zookeeperConnectionString;
        return self();
    }
    @Override
    protected void containerIsStarting(InspectContainerResponse containerInfo) {
        String command = """
                #!/bin/bash
                export SERVICE_OPTS="-Dhive.server2.support.dynamic.service.discovery=true -Dhive.zookeeper.quorum=%s -Dhive.server2.thrift.bind.host=<IP_ADDRESS> -Dhive.server2.thrift.port=%s"
                /entrypoint.sh
                """.formatted(zookeeperConnectionString, getMappedPort(randomPortFirst));
        copyFileToContainer(Transferable.of(command, 0777), STARTER_SCRIPT);
    }
    private int getRandomPort() {
        try (ServerSocket server = new ServerSocket(0)) {
            server.setReuseAddress(true);
            return server.getLocalPort();
        } catch (IOException exception) {
            throw new Error(exception);
        }
    }
}

import com.zaxxer.hikari.HikariConfig;
import com.zaxxer.hikari.HikariDataSource;
import org.apache.curator.framework.CuratorFramework;
import org.apache.curator.framework.CuratorFrameworkFactory;
import org.apache.curator.retry.ExponentialBackoffRetry;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.Test;
import org.testcontainers.containers.GenericContainer;
import org.testcontainers.containers.Network;
import org.testcontainers.junit.jupiter.Container;
import org.testcontainers.junit.jupiter.Testcontainers;
import javax.sql.DataSource;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.sql.Statement;
import java.time.Duration;
import java.util.List;
import java.util.Properties;
import static org.awaitility.Awaitility.await;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.is;
@SuppressWarnings({"SqlDialectInspection", "SqlNoDataSourceInspection", "resource"})
@Testcontainers
class ZookeeperServiceDiscoveryTest {
    private static final Network NETWORK = Network.newNetwork();
    @Container
    private static final GenericContainer<?> ZOOKEEPER_CONTAINER = new GenericContainer<>("zookeeper:3.9.3-jre-17")
            .withNetwork(NETWORK)
            .withNetworkAliases("foo")
            .withExposedPorts(2181);
    private final String jdbcUrlSuffix = ";serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2";
    private final String jdbcUrlPrefix = "jdbc:hive2://" + ZOOKEEPER_CONTAINER.getHost() + ":" + ZOOKEEPER_CONTAINER.getMappedPort(2181) + "/";
    @AfterAll
    static void afterAll() {
        NETWORK.close();
    }
    @Test
    void assertShardingInLocalTransactions() throws SQLException {
        try (GenericContainer<?> hs2FirstContainer = new HS2Container("apache/hive:4.0.1")
                .withNetwork(NETWORK)
                .withZookeeperConnectionString(ZOOKEEPER_CONTAINER.getNetworkAliases().get(0) + ":" + ZOOKEEPER_CONTAINER.getMappedPort(2181))
                .dependsOn(ZOOKEEPER_CONTAINER)) {
            hs2FirstContainer.start();
            awaitHS2(hs2FirstContainer.getFirstMappedPort());
            HikariConfig config = new HikariConfig();
            config.setDriverClassName("org.apache.hive.jdbc.HiveDriver");
            config.setJdbcUrl(jdbcUrlPrefix + jdbcUrlSuffix);
            DataSource dataSource = new HikariDataSource(config);
            extractedSQL(dataSource);
        }
    }
    private static void extractedSQL(final DataSource dataSource) throws SQLException {
        try (Connection connection = dataSource.getConnection();
             Statement statement = connection.createStatement()) {
            statement.execute("CREATE DATABASE demo_ds_0");
        }
    }
    private void awaitHS2(final int hiveServer2Port) {
        String connectionString = ZOOKEEPER_CONTAINER.getHost() + ":" + ZOOKEEPER_CONTAINER.getMappedPort(2181);
        await().atMost(Duration.ofMinutes(1L)).ignoreExceptions().until(() -> {
            try (CuratorFramework client = CuratorFrameworkFactory.builder()
                    .connectString(connectionString)
                    .retryPolicy(new ExponentialBackoffRetry(1000, 3))
                    .build()) {
                client.start();
                List<String> children = client.getChildren().forPath("/hiveserver2");
                assertThat(children.size(), is(1));
                return children.get(0).startsWith("serverUri=<IP_ADDRESS>:" + hiveServer2Port + ";version=4.0.1;sequence=");
            }
        });
        await().atMost(Duration.ofMinutes(1L)).ignoreExceptions().until(() -> {
            DriverManager.getConnection(jdbcUrlPrefix + jdbcUrlSuffix, new Properties()).close();
            return true;
        });
    }
}

Thanks for sharing! Let's revisit the code.
HS2Container looks for a random port on the host. Meanwhile, getMappedPort looks for random port to the given port on the container. So, the current code is looking for a port that doesn't exist on the container, only in the host. I see the HiveServer2 default port is 10000, so, the code should be getMappedPort(10000). However, checking some docs hive.server2.thrift.port changes the hive serve2 port, so, I guess your only option is using fixed port. This is unfortunate with some services where client and server exchange server metadata and client fails because the server sent original port instead of the random one. I wonder if adding a proxy would help. If you have a isolated project with proper tests workinf without testcontainers I can help.
The original title description is not valid because testcontainers checks ports on the container not the host. I'll move this to discussions.

No problem. Should I just close this issue? I prefer that there is no such proxy.


[#1191] fix script creation bug

This pr is to address #1191

@shawncrawley
This is pr is to address #1191. Just one line changed. Can you review this? thanks

Can one of the admins verify this patch?


ES6 export syntax not supported in transpiled handler

Expected behaviour: I can write my handler script using ES6 syntax, transpile it (along the lines discussed in #19), and then deploy using claudia

What actually happens: If I use the ES6 export syntax in my handler, validatePackage fails because it can't find the module's proxyRouter property, causing claudia update to fail with: handler.js does not export a Claudia API Builder instance

Link to a minimal, executable project that demonstrates the problem:

import ApiBuilder from 'claudia-api-builder';
export const api = new ApiBuilder();

This transpiles to:
function _interopRequireDefault(obj) { return obj && obj.__esModule ? obj : { default: obj }; }
var _claudiaApiBuilder = require('claudia-api-builder');
var _claudiaApiBuilder2 = _interopRequireDefault(_claudiaApiBuilder);
Object.defineProperty(exports, "__esModule", {
  value: true
});
exports.api = undefined;
var api = exports.api = new _claudiaApiBuilder2.default();

Steps to reproduce the problem:

The error is produced in validate-package.js here:
	try {
		apiModule = require(path.join(dir, apiModulePath));
	} catch (e) {
		console.error(e.stack || e);
		throw `cannot require ./${apiModulePath} after clean installation. Check your dependencies.`;
	}
	if (!apiModule[handlerMethod]) {
		if (restApiModule) {
			throw `${apiModulePath}.js does not export a Claudia API Builder instance`;
		} else {
			throw `${apiModulePath}.js does not export method ${handlerMethod}`;
		}
	}

I guess it's happening because of the dynamic require. Is there any way to make this work with ES6-style import/export?
Thanks!

Hi,
it looks as if your transpiled code is not exporting the api directly, but exporting an object with the property api (exports.api = ....). This is a bug in the transpiler, not the way how claudia loads things.

because AWS Lambda still works with Node 6, not node 8, perhaps you can target the transpiler to produce Node 6 compatible-code?

For the record I'm using babel, which is the most popular/common transpiler, with the following in my .babelrc:
{"presets": ["es2015", "es2016", "es2017"]}

So, nothing fancy or beyond the ordinary going on. I'm not sure how to ask babel to target node 6 compatible code.
It looks like right now dynamic imports using ES6-style import/export are not possible but they may be possible in the future. I guess there's nothing to be done here, but it's a little annoying that I have to modify my otherwise ES6 code to work with claudia (in practice, using module.exports = api works).
Thanks :)

Do you think it might be worth making a note of this on https://claudiajs.com/tutorials/using-babel.html?

This still seems to be an "issue" in version 5.0.1 building towards Node 8.10 using Babel 7.


expression data from Dan Jacobson preprint

don't see links, this may require reaching out
https://www.biorxiv.org/content/10.1101/2<IP_ADDRESS>712v2
Name of the dataset
A clear and concise description of the dataset and where you can download it from.
Mapping or relevant fields
A clear and concise description of what which fields you would want to be ingested.
If possible, highlight which fields map to nodes and which fields map to edges.
Refer to Data Preparation for guidelines on how the final transformed data should be represented.
Additional context
Add any other context, requests, concerns.

can you sketch what this might look like, how this would be modeled?

This is missing "A clear and concise description of what which fields you would want to be ingested. If possible, highlight which fields map to nodes and which fields map to edges."
Also, needs info about how ingesting this data source would benefit kg-covid-19.


feature: Change default hostPath paths

Hi,
While trying to deploy the helm chart for Aqua enforcer, we noticed it heavily relies on hostPath and hardcoded paths: https://github.com/aquasecurity/aqua-helm/blob/4f1c0cd56801f1d6c2b58ea6db66607628de1db6/enforcer/templates/enforcer-daemonset.yaml#L152
People with immutable nodes could have problems deploying this template, thus, a way to modify the path of the volumes is required.

Hi @pupseba
Our apologies that the response took so long time
The mounts in enforcer deployment are based on product requirements.
In addition, we can add some logic to our deployment based on platform type.
Do you have any specific scenario when this configuration does not fit your requirements?
Can you please share it with us and we will try to suggest some solution that will help as you and will not impact other our customers?

@pupseba we are closing this issue and your PR due to our previous comment
if you still think that this change is required for you, please take it with Aqua Support and open RFE for it


Mongo Cache Error

First of all, I want to express my gratitude to you for providing such a great product that has truly helped me a lot. However, I have encountered some issues now. When I use mongo as a cache, if the conditions in the featrue involve expressions such as in list, which require expressions, an error will be reported when the cache is set, and the information is as follows.
{"level":50,"time":1678091056765,"pid":29,"hostname":"growthbook-proxy-6c8586dfd6-p8s4m","err":{"type":"MongoServerError","message":"The dollar ($) prefixed field '$in' in 'entry.payload.features.demo-feature.rules.0.condition.id.$in' is not valid for storage.","stack":"MongoServerError: The dollar ($) prefixed field '$in' in 'entry.payload.features.demo-feature.rules.0.condition.id.$in' is not valid for storage.\n    at /usr/local/src/app/node_modules/mongodb/lib/operations/update.js:132:33\n    at /usr/local/src/app/node_modules/mongodb/lib/cmap/connection_pool.js:333:25\n    at /usr/local/src/app/node_modules/mongodb/lib/sdam/server.js:212:17\n    at handleOperationResult (/usr/local/src/app/node_modules/mongodb/lib/sdam/server.js:328:20)\n    at Connection.onMessage (/usr/local/src/app/node_modules/mongodb/lib/cmap/connection.js:239:9)\n    at MessageStream.<anonymous> (/usr/local/src/app/node_modules/mongodb/lib/cmap/connection.js:60:60)\n    at MessageStream.emit (node:events:513:28)\n    at processIncomingData (/usr/local/src/app/node_modules/mongodb/lib/cmap/message_stream.js:125:16)\n    at MessageStream._write (/usr/local/src/app/node_modules/mongodb/lib/cmap/message_stream.js:33:9)\n    at writeOrBuffer (node:internal/streams/writable:391:12)","index":0,"code":52,"codeName":"DollarPrefixedFieldName"},"msg":"Unable to refresh stale cache"}

{"defaultValue":false,"rules":[{"condition":{"id":{"$in":["1","2","3","4"]}},"force":true},{"condition":{"partnerId":"1"},"force":true}]}

Another inconvenience is that I am not able to customize the database name. I would like to put the cache collection of the proxy in the growthbook database as well.

Sorry for the slow reply, and thanks for bringing this to my attention. I'll have a look at these issues.

Hi @fivesmallq, I've resolved both of these issues in the latest build (1.0.14).
For customizing your Mongo connection, you now have 2 addtitional env vars:
CACHE_DATABASE_NAME
CACHE_COLLECTION_NAME

Wow, that's amazing! Lightning-fast speed, thank you so much. I will upgrade and test it later.