packer vagrant authentication error

Version (k3OS / kernel)

Architecture

Describe the bug

can't get packer to package a vagrant machine
To Reproduce

packer build vagrant.json
Expected behavior

a vagrant box
Actual behavior

it seems to run the provision steps but vagrant can't authenticate the second time
==> Builds finished but no artifacts were created.
Additional context

OS X Vagrant 2.2.7 Virtualbox 6.1.4
OUTPUT
virtualbox-iso: output will be in this color.
==> virtualbox-iso: Retrieving Guest additions
==> virtualbox-iso: Trying /Applications/VirtualBox.app/Contents/MacOS/VBoxGuestAdditions.iso
==> virtualbox-iso: Trying /Applications/VirtualBox.app/Contents/MacOS/VBoxGuestAdditions.iso
==> virtualbox-iso: /Applications/VirtualBox.app/Contents/MacOS/VBoxGuestAdditions.iso => /Applications/VirtualBox.app/Contents/MacOS/VBoxGuestAdditions.iso
==> virtualbox-iso: Retrieving ISO
==> virtualbox-iso: Trying https://github.com/rancher/k3os/releases/download/v0.9.0-rc4/k3os-amd64.iso
==> virtualbox-iso: Trying https://github.com/rancher/k3os/releases/download/v0.9.0-rc4/k3os-amd64.iso?checksum=sha256%3Af790f005b7c6da19b701f1a30d5786d15eb5b5b3b49837a67872769ce0709abc
k3os-amd64.iso 450.89 MiB / 450.89 MiB [==============================================================================================] 100.00% 2m20s
==> virtualbox-iso: https://github.com/rancher/k3os/releases/download/v0.9.0-rc4/k3os-amd64.iso?checksum=sha256%3Af790f005b7c6da19b701f1a30d5786d15eb5b5b3b49837a67872769ce0709abc => /Users/svefors/dev/projects/k3os/package/packer/vagrant/packer_cache/a3d15a65497858c1fd89a94cefd79ea5696d26ce.iso
==> virtualbox-iso: Starting HTTP server on port 8573
==> virtualbox-iso: Creating virtual machine...
==> virtualbox-iso: Creating hard drive...
==> virtualbox-iso: Creating forwarded port mapping for communicator (SSH, WinRM, etc) (host port 4072)
==> virtualbox-iso: Starting the virtual machine...
==> virtualbox-iso: Waiting 6s for boot...
==> virtualbox-iso: Typing the boot command...
==> virtualbox-iso: Using ssh communicator to connect: <IP_ADDRESS>
==> virtualbox-iso: Waiting for SSH to become available...
==> virtualbox-iso: Error waiting for SSH: Packer experienced an authentication error when trying to connect via SSH. This can happen if your username/password are wrong. You may want to double-check your credentials as part of your debugging process. original error: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none keyboard-interactive], no supported methods remain
==> virtualbox-iso: Deregistering and deleting VM...
==> virtualbox-iso: Deleting output directory...
Build 'virtualbox-iso' errored: Packer experienced an authentication error when trying to connect via SSH. This can happen if your username/password are wrong. You may want to double-check your credentials as part of your debugging process. original error: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none keyboard-interactive], no supported methods remain
==> Some builds didn't complete successfully and had errors:
--> virtualbox-iso: Packer experienced an authentication error when trying to connect via SSH. This can happen if your username/password are wrong. You may want to double-check your credentials as part of your debugging process. original error: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none keyboard-interactive], no supported methods remain
==> Builds finished but no artifacts were created.

packer is attempting to use password authentication for ssh, but that was disabled on k3os by #321.
You can get things working again by enabling password authentication in the cloud-config.yml file:
k3os:
  password: rancher
boot_cmd:
  - sed -i -r 's/^(\s*PasswordAuthentication\s*)no(\s*)$/\1yes\2/i' /etc/ssh/sshd_config

Though when you run vagrant up it’s going to add to .ssh/authorized_keys and PasswordAuthentication won’t be needed any more, so perhaps it would be better to have packer use temporary ssh keys as well?

packer is attempting to use password authentication for ssh, but that was disabled on k3os by #321.
You can get things working again by enabling password authentication in the cloud-config.yml file:
k3os:
  password: rancher
boot_cmd:
  - sed -i -r 's/^(\s*PasswordAuthentication\s*)no(\s*)$/\1yes\2/i' /etc/ssh/sshd_config

Though when you run vagrant up it’s going to add to .ssh/authorized_keys and PasswordAuthentication won’t be needed any more, so perhaps it would be better to have packer use temporary ssh keys as well?

sorry about this, good catch @davecardwell

Ran into the same issue with the vsphere-iso builder.  Thx for the example cloud-config. I used it like this in Packer:
"boot_command": [
"rancher",
"",
"sudo k3os install",
"",
"1",
"",
"Y",
"",
"http:///cloud-config.yml",
"",
"Y",
""
]
After that Packer could log in with rancher:rancher and complete the build. Don't forget to disable password auth again.

Can't get this to work. Using @stayfrostnl Packer procedure with the following cloud-config, but PassworkAuthentication stays set to 'no' and Packer fails.
Any hints would be much appreaciated :-)
k3os:
password: topsecret
boot_cmd:
- sed -i -r 's/^(\sPasswordAuthentication\s)no(\s*)$/\1yes\2/i' /etc/ssh/sshd_config
k3s_args:
- server
- "--disable-agent"
- "--no-deploy servicelb"
- "--no-deploy traefik"

token: "MyToken"

@mlgim https://github.com/cwebd/k3os-vagrant provides a packer build using virtualbox.

Ran into the same issue with the vsphere-iso builder. Thx for the example cloud-config. I used it like this in Packer:
"boot_command": [ "rancher", "<enter>", "sudo k3os install", "<enter>", "1", "<enter>", "Y", "<enter>", "http://<serverlocation>/cloud-config.yml", "<enter>", "Y", "<enter>" ]
After that Packer could log in with rancher:rancher and complete the build. Don't forget to disable password auth again.

Also running into issues with vsphere-iso builder. Not sure where my break is. Would you mind sharing your cloud-config.yml file? Wondering if I have a formatting issue.

@icanhazbeer cloud-config.yml is in https://github.com/cwebd/k3os-vagrant/blob/master/packer/cloud-config.yml
The packer build has been updated now so that the vagrant insecure key is inserted so that the ssh configuration does not need to be modified.
If you review:
https://github.com/cwebd/k3os-vagrant/tree/master/packer

thanks, I managed to solve my issue. I had some incorrect formatting, apparently i cant type :P


Fix Unkown_kind

Spotted while hacking on function generation support #270
It triggered my ocd and hence require prompt fixing! :sweat_smile:

Thanks!


jmelowry/add_defender

Yahoo NHL lineups are being generated with 1 defender, however lines should be composed of GGCCWWWDD.
Currently they are composed of GGCCWWWD
Here is the yahoo help page for reference: https://sports.yahoo.com/dailyfantasy/help/nhl/rules

Coverage remained the same at 93.991% when pulling 937604541c64866cb94d75d7f4439c709558ecc1 on jmelowry:patch-1 into 4f5375d7d2fb6b324947ea6d181f3423dadc28a8 on DimaKudosh:master.

Thanks for PR


⚠ ClamAV has degraded performance

In eafe227, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 80 ms

Resolved: ClamAV performance has improved in edaf46a after 5 minutes.


Improve derivation support for recursive ADTs

Fixes #2101

Codecov Report
Patch coverage has no change and project coverage change: -0.04% :warning:

Comparison is base (093756b) 84.91% compared to head (48d42c8) 84.87%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@                Coverage Diff                @@
##           series/0.14.x    #2187      +/-   ##
=================================================
- Coverage          84.91%   84.87%   -0.04%
=================================================
  Files                 76       76
  Lines               2731     2731
  Branches             371      374       +3
=================================================
- Hits                2319     2318       -1
- Misses               412      413       +1

Flag
Coverage Δ

2.12.18
84.74% <ø> (-0.04%)
:arrow_down:

2.13.11
84.85% <ø> (-0.08%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.
see 1 file with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@MartinHH, Looks like even if we get the compiler to generate these codecs at compile time, we will have a runtime issue with trees of sufficient size blowing up the stack.  I pushed some test changes to limit the test size so we can address this later, but on my machine i could upwards of 10,000 depth fine... in CI that number is much smaller, making me somewhat concerned about the real world limits.
As it stands right now, users will be forced to do something other than derivation, which i think is better than merging this right now and possibly getting a stack overflow at runtime.
If we still wanted to get this in, we need to change the way the encoder/decoder works to be more like Defer/Fix https://github.com/circe/circe/pull/2123 or some other trick to trampoline on the stack rather than blow it up.

As it stands right now, users will be forced to do something other than derivation, which i think is better than merging this right now and possibly getting a stack overflow at runtime.

Not trying to talk you into merging this (I really don't mind as I wasn't planning to use this), but it might be worth pointing out that the following already compiles (resulting in a similarly unsafe codec):
sealed trait Tree derives Codec.AsObject
case class Branch(l: Tree, r: Tree) extends Tree
case object Leaf extends Tree

On the one hand, I cannot wait for this aspect of circe in Scala 3 be improved (and I am sad with the lack of activity for almost 2 months), but I think this is the right place to point out a weird peculiarity which may be worth considering before merging anything.
@MartinHH mentioned, that derivation for his Tree structure compiles. While Martin is right, that is the following compiles:
import io.circe.Codec

sealed trait Tree
case class Branch(l: Tree, r: Tree) extends Tree
case object Leaf extends Tree
given Codec[Tree] = Codec.AsObject.derived[Tree]

the following doesn't:
import io.circe.Codec

sealed trait Tree
case class Branch(l: Tree, r: Tree) extends Tree
case object Leaf extends Tree
case class TreeWrapper(tree: Tree)
given Codec[TreeWrapper] = Codec.AsObject.derived[TreeWrapper]

The latter piece of code fails to compile with the infamous Maximal number of successive inlines (32) exceeded error. And the only thing that changed was moving from deriving for a recursive type to deriving for a type which contained a recursive type.


Fix commit status in repo list

Before:

TestOrg/testactions does have commit status, but won't display in All
After:

Same to #26179.

ps: Can not understand why SetSessionPagination does not check IsListAll.

ps: Can not understand why SetSessionPagination does not check IsListAll.

It's legacy code. I think we need to refacotr all of them.


Update AS OF SYSTEM TIME

AS OF SYSTEM TIME can now use some more complex expressions to compute the desired timestamp
Closes #3455.

This change is

http://cockroach-docs-review.s3-website-us-east-1.amazonaws.com/33311bdd4af87601a8258ea724c7a3137edbe235/

one example: we highly recommend that you run backups 10 seconds in the past to avoid interfering with production traffic and this is perfect for that

I don't think you can have expressions like now() or any other time function in the AOST clause. I think that we should just close this PR and issue because this change doesn't need to be documented beyond the release note. I can't come up with a good example of why a user would care about this. We made the change for internal code reasons, not any user-facing need.

Ok, thanks for info. Will close the issue / PR!


Extensions that add columns to Project and Namespaces list break styling on empty rows

Setup

Rancher version: Discovered in 2.9.0 dev build

Describe the bug

Adding kubewarden extension results in an additional column in the project and namespace list
This breaks the styling on the right of the `There are no namespaces defined

To Reproduce

Add kube warden extension
Nav to Cluster --> Projects/Namespaces

Result

row background for row in question stops before hitting table boundary

Expected Result

row background for row in question continues to table boundary

Screenshots

Additional context

Broken due to shell/components/ExplorerProjectsNamespaces.vue colspan for empty row being 5 which does not include extra column. Looks like other lists with templates that cover the row use fullColspan. Need to investigate if adding this (and ensuring slot has it) fixes the issue.

cc @jordojordo

This needs manual validation. Follow-up task created for automation https://github.com/rancher/qa-tasks/issues/1588

The extensions are not yet ready for Rancher 2.10 using the repo: https://github.com/rancher/ui-plugin-charts

Kubernetes 1.31 is not supported
The extensions version is restricted to below 2.10

Missing the annotation [catalog.cattle.io/ui-extensions-version](http://catalog.cattle.io/ui-extensions-version) which implies incompatibility with Rancher 2.10 and onwards

I will re-visit this once the official extensions support 2.10.

Moving back to test as per Dev input about example extensions that meet the requirements for validation.

Extensions for Vue3 is targeted for 2.10.1.

@gaktive Only the bump to Rancher Prime Extensions are 2.10.1, support for vue3 and extensions is still 2.10.0. Though this doesn't need to be a 2.10.0, the code has been merged there so we should be testing

Testing will resume here for v2.10.0 when/if the extensions are installable for that build or there are sample extensions that meets the requirements for validation which isn't the case for this issue as per offline conversation.

Can we label this with status/dev-validate and bring back in to 2.10.0?

We were able to repro and resolve this issue with a version of Kubewarden that was in development. @jordojordo have any updated Vue3 compatible versions of Kubewarden been published since we last worked on this issue?

have any Vue3 compatible versions of Kubewarden been published since we last worked on this issue?
Not yet, I'm waiting on a GA version of @rancher/shell to be published.

Validated on Rancher version: v2.10-fcb4b0e12dd94e908dcc39c4f941a660f22e5d51-head
UI version: master 1ed167d
Custom Kubewarden plugin version provided by Dev: 3.0.0-rc.1
The colspan is now increased as expected and the background behavior meets the expected result.


[BUG] doChallengeBoss is broken

Describe the bug
Should battle, but no :/
Stuck on boss level
Expected behavior
Multiple battles.
Screenshots/Logs

Desktop (please complete the following information):

OS: W10

Smartphone (please complete the following information):

Emulator: Bluestacks
Version 5.7...

Additional information
https://github.com/zebscripts/AFK-Daily/blob/599949ade4e4fe54a800aa6c9732942e0225abab/afk-daily.sh#L663

This is a weird one because it works for me and I can't really reproduce it. I have double fights, so it's not because it's multibattle or anything, at least in theory.
But I do see some inconsistencies here and there, that I'm, also unsure how to fix because I don't even know why they happen in the first place. I've edited the coords to check the big "VS" letters on the top there, maybe that'll help out a bit. I'm pretty sure no matter what, those letters will be there (which is good for consistency).

Closed for now, but feel free to re-open in case it's still happening.


OCM-5027 | feat: Added support for KubeletConfig resource

This PR adds support for the KubeletConfig resource introduced as part of XCMSTRAT-110. Currently this allows customers to modify the podPidsLimit field of the KubeletConfig for all worker/infra nodes on their ROSA / OSD Classic clusters.
By default, we allow a range of 4,096 -> 16,384 for the value of the podPidsLimit field on KubeletConfig. However, for some customers they are allowed to exceed this default range if they have a pre-agreed capability with Red Hat. For those customers, the supported range is 4,096 -> 3694303 (this is referred to as the "unsafe" range).
As validation happens in offline mode in Terraform, we can only provide a warning to a customer if they request a value in the "unsafe" range e.g > 16,384. The OCM backend enforces the validation, and the terraform apply will fail if a customer is requesting a podPidsLimit that is not supported for their organisation.
KubeletConfig is not currently supported on HCP clusters.

/ok-to-test

/retest

/retest-required

/test rosa-sts-private-critical-high-presubmit
/test rosa-sts-advanced-critical-high-presubmit

/retest

/approve
/lgtm


Reaction landing page

Things that are (currently) not handled by the template:

Various fields from Compound, ProductCompound, and ProductMeasurement

Hence the "Show" button to see the full record

CrudeComponents
More complicated workup fields
Observation images

Example: test.zip

Updates:

Added a top-level reaction summary using the compact representation from the search page
Removed some sections when they are empty: setup, conditions, notes, observations, workups


Update dependency for angular 1.5

So far I can see, in my scenario it works fine.

PR?

Sure: https://github.com/danielstern/ngAudio/pull/127

Solved #127


Updating a DID document

Hello,
I was trying to play around with this library to create and updating a DID and stumbled upon some issues:

Example to instantiate a new DID object and creating the DID document uses the full DID document's content - so far this makes sense for the create usecase. What is/should be the procedure to instantiating a DID object for an existing DID? E.g. the DID I created yesterday? new ION.DID('did:ion:XXX') does not work as a content array is expected.

How to update an existing DID document using this repository? I failed to find in the documentation how to apply the private key to sign off the update operation. Tried to check your lib's code and used option.signer to provide the key and the resulting update operation was signed, send, and accepted by the ION node (beta.ion.msidentity). But I do not see the changes in the DID document applied (but I can't find a spot on how to check about submitted operations state).
Is there a working example on how to update? The readme does not state the source of private key (only new ION.DID({...}) )

Thanks in advance

Did you manage to update an existing DID?


How to mock generateGuid for snapshot tests

Version
4.1.2
Here's what went wrong:
I'm attempting to do snapshot testing of a component that uses react-picky, but the snapshots
fail to match each time they're run, because generateGuid in lib/utils is generating a new guid
in parts of the markup.  Do you know how to mock this behavior in jest or could you provide a way to get around it?

Hi @mkelley33!
You have to mock utils module. Try to use it before first describe() method. I don't know about better solution. I also tried to create __mocks__ directory (see manual mocks) with mocked module and inherit all other methods, but it wasn't successful.
Picky.test.js (before first describe()):
jest.mock('../src/lib/utils', () => ({
  ...jest.requireActual('../src/lib/utils'),
  generateGuid: jest.fn().mockReturnValue('42'),
}));

You have to check that value with
const wrapper = mount(
  <Picky
    value={1}
    options={[1, 2, 3, 4, 5]}
    open={true}
  />
);
expect(wrapper.state('id')).toEqual('42');

Currently it works for all tests, but I don't know is it needed to put it to all tests or not.
Let me know please if I can help for you.

Thanks @Vadimkin
Picky could probably worth without the generated ID, it was just for the ARIA stuff, which I know embarrassingly little about. There is perhaps a more elegant way to deal with it, maybe using refs. 🤷♂

👍 thank you!


Remove static import of methods

Option 4 for avoiding static import of methods

seems like this PR is outdated.
closing for now. feel free to reopen.


Gradle 2.9 org.gradle.internal.classloader.CachingClassLoader.addURL() disabled

Expected Behavior
Gradle 2.9, org.gradle.internal.classloader.CachingClassLoader.addURL() disabled
Current Behavior
Exception in task dependencies: No signature of method: org.gradle.internal.classloader.CachingClassLoader.addURL() is applicable for argument types: (java.net.URL)
Context
Connected to the target VM, address: '<IP_ADDRESS>:51258', transport: 'socket'
===> Exception in task dependencies: No signature of method: org.gradle.internal.classloader.CachingClassLoader.addURL() is applicable for argument types: (java.net.URL) values: [file:/D:/Program%20Files/Git_repo/vayhapi/vayhapi-web/build/classes/main/]
Possible solutions: dump()

Sorry that you're having trouble with Gradle!
Your issue lacks information about how to reproduce the problem you're having. A minimal reproducible example can really help us track down and fix your problem. We may also be able to suggest workarounds or ways to avoid the problem if we can reproduce it.
We request you to provide a reproducer because we need to be sure we are looking at the exact problem you are facing.
The ideal way to provide a reproducer is to leverage our reproducer template.
This issue will be closed after 7 days unless you can provide more information.


GLTF Loader non-cached HTTP requests

Context
While debugging the Linux Dead lock, I logged all the xmlHTTPRequest and found the same requests for some .gltf. Textures are loaded multiple times (requests with related allocations). Loaded but not uploaded to VRAM as there is a cache mecanism but too late in the loading pipeline.
Pipeline
Textures found in the .gltf are loaded async in a buffer:
https://github.com/BabylonJS/Babylon.js/blob/af28ff68fcf6f782f8dec7f0494d5fced8a2948f/loaders/src/glTF/2.0/glTFLoader.ts#L2020
and
https://github.com/BabylonJS/Babylon.js/blob/af28ff68fcf6f782f8dec7f0494d5fced8a2948f/loaders/src/glTF/2.0/glTFLoader.ts#L2096
Then, this buffer is sent to the engine to create a texture with. But a texture cache discards that buffer if the URL is already present:
https://github.com/BabylonJS/Babylon.js/blob/b8238023e50655c58361136a88fcd7e88f06772d/src/Materials/Textures/texture.ts#L371
So, some buffers are discarded but the request and loading is done in NativeEngine. I guess there is a cache mecanism in the browser to overcome this kind of issue.  Maybe also the OS. But memory allocation and latency is a performance issue. Let alone bandwidth quota on mobile.
Fix proposal

Add a cache memory in xmlHTTPRequest with a configurable memory size
Add a cache in GLTFLoader to not load multiple times the same URL (I prefer this option as babylonjs will benefit as well)

ping @ryantrem

You can observe this behavior with GLTF Node NegativeScale (0) test scene :
https://www.babylonjs-playground.com/#DS8AA7#19


Automatic column detection

Hey! I thought about implementing automatic column detection to your library. It would be cool to pass a large csv without specifying each columns name. Tell me if you're interested in me implementing this functionality.

Hey @wojciechAdaszynski!
That's a neat idea! a-csv already has the headers option which parses the headers, so, maybe you can use that one here. :smile:
Contributions are welcome! :sparkle:


Test and correct / update RADseq de novo tutorial

I just finished a first pass on the entire tuto. If it's possible to look at it. Whishing you a good day!


File missing docs\images\subProgramComment.png

Describe the bug
A file docs\images\subProgramComment.png is missing causing a 404.
To Reproduce
Steps to reproduce the behavior:

Click the Link https://github.com/isg-stuttgart/vscode-isg-cnc/blob/main/docs\images\subProgramComment.png

Expected behavior
Show the picture instead of a 404.
Screenshots

Desktop (please complete the following information):

OS: Archlinux
Browser: Firefox
Version: 132.0.2

Additional context
Why is any information about my phone needed?

Closed with https://github.com/isg-stuttgart/vscode-isg-cnc/pull/166

@andreassteding-isg Thanks for reporting. The Problem is fixed and the picture will be visible on the next release.


Reaperscanstr closed

Source name
Reaper Scans
Source language
Turkish
Source link
https://reaperscanstr.com/
Other details
No response
Acknowledgements

[X] I have updated all installed extensions.
[X] I have opened WebView and checked that the source website is down.
[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a meaningful title with the source name.
[X] I will fill out all of the requested information in this form.

The site is not dead, please close the issues

The site is not dead, please close the issues

Reaper Scans (Turkish) is definitely dead though...


[Visibility Request] Page 11309 General Ledger Entries

Please make the following variables protected:

TempGLEntryBuf
ShowTotalAppliedAmount
IncludeEntryFilter

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.

In which version is this going to be shipped? Seems not to be in 16.4. Thanks in advanced.


problems after upgrade to 3.0

Hello in the documentation I read that the constructor that previously accepted two strings is now:
var fcm = new FcmSender (settings, httpClient);
it is not clear to me how these objects should be initialized
Old Code:
using (var fcm = new FcmSender(config["ServerKey"], config["SenderId"]))
{
	await fcm.SendAsync(deviceId, notification);
}

New Code:
FcmSettings sett = new FcmSettings();
sett.SenderId = config["SenderId"];
sett.SenderId = config["ServerKey"];
FcmSender fcm = new FcmSender(sett, new System.Net.Http.HttpClient());

I don't know if that's correct.
The using clause cannot be used because the object is not disposable.
What is the httpclient object for and how should it be instantiated?

@78fede78  This is all correct. One thing that you might want to do is wrap your HttpClient with using statement e.g.
using var http = new HttpClient();
var fcm = new FcmSender(settings, http);
await fcm.SendAsync(deviceId, notification);


V6 - Incorrect query when using '<Ref | Collections>.*'

Describe the bug
The generated queries from a find with a where.fields  using '.*'  to not populate all the fields of the relations which seems to be wrong.
To Reproduce
Steps to reproduce the behavior:

create the following two entities:

@Entity()
class A {
  @PrimaryKey()
  id: number;

  @Property({
    type: "text",
  })
  description: string;

  @ManyToOne(() => B)
  b: Ref<B>;
}

@Entity()
class B {
  @PrimaryKey()
  id: number;

  @Property({
    type: "text",
  })
  name: string;
}

do the following query

    const result = await this.em.findAll(A, {
      fields: ["b.*"],
      logging: { enabled: true }
    });

here is the generated query and received object

SQL:
select "s0"."id", "s0"."b_id" from "a_entity" as "s0"

received object:
{
  id: "..."
  b: {
    id: "..."
  }
}

Expected behavior
I think that the following query should be generated if i understand the api correctly:
select "s0".*, "h1"."id" as "b_id" from "b_entity" as "s0" left join "a_entity" as "h1" on "s0"."id" = "h1"."b_id"

Additional context
Also when i am populating the relation and try to get all fields like the following:
    const result = await this.em.findAll(A, {
      populate: ["b"],
      fields: ["*"],
      logging: { enabled: true }
    });

Here is the response:
{
  id: "....",
  b: {
    id: "..."
  }
}

Here is what I expected:
{
  id: "....",
  description: "..."
  b: {
    id: "...",
    name: "..."
  }
}

This issue may have the same origin as the one presented earlier.
Versions

Dependency
Version

node
18

typescript
5.0.4

mikro-orm
6.0.0-dev.82

your-driver
postgresql

I think that the following query should be generated if i understand the api correctly:

You need to enable joined strategy for that, by default it should issue two queries, one for each relation.

Also you are missing ref: true in your entity definition (unless you are using ts-morph).

I think that the following query should be generated if i understand the api correctly:

You need to enable joined strategy for that, by default it should issue two queries, one for each relation.
edit: or is that the second query only? why do you expect the join in there?

It seems to only generate one query without the joined strategy enabled.
I forgot that the join strategy is not the default, I didn't express myself correctly my concern is only about the received object that doesn't contains all fields.
I would expect to receive the following array:
[
  {
    id: "..."
    b: {
      id: "...",
      name: "..."
    }
  },
  ...
]

I am sry I have no time left tonight I didn't expect that you reply that fast, ty by the way !
I will investigate in more detail tomorrow based on the comments you told me.

It seems to only generate one query without the joined strategy enabled.

This is actually a bug in the implementation of local logging config, it's apparently not propagated in the populate calls. If you enable global logging, you will see two queries, and they seem to be correct to me:
select `a0`.`id`, `a0`.`b_id` from `a` as `a0`
select `b0`.`id`, `b0`.* from `b` as `b0` where `b0`.`id` in (1, 2) order by `b0`.`id` asc

But when I check the serialized form, I am also not getting the name properties there, so that's probably what you mean. Next time please distinguish the entity on runtime and what it serializes to.
Here is a repro:
import { Entity, Ref, LoadStrategy, ManyToOne, MikroORM, PrimaryKey, Property, wrap } from '@mikro-orm/sqlite';

@Entity()
class A {

  @PrimaryKey()
  id!: number;

  @Property()
  description!: string;

  @ManyToOne(() => B, { ref: true })
  b!: Ref<B>;

}

@Entity()
class B {

  @PrimaryKey()
  id!: number;

  @Property()
  name!: string;

}

let orm: MikroORM;

beforeAll(async () => {
  orm = await MikroORM.init({
    dbName: ':memory:',
    entities: [A, B],
    debug: true,
  });
  await orm.schema.createSchema();

  const b1 = orm.em.create(B, { name: 'b1' }, { persist: false });
  const b2 = orm.em.create(B, { name: 'b2' }, { persist: false });
  await orm.em.insertMany([b1, b2]);
  await orm.em.insertMany(A, [{ b: b1.id, description: 'd1' }, { b: b2.id, description: 'd2' }]);
});

afterAll(async () => {
  await orm.close(true);
});

test(`GH issue 4433 (select-in)`, async () => {
  const result = await orm.em.fork().find(A, {}, {
    fields: ['b.*'],
    // logging: { enabled: true },
    strategy: LoadStrategy.SELECT_IN,
  });
  console.log(result[0].b); // it has `name`
  expect(result.map(r => wrap(r).toObject())).toEqual([
    {
      b: { id: 1, name: 'b1' }, // but it's not in the serialized form, this fails
      id: 1,
    },
    {
      b: { id: 2, name: 'b2' },
      id: 2,
    },
  ]);
});

test(`GH issue 4433 (joined)`, async () => {
  const result = await orm.em.fork().find(A, {}, {
    fields: ['b.*'],
    // logging: { enabled: true },
    strategy: LoadStrategy.JOINED,
  });
  console.log(result);
});

I can also see the query is wrong with joined strategy - it actually uses two queries:
select `a0`.`id`, `a0`.`b_id`, `a0`.* from `a` as `a0` left join `b` as `b1` on `a0`.`b_id` = `b1`.`id`
select `b0`.`id`, `b0`.* from `b` as `b0` where `b0`.`id` in (1, 2) order by `b0`.`id` asc

Logging propagation is fixed here:
https://github.com/mikro-orm/mikro-orm/commit/0d22f7e7af378bb7987a0e6101ba187456f95256
I think the issue with nested partial loading of * is there in v5 already, will see if I can address it there.

But when I check the serialized form, I am also not getting the name properties there, so that's probably what you mean.

Yes that's what I meant.

Next time please distinguish the entity on runtime and what it serializes to.

Yes, sure.
I was in a rush and got confused. I apologize.

Logging propagation is fixed here:
0d22f7e

Thanks.

I think the issue with nested partial loading of * is there in v5 already, will see if I can address it there.
edit: actually the test is passing in v5, although the joined strategy partial loading is still broken, will keep this for v6 probably

Ok good news !

Closing as fixed via 61de246


Lines overflowing

Hey there @anseki  hope you are doing fine
I am using leader line in vuejs environment
the issue i am facing is my leaderlines get overflown if they are in a container that has scrollable its like they exist and passover everywhere

https://www.awesomescreenshot.com/video/8474666?key=5d128a422ea293f005288483990e5599
video reference

Hi @DAC696, thank you for the comment.
Sorry, my English is poor.
Do you mean that you want to hide the line when it is moved to outside of the scrollable element?
If so, this may help you:
https://github.com/anseki/leader-line/issues/54#issuecomment-499513836

No reply came, then this abandoned issue is closed.


Ratings in Recipes

For google star ratings, we would like to have them embedded in the Recipe schema. Is there a way to do that so that the recipes/ratings show together in the google rich snippets? Right now recipes show without star ratings.

The plugin doesn't currently have the ability to nest a Review within a Recipe, although the spec does support it. We'll flag this for a future release.

Awesome. I've done a pretty comprehensive search on the web, and nobody seems to do this feature well for recipes. This plugin is definitely one of the best for food bloggers already and I think this one feature would make a huge difference to food bloggers and really make this plugin stand out. Right now, to get this one feature, it forces people to get a foodie-specific theme with this functionality baked in.
Thanks so much.

Looking for this same option. Would love to have review capability for recipes.


Solr query permissions on PSU Only.

Solr search needs to return results for works and collections that are PSU only visibility, the content would not be accessible, but the metadata is.
WDLL

Add an ACL for public discovery access to Permissions#grant_authorized_access
Ensure that the ACL is removed with Permissions#revoke_authorized_access
Update WorkPolicy#show?
Update discovery_spec tests to ensure everything works correctly

Blocked by #293


feat: add support for styles debugging

to be introduced

Warnings

:warning:
There are no updates provided to CHANGELOG. Ensure there are no publicly visible changes introduced by this PR.

  Generated by :no_entry_sign: dangerJS


feat(update): Include a Locking message

What does this PR try to resolve?
This extends #13561 to cargo update.  I previously left it out because the locking message was redundant.  However the Locking message has been extended in #13754 to include the resolving policy which can be a useful point of interest (e.g. "why does cargo update do nothing? Oh, -Zminimal-versions is enabled").
I still left out the message for --precise because the user is overriding all of that.
I'd still like to extend all of this to cargo install (and maybe all resolves) but that is taking more investigation.
How should we test and review this PR?
Additional information

Thanks for the clarification!
@bors r+

:pushpin: Commit bec36fce99c5ea739828f98d9a43adf760c6243b has been approved by weihanglo
It is now in the queue for this repository.

:hourglass: Testing commit bec36fce99c5ea739828f98d9a43adf760c6243b with merge b9d913e532df8bb6e14a507376d9fbbcacdcc0b0...

:sunny: Test successful - checks-actions
Approved by: weihanglo
Pushing b9d913e532df8bb6e14a507376d9fbbcacdcc0b0 to master...


add lastPriceToken

Closes #447

Seems good, just the tests that need fixing I think

👍  can't give a review since i created it , so you have to approve it

good to go, but please do not have a release now
we will merge more breaking changes in one release


Rename variable names

See https://github.com/timble/illusion/blob/master/scss/tools/mixins/_spacing.scss#L5 and more
$defaultProperties > $defaults
$amount > $multiplier

Fixed in 2.0.0


Add support for language selection in preferences

Fixes #1001

Description
Add support for language selection in preferences.
This was a bit trickier than expected, namely the configuration of the language needs to be stored in a separate file, because we cannot use CONF at the stage the _ function is called.
I added a langconfig in the .spyder folder, which stores the language value ('en', 'fr' etc)
With this change I had to implement some new logic in the config dialog, so now comboboxes (so far) have an extra parameter call restart  (by default False).
Preferences dialog

Pop up message box
When the user changes one or more options flagged with restart=True and then clicks on Apply or Ok, the a message box will popup informing the user which options need to restart and asking if he/she wants to restart right away.

If the user accepts, any restart specific (defined) settings will be applied and spyder will restart automatically.
Note:
The message box is now available for any other combobox that is created with the restart=True parameter. Adding this functionality to other widgets will be left for another PR.
Todo

[x] Display language names In original Language
[x] Agree on additional file method and name
[x] Define when to pop the box to the user for restart

For another PR

Add the restart functionality to all the other widgets besides comboboxes

@ccordoba12 I updated with the fixes, when you have some time can you take a look at some questions I left in the description?

Can @spyder-ide/core-developers give this a try and give some comments?
git clone https://github.com/goanpeca/spyder.git spyder-lang
cd spyder-lang
git checkout -b language
git pull origin language
python bootstrap.py

This PR is waiting for some input from the @spyder-ide/core-developers ...
There are 2 ways of doing this.
1.) Ask the user if he wants to restart for the setting to take place, immediately after changing the setting
2.) Ask the user if he wants to restart for the setting to take place, only after pressing Ok or Apply
I implemented the first one.. but the second one could be also implemented if you consider is better.
@ccordoba12 any ideas, thoughts?

I vote 2 because:

there could be other settings which ask to reload (changing the icon theme, enabling plugins…)
It's annoying to have a popup without being able to check if the change was correct.

If 2 is chosen, it should be written that it requires a restart.

In that case of 2, I would make the popup appear at the end when clicking ok, or apply

Yep. Bonus points if the dialog contains the reason why the restart is needed (as a list so that other settings can give their information too)

Yep, you read my mind ;-)

@Nodd ok, this is now updated to ask only after pressing Ok or Apply and displays a list of all the options that require a restart (and were changed).
Can you give it a try @ccordoba12?
I would expect this to work in mac without changes but, @blink1073 can you give it a try?

The restart popup message is very nice!! It looks very professional! Thanks a lot for working on it @goanpeca!

Glad you like it, if this works on Mac, then it should be ready to merge

Everything seems fine to me now, but:

I still find the implementation too complex to follow.
We still need to test this PR on Mac :-)

Regarding 1.) hmmm
I tried to add all the needed documentation to understand. I will explain and you tell me where you find it confusing.
When someone wants to add a new translation they will create the folder in locale and put there the new translation, or variation of a translation.
Once they do this and run the code they will get an error because we need to have an updated language name (in native language) added to LANGUAGE_CODES. If they added a new language, no problem they just add i.e. 'de': 'Deutsch'... if it is a variation they just add i.e. 'pt_PT': 'Portuguese (Portugal)' and make sure the other variation (if existent) should be then 'pt_BR': 'Portuguese (Brazil)'
But the truth is, that the second case will most likely never happen...

I'll merge this tomorrow, after testing on Mac :-)

I will have to buy a Mac... ;-)

I can test today on my Mac if you prefer @ccordoba12.

That'd be great too @blink1073! I thought you weren't around ;-)

Was at the beach with the little ones.  ;-)

Una problema:
Cargando el inspector de objetos...
Traceback (most recent call last):
  File "/Users/silvester/workspace/spyder/spyderlib/spyder.py", line 3077, in main
    mainwindow = run_spyder(app, options, args)
  File "/Users/silvester/workspace/spyder/spyderlib/spyder.py", line 2963, in run_spyder
    main.setup()
  File "/Users/silvester/workspace/spyder/spyderlib/spyder.py", line 851, in setup
    self.set_splash(_("Loading outline explorer..."))
  File "/Users/silvester/workspace/spyder/spyderlib/spyder.py", line 2095, in set_splash
    self.debug_print(message)
  File "/Users/silvester/workspace/spyder/spyderlib/spyder.py", line 532, in debug_print
    debug_print(message)
  File "/Users/silvester/workspace/spyder/spyderlib/baseconfig.py", line 56, in debug_print
    print(*message, file=ss)
UnicodeEncodeError: 'ascii' codec can't encode character '\xf3' in position 27: ordinal not in range(128)

It looks like sys.stdout is encoded as ascii, thanks Apple...

Any suggestions?

Looking into it now.

Hmm, this works fine from an IPython prompt:  print('€', file=sys.stdout)

It might have something to do with the restart itself affecting sys.stdout.

If you say No when the popup box asks to restart... and then you close spyder...
and rerun from bootstrap...?
You get the same error?

If I restart after the failure itself it comes up fine.

If you run without DEBUG ?

Actually if I say NO then it restarts anyway...

hmm thats a bug then.. it should do that!

Oops, forget that last comment.

Can you run without Debug? and check again?

Running without debug is fine, and I can print unicode characters from the Spyder Python console.

It appears to just be the restart while in debug mode.

I tried using sys.stdout within debug_print, instead of the global variable, but it still did not work.

Can you try this...
Change baseconfig.py  line 52 with this
def debug_print(*message):
    """Output debug messages to stdout"""
    if DEBUG:
        ss = STDOUT
        message = tuple([m.encode('utf-8') for m in message])
        print(*message, file=ss)

And try with debug again?

That works but all of the strings print with b''

E.g.: b'Configuration de la fen\xc3\xaatre principale...'

and if you do this
def debug_print(*message):
    """Output debug messages to stdout"""
    if DEBUG:
        ss = STDOUT
        message = tuple([to_text_string(m) for m in message])
        print(*message, file=ss)

Nope, same UnicodeEncodeError.

Yep, I'll try Py2.

Same problem there.

If I turn DEBUG off, do the restart, and then try and print from with the Spyder Internal Console, I get the same error too.  The other consoles are note affected.

yes... the internal console seems to be the problem...
But that is something unrelated to the changes I did, seems to be a problem that has been sleeping for some time. I am trying some options

Yes, it appears to be related to the restart itself.

Yup, confirmed.  If I start in English mode and do just a restart, we get this behavior.

I will try linux.. one sec

Ok, I can reproduce it, ...fixing!! give me 10 mins

@blink1073
Can you add this line to spyderlib\spyder.py after line 2737
env['PYTHONIOENCODING'] = 'utf-8'

Perfecto!

both py2 and py3?

Oui.

debug_print is super nice, I use it all the time.

This env variable will change the behavior in scripts run from spyder ? This is bad, we had many bug reports because of this. We should change the encoding of the internal console only. I think I have a solution somewhere, I'll try to find it.

@Nodd I do not think it will be the case... but you can try with an example?

I can erase the environment variable in the restart_app.py just to be sure... so it is only used for the python process that spawns the new Spyder instance.

Maybe I can pop that variable in spyderlib\spyder.py so that it is not there after startup. It is needed when spawning the new Spyder process on restart, but should not be needed afterwards.

But what if the user had explicitly set that variable to something else?  Perhaps only set it if it is not in os.environ and not set to something else, then leave it there.

utf-8 all the things!

Well, if the user set it to soemthing else, the spyder will break...

Then I say set it and leave it.

Lets see what @Nodd brings :-)

I am good with this. Will use that dialog on restarting spyder for the icons as soon as this is merged.

@Nodd did you have some time to find what you were looking for? or
@ccordoba12  any comments on the fix needed to get this PR working properly, specifically this last commit

I used this in some scritps so that argparse help doesn't raise a UnicodeSomethingError when displaying a help message with utf8 symbols under python2:
import codecs
sys.stdout = codecs.getwriter('utf-8')(sys.stdout)

Since the internal console is basically sys.stdout, it might be used as-is.

and in python 3 ?

where should I put this code?

In python3 I didn't have this problem (the unicode support is way better).
It should be put at the same place where sys.stdout is redirected to the internal console, I guess. I have no idea where it is :disappointed:

If I'm not wrong, redirection of stdio for the internal console happens in redirect_internalshell_stdio, currently present in
https://github.com/spyder-ide/spyder/blob/master/spyderlib/spyder.py#L2435

I tried putting @Nodd  where @ccordoba12 without success

@Nodd I think I have it fixed, can you check the latest commit?
@blink1073, or @ccordoba12  can you please test on Mac? remember to run using
python bootstrap.py --debug

It works for me, other than the fact that the restart dialog was entirely in English.

Cause we need To rerun the localización scripts :+1:

@ccordoba12 can we merge this one so that @SylvainCorlay can add this to his icon PR :smile: ?

:+1:

Yeah seems I will need to rebase again anyway!

Is there any blocking point for this one?

@ccordoba12 this one should be ready to merge as well

:+1:

Merging :-) @ccordoba12?

Yes!! I'm having a problem with Qt5 but it seems to be with my system rather than with this PR (because I'm also having problems in master :-)

Thanks for your patience by the way ;-)

No prob ;-)
Ok, @SylvainCorlay, now is your turn :-)
Just add restart=True to the combobox, i.e.
        language_combo = self.create_combobox(_('Language'), choices,
                                              'interface_language',
                                              restart=True)


Adds a check to skip serialization-deserialization if request is for same node

Description

Category:
Bug Fix

Why these changes are required?
To improve performance of large requests handled on same node

Issues Resolved

Resolves #2724

Testing
Current test suite should pass.
Check List

[ ] New functionality includes testing
[ ] New functionality has been documented
[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

CI is currently failing with following to change introduced in core: https://github.com/opensearch-project/OpenSearch/pull/7465
 java.lang.NoClassDefFoundError: org/opensearch/core/common/io/stream/BaseStreamInput

CI is currently failing with following to change introduced in core: opensearch-project/OpenSearch#7465
 java.lang.NoClassDefFoundError: org/opensearch/core/common/io/stream/BaseStreamInput

Seems like a stale artifact issue that will be fixed on its own.

@nibix After updating GuiceHolder to contains an instance of TransportService and use that instance to fetch the localNode information, I'm getting different results:
cs.localNode() = {DiscoveryNode@12410} "{node_utest_n1_fnull_t353609252151333_num3}{cHyj4v__T_-MMs2s_____w}{_Rk3Y5VWTUeuzUrwCfr9Rg}{<IP_ADDRESS>}{<IP_ADDRESS>:6212}{m}{shard_indexing_pressure_enabled=true}"

connection.getNode() = {DiscoveryNode@12410} "{node_utest_n1_fnull_t353609252151333_num3}{cHyj4v__T_-MMs2s_____w}{_Rk3Y5VWTUeuzUrwCfr9Rg}{<IP_ADDRESS>}{<IP_ADDRESS>:6212}{m}{shard_indexing_pressure_enabled=true}"

localNode = {DiscoveryNode@12404} "{node_utest_n1_fnull_t353609252151333_num1}{1S3HEQAAQACnOySu_____w}{LUJGxVdsSoqtJ3kB3RZ4wg}{<IP_ADDRESS>}{<IP_ADDRESS>:7352}{d}{shard_indexing_pressure_enabled=true}"

the localNode value from ClusterState seems to be matching the value of the node the connection originated from (resulting in passing tests), but localNode value (retrieved from TransportService instance) is not matching connection.getNode() resulting in test failures.
Am i missing something here in terms of fetching localNode info from TransportService?

@nibix After updating GuiceHolder to contain an instance of TransportService and use that instance to fetch the localNode information, I'm getting different results

Oh, hm, interesting. I think I have an idea where this is coming from. Am I right in the assumption that you get that error when testing in a junit based test with a cluster embedded in the JVM?
If yes: This indeed collides with the GuiceHolder. Unfortunately, the GuiceHolder is at closer consideration a quite dirty hack to get ahold of dependencies which area only available by deprndecy injection. It uses for this a static attribute in the SecurityPlugin class. While in an JVM embedded cluster the nodes and also the instances of the SecurityPlugin can coexist very well together, this static class is a collision. The different nodes write in an undetermined fashion references to their own TransportService into it.
That's why you see other results there, at least if my assumption is correct.
Of course the more important part of the consideration would be "how can this be fixed". To be honest I also don't know at the moment. I will have to look at the code and think about it for a while. 😔

Oh, hm, interesting. I think I have an idea where this is coming from. Am I right in the assumption that you get that error when testing in a junit based test with a cluster embedded in the JVM?

Yes, I saw this while running one of the failing unit tests.

If yes: This indeed collides with the GuiceHolder. Unfortunately, the GuiceHolder is at closer consideration a quite dirty hack to get ahold of dependencies which area only available by deprndecy injection. It uses for this a static attribute in the SecurityPlugin class. While in an JVM embedded cluster the nodes and also the instances of the SecurityPlugin can coexist very well together, this static class is a collision. The different nodes write in an undetermined fashion references to their own TransportService into it.

If that is indeed the case, then do you think it might be worthwhile to look into using cluster state() as an alternative for getting localNode reference, cause that seems to be available at almost all time (except during cluster bootstrap using it Zen Discover mechanism, although I'm not a 100% sure whether the request will flow through security with some cluster state info during bootstrap).

Of course the more important part of the consideration would be "how can this be fixed". To be honest I also don't know at the moment. I will have to look at the code and think about it for a while. 😔

Thank you for helping with this.

If that is indeed the case, then do you think it might be worthwhile to look into using cluster state() as an alternative for getting localNode reference, cause that seems to be available at almost all time (except during cluster bootstrap using it Zen Discover mechanism, although I'm not a 100% sure whether the request will flow through security with some cluster state info during bootstrap).

Yes, I think using cluster state will be the best solution in the current situation.
Still, on a medium term, it might be recommendable to see whether it would be possible to get rid of the GuiceHolder hack and get access to the services provided by this on a better way. This probably needs a discussion with the core team.

@reta @dblock Thoughts? The problem at hand is to fetch localNode info for every transport request. There are two options:

using cluster state to retrieve localNode info. (seems to work all the time, with an AssertionError being thrown at bootstrap because it is expecting ClusterStateApplier as the calling class, but a catch block helps solve the problem)
using transport service via GuiceHolder. (doesn't work all the time)

For more details see the discussion above..

@DarshitChanpura the local node is the one where the plugin is bootstrapped, right? if that's the case, I think the most straightforward way is for security plugin to implement ClusterPlugin as well, which has the hook onNodeStarted(), a well defined place where the local node could be queried for.

@reta SecurityPlugin already implements ClusterPlugin and so I tried to initialize a variable: localNode = cs.localNode() inside onNodeStarted() and then to access it in SecurityInterceptor. However node initialization fails with Unable to start node NPE due to ClusterState cs which suggests that we might have to go through a different route to get localNode info (may not be a straighforward implementation).

However node initialization fails with Unable to start node NPE due to ClusterState cs which suggests that we might have to go through a different route to get localNode info (may not be a straighforward implementation).

This is super surprising,  could you access the local node using TransportService, not ClusterState ?

However node initialization fails with Unable to start node NPE due to ClusterState cs which suggests that we might have to go through a different route to get localNode info (may not be a straighforward implementation).

This is super surprising, could you access the local node using TransportService, not ClusterState ?

I couldn't. This is what I have currently implemented via GuiceHolder but it is not working. The only working solution I could get was via this:
        try {
            isSameNodeRequest = cs.localNode().equals(connection.getNode()); // using DiscoveryNode equals comparison here
        } catch (AssertionError e) {
            // do nothing
            log.info(e);
        }

This is super surprising, could you access the local node using TransportService, not ClusterState ?

Unfortunately, plugins can only get ahold of TransportService via hacks which can cause other trouble. See https://github.com/opensearch-project/security/pull/2765#issuecomment-1561556458 for details.
It would be cool if some more straight-forward possibility could be created to get a reference to TransportService from plugin code.

@reta @nibix How should we proceed w.r.t this PR?

@reta @nibix How should we proceed w.r.t this PR?

I most straightforward option I see is to consider enhancing onNodeStarted hook with DiscoveryNode node  argument - this (to me) looks like a missed contextual information for the hook from the beginning

The most straightforward option I see is to consider enhancing onNodeStarted hook with DiscoveryNode node argument - this (to me) looks like a missed contextual information for the hook from the beginning

This will involve adding a new declaration in ClusterPlugin class in core and modifying onNodeStarted here to pass localNode info:
https://github.com/opensearch-project/OpenSearch/blob/277eb3d59fb9edcae8575577ec135032e3563f2e/server/src/main/java/org/opensearch/node/Node.java#L1345
This might be out of scope for this PR and might need a discussion in core repo. Thoughts @reta ?

@reta I have pushed the changes. Thank you for your valuable guidance!!
This PR's CI is now blocked until https://github.com/opensearch-project/OpenSearch/pull/7919 is merged and an artifact with this change becomes available.

This is good, how are we measure the perf impact of this change?

Great question. I was thinking about taking before and after running times of the test-suite. Any other suggestions are much appreciated.

Could you post those findings on this pull request so we know what to expect if this change is merged?

@DarshitChanpura Can you resolve the conflicts on this PR? I see that the PR in core has been merged. After the conflicts are resolved is this PR good to finish review on?

Yes.

@DarshitChanpura Is the plugin install failure related to the change of the extension point in core?

Task :compileJava
/home/runner/work/security/security/src/main/java/org/opensearch/security/OpenSearchSecurityPlugin.java:1796: error: method does not override or implement a method from a supertype
@Override

@DarshitChanpura What is pending on this PR?

Looking into test-failures now.

@hardik-k-shah @krishna-ggk @parasjain1 Could you please re-review this?

Merging now. Any comments can be addressed in follow-ups.


Support for db resolvers

Hi,
I integrated a stable dbresolver into Bun. And used it in production with replicas. Hence. I was able to unblock db bottle necks and gain very low DB latencies in production.

I was hoping

to bring in support for sql plugins
to implement db resolvers in bun straight up

To summarize our discussion on Slack.

We should change Bun to accept SQLDB interface:

type SQLDB interface {
	Begin() (*sql.Tx, error)
	BeginTx(ctx context.Context, opts *sql.TxOptions) (*sql.Tx, error)
	Close() error
	Conn(ctx context.Context) (*sql.Conn, error)
	Driver() driver.Driver
	Exec(query string, args ...interface{}) (sql.Result, error)
	ExecContext(ctx context.Context, query string, args ...interface{}) (sql.Result, error)
	Ping() error
	PingContext(ctx context.Context) error
	Prepare(query string) (*sql.Stmt, error)
	PrepareContext(ctx context.Context, query string) (*sql.Stmt, error)
	Query(query string, args ...interface{}) (*sql.Rows, error)
	QueryContext(ctx context.Context, query string, args ...interface{}) (*sql.Rows, error)
	QueryRow(query string, args ...interface{}) *sql.Row
	QueryRowContext(ctx context.Context, query string, args ...interface{}) *sql.Row
}

It might require changing dbresolver or forking it.

We might also introduce some load balancing capabilities for Bun queries:

// LoadBalancer returns a database for the query.
type LoadBalancer interface {
    DBForQuery(query bun.Query) (bun.SQLDB, error)
}

db := bun.NewDB(sqldb, dialect, bun.WithLoadBalancer(lb))

// lb balancer should return leader nodes for *bun.InsertQuery and follower nodes
// for *bun.SelectQuery.


ids do not have a stability property.

First, thanks for creating this version of VirtualList. The 'sticky' feature is something I have been wanting.
Now the problem I am facing:
When a scroll list comes from a dynamic database and items are added or removed, a stability property is needed. That is to say for a given item, the loop id should always bind to the same item. From reading the source code, the id right now is item.index which when a database inserts new item, is not stable.
A possible solution:
svelte-virtual-list has the same issue. However I've addressed it in that code very simply and the same could be done here.
First add:
    export let indexing;

and use it in the loop:
        <div class="virtual-list-inner" style={innerStyle}>
		{#each items as item (indexing ? indexing(item.index) : item.index)}
			<slot name="item" style={item.style} index={item.index} />
		{/each}
	</div>

Alternatively, the call to indexing could be done inside the refresh function.

A bit late, but here you are :)


Provide object in IGListSingleSectionController size block

Fix #144

Coverage remained the same at 86.482% when pulling 17a185042f9c904b4e807eda88836aadd45d8560 on zhubofei:single-object into 9e8ad9626ab91326b6bbb75cb6927f8dbe134896 on Instagram:master.

Coverage remained the same at 86.482% when pulling 17a185042f9c904b4e807eda88836aadd45d8560 on zhubofei:single-object into 9e8ad9626ab91326b6bbb75cb6927f8dbe134896 on Instagram:master.

Coverage remained the same at 86.482% when pulling d82b939c623014d723af1a1b7b6584fea9e31373 on zhubofei:single-object into 9e8ad9626ab91326b6bbb75cb6927f8dbe134896 on Instagram:master.


better support for multiple parallelism in SegmentWriter and SegmentUploader default impl

In many use cases such as Spark / Flink ingestion, multiple instances of workers will be instantiated to upload segments in parallel
Currently the default SegmentWriter and SegmentUploader cannot support this easily: One needs to generate slightly different TableConfig and BatchIngestionConfig in order to modify the tmp directory name, segment name and other usages. Otherwise, spawning multiples instances within the same host will cause File write conflict.
Propose to: create a method to easily set parallelism index for SegmentWriter and Uploader; such as adding APIs to modify the parallelism index directly instead of via configuration changes.
CC @npawar, @xiangfu0

this should be closed by #7458


[OpenCTI] Engine attachment processor fail

Describe the bug a clear and concise description of what the bug is.
We deployed OpenCTI using the helm chart, and we tried writing our own deployment files based on the docker variant, but we keep running into the same error following examples and guide. It seems like it has issues with elastichsearch for running the correct engine.
We run OpenCTI with elasticsearch, rabbitmq, redis on a Kubernetes Cluster running 1.28.4 on containerd 1.7.2 with Helm 3.7.2 using the follwing variables:
What's your helm version?
3.7.2
What's your kubectl version?
1.28.4
Chart name
devops-ia/opencti
Chart version
1.2.2
Explain bug
Deployed using helm as wel as writing on k8s files based on docker-compose for OpenCTI, but we run into the same issue on both situations. After deploying all the components, only opencti-server seems to fail after 60-90 seconds:
{"category":"APP","errors":[{"attributes":{"genre":"TECHNICAL","http_status":500},"message":"Engine attachment processor configuration fail","name":"CONFIGURATION_ERROR","stack":"CONFIGURATION_ERROR: Engine attachment processor configuration fail\n    at error (/opt/opencti/build/src/config/errors.js:8:10)\n    at ConfigurationError (/opt/opencti/build/src/config/errors.js:70:53)\n    at /opt/opencti/build/src/database/engine.js:816:20\n    at processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at elConfigureAttachmentProcessor (/opt/opencti/build/src/database/engine.js:804:5)\n    at searchEngineInit (/opt/opencti/build/src/database/engine.js:292:32)\n    at checkSystemDependencies (/opt/opencti/build/src/initialization.js:31:3)\n    at platformStart (/opt/opencti/build/src/boot.js:14:7)"},{"message":"parse_exception\n\tRoot causes:\n\t\tparse_exception: No processor type exists with name [attachment]","name":"ResponseError","stack":"ResponseError: parse_exception\n\tRoot causes:\n\t\tparse_exception: No processor type exists with name [attachment]\n    at Won.request (/opt/opencti/build/node_modules/@elastic/elasticsearch/node_modules/@elastic/transport/src/Transport.ts:553:17)\n    at processTicksAndRejections (node:internal/process/task_queues:95:5)\n    at ydn.putPipeline (/opt/opencti/build/node_modules/@elastic/elasticsearch/src/api/api/ingest.ts:195:12)\n    at elConfigureAttachmentProcessor (/opt/opencti/build/src/database/engine.js:804:5)\n    at searchEngineInit (/opt/opencti/build/src/database/engine.js:292:32)\n    at checkSystemDependencies (/opt/opencti/build/src/initialization.js:31:3)\n    at platformStart (/opt/opencti/build/src/boot.js:14:7)"}],"level":"error","message":"Engine attachment processor configuration fail","timestamp":"2024-05-03T08:07:45.697Z","version":"6.0.10"} {"category":"APP","dependencyError":{"$metadata":{"attempts":3,"totalRetryDelay":188},"code":"ECONNREFUSED"},"level":"error","message":"[OPENCTI] System dependencies check failed","timestamp":"2024-05-03T08:07:46.197Z","version":"6.0.10"} {"category":"APP","errors":[{"attributes":{"genre":"TECHNICAL","http_status":500},"message":"An unknown error has occurred","name":"UNKNOWN_ERROR","stack":"UNKNOWN_ERROR: An unknown error has occurred\n    at error (/opt/opencti/build/src/config/errors.js:8:10)\n    at UnknownError (/opt/opencti/build/src/config/errors.js:76:47)\n    at Object._logWithError (/opt/opencti/build/src/config/conf.js:331:23)\n    at Object.error (/opt/opencti/build/src/config/conf.js:341:48)\n    at platformStart (/opt/opencti/build/src/boot.js:47:12)\n    at processTicksAndRejections (node:internal/process/task_queues:95:5)"},{"message":"","name":"AggregateError","stack":"AggregateError: \n    at internalConnectMultiple (node:net:1116:18)\n    at afterConnectMultiple (node:net:1683:7)"}],"level":"error","message":"Platform unmanaged direct error","timestamp":"2024-05-03T08:07:46.198Z","version":"6.0.10"}
From other deployments in the stack, like i.e. redis or rabbitmq, we can reach the elasticsearch deployment and receive data when curling it.
What you expected to happen?
OpenCTI to start and serve webportal for usage.
How to reproduce it?
Deploy the helm chart using the values files provided below.
Enter the changed values of values.yaml?
# -- Environment variables to configure application
# ref: https://docs.opencti.io/5.12.X/deployment/configuration/#platform
env:
  # Plain vars
  # foo: bar
  # my_env: my_value
  # APP OPENCTI
  APP__ADMIN__EMAIL<EMAIL_ADDRESS>  APP__ADMIN__PASSWORD: ***
  APP__ADMIN__TOKEN: ***
  APP__BASE_PATH: "/"
  APP__BASE_URL: "https://opencti.test.eu"
  OPENCTI_URL: "https://opencti.test.eu"
  # MINIO:
  MINIO__ENDPOINT: opencti-minio:9000
  MINIO_ACCESS_KEY: admin
  MINIO_SECRET_KEY: ***
  # ELASTICSEARCH
  ELASTICSEARCH__ENGINE_SELECTOR: elk
  ELASTICSEARCH__INDEX_PREFIX: opencti
  ELASTICSEARCH__URL: http://elasticsearch:9200
  # RABBITMQ
  RABBITMQ__HOSTNAME: opencti-rabbitmq
  RABBITMQ__PORT_MANAGEMENT: 15672
  RABBITMQ__PORT: 5672
  RABBITMQ__USERNAME: admin
  RABBITMQ__PASSWORD: ***
  # REDIS
  REDIS__HOSTNAME: opencti-redis-master
  REDIS__PORT: 6379
  REDIS__MODE: single
  # METRICS AND HEALTHCHECK
  APP__TELEMETRY__METRICS__ENABLED: true
  APP__HEALTH_ACCESS_KEY: ***

# -- ElasticSearch subchart deployment
# ref: https://github.com/bitnami/charts/blob/main/bitnami/elasticsearch/values.yaml
elasticsearch:
  # -- Enable or disable ElasticSearch subchart
  enabled: false

  ## Kernel settings modifier image
  ##
  sysctlImage:
    ## @param sysctlImage.enabled Enable kernel settings modifier image
    ##
    enabled: false

  # -- Elasticsearch cluster name
  clusterName: opencti-elastic

  extraEnvVars:
    - name: ES_JAVA_OPTS
      value: "-Xms512M -Xmx512M"

  ## @section Master-elegible nodes parameters
  master:
    # -- Deploy the Elasticsearch master-elegible nodes as master-only nodes. Recommended for high-demand deployments.
    masterOnly: true
    # -- Number of master-elegible replicas to deploy
    replicaCount: 1
    # -- Enable persistence using Persistent Volume Claims
    # ref: https://kubernetes.io/docs/user-guide/persistent-volumes/
    persistence:
      # -- Enable persistence using a `PersistentVolumeClaim`
      enabled: false

  # -- Data-only nodes parameters
  data:
    # -- Number of data-only replicas to deploy
    replicaCount: 1

    # -- Enable persistence using Persistent Volume Claims
    # ref: https://kubernetes.io/docs/user-guide/persistent-volumes/
    persistence:
      # -- Enable persistence using a `PersistentVolumeClaim`
      enabled: false

  # -- Ingest-only nodes parameters
  ingest:
    # --  Enable ingest nodes
    enabled: false

  # -- Coordinating-only nodes parameters
  coordinating:
    # -- Number of coordinating-only replicas to deploy
    replicaCount: 0

Enter the command that you execute when chart failing/misfunctioning
helm install opencti devops-ia/opencti -f values.yml -n namespace_name
More info
No response

I'm checking...

@remys89, can you try this? It's weird that the rest of the services can reach the Elasticsearch service but not OpenCTI.
Enforce fullnameOverride flag to set the name which use on ELASTICSEARCH__URL environment.
env:
...
  ELASTICSEARCH__ENGINE_SELECTOR: elk
  ELASTICSEARCH__INDEX_PREFIX: opencti
  ELASTICSEARCH__URL: http://elasticsearch:9200

elasticsearch:
  enabled: true
  fullnameOverride: elasticsearch

If you see the logs: ECONNREFUSED. So, here's my CI values: https://github.com/devops-ia/helm-charts/blob/main/charts/opencti/ci/ci-values.yaml, maybe can help you to set values.
Tell me if doesn't work.

I deployed it using our own elasticsearch in the same namespace, but i still seem to run into the same error. I will investigate more tomorrow and get back to you.

OpenCTI seems to boot up just fine now, had to adjust the token to a valid UUIDv4 and it works now, with our own elasticsearch instance. Had to disable the ES instance in the helm chart, since it requires some additional config for memory reservation. We can close this if its up to me.

Thanks for the update @remys89!


Click on notification doesn't go to the service in another workspace

Currently if when there is a notification, user should go to correct service in correct workspace.
Expected behaviour should be opening the service in:

Current workspace is the service is enabled in current workspace
First enabled workspace if the service is not enabled in current workspace but enabled in another workspace(s).
All services if the service is not enabled at any workspace(s)

Looks to be a bug in Franz itself and should be implemented there to help Franz users as well.


Question marks at the end swallowed

Looks like the example with just question marks is good now:
>>> segmenter.segment("??")
['??']

but the example with double question marks as a token at the end of a sentence still loses the question marks:
>>> segmenter.segment("T stands for the vector transposition. As shown in Fig. ??")
['T stands for the vector transposition.', 'As shown in Fig.']

looks like this is the minimal repro:
>>> segmenter.segment("Fig. ??")
['Fig.']

@danielkingai2 : Is given example - "Fig. ??" - actual sentence from some text? Seems like acronym with ?? would be less likely. Tried following:
In [2]: seg.segment('Fig. 20??')
Out[2]: ['Fig.', '20??']

In [3]: seg.segment('Fig. 20 ??')
Out[3]: ['Fig.', '20 ??']

In [4]: seg.segment('Fig. ??')
Out[4]: ['Fig.']

I would need to add fig abbreviation to keep above tokens intact. Will look into it

It can result from text that is parsed from a pdf. The problem is that it becomes challenging to use the output if the output text doesn't match the input text. It looks like it only happens when the ?? is at the end of the input sequence and after a sentence split by pysbd (see examples below). Would it be easier to add a case to handle a sequence ending with ?? rather than special casing the abbreviation?
>>> segmenter.segment("This text talks about Fig. ??. It is a figure.")
['This text talks about Fig.', '??.', 'It is a figure.']
>>> segmenter.segment("This text talks about Fig. ??.")
['This text talks about Fig.', '??.']
>>> segmenter.segment("This text talks about Fig. ?? .")
['This text talks about Fig.', '?? .']
>>> segmenter.segment("This text talks about Fig. ?? which is a figure.")
['This text talks about Fig.', '?? which is a figure.']
>>> segmenter.segment("This text talks about Fig ??")
['This text talks about Fig ??']
>>> segmenter.segment("This text talks about Fig. ??")
['This text talks about Fig.']

Yes, I agree.
Can you please try out adding following line after https://github.com/nipunsadvilkar/pySBD/blob/master/pysbd/processor.py#L182
txt = re.sub(r'☇$', '??', txt)

seems better with that, at least it doesn't truncate the text
>>> segmenter.segment("This text talks about Fig. ??")
['This text talks about Fig.', '??']
>>> segmenter.segment("This text talks about Fig ??")
['This text talks about Fig ?', '?']

Yes, I really need to come up with some assertion logic to map respective sentences to the original text. This is the main reason why I've been working on https://github.com/nipunsadvilkar/pySBD/tree/sentence-char-span branch because even if pysbd fails to find proper sentence. tok.is_sent_start would remain False and we will get an original text at the end

In the meantime, would you mind merging that fix you came up with?

Yes, sure!
And sorry for multiple bugs, I have ported it from https://github.com/diasks2/pragmatic_segmenter to python with all the rules as it is and getting those tests of golden rules pass.
Some of the issues (better to call it edge-cases) which you created earlier are not accounted for in pragmatic_segmenter. Though, I have tried to fix them in pysbd. Thanks for making pysbd more robust :)

No worries, I totally understand that you've ported the ruby gem! Definitely appreciate your responsiveness. Working with pdf parsed text causes all kinds of edge cases.

Yeah I concur, I myself work with lot of OCR text so know the pain of unformatted text

@danielkingai2 : Fixed above bug & have released char-span functionality today. Didn't release it yday since I wanted to add tests and update docs.

Nice!


rsx!{ "{foo::BAR}" } fails to compile: "expected } in format string"

Problem
rsx! doesn't seem to support paths: rsx!{ "{foo::BAR}" }
Steps To Reproduce
Add the following to a Dioxus app:
mod foo {
    const BAR: &str = "baz";
}

rsx! { "{foo::BAR}" };

Expected behavior
The code compiles.
Instead, fails with:
error: invalid format string: expected `'}'`, found `':'`
  --> src/main.rs:82:5
   |
20 |     rsx! { "{foo::BAR}" };
   |     ^^^^^^^^^^^^^^^^^^^^^ expected `}` in format string
   |
   = note: if you intended to print `{`, you can escape it using `{{`

Environment:

Dioxus version: My fork based on master (updates after Feb 14 are missing)
Rust version: 1.68.0
OS info: Linux
App platform: LiveView

Questionnaire

[ ] I'm interested in fixing this myself but don't know where to start
[ ] I would like to fix and I have a solution
[ ] I don't have time to fix this right now, but maybe later

@Demonthos: Alright. I saw one of your comments that stated "we now accept any arbitrary expression", so I thought this is a bug.
Are there plans to eliminate this restriction, or should this issue be closed?


Crawl links with query params without writing to the disk

Describe the feature
As discussed here: #nuxt/nuxt/issues/22197 with @danielroe, the query urls present on the html during the crawl process should be explored to discover links presents on the page even if they are not pre-rendered. (The most explicit example is for paginated pages like https://example.blog/?page=1).
I will make a PR about this soon.
Additional information

[X] Would you be willing to help implement this feature?

Thanks for opening issue here. I think it is a fairly valid reason to crawl links without writing to the disc 👍🏼  (we might have an option btw to disable feature...)