[LiveComponent] Accessing the original request when a component route is called

My application consist in some routes accessibile using a special parameter called _profile (behave like _locale special parameter):

60b276df-290f-4479-9ffc-ec7ca39dc297/page1

Component1
Component2

60b276df-290f-4479-9ffc-ec7ca39dc297/page2

Component3

Under both pages, my live components "live". These components need the profile object (I'm actually passing the profile to each component). Live component action will call some service, which in turn will use the provided profile:
LiveComponent -> Action -> Service (use the profile)
Do you have an idea on how to access the original request from the service, when a Live Component action is called? The RequestStack can't help here.

Additional notes: I also have a ProfileListener
Additional notes: I also have a ProfileListener which, on every request, check if we are under the "profile zone" and:

Add a Twig global variable named profile (so I can access the profile object in my Twig)
Add a router context parameter named _profile (so I don't need to specify _profile when generating urls)

<?php

namespace App\EventListener;

use App\Entity\Profile;
use App\Repository\ProfileRepository;
use Symfony\Component\EventDispatcher\EventSubscriberInterface;
use Symfony\Component\HttpKernel\Event\RequestEvent;
use Symfony\Component\HttpKernel\Exception\NotFoundHttpException;
use Symfony\Component\HttpKernel\KernelEvents;
use Symfony\Component\Routing\RequestContextAwareInterface;
use Twig\Environment;

class ProfileListener implements EventSubscriberInterface
{
    public function __construct(
        private ProfileRepository $repository,
        private Environment $twig,
        private ?RequestContextAwareInterface $router = null
    ) {
    }

    public function onKernelRequest(RequestEvent $event): void
    {
        $request = $event->getRequest();
        if (!$event->isMainRequest()) {
            return;
        }

        $profileUuid = $request->attributes->get('_profile');
        if (!$profileUuid) {
            return;
        }

        $profile = $this->repository->find($profileUuid);
        if (!$profile) {
            throw new NotFoundHttpException(
                sprintf('"%s" object not found by "%s".', Profile::class, self::class)
            );
        }

        $this->twig->addGlobal('profile', $profile);
        $this->router?->getContext()->setParameter('_profile', $profileUuid);
    }

    public static function getSubscribedEvents(): array
    {
        return [
            KernelEvents::REQUEST => 'onKernelRequest',
        ];
    }
}

There is currently no way to access the original request.
If your components need this "profile" information, it doesn't feel illogical to me that you pass it and make it a non-writable liveprop.
You also could use a custom route for your components, in which you'd set this profile id (like what's done with the locale) ?

You can use this "hack/custom solution" only for selected components, setting the route on the AsLiveComponent attribute

Thanks again @smnandre , I'm going to close this issue. So the final answer is "There is currently no way to access the original request." but there are some workarounds.
I'm going to investigate AsLiveComponent !

How to Recover Your Messenger Account
Losing access to your Messenger account can be frustrating, especially if it contains important messages and connections. Fortunately, there are several methods to recover your account and regain control. Here’s a step-by-step guide to help you through the process.

Use the “Forgot Password” Feature
If you can’t log in, the first step is to use the "Forgot Password" option on the Facebook login page, as password Messenger is tied to your Facebook account. Enter the email address or phone number associated with your account. You will receive a password reset link via email or a verification code via text message. Follow the instructions to create a new password and regain access.

Check Account Recovery Options
During the recovery process, Facebook may prompt you to verify your identity using alternate methods. This could include answering security questions, recognizing friends in tagged photos, or confirming recent activity on your account. Providing accurate information will improve your chances of successful recovery.

Use Trusted Contacts
If you’ve previously set up trusted contacts in your Facebook account, they can help you recover access. You’ll need to reach out to these friends and ask them to provide you with a special recovery code. Once you have the code, enter it during the recovery process to unlock your account.

Regain Access to Your Email or Phone Number
If the email or phone number linked to your account is no longer accessible, try recovering access to these first. Contact your email provider or phone carrier to restore the associated credentials. Once you have access, use the “Forgot Password” option again.

Contact Facebook Support
If none of the above methods work, you can reach out to Facebook’s support team. Use the "Help Center" on Facebook’s website and provide as much detail as possible about your account, such as your username, profile URL, or a government-issued ID for verification.

By following these steps, you can recover your Messenger account and secure it for future use. To prevent future issues, ensure you update your recovery options and enable two-factor authentication.


Animate Slide on Absolute Positioned Element

I have an element, a navigation menu, that is has position: absolute style applied so that it can easily lay over all the other content. However, with this position it seems the library will only do the fade transition. Is there a way to still use the slide transition with an element that is absolutely positioned?

Absolutely positioned child doesn't has effect to it’s parent element. So if you have
<Animate >

    ...

</Animate>

Animate can't calculate its children's height. A work around would be putting absolute as parent outside

  <Animate >
    ...
  </Animate>

I think slide should work this way.
Let me know if this helps:) @yuschick

Thanks, @MingruiZhang. I came up with the wrapper solution as well which worked out for me. Makes sense though about the absolute positioned item and the slide.
But great work on this library, it's quite helpful in my current project.


Backend to login/signup

Related Issue
[Cite any related issue(s) this pull request addresses. If none, simply state "None”]
Solved #632
Description
[Please include a brief description of the changes or features added]
I have made the backend for the login and signup page.
Type of PR

[ ] Bug fix
[x] Feature enhancement
[ ] Documentation update
[ ] Other (specify): _______________

Screenshots / videos (if applicable)
[Attach any relevant screenshots or videos demonstrating the changes]
I have covered everything in the video .
-> It throws error if duplicate user email is being registered.
-> It throws error if wrong password is given.
-> The password must be more than 6 characters.
-> The username takes place according to its email. for example if user email is<EMAIL_ADDRESS>then the username "sha" will be displayed.
-> Logout button is also there.
-> If the user has previous login credintials in there local storage they will redirect automatically.
https://github.com/Suchitra-Sahoo/AgriLearnNetwork/assets/102676949/c66b8eab-0817-4887-8ef2-f95b03bb4bf7
For<EMAIL_ADDRESS>
Checklist:

[x] I have performed a self-review of my code
[x] I have read and followed the Contribution Guidelines.
[x] I have tested the changes thoroughly before submitting this pull request.
[x] I have provided relevant issue numbers, screenshots, and videos after making the changes.
[x] I have commented my code, particularly in hard-to-understand areas.

Additional context:
[Include any additional information or context that might be helpful for reviewers.]
If you want the data to be in your firebase account make sure to replace the credentials.

The navbar looks like this after login alignment is not proper

logout option is overlapping and is not looking good

@Suchitra-Sahoo Okay I will do the changes.

Please assign the labels also.

Hey why had you reverted it and also backend is not working do I have to do it again?

@Suchitra-Sahoo


new: use fonctional args for Issue and Sign

This pr deprecated IssueCertificates and SignCSR in favor or Issue and Sign. The laters use functional arguments to make usage way easier. Options allows fine tuning as before, or provides high level versions that makes configuration easy.

/build - automatically fired by gogo with following PRs and commit SHAs v1.0.0
[
  {
    "project": "better-tg",
    "component": "tg",
    "pr-id": "15",
    "commit-sha": "96f3314f9dae003f55b5135aaea3ad461e361fbc"
  },
  {
    "project": "better-tg",
    "component": "backend",
    "pr-id": "345",
    "commit-sha": "9f7821dd12faeb8549242618c9400ca86b80b2ef"
  }
]

/build - automatically fired by gogo with following PRs and commit SHAs v1.0.0
[
  {
    "project": "better-tg",
    "component": "tg",
    "pr-id": "15",
    "commit-sha": "9fe124917983a0784bb53bb4096445f78b22a8e8"
  },
  {
    "project": "better-tg",
    "component": "backend",
    "pr-id": "345",
    "commit-sha": "1e62067565e5ca9b841171fd5e02a1a2f6e2194b"
  }
]

/build - automatically fired by gogo with following PRs and commit SHAs v1.0.0
[
  {
    "project": "better-tg",
    "component": "tg",
    "pr-id": "15",
    "commit-sha": "897f82e177efa9e76ecad464f49e63970c9f9fbd"
  },
  {
    "project": "better-tg",
    "component": "backend",
    "pr-id": "345",
    "commit-sha": "1e62067565e5ca9b841171fd5e02a1a2f6e2194b"
  }
]

/build - automatically fired by gogo with following PRs and commit SHAs v1.0.0
[
  {
    "project": "better-tg",
    "component": "tg",
    "pr-id": "15",
    "commit-sha": "d3d6cd28b64d5720fd3f1a22966cdb42c35a4bce"
  },
  {
    "project": "better-tg",
    "component": "backend",
    "pr-id": "345",
    "commit-sha": "1e62067565e5ca9b841171fd5e02a1a2f6e2194b"
  }
]

/build - automatically fired by gogo with following PRs and commit SHAs v1.0.0
[
  {
    "project": "better-tg",
    "component": "backend",
    "pr-id": "345",
    "commit-sha": "1e62067565e5ca9b841171fd5e02a1a2f6e2194b"
  },
  {
    "project": "better-tg",
    "component": "tg",
    "pr-id": "15",
    "commit-sha": "dd0ba01cd6f236c767f6cb5c08e6e51bdbd05140"
  }
]


Could not reproduce the answer recall for NQ dataset

Hi, I load nq/full-00000-of-00001.parquet and compute the answer recall based on
answers, context = item["answer"], item["context"]
is_retrieval = has_correct_answer(context, answers)
I could only get 0.8532 answer recall, which is below the reported number "88.53" in Table 1 of the paper.

Hi @tyu008, thanks for raising this question. First, the nq/full-00000-of-00001.parquet corresponds to the num_retrieval_units = 4 line, not the num_retrieval_units = 8 line. The QA performance has shown that the ideal long context for existing LLMs is around 30K. If we use a longer context, even if the retrieval performance is higher, the final QA result will degrade (as shown in Figure 3). Therefore, the correct number is 86.30 rather than 88.53. I will mark this more clearly in the repository.
Second, you are right. The current version's retrieval accuracy is 85.30. There is still a one-point gap between 85.30 and the 86.30 reported in the paper. I suspect I may have uploaded an older version of the final result. I will take a look and upload the correct one.
Thanks again for pointing it out!

Hi, @XMHZZ2018, thanks so much for your quick reply.
I am also trying to reproduce the results from max-P method in NQ dataset.
Following the paper, I divide each group into 512-token snippet, and use the maximum snippet similarity as the similarity for the group. But I could only obtain 67.5% answer recall using top-1 group, which is below the reported 71.69 in the paper.  I am using the exact same model BAAI/bge-large-en-v1.5 with fp_16.
I guess I might have some misalignment with you in chunking the group. May you share the cropped 512-token snippets? Thanks again

@tyu008 Sure! I think I know the main reason here. I avoid cross-document chunking; for example, if a new document starts, it will be in the next chunk. Previously, when I did cross-document chunking, I observed about a 5% to 10% degradation. (This issue becomes even more severe in HotpotQA since the document length is even smaller.) I assume the same thing happened to you. I will upload my chunking file to Hugging face soon so you can reproduce the results. I will ping you here after I finished.

@XMHZZ2018  Got it! Thanks a lot for your quick reply!
Look forward to your chunking file.


Parameterizing semantic operations

I'll start with an example grammar:
DateGrammar {
  Date = "now"   -- now
       | "later" -- later
}

And a semantic operation:
semantics.addOperation('extractDate', {
  'Date_now': function() {
    let now = new Date()
    return now
  },

  'Date_later': function() {
    let now = new Date()
    let later = new Date(now + 3 * 3600)
    return later
  }
})

Now, when writing tests for this semantic operation I'd like to set a fixed date instead of relying on the current time. I realize this probably wasn't possible short of wrapping everything in another scope or (gulp) using dynamic scope.
So I imagined something like this:
What if semantic operations could be parameterized?
let match = grammar.match('now')
let now = new Date(1440000000000)
let date = semantics(match).extract(now)
assert(+date === +now)

And the parameters could be accessed like this?
semantics.addOperation('extractDate', {
  'Date_now': function() {
    let now = this.parameters[0]
    return now
  },

  'Date_later': function() {
    let now = this.parameters[0]
    let later = new Date(now + 3 * 3600)
    return later
  }
})

This could be accessed via this.parameters, or this.context, or this.contextParameters, and would be the same value in all nodes.

Thanks, Stephan.
I'd love to enable Ohm's operations to take arguments, and have been thinking about how to do this for a while. This is a good time to make it a higher-priority item, so I'm glad you've opened this issue.
Accessing the operation's arguments via this in a semantic action doesn't seem right, though, because this corresponds to a CST (concrete syntax tree) node, not the operation. And the CST node doesn't have arguments, at least in your formulation.
I'll think about this a little more over the next week, and let you know if I come up with anything.
Cheers,
Alex

Alex, thank you for looking into this :)

I would think that the arguments should behave like regular function arguments -- so it should be possible to write a semantic action like this:
name: function(lastName, comma, firstName) {
  return lastName.stringValue('upper') + firstName.stringValue('lower');
}

...where stringValue is an operation that takes one parameter. For the case where you want all nodes to be parameterized by the same value, I think that closing over a variable is already a decent way to do it:
function makeActionDict(now) {
  return {
    Date_later: function() {
      return new Date(now + 3 * 3600);
    },
    ...
  };
}
semantics.addOperation('extractDate', makeActionDict(1440000000000));

For the case where you want all nodes to be parameterized by the same value, I think that closing over a variable is already a decent way to do it

Yes, that's true. My concern was for parsing incrementally (as input is gathered) where this might be creating lots of function objects. I'm not sure about the overhead involved though, parsing might actually be more expensive :)
Oh, and actually in my case it's probably fine to create an action dict at the start of the input, since the context won't have to change or exist for that long. Thanks :)

Yes, that's true. My concern was for parsing incrementally (as input is gathered) where this might be creating lots of function objects. I'm not sure about the overhead involved though, parsing might actually be more expensive :)

You can reuse the same semantics instance with different MatchResult instances, so the overhead shouldn't be a concern.

Fixed by https://github.com/cdglabs/ohm/commit/8e3ad7f84331f8be4cc60f846b4426cba6fba7c7.
It used to be that you could only add parameter-less operations to a semantics, e.g.,
myGrammar.semantics().addOperation('myOp', { ... });

This is still allowed, but it's now shorthand for:
myGrammar.semantics().addOperation('myOp()', { ... });

The first argument to Semantics.prototype.addOperation is now a string that specifies both the name of an operation, and (optionally) the names of its arguments. (This string is parsed using the NameAndFormals rule in the OperationsAndAttributes grammar, which you can see at src/operations-and-attributes.ohm.) So if we wanted myOp to take two parameters, x and y, we would write:
myGrammar.semantics().addOperation('myOp(x, y)', { ... });

These arguments must be supplied every time the operation is invoked, and are accessed by name via the node's args property, e.g.,
myGrammar.semantics().addOperation('myOp(x, y)', {
  Foo: function(bar) {
    return bar.myOp(this.args.x + 1, this.args.y * 2);
  },
  ...
});

(Stephan: as you can see, I changed my mind on the this thing. Thanks again for the suggestion! :))
Please let us know if you have any questions, comments, etc.
Cheers,
Alex

Alex, this looks great!
I'll change my current usage of my semantics and report back if I lean anything new.
Thank you for adding a major feature so quickly!

Glad you like it, Stephan! Looking forward to hearing whatever thoughts you have about it after you get a chance to use it.


Add various default options

Attempt to fix: https://github.com/killbill/killbill-plugin-framework-java/issues/82

Instead of exec-maven-plugin, create maven plugin, or separated script, I decided to give api-pojos more options and default value.
For example, If in your current directory, you have:

pojogen.jar
killbill-api project
killbill-plugin-framework-java project
Then calling java -jar pojogen.jar will generate all interfaces classes in killbill-api to boilerplate subpackage in killbill-plugin-framework-java.

See README.md file more explanation and options.

@xsalefter I merged the PR, but I think we should do a second pass:

There are 2 different set of instructions, the one in the 'README' and the one under the doc. I think we should merge those 2 as this gets confusing. Either keep a top level README that points to more detailed instructions under doc, or remove the doc altogether.

Regarding the setting.xml. I would suggest to check-in a version that works. Even better, if we could generate it - but if too much work, let's not do it. It needs to have all the packages acceptedPackages inside. I had created my own version at  the time by grepping from all package name under api. I had gotten the following:

<?xml version="1.0" encoding="UTF-8"?>
<Settings>
  <sourceDirectories>
    <sourceDirectory>input/project/src/main/java</sourceDirectory>
  </sourceDirectories>
  <dependencyDirectories>
    <dependencyDirectory>input/lib</dependencyDirectory>
  </dependencyDirectories>
  <outputSubpackage>boilerplate</outputSubpackage>
  <outputClassPrefix></outputClassPrefix>
  <outputClassSuffix>Imp</outputClassSuffix>
  <outputDirectory>output/project/src/main/java</outputDirectory>
  <resourceDirectory>output/project/src/main/resources</resourceDirectory>
  <testDirectory>output/project/src/test/java</testDirectory>
  <acceptedPackages>
  </acceptedPackages>
    <package>org.killbill.billing.tenant</package>
    <package>org.killbill.billing.tenant.api</package>
    <package>org.killbill.billing.util</package>
    <package>org.killbill.billing.util.customfield</package>
    <package>org.killbill.billing.util.nodes</package>
    <package>org.killbill.billing.util.entity</package>
    <package>org.killbill.billing.util.audit</package>
    <package>org.killbill.billing.util.queue</package>
    <package>org.killbill.billing.util.callcontext</package>
    <package>org.killbill.billing.util.api</package>
    <package>org.killbill.billing.util.tag</package>
    <package>org.killbill.billing.security</package>
    <package>org.killbill.billing.security.api</package>
    <package>org.killbill.billing.payment</package>
    <package>org.killbill.billing.payment.plugin</package>
    <package>org.killbill.billing.payment.plugin.api</package>
    <package>org.killbill.billing.payment.api</package>
    <package>org.killbill.billing.catalog</package>
    <package>org.killbill.billing.catalog.api</package>
    <package>org.killbill.billing.catalog.api.rules</package>
    <package>org.killbill.billing.osgi</package>
    <package>org.killbill.billing.osgi.api</package>
    <package>org.killbill.billing.osgi.api.config</package>
    <package>org.killbill.billing.usage</package>
    <package>org.killbill.billing.usage.api</package>
    <package>org.killbill.billing.account</package>
    <package>org.killbill.billing.account.api</package>
    <package>org.killbill.billing.entitlement</package>
    <package>org.killbill.billing.entitlement.api</package>
    <package>org.killbill.billing.currency</package>
    <package>org.killbill.billing.currency.api</package>
    <package>org.killbill.billing.overdue</package>
    <package>org.killbill.billing.overdue.api</package>
    <package>org.killbill.billing.invoice</package>
    <package>org.killbill.billing.invoice.api</package>
    <package>org.killbill.billing.invoice.api.formatters</package>
  <acceptedInterfaces>
  </acceptedInterfaces>
  <comparableTypes>
    <type>org.joda.time.DateTime</type>
  </comparableTypes>
  <builderClass>Builder</builderClass>
  <resolverClass>Resolver</resolverClass>
  <moduleClass>Module</moduleClass>
</Settings>

To summarize, we should end up with one clear set of instructions to follow, and if we need additional files (e.g setting.xml), they should either be checked-in or even better generated by a tool - same or different tool.

@sbrossie reviving old discussion. Sorry I just replied now:

Yeah, I'll try to clean up documentation.

I would suggest to check-in a version that works

This src/main/resources/settings.xml is working and valid settings.xml file. It is a default settings.xml if user not specify any settings.xml file when generating, for example: java -jar pojogen.jar. It has small different with your version, for example:
yours: <sourceDirectory>input/project/src/main/java</sourceDirectory>.
checked in: <sourceDirectory>./killbill-api/src/main/java</sourceDirectory>
I need to changes input to ./killbill-api so that users don't have to set anything. Of course I need to make an assumption that user with work with this project structure:
    .
    ├── pojogen.jar
    ├── killbill-api                      # project root directory
    ├── killbill-plugin-framework-java    # project root directory

.... It needs to have all the packages acceptedPackages inside

It has the same accepted packages with yours.

Looping Reshma, as she has a task to use the tool, so it will be great to see if the instructions are clear or not. @reshmabidikar Could you catch up on this discussion and work with Resa to improve the instructions. The goal is that when we need to use it, we don't have to think much... just follow the instructions.

The current instructions are helpful and I was able to generate the POJOs without much effort.
However, I'm listing some areas of improvement here:

Move Build instructions to the top of the Readme
The Usage section can be restructured to something like this: https://github.com/killbill/killbill-ee/wiki/Support-tools#usage-1.
Is example  really required? If not, maybe it can be removed


Race condition causing 422 Unprocessable Entity error

Problem
Sections can contain both .liquid and .json files. The .json files reference the .liquid files as a way to group them together. If a .json file is uploaded before the .liquid file is uploaded, a 422 Unprocessable Entity error is thrown.
Example
header-group.json and header.liquid are both stored in the /sections folder and are uploaded in this order.

header-group.json

header.liquid

{
  "name": "Header Group",
  "type": "header",
  "sections": {
    "header": {
      "type": "header",
      "settings": {
        "color_scheme": "scheme-1"
      }
    }
  },
  "order": [
    "header"
  ]
}

This is the header

{% schema %}
{
  "name": "Header",
  "class": "section-header",
  "settings": []
}

Failed

Success

The Cause
When uploading a theme, Syncify iterates through the files in alphabetical order (presumably). Because header-group.json is uploaded before header.liquid, the referenced section doesn't exist on the store.
If we were to run the upload argument again, it'll work, since the file is now present from the previous upload job.
Solution
Syncify should iterate through all the .liquid files first, then iterate through the .json files.

Which operation should is execute within? (like which command?) or should a check always apply when applying a sync?

Dev Build
If I were to run syncify store --theme dev --watch --hot, once I save the json file, I'll see this error since the liquid file will not exist. It would be as simple as saving the referenced file and we'll be good again. I don't think it's Syncify's job to handle this here, it's a waste of time.
Production Build
Now, if I were to run syncify store --theme dev --upload, I'll run into the problem. So I'd say the --upload flag needs to deal with this.


Impossible to start in server mode

Environmental Info:
K3s Version: 3s version v1.29.6+k3s1

Node(s) CPU architecture, OS, and Version: Linux MCSWKS01345 6.5.0-1025-oem #26-Ubuntu SMP PREEMPT_DYNAMIC Tue Jun 18 12:35:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Cluster Configuration: single node server

Describe the bug:

When installing k3s, it's impossible to make it start in server mode, it only starts as agent.The same command with foring version v1.29.5+k3s1 works as expected.
Steps To Reproduce:

Installed K3s:
Tried with
curl -sfL https://get.k3s.io | sh -s - --node-ip=<IP_ADDRESS> --resolv-conf /etc/resolv.conf --cluster-domain=cluster.priv --write-kubeconfig-mode=644
and
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server" sh -s - --node-ip=<IP_ADDRESS> --resolv-conf /etc/resolv.conf --cluster-domain=cluster.priv --write-kubeconfig-mode=644

But after installing, the process show "k3s agent"
When forcing the version to v1.29.5+k3s1 with the command
curl -sfL https://get.k3s.io | INSTALL_K3S_VERSION="v1.29.5+k3s1" sh -s - --node-ip=<IP_ADDRESS> --resolv-conf /etc/resolv.conf --cluster-domain=cluster.priv --write-kubeconfig-mode=644
it starts in server mode as expected
Expected behavior:

k3s should start in server mode
Actual behavior:

k3s starts in agent mode
Additional context / logs:

Are you sure you don't have K3S_URL exported in your environment? If you have that set, the script assumes you want an agent - since you're providing the address of an existing server.
Both of your examples give me servers:
root@systemd-node-1:/# curl -sfL https://get.k3s.io | sh -s - --node-ip=<IP_ADDRESS> --resolv-conf /etc/resolv.conf --cluster-domain=cluster.priv --write-kubeconfig-mode=644
[INFO]  Finding release for channel stable
[INFO]  Using v1.29.6+k3s1 as release
[INFO]  Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.29.6+k3s1/sha256sum-amd64.txt
[INFO]  Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.29.6+k3s1/k3s
[INFO]  Verifying binary download
[INFO]  Installing k3s to /usr/local/bin/k3s
[INFO]  Skipping installation of SELinux RPM
[INFO]  Creating /usr/local/bin/kubectl symlink to k3s
[INFO]  Creating /usr/local/bin/crictl symlink to k3s
[INFO]  Creating /usr/local/bin/ctr symlink to k3s
[INFO]  Creating killall script /usr/local/bin/k3s-killall.sh
[INFO]  Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO]  env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO]  systemd: Creating service file /etc/systemd/system/k3s.service
[INFO]  systemd: Enabling k3s unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.

root@systemd-node-1:/# curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="server" sh -s - --node-ip=<IP_ADDRESS> --resolv-conf /etc/resolv.conf --cluster-domain=cluster.priv --write-kubeconfig-mode=644
[INFO]  Finding release for channel stable
[INFO]  Using v1.29.6+k3s1 as release
[INFO]  Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.29.6+k3s1/sha256sum-amd64.txt
[INFO]  Downloading binary https://github.com/k3s-io/k3s/releases/download/v1.29.6+k3s1/k3s
[INFO]  Verifying binary download
[INFO]  Installing k3s to /usr/local/bin/k3s
[INFO]  Skipping installation of SELinux RPM
[INFO]  Creating /usr/local/bin/kubectl symlink to k3s
[INFO]  Creating /usr/local/bin/crictl symlink to k3s
[INFO]  Creating /usr/local/bin/ctr symlink to k3s
[INFO]  Creating killall script /usr/local/bin/k3s-killall.sh
[INFO]  Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO]  env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO]  systemd: Creating service file /etc/systemd/system/k3s.service
[INFO]  systemd: Enabling k3s unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.

Hi @brandond
Yes, it was a new machine and I'm sure the environment variable K3S_URL wasn't set. Did you check the process ? Because I had the same installer output but when I checked the process I could see "k3s agent" rather than "k3s server"

Please show the full output from your install script, as well as the contents of the resulting systemd unit.


Indiana mapping

completed new QDC mapping for Indiana Memory

Since IN is back in Ingestion1 I think we should close this out. @markbreedlove @guegueng any objections?

Closing this PR as IN was rolled back into ingestion1


Titleize navigation status tab

Update from job to Job on each job status tab title
before

After

Hmm... why? 🤔

Hmm... why? 🤔

I think it looks better/standarized, I feel like for In Progress jobs, it looks weird having the jobs word lowercase when the other two words are capitalized.

I think in that case, I'd go for changing In Progress to In-progress or something like that 🤔 I find Jobs capitalized everywhere a bit strange.

I think in that case, I'd go for changing In Progress to In-progress or something like that 🤔 I find Jobs capitalized everywhere a bit strange.

Okay, that makes sense, I'll close the PR, thanks for the follow up


improvementReduce partition column sample BE memory consumption. (#41203)

backport: https://github.com/apache/doris/pull/41203

run buildall


Notifications pages

This PR is for the notifications pages that we tested at Downview today.
There will be changes to this as a result of the UR.

missing CSS closing


Scrambled map when wide characters are present

As the title says.
For example, zoom in at location 39.761, 116.424 (Beijing, China).
I have made sure that my terminal is showing wide characters in monospace (2 chars wide on Chinese characters for example).

Yey, well spotted - I removed the once implemented POI-to-emoji logic because of the same reason, just wasn't aware that the Chinese characters are wider as well. Will be tackled!

Here is a screenshot of Shanghai.

@HarukaMa @Kuree pre-fix deployed to the telnet server, will be finished and pushed on the weekend, am incredibly busy with work atm. thanks for reporting in! will be solved with string-width


App website with manuals and FAQ

https://every-door.app/
For the beta, it needs:

a title and a description,
a picture (three stacked phones for each editing mode),
links to Play Store and App Store,
a short list of features,
embedded video (see #122),
and a link to FAQ pages in English and Russian.


Start gathering information about who uses the specification

Reason/Context
We know AsyncAPI is widely adopted by users but we do not have a way for users to tell us about it. Open question is if we should have one section with users or rather maybe Used by and Products integrations so community not only see how others use it but also how to integrate it to enrich products for their customers.
Description

Have a list of users and make it easy for anyone to be added to the list through PR
Such a list would feature the logo of the company, short description of use case or link to a blog post if possible. For example the logo of Axway with short description and link to https://devblog.axway.com/integration/asyncapi/. Something similar is on Kyma website https://kyma-project.io/#used-by.
Have list of users in README.md too?

@derberg I want to pick this up.

this one is pretty old 😄 I think better would be to devide it:

have on a website info about companies/vendors that build tools on AsyncAPI, like Axway, WSO2, SmartBear, SAP and others
and list of users would be populated using API mentioned in https://github.com/asyncapi/website/issues/204

so we have users and vendors separated. What do you think?

@derberg you're right, but does it has to be on a different route? I'd suggest it's on the same route, then we'll have 2 tabs that when clicked will change the view "Companies", "Users"

I personally prefer one under another than tabs, people do not click tabs really. Sections should be called "Vendors using AsyncAPI" and "Users of AsyncAPI"

@derberg guess you're right about that. But we'll be referencing each link from the homepage right?

But we'll be referencing each link from the homepage right?

what do you mean?
we should have these sections on homepage imho

@derberg Awesome!

basic work started https://github.com/asyncapi/website/pull/921

@derberg I am closing this issue as it is pretty old and we also have our first case study also on the website
We can reopen this if you find this relevant

@derberg I am closing this issue as it is pretty old and we also have our first case study also on the website
We can reopen this if you find this relevant


Fix the release build settings for document XML generation

Version
4.2.2
Describe the bug
The Microsoft.Bot.Builder.Azure NuGet package contains incorrect doc XML content.
To Reproduce
Steps to reproduce the behavior:

Download the Microsoft.Bot.Builder.Azure package, v 4.2.2.
Extract the contents.
Open microsoft.bot.builder.azure.4.2.2\lib\netstandard2.0\Micorosoft.Bot.Builder.Azure.xml
Find the entry for the CosmosDbCustomClientOptions.DocumentCollectionRequestOptions property.
Error: the summary contains a <see> tag with a cref value of !:https://docs.microsoft.com/en-us/dotnet/api/microsoft.azure.documents.client.requestoptions?view=azure-dotnet.

Expected behavior

The corresponding /// comments for this property contain the tag:
<see cref="Microsoft.Azure.Documents.Client.RequestOptions"/>.
This should generate the following doc XML tag:
<see cref="T:Microsoft.Azure.Documents.Client.RequestOptions"/>, not one that has a value that starts with !https://.

Additional context
See the ref page for the CosmosDbCustomClientOptions.DocumentCollectionRequestOptions Property to see how this renders.
The build is currently producing about 6 such errors.
[bug]

I've tried to duplicate this on the latest 4.3 codebase ("master"). In master, I had already completly re-done all of the project files - including how documentation is generated - as part of cleaning up our build.
That was PR #1314.
That PR moved all of the Doc Generation settings to the Build.Directory.Props file, and standardized all of the settings across the suite.
When I look at the Microsoft.Bot.Builder.Azure.xml file generated as part of a Documentation targeted build, all looks good. The XML looks like:
        <member name="T:Microsoft.Bot.Builder.Azure.CosmosDbCustomClientOptions">
            <summary>
            CosmosDB options for custom client.
            </summary>
        </member>

The entire file does not have a single microsoft.com reference in it, as evidenced here:
C:\git\botbuilder-dotnet\libraries\Microsoft.Bot.Builder.Azure\bin\Documentation\netstandard2.0>dir *.xml
 Volume in drive C has no label.
 Volume Serial Number is 2A77-7BE4

 Directory of C:\git\botbuilder-dotnet\libraries\Microsoft.Bot.Builder.Azure\bin\Documentation\netstandard2.0

02/07/2019  11:58 AM            20,067 Microsoft.Bot.Builder.Azure.xml
02/07/2019  11:58 AM           198,461 Microsoft.Bot.Builder.xml
02/07/2019  11:58 AM           181,371 Microsoft.Bot.Connector.xml
02/07/2019  11:58 AM           198,200 Microsoft.Bot.Schema.xml
               4 File(s)        598,099 bytes
               0 Dir(s)  343,090,954,240 bytes free

C:\git\botbuilder-dotnet\libraries\Microsoft.Bot.Builder.Azure\bin\Documentation\netstandard2.0>grep microsoft.com *.xml
Microsoft.Bot.Connector.xml:                https://docs.microsoft.com/bot-framework/rest-api/bot-framework-rest-connector-authentication
Microsoft.Bot.Schema.xml:            See https://msdn.microsoft.com/library/hh456380.aspx for a list of valid language codes

I than build a local Release build, which generates NuGet packages and XML packages. I extracted the XML file from the Nuget, and found the same results:
        <member name="T:Microsoft.Bot.Builder.Azure.CosmosDbCustomClientOptions">
            <summary>
            CosmosDB options for custom client.
            </summary>
        </member>

Lastly, I pulled down the most recent Nuget package from our daily feed. This also didn't show any issues.
I cannot duplicate this on the 4.3 codebase.
I'm going to mark this resolved. If I missed something, please let me know and I'm happy to look again.


Adding support for Octeon DPU family

Hello, requesting to add a support for Octeon DPU family.
I'm opening this Github issue per the instructions (https://github.com/k8snetworkplumbingwg/sriov-network-operator/blob/master/doc/supported-hardware.md):
Model: Marvell OCTEON Fusion CNF95XX
Vendor: 177d
Device: b600
Testing was done using OpenShift 4.10
$ oc get no -o json | jq -r '[.items[] | {name:.metadata.name, allocable:.status.allocatable}]'
[
  ...
  {
    "name": "worker0",
    "allocable": {
      "cpu": "31500m",
      "ephemeral-storage": "449603491915",
      "hugepages-1Gi": "0",
      "hugepages-2Mi": "0",
      "memory": "64700316Ki",
      "openshift.io/marvell_octeon": "3",
      "openshift.io/marvell_octeon_dpdk": "3",
      "pods": "250"
    }
  },
  ...
]

$ oc get sriovnetworknodestates.sriovnetwork.openshift.io -n openshift-sriov-network-operator worker0 -o yaml
apiVersion: sriovnetwork.openshift.io/v1
kind: SriovNetworkNodeState
metadata:
  creationTimestamp: "2022-06-30T09:00:50Z"
  generation: 4
  name: worker0
  namespace: openshift-sriov-network-operator
  ownerReferences:
  - apiVersion: sriovnetwork.openshift.io/v1
    blockOwnerDeletion: true
    controller: true
    kind: SriovNetworkNodePolicy
    name: default
    uid: d470af4b-1567-4713-81e2-daf801224b78
  resourceVersion: "16966615"
  uid: 9feaafa1-6b3a-410b-8e20-4f0c207dc4e9
spec:
  dpConfigVersion: "16965452"
  interfaces:
  - mtu: 9000
    name: ens259f0
    numVfs: 7
    pciAddress: 0000:81:00.0
    vfGroups:
    - deviceType: netdevice
      mtu: 9000
      policyName: policy-octeon
      resourceName: marvell_octeon
      vfRange: 0-3
    - deviceType: vfio-pci
      mtu: 9000
      policyName: policy-octeon-dpdk
      resourceName: marvell_octeon_dpdk
      vfRange: 4-6
status:
  interfaces:
  ...
  - Vfs:
    - deviceID: b603
      driver: Octeon_vf
      mac: 02:0f:00:0b:98:4a
      mtu: 9000
      name: ens259f0v0
      pciAddress: 0000:81:02.0
      vendor: 177d
      vfID: 0
    - deviceID: b603
      driver: Octeon_vf
      pciAddress: 0000:81:02.1
      vendor: 177d
      vfID: 1
    - deviceID: b603
      driver: Octeon_vf
      mac: 02:0f:00:0b:98:4e
      mtu: 9000
      name: ens259f0v2
      pciAddress: 0000:81:02.2
      vendor: 177d
      vfID: 2
    - deviceID: b603
      driver: Octeon_vf
      mac: 02:0f:00:0b:98:50
      mtu: 9000
      name: ens259f0v3
      pciAddress: 0000:81:02.3
      vendor: 177d
      vfID: 3
    - deviceID: b603
      driver: vfio-pci
      pciAddress: 0000:81:02.4
      vendor: 177d
      vfID: 4
    - deviceID: b603
      driver: vfio-pci
      pciAddress: 0000:81:02.5
      vendor: 177d
      vfID: 5
    - deviceID: b603
      driver: vfio-pci
      pciAddress: 0000:81:02.6
      vendor: 177d
      vfID: 6
    deviceID: b600
    driver: Octeon
    linkSpeed: 10000 Mb/s
    linkType: ETH
    mac: 02:0f:00:0b:98:49
    mtu: 9000
    name: ens259f0
    numVfs: 7
    pciAddress: 0000:81:00.0
    totalvfs: 128
    vendor: 177d
  syncStatus: Succeeded

kind: Pod
apiVersion: v1
metadata:
  name: demopod1
  namespace: openshift-sriov-network-operator
  uid: e6c783b2-0ac3-40b0-ba22-6c69b5ba9873
  resourceVersion: '15039131'
  creationTimestamp: '2022-07-04T08:37:04Z'
  annotations:
    k8s.ovn.org/pod-networks: >-
      {"default":{"ip_addresses":["<IP_ADDRESS>/23"],"mac_address":"0a:58:0a:83:01:cf","gateway_ips":["<IP_ADDRESS>"],"ip_address":"<IP_ADDRESS>/23","gateway_ip":"<IP_ADDRESS>"}}
    k8s.v1.cni.cncf.io/network-status: |-
      [{
          "name": "ovn-kubernetes",
          "interface": "eth0",
          "ips": [
              "<IP_ADDRESS>"
          ],
          "mac": "0a:58:0a:83:01:cf",
          "default": true,
          "dns": {}
      },{
          "name": "openshift-sriov-network-operator/example-sriovnetwork",
          "interface": "net1",
          "ips": [
              "<IP_ADDRESS>"
          ],
          "mac": "02:0f:00:0b:98:50",
          "dns": {},
          "device-info": {
              "type": "pci",
              "version": "1.0.0",
              "pci": {
                  "pci-address": "0000:81:02.0"
              }
          }
      }]
    k8s.v1.cni.cncf.io/networks: openshift-sriov-network-operator/example-sriovnetwork
    k8s.v1.cni.cncf.io/networks-status: |-
      [{
          "name": "ovn-kubernetes",
          "interface": "eth0",
          "ips": [
              "<IP_ADDRESS>"
          ],
          "mac": "0a:58:0a:83:01:cf",
          "default": true,
          "dns": {}
      },{
          "name": "openshift-sriov-network-operator/example-sriovnetwork",
          "interface": "net1",
          "ips": [
              "<IP_ADDRESS>"
          ],
          "mac": "02:0f:00:0b:98:50",
          "dns": {},
          "device-info": {
              "type": "pci",
              "version": "1.0.0",
              "pci": {
                  "pci-address": "0000:81:02.0"
              }
          }
      }]

Pod with Netdevice VF being consumed:
root@demopod1:/# ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1400
        inet <IP_ADDRESS>  netmask <IP_ADDRESS>  broadcast <IP_ADDRESS>
        inet6 fe80::858:aff:fe83:1cf  prefixlen 64  scopeid 0x20<link>
        ether 0a:58:0a:83:01:cf  txqueuelen 0  (Ethernet)
        RX packets 8  bytes 836 (836.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 10  bytes 796 (796.0 B)
        TX errors 0  dropped 1 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet <IP_ADDRESS>  netmask <IP_ADDRESS>
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

net1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 9000
        inet <IP_ADDRESS>  netmask <IP_ADDRESS>  broadcast <IP_ADDRESS>
        inet6 fe80::f:ff:fe0b:9850  prefixlen 64  scopeid 0x20<link>
        ether 02:0f:00:0b:98:50  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 10  bytes 796 (796.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

We successfully run ping with two such pods.
DPDK:
We spun two pods, pods were able to consume the pcie device:
Pod1:
# env
PCIDEVICE_OPENSHIFT_IO_MARVELL_OCTEON_DPDK=0000:81:02.5

Pod2:
# env
PCIDEVICE_OPENSHIFT_IO_MARVELL_OCTEON_DPDK=0000:81:02.4

Running testpmd on pod1:
dpdk-testpmd --proc-type=auto -a 0000:81:02.4,sdp_packet_mode=loop -l 0-1 --  --portmask=0x1 --rxq=1 --txq=1  --rxd=2048 --txd=2048 --port-topology=loop --nb-cores=1 -i
Running testpmd on pod2:
dpdk-testpmd --proc-type=auto -a 0000:81:02.5,sdp_packet_mode=loop -l 0-1 --  --portmask=0x1 --rxq=1 --txq=1  --rxd=2048 --txd=2048 --port-topology=loop --nb-cores=1 -i
Pushing packets:
testpmd> stop
Telling cores to stop...
Waiting for lcores to finish...

  ---------------------- Forward statistics for port 0  ----------------------
  RX-packets: 2240           RX-dropped: 0             RX-total: 2240
  TX-packets: 2272           TX-dropped: 0             TX-total: 2272
  ----------------------------------------------------------------------------

  +++++++++++++++ Accumulated forward statistics for all ports+++++++++++++++
  RX-packets: 2240           RX-dropped: 0             RX-total: 2240
  TX-packets: 2272           TX-dropped: 0             TX-total: 2272
  ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Done.

Fixed by #353
/close


[feature request] rbac role manifest

It's unclear what privileges the ingress controllers require.

Couldn't agree more @jhorwit2. Ideally all cluster services should ship sample RBAC roles, or at least a manifest of API resources and verbs they rely on.
It's the old chestnut from SELinux and App Armor, if the software authors don't publish the security requirements information, it requires every security conscious end-user to almost do a code audit to engineer a suitable least-privilege policy.
Is this something the automated build, test tools could do? E.g. monitor API calls during testing and spit out a manifest or a generated RBAC policy that is just what the tests need to run? Could be both a good leg-up for writing RBAC policies and useful as an audit tool.
As k8s matures it seems (to me) especially important for the core project to set a positive, secure-by-default example. Right now I see lots of early k8s users who are surprised and sometimes shocked when you tell them every one of their containers is getting, by default, an unrestricted, admin-strength Service Account token. Container break-outs make news, it is probably only a matter of time before k8s gets unnecessary negative PR for the default security stance.

Platforms like Tectonic are already doing RBAC by default and having to utilize a global very-permissive cluster role to make things like nginx-ingress and kube-lego work is kind of worrying. Would love to see more effort on this and eventually, an included nginx-rbac.yaml example for people deploying this, as well as documentation, which I can help with!

Maybe helpful as a starting point for people ending here after googling: This set of RBAC policies seems to work for me with the nginx controller 0.9.0-beta.2.
apiVersion: rbac.authorization.k8s.io/v1alpha1
kind: ClusterRole
metadata:
  name: ingress
rules:
- apiGroups:
  - ""
  - "extensions"
  resources:
  - configmaps
  - secrets
  - services
  - endpoints
  - ingresses
  - nodes
  verbs:
  - list
  - watch
- apiGroups:
  - "extensions"
  resources:
  - ingresses
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
- apiGroups:
  - "extensions"
  resources:
  - ingresses/status
  verbs:
  - update
---
apiVersion: rbac.authorization.k8s.io/v1alpha1
kind: RoleBinding
metadata:
  name: sa-binding
  namespace: nginx-ingress
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: admin
subjects:
  - kind: ServiceAccount
    name: default
    namespace: nginx-ingress
---
apiVersion: rbac.authorization.k8s.io/v1alpha1
kind: ClusterRoleBinding
metadata:
  name: ingress-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress
subjects:
  - kind: ServiceAccount
    name: default
    namespace: nginx-ingress

In my case all ingress-related resources are in the namespace 'nginx-ingress', similar to how https://github.com/jetstack/kube-lego/tree/master/examples/nginx/nginx is structured. I'm not yet sure whether I can actually nuke out the sa-binding ...

After some more painful debugging:
diff --git a/nginx-ingress-controller.yml b/nginx-ingress-controller.yml
index 06e3c77..29f2502 100644
--- a/nginx-ingress-controller.yml
+++ b/nginx-ingress-controller.yml
@@ -14,14 +14,23 @@ rules:
   resources:
   - configmaps
   - secrets
-  - services
   - endpoints
   - ingresses
   - nodes
+  - pods
   verbs:
   - list
   - watch
 - apiGroups:
+  - ""
+  resources:
+  - services
+  verbs:
+  - list
+  - watch
+  - get
+  - update
+- apiGroups:
   - extensions
   resources:
   - ingresses

Especially the 'update services' was hard to understand why, ultimately it is only needed when using named ports. See https://github.com/kubernetes/contrib/pull/766#issuecomment-210206052 for details.

Thank you for working this out @ankon ❤, I know how painful it is to reverse engineer permissions from code. I hope we can get this into the Ingress Controller documentation or examples - could you make a PR to add it and see what @aledbf thinks?

Especially the 'update services' was hard to understand why, ultimately it is only needed when using named ports. See kubernetes/contrib#766 (comment) for details.

The Ingress Controller needs update permissions to Services just to use annotations to duplicate the named port information from the Pod spec. This apparently stems from a pragmatic workaround for the limitation that EndPoints don't have a place to store the named port name specified in the Pod.
https://github.com/kubernetes/contrib/pull/766/commits/d455524b7bb09b8de95334c2d015e27e4096cf23#r59898626

This workaround also may be a broken approach? Since the annotation appears to assume that all Pods for a Service will use the same numeric port number for a particular named port. A Service label selector could easily encompass Pods from two Deployments, one where e.g. named port 'foo' is numeric port 80 and another where port 'foo' is numeric port 8080 (e.g. a blue/green deployment scenario). It might also break the Ingress during regular Deployment rolling updates, if the numeric port for a Pod named port is changed in the update. It seems like this Service annotation approach might fail in these cases? Or am I missing something @aledbf?

Or am I missing something @aledbf?

No. I need to review this and remove the annotation

could you make a PR to add it and see what @aledbf thinks?

:+1 to open a PR

fwiw, I am working on an audit2rbac (shamelessly taken from audit2allow) webhook authorizer that you can run that will gather API requests per user and generate RBAC roles that would cover the requests that were made. Adding that to your authz chain and exercising an app would make it easier to develop those roles (though rarely used permissions might still be tricky to trigger/capture)

Hi
Thanks very much for this!  I had to add the following to the ClusterRole in order to allow the ingress controller to read its configmap: -
- apiGroups:
  - ""
  resources:
  - configmaps
  verbs:
  - get

Just following up on this a bit.  I ran into this issue over the weekend and saw that there were mentions of creating a pull request for an example, but none had been made or accepted.  I created an example and pull request #747 based on what I found here and my own testing.  I'd appreciate any feedback on what I put together.

Question - is this closable now that we have an example?  Or does another step need to be taken?

Closing. Here there is an example to configure RBAC

fwiw, I am working on an audit2rbac (shamelessly taken from audit2allow)

For anyone interested, an initial version is now available at https://github.com/liggitt/audit2rbac

In case you are using helm charts and the nginx-ingress chart you just need to enable rbac as defined in the configuration section.
Just leaving this comment because it was not clear to me and I first tried to implement the example files on my own. Maybe it's helpful to someone else!


perf: Make use of System.arraycopy directly.

Motivation:
As ByteString is only copy Array[Byte], better to just use System.arraycopy
background: https://github.com/typelevel/cats-effect/pull/3963
Result:
Better performance.
Have no idea why a Vector[Byte] is backed by a Array[AnyRef]

@mdedetrich Would you like to take a look at this too, thanks.

@pjfanning @mdedetrich I just ran the benchmark.

@mdedetrich what do you think? I'm neutral to leaning slightly against. The perf gains seem small. In the end of the day, maybe we can delay till 1.1.0-M2.

Would you mind to explain how System.arraycopy and Arrays.copyOf relate (which the cats-effect change uses)?

Have no idea why a Vector[Byte] is backed by a Array[AnyRef]

Just out of interest, what does this relate to?

@jrudolph @pjfanning Let me explian:

I was looking at the change of CE, where they just call system.arrayCopy instead of Scala's Array.copy, and the Scala's Array.copy did some extra checking , but in this context, we knew it's always the Array[Byte], so we can just call Java's System.arrayCopy directly.

    if (srcClass.isArray && dest.getClass.isAssignableFrom(srcClass))
      java.lang.System.arraycopy(src, srcPos, dest, destPos, length)
    else
      slowcopy(src, srcPos, dest, destPos, length)

pekko/akka's ByteString extends the IndexedSeq and in the test, when we concating it with a Vector[Byte] which will cause problem, because the Vector[Byte] is actually backing by a Array[AnyRef], if I change the code to System.arrayCopy then an Array store exception will throw.  I asked on scala/bugs and that's not a design goal of Scala's stdlib. refs: https://github.com/scala/bug/issues/12931

    override def copyToArray[B >: Byte](dest: Array[B], start: Int, len: Int): Int = {
      // min of the bytes available to copy, bytes there is room for in dest and the requested number of bytes
      val toCopy = math.min(math.min(len, length), dest.length - start)
      if (toCopy > 0) {
        Array.copy(bytes, startIndex, dest, start, toCopy)
      }
      toCopy
    }

So I think we can just call the System.arraycopy where it fits. I asked on JVM group of china too, I was told the the CE's pr may not be faster on little bytes, and the results shows too, which is for SIMD,anyway, that's another story.

Thanks for the explanation, @He-Pin. Moving away from Array.copy seems like a good solution when you know it's an Array[Byte].
Regarding the integration of ByteString with scala collection, I think, this is one of its weakest parts, so I think it's fine not to be optimal here.

@pjfanning @mdedetrich it's 4% free lunch , why not?

@jrudolph Yes, I was shocked that in CE, they are not using the Array.copy method too, and that's why I come up with this .

Thanks for the detailed review.

@jrudolph Yes, I was shocked that in CE, they are not using the Array.copy method too, and that's why I come up with this .

Not sure what you mean?
It's pretty likely that java.util.Arrays.copyOf and System.arraycopy are both intrinsics, so the difference is likely small. There's the a chance that Arrays.copyOf is a slight bit faster, since it is a higher-level definition, so the JVM might be avoiding some bounds checks etc between creating the destination array and copying over the data (or at least with less analysis necessary in the JIT compiler).


Bybit - handle websocket response messages (coinmarginedfutures)

Authentication timed out for me a couple of times, which is why I looked more closely at the auth response messages
[WARN]  | EXCHANGE | 19/07/2023 15:54:03 | Bybit Asset Type coinmarginedfutures Received unhandled message on websocket: map[conn_id:... request:map[args:[...<PHONE_NUMBER>000 0xdeadbeef op:auth] ret_msg:error:request expired success:false]

Mercy Beaucoup😎