Assign users to group via API

As developer of external service which utilizes gate
I want to be able to assign users in my service to a group in gate
Gate should provide API for assigning and removing users from a group

@ajeygore, There are some services depends on Gate API to assign/remove user from group. It used to exist on nss_controller#add_user_to_group.
How can these external services do it now? should I create a new API?

Which services are those? They are using old auth token, So they need to use API token to do something. add_user_to_group shouldn't be used anymore, instead those service admins should be made group admins of those groups and they should come to gate and then manipulate, I don't think that services are doing that automatically. That's why I removed that from the nss_controller.

From what I understand, this service is operation portal (customer service and stuff)
it seems that their admins are not using gate ui at all, and the apps communicate behind the scene with gate to add user to groups using that API on nss_controller.

Okay, I shall discuss with them tomorrow about this

This issue has been communicated and the affected service will use gate user interface instead to add users to group.


cabal info doesn't work for local packages not published to hackage

I noticed packcheck cabal-new uses cabal info . to get the name of the package we're building. This is failing with:
Assertion failed
CallStack (from HasCallStack):
  assert, called at ./Distribution/Client/List.hs:427:3 in cabal-install-<IP_ADDRESS>-inplace:Distribution.Client.List

I believe cabal info just hits the global and user package databases, but not the inplace database in dist-newstyle/

Thanks for reporting. Do you have a pointer to the failing build? Is this in CI or on your local machine?
cabal info is used when building from source distribution and for install test. I can try not using it when these two options are disabled from the command line so that we have a workaround. This looks like a cabal bug, need to file an issue in upstream repo if not already there.

@mitchellwrosen see if #14 helps you.

@harendra-kumar This is on my local machine when running packcheck.sh cabal-new.
It may be a cabal bug - but I'm not totally sure if cabal info is supposed to give info about the local package. It's not clear from the output of cabal info --help
#14 looks like it may help, but there's still a || die "cabal info failed" that is triggered when cabal info . fails.

Ah, I just forgot to remove that. Let me fix it.

Fixed it, check it out now.

Here's the behavior I get now.
$ packcheck.sh cabal-new
...
Assertion failed
CallStack (from HasCallStack):
  assert, called at ./Distribution/Client/List.hs:427:3 in cabal-install-<IP_ADDRESS>-inplace:Distribution.Client.List
Error: Please use 'DISABLE_SDIST_BUILD=y' to avoid this issue.
Error: PACKAGE_FULL_NAME

$ DISABLE_SDIST_BUILD=y packcheck.sh cabal-new
...
Assertion failed
CallStack (from HasCallStack):
  assert, called at ./Distribution/Client/List.hs:427:3 in cabal-install-<IP_ADDRESS>-inplace:Distribution.Client.List
Error: Inconsistent package name [my-package] and package full name []
Error: PACKAGE_FULL_NAME

I pushed a fix for the error, try again. I hope it works this time.

It works! :)

@mitchellwrosen the issue may not be because the package is not on hackage. I just tested it with one of my packages that is not on hackage and it succeeds. Maybe it is something else in your cabal file that is making it fail? You may want to file a bug with cabal providing your cabal file that's making it fail.

Interesting, good to know. Thanks for the help regardless!

It is also possible that this is a bug in some specific version of cabal. I tested it with cabal-install version <IP_ADDRESS>. Which version is yours?

<IP_ADDRESS>

This version is not on hackage. I can see only <IP_ADDRESS> and then <IP_ADDRESS>. I tried cabal info with <IP_ADDRESS> and it succeeds on a package that is not on hackage.

@mitchellwrosen closing this as we have a workaround. You can file a bug on cabal-install if you see the problem even in the latest version.


Move PKCS #11 Mutual Auth Demo from LTS Dev branch

Move PKCS #11 Mutual Auth Demo from LTS Dev branch
Description
This change moves the PKCS #11 Mutual Auth demo. The demo has previously been reviewed, but the project needed to be updated to address changes to the folder structure.
Test Steps
Follow the demo instructions on freertos.org for the PKCS #11 mutual auth demo, and connect to AWS IoT Core.
Related Issue

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Nit: Maybe a better folder name for this demo would be MQTT_PKCS11_Mutual_Auth_Windows_Simulator because we might have to change the other demo names to coreMQTT_MQTT_Mutual_Auth_Windows_Simulator for consistency and that just doesn't look right. Although, this name may have been decided in the past already.

Thanks, this is a good thought. For now I prefer this. The other demos share a lot of code and I think it makes sense for them. In this case, I kept the PKCS mutual auth demo separate from the other ones, and is it has a lot more dependencies / more complex overall


Not able to get Credentials by calling CredentialProfileStoreChain when ProfilesLocation is null or empty

According to Accessing Credentials and Profiles in an Application, I do not need to set ProfilesLocation to retrieve Credential.  But I am only able to retrieve the Credential when I specific the ProfilesLocation.  Below is my code which is in a DLL Project.
        public CognitoAuthUserManagementService(string cognitoUserPoolId, string awsRegion)
        {
            _awsRegion = Amazon.RegionEndpoint.GetBySystemName(awsRegion);

            _cognitoUserPoolId = string.IsNullOrEmpty(cognitoUserPoolId) ? throw new SystemException(nameof(_cognitoClientProvider)) : cognitoUserPoolId;

            //var profileChain = new CredentialProfileStoreChain(@"C:\Users\hansl\.aws\credentials");
            var profileChain = new CredentialProfileStoreChain();

            if (profileChain.TryGetAWSCredentials("default", out AWSCredentials awsCredentials) == false)
            {
               // throw new Exception($"Not able to get AWSCredentials.");
            }

            _cognitoClientProvider = new AmazonCognitoIdentityProviderClient(awsCredentials, _awsRegion)
                              ?? throw new SystemException(nameof(_cognitoClientProvider));
        }

TryGetAWSCredentials() will always return false if I do not pass the shared credentials file path to CredentialProfileStoreChain constructor.
Even, I can specific the shared credentials file path in development, but what confuse me is how does this code work in production?  How should one get AWS credential profile in production in an Applicaiton?  I read that in production, an EC2 instance role credential will be used instead, but I do not see how this work with the code example of Configuring AWS Credentials
Expected Behavior
TryGetAWSCredentials() should return true even ProfilesLocation is null or empty
Current Behavior
TryGetAWSCredentials() will return false if ProfilesLocation is null or empty
Possible Solution

Steps to Reproduce (for bugs)
Call the above code from a windows console app
Context

How to get aws credentials without setting ProfilesLocation in production environment?
Your Environment

AWSSDK.Core version used: <IP_ADDRESS>
Service assembly and version used:
Operating System and version:  Windows 10 Pro Version	10.0.18362 Build 18362
Visual Studio version: 16.2
Targeted .NET platform:  .net core 3 preview 7

.NET Core Info
.NET Core SDK (reflecting any global.json):
 Version:   3.0.100-preview7-012821
 Commit:    6348f1068a

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.18362
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\3.0.100-preview7-012821\

Host (useful for support):
  Version: 3.0.0-preview7-27912-14
  Commit:  4da6ee6450

.NET Core SDKs installed:
  2.1.701 [C:\Program Files\dotnet\sdk]
  2.1.801 [C:\Program Files\dotnet\sdk]
  2.2.301 [C:\Program Files\dotnet\sdk]
  3.0.100-preview7-012821 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:

  Microsoft.AspNetCore.All 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.0.0-preview7.19365.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.0.0-preview7-27912-14 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.0.0-preview7-27912-14 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Project File
<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netcoreapp3.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Amazon.Extensions.CognitoAuthentication" Version="1.0.3" />
    <PackageReference Include="AWSSDK.CognitoIdentity" Version="<IP_ADDRESS>" />
    <PackageReference Include="AWSSDK.CognitoIdentityProvider" Version="<IP_ADDRESS>" />
  </ItemGroup>

</Project>

Hi @hanslai,
I'm not sure why this isn't working for you, but I can try to provide some guidance around it.  If what I say doesn't help, please provide more information regarding how you have tried to setup your credentials and what version of the Visual Studio Toolkit you are using, if any.
In the credentials configuration documentation that you previously linked, there are notes that should help you.  One thing that may be a potential problem is if @"C:\Users\hansl.aws\credentials" isn't resolved by %HOME%.aws\credentials wherever you are executing your code.  If you do have a file at %HOME%.aws\credentials with the proper format defined in the documentation, you should be getting credentials without having to define the location in your code.

First search the .NET credentials file for a profile with the specified name. If the profile isn't there, search %HOME%.aws\credentials. If the profile isn't there, search %HOME%.aws\config.

You may also want to verify that you haven't misconfigured credential-related environment variables or app settings.
Furthermore, there isn't a need to use the CredentialProfileStoreChain class in your code unless you want to create an abstraction between the NetSDKCredentialsFile (legacy, used in VS Toolkit and Powershell) and the AWSSharedCredentialsFile.
You could simply write new AmazonCognitoIdentityProviderClient(_awsRegion);
If your production environment is running in EC2, this should also work based on your IAM credentials and roles.

Credential and Profile Resolution
The AWS SDK for .NET searches for credentials in the following order and uses the first available set for the current application.
The client configuration, or what is explicitly set on the AWS service client.

BasicAWSCredentials that are created from the AWSAccessKey and AWSSecretKey AppConfig values, if they're available.

A credentials profile with the name specified by a value in AWSConfigs.AWSProfileName (set explicitly or in AppConfig).

The default credentials profile.

SessionAWSCredentials that are created from the AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN environment variables, if they're all non-empty.

BasicAWSCredentials that are created from the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables, if they're both non-empty.

IAM Roles for Tasks for Amazon EC2 Container Service (Amazon ECS) tasks.

EC2 instance metadata.

@klaytaybai Thanks for you help.
I found out that there is a C:\Users\hansl\AppData\Local\AWSToolkit\RegisteredAccounts.json in my user folder.  I tried so many things, I have no idea how or when that file was there and get created.  but once I delete that file.  just new AmazonCognitoIdentityProviderClient(_awsRegion); works fine now.
in fact, I was just use
new AmazonCognitoIdentityProviderClient(_awsRegion); at first, but it throw 503 Service Unavalilable Exception because of the RegisteredAccounts.json file was there.  That is why, I tried the CredentialProfileStoreChain class from Configuring AWS Credentials,
I am not sure what is the purpose of RegisteredAccounts.json file, is it safe to delete it?  and what is the purpose of that file if @"C:\Users<user>.aws\credentials" is already there?

Hi @hanslai.  I'm glad you were able to get it working.  I should have asked about that file too.  The RegisteredAccounts.json file is an encrypted credentials file that is specific to the .NET team's offerings and predates the cross-SDK SharedCredentialsFile found at ~/.aws/credentials or the config at ~/.aws/config.  It is created when setting profiles via the AWS Tools for Powershell or the AWS Visual Studio Toolkit.  In order to support backwards compatibility, these profiles have taken precedence.  Unfortunately this has resulted in many issues like yours, and we'll probably revisit whether we want to change how this is handled.  It is safe to delete the RegisteredAccounts.json file so long as you have your credentials backed up somewhere else and an alternative method for getting credentials in your projects, such as using the SharedCredentialsFile.
Now that you have deleted the RegisteredAccounts.json file, can you please try the simplified constructor new AmazonCognitoIdentityProviderClient(_awsRegion);?  I'd like to verify that it doesn't return that error for you.

Hi @klaytaybai thank you for you help.
just using  new AmazonCognitoIdentityProviderClient(_awsRegion) is working now.  Thanks.

Hi @klaytaybai I would also like to point of that the .net AWS document is kind of confusing and outdated especially with .net core.  Very often I spend hours search and reading document other then actual coding.  I believe if the document style is more clear and be more up to date, a lot more .net developer like myself will choice and enjoy AWS.  Thanks

Thanks for the feedback.  If you have more specific feedback about the documentation, please submit feedback via the buttons on the bottom of the documentation pages or create a GitHub issue for code-based documentation feedback.  We are ramping up our efforts to improve our documentation quality, so any customer feedback on documentation will help us focus our strategies and tasks.


Geo Search

Link to feature: https://meilisearch.productboard.com/feature-board/planning/features/5032241

Sorry @qdequele, my bad. I miss clicked an item inside product board.


(luksroot.nix) Add an option to request a pin when unlocking FIDO2 authenticator

Describe the bug
The NixOS module for LUKS partitions currently uses this line when unlocking a FIDO2 authenticator. This clearly works for the majority of cases, but unfortunately some authenticators require a separate pin for unlocking (e.g. yubikey 5). Acquiring this pin before unlocking can be as easy as adding the -P or --pin flag after fido2luks open. Therefore I am requesting that this option be added to the module.
Steps To Reproduce

Configure your LUKS partition as outlined in the NixOS manual (except adding -P after command for every fido2luks [command] and typing in the appropriate pin).
Reboot and enter the salt value if one has been configured.
Receive the following error:

Authentication Error { cause: FidoError(

This operating requires a PIN but none was provided.) }
No FIDO2 key found, falling back to normal open procedure.

Expected behavior
I expect there to be an option to require this pin in my NixOS configuration. Perhaps it could be named something like boot.initrd.luks.devices.<name>.fido2.requiresPin?

Describe the bug
The NixOS module for LUKS partitions currently uses this line when unlocking a FIDO2 authenticator. This clearly works for the majority of cases, but unfortunately some authenticators require a separate pin for unlocking (e.g. yubikey 5). Acquiring this pin before unlocking can be as easy as adding the -P or --pin flag after fido2luks open. Therefore I am requesting that this option be added to the module.
Steps To Reproduce
1. Configure your LUKS partition as outlined in the [NixOS manual ](https://nixos.org/manual/nixos/stable/#sec-luks-file-systems-fido2)(except adding -P after `command` for every `fido2luks [command]` and typing in the appropriate pin).

2. Reboot and enter the salt value if one has been configured.

3. Receive the following error:

Authentication Error { cause: FidoError(

This operating requires a PIN but none was provided.) }
No FIDO2 key found, falling back to normal open procedure.

Expected behavior
I expect there to be an option to require this pin in my NixOS configuration. Perhaps it could be named something like boot.initrd.luks.devices.<name>.fido2.requiresPin?

Hi there. I faced the same problem (and thought of the same solution).
I posted the fix here: https://github.com/peterwilli/nixpkgs/tree/fido2luks-requires-pin
However, I can't get my YubiKey 5 NFC to be recognized at boot. Using fido2luks in my running desktop to add a key works fine, for some reason.
I think that maybe it has to do that I had to use the unstable fido2luks to get it recognized in the running desktop. I'll start fiddling with that.
In the meantime, can others chip in and try it out? Also let me know if it's good for making a PR.

Just by talking about it in my previous comment, I managed to figure out I needed the unstable fido2luks, I managed to do that by downloading https://github.com/NixOS/nixpkgs/commit/5ebb1dca9bb as zip and copy the master fido2luks to that commit, and adding my requires-pin branch on top of it.
However, it still doesn't work! It turns out you don't have a tty on initrd. I have to update my branch to use read or something. See https://github.com/shimunn/fido2luks/issues/14#issuecomment-680178277 for more info on that matter.

I fixed it and got it running with my YubiKey 5 NFC. Anyone who wants to try it out for verification: https://github.com/NixOS/nixpkgs/pull/143981

In #149241, the device PIN is stored under /crypt-ramfs/passphrase, which can then be reused by other luks devices if boot.initrd.luks.reusePassphrases is true.

PR #149241 solves this same problem, however in #149241, the device PIN is stored under /crypt-ramfs/passphrase, which can then be reused by other LUKS devices if boot.initrd.luks.reusePassphrases is true.

I like your solution in where you use RamFS since it's sort of the same thing as what I do but reusable for next LUKS devices. I guess the answer as to which PR will be the best is if the ability to reuse a pin/passphrase weighs up to the guarantee a key will only be read once by one process at a time.
In any case it's cool to see how other people tackle this problem!

Hi all!
Just for information, since the last reply to this issue is almost a year old: is a solution to this problem still being worked on?
I would really like to add my SoloKey as a LUKS key, but the presence of the PIN seems to be a dealbreaker…

@Occhioverde
PR #149241 contains the most recent work regarding this feature request that I personally am aware of. Unfortunately, it doesn't seem like it has been updated very recently,  so I went ahead and dropped a similar message.
I probably can't help with any implementation, but this is definitely something I am still interested in.


Please add in some user-friendly articles

Why:
Closes [issue link]

What's being changed (if available, include any code snippets, screenshots, or gifs):

Check off the following:

[x] I have reviewed my changes in staging (look for the "Automatically generated comment" and click the links in the "Preview" column to view your latest changes).
[x] For content changes, I have completed the self-review checklist.

Writer impact (This section is for GitHub staff members only):

[x] This pull request impacts the contribution experience

[x] I have added the 'writer impact' label
[x] I have added a description and/or a video demo of the changes below (e.g. a "before and after video")

👋 @Saidimukasa Thanks for opening a PR! I'm not sure exactly what change you're hoping to make here, so I'm going to close this issue out.
In the future, please fill out the pull request template in the top comment in the pull request to help us understand the changes you're hoping to see.
Take a look at the contributing.md to learn how to contribute in this repo. You can open a discussion if you have any questions or check out issues with the help wanted label. :sparkles:

name: Merge Pull Request
uses<EMAIL_ADDRESS>


Build from conda 23.5.0 and constructor 3.4.3

Description

Checklist - did you ...

[ ] Add a file to the news directory (using the template) for the next release's release notes?

[ ] Add / update necessary tests?
[ ] Add / update outdated documentation?

any reason to not use constructor 3.4.4?

This PR started before 3.4.4 was available, I'll probably update this branch to conda 23.7.0 and constructor 3.4.5 (once ready).

@jezdez @dbast - This is now ready for review.


MT: Fiber::Context struct

The fiber context holds the current stack top pointer (for a saved stack), as well as a resumable flag that tells whether the fiber can be resumed (its context was fully saved) or is currently running, or is dead (its proc returned).
The resumable flag is required to prevent a fiber to be resumed while it's still running. This should usually not happen with the monothreaded scheduler, but still useful to find a buggy synchronisation primitive that would enqueue the same fiber multiple times (e.g. the current select implementation). This will be very frequent with a multithreaded scheduler where a thread B could try to resume a fiber that has just been enqueued by thread A but didn't fully store its context yet.
The fiber status, running or dead, is monitored using an @alive boolean. We can't set context.resumable to another value, because the scheduler eventually swaps the context to another fiber, which would overwrite context.resumable.
The scheduler now aborts with a fatal error whenever it would resume a running or dead fiber, which would otherwise lead to a segfault. It doesn't just ignore the fiber since to enqueue a fiber twice or a dead fiber is an error that must be fixed.

I added a third state to resumable an made it an atomic long. Weirdly enough, it didn't make the context switch slower, it may even be slightly faster :open_mouth:
The drawback is that we lost the @[Extern] annotation, because Atomic isn't allowed for extern structs.
The ASM still non atomically sets resumable, which I expect to be fine, since schedulers will atomically check resumable to be 1 then atomically transition it to 2, which are enough guarantees.

In the MT branch, the following segfaulted quickly. With the current patch, it didn't segfault anymore:
fiber = Fiber.new { loop { Fiber.yield } }
100.times { Crystal::Scheduler.enqueue(fiber) }
sleep 0

Note that adding @current_thread to Fiber, and atomically changing it wasn't as bad as I thought; roughly 20% slower than changing resumable to an atomic.

Also, can we use :preview_mt as a flag until the feature is stable enough?

With #7546 merged we can close this PR. That does not discard future improvements to the current state.


fix: Linting of charts

Fixes #87

@adityagajbhiye : Please review

@adityagajbhiye : Please review

Hi @adkumar1,
can you please also provide some details on this PR about the linting issue? It's not really visible, which of the changes fixes the lint part. Also, the PR seems to include changes that are unrelated to Chart linting. Please remove them from the PR

Hi,
Actually it was failing in test connection.
I think below file is not required which is applicable for test-connection. So this has been removed.
charts/orchestrator/templates/tests/test-connection.yaml

Hi, Actually workflow was failing in test connection. I think below file is not mandatorily required which is applicable for test-connection. So this has been removed. charts/orchestrator/templates/tests/test-connection.yaml

So this is not at all about linting, but testing. Could you please adjust this in your PR title?

Hi, Actually workflow was failing in test connection. I think below file is not mandatorily required which is applicable for test-connection. So this has been removed. charts/orchestrator/templates/tests/test-connection.yaml

So this is not at all about linting, but testing. Could you please adjust this in your PR title?

Updated the title


[backport] Cythonize memory hook

Backport of #722.
Note that this PR might break user code because  cupy.cuda.memory_hook.thread_local get unavailable from Python code. Although the API compatibility policy allows to remove thread_local because it is undocumented, it is advisable to use underscores to state that the name may be changed (e.g. use _thread_local instead of thread_local).

LGTM!


Upgrade duckdb to 0.9.2 and update dependencies

Running the upgrade script; also updated bindgen, arrow, and polars. Let me know if I should roll any dependency updates back.

Thanks for the PR. Would be great to get this in (plus #238 maybe 😎) and cut a new release. 🙏

@mlafeldt Since it moved under duckdb I'm not sure who approval and release access.
@wangfenjin Sorry to ping you, but are you still the best person to reach out to for PRs or is there someone under the DuckDB org that's taken over? Thank you again for starting this project 🙏

Thanks for the contribution. I no longer have the write permission for this repo, I’m sure someone will handle PR

Asked about code ownership in Discord.

Thanks! LGTM

Thanks for the contribution. I no longer have the write permission for this repo, I’m sure someone will handle the PR

Hi @wangfenjin, sorry about that, that was a mistake on my end. You should have write permissions and I now restored them.


no push access issue for homebrew prs

👋 it has been going for quite some time, I think it is time to fix it.
cc @alongir

@chenrui333 can you please advise if we need to move our fork outside of organization or providing write permissions for BrewTestBot  is enough?

@chenrui333  can you address?

providing write permissions for BrewTestBot is enough?

providing the bot write permission is good enough

similarly, is 72.3.83 a wrong tag?

@chenrui333

I gave write access for the pod but the invitation is pending for 2 days

@corest thanks!! 🙏 I am not managing the bot, so let's see how it goes with the new PRs. 👍

looks like still not working, https://github.com/Homebrew/homebrew-core/pull/199687

@chenrui333 the access to repo wasn't accepted. Not sure who is responsible for this, but without accepting the invite - it will keep failing


Update to Current Runtime Version (10.2.5), add automated build, and unit testing

Some open items from the repo review that are larger tasks

[x] Update to Current AGS .NET Runtime Released Version (10.2.5)
[x] Add msbuild configuration / steps
[x] Add unit tests and steps

See https://github.com/Esri/solutions-widgets-wpf if you need an example

v0.6.0 has been updated to use ArcGIS Runtime SDK .NET 10.2.5

@jmccausland - Will you be working on the last two items in this issue?

Fixed with pull request #61


Removed unused '[BlockStorage]' options

What this PR does / why we need it:
The BlockStorage options for the cloud provider were carried over from the legacy cloud provider and are no longer used in the new CSI'ful world. Their presence is confusing. Remove them.
Which issue this PR fixes(if applicable):
Special notes for reviewers:

Release note:

NONE

/ok-to-test

/override openstack-cloud-controller-manager-e2e-conformance-test

/test pull-cloud-provider-openstack-check

/test pull-cloud-provider-openstack-check

/test openstack-cloud-csi-cinder-e2e-test

Even though , in CSI we do support some options, but that are added separately , https://github.com/kubernetes/cloud-provider-openstack/blob/master/pkg/csi/cinder/openstack/openstack.go#L76 . So this change seems good to me. Thanks.

gate failure would be fixed by https://github.com/kubernetes/cloud-provider-openstack/pull/1817 , please retest once that PR is merged.

/test pull-cloud-provider-openstack-check

/lgtm

Thanks.
/lgtm
/approve

/override openstack-cloud-controller-manager-e2e-conformance-test

@ramineni Looks like whatever script that openstack-cloud-controller-manager-e2e-conformance-test job is supposed to run doesn't exist? From the logs:
 + WRAPPED_COMMAND_PID=202
+ wait 202
+ ./tests/ci-occm-e2e-conformance.sh
/usr/local/bin/runner.sh: line 114: ./tests/ci-occm-e2e-conformance.sh: No such file or directory

There is a tests/ci-occm-e2e.sh script. Perhaps that's what it's supposed to run?

/override openstack-cloud-controller-manager-e2e-conformance-test

@ramineni Looks like whatever script that openstack-cloud-controller-manager-e2e-conformance-test job is supposed to run doesn't exist? From the logs:
 + WRAPPED_COMMAND_PID=202
+ wait 202
+ ./tests/ci-occm-e2e-conformance.sh
/usr/local/bin/runner.sh: line 114: ./tests/ci-occm-e2e-conformance.sh: No such file or directory

There is a tests/ci-occm-e2e.sh script. Perhaps that's what it's supposed to run?

hey , yes . The job is being added in other PR and not merged yet, so its happening.


fix: unify dropping management in modern bpf probe

What type of PR is this?
/kind bug
Any specific area of the project related to this PR?
/area driver-modern-bpf
/area libscap-engine-modern-bpf
Does this PR require a change in the driver versions?
No
What this PR does / why we need it:
This PR carries 2 fixes:

allow computing the sum of all drops (full ring buffer + event too large) like in all our drivers.
unify the logic between our drivers. In our drivers n_evts is the number of events that the driver should send to userspace according to its configuration. So this counter includes also the number of drops (for example due to full buffers). Before this patch, the modern probe didn't consider drops inside the n_evts.

Which issue(s) this PR fixes:
Special notes for your reviewer:
Does this PR introduce a user-facing change?:
NONE

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Makes sense.
/milestone 0.10.0.

@FedeDP: The provided milestone is not valid for this repository. Milestones in this repository: [0.11.0, next-driver]
Use /milestone clear to clear the milestone.

In response to this:

Makes sense.
/milestone 0.10.0.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, FedeDP
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97,FedeDP]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: 301034266a2eeb516a658e598a34e2ea57d4c6d2

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, FedeDP, Molter73
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97,FedeDP,Molter73]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

This needs to be backported to the 0.10.0 release branch.

/milestone 0.10.1


post-deploy hook with curl does not work because curl is not available

Hi guys
Somehow I get a curl: not found error with the basic post-deploy example. Seems like the base image of ruby does not contain curl, and the post-deploy hook is run from within the kamal image and not the host server.
My workaround right now is to extend the kamal image and install curl afterwards.
Is there another way to use curl in post-deploy hook? Or is this a bug?

I'm not talking about curl/wget ON the server, but IN the kamal docker image. That's the current place where the post-deploy hook is executed.

Ensuring the tools required by your hooks are available in the image you're using is an exercise left to the programmer. Adding curl as a package as part of your Dockerfile is the correct solution.


Fixed glossary article not showing, when opening direct link on mobile devices

I fixed the issue with glossary articles not showing, when opening a direct link to an article on mobile devices as mentioned in Issue #2914.

Good morning @MikeMcC399, thanks for reporting the issue and reviewing!


Support of external IP address mapping

Summary
When you want to host TURN server on AWS EC2 for instance, the instance can have a private IP address, but not public address directly. It allows us to have an associated public IP address (or Elastic IP, etc, there will be a NAT in betwee), but TURN server does not know about the public IP address at all. Since it is TURN server's responsibility to report the public IP address to the TURN client, there need to be a way to tell the associated public IP address to the TURN server.
Motivation
To make pion/turn available for real use cases.
Describe alternatives you've considered
Tell the external IP address via:

command line option
environment variable

Additional context
coturn, for instance, have -X option for this purpose.

@Sean-Der Can we add this to the 2.2.0 project? If this, plus #82 is supported, I'd switch my TURN server to pion/turn. (how do I add this ticket to the project?)


vxlan mode may miss vtepMac for windows

Expected Behavior

vtepMac may miss for windows when updating to etcd.
Current Behavior

vtepMac may miss for windows when updating to etcd.
Possible Solution

When updating to etcd, we should check if vtepMap is "", if empty, we should retry.
Steps to Reproduce (for bugs)

Context

Your Environment

Flannel version:
Backend used (e.g. vxlan or udp):
Etcd version:
Kubernetes version (if used):
Operating System and version:
Link to your project (optional):

/assign


Build wrapping on top of wrap_predicate/4

Current scheme uses tricky rewriting/expansion of predicate goals inside other predicate bodies. Using the built-in wrap_predicate/4 should make it a bit more reliable.
Link to documentation:
https://www.swi-prolog.org/pldoc/doc/_SWI_/library/prolog_wrap.pl
Discussion from SWI Discourse:
https://swi-prolog.discourse.group/t/wrapping-predicates/668

I've run into a problem when a predicate is used multiple times in a clause - the same "aux" predicate is generated for each call, so there's a "discontiguous predicate" warning, not to mention the duplicate definitions which will cause unnecessary choice points (which will get pruned away at run-time by the if-then-else). Using wrap_predicate solves this problem also.

I pusblished version 1.0.0 that build on top of wrap_predicate. Everything is backwards compatible except exact error terms that now miss source location. Source location is reported by the built-in error reporting now.

One catch with using wrap_predicate -- it won't wrap some imported predicates, such as `aggregate:aggregate_all/3'.
I also get warnings such as Local definition of pykythe:base64_term/2 overrides weak import from pykythe_utils ... I'm guessing that this is because I put all the rdet/1 declarations in my main file and they now be in the places where the predicates are defined.


Add delivery date column

Addition of the a delivery_date column in the backend Azure SQL Database

delivery date added


[Automation] Generate Fluent Lite from Swagger app#package-2024-03

[Automation] Generate Fluent Lite from Swagger app#package-2024-03

API change check
APIView has identified API level changes in this PR and created following API reviews.
com.azure.resourcemanager:azure-resourcemanager-appcontainers


logs from client-go are not logged to stdout

Is this a BUG REPORT or FEATURE REQUEST?:

Uncomment only one, leave it on its own line:

/kind bug

/kind enhancement

What happened:
Increasing the verbosity level of virt-launcher does not increase the log verbosity for our structured logger and for client-go but the client-go logs are not written to stdout unless --logtostderr is not supplied too.
What you expected to happen:
That we wrap glog in our codbase and only write logs via our structured logger.

Have a POC. Will create a PR soon.


Bark_Push doesn't work with self-built Bark server

Qinglong version
2.15.1
Steps to reproduce
export BARK_PUSH="https://mybark.domain.name:port/abcdefg" as BARK_PUSH variable, then run a .js file, the running results couldn't be sent to the bark client.
What is expected?
The bark client could receive the proper msg.
What is actually happening?
Here is the error report:
RequestError: getaddrinfo ENOTFOUND mybark.domain.name
at ClientRequest. (/ql/data/scripts/node_modules/.pnpm/got@11.8.5/node_modules/got/dist/source/core/index.js:970:111)
at Object.onceWrapper (node:events:628:26)
at ClientRequest.emit (node:events:525:35)
at origin.emit (/ql/data/scripts/node_modules/.pnpm/@szmarczak+http-timer@4.0.6/node_modules/@szmarczak/http-timer/dist/source/index.js:43:20)
at TLSSocket.socketErrorListener (node:_http_client:481:9)
at TLSSocket.emit (node:events:513:28)
at emitErrorNT (node:internal/streams/destroy:151:8)
at emitErrorCloseNT (node:internal/streams/destroy:116:3)
at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:111:26) {
code: 'ENOTFOUND',
timings: {
start:<PHONE_NUMBER>926,
socket:<PHONE_NUMBER>927,
lookup:<PHONE_NUMBER>936,
connect: undefined,
secureConnect: undefined,
upload: undefined,
response: undefined,
end: undefined,
error:<PHONE_NUMBER>937,
abort: undefined,
phases: {
wait: 1,
dns: 9,
tcp: undefined,
tls: undefined,
request: undefined,
firstByte: undefined,
download: undefined,
total: 11
}
}
}
System Info
No response
Any additional comments?
When I try to use Python (notify.py) to send msg, it worked fine. Cause I modified the notify.py file manually, writing the variable content into the file, not as a environment variable. So does it have relationship with that? (Mean that I need to write the variable manually into the SendNotify.js )

使用最新镜像，检查你的bark服务是否和青龙容器相通


groups, talk: group links render with overlapping text

I'm not sure these have to be as small as they currently are - if we made them more like apple iOS references in text (more like a large rectangle) we could have a nice group image and all the info without it being squished like this.

You saw this in the Gallery context, yes?

I've seen it in gallery and in threads

Fixed in #1991


How can we transition from conventional OpenAI API calls to a Langchain-based approach?

Describe the issue
Most of the OpenAI API-based applications have shifted from conventional API calls to the Langchain approach. It provides a better solution for maintaining the context of the provided files in the OpenAI model and directs all queries solely to the files within the context. We are still using the conventional API calls.
What needs investigating
It needs to be investigated that whether we can shift from conventional API calls to a Langchain based solution in our API calls or not.
Additional context
LangChain is a framework designed to simplify the creation of applications using large language models. As a language model integration framework, LangChain's use-cases largely overlap with those of language models in general, including document analysis and summarization, chatbots, and code analysis.

This change can easily be integrated in the project. This change will also remove the need to ask the first question attached with the initial context. This change will also increase the relevancy of the answers. Secondly, we can get rid of splitting the files into 4096 characters and running through the loop.


BackPropagation Algorithm

Hi,
It looks like in the algorithm for backprop the variable output_layer is used before it's defined in the hidden_deltas list comprehension.
Also, do you have any tips for extending the algorithm for multi-layer networks?

Oops, you're right, it's not ever defined, I guess it should probably just be replaced with network[-1].
My sense is that people usually stick with just one hidden layer, see, e.g.:
http://stats.stackexchange.com/questions/181/how-to-choose-the-number-of-hidden-layers-and-nodes-in-a-feedforward-neural-netw

OK, it's fixed now.  Thanks!


Show logs and metrics of the log analytics pipelines

Describe the feature
As a customer, I can create the log analytics pipelines through the CLO console. The pipeline is composed by multiple AWS services. I am expecting the pipeline can show more details, such as, the Lambda concurrency, the SQS metrics. So i can validate if my log ingestion is working well.
Use Case

Diagnostic. As a customer, I can leverage the metrics and logs to figure out why logs are not ingested into the OpenSearch cluster.

Proposed Solution
Add monitoring, logs panel to the detail view of log analytics pipelines.

Other Information
No response
Solution version used
v1.0.0

new feature in v2


refactor(@embark/basic-pipeline): move module into own package

TODO: The embark eject-webpack command needs to be updated. @michaelsbradleyjr - do you have suggestions as to how we could port this over? We could assume that the basic-pipeline plugin will be installed, and then read the embark config and overrides from the file system, however this feels like we are sort adding a dependency to a plugin, which is not right. Any suggestions?

@emizzle the tslint.json and tsconfig.json files in packages/plugins/basic-pipeline should be removed, as they're not used in this new package.

do you have suggestions as to how we could port this over? We could assume that the basic-pipeline plugin will be installed, and then read the embark config and overrides from the file system, however this feels like we are sort adding a dependency to a plugin, which is not right. Any suggestions?

Quite tricky. I think the only good solution would be for the plugin to expose a command to the cli, but at present embark doesn't have a facility for that, i.e. everything's hardwired in cmd and cmd_controller of packages/embark.
Until we've implemented the ability for plugins to expose commands to the cli, any solution will be an unpleasant intertwining. For example: packages/embark cmd_controller could "look up" using find-up package and build the paths to the babel config and overrides inside embark-basic-pipeline/dist. In fact, that's the only decent solution I can think of for the time being.

@michaelsbradleyjr - I agree that the best solution would be to allow plugins to expose CLI commands for embark, though that may be difficult as plugins are loaded last, but that's a problem we can tackle in the future.
This begs the question -- do we need to be too concerned with ejecting webpack now that the pipeline is in it's own plugin? Maybe it could be a matter of simply updating the documentation?


Composer autoload fix

This will allow autoloading PDOWrapper class.

Awesome, thank for the contrib @tectiv3!!


Complete repo setup

We never finished getting this repo setup
Also bumping the Docker image to Python 3.12 in this PR
Closes #10
Closes #15

At this point, this branch so far out of date that any changes should be made in a new PR.


"modelProvider": true,
        "name": "Otto8",
        "source": {
          "lineNo": 1,
          "location": "/otto8-tools/otto8-model-provider/tool.gpt"
        },
        "workingDir": "/otto8-tools/otto8-model-provider"
      },
      "toolCategory": "provider",
      "toolResults": 0,
      "type": "callFinish",
      "usage": {

      }
    }
  },
  "spec": {
    "synchronous": true,
    "threadName": "t1-ks1fjxp4",
    "input": "[![Acorn Labs](https://www.acorn.io/wp-content/uploads/2024/10/acorn_logo_h_5b9f9fcaf6.svg)](https://www.acorn.io/)\n\n![](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2020%2014'%3E%3C/svg%3E)\n\nMenu Close\n\n- [Resources](https://www.acorn.io/resources/)\n  \n  - [Resources](https://www.acorn.io/resources/)\n    \n    - [![ ](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2022%200'%3E%3C/svg%3E)\n      \\\n      Tutorials](/resources/tutorials)\n    - [![ ](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2022%200'%3E%3C/svg%3E)\n      \\\n      Blog](/resources/blog)\n    - [![ ](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2022%200'%3E%3C/svg%3E)\n      \\\n      Tools](http://tools.gptscript.ai/)\n    - [![ ](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2022%200'%3E%3C/svg%3E)\n      \\\n      GitHub](https://github.com/gptscript-ai/gptscript)\n    - [![ ](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2022%200'%3E%3C/svg%3E)\n      \\\n      Events](/events)\n    - [![ ](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2022%200'%3E%3C/svg%3E)\n      \\\n      Docs](https://docs.gptscript.ai/)\n    - [![ ](data:image/svg+xml,%3Csvg%20xmlns='http://www.w3.org/2000/svg'%20viewBox='0%200%2022%200'%3E%3C/svg%3E)\n      \\\n      Discord](https://discord.com/invite/9sSf4UyAMC)\n  - [Learning Center](/resources/learning-center)\n    \n    - [Models](https://www.acorn.io/resources/learning-center/category/models/)\n      \n      - [OpenAI GPT-4](https://www.acorn.io/resources/learning-center/openai/)\n      - [Anthropic Claude](https://www.acorn.io/resources/learning-center/anthropic-claude/)\n      - [Cohere AI](https://www.acorn.io/resources/learning-center/cohere-ai/)\n      - [Google Gemini](https://www.acorn.io/resources/learning-center/google-gemini/)\n      - [Meta LLaMa](https://www.acorn.io/resources/learning-center/meta-llama/)\n      - [Mistral](https://www.acorn.io/resources/learning-center/mistral-ai/)\n      - [Mistral 7B](https://www.acorn.io/resources/learning-center/mistral-7b/)\n    - [Tools and Topics](https://www.acorn.io/resources/learning-center/category/tools-and-topics/)\n      \n      - [Fine-Tuning LLMs](https://www.acorn.io/resources/learning-center/fine-tuning-llm/)\n      - [Generative AI](https://www.acorn.io/resources/learning-center/generative-ai-applications/)\n      - [AI Agents](https://www.acorn.io/resources/learning-center/ai-agents/)\n      - [Claude API](https://www.acorn.io/resources/learning-center/claude-api/)\n      - [Gemini API](https://www.acorn.io/resources/learning-center/google-gemini-api/)\n      - [LLM Application Development](https://www.acorn.io/resources/learning-center/llm-application-development/)\n      - [LLM Security](https://www.acorn.io/resources/learning-center/llm-security/)\n      - [Prompt Engineering](https://www.acorn.io/resources/learning-center/prompt-engineering/)\n    - [Use Cases](https://www.acorn.io/resources/learning-center/category/use-cases/)\n      \n      - [Retrieval Augmented Generation (RAG)](https://www.acorn.io/resources/learning-center/retrieval-augmented-generation/)\n      - [AI Copilots](https://www.acorn.io/resources/learning-center/ai-copilots/)\n      - [AI Image](https://www.acorn.io/resources/learning-center/ai-image-generation/)\n      - [AI Video Generators](https://www.acorn.io/resources/learning-center/ai-video-generators/)\n      - [AI Summarization](https://www.acorn.io/resources/learning-center/ai-summarization/)\n      - [Code Interpreter](https://www.acorn.io/resources/learning-center/code-interpreter/)\n    \n    [Explore All Articles](/resources/learning-center)\n- [Docs](https://docs.gptscript.ai/)\n- [Tools](http://tools.gptscript.ai/)\n- [Discord](https://discord.com/invite/9sSf4UyAMC)\n- [GitHub](https://github.com/gptscript-ai/gptscript)\n- [Company](https://www.acorn.io/about-us/)\n\n[Try GPTScript](https://github.com/gptscript-ai/gptscript?tab=readme-ov-file#1-install-the-latest-release)\n\n[Request a Demo](/contact)\n\n[Learning Center](/resources/learning-center)\n\n# Fine-Tuning LLMs: Top 6 Methods, Challenges and Best Practices\n\n###### June 6, 2024 by acorn labs\n\n## What Does It Mean to Fine-Tune LLMs?\n\nFine-tuning Large Language Models (LLMs) involves adjusting pre-trained models on specific datasets to enhance performance for particular tasks. This process begins after general training ends. Users provide the model with a more focused dataset, which may include industry-specific terminology or task-focused interactions, with the objective of helping the model generate more relevant responses for a specific use case.\n\nFine-tuning allows the model to adapt its pre-existing weights and biases to fit specific problems better. This results in improved accuracy and relevance in outputs, making LLMs more effective in practical, specialized applications than their broadly trained counterparts. While fine-tuning can be highly computationally intensive, new techniques like Parameter-Efficient Fine-Tuning (PEFT) are making it much more efficient and possible to run even on consumer hardware.\n\nFine-tuning can be performed both on open source LLMs, such as Meta LLaMA and Mistral models, and on some commercial LLMs, if this capability is offered by the model’s developer. *(Learn more in our detailed guide to [fine tuning Llama 2](https://www.acorn.io/resources/learning-center/fine-tuning-llama-2))*. For example, OpenAI allows fine tuning for GPT-3.5 and GPT-4. This is part of an extensive series of guides about [machine learning](https://www.aporia.com/learn/machine-learning-model/machine-learning-models-use-cases-operations/).\n\n## Fine-Tuning vs. Embeddings vs. Prompt Engineering\n\n**Fine-tuning** is a method where a pre-trained model is further trained (or fine tuned) on a new dataset specific to a particular task. This technique involves adjusting the weights across all layers of the model, based on the new data. It allows the model to specifically cater to nuanced tasks and often results in higher performance for specialized applications.\n\n**Embeddings** refer to dense vector representations of words or phrases, which are typically obtained during the initial training of a model. Instead of adjusting the entire model, embeddings can be extracted and used as static input features for various downstream tasks. This approach does not modify the pre-trained model but leverages the learned representations. It’s generally faster and less resource-intensive than fine-tuning.\n\n**Prompt engineering** is another way to adjust LLMs to specific tasks. Adding more context, examples, or even entire documents and rich media, to LLM prompts can cause models to provide much more nuanced and relevant responses to specific tasks. Prompt engineering is considered more limited than fine-tuning, but is also much less technically complex and is not computationally intensive.\n\n***Learn more in our detailed guide to LLM fine tuning vs embedding (coming soon)***\n\n## When Does Your Business Need a Fine-Tuned Model?\n\nHere are a few primary use cases for fine-tuned LLMs:\n\n### Specificity and Relevance\n\nA fine-tuned model excels in providing highly specific and relevant outputs tailored to your business’s unique needs. Unlike general models, which offer broad responses, fine-tuning adapts the model to understand industry-specific terminology and nuances. This can be particularly beneficial for specialized industries like legal, medical, or technical fields where precise language and contextual understanding are crucial.\n\n### Improved Accuracy\n\nFine-tuning significantly enhances the accuracy of a language model by allowing it to adapt to the specific patterns and requirements of your business data. When a model is fine-tuned, it learns from a curated dataset that mirrors the particular tasks and language your business encounters. This focused learning process refines the model’s ability to generate precise and contextually appropriate responses, reducing errors and increasing the reliability of the outputs.\n\n### Data Privacy and Security\n\nIn many industries, maintaining data privacy and security is paramount. By fine-tuning a language model on proprietary or sensitive data, businesses can ensure that their unique datasets are not exposed to third-party risks associated with general model training environments. Fine-tuning can be conducted on-premises or within secure environments, keeping data control in-house.\n\n### Customized Interactions\n\nBusinesses that require highly personalized customer interactions can significantly benefit from fine-tuned models. These models can be trained to understand and respond to customer queries with a level of customization that aligns with the brand’s voice and customer service protocols. For instance, a fine-tuned model in a retail business can understand product-specific inquiries, offer personalized recommendations, understand company policies, and handle complex service issues more effectively than a general model.\n\n## Top 6 LLM Fine-Tuning Methods\n\nHere are some of the ways that large language models can be fine tuned.\n\n### 1. Instruction Fine-Tuning\n\nInstruction fine-tuning involves training a model using examples that demonstrate how it should respond to specific queries. For instance, to improve summarization skills, a dataset with instructions like \"summarize this text\" followed by the actual text is used.\n\nThis method helps the model learn to follow specific instructions and improve its performance in targeted tasks by understanding the expected outputs from given prompts. This approach is particularly useful for enhancing the model’s ability to handle various task-specific instructions effectively.\n\n### 2. Parameter-Efficient Fine-Tuning (PEFT)\n\nPEFT updates only a small subset of the model’s parameters during training, significantly reducing the memory and computational requirements compared to full fine-tuning. Techniques like LoRA (Low-Rank Adaptation) and QLoRA (Quantized Low-Rank Adaptation) can reduce the number of trainable parameters by thousands of times.\n\nThis method helps manage hardware limitations and prevents the phenomenon of ‘catastrophic forgetting’, maintaining the model’s original knowledge while adapting to new tasks. By focusing on specific components, PEFT makes the fine-tuning process more efficient and cost-effective, especially for large models.\n\n### 3. Task-Specific Fine-Tuning\n\nTask-specific fine-tuning focuses on adjusting a pre-trained model to excel in a particular task or domain using a dedicated dataset. This method typically requires more data and time than transfer learning but achieves higher performance in specific tasks, such as translation or sentiment analysis.\n\nDespite its effectiveness, it can lead to catastrophic forgetting, where the model loses proficiency in tasks it was previously trained on. However, by tailoring the model to specific requirements, task-specific fine-tuning ensures high accuracy and relevance for specialized applications.\n\n### 4. Transfer Learning\n\nTransfer learning leverages a model trained on a broad, general-purpose dataset and adapts it to specific tasks using task-specific data. This method is useful when data or resources are limited, as it builds on the knowledge already embedded in the pre-trained model, offering improved learning rates and accuracy with less training time compared to training from scratch.\n\nTransfer learning enables the efficient reuse of models like GPT or BERT for new applications, providing a strong foundation for further customization.\n\n***Learn more in our detailed guide to fine tuning vs transfer learning (coming soon)***\n\n### 5. Multi-Task Learning\n\nMulti-task learning trains a model on a dataset containing examples for multiple tasks, such as summarization, code translation, and entity recognition. This approach helps the model improve performance across different tasks simultaneously and avoids catastrophic forgetting.\n\nHowever, this method requires a large amount of diverse data, which can be challenging to assemble. The comprehensive training enables the model to handle various tasks proficiently, making it suitable for environments where versatile performance is necessary.\n\n### 6. Sequential Fine-Tuning\n\nSequential fine-tuning adapts a model to a series of related tasks in stages. For example, a general language model might first be fine-tuned for medical language and subsequently for pediatric cardiology. This method ensures the model retains its performance across various specialized domains, building on each successive fine-tuning step to refine its capabilities further.\n\nBy sequentially adapting to increasingly specific datasets, the model can achieve high proficiency in niche areas while maintaining a broad understanding of the general domain.\n\n## What Is Retrieval Augmented Generation (RAG)?\n\nRetrieval Augmented Generation (RAG) is a technique that combines natural language generation with information retrieval to enhance a model’s outputs with up-to-date and contextually relevant information. RAG integrates external knowledge sources, ensuring that the language model provides accurate and current responses. This method is particularly useful for tasks requiring precise, timely information, as it allows continuous updates and easy management of the knowledge base, avoiding the rigidity of traditional fine-tuning methods.\n\nRAG systems can dynamically retrieve information during generation, making them highly adaptable to changing data and capable of delivering more relevant and informed outputs. This technique is beneficial for applications where the accuracy and freshness of information are critical, such as customer support, content creation, and research. By leveraging RAG, businesses can ensure their language models remain current and provide high-quality responses that are well-grounded in the latest information available.\n\nNotable examples of the use of RAG are the [AI Overviews](https://blog.google/products/search/generative-ai-google-search-may-2024/) feature in Google search, and [Microsoft Copilot in Bing](https://www.bing.com/chat?form=CONVRD), both of which extract data from a live index of the Internet and use it as an input for LLM responses.\n\n## How to Choose a Pre-Trained Model for Fine-Tuning\n\nHere’s an overview of the process of identifying an existing LLM for fine-tuning.\n\n### Define the Task\n\nBefore fine-tuning, clearly define the model’s intended task. Understanding the task’s requirements helps in selecting a model whose pre-trained capabilities align closely with the end objectives. For example, it may involve classification, regression, or generative tasks.\n\nAn accurate task definition also aids in determining the necessary data scope for model fine-tuning. This can prevent potential performance degradation due to underfitting or overfitting during the fine-tuning phase.\n\n### Understand the Model Architecture\n\nGet familiar with different model architectures to select the most suitable one for your task. Each architecture has strengths and limitations based on its design principles, layers, and the type of data it was initially trained on.\n\nUnderstanding these characteristics can significantly impact the success of fine-tuning, as certain architectures might be more compatible with the nature of your specific tasks.\n\n### Assess Strengths and Weaknesses\n\nEvaluate the strengths and weaknesses of the model options. Some models may excel at handling text-based tasks while others may be optimized for voice or image recognition tasks. Standardized benchmarks, which you can find on LLM leaderboards, can help compare models on parameters relevant to your project.\n\nAdditionally, consider the model’s performance trade-offs such as accuracy, processing speed, and memory usage, which can affect the practical deployment of the fine tuned model in real-world applications.\n\n### Match with Task Requirements\n\nEnsure the pre-trained model’s capabilities match the demands of the task. This involves comparing the model’s training data, learning capabilities, and output formats with what’s needed for your use case. A close match between the model’s training conditions and your task’s requirements can enhance the effectiveness of the re-training process.\n\n***Related content: Read our guide to fine tuning LLM tutorial (coming soon)***\n\n## Challenges and Limitations of LLM Fine-Tuning\n\nHere are some of the challenges involved in fine-tuning large language models.\n\n### Overfitting\n\nOverfitting occurs when a model is trained so closely to the nuances of a specific dataset that it performs exceptionally well on that data but poorly on any data it hasn’t seen before. This is particularly problematic in fine-tuning because the datasets used are generally smaller and more specialized than those used in initial broad training phases.\n\nSuch datasets can include rare or unique examples that do not represent a broader population, causing the model to learn these as common features. Overfitting results in a model that lacks the ability to generalize, which is critical for practical applications where the input data may vary significantly from the training data.\n\n### Catastrophic Forgetting\n\nCatastrophic forgetting refers to a situation where a neural network, after being fine-tuned with new data, loses the information it had learned during its initial training. This challenge is especially significant in the fine-tuning of LLMs because the new, task-specific training can override the weights and biases that were useful across more general contexts.\n\nFor example, a model trained initially on a broad range of topics might lose its ability to comprehend certain general concepts if it is intensely retrained on a niche subject like legal documents or technical manuals.\n\n### Bias Amplification\n\nBias amplification is when inherent biases in the pre-trained data are intensified. During fine-tuning, a model may not only reflect but also exacerbate biases present in the new training dataset.\n\nFor example, if a dataset for fine-tuning an LLM on job application reviews contains biases against certain demographic groups, the model might amplify this bias, leading to discriminatory behavior in automated screening processes. This underscores the need for careful selection of datasets to avoid reinforcing harmful stereotypes or unfair practices in model outputs.\n\n### Hyperparameter Tuning Complexity\n\nHyperparameters, such as learning rate, batch size, and the number of epochs during which the model is trained, have a major impact on the model’s performance. These parameters need to be carefully adjusted to strike a balance between learning efficiently and avoiding overfitting. The optimal settings for hyperparameters vary between different tasks and datasets.\n\nThe process of identifying the right hyperparameter settings is time-consuming and computationally expensive, requiring extensive use of resources to run numerous training cycles. However, standardized methods, frameworks, and tools for LLM tuning are emerging, which aim to make this process easier.\n\n## LLM Fine-Tuning Best Practices\n\nHere are some of the measures you can take to ensure an effective LLM fine-tuning process.\n\n### Start with a Small Model\n\nBeginning with a smaller model can simplify the fine-tuning process. Smaller models require less computational power and memory, allowing for faster experimentation and iteration. This approach is particularly beneficial when resources are limited. Once the process is optimized on a smaller scale, the insights gained can be applied to fine-tune larger models.\n\n### Experiment with Different Data Formats\n\nExperimenting with various data formats can significantly enhance the effectiveness of fine-tuning. By including diverse input types—such as structured data, unstructured text, images, or even tabular data—models can learn to handle a broader range of real-world scenarios. This helps build versatility in the model’s responses, ensuring it performs well across different contexts and input variations.\n\n### Start with Subsets of Data\n\nStarting with fine-tuning on smaller subsets of the dataset allows for quicker iterations and helps identify potential issues early in the training process. By gradually scaling up to the full dataset, you can fine-tune hyperparameters and make necessary adjustments without expending excessive resources.\n\n### Ensure the Dataset Is High-Quality\n\nThe dataset should be representative of the specific task and domain to ensure the model learns the relevant patterns and nuances. High-quality data minimizes noise and errors, allowing the model to generate more accurate and reliable outputs. Investing time in curating and cleaning the dataset ensures improved model performance and generalization capabilities.\n\n### Use Hyperparameters to Optimize Performance\n\nHyperparameter tuning is vital for optimizing the performance of fine-tuned models. Key parameters like learning rate, batch size, and the number of epochs must be adjusted to balance learning efficiency and overfitting prevention. Systematic experimentation with different hyperparameter values can reveal the optimal settings, leading to improvements in model accuracy and reliability.\n\n## Building LLM Applications with Acorn\n\nVisit [https://gptscript.ai](https://gptscript.ai) to download GPTScript and start building today. As we expand on the capabilities with GPTScript, we are also expanding our list of tools. With these tools, you can create any application imaginable: check out [tools.gptscript.ai](https://tools.gptscript.ai/) to get started.\n\n## See Additional Guides on Key Machine Learning Topics\n\nTogether with our content partners, we have authored in-depth guides on several other topics that can also be useful as you explore the world of [machine learning](https://www.aporia.com/learn/machine-learning-model/machine-learning-models-use-cases-operations/).\n\n### [Advanced Threat Protection](https://www.cynet.com/advanced-threat-protection/advanced-threat-protection-a-real-time-threat-killer-machine/)\n\n*Authored by Cynet*\n\n- [Advanced Threat Protection: A Real-Time Threat Killer Machine](https://www.cynet.com/advanced-threat-protection/advanced-threat-protection-a-real-time-threat-killer-machine/)\n- [Advanced Threat Detection: Catch & Eliminate Sneak Attacks](https://www.cynet.com/advanced-threat-protection/advanced-threat-detection-stopping-advanced-attacks-in-their-tracks/)\n- [What is Network Analytics? From Detection to Active Prevention](https://www.cynet.com/advanced-threat-protection/network-analytics-from-detection-to-active-prevention/)\n\n### [Multi GPU](https://www.run.ai/guides/multi-gpu)\n\n*Authored by Run.AI*\n\n- [Multi GPU: An In-Depth Look](https://www.run.ai/guides/multi-gpu)\n- [Keras Multi GPU: A Practical Guide](https://www.run.ai/guides/multi-gpu/keras-multi-gpu-a-practical-guide)\n- [How to Build Your GPU Cluster: Process and Hardware Options](https://www.run.ai/guides/multi-gpu/gpu-clusters)\n\n### [Best LLM](https://www.acorn.io/resources/learning-center/best-llm)\n\n*Authored by Acorn*\n\n- [Best LLM: Benchmarks, Leaderboards, & the World’s 8 Smartest LLMs](https://www.acorn.io/resources/learning-center/best-llm)\n- [Leaderboard of LLM Leaderboards: Top 7 LLM Listings & Their Criteria](https://www.acorn.io/resources/learning-center/llm-leaderboards)\n- [Open LLM Leaderboard: Benchmarks, Model Types & Filters Explained](https://www.acorn.io/resources/learning-center/open-llm-leaderboard#other-filter-options-in-the-open-llm-leaderboard)\n\n## Related Articles\n\n- [AI Copilots: Enterprise Use Cases and Key Considerations](https://www.acorn.io/resources/learning-center/ai-copilots/)\n- [Parameter-Efficient Fine-Tuning (PEFT): The Basics and a Quick Tutorial](https://www.acorn.io/resources/learning-center/parameter-efficient-fine-tuning-peft/)\n- [Aisera: Overview of Platform, Solutions, Pros and Cons](https://www.acorn.io/resources/learning-center/aisera/)\n- [Fine-Tuning Llama 2 with Hugging Face PEFT Library](https://www.acorn.io/resources/learning-center/fine-tuning-llama-2/)\n- [Prompt Engineering in ChatGPT: 9 Proven Techniques](https://www.acorn.io/resources/learning-center/prompt-engineering-in-chatgpt/)\n- [LLM Application Development: Tutorial & 7 Steps to Production Apps](https://www.acorn.io/resources/learning-center/llm-application-development/)\n- [Open LLM Leaderboard: Benchmarks, Model Types & Filters Explained](https://www.acorn.io/resources/learning-center/open-llm-leaderboard/)\n- [Best LLM: Benchmarks, Leaderboards, & the World’s 8 Smartest LLMs](https://www.acorn.io/resources/learning-center/best-llm/)\n\nMenu\n\nMenu Close\n\n- [About Us](https://www.acorn.io/about-us/)\n- [Contact Us](https://www.acorn.io/contact/)\n- [Tutorials](/resources/tutorials)\n- [Blog](/resources/blog)\n- [Events](/events)\n- [Discord](https://discord.com/invite/9sSf4UyAMC)\n- [Open Source](https://www.acorn.io/resources/blog/open-source/)\n\nTo unsubscribe at any time please see our [Privacy Policy](https://www.acorn.io/privacy-policy).\n\n#### Get Started with GPTScript\n\n[Install GPTScript](https://github.com/gptscript-ai/gptscript?tab=readme-ov-file#1-install-the-latest-release)\n\nCopyright © 2024. All rights reserved. Acorn Labs, Inc.\n\n[Terms of Service](https://www.acorn.io/terms-of-use)\n\n- [GitHub](https://github.com/gptscript-ai/gptscript)\n- [Twitter](https://x.com/acornlabs)\n- [YouTube](https://www.youtube.com/c/AcornLabs)\n- [LinkedIn](https://www.linkedin.com/company/acorn-io/)",
    "tool": "\"website-cleaner\"",
    "defaultModel": "llm"
  },
  "status": {
    "state": "error",
    "output": "",
    "endTime": "2024-11-28T00:23:32Z",
    "error": "failed to run: failed calling model for completion: error, The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if you keep seeing this error."
  }
}

Will hopefully be fixed as part of https://github.com/otto8-ai/otto8/issues/753

Following 2 ingestion errors seen when ingesting - cry.org

failed to clean website content: failed to load documents from file "ws:///Corp Docs/Accounting/Accounts Payable/CSC/CSCINV20231204_8386860_86115235738.pdf" using loader "": you registered the "commonmark" plugin but the "base" plugin is also required

failed to clean website content: failed to run: failed calling model for completion: Post "http://<IP_ADDRESS>:11009/v1/chat/completions": read tcp <IP_ADDRESS>:60424-><IP_ADDRESS>:11009: read: connection reset by peer

Tested Ingestion of few websites with these change.
I have not encountered the errors reported in this issue relating to  website cleanup.
I am seeing few ingestion failures relating to - failed to clean website content: run exceeded maximum time of 10m0s which is tracked in https://github.com/obot-platform/obot/issues/926.
Closing this issue.


Add configurable option for role field/claim instead of hard-coded dial_roles

Description
Add configurable option for role field/claim instead of hard-coded dial_roles
Use case/motivation
No response
Related issues
No response
Confidential information

[X] I confirm that do not share any confidential information

Verified on staging with DevOps help, seems not working, waiting for new fix

Verified on staging with the help of @dspashynskyi
DIAL_ROLES_FIELD was configured to set new user-administrator in Keycloak and in Auth0 providers.
Please use https://github.com/epam/ai-dial-chat/tree/development/apps/chat for the details.


Auto install shared service names need to be formatted correctly.

The auto installed shared services have long names that make no sense to users:

In addition, the certs shared service probably does not want to be listed. Do we need a "show ito users" (as opposed to admins) boolean property?
@jjgriff93 @damoodamoo thoughts?

this would be better:

yea we can change the display_name in the auto-install, happy to take it


t3D header not correct yet

Describe the bug
Some issues with t3D header:
Missing:

vertical position 1/2/3/n is not supported, should be added to QuantityUnitPair (or new object)
timeInterpolation not written to file for T3D blocks
verticalPositionType "ZDatum" but is not available in VerticalPositionType class
verticalPositionType "ZSurf" is missing (newly added to this issue, Sep 29 '22)
Inconsistencies:
"verticalPositions" is written, while "Vertical position specification" is expected (in 2D3D manual). Does both work?
Answer: in kernel: yes, in GUI: no. And we should move to 1 consistent keyword. For now: support the original keyword: "Vertical position specification" (does Pydantic's alias allow whitespace in keys?)
Background for D-HYDRO developers, see internal issue UNST-5801.
"verticalPositionType " is written, while "Vertical position type" is expected (in 2D3D manual). Does both work?
Answer: same as previous one. For now, only support "Vertical position type".

To Reproduce
from pathlib import Path
import numpy as np

from hydrolib.core.io.bc.models import (
    ForcingModel,
    QuantityUnitPair,
    TimeInterpolation,
    T3D,
    VerticalPositionType,
)

bc_south = ForcingModel()
depth_list = [-2000,-1000,-500,-100]
times_list = np.array([0,60,120,180,240,300,360,420])
datablock = np.random.uniform(low=-40, high=130.3, size=(len(times_list),len(depth_list)))
datablock_incltime = np.concatenate([times_list[:,np.newaxis],datablock],axis=1)
list_QUP_perlayer = [QuantityUnitPair(quantity='salinity', unit='1e-3') for iL in range(len(depth_list))] #TODO REPORT: verticalposition 1/2/3/n is not supported
ts_one = T3D(name='3d_bnd_0001',
             verticalpositions=depth_list, #TODO: should be "Vertical position specification = [..]" but is verticalPositions = [..]" (both possible?)
             verticalInterpolation='linear',
             verticalPositionType=VerticalPositionType('ZBed'), #TODO: should be "Vertical position type = zdatum" but is "verticalPositionType = ZBed" (zdatum is niet beschikbaar)
             quantityunitpair=[QuantityUnitPair(quantity="time", unit="seconds since 2022-01-01 00:00:00 +00:00")]+list_QUP_perlayer,
             timeinterpolation=TimeInterpolation.linear, #TODO: not passed on to bc file
             datablock=datablock_incltime.tolist(),
             )
bc_south.forcing.append(ts_one)
bc_south.save(filepath=Path("steric_south2.bc"))

Expected behavior
described in description
Version info (please complete the following information):

OS: Windows, Anaconda, Spyder
Version: 0.3.0

@tim-vd-aardweg : I've update the original issue description at the top.
Note the correct naming: "Vertical position specification", "vertical position", "Vertical position specification".

@arthurvd Thanks for checking this. So we basically have to adhere to the 2D3D manual for the .bc file then? So we also have to change timeInterpolation (which is the current implementation in Hydrolib) to Time Interpolation

@arthurvd Thanks for checking this. So we basically have to adhere to the 2D3D manual for the .bc file then? So we also have to change timeInterpolation (which is the current implementation in Hydrolib) to Time Interpolation?

No, let's keep that at the new "timeInterpolation" keyword already. I am preparing an issue where we make kernel+GUI consistent again with all new lowerCamelCase namings, and timeInterpolation is now already supported by both kernel and GUI.

Ok!

I will


nav

Hey @suryanshsingh2001 can you please check the pr.

Solves #79

Hey @KavyaBS123,
It looks like there are deployment errors, likely due to the missing lucide-react dependency. Please take a look and refer to the PR rules and guidelines in the README for fixes.
Feel free to create a new one once you're ready.
Thanks!

Hey @suryanshsingh2001 the issue was related to deployment,  and  only you can resolve it as you are the memeber of the repo.
I would be grateful if you could see it once again

Hey @suryanshsingh2001 the issue was related to deployment, and only you can resolve it as you are the memeber of the repo. I would be grateful if you could see it once again

Hey @KavyaBS123, I checked the error logs,
Error: [vite]: Rollup failed to resolve import "lucide-react" from "/vercel/path0/client/src/Components/Header.jsx".


ENH: add in-draft to the SPEC template and to the listed, but still WIP specs

This goes along with https://github.com/scientific-python/scientific-python.org/pull/668
I left SPEC6 and SPEC8 alone as those maybe ready to get their first endorsements, but if needed we can add the draft status after next weeks meeting.

Ah, cool, thanks for this! Yes, I think the others we can start gathering endorsements for.
If you have time, perhaps also update the quickstart.py template?

If you have time, perhaps also update the quickstart.py template?

I assumed I did, but will have another look at it.

You did :upside_down_face: Thank you!


Delay in showing annotation jobs have been submitted

This morning I submitted 4 annotation jobs over the course of 5-8 minutes.  The jobs tracking page does not reflect the submission.  Why is there a delay this long?

They appeared as completed before they ever appeared in the job tracker.

Did the job window come up quickly for you or was there a delay? Wondering if it is related to the problem with rendering that page when there is a large number of jobs.
If the job page is pulling jobs incrementally, we might have an issue with sort ordering and the new jobs not coming up in initial retrieves when they don't yet have a start or completion time. I'll take a look at that possibility today.

I opened the jobs window pretty easily, and then clicked on the Started column.  They weren't showing up in the queue.  I've seen this before.
----- Original Message -----
From: "olsonanl"<EMAIL_ADDRESS>To: "PATRIC3"<EMAIL_ADDRESS>Cc: "Rebecca Wattam"<EMAIL_ADDRESS>"Author"<EMAIL_ADDRESS>Sent: Wednesday, October 26, 2016 9:47:30 AM
Subject: Re: [PATRIC3/patric3_website] Delay in showing annotation jobs have been submitted (#1170)
Did the job window come up quickly for you or was there a delay? Wondering if it is related to the problem with rendering that page when there is a large number of jobs.
If the job page is pulling jobs incrementally, we might have an issue with sort ordering and the new jobs not coming up in initial retrieves when they don't yet have a start or completion time. I'll take a look at that possibility today.
--
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub:
https://github.com/PATRIC3/patric3_website/issues/1170#issuecomment-256352368

@nconrad, see if you can fix this.

This issue doesn't seem easy to replicate.  It sounds like this could have even been when @ARWattam 's polling was getting blocked by ANL, @olsonanl ?   I wonder if we will have that issue again with the 5 second timeout polling.  Polling will be more often, but payload is only a couple hundred bytes.


Lag on the screen when Mouse Down

When I'm moving the selected area over the screen, it runs smoothly, but only when I release the cursor it takes a little time to re-rerun.
Can I do something to remove this lag?

this is an issue we have seen before. I can't reproduce it my self. i Would love to help just need a way to test it.

this is fixed with block rendering, try this out http://codepen.io/Crackeraki/pen/XmEdPx


Can I ask if you have tensorflow version or caffe version of the model?

Hi, unfortunately, we only have PyTorch version now, but we have the plan to implement the code in the other library and will upload it sooner or later.

Alright，good to know


Generating @context-files

Hi,
is there a default way for generating @context-files  based on customized Smart Data Models? The context-file-generator used in this tutorial seems to be based on @jason-fox swagger-datamodel-test tool.
Will this tool generate NGSI-LD v1.4.1 conform files?

@albertoabellagarcia thanks for the links. Somehow I completely overlooked the "Tool" menu on the Smart Data Model website. I will have a look at it.


Addendum Epic_2: Token-based Representation

As described in Epic_4, the dataset contains for each sensor measurements and meta-data. Among others, the meta-data contains the hash of the measurement.

The hash is used to construct the signature: for better efficiency, the measurements are first hashed and then the hash is asymmetrically encrypted. Consequently, to verify the signatures, the hash is required.

Given that (1) the hashes are already part of the meta-data and (2) are required for on-chain signature verification anyway, they represent a good candidate for the token-based representation. Furthermore, they can be eventually be used by the data consumer to verify the integrity of the measurements.

Since the dataset may contain measurements of many sensors and a one-to-one mapping of token and dataset is desirable, it may make sense to aggregate the individual hash value into a single root hash that represents the dataset. Therefore, Hash Trees can be used.

Closed, because the process is clarified and modified.


Receiving S3 input stream abort warnings when using an S3 maven repository

Expected Behavior
Executing a build with an S3 maven repository runs without warnings.
Current Behavior
Executing a build with an S3 maven repository produces multiple instances of the following warning.

WARNING: Not all bytes were read from the S3ObjectInputStream, aborting HTTP connection. This is likely an error and may result in sub-optimal behavior. Request only the bytes you need via a ranged GET or drain the input stream after use.

Context
We are using S3 as a maven repository for some of our libraries. When executing builds that rely on these dependencies, the above warning is being output multiple times.
After some investigation, I believe that this is an issue with the getMetaData method on S3ResourceConnector in conjunction with the getMetaData method on S3Client. The getMetaData method of S3Client returns an S3Object instead of just the metadata. It appears this is because of an issue with the GetObjectMetadataRequest as seen in the comment shown below. The S3ResourceConnector then gets the metadata and finally closes the S3Object. This warning is occurring because S3Object's content is never read before it is closed.
S3Client
public S3Object getMetaData(URI uri) {
    LOGGER.debug("Attempting to get s3 meta-data: [{}]", uri.toString());
    //Would typically use GetObjectMetadataRequest but it does not work with v4 signatures
    return doGetS3Object(uri, true);
}

S3ResourceConnector
public ExternalResourceMetaData getMetaData(URI location, boolean revalidate) {
    LOGGER.debug("Attempting to get resource metadata: {}", location);
    S3Object s3Object = s3Client.getMetaData(location);
    if (s3Object == null) {
        return null;
    }
    try {
        ObjectMetadata objectMetadata = s3Object.getObjectMetadata();
        return new DefaultExternalResourceMetaData(location,
            objectMetadata.getLastModified().getTime(),
            objectMetadata.getContentLength(),
            objectMetadata.getContentType(),
            objectMetadata.getETag(),
            null); // Passing null for sha1 - TODO - consider using the etag which is an MD5 hash of the file (when less than 5Gb)
    } finally {
        IoActions.closeQuietly(s3Object);
    }
}

I was able to confirm this with a simple test. This replicates the calls as they are above. When executed this will produce the warning. When explicitly calling abort on the object content prior to calling close, it does appear to stop this warning from occurring. Alternatively, reading through all of the bytes of the object also clears this warning.
public static void main(String[] args) throws Exception {
    GetObjectRequest gor = new GetObjectRequest("bucket", "key");
    S3Object s3Object = AmazonS3Client.builder().build().getObject(gor);
    ObjectMetadata objectMetadata = s3Object.getObjectMetadata();
    objectMetadata.getLastModified().getTime();
    objectMetadata.getContentLength();
    objectMetadata.getContentType();
    objectMetadata.getETag();
//        s3Object.getObjectContent().abort();

    s3Object.close();
}

I also see this a lot... really annoying!!

Any update to this? We use S3 Maven for our internal plugins and dependencies at my company and would like to know if the fix prescribed here will be implemented in some way.

+1 on this. Our console output is polluted by tens of occurrences of this message.

Please solve this issue! We get a lot of these warnings and they make the whole console output very difficult to analyse.

Is anyone aware of a way to filter out these messages from the gradle output? I can't find a way to do so with custom loggers in Gradle.

+1


MySQL mediumtext

I got a runtime error that I tracked down to a table with MySQL mediumtext columns. I got past it by deleting the table as I didn't need in for my new PG DB.
The type was coming through into here as :none
https://github.com/jhartwell/Plsm/blob/master/lib/io/export.ex#L6
Which then failed to match any of the map_type signatures:
https://github.com/jhartwell/Plsm/blob/master/lib/io/export.ex#L12
I didn't dig deeper but suppose mediumtext is uknown in the type mappings and coming through as :none.

plsm 2.4.0 (https://github.com/jhartwell/Plsm.git) (mix)
locked at f1ee062
ok


[Native whitelist] add elmfire

Resolves issue #65

@evancz Please confirm, or reject.

Alright, we shall see!


Anchor links in exhibition guides

We need to be able to link to specific media or parts of the guides. This ticket is to explore how anchor links can help us do that.

Closing; follow-up work is in #8463.


V2_MetaFunction doesn't expect an array to be returned

What version of Remix are you using?
1.8.1
Steps to Reproduce
Create a meta export like:
export let meta: V2_MetaFunction = () => {
  return [
    { title: "Some App" },
    { property: "charset", content: "utf-8" },
    { property: "viewport", content: "width=device-width,initial-scale=1" },
    { property: "theme-color", content: "#1C488B" },
  ];
};

Expected Behavior
TypeScript should understand that the return type will be HtmlMetaDescriptor[].
Actual Behavior
It thinks the return type should be HtmlMetaDescriptor.^source

There appears to be a working type in the react package, but not the server-runtime package. Closing.


Duplicates

Adding additional duplicate removal capabilities. This branch was started from branch threshold.

While I approve of these changes, I am going to close this PR and merge it into main directly after approving thresholds, in order to not mix the two distinct features into one PR.

Looking to see if I can change the destination to main.


Update Dogs.json

added breed Dorgi and Doberdor

Hey @Sonal-Maurya
I am so sorry but this repo has been excluded from Hacktoberfest, yet, I'll take your Pull Request for the sake of Contribution
Thank you so much!

Any repo that is actived in hacktober


MINOR: update required MacOS version

MacOS 10.14 should actually work, too. At least our tests passed while they fail on 10.13.
This PR should be cherry-picked to 2.6 branch.
Call for review @vvcephei

Merged to trunk and cherry-picked to 2.6 branch.


Remove react-native-windows as a direct dependency of react-native-video

Fix #729
Fix #720
Fix #640
Fix #567
Hi there, thanks for this package! When installing this package, it also bundled react-native-windows into my app since it's a dependency of this project. Since it's not required to have react-native-windows for this package to work, it shouldn't be required. We're also not adding react-native itself as a dependency so I don't see why the windows version should be.
Thanks in advance for reviewing!

Hope this PR can be merged soon. There're multiple issues caused by this dependency: #729 #720 #640 . In the meantime we have to fork this repo to make things work for our project.

+1

+1

+1

@radko93 Is there any chance of getting this merged in quickly? I really need to use this library but the react-native-windows dependency is causing the iOS side to fail due to this bug in react-native-windows: https://github.com/Microsoft/react-native-windows/issues/1255

I'm not a collaborator so I can't merge it.

+1

Who has write access? @mattapperson?

@radko93 Oh sorry I saw your name on an earlier pull request and thought it was you merging it, my bad.

thanx @mattapperson !
Could you also republish the package please?

Thanks for your efforts :)

Hey thanks for merging! Is is possible to do a new release with this included @mattapperson?

@mattapperson Any chance of getting this published to npm today?

Yes I will be cutting a new release this evening

Published at 2.0.0 due to potential breaking changes in this release


Update adopter use case: Hurricane Engines

What type of PR is this?
/kind cleanup
What this PR does / why we need it:
Update adopters' information according to the latest feedback.
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:

/assign @pidb
You can get the preview from :

English: https://deploy-preview-236--karmada.netlify.app/docs/casestudies/adopters
Chinese: https://deploy-preview-236--karmada.netlify.app/zh/docs/next/casestudies/adopters/


[Feature] Merge form button with normal button widget

Description
Today form buttons are a special widget which reset parent forms and get disabled based on the parent forms logic. This leads to a bit of confusion on their usage and makes it hard to add a new form button if it's deleted. Ideally, these should work the same as regular buttons inside any container

Duplicate of #3613


CI/CD Creation Ticket

@Pyatakov @danielnorkin @anvabr To further Scope Out this ticket based on our Sprint Grooming Session today, 11/24. This ticket goes hand in hand with Issue#40
Need to Add DOD and Acceptance Criteria.

This is done, I re-opened and updated ticket #4