Could not find a matching NEFF for your HLO in this directory

System Info
I'm compiling a fine-tuned Llama 3.1 70B model with the below system info on an inf2.48xlarge machine. I'm using neuronX TGI 0.0.25 with AWS Sagemaker. I get the below error:

FileNotFoundError: [Errno 2] No such file or directory: '/opt/ml/model/compiled/e28e32d0143dad6277a9.neff'
FileNotFoundError: Could not find a matching NEFF for your HLO in this directory. Ensure that the model you are trying to load is the same type and has the same parameters as the one you saved or call "save" on this model to reserialize it.

Platform:

- Platform: Linux-6.8.0-1015-aws-x86_64-with-glibc2.35
- Python version: 3.10.12

Python packages:

- `optimum-neuron` version: 0.0.25
- `neuron-sdk` version: 2.20.0
- `optimum` version: 1.22.0
- `transformers` version: 4.43.2
- `huggingface_hub` version: 0.26.2
- `torch` version: 2.1.2+cu121
- `aws-neuronx-runtime-discovery` version: 2.9
- `libneuronxla` version: 2.0.4115.0
- `neuronx-cc` version: <IP_ADDRESS>+56dc5a86
- `neuronx-distributed` version: 0.8.0
- `neuronx-hwm` version: NA
- `torch-neuronx` version: <IP_ADDRESS>.3.0
- `torch-xla` version: 2.1.4
- `transformers-neuronx` version: 0.12.313

Neuron Driver:

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

aws-neuronx-collectives/now <IP_ADDRESS>-17a033bc8 amd64 [installed,local]
aws-neuronx-dkms/now <IP_ADDRESS> amd64 [installed,local]
aws-neuronx-runtime-lib/now <IP_ADDRESS>-6e27b8d5b amd64 [installed,local]
aws-neuronx-tools/now <IP_ADDRESS> amd64 [installed,local]

This is my compilation command:

optimum-cli export neuron -m orig-llama/ --batch_size 4 --task text-generation --sequence_length 4096 --num_cores 24 --auto_cast_type bf16 ./neuron-llama-throughput

Here is my TGI env:

sagemaker_model_env = {
    "SM_MODEL_DIR" = "/opt/ml/model"
    "HF_MODEL_ID" = "/opt/ml/model"
    "HF_NUM_CORES" = "24"
    "HF_BATCH_SIZE" = "4"
    "HF_SEQUENCE_LENGTH" = "4096"
    "HF_AUTO_CAST_TYPE" = "bf16"
    "MAX_BATCH_SIZE" = "4"
    "MAX_INPUT_TOKENS" = "3072"
    "MAX_TOTAL_TOKENS" = "4096"
    "MESSAGES_API_ENABLED" = "false"
    "MAX_BATCH_PREFILL_TOKENS" = "16384"
    "MAX_BATCH_TOTAL_TOKENS" = "20000"
    "ROPE_SCALING" = "dynamic"
    "ROPE_FACTOR" = "8.0"
  }

Who can help?
@dacorvo @JingyaHuang
Information

[ ] The official example scripts
[ ] My own modified scripts

Tasks

[ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[ ] My own task or dataset (give details below)

Reproduction (minimal, reproducible, runnable)
Compile Llama 3.1 70B using the below system info on inf2.48xlarge and run on neuronX TGI version 0.0.25
Platform:

Platform: Linux-6.8.0-1015-aws-x86_64-with-glibc2.35
Python version: 3.10.12

Python packages:

optimum-neuron version: 0.0.25
neuron-sdk version: 2.20.0
optimum version: 1.22.0
transformers version: 4.43.2
huggingface_hub version: 0.26.2
torch version: 2.1.2+cu121
aws-neuronx-runtime-discovery version: 2.9
libneuronxla version: 2.0.4115.0
neuronx-cc version: <IP_ADDRESS>+56dc5a86
neuronx-distributed version: 0.8.0
neuronx-hwm version: NA
torch-neuronx version: <IP_ADDRESS>.3.0
torch-xla version: 2.1.4
transformers-neuronx version: 0.12.313

Neuron Driver:
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
aws-neuronx-collectives/now <IP_ADDRESS>-17a033bc8 amd64 [installed,local]
aws-neuronx-dkms/now <IP_ADDRESS> amd64 [installed,local]
aws-neuronx-runtime-lib/now <IP_ADDRESS>-6e27b8d5b amd64 [installed,local]
aws-neuronx-tools/now <IP_ADDRESS> amd64 [installed,local]
Expected behavior
The neuronX TGI server to start

I want to rule out an SDK mismatch between the compilation environment and the hosting environment.
Are you deploying on SageMaker?  What image are you using?
If you are not deploying on SageMaker, try compiling using the TGI image itself.
See the example here:
https://github.com/huggingface/optimum-neuron/tree/main/benchmark/text-generation-inference/performance#compiling-the-model

Should be fixed by #743


guess I missed that paragraph but...   "@@"(Js.log, Js.Float.toString(param))

@@ is a custom infix operator. In your example it is used like a regular function.
Right now ReScript doesn't support custom infix operator syntax, that's why you need to escape it with \"@@" and then later on use it as a function \"@@"(arg1, arg2).
I assume this is the @@ operator is coming from the Pervasives module, which has following definition:
Application operator: g @@ f @@ x is exactly equivalent to g (f (x)).

It's not something we'd recommend to use, since you can achieve pretty much the same thing with the way easier to understand pipe syntax: param->Js.Float.toString->Js.log

@@ is a custom infix operator. In your example it is used like a regular function.
Right now ReScript doesn't support custom infix operator syntax, that's why you need to escape it with \"@@" and then later on use it as a function \"@@"(arg1, arg2).
I assume this is the @@ operator is coming from the Pervasives module, which has following definition:
Application operator: g @@ f @@ x is exactly equivalent to g (f (x)).

It's not something we'd recommend to use, since you can achieve pretty much the same thing with the way easier to understand pipe syntax: param->Js.Float.toString->Js.log

You have put to much faith in my capacity to understand your answer but... I get that it's a monkey path and that I should look into Pervasives ;-)
Thanks

You have put to much faith in my capacity to understand your answer but... I get that it's a monkey path and that I should look into Pervasives ;-)
Thanks

@ruifortes where did you find this information? if you found that in the language manual, we need to fix it

@ruifortes where did you find this information? if you found that in the language manual, we need to fix it

yep...in the Belt documentation but just went there and it's fixed now,
Maybe it was a glitch updating the site

yep...in the Belt documentation but just went there and it's fixed now,
Maybe it was a glitch updating the site

ahh...found it https://rescript-lang.org/docs/manual/latest/api/js/float

ahh...found it https://rescript-lang.org/docs/manual/latest/api/js/float

@ryyppy @ruifortes working on a PR that converts the syntax away from this. Just FYI so that we don't double up work. HMU if this is already being worked on.

@ryyppy @ruifortes working on a PR that converts the syntax away from this. Just FYI so that we don't double up work. HMU if this is already being worked on.

@ruifortes see referenced PR for what this potentially should have looked like. I do have an outstanding question of whether or not it is idiomatic to do a normal function call first, then pipe for all subsequent calls, or to pipe from the get-go like in the PR, but either would WORK, and hopefully are less confusing than the examples that prompted your question

@ruifortes see referenced PR for what this potentially should have looked like. I do have an outstanding question of whether or not it is idiomatic to do a normal function call first, then pipe for all subsequent calls, or to pipe from the get-go like in the PR, but either would WORK, and hopefully are less confusing than the examples that prompted your question


closes #320: implement findCollections command

What this PR does:
Implements the findCollections command.. Lists only jsonapi collections in a namespace. Example similar to findNamespaces:
$ curl -X 'POST' \
>   'http://localhost:8080/v1/cycling' \
>   -H 'accept: application/json' \
>   -H 'X-Cassandra-Token: 90e68c40-c254-43cb-87d2-91c9003de546' \
>   -H 'Content-Type: application/json' \
>   -d '{
>   "findCollections": {}
> }' | jq .

{
  "status": {
    "existingCollections": [
      "events"
    ]
  }
}

Which issue(s) this PR fixes:
Fixes #320

@johnsmartco we would also need spec for this new command as well..

Converted to draft as suggested by @maheshrajamani

@jeffreyscarpenter @maheshrajamani This was ready to review a long time ago, I don't understand why is this moved to draft and still not reviewed?


Improved error reporting using ERR_CLEANUP_CLUSTER_RESOURCES when hibernating a cluster

How to categorize this PR?
/area ops-productivity
/kind enhancement
What this PR does / why we need it:
Previously, when a pod was still running while gardener was waiting for all pods to be in a non running state, there was no distinction between a pod in system namespace and a pod in non system namespace. Now, different error messages will be returned depending on the namespace. This is especially useful when hibernating a cluster since the user will be provided with much better feedback about why his cluster refused to hibernate
Which issue(s) this PR fixes:
Fixes #8887
Special notes for your reviewer:
Release note:
When hibernating a cluster, Gardener now assigns an error code ERR_CLEANUP_CLUSTER_RESOURCES to shoot clusters if (user) pods are still running in non kube-system namesapces

/assign @timuthy

Converting to draft until PR format is in a good shape (clarification offline).

/ok-to-test

@benedictweis Can you please fix the release note? Some characters seem to have overtaken others...

No further comments from my side. Can you please check the failing unit test?

Forgot to run make tidy, should be fixed now.


Co.chat - add return chatlog

What this PR does

Adds return_chatlog as an option param to co.chat
Returns the chatlog if return_chatlog is set to true
Adds unit tests for the new functionality
Closes CNV-288

How it was tested

Unit tests

also please bump the patch version in setup.py

I'll also do the same for another variable, should I update the patch version twice or can it wait for the next one? @mkozakov

also please bump the patch version in setup.py

I'll also do the same for another variable, should I update the patch version twice or can it wait for the next one? @mkozakov

If you're updating in two different PRs, update the version separately


Conflit with GABZ

First of all
Did you try disabling bob74_ipl to see if the issue is still there? Yes
Did you use the latest version of bob74_ipl ? Yes
Did you use an up to date server artifact ? Yes
Describe the bug
Hello, I have a conflict with the PDM of Gabz and the casemate hospital.

To Reproduce
Coordinates or map screenshot of the location.
Expected behavior
A clear and concise description of what you expected to happen, a screenshot from GTA Online if relevant is welcome.
Screenshots
If applicable, add screenshots to help explain your problem.
Versions
3095

Resource version

Please try disabling the fixes added in 2944:
MercenariesFixes = exports['bob74_ipl']:GetMercenariesFixesObject()

MercenariesFixes.Enable(false)

not fixing the issue ?
have disabled the mercenaries fixes but still see the airco boxes in the garage door and the glitch in the pillbox garage
i have nothing else running what can cause this problem

not fixing the issue ? have disabled the mercenaries fixes but still see the airco boxes in the garage door and the glitch in the pillbox garage i have nothing else running what can cause this problem

well i have fixed the pdm with the 2 boxes only the garage from pillbox not

@SubwooferNL how did you fix the PDM issue?

There are fixes for these in the gabz Discord I think.


[WIP] Custom Integrations UI

refs https://github.com/TryGhost/Ghost/issues/9865, https://github.com/TryGhost/Ghost/issues/9942

integration, api-key, and webhook models and respective mirage mocks
moves integration routes around to match ember's concept of nested routes (nested routes reflect nested UI not nested URLs)
adds custom integrations list to integrations screen

NB: the enableDeveloperExperiments flag currently needs to be enabled in the config.development.json file for the custom integrations UI to be displayed.
TODO:

[ ] new integration modal

[ ] handle duplicate name validation

[ ] list custom integrations on integrations index screen
[ ] /settings/integrations/:id/ screen
[ ] new webhook modal

Coverage decreased (-0.04%) to 71.725% when pulling 230fe99f98b08fc670fdabc8a17c1a2801006042 on kevinansfield:integrations-custom into d878d2e31fd34e8f4be4bf2fe55d8fa838dab68b on TryGhost:master.

Merging as-is for now so that it's easier to to test server-side components and build on top with smaller PRs.


[Question] Obtaining Relative Position of Tap

Hi everyone,
I was glad the in recent versions, the tap callbacks were included into photo view. However, I am currently trying to obtain the relative position of tap, irregardless of scale value. This means that even at different scales, when a user taps on the same point/position, it should return the same position value.
The following onTapUp callback implementation would give me the position of a point tapped. However, upon changing to another scale and tapping at the same point, a different value was returned.
onTapUp: (buildContext, tapUpDetails, photoViewControllerValue){
    print("position: " + tapUpDetails.globalPosition.toString());
}

Using the code photoViewControllerValue.position.dx/photoViewControllerValue.scale gave me different values, even when tapping on the same point.
As such, I would require the help of everyone to lead me towards the right direction in getting this done.
Thank you!

@darrylong Did you solve it? I am too working on an app and need this. This comment says that is done but I am not able to see how to use that.

Kind of – the 2.0 comes from the fact that the localPosition starts in the top left corner, while the controller uses coordinates that start in the center of the widget. Hence the translation between them is half the widget size (the minus signs are because it also inverts the coordinates for some reason).
In your case, the context of onTapUp might be a context where the obtained size is different, but it's hard to tell without your full code.


fix: checked data will disappear when the dataRow is refresh

PTAL

@LiuJi-Jim can take a look at Vue code.


[bug] SvelteKit got upgraded - Tauri not working with<EMAIL_ADDRESS>(V2)

Describe the bug
After carefully following the quickstart guide for SvelteKit, my Tauri application showed up blank (white/blank UI, but working on localhost:localhost:5173).
I noticed that recently SvelteKit got upgraded to version 2.
Workaround
So i tried using older packages, which is a workaround for now:
npm create<EMAIL_ADDRESS>my-app
npm install --save-dev<EMAIL_ADDRESS>Reproduction
Follow instructions on https://tauri.app/v1/guides/getting-started/setup/sveltekit
Expected behavior
Welcome to SvelteKit
Visit kit.svelte.dev to read the documentation

... Showing up in window - but it doesn't.
:(
Full tauri info output
> tauri info

[✔] Environment
    - OS: Mac OS 10.15.7 X64
    ✔ Xcode Command Line Tools: installed
    ✔ rustc: 1.75.0 (82e1608df 2023-12-21)
    ✔ cargo: 1.75.0 (1d8b05cdd 2023-11-20)
    ✔ rustup: 1.26.0 (5af9b9484 2023-04-05)
    ✔ Rust toolchain: stable-x86_64-apple-darwin (default)
    - node: 21.5.0
    - pnpm: 8.6.9
    - npm: 10.2.5

[-] Packages
    - tauri [RUST]: 1.5.4
    - tauri-build [RUST]: 1.5.1    - wry [RUST]: 0.24.7
    - tao [RUST]: 0.16.5
    - @tauri-apps/api : not installed!    - @tauri-apps/cli [NPM]: 1.5.9

[-] App
    - build-type: bundle
    - CSP: unset
    - distDir: ../build
    - devPath: http://localhost:5173/
    - framework: Svelte
    - bundler: Vite

### Stack trace

_No response_

### Additional context

[The migration to V2](https://kit.svelte.dev/docs/migrating-to-sveltekit-2#vitepreprocess-is-no-longer-exported-from-sveltejs-kit-vite) doesn't seem to be that bad, but this is my first time using Tauri so i'm not even exactly sure how the tech works or where i would begin to debug.

Hmm, i assume the issue is because they may have upgraded the minimal browser versions (for dev mode at least). The webview in macOS 10.15 is basically Safari 13. I don't have access to my 10.15 vm right now so this is only a guess. (it still works for me on the latest macos/windows versions fwiw)
Are there any errors in the devtools console of the webview?

I just went through is, and after a couple of tries, got the expected result following the exact instructions. The first two tries gave me Os Error Code 2 - not sure if this was because I selected 'Try Svelte 5 preview (unstable)'

Ah. Now I am running into ```% npm run tauri dev

<EMAIL_ADDRESS>tauri
tauri dev

   Error failed to get cargo metadata: No such file or directory (os error 2)```


Documentation bug for "/{application}/{profile}[/{label}]"

The documentation says:
The HTTP service has resources in the following form:

/{application}/{profile}[/{label}]
....

which is incorrect.
The correct form would be /{application}/[{profile}]/{label} as profiles can be many. It would also not hurt to have a minor example of this.

It's not incorrect. Label is optional, profile is not

Profile can be a comma separated list, but it isn't optional which is what brackets denote

thank you @spencergibb, got my answer, closing.