Choosing Search Marketplace causes VSCODE to search endlessly (no timeout)

For the past month I have noticed that I am unable to see any extensions which were installed or able to see any extensions which I could choose to install. The activity cursor moves across the screen (left to right) and never seems to finish. I can use the filter "@sort:name " and the cursor stops moving. But then no inventory of possible extension shows.
Version: 1.39.1
Commit: 88f15d17dca836346e787762685a40bb5cce75a8
Date: 2019-10-10T23:30:33.955Z
Electron: 4.2.10
Chrome: 69.0.3497.128
Node.js: 10.11.0
V8: 6.9.427.31-electron.0
OS: Linux x64 4.19.79-1-MANJARO
Steps to Reproduce:
1.Open VS Code
2.Open Extensions Market place.
3.Marketplace busy cursor will update showing no extensions loading.

Does this issue occur when all extensions are disabled?: Yes/No

Did some additional looking around and noticed that \Home.vscode\extensions directory does not appear to update. I can see all the plugins I had previously installed when extensions was working.
To troubleshoot I renamed the 'extensions' directory to 'extensions-old'. When I launch Visual Studio Code the extensions directory gets re-created but no content is put in the directory.

Any update on this issue?
Do you need me to disable the extension? If yes, how should I do it?

Can you please try with our insiders and let us know if you see the issue?
https://code.visualstudio.com/insiders/

I re-tested 2 days ago with the below release and it all of a sudden the market place started working. Its too bad we did not capture any debugging prior.
Version: 1.40.0
Commit: 86405ea23e3937316009fc27c9361deee66ffbf5
Date: 2019-11-06T18:14:08.920Z
Electron: 6.1.2
Chrome: 76.0.3809.146
Node.js: 12.4.0
V8: 7.6.303.31-electron.0
OS: Linux x64 4.19.81-1-MANJARO snap


Notebook getting started experience challenges/questions

Scenario 1

User creats a new notebook using Create Blank Notebook
The notebook will default to Python (for new users)
User will get a prompt to install Python extension (bottom right prompt)
Is this good? What if the user wanted to get strated with TypeScript or Julia?

Scenario 2

User creats a new notebook by creating a new file in VSC explorer new.ipynb
The notebook will default to plaintext (for new users)
Ther'es a slight difference between this scneario and previous.
Why is it plaintext, should it be python
However now user will get prompted to isntall Python, if the user was planning on working on TypeScript.

Scenario 3

User has anaconda installed (but no jupyter kernels, in global path)
User has Jupyter extension installed
User opens a notebook (they get the dummy Python 3 placeholder kenrel)
User hits the run button, they get prpompted to install Python extension
User installs it
Now what?
Ok, user hits run button again, but gets the exact same message
Also the kernel picker doesn't display the new kernels.

Problems

Annoying Prompts

Getting prompted to install Python when user intents on creating TS notebook
Getting prompted to install Python when user is just planning on opening a notebook to read (not exefcute)

Solution is to Reload VS Code

Even after installing Python extensions, the run button doesn't work (need to reload vs code)
Even after installing Python extension, the list of kernels hasn't changed (need to reload vs code)

Suggestion

Recommend installing Python only when the user hits the run button
Fetch the list of the kernels once the Python extension has been activated IPythonApiProvider.onDidActivatePythonExtension
When Python extension is installed and user attempts to run a cell

The dummy kernel should wait for kernel search to complete
After this, this dummy kernel should be upgraded to the preferred kernel (same solution we use here https://github.com/microsoft/vscode-jupyter/issues/5861)
I.e. if user has 10 pytho kernels, this placeholder will end up being upgraded to one of the 10 (it must be the preferred item).
Next after all of this, the run continues and starts the kernel.
NOTE: Disposing this kernel controller will not work, as its already associated with the document (user ran a cell)
Problem with this solution, next time user loads VSCode and opens this notebook, the placeholder kernel no longer exists, hecne when user attempts to run a cell, they'll get prompted once again to select a kernel (no ideal)
NOTE: The kernel that was used in both cases must be the same, we need to upgrade the placeholder to the preferred kernel.

@claudiaregio /cc

Working on this with @claudiaregio right now.


Trades not considering fees

Hi, first of all, amazing tool!
I'm exploring node-binance-trader with BvA, and I'm seeing that the trades doesn't consider fees, so it tries to sell the same amount of coins it has bought, causing insufficient funds error in Binance.
I'm using the "BVA3H+ ALTS" strategy and deployed the trader to Heroku from https://github.com/jsappme/nbt-binance-auto-trader project.
Am I missing something?
Thank you

yes you always need to have some BNB on your wallet(s) to pay for the fees.


Taxon constraint: adhesive extracellular matrix(GO:0062130) to Drosophila adhesive extracellular matrix

Please provide as much information as you can:

GO term ID label:
Replace ID label Adhesive extracellular matrix (GO:0062130) by "Drosophila adhesive extracellular matrix"

Request to add a taxon constraint:
Drosophila melanogaster

**only in taxon: **
Drosophila melanogaster

**never in taxon: **

Request to remove a taxon constraint:
The old term is misleading as in common usage the term "adhesive extracellular matrix" refers to the group of gene products that exert adhesive functions within the extracellular matrix (GO:0031012), while this term is actually defining a subset of drosophila gene products; the reference says "These glands synthesize and secrete massive amounts of a glue which attaches the pupae to the substrate during metamorphosis.", this is taxon-specific.
We thus recommend changing the name to "drosophila adhesive extracellular matrix" or the synonym "puparial glue" and also change the parent term.

Change the parent to Specialized extracellular matrix (see #29265)

Supporting evidence if available (e.g PMID):
Keep the same reference and definition: PMID:825230

@DaiqingChen : Add link to slide deck and insert screenshot of revised slide 2

Hi @DaiqingChen
You can find some of the original discussion that led to the creation of this term here:

#17502

Interestingly, the original request from @hattrill was for a Drosophila-specific term, but after discussion we decided to introduce a very broad and general term.
Note also that it used to be common practice to attach the PMID that was used for the first annotation to the term definition, even if the PMID is much more specific. This is really confusing. We have since stopped this practice.
I agree with your broader analysis. The current term as it exists is far too broad, as it covers a huge range of secreted products that are different structurally and evolutionary. I support "repurposing" this term to be more specific as it was originally intended.
I think this is a good time to do some general cleanup. Let's look at some similar terms:

GO BP term "puparial adhesion" https://amigo.geneontology.org/amigo/term/GO:0007594
FlyBase gene group term "glue protein" https://flybase.org/reports/FBgg0001189.htm

The gene sets for the 3 terms are identical (modulo inclusion of Sgs9, which was purported to exist in 1976 but has not been genomically characterized)
It seems odd to me that we have 3 terms to describe the same set of genes, yet in GO we report "function unknown" for most of them. The biophysical function of these genes are understood in exquisite detail! The function for some of the genes is reported as "structural constituent of ECM" but this is inconsistently applied and IMO is pretty indirect.
I think the most conservative course of action here is to repurpose this term as you suggest in https://github.com/geneontology/go-ontology/issues/29265#issuecomment-2557670020 under "specialized ECM", relabel and redefine to be less generic, add a taxon constraint. I think the correct TC is NCBITaxon:480117 ! Cyclorrhapha (see https://link.springer.com/chapter/10.1007/978-3-319-40740-1_15) but putting "Dipteran" in the name is good enough for clarity.
I would make an equivalence xref to FlyBase:FBgg0001189 such that gene sets are guaranteed to be the same (though this seems fairly stable)
And not to overload this ticket, but I'm questioning the utility of:
id: GO:0007594
name: puparial adhesion
namespace: biological_process
def: "The adhesion of the puparia of Diptera to their substrate; normally effected by a 'glue' secreted by the larval salivary gland and expectorated at the time of pupariation." [GOC:ma]
synonym: "puparial glue" RELATED []
is_a: GO:0022609 ! multicellular organism adhesion to substrate
relationship: part_of GO:0007591 ! molting cycle, chitin-based cuticle

Is this truly a biological process, or more of a biophysical process? I think if we do want to represent the GO-BP aspect of glue genes we should have a term for the secretory pathway for generating them, which is well understood with defined starts and ends (though the glue genes themselves would not be annotated here). It seems odd to try and force the glue genes into a biological process.
I also wonder if we want to do a better job of gene products whose function is biophysical. We have the "structural constituent" branch but this seems to be applied consistently.
(oh and the reference for the FBgg0001189 is Naba et al, we have come full circle :-)

@cmungall Thanks for the feedback!
Your proposal seem reasonable. If we need additional insight, I can reach out to our ECM Drosophila expert.


chunks[startIndex].slice is not a function

I'm trying to use this is Astro. Unfortunately, when at least one callout is present on the page, rendering crashes:
Failed to parse Markdown file "C:\Entwicklung\repos\eiko-wagenknecht.de\src\content\posts\markdown-test.md":
chunks[startIndex].slice is not a function

Stack Trace:
TypeError: Failed to parse Markdown file "C:\Entwicklung\repos\eiko-wagenknecht.de\src\content\posts\markdown-test.md":
chunks[startIndex].slice is not a function
    at sliceChunks (file:///C:/Entwicklung/repos/eiko-wagenknecht.de/node_modules/.pnpm/micromark@4.0.0/node_modules/micromark/lib/create-tokenizer.js:512:32)
    at Object.sliceStream (file:///C:/Entwicklung/repos/eiko-wagenknecht.de/node_modules/.pnpm/micromark@4.0.0/node_modules/micromark/lib/create-tokenizer.js:153:12)
    at insideTitle (file:///C:/Entwicklung/repos/eiko-wagenknecht.de/node_modules/.pnpm/remark-callout@1.1.1/node_modules/remark-callout/dist/lib/micromark-syntax.js:105:33)
    at afterTypePotentialWhitespace (file:///C:/Entwicklung/repos/eiko-wagenknecht.de/node_modules/.pnpm/remark-callout@1.1.1/node_modules/remark-callout/dist/lib/micromark-syntax.js:100:16)
    at go (file:///C:/Entwicklung/repos/eiko-wagenknecht.de/node_modules/.pnpm/micromark@4.0.0/node_modules/micromark/lib/create-tokenizer.js:222:13)
    at main (file:///C:/Entwicklung/repos/eiko-wagenknecht.de/node_modules/.pnpm/micromark@4.0.0/node_modules/micromark/lib/create-tokenizer.js:208:9)
    at Object.write (file:///C:/Entwicklung/repos/eiko-wagenknecht.de/node_modules/.pnpm/micromark@4.0.0/node_modules/micromark/lib/create-tokenizer.js:129:5)
    at fromMarkdown (file:///C:/Entwicklung/repos/eiko-wagenknecht.de/node_modules/.pnpm/mdast-util-from-markdown@2.0.1/node_modules/mdast-util-from-markdown/lib/index.js:173:66)
    at parser (file:///C:/Entwicklung/repos/eiko-wagenknecht.de/node_modules/.pnpm/remark-parse@11.0.0/node_modules/remark-parse/lib/index.js:33:12)
    at Function.parse (file:///C:/Entwicklung/repos/eiko-wagenknecht.de/node_modules/.pnpm/unified@11.0.5/node_modules/unified/lib/index.js:668:12)

Input:
> [!example]
> Example

package.json:
  "dependencies": {
    "@astrojs/mdx": "3.1.2",
    "@astrojs/partytown": "2.1.1",
    "@astrojs/react": "3.6.0",
    "@astrojs/rss": "4.0.7",
    "@astrojs/sitemap": "3.1.6",
    "@astrojs/tailwind": "5.1.0",
    "@radix-ui/react-dialog": "1.1.1",
    "@types/mdast": "4.0.4",
    "astro": "4.11.5",
    "hast-util-from-html-isomorphic": "2.0.0",
    "react": "18.3.1",
    "react-dom": "18.3.1",
    "rehype-autolink-headings": "7.1.0",
    "rehype-slug": "6.0.0",
    "remark-callout": "^1.1.1",
    "remark-github-blockquote-alert": "1.2.1",
    "remark-obsidian-callout": "1.1.3",
    "remark-toc": "9.0.0",
    "sanitize-html": "2.13.0",
    "unified": "11.0.5",
    "unist-util-visit": "5.0.0"
  },
  "devDependencies": {
    "@astrojs/check": "0.8.0",
    "@biomejs/biome": "1.8.3",
    "@commitlint/cli": "19.3.0",
    "@commitlint/config-conventional": "19.2.2",
    "@commitlint/types": "19.0.3",
    "@tailwindcss/forms": "0.5.7",
    "@tailwindcss/typography": "0.5.13",
    "@types/react": "18.3.3",
    "@types/react-dom": "18.3.0",
    "lefthook": "1.7.1",
    "prettier": "3.3.2",
    "prettier-plugin-astro": "0.14.0",
    "prettier-plugin-tailwindcss": "0.6.5",
    "tailwindcss": "3.4.4",
    "typescript": "5.5.3"
  }

The other alternative plugins work fine (with their limited functionality).

Hi, thank you for your report and sorry for the late reply. This issue looks a little weird, and it might be related to  depencencies this package use. I have found a similar issue here in the micromark GitHub repo. The dependency "micromark-utils-character" might be causing the problem. Further investigation is needed, and I will work on this to reach a solution when I can spare some time.
Testing locally on my machine works fine. My exact local version of dependency "micromark-util-character" is 2.0.1.
As a temporary fix, maybe specifying
"micromark-util-character": "2.0.1"

in your package.json can solve your problem (note there is no caret symbol before the version number).

Pinning 2.0.1 does not fix this. Thanks for investigating, though!


Problems running run_docker

I've been trying to start the docker(/run_docker) for a while with the aim of having the rest API enabled and being able to start developing my own controller.
The problem is that I am not being able to access the API services.
This is what I running:

What should be the procedure to follow to have the rest API available?

Hello,
have you exposed the port you are using (8021 & 8020) in the ENV variable
$PORTS?
Take a look at the run_docker script.
Il giorno mar 29 giu 2021 alle ore 14:25 gilAsier @.***>
ha scritto:

I've been trying to start the docker(/run_docker) for a while with the aim
of having the rest API enabled and being able to start developing my own
controller.
The problem is that I am not being able to access the API services.
This is what I running:
[image: image]
https://user-images.githubusercontent.com/4936241/123796695-adcdfb00-d8e5-11eb-952c-49cbe3ef6f51.png
What should be the procedure to follow to have the rest API available?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/hyperledger/aries-cloudagent-python/issues/1277, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ABOSNHU3WH6RFMARNALDT5DTVG3THANCNFSM47QAIG4A
.

That was the problem. Thanks you for the help!


Add Apalache server integration

Closes #822
This changest is focused on providing the minimal functionality needed for integration with the apalache gRPC server.
Unfortunately, due the the opsy nature of this goal, I've had to touch quite a few places. Sorry for the big diff.
The good news is: 3/4th of the diff is either package updates or relatively low density integration tests. So it's really not as bad as it looks :)
Much functionality is yet to be added: but this foundation will let us add the next steps in small iterations (such as more useful error handing, transparent management of apalache, and production of ITF traces).

[x] Tests added for any new code
[x] Ran npm run format (or had formatting run automatically on all files edited)
[x] Documentation added for any new functionality
[x] Entries added to the respective CHANGELOG.md for any new functionality
[x] Feature table on README.md updated for any listed functionality

Thanks for the review1

I thought this was merged already!


DC-560: On PRs, run integration tests with just-built docker container

This changes our PR process to run integration tests using the jib-built docker image instead of starting a server using gradle bootRun. This has the benefit of finding problems with the docker build process before a PR is merged up.
As part of this work, the jib job is now a step inside the integration-tests job. It could be done separately, but no other job uses the built image, and uploading/downloading the image as an artifact takes an extra ~2 minutes.
A further improvement on this process would be to deploy the built image to a GKE cluster and run the tests there, possibly using a BEE. That would also help debug our helm deployment configuration, and provide a more realistic test environment.
To see an example of a failed run with the boot run log output: https://github.com/DataBiosphere/terra-data-catalog/actions/runs/3565847046/jobs/5991554899
Example of failed run due to incorrect jib image creation: https://github.com/DataBiosphere/terra-data-catalog/actions/runs/3567373980/jobs/5995087936

@s-rubenstein

Do we have a sense of how long this adds to the runtime of our integration tests? Is that the 2 minutes called out in the PR description?

If only GitHub workflows was a real CI system then we could see some nice plots of job duration over time...
Looking at a bunch of passing test runs from this branch, integration tests take about 8 min 30 sec. Looking at builds from other PRs, the integration test job currently takes about 8 min 10 sec. So this appears to add about 20s overhead to the integration test and the build workflow overall.
It looks like the web app startup is faster, since it skips gradle, but now the jib build must be done before running the integration tests. Before it was done in parallel in a different job, so its time didn't affect the tests.


XLA compiled Embedding could work for out-of-bound input

Issue type
Bug
Have you reproduced the bug with TensorFlow Nightly?
Yes
Source
source
TensorFlow version
2.15.0-dev20230921
Custom code
Yes
OS platform and distribution
No response
Mobile device
No response
Python version
No response
Bazel version
No response
GCC/compiler version
No response
CUDA/cuDNN version
No response
GPU model and memory
No response
Current behavior?
XLA compiled Embedding could work for out-of-bound input. In the following code, the model has an embedding layer using tf.keras.layers.Embedding(64, 128), which means it has 64 possible tokens (usually representing words or items, indexed from 0 to 63).
The input to it is tf.constant([64], dtype=tf.int32), which is out of range because 64 exceeds the highest valid token index 63.
If we run the model without XLA compilation, it will raise an error as expected. However, after XLA compilation, the model could process such invalid input.
Standalone code to reproduce the issue
import tensorflow as tf
"""
XLA compiled
"""
class Model(tf.keras.Model):

    def __init__(self):
        super(Model, self).__init__()
        self.embedding = tf.keras.layers.Embedding(64, 128)

    @tf.function(jit_compile=True)
    def call(self, x1, x2):
        x3 = self.embedding(x1)
        return (x3 * x2)

tf.random.set_seed(42)
m = Model()
input_1 = tf.constant([64], dtype=tf.int32)
input_2 = tf.constant([[[[10.0]]]], dtype=tf.float32)
print(m(input_1, input_2))

"""
Without XLA
"""
class Model(tf.keras.Model):

    def __init__(self):
        super(Model, self).__init__()
        self.embedding = tf.keras.layers.Embedding(64, 128)

    def call(self, x1, x2):
        x3 = self.embedding(x1)
        return (x3 * x2)

tf.random.set_seed(42)
m = Model()
input_1 = tf.constant([64], dtype=tf.int32)
input_2 = tf.constant([[[[10.0]]]], dtype=tf.float32)
print(m(input_1, input_2))

"""
InvalidArgumentError: Exception encountered when calling layer 'embedding_4' (type Embedding).

{{function_node __wrapped__ResourceGather_device_/job:localhost/replica:0/task:0/device:CPU:0}} indices[0] = 64 is not in [0, 64) [Op:ResourceGather] name:
"""

Relevant log output
No response

@YangChenyuan,
Apologies for the delay. I tried to execute the mentioned code on tf-nightly(2.16.0-dev20231101) and the code was executed without any issue/error and the respective outputs are also the same in both the cases. Kindly find the gist of it here. Thank you!


Possible Out of Index bug when reasoning with Qwen

System Info
python==3.10.15
cuda==11.8-8.8.1
torch==2.4.0
The latest version of code
GPU A100_40G * 8
Who can help?
@ziyuwan @Gebro13 @mengfn @gzqaq @YanSong97 @i
Information

[X] The official example scripts
[ ] My own modified scripts

Tasks

[X] An officially supported task in the codebase (such as scrips/, ...)
[ ] My own task or dataset (give details below)

Reproduction

run prm/code/fintune_qwen.py to get prm based on Qwen2.5-Math-1.5B. (math-shepherd is worked)
run scripts create_service_qwen2.5_math_hf.sh to start service for eval. (NUM_LM_WORKER=2, NUM_RM_WORKER=2)
run cot_greey.sh for eval Math dataset

When I set NUM_LM_WORKER=1, NUM_RM_WORKER=1, it could work successfully.
91%|███████████████████████████████▊         | 454/500 [1:33:18<06:55,  9.04s/it]
But set NUM_LM_WORKER=2, NUM_RM_WORKER=2, it could fail.
0%|▏                                  | 1/500 [00:10<1:28:42, 10.67s/it]
Traceback (most recent call last):
...
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
I try to locate error, so I print some message near llm_service/worker/inference.py line:190
    if temperature < 1e-5 or top_p < 1e-8:  # greedy
        tmp, indices = torch.topk(last_token_logits, 2)
        tokens = [int(index) for index in indices.tolist()]

        if tokens[0] > 151936:
            print(f"prompt:{prompt}")
            print(f"tmp:{tmp}")
            print(f"last_token_logits:{last_token_logits} {last_token_logits.shape}")
            print(f"indices:{indices}")
            print(f"tokens:{tokens}")

I got the wrong information in all four experiments with different prompts. (Out of Index Error)
2024-11-09 12:21:48 | INFO | stdout | prompt:<|im_start|>system
2024-11-09 12:21:48 | INFO | stdout | Please reason step by step, and put your final answer within \boxed{{}}.<|im_end|>
2024-11-09 12:21:48 | INFO | stdout | <|im_start|>user
2024-11-09 12:21:48 | INFO | stdout | If $f(x) = \frac{3x-2}{x-2}$, what is the value of $f(-2) +f(-1)+f(0)$? Express your answer as a common fraction.<|im_end|>
2024-11-09 12:21:48 | INFO | stdout | <|im_start|>assistant
2024-11-09 12:21:48 | INFO | stdout |
2024-11-09 12:21:48 | INFO | stdout | tmp:tensor([nan, nan], device='cuda:0', dtype=torch.float16)
2024-11-09 12:21:48 | INFO | stdout | last_token_logits:tensor([nan, nan, nan,  ..., nan, nan, nan], device='cuda:0',
2024-11-09 12:21:48 | INFO | stdout |        dtype=torch.float16) torch.Size([151936])
2024-11-09 12:21:48 | INFO | stdout | indices:tensor([9223231297218904063,<PHONE_NUMBER>218904063], device='cuda:0')
2024-11-09 12:21:48 | INFO | stdout | tokens:[9223231297218904063,<PHONE_NUMBER>218904063]
2024-11-09 12:23:17 | INFO | stdout | prompt:<|im_start|>system
2024-11-09 12:23:17 | INFO | stdout | Please reason step by step, and put your final answer within \boxed{{}}.<|im_end|>
2024-11-09 12:23:17 | INFO | stdout | <|im_start|>user
2024-11-09 12:23:17 | INFO | stdout | If $f(x) = \frac{3x-2}{x-2}$, what is the value of $f(-2) +f(-1)+f(0)$? Express your answer as a common fraction.<|im_end|>
2024-11-09 12:23:17 | INFO | stdout | <|im_start|>assistant
2024-11-09 12:23:17 | INFO | stdout |
2024-11-09 12:23:17 | INFO | stdout | tmp:tensor([ 0.3206, -0.0068], device='cuda:0', dtype=torch.float16)
2024-11-09 12:23:17 | INFO | stdout | last_token_logits:tensor([nan, nan, nan,  ..., nan, nan, nan], device='cuda:0',
2024-11-09 12:23:17 | INFO | stdout |        dtype=torch.float16) torch.Size([151936])
2024-11-09 12:23:17 | INFO | stdout | indices:tensor([571746046575616,<PHONE_NUMBER>99872], device='cuda:0')
2024-11-09 12:23:17 | INFO | stdout | tokens:[571746046575616,<PHONE_NUMBER>99872]
2024-11-09 12:25:45 | INFO | stdout | prompt:<|im_start|>system
2024-11-09 12:25:45 | INFO | stdout | Please reason step by step, and put your final answer within \boxed{{}}.<|im_end|>
2024-11-09 12:25:45 | INFO | stdout | <|im_start|>user
2024-11-09 12:25:45 | INFO | stdout | What is the smallest positive perfect cube that can be written as the sum of three consecutive integers?<|im_end|>
2024-11-09 12:25:45 | INFO | stdout | <|im_start|>assistant
2024-11-09 12:25:45 | INFO | stdout |
2024-11-09 12:25:45 | INFO | stdout | tmp:tensor([nan, nan], device='cuda:0', dtype=torch.float16)
2024-11-09 12:25:45 | INFO | stdout | last_token_logits:tensor([nan, nan, nan,  ..., nan, nan, nan], device='cuda:0',
2024-11-09 12:25:45 | INFO | stdout |        dtype=torch.float16) torch.Size([151936])
2024-11-09 12:25:45 | INFO | stdout | indices:tensor([9223231297218904063,<PHONE_NUMBER>218904063], device='cuda:0')
2024-11-09 12:25:45 | INFO | stdout | tokens:[9223231297218904063,<PHONE_NUMBER>218904063]
2024-11-09 12:27:39 | INFO | stdout | prompt:<|im_start|>system
2024-11-09 12:27:39 | INFO | stdout | Please reason step by step, and put your final answer within \boxed{{}}.<|im_end|>
2024-11-09 12:27:39 | INFO | stdout | <|im_start|>user
2024-11-09 12:27:39 | INFO | stdout | What is the smallest positive perfect cube that can be written as the sum of three consecutive integers?<|im_end|>
2024-11-09 12:27:39 | INFO | stdout | <|im_start|>assistant
2024-11-09 12:27:39 | INFO | stdout |
2024-11-09 12:27:39 | INFO | stdout | tmp:tensor([nan, nan], device='cuda:0', dtype=torch.float16)
2024-11-09 12:27:39 | INFO | stdout | last_token_logits:tensor([nan, nan, nan,  ..., nan, nan, nan], device='cuda:0',
2024-11-09 12:27:39 | INFO | stdout |        dtype=torch.float16) torch.Size([151936])
2024-11-09 12:27:39 | INFO | stdout | indices:tensor([9223231297218904063,<PHONE_NUMBER>218904063], device='cuda:0')
2024-11-09 12:27:39 | INFO | stdout | tokens:[9223231297218904063,<PHONE_NUMBER>218904063]
Expected behavior
The expectation is to be able to run scripts properly.

hi @Dada-Cloudzxy
Thank you for discovering this issue.

run scripts create_service_qwen2.5_math_hf.sh to start service for eval. (NUM_LM_WORKER=2, NUM_RM_WORKER=2)
The hf model runner was adapted from Qwen-Math, it is still an inefficient version, so a quick solution is to use vllm API instead, can you use vLLM by run create_service_qwen2.5_math_vllm.sh?

In the meanwhile, we will try to fix this bug.

Thank you very much, I will give it a try

Thank you very much, I will give it a try

Hi, I failed to reproduce your error following the instructions. Could you provide more information regarding the error message? Such as the error message in each worker session. Many thanks.

@YanSong97 Hi, I met the same bug, the only difference was model. Concretely, I finetuned Meta-Llama-3-8B using my prm/code/finetune_llama.py to get PRM (llama3_prm_checkpoint-6358) and use Meta-Llama-3-8B-Instruct for reason eval on MATH. The modified code to reproduce the bug I met is in my forked repo: Repo Link, Meta-Llama-3-8B and Meta-Llama-3-8B-Instruct were downloaded from huggingface, PRM (llama3_prm_checkpoint-6358) checkpoint and steps to reproduce it are in release of my forked repo: ckpt link.
Here are some extra detailed info may be useful:
Bug info
2024-11-12 02:09:42 | ERROR | stderr |   File "/xxx/openr/reason/llm_service/workers/inference.py", line 202, in generate_stream
2024-11-12 02:09:42 | ERROR | stderr |     torch.log_softmax(logits[0, -1, :], dim=-1)[token].tolist()
2024-11-12 02:09:42 | ERROR | stderr | IndexError: index<PHONE_NUMBER>630828155 is out of bounds for dimension 0 with size 128256
Sys info
python==3.10.15
cuda==11.6
torch==2.4.0
The latest version of code
GPU A6000_48G * 4
Reproduction

download or reproduce PRM (llama3_prm_checkpoint-6358) checkpoint
run script bash reason/llm_service/create_service_llama3_8b_instruct_hf.sh in my forked repo to start service for eval (NUM_LM_WORKER=2, NUM_RM_WORKER=2, the bug still occurs when these two workers set to 1)
run script bash scripts/eval/beam_search_MATH_llama3_8b_instruct.sh in my forked repo to eval Math dataset


[Bug]: Qwen2-VL-72B Inference on Multiple-GPUs

Your current environment

The output of `python collect_env.py`
PyTorch version: 2.4.0+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.6 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
Clang version: Could not collect
CMake version: version 3.26.4
Libc version: glibc-2.31

Python version: 3.10.13 (main, Sep 11 2023, 13:44:35) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-4.18.0-372.76.1.el8_6.x86_64-x86_64-with-glibc2.31
Is CUDA available: True
CUDA runtime version: 12.1.105
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration:
GPU 0: Tesla V100-SXM2-32GB
GPU 1: Tesla V100-SXM2-32GB

Nvidia driver version: 525.60.13
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.9.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.9.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.9.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.9.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.9.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.9.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.9.0
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Byte Order:                      Little Endian
Address sizes:                   46 bits physical, 48 bits virtual
CPU(s):                          4
On-line CPU(s) list:             0-3
Thread(s) per core:              1
Core(s) per socket:              1
Socket(s):                       4
NUMA node(s):                    1
Vendor ID:                       GenuineIntel
CPU family:                      6
Model:                           85
Model name:                      Intel Xeon Processor (Cascadelake)
Stepping:                        5
CPU MHz:                         3092.672
BogoMIPS:                        6185.34
Virtualization:                  VT-x
Hypervisor vendor:               KVM
Virtualization type:             full
L1d cache:                       128 KiB
L1i cache:                       128 KiB
L2 cache:                        16 MiB
L3 cache:                        64 MiB
NUMA node0 CPU(s):               0-3
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Mmio stale data:   Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown
Vulnerability Retbleed:          Mitigation; Enhanced IBRS
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Enhanced IBRS, IBPB conditional, RSB filling, PBRSB-eIBRS SW sequence
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Mitigation; TSX disabled
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves arat umip pku ospke avx512_vnni md_clear arch_capabilities

Versions of relevant libraries:
[pip3] numpy==1.26.2
[pip3] nvidia-cublas-cu12==<IP_ADDRESS>
[pip3] nvidia-cuda-cupti-cu12==12.1.105
[pip3] nvidia-cuda-nvrtc-cu12==12.1.105
[pip3] nvidia-cuda-runtime-cu12==12.1.105
[pip3] nvidia-cudnn-cu12==<IP_ADDRESS>
[pip3] nvidia-cufft-cu12==<IP_ADDRESS>
[pip3] nvidia-curand-cu12==<IP_ADDRESS>
[pip3] nvidia-cusolver-cu12==<IP_ADDRESS>
[pip3] nvidia-cusparse-cu12==<IP_ADDRESS>
[pip3] nvidia-ml-py==12.560.30
[pip3] nvidia-nccl-cu12==2.20.5
[pip3] nvidia-nvjitlink-cu12==12.6.77
[pip3] nvidia-nvtx-cu12==12.1.105
[pip3] pyzmq==26.2.0
[pip3] torch==2.4.0
[pip3] torchaudio==2.1.2
[pip3] torchelastic==0.2.2
[pip3] torchvision==0.19.0
[pip3] transformers==4.45.2
[pip3] triton==3.0.0
[conda] blas                      1.0                         mkl
[conda] cuda-cudart               12.1.105                      0    nvidia
[conda] cuda-cupti                12.1.105                      0    nvidia
[conda] cuda-libraries            12.1.0                        0    nvidia
[conda] cuda-nvrtc                12.1.105                      0    nvidia
[conda] cuda-nvtx                 12.1.105                      0    nvidia
[conda] cuda-opencl               12.3.101                      0    nvidia
[conda] cuda-runtime              12.1.0                        0    nvidia
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
[conda] libcublas                 <IP_ADDRESS>                     0    nvidia
[conda] libcufft                  <IP_ADDRESS>                      0    nvidia
[conda] libcufile                 <IP_ADDRESS>                       0    nvidia
[conda] libcurand                 <IP_ADDRESS>                    0    nvidia
[conda] libcusolver               <IP_ADDRESS>                     0    nvidia
[conda] libcusparse               <IP_ADDRESS>                     0    nvidia
[conda] libjpeg-turbo             2.0.0                h9bf148f_0    pytorch
[conda] libnpp                    <IP_ADDRESS>                     0    nvidia
[conda] libnvjitlink              12.1.105                      0    nvidia
[conda] libnvjpeg                 <IP_ADDRESS>                     0    nvidia
[conda] mkl                       2023.1.0         h213fc3f_46344
[conda] mkl-service               2.4.0           py310h5eee18b_1
[conda] mkl_fft                   1.3.8           py310h5eee18b_0
[conda] mkl_random                1.2.4           py310hdb19cb5_0
[conda] numpy                     1.26.2          py310h5f9d8c6_0
[conda] numpy-base                1.26.2          py310hb5e798b_0
[conda] nvidia-cublas-cu12        <IP_ADDRESS>                 pypi_0    pypi
[conda] nvidia-cuda-cupti-cu12    12.1.105                 pypi_0    pypi
[conda] nvidia-cuda-nvrtc-cu12    12.1.105                 pypi_0    pypi
[conda] nvidia-cuda-runtime-cu12  12.1.105                 pypi_0    pypi
[conda] nvidia-cudnn-cu12         <IP_ADDRESS>                 pypi_0    pypi
[conda] nvidia-cufft-cu12         <IP_ADDRESS>                pypi_0    pypi
[conda] nvidia-curand-cu12        <IP_ADDRESS>               pypi_0    pypi
[conda] nvidia-cusolver-cu12      <IP_ADDRESS>               pypi_0    pypi
[conda] nvidia-cusparse-cu12      <IP_ADDRESS>               pypi_0    pypi
[conda] nvidia-ml-py              12.560.30                pypi_0    pypi
[conda] nvidia-nccl-cu12          2.20.5                   pypi_0    pypi
[conda] nvidia-nvjitlink-cu12     12.6.77                  pypi_0    pypi
[conda] nvidia-nvtx-cu12          12.1.105                 pypi_0    pypi
[conda] pytorch-cuda              12.1                 ha16c6d3_5    pytorch
[conda] pytorch-mutex             1.0                        cuda    pytorch
[conda] pyzmq                     26.2.0                   pypi_0    pypi
[conda] torch                     2.4.0                    pypi_0    pypi
[conda] torchaudio                2.1.2               py310_cu121    pytorch
[conda] torchelastic              0.2.2                    pypi_0    pypi
[conda] torchvision               0.19.0                   pypi_0    pypi
[conda] transformers              4.45.2                   pypi_0    pypi
[conda] triton                    3.0.0                    pypi_0    pypi
ROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: dev
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
GPU0    GPU1    CPU Affinity    NUMA Affinity
GPU0     X      NV2     0-3             N/A
GPU1    NV2      X      0-3             N/A

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

Model Input Dumps
No response
🐛 Describe the bug
running the Qwen2-VL-72B-INT4 GPTQ quantized model on multiple GPU.
After running the model checkpoints are loading properly as shown below but after loading it is throwing an error because of memory issue.

The complete process logs are below:
==========
== CUDA ==
CUDA Version 12.1.1
Container image Copyright (c) 2016-2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license
A copy of this license is made available in this container at /NGC-DL-CONTAINER-LICENSE for your convenience.
/opt/conda/lib/python3.10/site-packages/vllm/connections.py:8: RuntimeWarning: Failed to read commit hash:
No module named 'vllm._version'
from vllm.version import version as VLLM_VERSION
INFO 10-15 06:52:06 api_server.py:528] vLLM API server version dev
INFO 10-15 06:52:06 api_server.py:529] args: Namespace(subparser='serve', model_tag='Models/qwen2-VL-72B/', config='', host='<IP_ADDRESS>', port=8000, uvicorn_log_level='info', allow_credentials=False, allowed_origins=[''], allowed_methods=[''], allowed_headers=['*'], api_key='abc123', lora_modules=None, prompt_adapters=None, chat_template=None, response_role='assistant', ssl_keyfile=None, ssl_certfile=None, ssl_ca_certs=None, ssl_cert_reqs=0, root_path=None, middleware=[], return_tokens_as_token_ids=False, disable_frontend_multiprocessing=False, enable_auto_tool_choice=False, tool_call_parser=None, tool_parser_plugin='', model='/opt/app-root/src/llm-storate/Models/qwen2-VL-72B/', tokenizer=None, skip_tokenizer_init=False, revision=None, code_revision=None, tokenizer_revision=None, tokenizer_mode='auto', trust_remote_code=False, download_dir=None, load_format='auto', config_format='auto', dtype='half', kv_cache_dtype='auto', quantization_param_path=None, max_model_len=8192, guided_decoding_backend='outlines', distributed_executor_backend=None, worker_use_ray=False, pipeline_parallel_size=1, tensor_parallel_size=2, max_parallel_loading_workers=None, ray_workers_use_nsight=False, block_size=16, enable_prefix_caching=False, disable_sliding_window=False, use_v2_block_manager=True, num_lookahead_slots=0, seed=0, swap_space=4, cpu_offload_gb=0, gpu_memory_utilization=0.9, num_gpu_blocks_override=None, max_num_batched_tokens=None, max_num_seqs=256, max_logprobs=20, disable_log_stats=False, quantization=None, rope_scaling=None, rope_theta=None, enforce_eager=False, max_context_len_to_capture=None, max_seq_len_to_capture=8192, disable_custom_all_reduce=False, tokenizer_pool_size=0, tokenizer_pool_type='ray', tokenizer_pool_extra_config=None, limit_mm_per_prompt=None, mm_processor_kwargs=None, enable_lora=False, max_loras=1, max_lora_rank=16, lora_extra_vocab_size=256, lora_dtype='auto', long_lora_scaling_factors=None, max_cpu_loras=None, fully_sharded_loras=False, enable_prompt_adapter=False, max_prompt_adapters=1, max_prompt_adapter_token=0, device='auto', num_scheduler_steps=1, multi_step_stream_outputs=True, scheduler_delay_factor=0.0, enable_chunked_prefill=None, speculative_model=None, speculative_model_quantization=None, num_speculative_tokens=None, speculative_disable_mqa_scorer=False, speculative_draft_tensor_parallel_size=None, speculative_max_model_len=None, speculative_disable_by_batch_size=None, ngram_prompt_lookup_max=None, ngram_prompt_lookup_min=None, spec_decoding_acceptance_method='rejection_sampler', typical_acceptance_sampler_posterior_threshold=None, typical_acceptance_sampler_posterior_alpha=None, disable_logprobs_during_spec_decoding=None, model_loader_extra_config=None, ignore_patterns=[], preemption_mode=None, served_model_name=['qwen2-vl'], qlora_adapter_name_or_path=None, otlp_traces_endpoint=None, collect_detailed_traces=None, disable_async_output_proc=False, override_neuron_config=None, scheduling_policy='fcfs', disable_log_requests=False, max_log_len=None, disable_fastapi_docs=False, dispatch_function=<function serve at 0x7f49d2799ea0>)
INFO 10-15 06:52:06 api_server.py:166] Multiprocessing frontend to use ipc:///tmp/a6394202-5f62-4042-9b71-56cdf6c0ea36 for IPC Path.
INFO 10-15 06:52:06 api_server.py:179] Started engine process with PID 33
You are using a model of type qwen2_vl to instantiate a model of type . This is not supported for all configurations of models and can yield errors.
/opt/conda/lib/python3.10/site-packages/vllm/connections.py:8: RuntimeWarning: Failed to read commit hash:
No module named 'vllm._version'
from vllm.version import version as VLLM_VERSION
You are using a model of type qwen2_vl to instantiate a model of type . This is not supported for all configurations of models and can yield errors.
WARNING 10-15 06:52:16 config.py:306] gptq quantization is not fully optimized yet. The speed can be slower than non-quantized models.
INFO 10-15 06:52:16 config.py:887] Defaulting to use mp for distributed inference
WARNING 10-15 06:52:21 config.py:306] gptq quantization is not fully optimized yet. The speed can be slower than non-quantized models.
INFO 10-15 06:52:21 config.py:887] Defaulting to use mp for distributed inference
INFO 10-15 06:52:21 llm_engine.py:237] Initializing an LLM engine (vdev) with config: model='/opt/app-root/src/llm-storate/Models/qwen2-VL-72B/', speculative_config=None, tokenizer='/opt/app-root/src/llm-storate/Models/qwen2-VL-72B/', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config=None, rope_scaling=None, rope_theta=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.float16, max_seq_len=8192, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=2, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=gptq, enforce_eager=False, kv_cache_dtype=auto, quantization_param_path=None, device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='outlines'), observability_config=ObservabilityConfig(otlp_traces_endpoint=None, collect_model_forward_time=False, collect_model_execute_time=False), seed=0, served_model_name=qwen2-vl, use_v2_block_manager=True, num_scheduler_steps=1, chunked_prefill_enabled=False multi_step_stream_outputs=True, enable_prefix_caching=False, use_async_output_proc=True, use_cached_outputs=True, mm_processor_kwargs=None)
WARNING 10-15 06:52:21 multiproc_gpu_executor.py:53] Reducing Torch parallelism from 4 threads to 1 to avoid unnecessary CPU contention. Set OMP_NUM_THREADS in the external environment to tune this value as needed.
INFO 10-15 06:52:21 custom_cache_manager.py:17] Setting Triton cache manager to: vllm.triton_utils.custom_cache_manager:CustomCacheManager
INFO 10-15 06:52:21 utils.py:581] Port 8000 is already in use, trying port 8001
INFO 10-15 06:52:21 utils.py:581] Port 8001 is already in use, trying port 8002
INFO 10-15 06:52:21 selector.py:224] Cannot use FlashAttention-2 backend for Volta and Turing GPUs.
INFO 10-15 06:52:21 selector.py:115] Using XFormers backend.
/opt/conda/lib/python3.10/site-packages/xformers/ops/fmha/flash.py:211: FutureWarning: torch.library.impl_abstract was renamed to torch.library.register_fake. Please use that instead; we will remove torch.library.impl_abstract in a future version of PyTorch.
@torch.library.impl_abstract("xformers_flash::flash_fwd")
/opt/conda/lib/python3.10/site-packages/xformers/ops/fmha/flash.py:344: FutureWarning: torch.library.impl_abstract was renamed to torch.library.register_fake. Please use that instead; we will remove torch.library.impl_abstract in a future version of PyTorch.
@torch.library.impl_abstract("xformers_flash::flash_bwd")
/opt/conda/lib/python3.10/site-packages/vllm/connections.py:8: RuntimeWarning: Failed to read commit hash:
No module named 'vllm._version'
from vllm.version import version as VLLM_VERSION
(VllmWorkerProcess pid=56) INFO 10-15 06:52:30 selector.py:224] Cannot use FlashAttention-2 backend for Volta and Turing GPUs.
(VllmWorkerProcess pid=56) INFO 10-15 06:52:30 selector.py:115] Using XFormers backend.
(VllmWorkerProcess pid=56) /opt/conda/lib/python3.10/site-packages/xformers/ops/fmha/flash.py:211: FutureWarning: torch.library.impl_abstract was renamed to torch.library.register_fake. Please use that instead; we will remove torch.library.impl_abstract in a future version of PyTorch.
(VllmWorkerProcess pid=56)   @torch.library.impl_abstract("xformers_flash::flash_fwd")
(VllmWorkerProcess pid=56) /opt/conda/lib/python3.10/site-packages/xformers/ops/fmha/flash.py:344: FutureWarning: torch.library.impl_abstract was renamed to torch.library.register_fake. Please use that instead; we will remove torch.library.impl_abstract in a future version of PyTorch.
(VllmWorkerProcess pid=56)   @torch.library.impl_abstract("xformers_flash::flash_bwd")
(VllmWorkerProcess pid=56) INFO 10-15 06:52:31 multiproc_worker_utils.py:216] Worker ready; awaiting tasks
INFO 10-15 06:52:34 utils.py:1008] Found nccl from library libnccl.so.2
INFO 10-15 06:52:34 pynccl.py:63] vLLM is using nccl==2.20.5
(VllmWorkerProcess pid=56) INFO 10-15 06:52:34 utils.py:1008] Found nccl from library libnccl.so.2
(VllmWorkerProcess pid=56) INFO 10-15 06:52:34 pynccl.py:63] vLLM is using nccl==2.20.5
INFO 10-15 06:52:34 custom_all_reduce_utils.py:204] generating GPU P2P access cache in /.cache/vllm/gpu_p2p_access_cache_for_0,1.json
INFO 10-15 06:52:51 custom_all_reduce_utils.py:242] reading GPU P2P access cache from /.cache/vllm/gpu_p2p_access_cache_for_0,1.json
(VllmWorkerProcess pid=56) INFO 10-15 06:52:51 custom_all_reduce_utils.py:242] reading GPU P2P access cache from /.cache/vllm/gpu_p2p_access_cache_for_0,1.json
INFO 10-15 06:52:51 utils.py:581] Port 8000 is already in use, trying port 8001
INFO 10-15 06:52:51 utils.py:581] Port 8001 is already in use, trying port 8002
INFO 10-15 06:52:51 utils.py:581] Port 8002 is already in use, trying port 8003
INFO 10-15 06:52:51 shm_broadcast.py:241] vLLM message queue communication handle: Handle(connect_ip='<IP_ADDRESS>', local_reader_ranks=[1], buffer=<vllm.distributed.device_communicators.shm_broadcast.ShmRingBuffer object at 0x7f1498634b20>, local_subscribe_port=8003, remote_subscribe_port=None)
INFO 10-15 06:52:51 model_runner.py:1060] Starting to load model /opt/app-root/src/llm-storate/Models/qwen2-VL-72B/...
(VllmWorkerProcess pid=56) INFO 10-15 06:52:51 model_runner.py:1060] Starting to load model /opt/app-root/src/llm-storate/Models/qwen2-VL-72B/...
INFO 10-15 06:52:51 selector.py:224] Cannot use FlashAttention-2 backend for Volta and Turing GPUs.
INFO 10-15 06:52:51 selector.py:115] Using XFormers backend.
(VllmWorkerProcess pid=56) INFO 10-15 06:52:51 selector.py:224] Cannot use FlashAttention-2 backend for Volta and Turing GPUs.
(VllmWorkerProcess pid=56) INFO 10-15 06:52:51 selector.py:115] Using XFormers backend.
Loading safetensors checkpoint shards:   0% Completed | 0/11 [00:00<?, ?it/s]
Loading safetensors checkpoint shards:   9% Completed | 1/11 [00:03<00:32,  3.20s/it]
Loading safetensors checkpoint shards:  18% Completed | 2/11 [00:06<00:29,  3.26s/it]
Loading safetensors checkpoint shards:  27% Completed | 3/11 [00:09<00:25,  3.23s/it]
Loading safetensors checkpoint shards:  36% Completed | 4/11 [00:12<00:22,  3.18s/it]
Loading safetensors checkpoint shards:  45% Completed | 5/11 [00:15<00:18,  3.14s/it]
Loading safetensors checkpoint shards:  55% Completed | 6/11 [00:18<00:14,  3.00s/it]
Loading safetensors checkpoint shards:  64% Completed | 7/11 [00:21<00:11,  2.87s/it]
Loading safetensors checkpoint shards:  73% Completed | 8/11 [00:22<00:07,  2.46s/it]
Loading safetensors checkpoint shards:  82% Completed | 9/11 [00:25<00:05,  2.60s/it]
Loading safetensors checkpoint shards:  91% Completed | 10/11 [00:28<00:02,  2.73s/it]
Loading safetensors checkpoint shards: 100% Completed | 11/11 [00:30<00:00,  2.56s/it]
Loading safetensors checkpoint shards: 100% Completed | 11/11 [00:30<00:00,  2.81s/it]
INFO 10-15 06:53:24 model_runner.py:1071] Loading model weights took 20.0451 GB
(VllmWorkerProcess pid=56) INFO 10-15 06:53:25 model_runner.py:1071] Loading model weights took 20.0451 GB
INFO 10-15 06:53:39 distributed_gpu_executor.py:57] # GPU blocks: 1858, # CPU blocks: 1638
INFO 10-15 06:53:39 distributed_gpu_executor.py:61] Maximum concurrency for 8192 tokens per request: 3.63x
(VllmWorkerProcess pid=56) INFO 10-15 06:53:45 model_runner.py:1402] Capturing the model for CUDA graphs. This may lead to unexpected consequences if the model is not static. To run the model in eager mode, set 'enforce_eager=True' or use '--enforce-eager' in the CLI.
(VllmWorkerProcess pid=56) INFO 10-15 06:53:45 model_runner.py:1406] CUDA graphs can take additional 1~3 GiB memory per GPU. If you are running out of memory, consider decreasing gpu_memory_utilization or enforcing eager mode. You can also reduce the max_num_seqs as needed to decrease memory usage.
INFO 10-15 06:53:46 model_runner.py:1402] Capturing the model for CUDA graphs. This may lead to unexpected consequences if the model is not static. To run the model in eager mode, set 'enforce_eager=True' or use '--enforce-eager' in the CLI.
INFO 10-15 06:53:46 model_runner.py:1406] CUDA graphs can take additional 1~3 GiB memory per GPU. If you are running out of memory, consider decreasing gpu_memory_utilization or enforcing eager mode. You can also reduce the max_num_seqs as needed to decrease memory usage.
(VllmWorkerProcess pid=56) INFO 10-15 06:54:36 custom_all_reduce.py:233] Registering 4025 cuda graph addresses
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231] Exception in worker VllmWorkerProcess while processing method initialize_cache: CUDA out of memory. Tried to allocate 464.00 MiB. GPU 1 has a total capacity of 31.74 GiB of which 346.12 MiB is free. Process 3040327 has 31.39 GiB memory in use. Of the allocated memory 25.18 GiB is allocated by PyTorch, with 504.07 MiB allocated in private pools (e.g., CUDA Graphs), and 227.17 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables), Traceback (most recent call last):
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]   File "/opt/conda/lib/python3.10/site-packages/vllm/executor/multiproc_worker_utils.py", line 224, in _run_worker_process
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]     output = executor(*args, **kwargs)
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]   File "/opt/conda/lib/python3.10/site-packages/vllm/worker/worker.py", line 271, in initialize_cache
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]     self._warm_up_model()
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]   File "/opt/conda/lib/python3.10/site-packages/vllm/worker/worker.py", line 287, in _warm_up_model
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]     self.model_runner.capture_model(self.gpu_cache)
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]   File "/opt/conda/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]     return func(*args, **kwargs)
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]   File "/opt/conda/lib/python3.10/site-packages/vllm/worker/model_runner.py", line 1522, in capture_model
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]     graph_runner.capture(**capture_inputs)
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]   File "/opt/conda/lib/python3.10/site-packages/vllm/worker/model_runner.py", line 1785, in capture
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]     self.model(
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]   File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]     return self._call_impl(*args, **kwargs)
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]   File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]     return forward_call(*args, **kwargs)
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]   File "/opt/conda/lib/python3.10/site-packages/vllm/model_executor/models/qwen2_vl.py", line 1092, in forward
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]     hidden_states = self.model(
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]   File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]     return self._call_impl(*args, **kwargs)
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]   File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]     return forward_call(*args, **kwargs)
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]   File "/opt/conda/lib/python3.10/site-packages/vllm/model_executor/models/qwen2.py", line 288, in forward
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]     hidden_states, residual = layer(
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]   File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]     return self._call_impl(*args, **kwargs)
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]   File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]     return forward_call(*args, **kwargs)
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]   File "/opt/conda/lib/python3.10/site-packages/vllm/model_executor/models/qwen2.py", line 220, in forward
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]     hidden_states = self.mlp(hidden_states)
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]   File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]     return self._call_impl(*args, **kwargs)
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]   File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]     return forward_call(*args, **kwargs)
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]   File "/opt/conda/lib/python3.10/site-packages/vllm/model_executor/models/qwen2.py", line 79, in forward
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]     gate_up, _ = self.gate_up_proj(x)
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]   File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]     return self._call_impl(*args, **kwargs)
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]   File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in call_impl
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]     return forward_call(*args, **kwargs)
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]   File "/opt/conda/lib/python3.10/site-packages/vllm/model_executor/layers/linear.py", line 371, in forward
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]     output_parallel = self.quant_method.apply(self, input, bias)
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]   File "/opt/conda/lib/python3.10/site-packages/vllm/model_executor/layers/quantization/gptq.py", line 242, in apply
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]     output = ops.gptq_gemm(reshaped_x, layer.qweight, layer.qzeros,
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]   File "/opt/conda/lib/python3.10/site-packages/vllm/_custom_ops.py", line 45, in wrapper
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]     return fn(*args, **kwargs)
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]   File "/opt/conda/lib/python3.10/site-packages/vllm/_custom_ops.py", line 274, in gptq_gemm
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]     return torch.ops._C.gptq_gemm(a, b_q_weight, b_gptq_qzeros, b_gptq_scales,
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]   File "/opt/conda/lib/python3.10/site-packages/torch/ops.py", line 1061, in call
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]     return self._op(*args, **(kwargs or {}))
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231] torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 464.00 MiB. GPU 1 has a total capacity of 31.74 GiB of which 346.12 MiB is free. Process 3040327 has 31.39 GiB memory in use. Of the allocated memory 25.18 GiB is allocated by PyTorch, with 504.07 MiB allocated in private pools (e.g., CUDA Graphs), and 227.17 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
(VllmWorkerProcess pid=56) ERROR 10-15 06:54:36 multiproc_worker_utils.py:231]
INFO 10-15 06:54:36 custom_all_reduce.py:233] Registering 4025 cuda graph addresses
INFO 10-15 06:54:37 multiproc_worker_utils.py:121] Killing local vLLM worker processes
Process SpawnProcess-1:
Traceback (most recent call last):
File "/opt/conda/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/opt/conda/lib/python3.10/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/opt/conda/lib/python3.10/site-packages/vllm/engine/multiprocessing/engine.py", line 392, in run_mp_engine
engine = MQLLMEngine.from_engine_args(engine_args=engine_args,
File "/opt/conda/lib/python3.10/site-packages/vllm/engine/multiprocessing/engine.py", line 141, in from_engine_args
return cls(
File "/opt/conda/lib/python3.10/site-packages/vllm/engine/multiprocessing/engine.py", line 78, in init
self.engine = LLMEngine(*args,
File "/opt/conda/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 349, in init
self._initialize_kv_caches()
File "/opt/conda/lib/python3.10/site-packages/vllm/engine/llm_engine.py", line 497, in _initialize_kv_caches
self.model_executor.initialize_cache(num_gpu_blocks, num_cpu_blocks)
File "/opt/conda/lib/python3.10/site-packages/vllm/executor/distributed_gpu_executor.py", line 67, in initialize_cache
self._run_workers("initialize_cache",
File "/opt/conda/lib/python3.10/site-packages/vllm/executor/multiproc_gpu_executor.py", line 192, in _run_workers
driver_worker_output = driver_worker_method(*args, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/vllm/worker/worker.py", line 271, in initialize_cache
self._warm_up_model()
File "/opt/conda/lib/python3.10/site-packages/vllm/worker/worker.py", line 287, in _warm_up_model
self.model_runner.capture_model(self.gpu_cache)
File "/opt/conda/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/vllm/worker/model_runner.py", line 1522, in capture_model
graph_runner.capture(**capture_inputs)
File "/opt/conda/lib/python3.10/site-packages/vllm/worker/model_runner.py", line 1785, in capture
self.model(
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/vllm/model_executor/models/qwen2_vl.py", line 1092, in forward
hidden_states = self.model(
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/vllm/model_executor/models/qwen2.py", line 288, in forward
hidden_states, residual = layer(
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/vllm/model_executor/models/qwen2.py", line 220, in forward
hidden_states = self.mlp(hidden_states)
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/vllm/model_executor/models/qwen2.py", line 79, in forward
gate_up, _ = self.gate_up_proj(x)
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in call_impl
return forward_call(*args, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/vllm/model_executor/layers/linear.py", line 371, in forward
output_parallel = self.quant_method.apply(self, input, bias)
File "/opt/conda/lib/python3.10/site-packages/vllm/model_executor/layers/quantization/gptq.py", line 242, in apply
output = ops.gptq_gemm(reshaped_x, layer.qweight, layer.qzeros,
File "/opt/conda/lib/python3.10/site-packages/vllm/_custom_ops.py", line 45, in wrapper
return fn(*args, **kwargs)
File "/opt/conda/lib/python3.10/site-packages/vllm/_custom_ops.py", line 274, in gptq_gemm
return torch.ops._C.gptq_gemm(a, b_q_weight, b_gptq_qzeros, b_gptq_scales,
File "/opt/conda/lib/python3.10/site-packages/torch/ops.py", line 1061, in call
return self._op(*args, **(kwargs or {}))
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 464.00 MiB. GPU 0 has a total capacity of 31.74 GiB of which 314.12 MiB is free. Process 3039754 has 31.42 GiB memory in use. Of the allocated memory 25.18 GiB is allocated by PyTorch, with 504.07 MiB allocated in private pools (e.g., CUDA Graphs), and 227.17 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
Before submitting a new issue...

[X] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

I think this is because the FlashAttention-2 backend is not enabled and XFormers backend is used, which resulting in a increase need for vRAM. In addition, I ran Qwen2-VL-7B-Instruct-GPTQ-Int4 with vllm v0.6.3 and encountered a similar problem. Will vllm support FlashAttention-2 in the near future?

We don't support quantization for VLMs at the moment. I have added this to the issue tracker. cc @mgoin @Isotr0py

Qwen2-VL with GPTQ/AWQ quantization is supported because Qwen team only quantized the LLM backbone for Qwen2-VL, which is compatible with current VLM quantization implementation on VLLM.
@bhupendra1324 You can try adding enforce_eager=True and reduce max_num_seqs=1 to reduce the VRAM usage.

I think this is because the FlashAttention-2 backend is not enabled and XFormers backend is used, which resulting in a increase need for vRAM. Will vllm support FlashAttention-2 in the near future?

If you are running on GPU like V100/T4, it's not possible to use Flash-Attention2 backend because FA2 requires GPU with Ampere or newer architecture.

@Isotr0py  I am using V100 GPUs. Also, it checks for FlashAttention compatibility and loads in other backend. As you can see the model is loading but it is taking only one GPU for loading and throwing CUDA out of memory error.

According to the log above, the model has been loaded to 2 GPUs correctly: the Qwen2-VL-72B with 4bit weights needs about 44GB VRAM for loading. And each GPU took about 20GB for loading, which should work as expected.
Seems that OOM occurred during profile run, which will run with the largest available dummy image, you may need to reduce the max_pixels in preprocessor_config.json as well.

@Isotr0py thank you for your inputs, reducing the max_pixels or adding up more GPU worked.


Slightly increment size for TestIncrementalBackup

This should make the test more reliable, it should hit the accidental
"repo is has grown too much" way less often.

Codecov Report

Merging #1699 into master will increase coverage by 0.03%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #1699      +/-   ##
==========================================
+ Coverage   52.34%   52.37%   +0.03%
==========================================
  Files         148      148
  Lines       11639    11639
==========================================
+ Hits         6092     6096       +4
+ Misses       4606     4603       -3
+ Partials      941      940       -1

Impacted Files
Coverage Δ

internal/backend/rclone/backend.go
70.32% <0%> (-1.94%)
:arrow_down:

internal/archiver/archiver.go
64.25% <0%> (+0.21%)
:arrow_up:

internal/backend/s3/s3.go
63.07% <0%> (+2.3%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 22d5061...4544a77. Read the comment docs.


 Do you have any recommendations as to how to remove it? Thank you.

I don't think I've encountered that before. It looks like it could either be a rounding/1-pixel inaccuracy in the renderer, or something with the texturing/texture extraction. I wonder where it's coming from - can you render the model onto a white background? If the black edge is then still there, then it's most likely coming from the texturing.

The black edge is still there with a white background, but the isomap looks fine to me.

Ok, interesting. Can you provide the input landmarks and image, and which fitting exactly you ran and with which parameters?
Does it occur with every image/rendering, or only with a few?
I'm actually not using eos's rendering much, you can also just export an obj and then use MeshLab/Blender for rendering, or use some OpenGL rendering, which I'm sure will render it better than eos's software rendering.

@patrikhuber Do you have  any OpenGL rendering code ? Simple demo code is enough.

@chrisluu I don't have any OpenGL code. But it should be easy to do, as all matrices should be OpenGL compatible, and glm works well with OpenGL.

@patrikhuber Alright, I not familiar with OpenGL but I will have a try.Thanks.

Apologies for the delay. The black line issue happens to all images. I guess it's a problem with the texture. For now I'm still using eos's rendering and doing something like this
for each vertex in mesh:
project the vertex onto the 2d image
assign the pixel color as the vertex color
I don't know if this method is favourable. I'll experiment more with the texture some time in late-April.


[pixman] Fix x86 builds not using MSVC

Fixes the #Todo comment in pixman's portfile.cmale

There are two situations when modifying files under vcpkg/ports/portName:

If a version is released upstream, update the "version" field in vcpkg.json and count "port-version" from 0 (port-version is a field that records changes specific to vcpkg, 0 generally means deleting "port-version" line). Reference: 38749/files.
If the upstream has not released a new version, then after modifying other files, please accumulate the "port-version" value to record this modification. Reference: 37966/files.

Please read the documentation: maintainer guide#versioning.

At present, the version information of your commit is inconsistent with your local one:
##[error]Found the following errors:
##[error]versions/p-/pixman.json(2,): error : while parsing versions for pixman from version/p-/pixman.json
0.43.4 is declared with f7e1a97a6af1ad657489f901a6ca14db24652eff, but the local port has a different SHA a4e66badb15b7db89e768b1a1943583bff608b2c.

Please run ./vcpkg x-add-version testPort [--overwrite-version] before each push:

For any file modification under the ports/testPort/ folder, please commit after modification
Run ./vcpkg x-add-version portName to generate a series of version according to your commit info (Add --overwrite-version when generating other than the first time)
Commit verison information again
Push together

--overwrite-version
After the first modification, please run ./vcpkg x-add-version testPort to generate the corresponding json; for subsequent modifications of the same version, please use ./vcpkg x-add-version portName --overwrite-version to update git-tree.
maintainer guide#versioning.

Note: I will be converting your PR to draft status. When you're ready, please revert to "ready for review".

@microsoft-github-policy-service agree company="Flole"


RouteHandler nesting requirement?

If I have a tree like this

Home

Inbox

Message x

I found it a little surprising that when I started with react-router which is that if I don't nest a RouteHandler throughout an entire tree: Home and Inbox, click a link to say Message x, will end me up at Home.
It would make more sense to me that react-router would apply the highest level available RouteHandler to everything below it.
Because, if Inbox is its own page design which has nothing that transfers to Message x except the url, you end up using stubs for this, right? In the example, what if message takes over the entire inbox page?

Home

Inbox stub

Inbox
Message x

Perhaps I'm not yet getting it all correctly.

Sorry, can you give a simple (pseudo)code/config example to make this more clear?

I'm not fully understanding, but let me take stab at it.  Are you saying:

Your Message view does not share UI with Inbox?
 The point of nesting is to share UI. You can still "nest" the urls.
<Route handler={App}>
  <Route name="inbox" path="/inbox" handler={Inbox}/>
  <Route name="message" path="/inbox/messages/:messageId" handler={Inbox}/>
</Route>


SiteSystems to CMBoundaryGroups

Pull Request (PR) description
Added the capability to match\add\remove Site Systems to Boundary Groups.  Updated Unit tests and all associated files.
This Pull Request (PR) fixes the following issues

Task list

[X] Added an entry to the change log under the Unreleased section of the
file CHANGELOG.md. Entry should say what was changed and how that
affects users (if applicable), and reference the issue being resolved
(if applicable).
[X] Resource documentation added/updated in README.md.
[X] Resource parameter descriptions added/updated in README.md, schema.mof
and comment-based help.
[X] Comment-based help added/updated.
[X] Localization strings added/updated in all localization files as appropriate.
[X] Examples appropriately added/updated.
[X] Unit tests added/updated. See DSC Community Testing Guidelines.
[ ] Integration tests added/updated (where possible). See DSC Community Testing Guidelines.
[X] New/changed code adheres to DSC Community Style Guidelines.

This change is

Thought about adding Security Scopes to this resource before merging.


to_webp use cases?

Hi, the to_webp flag is not so usefull since the <picture> tag is made also for that:
<picture>
  <source type="image/webp" srcset="flower.webp">
  <source type="image/jpeg" srcset="flower.jpg">

</picture>

It will try to load the .webp if supported, if not it will try with .jpg. If the picture tag isn't supported at all old browsers will show the  with the default src...
I can't figure out when using it.. maybe it can be removed to optimise the code?

Hi and thank you for opening this.
The to_webp option in fact is used by many, since the picture tag is the most verbous one when you need to do responsive images and all your image variants have the width/height ratio.
For brevity, and for performance optimization, it is much better to use the srcset and sized attributes of the img tag.
If you wanted to have responsive images using the picture tag you are, of course, free to do it, but please consider the length of the HTML portion which will be required to just render a single image.

If you want to see use cases of to_webp please take a look at the demos folder. Try and get the same result using the picture tag and see the difference.

Sorry @verlok i don't get the point. Since i can add both media and srcset to the source elements, why not just preload the <picture>? By this way i just need to declare my semantic html with the resources i need.
<picture>
  <source type="image/webp" media="(min-width: 650px)"  srcset="flower.webp">
  <source type="image/jpeg" media="(min-width: 465px)" srcset="flower.jpg">

</picture>

I just think that to_webp is something outside the scope of this library that should only allow you to preload images, not manipulating them. This option will disable the native fallback provided by the html.

@equinusocio thanks for your reply.
I think your HTML example is very short since you have only 2 images: one webp and one jpg.
Consider the following html:

This provides 5 images URLs in the srcset that the browser could choose to download depending on the match between the sizes attribute and the width resolved by the browser matching viewport width and the screen density.
To obtain the same result using the picture tag, and supporting the webp format, you would need to do write tons of lines of source tags.

I figured out the picture tag would need to be something like this:
<picture>
	<source type="image/jpeg" media="" data-srcset="img/image_440.jpg 1x, img/image_660.jpg 2x">
	<source type="image/jpeg" media="(min-width: 768px)" data-srcset="img/image_330.jpg 1x, img/image_660.jpg 2x">
	<source type="image/jpeg" media="(min-width: 1024px)" data-srcset="img/image_220.jpg 1x, img/image_440.jpg 2x">
	<source type="image/webp" media="" data-srcset="img/image_440.jpg 1x, img/image_660.jpg 2x">
	<source type="image/webp" media="(min-width: 768px)" data-srcset="img/image_330.jpg 1x, img/image_660.jpg 2x">
	<source type="image/webp" media="(min-width: 1024px)" data-srcset="img/image_220.jpg 1x, img/image_440.jpg 2x">

</picture>

And this only for one image serving 3 media queries and 5 image sizes.
The weight of the HTML would become much heavier the more media queries and image sizes you want to serve.
Imagine this multiplied for all the images in a product listing page.

I get your point, i just don't like this api because i think it's a bit out of the lazyloading scope. Btw i think your (complex) example can be reduced to something like this:
<picture>
	<source type="image/webp" srcset="image_440.webp 1x, image_660.webp 2x" sizes="(min-width: 768px) 330px, (min-width: 1024px) 220px, 440px" >
  <source type="image/jpeg" srcset="image_440.jpg 1x, image_660.jpg 2x" sizes="(min-width: 768px) 330px, (min-width: 1024px) 220px, 440px" >

</picture>

Where (extension free):

Use image_440 on 1x or use image_660 on 2x
If (min-width: 768px) show the image at 330px (intrinsic size)
if (min-width: 1024px) show the image 220px (intrinsic size)
By default show the image at 440px (intrinsic size)

Combining these rules the browser will load the bigger file based on the screen density, but it will change the intrisic size (rendered sizes) of the image based on the media queries.

Btw your (complex) example can be reduced to something like this:

Oh my god, I didn't know that sizes attribute could be used in the source tag too. I'll definitely take a look into that. If it solves, lets be declarative in the HTML and leave lazyload out of this.
The thing you got wrong, though, is that you used the x descriptor in the srcset, in conjunction with the sizes attribute. Check this guide. But I got your point.
Thanks for now.

I agree with you though, this goes beyond the purpose of lazyloading.
I'm gonna try if this works.
<picture>
  <source
    type="image/webp"
    srcset="img/image_220.webp 220w,
      img/image_330.webp 330w,
      img/image_440.webp 440w,
      img/image_550.webp 550w,
      img/image_660.webp 660w"
    sizes="(min-width: 768px) 330px, (min-width: 1024px) 220px, 440px"
  >
  <source
    type="image/jpeg"
    srcset="img/image_220.jpg 220w,
      img/image_330.jpg 330w,
      img/image_440.jpg 440w,
      img/image_550.jpg 550w,
      img/image_660.jpg 660w"
    sizes="(min-width: 768px) 330px, (min-width: 1024px) 220px, 440px"
  />

</picture>

If it does, I might consider deprecating the to_webp option.

Nope :) srcset can contains w or x descriptors:
From the spec

Zero or one of the following:

A width descriptor, consisting of: ASCII whitespace, a valid non-negative integer giving a number greater than zero representing the width descriptor value, and a U+0077 LATIN SMALL LETTER W character.
A pixel density descriptor, consisting of: ASCII whitespace, a valid floating-point number giving a number greater than zero representing the pixel density descriptor value, and a U+0078 LATIN SMALL LETTER X character.

Hey! I've tried the following HTML document in different browser (not on Safari yet), and it works great on every browser. Opera, Chrome and Firefox download the fake webp files, MS Edge the jpeg files, IE the fallback one.

		<title>Picture!</title>
		<meta name="viewport" content="width=device-width, initial-scale=1.0" />

		<picture>
			<source
				type="image/webp"
				srcset="
					https://placehold.it/220?text=220.webp 220w,
					https://placehold.it/330?text=330.webp 330w,
					https://placehold.it/440?text=440.webp 440w,
					https://placehold.it/660?text=660.webp 660w,
					https://placehold.it/880?text=880.webp 880w
				"
				sizes="(min-width: 1024px) 220px, (min-width: 768px) 330px, 440px"
			/>
			<source
				type="image/jpeg"
				srcset="
					https://placehold.it/220?text=220.jpg 220w,
					https://placehold.it/330?text=330.jpg 330w,
					https://placehold.it/440?text=440.jpg 440w,
					https://placehold.it/660?text=660.jpg 660w,
					https://placehold.it/880?text=880.jpg 880w
				"
				sizes="(min-width: 1024px) 220px, (min-width: 768px) 330px, 440px"
			/>

		</picture>

The only strange thing is that my version of MS Edge (which is quite old) downloads the wrong size (660w) of the image, but the others do good.

I'm wondering if one could put everything inside the img tag except the webp version, which could go in the source tag.
 It still works. It's the shortest version of the markup if one needs to have responsive images.
I think I'll drop the to_webp option then. Let me sleep over it. Update tomorrow (GMT+1).

It also already works with lazyload, no need to do anything else.

		<title>Picture!</title>
		<meta name="viewport" content="width=device-width, initial-scale=1.0" />

		<picture>
			<source
				type="image/webp"
				data-srcset="
					https://placehold.it/220?text=220.webp 220w,
					https://placehold.it/330?text=330.webp 330w,
					https://placehold.it/440?text=440.webp 440w,
					https://placehold.it/660?text=660.webp 660w,
					https://placehold.it/880?text=880.webp 880w
				"
				sizes="(min-width: 1024px) 220px, (min-width: 768px) 330px, 440px"
			/>

		</picture>

I'm going to pick the code in my picture gist and place it in a new demo in the demos folder.
I already released version 10.20.1 which fixed a bug with the previous code (data-sizes -> sizes in the source tag), so now it's working properly and to_webp became quite useless indeed.
Cheers.

@verlok Nice! :)

Should we clean everything related to webp  (webp detection support, to_webp) to clean up a bit then bump the version number to a major one?

@ipernet Sounds good. Could be. way to clean up and drop the support to version 8.x at once (see #306)

I've just noticed in the MDN documentation for the source tag that the sizes and srcset attributes are experimental and should not be used in production. Even if in my demo, everything works fine in every browser.
I guess that I cannot ask LazyLoad users to use the above discussed markup to serve WebP images only where supported. So maybe for now the to_webp option still makes sense?
@equinusocio

The W3C spec for the source tag doesn't mention anything about those attributes being experimental, though. 🤔

See demos/webp_native.html for HTML, live demo here https://www.andreaverlicchi.eu/lazyload/demos/webp_native.html

The source tag and relative attributes are part of the living standard. They are not experimental anymore.
https://html.spec.whatwg.org/multipage/embedded-content.html#the-source-element
I will update the mdn page.

@verlok Done. The experimental flags are gone.

Great! Thanks. Will proceed as @ipernet suggested then.

Hi everyone,
I'm working on the renewal of LazyLoad in the feature/v11 branch, if you want to take a look.
I took the change to refactor the code and add something here, remove something there.
There are a couple of new features but the code is so optimized then the minified library is even smaller.
11.0.0 Changelog (temporary)

Changed bundle file name of ES Module from lazyload.es2015.js to lazyload.esm.js
Removed the to_webp option (see issue #288)
Ceased support and development of LazyLoad v.8 (see issue #306)
Callbacks renewal :: POSSIBLE BREAKING CHANGE

Callback callback_enter has changed its meaning! It is now called whenever an element enters the viewport, even if load_delay is set
Callback callback_exit (new) is called whenever an element exits the viewport, even if load_delay is set
Callback callback_reveal (new) is called when an element is about to be revealed, and its attribute values were copied from the data- attributes to the actual ones.
Callback callback_set was removed. You can use callback_reveal instead.

Private methods like _setObserver, _onIntersection etc. are now hidden and protected.
Added the auto_unobserve boolean option.

#310 closes this I guess.

please bring it back! what about background images? you missed them entirely with your discussion.

@hyteq As I said, I think is outside the lazy load scope. You can follow multiple ways to load web as background image with CSS. One of them is described here:
https://css-tricks.com/using-webp-images/

@hyteq As I said, I think is outside the lazy load scope. You can follow multiple ways to load web as background image with CSS. One of them is described here:
https://css-tricks.com/using-webp-images/

this does not help at all. your mentioned solution has three disadvantages:

it requires another 3rd party package with a little overhead
its not possible to store the image url inside of the html tag element. i need to use a css file at all for setting all the background images for my entire website
the 2nd point brings much larger css files that are not needed anyway. this produces much more traffic as well. even google's pagespeed tells you to reduce unused css rules, so this will reduce your entire seo rating on google

lets said again: please bring it back!
it doesnt matter if its outside of the scope of lazyloading.
its quite a feature and nothing else.
dont just remove it when somebody is not happy about it.
i agree, there is another better way to give the part of loading webp images to the browser using the picture tag. but dont remove such a feature unless there is no equivalent way to do so with background-images. the mentioned option above is it not.

There is not standard ways to lazyload assets from CSS. The only way you can do that is loading the relative (scoped) CSS only when the HTML is loaded, for example using one of the current component-based frameworks like Vue, React, Svelte, Angular. Additionally you can use postcss to convert background to webp
https://github.com/ai/webp-in-css


How to modify the four vertex

we drawed the rectangular. we want to modify the rectangular by four vertex.

another vertex don't work,

Hi, sorry but I'm not sure what is the issue here. Do you want to move the whole polygon?

I think that you want to resize rectangle (all vertex) when you drag any vertex. If you draw rectangular, its like polygon, modify works same like for polygon. Every vertex move alone.  if you use geometryFunction  for own geometry, you set draw type (https://openlayers.org/en/latest/examples/draw-shapes.html)
Modify for own geometry isnt in openlayers, i think. Openlayers can modify only default type (polygon, point, line, circle). I try to solve simuliar problem, but its not EZ.


route throw error when it deserialize a StreamingResponse/FileResponse (Fastapi)

Hello.
I'm having trouble downloading PDF files from the microservice. I suppose that the maqueRequest method only decodes JSON responses. In my example, the gateway throws an exception since the microservice returns a StreamingResponse.
with async_timeout.timeout(delay=timeout):
        async with aiohttp.ClientSession(headers=headers) as session:
            async with session.request(
                    method=method, url=url, params=query, data=data) as response:
                response_json = await response.json()
                decoded_json = decode_json(data=response_json)
                return decoded_json, response.status, response.headers

My method signature
@pmrush_billing_information_router.get("/api/billing/invoices/download/{invoice_number}", response_model=Any,
                                          tags=["billing_information"])
   @pm_logger
   async def download_invoice(self, invoice_number: str, auth_user: UserInfoResponse = Depends(get_current_user)):

      ......

       invoice_pdf = requests.get(download_link, stream=True)
       output = BytesIO(invoice_pdf.content)
       output.seek(0)
       headers = {
           'Content-Disposition': 'attachment; filename="invoice.pdf"'
       }
       return StreamingResponse(output, headers=headers)

Api gateway signature:
@staticmethod
   @route(
       request_method=api_gateway_billing_information_router.get,
       service_url=SERVICE_URL,
       gateway_path='/api/billing/invoices/download/{invoice_number}',
       service_path='/api/billing/invoices/download/{invoice_number}',
       query_params=["invoice_number"],
       body_params=[],
       status_code=status.HTTP_200_OK,
       tags=['billing_information'],
       response_model=Any
   )
   @pm_logger
   async def download_invoice(invoice_number: str, request: Request, response: Response,
                              token: str = Depends(reuseable_oauth)):
       pass

Hello Any update for this feature ?

Hello Any update for this feature ?

Hello, I'll take a look the other day, I left the city :(

@BrandeeDev Hi, I was somehow able to add support for SteamingResponse. Look in the Pull request #7

Hallo, this feature is CRITICAL in many scenarios... a lot of people need to download - through the API gateway - a binary resource... can you please pay attention to our requests?


Roll engine c96f3536f316..c5e30553cd1a (1 commits)

git@github.com:flutter/engine.git/compare/c96f3536f316...c5e30553cd1a
git log c96f3536f316..c5e30553cd1a --no-merges --oneline
2019-08-14<EMAIL_ADDRESS>Roll src/third_party/skia 735f1d2f0208..c061d31c2415 (17 commits) (flutter/engine#11008)
The AutoRoll server is located here: https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+/master/autoroll/README.md
If the roll is causing failures, please contact the current sheriff (stuartmorgan@google.com), and stop
the roller if necessary.

Auto-roller completed checks. About to merge.


Reduce CI duration

Currently CI duration has exceeded 20mins. Need to breakdown things further

https://github.com/open-traffic-generator/conformance/actions/runs/3959593259


Add commit name check

Added commit name check via GitHub Actions and at contributor's git via githooks

@SSSKrut Could you check this warning: https://github.com/butvinm/nobutt/pull/15/checks#step:3:6


extend/pathname: fix text_executable? regex

Otherwise we might match binary sequences that happen to be 0A 23 21 <regular non-space character>.

Makes sense, thanks again @Bo98!


Lambda layer to slice input not being applied

This is not the same as #1747 - the output_shape of my Lambda layer is fine, but during training the layer seems to be leaving the input unchanged: https://github.com/danielhers/ucca/blob/lstm/parsing/classifiers/nn/lstm.py#L124
This network is supposed to be slicing the inputs to (*, self._output_max_len, *) (first dimension is the batch size, of course, and the last varies per input but it should be OK for concatenation).
When running it, I am getting the following:
ValueError: all the input array dimensions except for the concatenation axis must match exactly
Apply node that caused the error: Join(TensorConstant{2}, Reshape{3}.0, Reshape{3}.0, Subtensor{int64::}.0)
Toposort index: 102
Inputs types: [TensorType(int8, scalar), TensorType(float32, 3D), TensorType(float32, 3D), TensorType(float32, 3D)]
Inputs shapes: [(), (0, 1000, 20), (0, 1000, 100), (0, 1500, 6)]
Inputs strides: [(), (80000, 80, 4), (400000, 400, 4), (36000, 24, 4)]
Inputs values: [array(2, dtype=int8), array([], shape=(0, 1000, 20), dtype=float32), array([], shape=(0, 1000, 100), dtype=float32), array([], shape=(0, 1500, 6), dtype=float32)]
Outputs clients: [[Elemwise{Composite{Switch(i0, (i1 * i2 * i3), i2)}}[(0, 2)](InplaceDimShuffle{x,x,x}.0, TensorConstant{(1, 1, 1) of 2.0}, Join.0, Elemwise{Composite{Cast{float32}(LT(i0, i1))}}[(0, 0)].0)]]

Here 1000 is self._output_max_len, 1500 is self._max_len, and I have 3 inputs to the network (batch size, self._max_len, param.num shown here):
numeric: (10, 1500, 6)
e: (10, 1500, 4)
w: (10, 1500, 1)

So the concatenation at line 130 is throwing an error because not all inputs are the same modulo the last dimension, namely the input corresponding to the numeric input is still (10, 1500, 6), as if the slicer layer did nothing. How can that be?

Post your code, please.

Sorry, thought the link would suffice.
This is the relevant part:
    for suffix, param in self.feature_params.items():
        if config.Config().args.verbose:
            print("%s: %s" % (suffix, param))
        i = Input(shape=(self._max_len, param.num), dtype="float32" if param.numeric else "int32", name=suffix)
        inputs.append(i)
        slicer = Lambda(lambda a: a[self._input_max_len:], output_shape=(self._output_max_len, param.num))
        if param.indexed:  # w, t
            # first half is input representation encoding
            x = Lambda(lambda a: a[:self._input_max_len, 0], output_shape=(self._input_max_len,))(i)
            x = Embedding(output_dim=param.dim, input_dim=param.size, init=self._init,
                          weights=param.init, input_length=self._input_max_len, W_regularizer=self._regularizer)(x)
            rep = x#Bidirectional(LSTM(self._layer_dim, activation=self._activation, init=self._init,
            #                          return_sequences=True))(x) TODO uncomment
            # second half is indices into first half
            idx = slicer(i)
            x = merge([rep, idx], mode=lambda a: a[0][:, a[1]],
                      output_shape=(self._output_max_len, param.num, param.dim))#self._layer_dim))
            x = Reshape((self._output_max_len, param.num * param.dim))(x)#self._layer_dim))(x)
        else:  # not indexed: e, A, p, x or numeric
            x = slicer(i)  # TODO why is the shape wrong here for numeric input?
            if not param.numeric:
                x = Embedding(output_dim=param.dim, input_dim=param.size, init=self._init, weights=param.init,
                              input_length=self._output_max_len, W_regularizer=self._regularizer)(x)
                x = Reshape((self._output_max_len, param.dim))(x)
        encoded.append(x)
    x = merge(encoded, mode="concat", concat_axis=-1) if len(encoded) > 1 else encoded[0]

This is in the same as #1747, there the slicing was correct, op just forgot to set output_shape. Here you are slicing it wrong. It should be:
slicer = Lambda(lambda a: a[:, self._input_max_len:, :], output_shape=(self._output_max_len, param.num))

And,
 x = Lambda(lambda a: a[:, :self._input_max_len, 0], output_shape=(self._input_max_len,))(i)

Thanks for the answer.
I see, so I need to include the batch axis too!
Not sure about the first line, though - I should have explained it better:
The first self._input_max_len time points contain the input for my indexed params, and the last self._output_max_len time points of the same inputs contain indices into the first self._input_max_len.
For my non-indexed params, the first self._input_max_len time points are always zero, and the last self._output_max_len time points contain the actual data. I made it that way because I thought the inputs must all be the same length.
Then each output time point is supposed to look at the input received at the same time point (which for some of the params may be indices into the first self._input_max_len time points).
I changed these two lines to
slicer = Lambda(lambda a: a[:, self._input_max_len:], output_shape=(self._output_max_len, param.num))

and
x = Lambda(lambda a: a[:, :self._input_max_len, 0], output_shape=(self._input_max_len,))(i)

and now I'm getting this:
ValueError: all the input array dimensions except for the concatenation axis must match exactly
Apply node that caused the error: Join(TensorConstant{2}, Subtensor{::, :int64:}.0, Reshape{3}.0, Reshape{3}.0)
Toposort index: 105
Inputs types: [TensorType(int8, scalar), TensorType(float32, 3D), TensorType(float32, 3D), TensorType(float32, 3D)]
Inputs shapes: [(), (10, 1000, 6), (100, 1000, 100), (40, 1000, 20)]
Inputs strides: [(), (36000, 24, 4), (400000, 400, 4), (80000, 80, 4)]
Inputs values: [array(2, dtype=int8), 'not shown', 'not shown', 'not shown']
Outputs clients: [[Elemwise{Composite{Switch(i0, (i1 * i2 * i3), i2)}}[(0, 2)](InplaceDimShuffle{x,x,x}.0, TensorConstant{(1, 1, 1) of 2.0}, Join.0, Elemwise{Composite{Cast{float32}(LT(i0, i1))}}[(0, 0)].0)]]

So the batch size seems to be off. It's admittedly better than a batch size of 0, but I don't understand why it's 10, 100 and 40, respectively. I'm feeding a batch size of 10 for all inputs.

I guess my problem is not with the slicing, but rather that my e input has an extra axis that I want to use, but the Embedding doesn't allow that. Namely, at each output time point I want to feed 4 separate e elements, embed them and then just flatten them so that the last dimension would be 80=4*20. But I can't do that because for some reason the 4 gets pushed into the batch size.
I don't want separate Embedding layers because all 4 elements should be embedded using the same lookup table. And I can't really do the trick I did with the w elements (feeding them in the first self._input_max_len time points and then indexing them) because they are relevant per output time point, not per input time point.
And even if I could artificially feed all possible e values in the first self._input_max_len time points and then just index into them, I don't know how to handle the fact that I have 4 separate indices into them. This is also the problem with the x = merge([rep, idx], mode=lambda a: a[0][:, a[1]], line: it gives an effective output shape of (100, 1000, 100) for some reason. I fixed that by changing it to x = merge([rep, idx], mode=lambda a: a[0][:, a[1][0]], which is fine because I really do have only one w index per time point right now, but if I want to have more than that (say, 4) it seems to be getting pushed into the batch axis again...

tl;dr The original issue with slicing using Lambda has been solved. For further help, you will have to state your problem in a much more simpler manner, along with a simplified a code snippet (not just dump code from your actual program.. for e.g.. I don't know what is your e, nor the elements in the encoded list ) which demonstrates what you are trying to do.

Fair enough. Thanks a lot for your help so far!