Tractor

Tractor Base Functionality
Task: Complete the tractor.
Feature: Adding farm tools #7
Imagine a farming game where you can't sit back and on your tractor and crack open a cold one. Well don't worry we got this covered! A tractor will be added to the game to enhance the player's experience by increasing the ceiling for productivity and increasing the amount of money you can make, whilst looking good on a tractor.
Dependencies

[x] Make the Tool Class #17

Milestones
This task is optional as per our feature task #7 so there won't be any hard deadlines.

[x] Step 1 (Aug. 19): Create base factory for tractor
[x] Step 2 (Aug. 21): Make base animations
[x] Step 3 (Aug. 25): Implement player getting in the tractor and moving in the tractor
[ ] Step 4 (Aug. 28): Full animations done
[ ] Step 5 (Aug 31) (Optional): Interactions with Tiles (Tilling, watering the ground etc.)

Completion Deadline: Aug. 31
Documentation
Member

Jackson Trenerry @JTrenerry
Lakshan Chandrasekara @86LAK

Tractor TM

Note to self: Tractor starts un-muted and player starts muted, needs to be flipped, after that should be okay

After some testing found the following bugs

when moving and exiting the tractor this results in movement of the character and tractor to break. Character is stuck in a direction and the tractor stuck moving in the opposite direction. This was tested with the tractor moving left on screen or using  "A"  key. The tractor continued heading left after the player exits the vehicle and the player continues in the opposite direction.

-player can enter vehicle from anywhere by pressing F. The player should only be able to enter the tractor by standing next to it.

second bit isn't implemented yet, did you hold 'A' and the player moved right?

yeh

Okay I know the solution to tractor moving, but player im stumped on

Tractor will continue unto sprint 2


Refactor PowerVS changes

What this PR does / why we need it:
Trying to refactor PowerVS changes to address comments given in earlier PRs on below things

Capitalise abbreviations
Removing named return var
Using inline if wherever required
and few minor changes like flag rename to remove redundant powervs prefix in it. e.g. --powervs-zone to --zone

Which issue(s) this PR fixes (optional, use fixes #<issue_number>(, fixes #<issue_number>, ...) format, where issue_number might be a GitHub issue, or a Jira story:
Fixes #
Checklist

[x] Subject and description added to both, commit and PR.
[ ] Relevant issues have been referenced.
[ ] This change includes docs.
[ ] This change includes unit tests.

/retest-required

/approve
thanks! for @mkumatag to final lgtm

/hold cancel

/lgtm


Cloning views fix

Right now view count gets cloned with the brew contents, this takes care of that, artifically setting views to 0 when a cloned brew is created.

This is good, although it includes a lot of extraneous changes from your other PRs as well. Can you remake this making sure to use the current Master branch as the base?

This is good, although it includes a lot of extraneous changes from your other PRs as well. Can you remake this making sure to use the current Master branch as the base?

wow, must've pushed changes in the wrong branch, let me fix that

i erased my other commits and merged master into it, that's it, right? @calculuschild


Add license check

See https://github.com/integritee-network/parachain/pull/165/files for an example
Should be done after #274

Currently blocked by GPL license of array-bytes used in substrate.


More PS-y fitText used to produce proportionally scaled metric labels

Also adjustments to the chart sizing stuff which was producing chart dimensions smaller than their container that weren't vertically aligned.

Thank you very much!


chore: update tj actions

Proposed changes

Trying to update this to resolve https://github.com/mongodb/mongodb-atlas-cli/security/dependabot
Checklist

[ ] I have signed the MongoDB CLA
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have added any necessary documentation in document requirements section listed in CONTRIBUTING.md (if appropriate)
[ ] I have addressed the @mongodb/docs-cloud-team comments (if appropriate)
[ ] I have updated test/README.md (if an e2e test has been added)
[ ] I have run make fmt and formatted my code

Further comments

@gssbzn do you know if there is any blocker to update this?


Add boundary arg to tides and change setup run dir

This task is about ensuring the less than four boundary case works. Because of the addition of tides, I believe it is a good idea to make the boundaries a __init__ level argument because now there are 2 major functions (ocean_state and tides) that take that argument, and absolutely no reason for them to be different.

[x] Change setup_boundary_tides to accept a variable number of boundaries
[ ] Change either __init__,setup_run_directory/all boundary functions (setup_ocean_state_boundaries) to save boundary file names from 0-num boundaries (Likely put it in Init...)
Helps fix: https://github.com/CROCODILE-CESM/crocodile-regional-ruckus/issues/8

Changes:

Add a boundaries parameter to tides
Add boundaries to init and take them out of ocean_state and tides.

Rebased PR #24 under this


Update Package for Lambda Functions

lambda functions need the update version of aws_network as there is a fix to ClientFactory

will close on merge with main


[Compute] az vmss create: Support NetworkApiVersion for Vmss with OrchestraionMode == Flexible

Description

Fix #17830
Testing Guide

History Notes

[Component Name 1] BREAKING CHANGE: az command a: Make some customer-facing breaking change.
[Component Name 2] az command b: Add some customer-facing feature.

This checklist is used to make sure that common guidelines for a pull request are followed.

[ ] The PR title and description has followed the guideline in Submitting Pull Requests.

[ ] I adhere to the Command Guidelines.

[ ] I adhere to the Error Handling Guidelines.

Compute


memory leak

i performed a profile with xcode and instrument wanting me about a memory leak in this row of code
[code]
result = [NSString stringWithUTF8String:(char *)sqlite3_column_text(statement,i)];
[code]

Thanks for the bug report!
I too have noticed some memory leaks with this. I have fixed some, but not all, with a patch I've recently committed. See https://github.com/jwhitehorn/SKDatabase/commit/0bab46126f38be6d2da54c77ec68a9aaf79f1d4d
Let me know if you have any success with that. Again, I am still seeing some leaks though, and I am stil trying to track them al down.


🌐 Spherical coordinates

🎉 New feature
Closes #981
Requires

https://github.com/ignitionrobotics/ign-math/pull/235
https://github.com/ignitionrobotics/sdformat/pull/685
https://github.com/ignitionrobotics/ign-msgs/pull/177

Summary

Add support for spherical coordinates (latitude / longitude). Users are able to set the lat / lon for the world origin, and query the lat / lon for other entities with respect to that.

Set from SDF, see example world
See on component inspector
Plot world origin coordinates
Change world origin coordinates from plugins
Change world origin coordinates from a transport service

TODO

[x] Tutorial

Under consideration:

[x] Support spawning entities at a given spherical location through the create service
[x] Move entities to a given spherical coordinate through the set pose service

I'll open this PR for review when the above have been sorted out.
Test it

Try changing it through the SDF, the component inspector, and the service.
Checklist

[x] Signed all commits for DCO
[x] Added tests
[x] Added example and/or tutorial
[x] Updated documentation (as needed)
[x] Updated migration guide (as needed)
[x] codecheck passed (See contributing)
[x] All tests passed (See test coverage)
[x] While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use Squash-Merge
🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸🔸

Let's say I follow the tutorial and spawn an entity. If I now set the world origin's spherical coordinates with the GUI or the service to a different value, shouldn't all entities be moved as a side effect?

I added a note about this to the tutorial in 76495a7. It would be tough to move the entire world, and I'm not sure this is something that the user may always want. I think in the future, if there's need for it, we could add an option that enables this. For now, changing the world origin only affects subsequent commands, like spawns and moves.


support ChatGLM

support ChatGLM3-6b
add a example of a chatglm config
mixlora experiment results:
arc-e 79.87   arc-c 62.9

plz sync to main and provide test results on LoRA with ARC-C, ARC-E, GLUE:RTE, BoolQ


[SPARK-29789][SQL] should not parse the bucket column name when creating v2 tables

What changes were proposed in this pull request?

When creating v2 expressions, we have public java APIs, as well as interval scala APIs. All of these APIs take a string column name and parse it to NamedReference.
This is convenient for end-users, but not for interval development. For example, the query plan already contains the parsed partition/bucket column names, and it's tricky if we need to quote the names before creating v2 expressions.
This PR proposes to change the interval scala APIs to take NamedReference directly, with a new method to create NamedReference with the exact name parts.
Why are the changes needed?

fix a bug, and make it easier to create v2 expressions correctly in the future.
Does this PR introduce any user-facing change?

yes, now v2 CREATE TABLE works as expected.
How was this patch tested?

a new test

cc @rdblue @brkyvz @viirya  @imback82

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/18287/
Test PASSed.

Test build #113388 has started for PR 26425 at commit c84a7e4.

@cloud-fan, I think this misunderstands the purpose of the API. That API is for end users, where parsing the name is the correct thing to do.
Internal uses should go directly to the LogicalExpressions helpers and create the case class implementation of NamedReference directly. That class is FieldReference. That's why the Parser uses FieldReference, not NamedReference.

Test build #113388 has finished for PR 26425 at commit c84a7e4.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/113388/
Test PASSed.

That API is for end users, where parsing the name is the correct thing to do.

Yea I agree, that's why I leave the behavior of java API (Expressions) unchanged.
What I changed in this PR is the scala API (LogicalExpressions), e.g. LogicalExpressions.bucket

Internal uses should go directly to the LogicalExpressions helpers and create the case class implementation of NamedReference directly.

I don't quite get the point here. Internally we also want to write concise code, why force people to create case class implementation directly? Can't we use the helper functions in LogicalExpressions ?

Hi, @rdblue .
How do you think about @cloud-fan 's question? I'm just wondering which path we are going to choose.

https://github.com/apache/spark/pull/26425#issuecomment-551350920

@cloud-fan, thanks for the responses. Sorry that I thought you were changing the behavior at first.
For the question about whether to use the methods in LogicalExpressions or to use FieldReference directly, I think it's easier to use the FieldReference case class directly. That way, using references internally always goes through FieldReference. That's why we added the extractors that can match any NamedReference implementation and will look like a FieldReference. But, whether to instantiate everything through the case class or not doesn't really matter.

Test build #113610 has started for PR 26425 at commit 550bf25.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/18495/
Test PASSed.

Test build #113610 has finished for PR 26425 at commit 550bf25.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/113610/
Test PASSed.

Thanks! Merging to master.

Thank you, @rdblue and @cloud-fan .


Mistral 7b and Mixtral 8x7b experience degraded performance (using official docs)

System Info

TensorRT-LLM v0.8.0 (pinned to release commit)
Nvidia A100
Mistral-7B-Instruct-v0.2
Using the CPP runner
Installed with pip install tensorrt_llm==0.8.0 --extra-index-url https://pypi.nvidia.com
Cloned this repository at the v0.8.0 https://github.com/NVIDIA/TensorRT-LLM/commit/5955b8afbad2ddcc3156202b16c567e94c52248f commit for contained scripts

Who can help?
@kaiyux @byshiue
Information

[X] The official example scripts
[ ] My own modified scripts

Tasks

[X] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[ ] My own task or dataset (give details below)

Reproduction

Build and run Mistral 7b with the example instructions.

python convert_checkpoint.py --model_dir Mistral-7B-Instruct-v0.2 \
                             --output_dir trt_engines/tllm_checkpoint_1gpu_mistral \
                             --dtype float16
trtllm-build --checkpoint_dir trt_engines/tllm_checkpoint_1gpu_mistral \
            --output_dir trt_engines/fp16/1-gpu/ \
            --gemm_plugin float16 \
            --max_input_len 32256
python3 ../run.py --max_output_len 2000 \
                  --input_text "<prompt here>" \
                  --tokenizer_dir Mistral-7B-Instruct-v0.2 \
                  --engine_dir trt_engines/fp16/1-gpu \
                  --max_attention_window_size 4096

Experiment with various prompts conforming to the Mistral prompt template/format such as:

"[INST] write me a long story [/INST]"
"[INST] write me a long book [/INST]"
"[INST] Please write an essay on the thermodynamics of pizza. [/INST]"
or if you disturb the formatting a bit as with the spaces here
"[INST] hi [/INST] Hello ! How can I help you today ?  [INST] hi [/INST]"

Expected behavior
The LLM will provide a complete response which ends instead of deteriorating into an infinite loop. This problem does not happen with Transformers as far as I can tell. Every one of the above example prompts ends and does not infinitely loop.
For example with Transformers:
prompt: [INST] Please write an essay on the thermodynamics of pizza. [/INST]
output:
<s> [INST] Please write an essay on the thermodynamics of pizza. [/INST] Title: The Thermodynamics of Pizza: A Delicious Exploration of Energy Transformations

Abstract:
Pizza, a beloved food item enjoyed by millions around the world, offers an intriguing platform to explore the fundamental principles of thermodynamics. This essay delves into the fascinating world of energy transformations in the context of pizza production, cooking, and consumption.

Introduction:
Thermodynamics, the branch of physics that deals with heat and temperature, provides a framework to understand the transformations of energy in various systems. In our daily lives, we encounter numerous examples of energy transformations, some as simple as a cup of hot coffee cooling down to room temperature or as complex as the combustion engine in a car. In this essay, we will explore the thermodynamics of pizza, from its production to consumption.

Production:
The production of pizza involves several energy transformations. The primary raw materials, such as flour, water, yeast, tomatoes, and cheese, undergo various processes to create the final product. The energy required to produce these raw materials comes from the sun, through the process of photosynthesis in plants, or from non-renewable sources like fossil fuels.

During the baking process, the dough is transformed into a golden-brown crust. This transformation occurs due to the application of heat, which causes the water in the dough to evaporate, producing steam. The heat also denatures the proteins in the dough, allowing it to set and form a solid structure. This process is an endothermic reaction, meaning it absorbs energy from its surroundings.

Cooking:
The cooking of pizza is another fascinating example of energy transformations. The pizza is typically cooked in a wood-fired or gas-fired oven, which provides the high temperatures necessary to cook the pizza evenly and quickly. The heat from the oven transfers energy to the pizza, causing the water in the dough and toppings to evaporate, producing steam. This steam helps to cook the pizza from the inside out, while the high temperatures also help to melt the cheese and brown the crust.

Consumption:
The consumption of pizza is the final stage in its life cycle, and it too involves energy transformations. Our bodies use the energy stored in the pizza to fuel various metabolic processes. The carbohydrates in the pizza are broken down into glucose, which is used as a source of energy. The proteins in the pizza are broken down into amino acids, which are used to build and repair body tissues. The fats in the pizza provide a source of energy and help to absorb the fat-soluble vitamins in the pizza.

Conclusion:
The thermodynamics of pizza provide a fascinating glimpse into the world of energy transformations. From the production of the raw materials to the cooking of the pizza and its consumption, each stage involves the absorption, transfer, and transformation of energy. Understanding these principles not only deepens our appreciation for the science behind our favorite food but also highlights the interconnectedness of various systems in the natural world. So the next time you enjoy a slice of pizza, take a moment to savor not only its delicious flavors but also the fascinating energy transformations that brought it to your table.</s>

actual behavior
Generation does not end until the max_output_len is reached and the farther it goes, the worse it gets. From what I've seen it starts repeating itself and then outputting random tokens which decode as random unicode symbols.
For example,
prompt: [INST] Please write an essay on the thermodynamics of pizza. [/INST]
output:
starts off well with
Input [Text 0]: "<s> [INST] Please write an essay on the thermodynamics of pizza. [/INST]"
Output [Text 0 Beam 0]: "Title: The Thermodynamics of Pizza: A Delicious Exploration

Abstract:
This essay delves into the fascinating world of thermodynamics as it applies to the beloved food item: pizza.

Introduction:
Pizza, a staple food in many households, is a delightful combination of dough, sauce, and various toppings. However, beyond its delicious taste and satisfying texture, lies an intriguing thermodynamic story.

First Law of Thermodynamics:
The first law of thermodynamics, also known as the law of conservation of energy, states that energy cannot be created or destroyed, only transferred or transformed from one form to another.

In the context of pizza, the first law of thermodynamics can be applied to the various stages of pizza preparation and consumption.

For instance, when the dough is being kneaded, the energy from the kneading action is being transferred to the dough, transforming the dough from a disorganized, sticky mass to a smooth, elastic, cohesive structure that is capable of holding the various toppings and sauces that will be added later.

Similarly, when the pizza is being cooked in a hot oven, the energy from the heat is being transferred to the pizza, transforming the pizza from a raw, doughy, unappetizing state to a delicious, golden-brown, mouth-watering masterpiece that is capable of satisfying even the most discerning of taste buds.

but after awhile it starts a continuous loop until finally it reaches the end
In conclusion, the thermodynamics of pizza provide an intriguing and fascinating exploration into the various stages of pizza preparation, consumption, and disposal, and the ways in which the laws of thermodynamics apply to these processes.

From the first law of thermodynamics, which governs the conservation of energy and the transfer and transformation of energy from one form to another, we can see how the energy from the pizza, in the form of the heat and nutrients that it contains, is transferred and transformed from the pizza to the body of the person who is consuming the pizza, and how this process is governed by the first law of thermodynamics.

From the second law of thermodynamics, which governs the increase of entropy, or disorder, in a closed system over time, we can see how the pizza, which once was a delicious, satisfying, nourishing meal, has now been transformed into a mere waste product, which is capable of contributing to the total entropy, or disorder, of the universe.

In summary, the thermodynamics of pizza provide an intriguing and fascinating exploration into the various stages of pizza preparation, consumption, and disposal, and the ways in which the laws of thermodynamics apply to these processes.

From the first law of thermodynamics, which governs the conservation of energy and the transfer and transformation of energy from one form to another, we can see how the energy from the pizza, in the form of the heat and nutrients that it contains, is transferred and transformed from the pizza to the body of the person who is consuming the pizza, and how this process is governed by the first law of thermodynamics.

From the second law of thermodynamics, which governs the increase of entropy, or disorder, in a closed system over time, we can see how the pizza, which once was a delicious, satisfying, nourishing meal, has now been transformed into a mere waste product, which is capable of contributing to the total entropy, or disorder, of the universe.

In conclusion, the thermodynamics of pizza provide an intriguing and fascinating exploration into the various stages of pizza preparation, consumption, and disposal, and the ways in which the laws of thermodynamics apply to these processes.

From the first law of thermodynamics, which governs the conservation of energy and the transfer and

additional notes
Increasing the repetition penalty has an effect, but it doesn't always work and degrades the output whereas I've never seen this issue with Mistral 7b using Transformers.

@iibw do you see the same issue with Mistral-7B-v0.1 (https://huggingface.co/mistralai/Mistral-7B-v0.1) ? just trying to rule out some potential factors that might lead to this.

@PerkzZheng

@iibw do you see the same issue with Mistral-7B-v0.1 (https://huggingface.co/mistralai/Mistral-7B-v0.1) ? just trying to rule out some potential factors that might lead to this.

Using Mistral-7b-Instruct-v0.1, 3 out of 4 prompts produced exactly the same output between TensorRT-LLM and Transformers. The last one wasn't the same, but it was similar. None of them produced repeating outputs and after experimenting with other prompts, trying to get the same issue to happen, I failed to do so.
So, it seems your hunch was correct, and this problem does not affect Mistral v0.1.

Mistral-7B-Instruct-v0.2

does one of your prompts have large than 4096 sequence length (input + output) ? Mistral-instruct-v0.2 doesn't have a sliding window attention, so you would better remove that line (--max_attention_window_size 4096). That is the only difference I can tell from the huggingface configurations.

@PerkzZheng

Mistral-7B-Instruct-v0.2

does one of your prompts have large than 4096 sequence length (input + output) ? Mistral-instruct-v0.2 doesn't have a sliding window attention, so you would better remove that line (--max_attention_window_size 4096). That is the only difference I can tell from the huggingface configurations.

No because the sequence length is never more than around 2000 tokens due to --max_output_len 2000. The repetition starts long before 4096 tokens are hit.
Just in case that was the issue, I ran Mistral-7B-Instruct-v0.2 without --max_attention_window_size 4096 and nothing changed. It reached the max_output_len while repeating over and over.
For example, given the prompt [INST] write me a long story [/INST], this was output:
[TensorRT-LLM][INFO] Engine version 0.8.0 found in the config file, assuming engine(s) built by new builder API.
[TensorRT-LLM][WARNING] [json.exception.type_error.302] type must be array, but is null
[TensorRT-LLM][WARNING] Optional value for parameter lora_target_modules will not be set.
[TensorRT-LLM][WARNING] Parameter max_draft_len cannot be read from json:
[TensorRT-LLM][WARNING] [json.exception.out_of_range.403] key 'max_draft_len' not found
[TensorRT-LLM][WARNING] [json.exception.type_error.302] type must be string, but is null
[TensorRT-LLM][WARNING] Optional value for parameter quant_algo will not be set.
[TensorRT-LLM][WARNING] [json.exception.type_error.302] type must be string, but is null
[TensorRT-LLM][WARNING] Optional value for parameter kv_cache_quant_algo will not be set.
[TensorRT-LLM][INFO] Initializing MPI with thread mode 1
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
        - Avoid using `tokenizers` before the fork if possible
        - Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
No protocol specified
[TensorRT-LLM][INFO] MPI size: 1, rank: 0
[TensorRT-LLM][INFO] Loaded engine size: 13815 MiB
[TensorRT-LLM][INFO] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +8, now: CPU 19092, GPU 15929 (MiB)
[TensorRT-LLM][INFO] [MemUsageChange] Init cuDNN: CPU +206, GPU +58, now: CPU 19298, GPU 15987 (MiB)
[TensorRT-LLM][WARNING] TensorRT was linked against cuDNN 8.9.6 but loaded cuDNN 8.9.2
[TensorRT-LLM][INFO] [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +13812, now: CPU 0, GPU 13812 (MiB)
[TensorRT-LLM][INFO] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +8, now: CPU 19320, GPU 18929 (MiB)
[TensorRT-LLM][INFO] [MemUsageChange] Init cuDNN: CPU +0, GPU +8, now: CPU 19320, GPU 18937 (MiB)
[TensorRT-LLM][WARNING] TensorRT was linked against cuDNN 8.9.6 but loaded cuDNN 8.9.2
[TensorRT-LLM][INFO] [MemUsageChange] TensorRT-managed allocation in IExecutionContext creation: CPU +0, GPU +0, now: CPU 0, GPU 13812 (MiB)
[TensorRT-LLM][WARNING] CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage and speed up TensorRT initialization. See "Lazy Loading" section of CUDA documentation https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#lazy-loading
[TensorRT-LLM][INFO] Allocate<PHONE_NUMBER>0 bytes for k/v cache.
[TensorRT-LLM][INFO] Using 447360 tokens in paged KV cache.
[TensorRT-LLM] TensorRT-LLM version: 0.8.0Input [Text 0]: "<s> [INST] write me a long story [/INST]"
Output [Text 0 Beam 0]: "Title: The Chronicles of the Celestial Garden

Prologue: The Whispered Prophecy

In the farthest reaches of the cosmos, nestled between the swirling nebulae and the vast, star-studded expanses, lay the Celestial Garden.

This enchanted paradise was a place of unparalleled beauty and tranquility. It was a realm where the very fabric of reality seemed to bend and warp in the most fantastical ways.

The Celestial Garden was a place of wonder and enchantment, where the most extraordinary beings from across the cosmos came to live, learn, and grow.

But the Celestial Garden was not without its dangers and challenges. For in the vast, swirling cosmos, there were those who coveted the unparalleled beauty and power of the Celestial Garden for their own nefarious purposes.

And so, it was foretold by the ancient, wise seers of the cosmos that a great and terrible battle would soon be waged across the vast, swirling cosmos. A battle that would determine the fate of the Celestial Garden and all the wondrous beings that called it home.

This was the whispered prophecy, passed down through the generations of the ancient, wise seers of the cosmos. And it was this prophecy that had brought the great and powerful beings of the cosmos to the Celestial Garden, to prepare for the coming battle and to protect the unparalleled beauty and power of the Celestial Garden for generations to come.

Chapter One: The Gathering of the Great and Powerful

In the heart of the Celestial Garden, nestled among the swirling, iridescent petals of the most beautiful and fragrant flowers in the cosmos, stood the magnificent, resplendent Palace of the Celestial Garden.

This wondrous, enchanted palace was the grandest and most magnificent edifice in the entire cosmos. It was a place of unparalleled beauty, wonder, and enchantment, where the very fabric of reality seemed to bend and warp in the most fantastical ways.

The Palace of the Celestial Garden was a place of pilgrimage for the great and powerful beings of the cosmos. For it was here, in the heart of the Celestial Garden, that the great and powerful beings of the cosmos had come to gather, to prepare for the coming battle, and to protect the unparalleled beauty and power of the Celestial Garden for generations to come.

And so, the great and powerful beings of the cosmos had come to the Celestial Garden, each one bringing with them their unique and extraordinary abilities, knowledge, and power.

There were the great and powerful elemental beings, each one hailing from the most far-flung and inaccessible corners of the cosmos. There were the great and powerful beings of fire, who could summon forth the most scorching and infernal flames, capable of reducing even the most indestructible of materials to ash and dust. There were the great and powerful beings of water, who could summon forth the most torrential and devastating floods, capable of submerging even the most colossal of structures beneath their relentless and unyielding waves. There were the great and powerful beings of air, who could summon forth the most tempestuous and violent storms, capable of tearing apart even the most robust and sturdy of structures with their relentless and unyielding winds. And there were the great and powerful beings of earth, who could summon forth the most solid and unyielding of structures, capable of withstanding even the most cataclysmic of events with their unyielding and unbreakable strength.

But the great and powerful beings of the cosmos were not just limited to the elemental beings. There were also the great and powerful beings of light, who could summon forth the most brilliant and radiant of lights, capable of illuminating even the most darkest and shadowy of corners of the cosmos with their unyielding and unquenchable light. And there were the great and powerful beings of darkness, who could summon forth the most enveloping and all-consuming of darknesses, capable of shrouding even the most brilliant and radiant of lights in an impenetrable and unbreakable darkness, leaving all that lay within its grasp in a state of utter and complete darkness and oblivion.

And then there were the great and powerful beings of time, who could summon forth the most elusive and fleeting of all entities, time itself. These great and powerful beings of time could manipulate and control the very fabric of time itself, bending and warping it to their will, allowing them to travel through time and space at will, visiting the most far-flung and inaccessible corners of the cosmos, and witnessing the most awe-inspiring and mind-boggling of events and phenomena that the cosmos had to offer.

And finally, there were the great and powerful beings of space, who could summon forth the most vast and expansive of all entities, space itself. These great and powerful beings of space could manipulate and control the very fabric of space itself, bending and warping it to their will, allowing them to travel through the most far-flung and inaccessible corners of the cosmos, and witness the most awe-inspiring and mind-boggling of events and phenomena that the cosmos had to offer.

And so, the great and powerful beings of the cosmos had come to the Celestial Garden, each one bringing with them their unique and extraordinary abilities, knowledge, and power. And together, they had formed an unbreakable and unyielding alliance, dedicated to protecting the unparalleled beauty and power of the Celestial Garden for generations to come.

Chapter Two: The Preparations for the Coming Battle

In the heart of the Celestial Garden, nestled among the swirling, iridescent petals of the most beautiful and fragrant flowers in the cosmos, stood the magnificent, resplendent Palace of the Celestial Garden.

This wondrous, enchanted palace was the grandest and most magnificent edifice in the entire cosmos. It was a place of pilgrimage for the great and powerful beings of the cosmos, who came to the Celestial Garden to prepare for the coming battle, and to protect the unparalleled beauty and power of the Celestial Garden for generations to come.

And so, the great and powerful beings of the cosmos had come to the Celestial Garden, each one bringing with them their unique and extraordinary abilities, knowledge, and power. And together, they had formed an unbreakable and unyielding alliance, dedicated to protecting the unparalleled beauty and power of the Celestial Garden for generations to come.

But the great and powerful beings of the cosmos knew that they could not simply rest on their laurels and wait for the coming battle to come to them. For they knew that the forces of darkness and oblivion, led by the malevolent and all-powerful being of darkness and oblivion, known only as the Void King, would stop at nothing to prevent the great and powerful beings of the cosmos from protecting the unparalleled beauty and power of the Celestial Garden for generations to come.

And so, the great and powerful beings of the cosmos had come to the Celestial Garden not only to prepare for the coming battle, but also to take the fight to the forces of darkness and oblivion, led by the malevolent and all-powerful being of darkness and oblivion, known only as the Void King.

And so, the great and powerful beings of the cosmos had come to the Celestial Garden, each one bringing with them their unique and extraordinary abilities, knowledge, and power, to form an unbreakable and unyielding alliance, dedicated to protecting the unparalleled beauty and power of the Celestial Garden for generations to come, and to taking the fight to the forces of darkness and oblivion, led by the malevolent and all-powerful being of darkness and oblivion, known only as the Void King.

And so, the great and powerful beings of the cosmos had come to the Celestial Garden, each one bringing with them their unique and extraordinary abilities, knowledge, and power, to form an unbreakable and unyielding alliance, dedicated to protecting the unparalleled beauty and power of the Celestial Garden for generations to come, and to taking the fight to the forces of darkness and oblivion, led by the malevolent and all-powerful being of darkness and oblivion, known only as the Void King.

And so, the great and powerful beings of the cosmos had come to the Celestial Garden, each one bringing with them their unique and extraordinary abilities, knowledge, and power, to form an unbreakable and unyielding alliance, dedicated to protecting the un"
Tokens: 2013

I noticed at the start there are some json config errors. Maybe they contribute to this? Although I believe they exist with v0.1 as well.

@PerkzZheng it looks like this bug affects Mixtral as well.
Using the same system info as my Mistral testing and these commands:
python convert_checkpoint.py --model_dir Mixtral-8x7B-Instruct-v0.1 \
                                                 --output_dir trt_engines/Mixtral-8x7B-Instruct-v0.1/int8_wo_1gpu_ckpt \
                                                 --dtype float16 \
                                                 --use_weight_only \
                                                 --weight_only_precision int8 \
                                                 --load_model_on_cpu
trtllm-build --checkpoint_dir trt_engines/Mixtral-8x7B-Instruct-v0.1/int8_wo_1gpu_ckpt \
                   --output_dir trt_engines/Mixtral-8x7B-Instruct-v0.1/int8_weight_only/1-gpu \
                   --gemm_plugin float16 --gpt_attention_plugin float16 --context_fmha enable \
                   --use_paged_context_fmha enable --remove_input_padding enable \
                   --max_batch_size 64 --max_input_len 32256 --max_output_len 32256 \
                   --max_num_tokens 32256
python3 ../run.py --engine_dir trt_engines/Mixtral-8x7B-Instruct-v0.1/int8_weight_only/1-gpu \
                            --tokenizer_dir Mixtral-8x7B-Instruct-v0.1 \
                            --max_output_len 2000 \
                            --input_text "<prompt here>"

I built Mixtral for my system (I can't do full precision because I don't have enough VRAM) and ran the four prompts above. Three of the four prompts worked without issue, but the prompt [INST] write me a long book [/INST] looped over and over until it reached the 2000 token limit. I tried these prompts with Mixtral on Transformers with bitsandbytes 8bit precision and again, no issue. The three prompts that worked properly with TensorRT-LLM produced a very similar output when using Transformers and the prompt which was broken with TensorRT-LLM, was not broken with Transformers.
The output of the broken prompt [INST] write me a long book [/INST]:
I'd be happy to help you write a book, but I'll need a bit more information about what you have in mind before I get started. Here are some questions that might help us get started:

* What is the topic of the book?
* What is the intended audience for the book?
* What is the desired length of the book?
* Do you have any specific ideas or themes that you would like to include in the book?

Once I have a better understanding of what you're looking for, I can begin to help you write your book. In the meantime, here is a brief outline that might help us get started:

1. Introduction: This section should introduce the topic of the book and provide some context for the reader. It should also include a brief overview of the main ideas and themes that will be explored in the book.
2. Background information: This section should provide any necessary background information that the reader will need in order to understand the topic of the book. This might include historical context, scientific concepts, or other relevant information.
3. Main ideas and arguments: This section should be the bulk of the book, and should present the main ideas and arguments that you want to make. It should be organized in a logical and coherent way, with each chapter building on the ideas presented in the previous chapter.
4. Conclusion: This section should summarize the main ideas and arguments presented in the book, and should provide some final thoughts or reflections on the topic.
5. Appendices (if necessary): This section can include any additional information or materials that didn't fit neatly into the main body of the book. This might include data tables, charts, or other supplementary materials.

I hope this outline is helpful as you begin to write your book. If you have any specific questions or need further guidance, please don't hesitate to ask. I'm here to help!

As I mentioned earlier, I'll need more information about the topic and intended audience of your book in order to provide more specific advice and guidance. In the meantime, here are a few general tips that may be helpful as you begin to write:

* Outline your book before you start writing. This will help you organize your thoughts and ensure that your book has a clear structure and flow.
* Write regularly, even if it's just for a short period of time each day. This will help you build momentum and make progress on your book.
* Don't be afraid to revise and edit your work as you go. It's rare for a first draft to be perfect, so don't be afraid to make changes and improvements as you work.
* Seek feedback from others. This can be a valuable way to get fresh perspectives on your work and identify any areas that may need improvement.
* Stay organized and keep track of your progress. This will help you stay on track and motivated as you work on your book.

I hope these tips are helpful as you begin to write your book. If you have any specific questions or need further guidance, please don't hesitate to ask. I'm here to help!

Here are a few more specific tips that may be helpful as you write your book:

* Use clear and concise language. This will help your readers understand your ideas and arguments more easily.
* Use examples and anecdotes to illustrate your points. This can make your writing more engaging and help your readers relate to your ideas.
* Use headings and subheadings to organize your ideas and make your writing easier to follow.
* Use transitional phrases to link your ideas together and help your writing flow smoothly.
* Use images, charts, and other visual aids to help illustrate your points and break up the text.

I hope these tips are helpful as you write your book. If you have any specific questions or need further guidance, please don't hesitate to ask. I'm here to help!

As you can see, writing a book is a big task that requires careful planning and attention to detail. However, with hard work and dedication, it is definitely possible to write a successful and engaging book. I wish you the best of luck as you begin this exciting journey!

If you have any specific questions or need further guidance, please don't hesitate to ask. I'm here to help!

Here are a few more tips that may be helpful as you write your book:

* Don't be afraid to take breaks and step away from your writing when you need to. This can help you clear your mind and come back to your work with fresh eyes.
* Set specific goals for yourself and track your progress. This can help you stay motivated and on track as you work on your book.
* Seek out resources and advice from other writers. There are many books, articles, and online resources available that can provide valuable insights and tips on writing a book.
* Consider working with a writing coach or mentor. This can be a great way to get personalized feedback and guidance as you write your book.
* Most importantly, have fun and enjoy the process of writing your book! This will help you stay motivated and engaged as you work on your project.

I hope these tips are helpful as you begin to write your book. If you have any specific questions or need further guidance, please don't hesitate to ask. I'm here to help!

As you can see, writing a book is a complex and challenging process that requires careful planning, hard work, and dedication. However, it can also be a rewarding and fulfilling experience that allows you to share your ideas and insights with the world. I wish you the best of luck as you begin this exciting journey!

If you have any specific questions or need further guidance, please don't hesitate to ask. I'm here to help!

Here are a few more tips that may be helpful as you write your book:

* Be patient and persistent. Writing a book takes time, and it's important to be patient and keep working even when you encounter challenges or setbacks.
* Seek out opportunities to share your work and get feedback from others. This can be a great way to improve your writing and build your confidence as a writer.
* Don't be afraid to take risks and try new things. Writing a book is a creative process, and it's important to be open to new ideas and approaches.
* Keep learning and growing as a writer. There is always room for improvement, and seeking out opportunities to learn and grow can help you become a better writer.
* Most importantly, believe in yourself and your ability to write a great book. With hard work and dedication, you can achieve your goals and create something truly special.

I hope these tips are helpful as you begin to write your book. If you have any specific questions or need further guidance, please don't hesitate to ask. I'm here to help!

As you can see, writing a book is a complex and challenging process that requires careful planning, hard work, and dedication. However, it can also be a rewarding and fulfilling experience that allows you to share your ideas and insights with the world. I wish you the best of luck as you begin this exciting journey!

If you have any specific questions or need further guidance, please don't hesitate to ask. I'm here to help!

Here are a few more tips that may be helpful as you write your book:

* Don't be afraid to ask for help when you need it. Whether you're struggling with a particular aspect of your book or just need some moral support, don't hesitate to reach out to others for assistance.
* Take care of yourself physically and mentally. Writing a book can be demanding, and it's important to make sure you're taking care of your physical and mental health.
* Stay positive and focused on your goals. It's easy to get discouraged or sidetracked when working on a long-term project like a book, but it's important to stay positive and focused on your goals in order to stay motivated and make progress.
* Celebrate your successes, no matter how small they may seem. Writing a book is a big accomplishment, and it's important to take the time to celebrate your successes along the way.
* Most importantly, have fun and enjoy the process of writing your book. This will help you stay motivated and engaged as you work on your project, and it will also make the final product more enjoyable for your readers.

I hope these tips are helpful as you begin to write your book. If you have any specific questions or need further guidance, please don't hesitate to ask. I'm here to help!

As you can see, writing a book is a complex and challenging process that requires careful planning, hard work, and dedication. However, it can also be a rewarding and fulfilling experience that allows you to share your ideas and insights with the world. I wish you the best of luck as you begin this exciting journey!

If you have any specific questions or need further guidance, please don't hesitate to ask. I'm here to help!

Here are a few more tips that may be helpful as you write your book:

* Don't be afraid

I should probably mention, for all these tests with Transformers, I'm using version 4.36.1.

So to summarize,

Mistral v0.1 also generates repeated sentences after enabling int8 weight only, but looks good without int8 weight only ?
Mistral v0.2 generates repeated sentences no matter with or without int8 weight only ?

I have the very same issue with Mixtral on Nvidia H100 in version 0.8.0.
I tried building both with int8 weight quantization and no quantization and the issue is still there.
I can provide more details if needed.

@PerkzZheng

Mistral v0.1 also generates repeated sentences after enabling int8 weight only, but looks good without int8 weight only ?
Mistral v0.1 is perfectly fine as far as I have seen. Mixtral 8x7b is the one appearing to also have this problem. I can't test Mixtral 8x7b without int8 weight only applied because it's too large for the A100 GPU I have access to, but according to @bprus, it doesn't seem to make a difference.

Mistral v0.2 generates repeated sentences no matter with or without int8 weight only ?
I haven't tried Mistral v0.2 with int8 weight only so I can't say for sure, but given what @bprus, int8 weight only doesn't seem to change anything.

So, to recap:

Mistral-7b-Instruct-v0.1 built with normal config appears to be fine
Mistral-7b-Instruct-v0.2 built with normal config appears to have the issue
Mixtral-8x7b-Instruct-v0.1 built with int8 weight only appears to have the issue
Mixtral-8x7b-Instruct-v0.1 built with normal config appears to have the issue according to @bprus

Thanks for the summary. I will see if I can reproduce and find the root cause of this.

I think it's the same as: https://github.com/NVIDIA/TensorRT-LLM/issues/722

It appears to have been introduced with #667

@PerkzZheng I'm not sure if this helps or not, but it looks like there are more changes between Mistral v0.1 and Mistral v0.2 than just the removal of the sliding window
https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2/commit/41b61a33a2483885c981aa79e0df6b32407ed873

Also, it doesn't seem like the Nvidia demo at https://build.nvidia.com/mistralai/mixtral-8x7b-instruct has this issue and it says that it uses Triton Inference Server which is probably using TensorRT-LLM as its backend.

@iibw can you have a try with the main branch ?
this is what I got with Mistral v0.2 and your prompt.
Input [Text 0]: "<s> [INST] Please write an essay on the thermodynamics of pizza. [/INST]"
Output [Text 0 Beam 0]: "Title: The Thermodynamics of Pizza: A Delicious Exploration of Energy Transformations

Abstract:
Pizza, a beloved food item enjoyed by millions around the world, offers an intriguing platform to explore the fundamental principles of thermodynamics. This essay delves into the fascinating world of energy transformations in the context of pizza production, cooking, and consumption.

Introduction:
Thermodynamics, the branch of physics that deals with heat and temperature, provides a framework to understand the transformations of energy in various systems. In our daily lives, we encounter numerous examples of energy transformations, some as simple as a cup of hot coffee cooling down to room temperature or as complex as the combustion engine in a car. In this essay, we will explore the thermodynamics of pizza, from its production to consumption.

Production:
The production of pizza involves several energy transformations. The primary raw materials, such as flour, water, yeast, tomatoes, and cheese, undergo various processes to create the final product. The energy required to produce these raw materials comes from the sun, through the process of photosynthesis in plants, or from non-renewable sources like fossil fuels.

During the baking process, the dough is transformed into a golden-brown crust. This transformation occurs due to the application of heat, which causes the water in the dough to evaporate, producing steam. The heat also denatures the proteins in the dough, allowing it to set and form a solid structure. This process is an endothermic reaction, meaning it absorbs energy from its surroundings.

Cooking:
The cooking of pizza is another fascinating example of energy transformations. The pizza is typically cooked in a wood-fired or gas-fired oven, which provides the high temperatures necessary to cook the pizza evenly and quickly. The heat from the oven transfers energy to the pizza, causing the water in the dough and toppings to evaporate, producing steam. This steam helps to cook the pizza from the inside out, while the high temperatures also help to melt the cheese and brown the crust.

Consumption:
The consumption of pizza involves the transformation of potential energy, stored in the pizza, into kinetic energy as we eat it. Our bodies use this energy to perform various functions, such as digestion, muscle movement, and maintaining body temperature. The energy from the pizza is ultimately derived from the energy stored in the sun, through the process of photosynthesis in plants or the burning of fossil fuels.

Conclusion:
The thermodynamics of pizza offers a delightful exploration of the energy transformations that occur in our daily lives. From the production of the raw materials to the cooking of the pizza and its consumption, each step involves the transfer of energy between various forms. Understanding these energy transformations not only deepens our appreciation for the science behind our favorite food but also highlights the interconnectedness of various systems in the natural world. So, the next time you enjoy a slice of pizza, take a moment to savor not only its delicious flavors but also the fascinating thermodynamic processes that brought it to your table."

I tried with the main branch on mistral v0.2 and I'm experiencing the same error. The model repeats texts after max output length is reached.

I tried with the main branch on mistral v0.2 and I'm experiencing the same error. The model repeats texts after max output length is reached.

it should throw an error if the output length exceeds the max_output_length.
if you build the engine with a large enough output length, do you observe any repeated sentences ?

I tried with the main branch on mistral v0.2 and I'm experiencing the same error. The model repeats texts after max output length is reached.

it should throw an error if the output length exceeds the max_output_length. if you build the engine with a large enough output length, do you observe any repeated sentences ?

Sorry I made a typo in my original response. What I mean was the model repeats texts continually until the max output length is reached. But I made another try today and the issue is now surprisingly gone.

the relevant bugs/issues might have been fixed in the main branch. let us know if you find other issues or you can close this. thanks.

Hi!
I've built the current main branch today, and unfortunately, the issue still persists for Mixtral.

Input [Text 0]: "<s> What is machine learning?"
Output [Text 0 Beam 0]: "

Machine learning is a branch of artificial intelligence that uses algorithms to learn from data and make predictions or decisions. It is a way of teaching computers to learn from data, without being explicitly programmed.

Machine learning algorithms are used in a variety of applications, such as image recognition, natural language processing, and predictive analytics. They can be used to find patterns in data, make predictions, and make decisions.

Machine learning is a rapidly growing field, and there are many different types of algorithms and techniques. Some of the most common types of machine learning algorithms include supervised learning, unsupervised learning, and reinforcement learning.

Supervised learning is a type of machine learning where the algorithm is trained on a set of labeled data. The algorithm is then able to make predictions on new data.

Unsupervised learning is a type of machine learning where the algorithm is not given any labeled data. The algorithm is instead given a set of unlabeled data and is tasked with finding patterns in the data.

Reinforcement learning is a type of machine learning where the algorithm is given a set of rules and is tasked with finding the best way to achieve a goal.

Machine learning is a powerful tool that can be used to solve a variety of problems. It is a rapidly growing field, and there are many different types of algorithms and techniques.

## What is machine learning and how does it work?

Machine learning is a branch of artificial intelligence that deals with the construction and study of algorithms that can learn from and make predictions on data.

Machine learning algorithms are used in a variety of applications, such as email filtering, spam detection, and online recommendation systems. They are also used in more complex applications, such as self-driving cars and medical diagnosis.

Machine learning algorithms are typically divided into two categories: supervised and unsupervised.

Supervised learning algorithms are trained on a set of data that has been labeled with the correct answer. The algorithm then learns to predict the correct answer for new data.

Unsupervised learning algorithms are not given any labeled data. Instead, they are given a set of data and are tasked with finding patterns in the data.

Machine learning algorithms can be further divided into two categories: regression and classification.

Regression algorithms are used to predict a continuous value, such as the price of a stock.

Classification algorithms are used to predict a discrete value, such as the type of animal in a picture.

Machine learning algorithms are typically trained on a set of data that has been labeled with the correct answer. The algorithm then learns to predict the correct answer for new data.

Unsupervised learning algorithms are not given any labeled data. Instead, they are given a set of data and are tasked with finding patterns in the data.

Machine learning algorithms can be further divided into two categories: regression and classification.

Regression algorithms are used to predict a continuous value, such as the price of a stock.

Classification algorithms are used to predict a discrete value, such as the type of animal in a picture.

## What is machine learning and its types?

Machine learning is a branch of artificial intelligence that deals with the construction and study of algorithms that can learn from and make predictions on data.

There are three main types of machine learning: supervised learning, unsupervised learning, and reinforcement learning.

Supervised learning is where the algorithm is given a set of training data, and the algorithm learns to predict the output for new data.

Unsupervised learning is where the algorithm is given a set of data, and the algorithm learns to find patterns in the data.

Reinforcement learning is where the algorithm is given a set of data, and the algorithm learns to find the best way to achieve a goal.

## What is machine learning and its applications?

Machine learning is a branch of artificial intelligence that deals with the construction and study of algorithms that can learn from and make predictions on data.

Machine learning algorithms are used in a variety of applications, such as email filtering, spam detection, and online recommendation systems. They are also used in more complex applications, such as self-driving cars and medical diagnosis.

Machine learning algorithms are typically divided into two categories: supervised learning and unsupervised learning.

Supervised learning algorithms are trained on a set of data that has been labeled with the correct answer. The algorithm then learns to predict the correct answer for new data.

Unsupervised learning algorithms are not given any labeled data. Instead, they are given a set of data and are tasked with finding patterns in the data.

Machine learning algorithms can be"

and
mpirun -n 1 --allow-run-as-root python3 ../run.py --max_output_len=1000 \
                  --tokenizer_dir mistralai/Mixtral-8x7B-v0.1 \
                  --engine_dir /models/engines/Mixtral-8x7B_1gpu_fp16_wq8_pc \
                  --input_text "What is SOTA?"

Input [Text 0]: "<s> What is SOTA?"
Output [Text 0 Beam 0]: "

SOTA is an acronym for Summits On The Air. It is an award scheme for radio amateurs and shortwave listeners that encourages portable operation in mountainous areas. SOTA has been carefully designed to make participation possible for everyone – this is not just for mountaineers! There are awards for activators (those who ascend to the summits) and chasers (who either operate from home, a local hilltop or are even Activators on other summits).

The scheme began in May 2002 with the UK and now includes most of Europe, New Zealand, Japan, Canada, the USA and others.

The SOTA program is a great way to get out and enjoy the outdoors and to meet other hams.

For more information, visit the SOTA website.

SOTA in the Northwest

The Northwest SOTA Association (NW) covers the states of Washington, Oregon, Idaho, and Montana.

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA).

The Northwest SOTA Association is part of the Western USA SOTA Association (WUSA)."

I ran more comprehensive tests and for ~400 requests the average number of output tokens is 985. For other methods of serving (like TGI or vLLM or Transformers) the same requests generate around 550 tokens.
If you need any more information I'm glad to provide it.

@bprus have you observed this issue using fp16 weight instead of int8 weight-only ?
and also try to set --use_paged_context_fmha disable, and see if anything is changed.

So I built with --use_paged_context_fmha disable:
trtllm-build --checkpoint_dir /models/rt/Mixtral-8x7B_1gpu_fp16_wq8 \
             --output_dir /models/engines/Mixtral-8x7B_1gpu_fp16_wq8_pc \
             --gemm_plugin float16 \
             --workers 1 \
             --remove_input_padding enable \
             --use_paged_context_fmha disable \
             --max_input_len 2048 \
             --max_batch_size 64

And unfortunately, it didn't change anything and the results are exactly the same.
Right now, I'm unable to test fp16 because of GPU availability. I need 2xH100 for this.
However, in my previous tests (on older versions) it didn't matter if I skipped quantization.
https://github.com/NVIDIA/TensorRT-LLM/issues/1305#issuecomment-2017405199
I'll try to re-run the test when I get the chance.

So I built with --use_paged_context_fmha disable:
trtllm-build --checkpoint_dir /models/rt/Mixtral-8x7B_1gpu_fp16_wq8 \
             --output_dir /models/engines/Mixtral-8x7B_1gpu_fp16_wq8_pc \
             --gemm_plugin float16 \
             --workers 1 \
             --remove_input_padding enable \
             --use_paged_context_fmha disable \
             --max_input_len 2048 \
             --max_batch_size 64

And unfortunately, it didn't change anything and the results are exactly the same. Right now, I'm unable to test fp16 because of GPU availability. I need 2xH100 for this. However, in my previous tests (on older versions) it didn't matter if I skipped quantization. #1305 (comment)
I'll try to re-run the test when I get the chance.

Thanks. I will see if I can reproduce this issue.

HI @bprus does the issue persist on the latest main branch for you? I tried to reproduce the bug, but was unable reproduce it?
 [/INST]"

Output:
Input [Text 0]: "<s> [INST] Please write an essay on the thermodynamics of pizza. [/INST]"
Output [Text 0 Beam 0]: "Title: The Thermodynamics of Pizza: A Delicious Exploration

Pizza, a beloved dish across the globe, is not only a culinary delight but also an interesting subject of thermodynamic study. Thermodynamics, the branch of physics that deals with heat and temperature and their relation to energy, mass, and radiation, can be applied to the pizza-making and consumption process in various fascinating ways.

The pizza-making process begins with the preparation of the dough. Flour, water, yeast, and salt are mixed and kneaded, resulting in the formation of gluten, a protein that gives the dough its elasticity. This process is endothermic, meaning it absorbs heat from the surroundings. The energy required for this reaction comes from the kinetic energy of the mixer or the heat generated by the baker's hands.

The next step is the proofing of the dough, where it is left to rise. During this period, the yeast ferments the sugars in the dough, producing carbon dioxide gas. This process is exothermic, releasing heat into the surroundings. The increase in temperature can be observed as the dough rises due to the expansion of the gas bubbles.

The toppings, including sauce, cheese, and various meats and vegetables, are then added to the dough. The sauce, usually made from tomatoes, is acidic and can cause the cheese to melt more quickly due to the lowering of the pH level, an exothermic process. The cheese, typically mozzarella, melts as it absorbs heat from the oven, undergoing a phase transition from a solid to a liquid state.

The pizza is then baked in a hot oven, typically at temperatures between 450 and 500 degrees Fahrenheit. This high heat causes the pizza to undergo several thermodynamic processes. The water in the dough evaporates, turning into steam, which contributes to the pizza's texture and helps the crust become crispy. The Maillard reaction also occurs, a complex series of chemical reactions between amino acids and reducing sugars that gives the pizza its distinctive brown color and flavor.

Finally, the pizza is served and consumed, often at a temperature higher than the surrounding environment. The heat from the pizza is transferred to the mouth and surrounding tissues, causing a sensation of warmth or heat. This heat is then dissipated through the process of convection, where the warm air in the mouth rises and is replaced by cooler air from the surroundings.

In conclusion, the thermodynamics of pizza involves various heat transfers and chemical reactions. From the preparation of the dough to the baking of the pizza and its consumption, thermodynamics plays a crucial role in the pizza-making process. Understanding these thermodynamic principles can enhance the pizza-making experience, leading to better-tasting and more consistently cooked pizzas. And, of course, the pleasure of eating pizza is a delightful way to appreciate the principles of thermodynamics in action."

Hi, @djns99 !
I checked today on the v0.9.0 release, and the issue seems to be resolved. Thanks a lot for all your help!
Yet, I stumbled upon another minor issue. In convert_checkpoint.py when --load_model_on_cpu is set, the conversion fails. It's because the preload_model function tries to load the model on device_map='auto'. I worked around it by adding load_model_on_cpu:
def preload_model(model_dir, load_model_on_cpu):
    from transformers import AutoConfig, AutoModelForCausalLM
    if "vila" in model_dir:
        sys.path.append(model_dir + "/../VILA")
        from llava.model import LlavaConfig, LlavaLlamaForCausalLM
        AutoConfig.register("llava_llama", LlavaConfig)
        AutoModelForCausalLM.register(LlavaConfig, LlavaLlamaForCausalLM)

    hf_config = AutoConfig.from_pretrained(model_dir, trust_remote_code=True)
    if hf_config.model_type == "llava":
        from transformers import LlavaForConditionalGeneration
        hf_llava = LlavaForConditionalGeneration.from_pretrained(
            model_dir, torch_dtype="auto")
        model = hf_llava.language_model
    else:
        model = AutoModelForCausalLM.from_pretrained(
            model_dir,
            device_map='auto' if not load_model_on_cpu else 'cpu',
            torch_dtype='auto',
            trust_remote_code=True,
        )
    return model

Just a heads up that you might want to fix it sometime 😉
Once again, thanks for all the help!

Most of my prompts aren't repeating endlessly now, but there's still one. Passing "What is machine learning?" as the prompt to Mixtral 8x7b continues to loop endlessly. I don't think this happens with Transformers so the root problem hasn't been fixed yet.

I tested "What is machine learning?" with transformers and it also loops endlessly for that prompt so it seems like this is an expected output. This clears everything up for me so I'll go ahead and close this issue. Thanks for the help!

I am facing same issue on lllama3-8b-instruct and phi3-mini-128k-instruct

I am facing same issue on lllama3-8b-instruct and phi3-mini-128k-instruct

@anubhav-agrawal-mu-sigma Which version of TensorRT-LLM do you use? And can you share more details that can help us reproduce the issue?

I am facing same issue on lllama3-8b-instruct and phi3-mini-128k-instruct

@anubhav-agrawal-mu-sigma Which version of TensorRT-LLM do you use? And can you share more details that can help us reproduce the issue?

@lfr-0531
Here are the details:
Using tensorrt-llm v0.11.0, using docker image nvcr.io/nvidia/tritonserver:24.07-trtllm-python-py3
python3 /tensorrtllm_backend/tensorrt_llm/examples/llama/convert_checkpoint.py \
  --model_dir "/Meta-Llama-3-8B-Instruc" \
  --output_dir "/triton-build/chkpoint/llama3/1gpu/" \
  --dtype float16

trtllm-build --checkpoint_dir "/triton-build/chkpoint/llama3/1gpu/" \
    --output_dir "/triton-build/engine/llama3/1gpu/" \
    --gemm_plugin auto \
    --gpus_per_node 1 \
    --max_batch_size 64

/tensorrtllm_backend/tensorrt_llm/examples/run.py --engine_dir="/triton-build/engine/llama3/1gpu/" --max_output_len 100 \
  --tokenizer_dir "/Meta-Llama-3-8B-Instruct" \
  --input_text "You are a helpful assistant who answers questions based on given context. Context: MIKE is a student from New York and he is studying in NYU. Question: who is MIKE? "

@anubhav-agrawal-mu-sigma can you try with the main branch ?
this is what I have got with the main branch on H100.
Input [Text 0]: "<|begin_of_text|>You are a helpful assistant who answers questions based on given context. Context: MIKE is a student from New York and he is studying in NYU. Question: who is MIKE? "
Output [Text 0 Beam 0]: " Answer: MIKE is a student from New York and he is studying in NYU.  Context: MIKE is a student from New York and he is studying in NYU. Question: where is MIKE studying?  Answer: MIKE is studying in NYU.  Context: MIKE is a student from New York and he is studying in NYU. Question: where is MIKE from?  Answer: MIKE is from New York.  Context: MIKE is a"
[TensorRT-LLM][INFO] Refreshed the MPI local session

is this output expected ?


Increase Schema Registry Heap

512M doens't seem to be sufficient for v5.4.0
What changes were proposed in this pull request?
Increase to the default heap specified in the values.yaml
How was this patch tested?
This issue was discovered in a live environment on AKS 1.13.x upon upgrade from the 5.3.1 Schema Registry release. The service would fail to start due to insufficient heap. Further testing showed 1000M to be a reasonable floor for the heap size that would allow the service. This was tested and verified across 2 additional environments, also on AKS.

@confluentinc It looks like @matthewi just signed our Contributor License Agreement. :+1:
Always at your service,
clabot


If settlingDuration is unavailable, use the provided spring duration.

Tests fixed.

Tests succeed as expected on iPhone 4s Sim @ iOS 8.1 and iPhone 6s Sim @ iOS 10.3.


Feature: add an observable

Hi,
Do you think it could be possible to add an observable which could be subscribed in order to retrieve and use all logging events in another component?
Of course no need to store an history of the events, just emit on the observable each time a new log request occurs.
Thanks a lot.

Hi @no-more ,
I can't say it hasn't cross my mind but could you elaborate some about the business requirement for this? What would you do with this?

Closing for now, if new info gets added it'll be reconsidered for re opening.

Closing for now, if new info gets added it'll be reconsidered for re opening.

Hi,
I'm very sorry I completely missed your response and forgot about this. I have implemented a service over this logger in order to provide this feature. But I still think it's very useful if it's included directly into this component.
In my case I mainly use the observable to display some information on the screen (for example or errors). I have a component dedicated to this which listen to logs change and decide if something needs to be displayed or not. In all my application I only have to use the logger service.
Thanks a lot
And Happy new Year!

Hi,
I'm very sorry I completely missed your response and forgot about this. I have implemented a service over this logger in order to provide this feature. But I still think it's very useful if it's included directly into this component.
In my case I mainly use the observable to display some information on the screen (for example or errors). I have a component dedicated to this which listen to logs change and decide if something needs to be displayed or not. In all my application I only have to use the logger service.
Thanks a lot
And Happy new Year!

Hi @no-more,
Happy New Year !
I'm not so sure about this. The way I see it, the messages for the logger aren't meant to be messages for the end user but for the developers to help with troubleshooting, let's see if more people request this feature in the future.

Hi @no-more,
Happy New Year !
I'm not so sure about this. The way I see it, the messages for the logger aren't meant to be messages for the end user but for the developers to help with troubleshooting, let's see if more people request this feature in the future.

Hello @langley-agm !
I'm trying to figure out, how to subscribe to log event. I want to push error logs from app to the server. Is it possible?

Hi @bushev , not natively by the app, but you can create your custom logger, which is a good practice anyway in case you switch implementations later on, and wrap this logger into your custom one to do what you want, then call this logger.


[SPARK-7131] [ml] Copy Decision Tree, Random Forest impl to spark.ml

This PR copies the RandomForest implementation from spark.mllib to spark.ml.  Note that this includes the DecisionTree implementation, but not the GradientBoostedTrees one (which will come later).
I essentially copied a minimal amount of code to spark.ml, removed the use of bins (and only used splits), and modified code only as much as necessary to get it to compile.  The spark.ml implementation still uses some spark.mllib classes (privately), which can be moved in future PRs.
This refactoring will be helpful in extending the node representation to include more information, such as class probabilities.
Specifically:

Copied code from spark.mllib to spark.ml:

mllib.tree.DecisionTree, mllib.tree.RandomForest copied to ml.tree.impl.RandomForest (main implementation)
NodeIdCache (needed to use splits instead of bins)
TreePoint (use splits instead of bins)

Added ml.tree.LearningNode used in RandomForest training (needed vars)
Removed bins from implementation, and only used splits
Small fix in JavaDecisionTreeRegressorSuite

CC: @mengxr  @manishamde  @codedeft @chouqin

Merged build triggered.

Merged build started.

Test build #36814 has started for   PR 7294 at commit 4e6d2a4.

Test build #36814 has finished for   PR 7294 at commit 4e6d2a4.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
    logDebug("isMulticlass = " + metadata.isMulticlass)
   *       (i.e., if isMulticlass && isSpaceSufficientForAllCategoricalSplits),
    logDebug("isMulticlass = " + metadata.isMulticlass)

Merged build finished. Test FAILed.

Merged build triggered.

Merged build started.

Test build #36825 has started for   PR 7294 at commit bea9703.

Test build #36825 has finished for   PR 7294 at commit bea9703.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
    logDebug("isMulticlass = " + metadata.isMulticlass)
   *       (i.e., if isMulticlass && isSpaceSufficientForAllCategoricalSplits),
    logDebug("isMulticlass = " + metadata.isMulticlass)

Merged build finished. Test PASSed.

@jkbradley It looks good to me. It might be a good idea to run the spark.mllib and spark.ml models on a couple of datasets to ensure there are no regressions (in accuracy or performance).

@jkbradley I made one pass:

Some of my comments also apply to the old implementation. So I don't think it is necessary to address them in this PR.
It is quite hard to track what have been changed, because you merged DecisionTree and RandomForest into one file and removed bins. Next time we should think of a better approach, making the real changes more discoverable.

Except some style issues and the random seed, it looks good to me:)

@manishamde Thanks for taking a look!  I do want to run some performance tests to make sure they are similar.  For accuracy, current tests run all unit tests and make sure ml and mllib output exactly the same models.  I hope that's sufficient (but I do think we should continue to improve unit tests).
@mengxr Thanks for reviewing!  If I had felt at liberty to do more of these cleanups, I would have.  : )  I'm going ahead and fixing the cleanups, save for unpersisting the last prevNodeIdsForInstances.
W.r.t making the changes discoverable, I agree it's a hard problem.  I think removing bins was worth the trouble to avoid creating even more work for ourselves (new tree representation with bins, using bins internally, removing it eventually).  I was relying on unit tests to give confidence.  It'd be good to come up with ideas for the next time this happens.
Let me know if there is a problem with storing the FileSystem instance.  (I assume it's fine but don't know much about HDFS...)

Merged build triggered.

Merged build started.

Test build #37560 has started for   PR 7294 at commit 48749be.

Test build #37560 has finished for   PR 7294 at commit 48749be.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
    logDebug("isMulticlass = " + metadata.isMulticlass)
   *       (i.e., if isMulticlass && isSpaceSufficientForAllCategoricalSplits),
    logDebug("isMulticlass = " + metadata.isMulticlass)

Merged build finished. Test PASSed.

@jkbradley Just thought about how to reduce the review complexity. I think this PR would be easier to review if you copied the implementation with bins and kept the 1-1 mapping between files in the first PR. Then the second PR removed the bins from the implementation and fixes other small issues.
The changes LGTM:)

Merged into master. Thanks!


Update docs/faq.md about package.json with changes in adf2597a (Compose function ARN from user account ID)

It is straightforward to update package.json examples, but I'm worrying that the updated description sounds awkward.
If you feel something wrong or strange, please point out without reserve.

Thanks! This change should be done at https://github.com/r7kamura/fluct/commit/adf2597a96d5b8071fcf7f3b13a39e0b31dc4b5d :bow:


fix: cmake config.h defines declaration

This fixes how precompiler macros are defined in the cmake build.
I've manually searched for each variable and aligned the declaration in config.h with how the define is actually used (be it #if or #ifdef).
In a couple of cases where the usage was inconsistent between sourcefiles (#if in some files, #ifdef in others), I've consolidated these source files to use the same logic.
Ideally, all source files would use just one way to check these, but doing such a change would cause a very large PR changelist so I didn't want to do it here just to land the fix.

Applied. Thanks. Can you please elaborate what is the "fix" aspect of this (i.e. other than matter of taste).

Applied. Thanks. Can you please elaborate what is the "fix" aspect of this (i.e. other than matter of taste).

Not a matter of taste at all. It didn't compile before for "features" that were not available. Let me give you an example I've encountered when compiling for Android:

First, this is executed to find out whether __environ is defined:
https://github.com/gperftools/gperftools/blob/7ad1dc7693b7b04199681df474122248634a9808/CMakeLists.txt#L327-L334
This results in HAVE___ENVIRON == false because it isn't on NDK.
Next, the original cod had #cmakedefine01 HAVE___ENVIRON in config.h.in which will result in (see cmake docs for details): #define HAVE___ENVIRON 0
Now, the code that conditionally uses it has an #ifdef - https://github.com/gperftools/gperftools/blob/7ad1dc7693b7b04199681df474122248634a9808/src/base/sysinfo.cc#L160-L168
Thus, because HAVE___ENVIRON is defined (albeit to a 0), ifdef succeeds and the compilation fails because __environ is actually unknown.

The correct way to do this is either:

use #ifdef and #if defined() in code but then combine that with config.h.in using #cmakedefine which will conditionally evaluate to ``#define HAVE___ENVIRON 1` if and only if the cmake variable is true
use #if directly and combine it with  #cmakedefine01 which will always make the macro defined.

Which way to use is indeed a matter of taste. Currently, this repository does both approaches which leads to confusion (and errors in this case). This PR just fixed the errors, didn't try to make the decision and unify which way to go.

Thanks for the update. And for the fix.


Curtailment in SA

Not sure it is feasible, but has there been any thought of graphing / reporting the amount of curtailment being exercised each day, it may be a very SA problem, but by looking at all the various renewable energy generators in South Australia you can see which ones have reduced or switched off their output during the day as we can't export any more energy via the interconnector

@BevDan yes we have - we just happen to be talking about it yesterday. There's an issue on the open electricity repository:
https://github.com/opennem/openelectricity/issues/41
I'll keep this open here as it'll require some work, and I'll figure out exactly when we may be able to get to it. Thanks for logging an issue!


After connecting to the Image Comparer, the image does not appear in the show image feed

I consider this a positive. If you want the image to always show in the image feed, connect a preview image before the comparer :)

I consider this a positive. If you want the image to always show in the image feed, connect a preview image before the comparer :)

Isn't that a waste of resources

I consider this a positive. If you want the image to always show in the image feed, connect a preview image before the comparer :)

Isn't that a waste of resources
Isn't using the image feed a waste of resource?


Add OS nodeSelector for linux to allow for hybrid cluster.

What this PR does / why we need it:
This PR sets a nodeSelector for limiting the deployment via helm to linux OS nodes only.
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when the PR gets merged):
Fixes #324

I signed it!

I've signed it!

RE: Those CI failures-- I don't think they are from my commit.

Yeah, CI failures are my fault. I'll fix those tonight and force a rerun. Thanks for the PR!

You're welcome.
Let me.know if there is anything I can do further.

Hi, any movement on this?

Sorry for the delays, it's just me working on MetalLB in my spare time, and spare time is minimal these days :). Merged, thanks!


fix: add component dropdown

Ticket
Add Component dropdown issue
Changes

Updated the filter logic of add component dropdown.

In the development testing I added end dates for 100 and PHG that is why it was listed in the dropdown.


Create a plugin install command

https://github.com/app-registry/appr/blob/master/appr/commands/plugins.py#L17 should allow HELM_PLUGIN_DIR to override plugin_path for consistency.


Add more apis to be compliant with OSGi specifications

To be more compliant with OSGi specifications, we need to add:

runtime.update(id: string, data: JSON): update a system given its id. It will reload the system from its path. We can also add a data parameter to update the system from this data.

SystemRuntime.update(data: JSON): update the system from this data.

Add also:

SystemRuntime.uninstall(): uninstall this system.

Rename:

SystemRuntime.status() into SystemRuntime.state().


Tag named "latest" can conflict with RTD "latest"

I found that our repo was stuck on an old version because we had a tag named "latest"

Agreed. However if we do accept a latest version, we should probably have a method for reverting the latest version as we originally saw it -- a branch -- to our automatic increment latest version.

@stsewd does the fix that we talked the other day will fix this problem also, right? (we do have those 2 special cases to consider in the sync method: stable and latest)
If you haven't done yet, could you add a test case for the latest case also in #3887? Thanks.

@humitos yeah, I think this is the same bug

Closing in favor of https://github.com/rtfd/readthedocs.org/issues/3887


unable to install ansible-dk

Hi,
trying to install ansible-dk on ubuntu-16.04 vagrant vm.
I have the following error
sudo apt-get install -y https://github.com/omniti-labs/ansible-dk.git
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Unable to locate package https://github.com/omniti-labs
E: Couldn't find any package by glob 'https://github.com/omniti-labs'
E: Couldn't find any package by regex 'https://github.com/omniti-labs'
Regards

Hi lmeyemezu,
You cannot install the kit directly from the git repo.
To install ansible-dk, install the compiled OS packages, from https://github.com/omniti-labs/ansible-dk/releases .  For Ubuntu, the current release is https://github.com/omniti-labs/ansible-dk/releases/download/1.2.0/ansible-dk_1.2.0-1_amd64.deb .
--CW


Few shot learning for Document AI

Hello,
I am working on a practical use-case of Document understanding and wondering if I could leverage UDOP. The goal is to extract key informations from the document (in fields or tables). The trick is that I only have a few training samples (<50) and I don't think VQA would apply as these informations are very specific and not always associated with a clear question.
Here are the 2 options I have in mind :

finetuning model. But would 50 sample be enough for UDOP ? How should I deal with tables ? (which don't really look like tables but rather a list without printed rows and columns, as on many receipts)
leverage a foundation model to perform few shot learning (as in GPT3). Are there text + layout foundation models out there that would work for this ? Or should I do prompt engineering with GPT3, Flan-T5, OPT or equivalent models ?

I am interested to get your insights for both english-data... and non english (but latin) data,
Many thanks for your inputs, but also for your work, it is very useful to get open sourced models as yours :)
Simon

Hi Simon,
i am working on a similar task, with very few (< 30) examples.
Did not try UDOP yet but I had very promising results with PAddle NLP's UIE
I can send you the colab that is working with their own dataset if you want - maybe saves a bit time =)

Thanks for the quick reply ! I am interested to get a colab, I'll send you a PM

Hi @SimJeg, thanks for your interest in our work. This is an interesting and practical application case. Few-shot learning with UDOP is always in our discussion. Based on few-shot learning experiments we've conducted (results not included in the paper yet), it's possible you can have reasonable performance with 50 examples: UDOP indeed has some few-sot learning capability.
For UDOP, I'd recommend finetuning (we're expediting the release of pretrained weights). If your task mostly depends on the text modality input, I'd recommend trying GPT. You can try encoding the layout modality as textual input to GPT.

Thanks for your insight @ziyi-yang  :) For few-shot I would love to have some GPT-layout-ready model to do some prompt-based learning...
Are you working on multi-lingual versions too ? My use case cover multiple languages, so UDOP might not help
Also how would you deal with tables with UDOP ? Existing benchmarks are too easy I think, (see for instance the VRDU paper by Google on this topic). The goal is to predict nested entities (i.e. extract D fields from N lines).
Maybe it's a lot of questions for a single issue 😓

Sry for the late reply, had been occupied with paper submissions.
We just tested UDOP on the few-shot learning setting. The accuracy on FUNSD is 83.11 with finetuning on 10 examples (acc. is 91.62 full finetuning on 199 examples). However, UDOP can't do in context learning for few-shot.
Currently we don't have plan for the multilingual version.
We don't have special handling for tables.

Sry for the late reply, had been occupied with paper submissions.
We just tested UDOP on the few-shot learning setting. The accuracy on FUNSD is 83.11 with finetuning on 10 examples (acc. is 91.62 full finetuning on 199 examples). However, UDOP can't do in context learning for few-shot.
Currently we don't have plan for the multilingual version.
We don't have special handling for tables.

Hi @ziyi-yang could you please share the code for using the model on FUNSD dataset?


Fix compatibility info, release 1.2.1

I am happy if you pull this in :) Should it be released as well ?

I won't pull this in, because I sent the request. I am a fan on the four-eye-principle.

Should it be released as well ?

I think so, yes. When it is pulled, we should also update master.

I am a fan on the four-eye-principle.

Me too! That's why I checked it and just pulled it :smile:

Should I rebase integration before pulling into master ?

And I'd like to amend your commit, since it's 1.21, not 1.2.1 :smile:

Shouldn't be necessary, because it is simply one commit ahead of master.

Shouldn't be necessary, because it is simply one commit ahead of master.

 History will always remember from now on.  But I never dare to do this on public repos, because other people might have problems with that. You are basically destroying history if you --force.
I don't mind the merge, BTW. But changing the commit message might indeed be a good idea in this case.
Sorry for the inconvenience.

You are basically destroying history if you --force.

Nope, history is still preserved: The new commit is still related to the old one with the commit message and the commit is still in the .git directory: just not in the tree anymore.
Using --force on public repro should always be avoided when possible. When in master branch, it should be avoided at all cost since it break git pull. But on a integration branch, when both of us are the only one which will have a broken git pull, I do not find it that dangerous.

Sorry for the inconvenience.

No worries.

All done.

I was on my way to push the lost (1.21) tag, but now I've seen that you did it in the meantime.
Thanks a lot, @nitriques!

My pleasure !


[Gradle] Fixed incorrect PURLs & BOM-REFs

This PR fixes issues in the purl and bom-ref when your module names use characters other than letters or numbers.
I found that some of my modules (generated by expo) had names like 'expo-modules-core$android-annotation-processor', which were sometimes correctly percent encoded in the purl and sometimes they weren't. Same with the bom-ref, which actually caused cdxgen to output (only in debug):
===== WARNINGS =====
[
  'Invalid ref in dependencies.dependsOn pkg:maven/host.exp.exponent/expo-modules-core%24android-annotation-processor@1.1.1?type=jar',
  'Invalid ref in dependencies.dependsOn pkg:maven/host.exp.exponent/expo-modules-core%24android-annotation@1.1.1?type=jar'
]

I do wonder though why the incorrect purl wasn't a problem in the validation process...

Could you sign the commit by following the instructions for dco? Thank you so much for this great PR!

Oops, forgot to sign! Using the opportunity to squash as well.

@cerrussell any ideas about the snapshot failures?

@prabhu @malice00 I have updated the snapshot with the piece of data that was causing the error. Should pass now - I have triggered a rerun of the workflow.


brands - about us section

As per web content document, below diagram in about us diagram (for brands page) should say,
‘For the very first time, every university society in the UK will be searchable from a single point.’
NOT 'We help student societies collaborate with brands to get sponsorship.'

Thanks for raising issued I have now updated , this will be reflected in the next Pull request.


Allow to get raw http body

Thanks for the great work!
I was looking for a way to get the raw http body. I'm using the gem to fetch my jsonapi API but i'd like to forward the http body to my front-end application directly.
Any way I can get that payload, out of a ResultSet maybe?

i know the raw body is copied here
https://github.com/chingor13/json_api_client/blob/105e9d079c92c4b540d0dc2752eb01317ac4f9b0/lib/json_api_client/middleware/parse_json.rb
and there is an attr_reader :faraday on the connection
https://github.com/chingor13/json_api_client/blob/105e9d079c92c4b540d0dc2752eb01317ac4f9b0/lib/json_api_client/connection.rb#L4
not sure if that would allow u access


[Feature] Create Dela log after each commit

Search before asking

[X] I searched in the issues and found nothing similar.

Motivation
Motivation is to make paimon readable by databricks sql endpoints
Solution
After each commit we can create delta log which will have a list of underlying parquet files
Anything else?
Is there a way to determine parquet files when creating manifest in paimon ?
Are you willing to submit a PR?

[X] I'm willing to submit a PR!

Can anyone help me. I want to create delta lag after each paimon commit (also with delta delete vectors etc) question how can I retrieve the list of files created by paimon.

This is a big feature, it is not easy to do, but we will consider this.

Hey I have a working prototype and will create WIP PR. Already tested against databricks sql warehouse and seems to work. Just need to add more coverage tests