Obtain Dockstore command line script and jar from dockstore-cli repo

Addresses https://ucsc-cgl.atlassian.net/browse/DOCK-1334

Currently the CLI jar looks on the Dockstore webservice GitHub repo at the releases to find which is the latest or a prerelease; but should actually be looking at the dockstore-cli repo to find out what the dockstore-cli releases are. So now it thinks the latest CLI release is 1.8.5 when the dockstore-cli release is really 1.8.0.
When the CLI jar locates the desired release or prerelease it download the 'dockstore' script from that release. The 'dockstore' script is then responsible for downloading the correct CLI JAR.
So I change the paths in CLI jar to point to dockstore-cli but then it wants to download the 'dockstore' script from the dockstore-cli repo but it isn't there. So we put the 'dockstore' script on the Dockstore CLI release page.
There is a problem for a user that has an old CLI JAR that is still looking on the webservice release page to find latest or prerelease versions.
Here is a potential solution:

Keep the 'dockstore' script on dockstore/dockstore and also on dockstore/dockstore-cli.

Make a dockstore-cli release 1.8.5 and then releases named the same as dockstore/dockstore for a couple of releases until most people who have an old dockstore-cli jar have upgraded. Then eventually we can figure out how to have the UI determine what the latest release is for dockstore-cli and point quickstart there to get the dockstore cli script.

Then we remove the dockstore-cli script from docstore/dockstore and provide some kind of message/documentation telling them to download the script from dockstore-cli repo if the user cannot upgrade.

Make a dockstore-cli release 1.8.5 and then releases named the same as dockstore/dockstore for a couple of releases until most people who have an old dockstore-cli jar have upgraded. Then eventually we can figure out how to have the UI determine what the latest release is for dockstore-cli and point quickstart there to get the dockstore cli script.
Then we remove the dockstore-cli script from docstore/dockstore and provide some kind of message/documentation telling them to download the script from dockstore-cli repo if the user cannot upgrade.

I think this is the tricky bit, we're already considering 1.8.6 so this redirect script would need to be brought forward for releases in the immediate future. Which is reasonable, but need to watch out.

This looks good to me. It's the overall flow that I'm still a little fuzzy on -- the interaction with the script file.

So the script file is in a few places

attached releases of the webservice
installed in people's client machines

When the script file runs without a jar around, it downloads it.
I don't recall if it actually downloads a new script to replace the old one (One second thought, I don't think it does).

@denis-yuen @coverbeck What's the best way to move forward on this?

Could be a short storytime discussion to layout the issue and brainstorm a better solution.

Story time bug explanation and fix proposal doc https://docs.google.com/document/d/1mASD-W8MOPxgauRCNhhz3TD91eLBzeN9q1EQihsU0-w/edit?usp=sharing

I'd like to move forward with the fix outlined at story time. Is this acceptable?


torch.dataloader may freeze if ClearML and Speechbrain are used simultaneously

Hi. I faced a curious issue related to ClearML (docs, github) and Speechbrain.
I use ClearML to track my experiments. In order to track something I have to create a Task object. It tries to configure many things on init.
Also I have to call create_experiment_directory method from Speechbrain in order to set up a folder for my experiment.
I discovered that a train may completely freeze if I call create_experiment_directory method after Task.init method. Due to stacktrace it hangs inside this cycle in pytorch dataloader.
Stacktrace:
Traceback (most recent call last):
File "/home/alexandr.pankratov/pipelines_over_speech_brain/train.py", line 208, in <module>
run_train(hparams, run_opts, tracker_task)
File "/home/alexandr.pankratov/pipelines_over_speech_brain/train.py", line 67, in run_train
asr_brain.fit(
File "/home/alexandr.pankratov/pipelines_over_speech_brain/brains.py", line 167, in fit
self._validate(valid_loaders, epoch, disable_progress_bar=disable_progress_bar,
File "/home/alexandr.pankratov/pipelines_over_speech_brain/brains.py", line 239, in _validate
for batch in tqdm(loader, desc=loader.dataset.dataset_name, dynamic_ncols=True,
File "/opt/conda/lib/python3.9/site-packages/tqdm/std.py", line 1180, in iter
for obj in iterable:
File "/opt/conda/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 521, in next
data = self._next_data()
File "/opt/conda/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1186, in _next_data
idx, data = self._get_data()
File "/opt/conda/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1152, in _get_data
success, data = self._try_get_data()
File "/opt/conda/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 990, in _try_get_data
data = self._data_queue.get(timeout=timeout)
File "/opt/conda/lib/python3.9/multiprocessing/queues.py", line 113, in get
if not self._poll(timeout):
File "/opt/conda/lib/python3.9/multiprocessing/connection.py", line 262, in poll
return self._poll(timeout)
File "/opt/conda/lib/python3.9/multiprocessing/connection.py", line 429, in _poll
r = wait([self], timeout)
File "/opt/conda/lib/python3.9/multiprocessing/connection.py", line 936, in wait
ready = selector.select(timeout)
File "/opt/conda/lib/python3.9/selectors.py", line 416, in select
fd_event_list = self._selector.poll(timeout)
File "/home/alexandr.pankratov/.local/lib/python3.9/site-packages/clearml/task.py", line 3627, in signal_handler
return org_handler if not callable(org_handler) else org_handler(sig, frame)
KeyboardInterrupt

Also there were warnings on dataloader creation, and the number of these warnings is equal to num_workers in dataloader. I'm not sure about the reason of these warnings.
Warning:
Exception ignored in: <function _after_at_fork_child_reinit_locks at 0x7f04644929d0>
Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/logging/__init__.py", line 255, in _after_at_fork_child_reinit_locks
    handler._at_fork_reinit()
  File "/opt/conda/lib/python3.9/logging/__init__.py", line 894, in _at_fork_reinit
    self.lock._at_fork_reinit()
AttributeError: 'NoneType' object has no attribute '_at_fork_reinit'

It requires quite a lot of time to freeze, usually my code freezes in one hour. I couldn't find any dependency on time, data, etc.
But if I call create_experiment_directory method before Task.init everything looks fine and both problems (warnings and freeze) do not appear.
There is an example of code that is affected by the issue. It's not a repro because it depends on my data, code and config but it is quite small so I think it may be useful. I ran it using pytorch.DataParallel but I'm not sure if it's important. There is no model and no train so problem is unrelated to the specifics of train procedure.
import sys
import os
import speechbrain as sb
import torch
from tqdm import tqdm
from torch.utils.data import DataLoader
from data_pipelines import configure_data_pipelines
from sampler import configure_train_batch_sampler
from hyperpyyaml import load_hyperpyyaml
from clearml import Task

if __name__ == "__main__":
    # Reading command line arguments
    hparams_file, run_opts, overrides = sb.parse_arguments(sys.argv[1:])

    # Load hyperparameters file with command-line overrides
    with open(hparams_file) as fin:
        hparams = load_hyperpyyaml(fin, overrides)

    torch.cuda.set_device(0)

    datasets = configure_data_pipelines(hparams)
    hparams["train_dataloader_opts"]["batch_sampler"] = configure_train_batch_sampler(datasets, hparams, run_opts)
    train_set = datasets['train']
    valid_sets = datasets['valid']

    experiment_tracker_api_url = hparams['experiment_tracking']['api_url']
    experiment_tracker_web_url = hparams['experiment_tracking']['web_url']
    os.environ['CLEARML_API_URL'] = experiment_tracker_api_url
    os.environ['CLEARML_WEB_URL'] = experiment_tracker_web_url

    Task.force_requirements_env_freeze()
    tracker_task = Task.init(project_name='PROJECT_NAME', task_name='run_name')

    sb.create_experiment_directory(
        experiment_directory=hparams["output_folder"],
        hyperparams_to_save=hparams_file,
        overrides=overrides,
    )
    if not isinstance(train_set, DataLoader):
            train_set = sb.dataio.dataloader.make_dataloader(
                train_set, **hparams["train_dataloader_opts"]
            )

    valid_loaders = []
    if valid_sets is not None:
            for valid_set in valid_sets:
                valid_loaders += [sb.dataio.dataloader.make_dataloader(
                    valid_set, **hparams["valid_dataloader_opts"],
                )]

    for _ in range(1000000):
        for batch in tqdm(train_set):
            batch.to('cuda')
        for loader in valid_loaders:
            for batch in tqdm(loader):
                batch.to('cuda')
    tracker_task.close()

I'm not sure if the Speechbrain repo is the right place to post this issue, but maybe you can give me some advice about the сauses of the problem.
My enviroment:
Ubuntu 20.04.4
python 3.9.7
clearml	1.6.2
speechbrain	0.5.11
torch	1.10.1
cuda11.3
GPU: A100/V100

I'm not sure about it but maybe this os.fork patch is related to the issue
https://github.com/allegroai/clearml/blob/0397f2b41e41325db2a191070e01b218251bc8b2/clearml/task.py#L636
https://github.com/allegroai/clearml/blob/9f1487a9235fb449ee169ceeb1c459361191a382/clearml/binding/environ_bind.py#L88

Hi @kokamido,
We've released and rc with a fix to this issue, could you please test by installing pip install clearml==1.6.3rc1

Hello @kokamido,
any news on this issue? did you tried installing clearml==1.6.3rc1 ?

Hello,
There has been no activity for a very long time. Therefore, I am closing this issue.
Feel free to reopen if needed. Thanks! :)

@kokamido can u solve the problem?


Fix missing null ref check causing extension issues

This PR fixes #21973

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 1 (0.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.01%) to 41.913%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/sql/workbench/services/connection/browser/connectionWidget.ts
0
1
0.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.01%

Covered Lines:
28661

Relevant Lines:
63508

💛  - Coveralls


Some CSS fixes

Could you give me write access to your repo so I can add branches directly to it?

Changes manually merged in #53


Static file serving shows "Page Not Found", but loading on page refresh

Have you read the Contributing Guidelines on issues?

[X] I have read the Contributing Guidelines on issues.

Prerequisites

[X] I'm using the latest version of Docusaurus.
[X] I have tried the npm run clear or yarn clear command.
[X] I have tried rm -rf node_modules yarn.lock package-lock.json and re-installing packages.
[ ] I have tried creating a repro with https://new.docusaurus.io.
[ ] I have read the console error message carefully (if applicable).

Description
I am serving some static files (docx, pdf or png images) by putting them to the static/ folder and referencing them with a type: 'link' in sidebars.js:
    {
        type: 'category',
        label: 'Pictures',
        items: [
            {
                type: 'link',
                label: 'My Pic',
                href: 'mypic.png',
            },

        ]
    },

But clicking on the rendered menu__link gives a Page Not Found. By reloading the page, the document is loaded. Navigating back (with the browser) will not work again, one has to reload that page to show. What happens here?
Docusaurus: 2.0.0-beta.20
Google Chrome:  101.0.4951.64
Ubuntu 20.04.4 LTS
Reproducible demo
No response
Steps to reproduce

Put a file into the static directory, e.g. docusaurus_root/static/mypic.png
Create a link in a manual sidebar in sidebars.js:

    {
        type: 'category',
        label: 'Pictures',
        items: [
            {
                type: 'link',
                label: 'My Pic',
                href: 'mypic.png',
            },

        ]
    },

Serve the site through docusaurus own server in development mode:

npm run serve

Try to load the file in a browser by clicking on the link rendered.
You will get a Page Not Found page.
Reload the page in the browser.
The file will load and it will be shown.

Expected behavior
I expect the file to load by clicking on the rendered link.
Actual behavior
You will get a Page Not Found page instead.
Your environment

Docusaurus version used: 2.0.0-beta.20
Environment name and version: Chrome 101, Node.js 16.14.2
Operating system and version: Ubuntu 20.04.2 LTS

Self-service

[ ] I'd be willing to fix this bug myself.

Try href: 'pathname:///mypic.png',?
Documentation: https://docusaurus.io/docs/advanced/routing#escaping-from-spa-redirects

Thank you very much! That spared a lot of time and effort.


Blind positions are incorrect, used to be OK

I have 8 roller shades controlled with ESPSomfy RTS and integrated with HA using the HACS integration.  When I set up the system, I timed the open & close times, added & paired the shades & linked the remotes.
I don't know when or how it happened, but I noticed this morning that 6 of the 8 shades do not show the correct position in HA or on the ESPSomfy RTS web interface.  It appears HA and the web interface are in agreement with the "incorrect" positions, however.
Can you please let me know how to correct the reported blind positions?  Do I need to delete & re-add the shades & remotes?
Attached is a screenshot of the web interface.  All of the blinds are currently open 100%, but only the Master_BR_West and Casita shades reflect the position correctly.
I was running 1.4.6 but updated a few minutes ago to 1.5.3.  I am running HACS integration 1.0.12.
Thanks for your help!

No don't delete anything just press the up button on the ones that are out of sync in the ESPSomfyRTS interface and wait for them to finish.  Then update to v1.5.3

And Home Assistant now shows all of the shades as "open."  I'm going to close this issue to save you the hassle.
Thanks & have a super day!


Middleware proxy doesn't change the host header so can't be used with an API on a shared IP

Describe the bug
The middleware proxy in volto breaks with a connection refused if you the api path is set to api that uses the host header to direct the request.
To Reproduce

Deploy the plone backend behind a proxy that uses a host header to direct the request. Say api.myapp
Deploy the volto frontend to www.myapp and set the RAZZLE_API_PATH=https://api.myapp
Requests to https://api.myapp/++api++ will fail (because they will be sent with "Host: www.myapp" not "Host: api.myapp".

Expected behavior
Set the right host header.
https://github.com/plone/volto/blob/a34da0bdf63190e6209a8821e4af801eb2c8b8ff/packages/volto/src/express-middleware/devproxy.js#L49
Should also probably set X-Forwarded-Host etc - https://jdudzik.medium.com/posthog-reverse-proxy-with-node-express-js-480c1372f608
Screenshots
N/A
Software (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Volto Version [e.g. 8.5.0]
Plone Version [e.g. 5.2.2]
Plone REST API Version [e.g. 7.0.1]

Additional context
Add any other context about the problem here.

I can work on this issue

Hey, I fixed the issue with the Host header! The problem was that the proxy was sending requests with the wrong Host, so I updated the onProxyReq function to set it correctly to the API backend (api.myapp). Now, requests are going to the right place, and the “connection refused” error should be gone.
Let me know if it works for you!


hb-status reports wrong port when multiple instances

Describe The Bug
When multiple instances of homebridge are running (I had a specific reason to do this.)
hb-service status; hb-service --service-name=homebridge2 status; hb-service --service-name=homebridge4 status; hb-service --service-name=homebridge5 status
ℹ Testing hb-service is running on port 8581...
✔ Homebridge UI Running
ℹ Testing hb-service is running on port 8581...
✔ Homebridge UI Running
ℹ Testing hb-service is running on port 8581...
✔ Homebridge UI Running
ℹ Testing hb-service is running on port 8581...
✔ Homebridge UI Running
The actual ports are
51793, 51956, 51362, 51369
Logs
No response
Config
No response
Homebridge UI Version
homebridge-config-ui-x v4.50.4
Homebridge Version
Homebridge v1.6.1 (HAP v0.11.1) (Mac Mini)
Node.js Version
Node.js v18.16.0
Operating System
macOS
Environment Info

[ ] Using Docker?
[ ] Using Hyper-V?
[X] Using hb-service?

Raspberry Pi Model
None

hb-service install --service-name homebridge --port 8081
sudo hb-service uninstall --service-name homebridge6 --port 8582
sudo hb-service install --service-name homebridge2 --port 8582
sudo hb-service install --service-name homebridge3 --port 8583
sudo hb-service install --service-name homebridge4 --port 8584
sudo hb-service install --service-name homebridge5 --port 8585

@TeslaOwnerTips If you look at the source behind hb-service you can see that the status command checks for the homebridge UI and not individual instances.   So the behaviour is as per design


update deps

Codecov Report

Merging #95 (3b7e8c4) into main (b5be3c8) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main      #95   +/-   ##
=======================================
  Coverage   88.84%   88.84%
=======================================
  Files          16       16
  Lines         251      251
=======================================
  Hits          223      223
  Misses         24       24
  Partials        4        4

Impacted Files
Coverage Δ

app/cache/cache.go
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b5be3c8...3b7e8c4. Read the comment docs.


Error in expert api

the next url gives a traceback
http://impactu.colav.co:8080/api/affiliation?section=research&tab=products&page=4&max=100
this is requiered to download google scholar data

A ternary operator was added to control the error. https://github.com/colav-playground/hunabku_impactu/commit/33bfb4dff888a58fcc67de522a14150ee7c1e038


Overworld stutters occasionally when the player moves

Sometimes when the player moves, the overworld suddenly jerks to the left for one frame.
This is likely a bug in the overworld camera code, either how it is positioned or scaled.

The overworld stutter occurs when the _process function is called with a high delta (>50 ms). This occurs once a minute or so, although it appears to be more frequent when the game is initially launched.
I experimented with several attempted fixes:

Changing the camera code to run in _physics_process instead of _process
lerping the Camera2D's position so that it would move slower (this is redundant with the camera's built-in smoothing)
adjusting the Camera2D's smoothing parameters
Changing various project settings:

Enabling display/window/size/fullscreen. According to Godot's documentation, Windows is known to cause stutter in windowed games but gives games special priority in fullscreen. This appears to be untrue. On my PC, the camera stutters equally in windowed and fullscreen modes.
Switching the physics/2d/thread_model to the default value of Single-safe. Intuitively, this felt it could be tied to rendering/physics threads but this had no effect.
Enabling rendering/2d/snapping/use_gpu_pixel_snap. Several reddit threads (1, 2) indicate that this helps for 2D games, but it did not improve the stutter.

Exploration with the debugger. I could verify there was a very large performance spike when jitter occurred, although as with all other instances using the profiler, it did not stem from anything concrete.

After investigation, I'm leaning towards Godot's official documentation that this is just the way games work in Windows, I guess. I have no further ideas for how to investigate or fix this stutter.


Add receipt thumbnail to Search

Details
Adds receipt thumbnail to the Search page
Fixed Issues
$ https://github.com/Expensify/App/issues/41629
Tests
Pre-requisite: an account with money requests

Navigate to the Search page, Profile > Troubleshoot > New search page
Verify that you can see the transaction receipt thumbnail
Verify that if the transaction doesn't have a receipt, you can see the default placeholder
Verify that if you add a receipt and reload the page, you can see the receipt thumbnail

[x] Verify that no errors appear in the JS console

Offline tests

Verify that going offline changes the default receipt icon to the offline icon

QA Steps
Same as tests

[ ] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native

iOS: mWeb Safari

MacOS: Chrome / Safari
https://github.com/Expensify/App/assets/22219519/261d49c3-336b-4cc6-8da8-482af23887f0

MacOS: Desktop

@Expensify/design I kicked off an adhoc build. Could you please take a look at the receipt thumbnails in the search page once the build is ready? One thing to note is that I had to make the column wider than it is in Figma, otherwise the Receipt header would be truncated.

@luacmartins I feel dumb for asking, but how do I get to the search page on that build?

Oh sorry, I should have listed out the steps. The search page is hidden right now and can only be accessed via Profile > Troubleshoot > New search page

Ok here are a couple things I'm noticing:

Your table headers are set in our label size (13px), but in Figma we have them designed at our micro size (11px)
The receipt plus icon should be our icon color—right now it's a bit too close to the background color

@Expensify/design right now we have the background color of the receipt plus container set to row-hover, which looks good for the static and selected states, but causes the background to disappear when the row is hovered (which IMO is not ideal). We'll need to figure out how we want to handle that.

Ok, addressed the comments above and kicked off another adhoc build. Let me know what you think!

One thing to note is that I had to make the column wider than it is in Figma, otherwise the Receipt header would be truncated.

So in Figma, we are using a wrapper to wrap the "Receipts" text. We give the wrapper a fixed width of 36px like the receipt, but we allow it to have visible overflow so the centered text inside of it can still fully show. Does that makes sense, and is that something we can do here?

right now we have the background color of the receipt plus container set to row-hover, which looks good for the static and selected states, but causes the background to disappear when the row is hovered (which IMO is not ideal). We'll need to figure out how we want to handle that.

Agree, so @luacmartins can we give the receipt thumbnail a different color when the row is hovered? Perhaps we use the next step up which is product400 I think.

Done!

Dang this looks pretty great to me! Love how this is shaping up 😻

@luacmartins I do not see the receipt thumbnail in the web ad-hoc build as shown below. Where am I going wrong?
https://github.com/Expensify/App/assets/3004659/24ddf0eb-0765-4568-8381-21f06a57acbd

Hmm do you have any errors in the console?

Hmm do you have any errors in the console?

Good point. Looks like a CORS issue as the images could not be fetched.

Hmm can you see other attachments? Not sure if that's an issue with adhoc builds.

Hmm can you see other attachments? Not sure if that's an issue with adhoc builds.

I noticed this problem first with the local build of the PR branch. To reconfirm I checked with the ad-hoc build and I noticed the same problem there. By the way, I can see the receipt in the expense report as demonstrated in the video here.

Thanks! I'll take a look soon!

@grgia it'd be great if you could review this one

@rojiphil could you try your local build removing the isAuthTokenRequired prop? Do you still see errors then?

could you try your local build removing the isAuthTokenRequired prop? Do you still see errors then?

I tried after setting isAuthTokenRequired to false. But I could still see the errors.

Updated!

@luacmartins The ReceiptPlus icon is not displayed on the local build of android native platform. The attached test video demonstrates this. But I do not notice this problem on other platforms though.
What could be going wrong here?
https://github.com/Expensify/App/assets/3004659/145e3834-7c9b-47bb-8258-869a4b288a80

@luacmartins @rojiphil I did some investigation for this bug and it looks like ExpoImage can't handle empty string properly. On iOS and web what we actually see is a fallback icon. On Android empty string won't trigger onLoadFailure in the ThumbnailImage
I think we should report this problem to the expo team and maybe we could do a simple check for an empty string somewhere in our wrapper for ExpoImage and trigger.
Something like that in BaseImage.native.tsx works.
useEffect(() => {
  if (source?.uri !== '') {
    return;
  }
  onError?.();
}, [source, onError]);

There is something wrong with types for source. It doesn't see uri. But it's there.

@rojiphil I believe the issue is fixed now, could you please review again?

@rojiphil can you please review again?

Looks good so far,

PDF receipt is not showing icon @luacmartins

Looks good on mobile

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by https://github.com/luacmartins in version: 1.4.77-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/puneetlath in version: 1.4.77-11 🚀

platform
result

🤖 android 🤖
failure ❌

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/puneetlath in version: 1.4.77-11 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


Saving topomap animation doesn't include plot titles (time points)

MWE:
import numpy as np
from mne.datasets import sample
from mne import read_evokeds

path = sample.data_path()
fname = path + '/MEG/sample/sample_audvis-ave.fif'
condition = 'Left Auditory'
evoked = read_evokeds(fname, condition=condition, baseline=(None, 0))

times = np.linspace(start=evoked.times[0], stop=evoked.times[-1], num=25,
                    endpoint=True)

fig, anim = evoked.animate_topomap(times=times,
                                   ch_type='eeg',
                                   frame_rate=2,
                                   time_unit='s',
                                   butterfly=False,
                                   blit=False,
                                   show=False)

anim.save('/tmp/foo.gif')

Output
MovieWriter ffmpeg unavailable; using Pillow instead.
Initializing animation...

Whereas the respective Matplotlib figures contain the titles; they simply seem to get lost during saving:
https://mne.tools/dev/auto_examples/visualization/plot_evoked_topomap.html#animating-the-topomap

I'll work on this!


empty file

Hey, Hope you're doing well.
I just tested the js file and it giving me empty file. il also tested the firefox extension and i have the same problem. Am'I missing something ?
I'm testing with this profile https://www.tiktok.com/@fullhistoiresredditfr (it's mine🙃)
Thanks

Hi! I tried the script and it seems to be working. Have you enabled some custom options for video downloading?
I've also pushed an update to the script (I'll upload it on Mozilla Addons later) that adds a new method for extracting the video container. Try enabling the "get_video_container_from_e2e" option from the "advanced" object in the "scriptOptions" variable, and let me know if in that way the script works.
If you're still having issues, would you mind sending the HTML class of the video container? Maybe TikTok is rolling out a new change in the HTML layout and I'll need to fix the script. Let me know if that worked!

Hey,
It start to work when i thick the "Use the [data-e2e] attributes for getting the video container" in advanced setting.
Thanks. Have a great day

Glad I helped!


Skip funding confirmation

This PR introduces the ability to skip a channel funding transaction confirmation, making the channel fully operational before its on-chain confirmation (aka a zero-conf channel).
Till confirmation, this channel requires trust between its two parties and in the case of a remote initiator, it puts the received funds of the local party at risk.
Nevertheless, there are cases where it makes sense to support this behavior. For example, in cases both parties decide to trust each other. Or, in cases where trust between the parties already exists (buying a pre-loaded channel from a service like Bitrefill).
We at Breez want this capability to support the following:
Immediate on-boarding:
Currently, new users need to wait for channel confirmation before being able to receive. This PR, together with https://github.com/lightningnetwork/lnd/pull/4018, enables us to provide a far better user experience as follows:
Let's say Alice wants to pay a new Breez user, Bob.

Bob is connected to the Breez node and issues an invoice with a routing hint that points to a fake channel between Bob and the Breez node.
When Alice pays Bob's invoice, the Breez node intercepts the HTLC and holds it.
Then, the Breez node does the following:

Opens a channel to Bob.
Uses the fake short channel id in the routing hints to skip funding confirmation.
Waits for the channel to become active.
Pays Bob the original Alices' tx (potentially, minus a service fee https://github.com/lightningnetwork/lnd/pull/4423)
Settles backword.

When Bob has a new pending channel it is his choice whether to call SkipFundingConfirmation or use the default behavior that requires confirmation.
In case Bob decided to skip confirmation all received payments are held, giving him control whether to settle additional payments and add to existing trust between the parties.

Behavior:
In order to skip confirmation, the funding locked message should be sent and received on both sides.
Both parties need to agree on a “fake” short channel id and because the protocol doesn't define any kind of message that supports it,  a simplified approach was taken where both parties use the "temporary_channel_id" 58 first bits to create a "fake" short channel id. This leaves 18 bits for the block height which ensures it will be less than 2^18 = 262144.
This PR is a work in progress on which I would like to get early feedback regarding the overall approach.

@roeierez thanks for the PR, good to get this discussion going! This has been a highly requested feature, and I do think we should look to support it.
IMO the main thing we have to consider is that currently there is critical forwarding info that is indexed by short channel id, and the logic assumes that the channel is only ever used after the short channel id is final, see forwarding packages. In practice, things will break if the short channel id is changed after using the channel as is done in this PR. Further, various parts of the codebase assume that UpdateShortChannelID is only called once—it's been two+ years since that code has been touched, so we want to make sure we aren't missing anything critical.
I think there is a preliminary step before this proposal is possible, i.e. migrating the forwarding packages to use a key that is static for the lifetime of the channel, which probably means either channel id or channel point. In theory this would allow us to mutate the short channel id on the fly, though i admit i have not thought about all the possible edge cases in the switch if this mapping is modified.

@cfromknecht Appreciate you jumped in and your input is exactly what I am trying to collect here. The WIP prefix will only be removed once I feel it is ready and currently there is some way to go.
I will definitely look into the forwarding_package and then we can continue the discussion. As for the switch, since the code there already had the mechanism to refresh the short channel id I am more comfortable with the small change there but I will get the safety feeling only after I will implement the required tests.

I think there is a preliminary step before this proposal is possible, i.e. migrating the forwarding packages to use a key that is static for the lifetime of the channel, which probably means either channel id or channel point. In theory this would allow us to mutate the short channel id on the fly, though i admit i have not thought about all the possible edge cases in the switch if this mapping is modified.

@cfromknecht I looked into it and thought it is better to break the preliminary steps into separate PRs. Started with this https://github.com/lightningnetwork/lnd/pull/4459. If this looks reasonable to you I will continue to add the missing migration that will enable us to move forward.

I did some major changes to the initial PR by favoring some conventions over flexibility.
In general, I've adopted the keysend model: the risk in an unconfirmed channel is not in the channel itself, rather in actually accepting a payment while the channel is unconfirmed. In some cases, one would want to accept such payments (and the risk involved) on their amount. Others may use a different criteria that are not known at the time of opening.
Based on that, this approach works as follows:

The non-funding node (fundee) decides whether to skip confirmation by sending min_depth=0 in accept_channel message.
The funder then skips confirmation and sends funding_locked (as the funder is not at risk).
Both nodes uses a convention for the short channel id by applying a simple transformation on the random pendingID. This convention is simple and doesn't require additional RPC layer as suggested in the previous version.
All received payments on an unconfirmed channel are held and require the user to explicitly settle them. This mechanism makes it safe to skip confirmation and allows users to selectively approve payments and control their risk

I think it may be better to first start a conversation on the spec level w.r.t how zero-conf channels are negotiated and implemented. Otherwise, we may end up shipping with a version that only works with lnd and not the other implementations (all of which I think have some form of 0-conf channels being used in the wild?).

Started here: https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-August/002780.html

Rebased

Thinking about this a bit more, what if we just made the zero conf chan funding into a new channel type? This would address the config issues (always accepting to zero conf), as it would effectively let the initiator attempt to propose a zero conf channel.

Thinking about this a bit more, what if we just made the zero conf chan funding into a new channel type? This would address the config issues (always accepting to zero conf), as it would effectively let the initiator attempt to propose a zero conf channel.

Sounds interesting. I will check this direction.

Thinking about this a bit more, what if we just made the zero conf chan funding into a new channel type? This would address the config issues (always accepting to zero conf), as it would effectively let the initiator attempt to propose a zero conf channel.

Went over the code again and I think we can add another ChanType flag for zero conf.  I think the main advantage would be to stop relying on the fake id (isFake method) to determine funding flow and instead use the channel type.
The config option is for the responder to determine if the accept_channel message may include min_depth=0 on the case the remote peer supports it. The acceptor still have the opportunity to override this decision using the ChannelAcceptor hook and return a positive min_depth.
As for the initiator, if he provisions the SkipFundingConfirmationOptional feature currently it means he will always accept min_depth=0 as a response for chan_accept message (the risk is not on his side as he is the funder). We can limit this behaviour by adding another field for the open channel request (allowSkipConfirmation) and consider it when handling the chan_accept message. @Roasbeef is that what you meant?

Is there a BOLT or blip writeup of the spec changes to this? This is something that is going to really want cross-implementation compatibility and I don't see a writeup to provide feedback against. anywhere

Is there a BOLT or blip writeup of the spec changes to this? This is something that is going to really want cross-implementation compatibility and I don't see a writeup to provide feedback against. anywhere

hi @TheBlueMatt yes, @roeierez wrote up their approach on the mailing list almost a full year ago. If bLIPs (or something similar) were a thing I think it would've received more feedback from other implementations over the past 12 months. Hopefully we can tweak the process to ensure the next feature like this doesn't suffer the same fate!

Hey everyone! Just checking in on this and the overall status for 0conf support in LND. Last I heard, there was an intention to get support into the next release. Where do things stand now, and is there anything any of us can do to facilitate? Thanks!

Hi, we're still working through some possible edge cases that exist in this PR as well as the portion scheduled to land in the spec, which now includes support for arbitrary channel aliases, which improves privacy as you no longer leak the location of your funding transaction on chain when creating an invoice with hop hints.

Yeah I didn't check the new code yet but I hoped for compatibility with Rusty's BOLT spec proposal.
Awesome to hear. 💯

@cfromknecht: review reminder

@roeierez, remember to re-request review from reviewers for your latest update

@cfromknecht: review reminder

@roeierez, remember to re-request review from reviewers for your latest update

@cfromknecht: review reminder

@roeierez, remember to re-request review from reviewers for your latest update

@cfromknecht: review reminder

@cfromknecht: review reminder
@roeierez, remember to re-request review from reviewers when ready

@cfromknecht: review reminder
@roeierez, remember to re-request review from reviewers when ready

@cfromknecht: review reminder
@roeierez, remember to re-request review from reviewers when ready

@cfromknecht: review reminder
@roeierez, remember to re-request review from reviewers when ready

@cfromknecht: review reminder
@roeierez, remember to re-request review from reviewers when ready

@cfromknecht: review reminder
@roeierez, remember to re-request review from reviewers when ready

Has been replaced by https://github.com/lightningnetwork/lnd/pull/5955


Restrict Permissions

Do we really have to allow full read/write access to the entire public to use this feature?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e9e73fcc-e961-749e-13e2-11fe5b2691dd
Version Independent ID: 96e45113-f396-63af-ba9f-2e5c1e55a107
Content: Hosting Windows 10 app packages on AWS for web install - MSIX
Content Source: msix-src/app-installer/web-install-aws.md
Product: msix
GitHub Login: @mcleanbyron
Microsoft Alias: mcleans

the write access is required to upload content to the bucket for the owner. The public will only require read permissions.

Ok, do we really have to give complete public read access to our s3 bucket? I tried applying permissions and using a signed url but App Installer rejects the URL.

Yes. You will need to give read access to your S3 bucket. There might be a way to only provide read access to a particular folder.
What is the error that you see in App Installer?


Discussion - Atlas and GAIA for BNS v2

Opening up this ticket just so we can have a discussion around it. Its entirely possible that there are no changes that will be required in atlas or gaia for the upgrade, but thats what we have to figure out.
Atlas network: runs on the stacks node and is responsible for tracking/storing/replicating attachment data.
GAIA: currently unsure how GAIA comes into play for BNS names
Some initial questions:

Zonefile hashes are stored on the BNS contract in a map. Will migrating those ensure zone-file attachments are available for BNS v2?
What is GAIA's role in BNS, what information does it store?
Stacks API has a separate table for zone files that it uses exclusively for the BNS apis. Are those zone files a separate entry or are those synced with stacks-nodes? How does Stacks API populate its own entries?

PS. This is just to get the conversation started, I'll keep on adding more information as I find it.
Known issues:

https://github.com/stacks-network/stacks-core/issues/3270#issue-1357482437

By default they are set to track .bns events here

This will have to be updated, nodes will have to track the new bns contract

Can you open an issue on the Stacks blockchain repo so this gets done prior to BNSv2 going live?  Presumably, this would be part of the Nakamoto release.

Apparently, there does not seem to be any other link between BNS and gaia. With BNS v2 its unlikely that any change will be required as long as zone files are maintained.

Gaia is defunct, I'm afraid.  The Atlas system, however, still enjoys first-class support, especially in the service of .btc.us domains.

The Atlas system, however, still enjoys first-class support
Good to know.
Looking forward to support for https://github.com/stacks-network/stacks-core/issues/3270


yarn/npm install failed

after pull the newest master branch, yarn/npm install, throw error:
><EMAIL_ADDRESS>postinstall /Users/luoguochun/tmp/electron/ele-react
> node -r babel-register internals/scripts/CheckNativeDep.js && npm run flow-typed && npm run build-dll && electron-builder install-app-deps && node node_modules/fbjs-scripts/node/check-dev-engines.js package.json

/Users/luoguochun/tmp/electron/ele-react/internals/scripts/CheckNativeDep.js:2
import fs from 'fs';
       ^^

SyntaxError: Unexpected identifier
    at new Script (vm.js:74:7)
    at createScript (vm.js:246:10)
    at Object.runInThisContext (vm.js:298:10)
    at Module._compile (internal/modules/cjs/loader.js:657:28)
    at loader (/Users/luoguochun/tmp/electron/ele-react/node_modules/babel-register/lib/node.js:144:5)
    at Object.require.extensions.(anonymous function) [as .js] (/Users/luoguochun/tmp/electron/ele-react/node_modules/babel-register/lib/node.js:154:7)
    at Module.load (internal/modules/cjs/loader.js:599:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
    at Function.Module._load (internal/modules/cjs/loader.js:530:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:742:12)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>postinstall: `node -r babel-register internals/scripts/CheckNativeDep.js && npm run flow-typed && npm run build-dll && electron-builder install-app-deps && node node_modules/fbjs-scripts/node/check-dev-engines.js package.json`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>postinstall script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR!      /Users/luoguochun/.npm/_logs/2018-07-12T02_59_44_663Z-debug.log
^_^@/Users/luoguochun/tmp/electron/ele-react]$ vim package.json

the newest release version V0.13.3 doesn't have any problem.
env:
os: macos 10.12.6
node: v10.6.0
yarn: 1.7.0
npm: 6.1.0

Try rm -rf node_modules && yarn cache clean && yarn

@amilajack , I tried this way(rm -rf node_modules && yarn cache clean && yarn) several times, but produce the same error(only happen in the master branch)! 😕

I had the same issue before noticing I failed to copy hidden files located at the root of the project.

I can't reproduce (at 118560b), both npm and yarn work. @buf1024 we have different node versions and different OS.
@Wharenn can you attach your env info?
env:
os: ubuntu 18.04
node: v8.9.4
yarn: 1.7.0
npm: 6.1.0

As I said, it was a fail on my side, I did a cp of the directory and omitted to copy hidden files ;) Everything works fine now :) Thanks


batch operations: add tuple_index field in error object

CRUD can be used by projects written in programming languages
other than lua (at the moment CRUD batch operations are used in tt import).
In this case the format of the tuple from error object and the format of tuple
representation in the application are different. Therefore, it becomes
difficult to understand on which tuple the operation was interrupted.
The solution is to add tuple_index field in error object.
This field should contain index of tuple in original batch.

The data is in the operation_data field (because it may contain a tuple, an object or an upsert {tuple, ops} operation). So it seems meaningful for me to name the field with the index as operation_index.

The data is in the operation_data field (because it may contain a tuple, an object or an upsert {tuple, ops} operation). So it seems meaningful for me to name the field with the index as operation_index.

The data is in the operation_data field (because it may contain a tuple, an object or an upsert {tuple, ops} operation). So it seems meaningful for me to name the field with the index as operation_index.

Maybe operation_data_index?


Cannot Connect to Cider

Info:
Cider Version: 1.4.0
Cider Remote Version 1.0.1(3)
Issue:
When trying to connect, all I see is a spinning circle that says "Scanning Cider Remote Instance". When I go to scan a QR code, I cant find one in the app. Guessing that Version 1.4.0 does not support Cider Remote. Please let me know!

I currently am using the Cider Client on Ubuntu but have a dual boot to Windows 10. I think the issue is that I am not on the alpha version and just the base version. Going to see if I can get it to update to alpha, mess around with the firewall a bit, and will report back. On Version 1.4.0, when you click "Web Remote" all it does it take me to Cider's main page to download Cider Remote.
Thank you for your suggestions!


About the management of the default certificate for ASE

[Enter feedback here]
From the following description, I recognize that the default TLS/SSL certificate for *.appserviceenvironment.net is managed by MS for the ILB ASEs made after May 2019.

The zone named ..appserviceenvironment.net is globally unique. Before May 2019, customers were able to specify the domain suffix of the ILB ASE. If you wanted to use .contoso.com for the domain suffix, you were able do so and that would include the scm site. There were challenges with that model including; managing the default TLS/SSL certificate, lack of single sign-on with the scm site, and the requirement to use a wildcard certificate. The ILB ASE default certificate upgrade process was also disruptive and caused application restarts. To solve these problems, the ILB ASE behavior was changed to use a domain suffix based on the name of the ASE and with a Microsoft owned suffix. The change to the ILB ASE behavior only affects ILB ASEs made after May 2019. Pre-existing ILB ASEs must still manage the default certificate of the ASE and their DNS configuration.

In other words, I recognize that the default certificate which will expire soon is updated automatically by MS without the user action.
If my recognition is wrong, please let me know.
If my recognition is right, could you explicitly write that the default certificate is managed by MS and the user don't need some action to update the default certificate which will expire soon?
In addition, it would be better if the detail of the renewal process (when the expired certificate is renewal, whether there is any log which shows the update of the default certificate, and so on) is written.
We are often asked by customers about that and would be glad if you would consider it.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e754567e-9482-5693-5ef8-6cc811ea7f57
Version Independent ID: 7c87e9ce-cf7f-120d-e6a3-9b605e231ef3
Content: Use and manage an App Service Environment - Azure App Service Environment
Content Source: articles/app-service/environment/using-an-ase.md
Service: app-service
GitHub Login: @madsd
Microsoft Alias: madsd

@atsushinak
Thanks for your feedback! We will investigate and update as appropriate.

@atsushinak Thank you for the feedback and request for clarification.
Your understanding is correct that any domain owned by Microsoft such as *.azurewebsites.net, *appserviceenvironment.net, p.azurewebsites.net, etc. and their respective SSL/TLS certificates are managed and rotated by Microsoft and do not require action on behalf of the customer.
As a result, we added an update to the document to provide the clarity needed. The change should be public within ~24 hours.
In regards to Microsoft owned and managed TLS/SSL certificate rotations being recorded into customer logs, this would need to be a feature request. We are meeting with the product group later this week and we will share your idea with them to see if this is feasible as a feature down the road.
We will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.


fix garbled file's title on leftfovers.

When file's title on leftovers contains multi-bytes character,
copied file's title was garbled.
Drive.Files.get(id, {alt: 'media'}) cannot specify content encoding.
The content encoding is expected UTF-8.
So, getBlob().getDataAsString() Instead of Files.get().

First, let me say thank you for contributing!
Second, what functional change does this achieve? Are you getting garbled file names during the copy process or is it just the "leftovers" file you're trying to fix. I'm less inclined to introduce changes if its just the leftovers file - the titles in that file are not used after the top-level folder is created.

Nevermind my second comment, I realized where this is used. Let me try to find some time this weekend to do some testing and make sure this doesn't break anything so I can get this merged.


ClientModel prototype: paging approach where service creates subtypes

Centered around ideas:

PageCollection is a client
3p services will have different types of page tokens
Supports rehydration
Types are provided to support generation from TypeSpec, not end-user use

API change check
APIView has identified API level changes in this PR and created following API reviews.
System.ClientModel

Closing in favor of https://github.com/Azure/azure-sdk-for-net/pull/44825


Consolidate logging in istio-cni

Describe the feature request
Right now if you want to enable logging for the istio-cni daemonset you might try to supply the following granular logLevel config:
level: default:info,cni:debug
this will only show debug logs for the cni plugin (used by ambient or sidecar) but will not show any logs for the ambient controller, or the CNI plugin (sidecar or ambient) install logic, which are registered as different subcomponents within istio-cni - for that you need (I think):
level: default:info,cni:debug,ambient:debug,install:debug
This is confusing, and nonobvious. We should consider using either a common component name for log signaling for everything in istio-cni (istio-cni maybe), or something a bit smarter.
Describe alternatives you've considered
Affected product area (please put an X in all that apply)
[ ] Ambient
[x] Docs
[ ] Dual Stack
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
Affected features (please put an X in all that apply)
[ ] Multi Cluster
[ ] Virtual Machine
[ ] Multi Control Plane
Additional context

Why not just set all:debug if you want everything?

Why not just set all:debug if you want everything?

That will work if you explicitly set cni.logLevel directly/by itself yep. And since cni.logLevel currently ignores the global setting, that's fine, but that still results in having to do something like
--set values.cni.logLevel="all:debug" --set values.global.logging.level="all:info"
ideally I'd want cni.logLevel to inherit the global log config, and it cannot as long as we have :point_up:

Hello, a newbie here. Can you guys help me understand this a bit more

@yas8say Sorry! I ended up fixing this in https://github.com/istio/istio/pull/51074 myself because I was already working in the area.
TYVM for volunteering!


sc-8193 : update sentry to the latest version

Scope of changes
sc-8193 : update sentry to the latest version
Type of change

[ ] bug fix
[ ] new feature
[ ] documentation
[x] other (refactor)

Acceptance criteria
sc-8193 : update sentry to the latest version
Author checklist

[ ] I have manually tested the change and/or added automation in the form of unit tests or integration tests
[ ]  I have updated the dependencies list
[ ]  I have recompiled and included new protocol buffers to reflect changes I made
[ ]  I have added new test fixtures as needed to support added tests
[x]   Check this box if a reviewer can merge this pull request after approval (leave it unchecked if you want to do it yourself)
[x]  I have moved the associated Shortcut story to "Ready for Review"

Reviewer(s) checklist

[ ] Any new user-facing content that has been added for this PR has been QA'ed to ensure correct grammar, spelling, and understandability.

Codecov Report

Merging #749 (d67e453) into user-ui-post-refactoring-1.5.0 (f82eb8e) will decrease coverage by 11.66%.
The diff coverage is n/a.

@@                         Coverage Diff                         @@
##           user-ui-post-refactoring-1.5.0     #749       +/-   ##
===================================================================
- Coverage                           43.53%   31.86%   -11.67%
===================================================================
  Files                                 584      479      -105
  Lines                               17544     5451    -12093
  Branches                             1329     1329
===================================================================
- Hits                                 7637     1737     -5900
+ Misses                               8718     3526     -5192
+ Partials                             1189      188     -1001

Impacted Files
Coverage Δ

...github.com/trisacrypto/directory/pkg/utils/week.go

...m/trisacrypto/directory/pkg/sectigo/serializers.go

.../trisacrypto/directory/pkg/trtl/replica/replica.go

.../trisacrypto/directory/pkg/gds/store/trtl/index.go

....com/trisacrypto/directory/pkg/bff/auth/options.go

...trisacrypto/directory/pkg/utils/whisper/whisper.go

.../trisacrypto/directory/pkg/gds/models/v1/models.go

...com/trisacrypto/directory/pkg/utils/wire/rewire.go

...sacrypto/directory/pkg/gds/models/v1/pagination.go

...thub.com/trisacrypto/directory/pkg/bff/parallel.go

... and 95 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


archi: build package from source and bump to 5.1.0

Description of changes
Builds this package from source, additionally this PR bumps Archi to it's most recent version.

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

This build doesn't seem to be reproducible, I marked this PR as draft till this issue was solved.

Due to the reproducibility issues this PR is superset by #228775


ordering of option declarations should not prohibit forward reference in default procs

Describe the bug
the :default keyword supports an "instance_exec" type of evaluation whereby default: -> { foo + bar }' reference the object in questions' fooand barmethods.   however, when eitherfooorbarare, themselves, dry-initialized attributes, the default value will resolve tonil` due to being referenced prior to 'dry initialization'
To Reproduce
require 'dry-initializer'

class A
  extend Dry::Initializer
  option :foo, default: -> { bar }
  option :bar, default: -> { 42 }
end.new

class B
  extend Dry::Initializer
  option :bar, default: -> { 42 }
  option :foo, default: -> { bar }
end.new

pp A.new #=> #<A:0x0000000103009958 @bar=42, @foo=nil>
pp B.new #=> #<B:0x000000010302aea0 @bar=42, @foo=42>

Expected behavior
@bar == @foo == 42
My environment
n/a

How would it work, though? Implementation-wise I mean.

essentially, the reader must be read consistent.   specifically, the first invocation needs to memo-ize the default proc, vs simply looking at the list of options.  eg.

ivar = "@#{ name }"

defined_method(name) do
  initialize_default_value_for(name) unless instance_variable_defined?(ivar)
  instance_variable_get(ivar)
end

i'll see if i can gin up a PR

But this would break the assumption an object is fully initialized after .new call. Normally, I don't mutate objects meaning I call .freeze after initialization. Your suggestion will break in such a use case.

not necessarily.  one would simple need to hit all readers to initialize the values before freezing.   freezing is nice but it also means that objects are expensive, as memoization is good for speed.   regardless, it's a bug-ette in dry.rb now, since a specified default is not applied, depending on ordering.  if this is by design so be it but i'm guessing it's a side-effect of other design goals.  i think all goals can be achieve.  the code is fairly abstract but i'm working on something now....

Freezing is not about the cost actually. It's a technique of separating boot time from runtime. An application stack and its constituents are meant to be immutable, hence frozen.
Anyway, handling such cases is somewhat problematic:
  option :foo, default: -> { bar }
  option :bar, default: -> { foo }


new feature

Hi. How about adding functionality?
You can take similar rules from the site itself.
Need to sort by popularity. This will greatly simplify the search for products with a lot of reviews.

It is already sorted by popularity, which is a default sort. Be advised that

popularity in terms of violet marketplace is not a feedback ratio, but more like a set of its own heuristics. Probably it doesn't even include feedbacks quantity as a sort signal. Or does, who knows?
every search request is personalized (by search history, geolocation, ad reactions and many more factors). In current implementation it's more like a snapshot of a search which was actually made some time ago, but even so sort is always mutating depending on marketplace algorithms maintained by a huge marketing team, so be prepared that it doesn't match the order when you're traversing the violet marketplace.
there's also a sort by rating. I tried it, but it also doesn't correlate with feedback quantity.

I'm gonna close the issue since there's not so much to add. As for me - I don't use search much since there's a better discover route:

visit an item
look into person's feedback list
find items of interest
repeat until infinity
But I do believe that every person uses the tool differently. Sorry that I can't help much on this one.


Support Python 3.8

Not much to say about this. The tests all pass and things look OK. The linters are updated.

:fish: lgtm!


Community contribution -  optimum.exporters.onnx support for new models!

Following what was done by @ChainYo in Transformers, in the ONNXConfig: Add a configuration for all available models issue, the idea is to add support for exporting new models in optimum.exporters.onnx.
This issue is about the working group specially created for this task. If you are interested in helping out, reply here, take a look at this organization, or add  ChainYo#3610 on discord.
We want to contribute to Hugging Face's ONNX export implementation for all available models on Hugging Face Hub. There are already a lot of architectures implemented for converting PyTorch models to ONNX, but we need more! We need them all!
Feel free to join us in this adventure! Join the org by clicking here
Here is a non-exhaustive list of models that all models available:

[x] Albert
[x] BART
[x] BeiT
[x] BERT
[ ] BigBird (Critical issue: https://github.com/huggingface/optimum/issues/754#issuecomment-1429830467)
[ ] BigBirdPegasus (Critical issue: https://github.com/huggingface/optimum/issues/754#issuecomment-1429830467)
[x] Blenderbot
[x] BlenderbotSmall
[ ] BLIP-2
[x] BLOOM
[x] CamemBERT
[ ] CANINE
[x] CLIP
[x] CodeGen
[x] ConvNext
[x] ConvBert
[ ] CTRL
[ ] CvT
[x] Data2VecText
[x] Data2VecVision
[x] Deberta
[x] DebertaV2
[x] DeiT
[ ] DecisionTransformer
[x] DETR
[x] Distilbert
[ ] DPR
[ ] DPT
[x] ELECTRA
[ ] FNet
[ ] FSMT
[x] Flaubert
[ ] FLAVA
[ ] Funnel Transformer
[ ] GLPN
[x] GPT2
[x] GPTJ
[x] GPT-Neo
[x] GPT-NeoX
[x] Hubert
[x] I-Bert
[x] ImageGPT 🛠 @adit299
[ ] LED
[x] LayoutLM
[ ] LayoutLMv2 (but 🛠 in Transformers)
[x] LayoutLMv3
[ ] LayoutXLM
[ ] LED
[x] LeViT
[ ] 🛠 Longformer (Critical issue: https://github.com/huggingface/optimum/issues/776#issuecomment-1429680121)
[x] LongT5
[ ] Luke (but 🛠 in Transformers)
[ ] Lxmert
[x] M2M100
[ ] MaskFormer
[x] mBart
[ ] MCTCT
[x] MPNet
[x] MT5
[x] MarianMT
[ ] MegatronBert
[x] MobileBert
[x] MobileViT
[ ] Nyströmformer
[x] OpenAIGPT-2
[ ] OPT ((but 🛠 in Transformers)
[x] OWLViT
[ ] Pix2Struct
[x] PLBart
[x] Pegasus
[x] Perceiver
[x] PoolFormer
[ ] ProphetNet
[ ] QDQBERT
[ ] RAG
[ ] REALM
[ ] Reformer (but 🛠 in Transformers)
[x] RemBert
[x] ResNet
[x] RegNet 🛠 @asrimanth
[ ] RetriBert
[x] RoFormer
[x] RoBERTa
[x] SEW
[x] SEW-D
[x] SegFormer
[x] Speech2Text
[ ] Speech2Text2
[x] Splinter
[x] SqueezeBERT
[x] Swin Transformer
[x] T5
[ ] TAPAS 🛠 @someshfengde
[ ] TAPEX
[ ] Transformer XL
[x] TrOCR
[ ] UniSpeech
[ ] UniSpeech-SAT
[ ] VAN
[x] ViT
[ ] Vilt
[ ] VisualBERT
[x] Wav2Vec2
[x] WavLM
[x] Whisper
[ ] XGLM
[x] XLM
[ ] XLMProphetNet
[x] XLM-RoBERTa
[x] XLM-RoBERTa-XL
[ ]  XLNet (but 🛠 in Transformers)
[x] YOLOS
[ ] Yoso

🛠 next to a model suggests that the PR is in progress. If there is nothing next to a model, it means that ONNX does not yet support the model, and thus we need to add support for it.
If you need help implementing an unsupported model, here is a guide from HuggingFace Optimum documentation.

Hi! I'm trying to add support for VisualBERT, which works for VQA, VCR, NLVR and RPG.
Since the guide says that "When inheriting from a middle-end class, look for the one handling the same modality / category of models as the one you are trying to support.", I'm using TextAndVisionOnnxConfig because this is a multimodal model. Then initialized NORMALIZED_CONFIG_CLASS = NormalizedTextAndVisionConfig
I this OK so far?
The problem comes when implementing the inputs property... What is it that this property specifies? In the guide, I see that this inputs are exactly BERT's tokenizer's output keys, and values are the tensor dimensions for each key of the tokenizer's output. This will vary task-wise so I'd have to make a different axis for each task. Is this ok?
Thanks for the help!

Hi @mszsorondo , indeed the page https://huggingface.co/docs/transformers/serialization#export-to-onnx is a bit outdated. I'll do a PR to fix it. In you EDIT II, were you referring to this page?
I'd recommend to refer to: https://huggingface.co/docs/optimum/main/en/exporters/onnx/usage_guides/contribute . If you see any issue / unclear steps in the guide, don't hesitate to open a PR!
As for VisualBERT, I guess you haven't picked the easiest one :) I think you can leave VisualBertForPreTraining aside, it's probably better to support the rest for inference.
Indeed NORMALIZED_CONFIG_CLASS = NormalizedTextAndVisionConfig seems good.

The problem comes when implementing the inputs property... What is it that this property specifies? In the guide, I see that this inputs are exactly BERT's tokenizer's output keys, and values are the tensor dimensions for each key of the tokenizer's output. This will vary task-wise so I'd have to make a different axis for each task. Is this ok?
EDIT: I see VisualBERT is implemented separately by task, but VisualBertForPreTraining is also provided for customized down-stream tasks. Should I implement a diferent configuration for each task?

I don't think you need to implement configs for each tasks. Apparently all tasks take as inputs input_ids, token_type_ids, attention_mask, visual_embeds, visual_token_type_ids, visual_attention_mask. The VisualBertForRegionToPhraseAlignment seem to have an additional region_to_phrase_position input.
To implement the input method, you need to specify which inputs / outputs the model takes, and what are the dynamic axis: for example, for CLIP, that is https://github.com/huggingface/optimum/blob/9ac17034b6cb27da23499393598086f0b3b9223d/optimum/exporters/onnx/model_configs.py#L523-L528
You can very well do an if/else in the input/output keys (or axis) depending on the task, for example BART: https://github.com/huggingface/optimum/blob/9ac17034b6cb27da23499393598086f0b3b9223d/optimum/exporters/onnx/model_configs.py#L382-L389

Thanks for your help @fxmarty

In your EDIT II, were you referring to this page?

I was actually referring to the second guide (https://huggingface.co/docs/optimum/main/en/exporters/onnx/usage_guides/contribute), there are some minor issues with two function calls at the export step + one lacking import. Submitted PR #662
I advanced with the inputs function and did the export step, and indeed got an error regarding visual_embeds (surely this is also a problem for visual_token_type_ids, visual_attention_mask and region_to_phrase_position as you suggest), so I'll go for the new input generator.

Hi @michaelbenayoun!
Is someone working on adding the Pegasus ONNX config?
If not, I would like to look into it 😄(under your guidance, since I haven't done written a ONNXConfig yet)

Hi @bhavnicksm , @mht-sharma just merged the Pegasus ONNX config yesterday! https://github.com/huggingface/optimum/pull/620

Hi @bhavnicksm , @mht-sharma just merged the Pegasus ONNX config yesterday! https://github.com/huggingface/optimum/pull/620

I tried to use Pegasus with an inference right now using ORTModelforSeq2SeqLM and it gave me an error saying Pegasus is not yet supported.
I installed optimum directly from source here

@bhavnicksm Can you open an issue in Optimum? We can track it there!

@fxmarty Please re-open this. 🤗

Thanks!

I can look into ImageGPT, if it has not yet been claimed.

Feel free! Don't hesitate to ask any question if needed.

Can I take TAPAS if it's not yet been claimed?

Hello, Can I work on RegNet?

Yes to both, feel free!
I updated the list saying that you are working on it.

Hi @michaelbenayoun, I went into the codebase recently and I think the list above may not be the latest update. I found that a few models such as

PoolFormer
Hubert
MPnet
wav2vec

already have their own configurations in this file.

thank you @hazrulakmal , I updated the list!

@fxmarty Re-open this, please 🤗

@fxmarty working on FLAVA

Hi,
I am really lost in something which relates to exporting DeBERTa in ONNX. I have the following code -
tokenizer = AutoTokenizer.from_pretrained("microsoft/deberta-base")
model = ORTModelForTokenClassification.from_pretrained("microsoft/deberta-base", export=True)
pipe = pipeline("fill-mask", model=model, tokenizer=tokenizer)
pipe("I am a [MASK] engineer. I have worked in JS, CPP, Java, J2ME, and Python. I know Oracle and MySQL")

Results into this error -
---------------------------------------------------------------------------
InvalidArgument                           Traceback (most recent call last)
Cell In[8], line 1
----> 1 pipe("I am a [MASK] engineer. I have worked in JS, CPP, Java, J2ME, and Python. I know Oracle and MySQL")

File ~/skill_extraction/skill_extraction/lib/python3.10/site-packages/transformers/pipelines/fill_mask.py:239, in FillMaskPipeline.__call__(self, inputs, *args, **kwargs)
    217 def __call__(self, inputs, *args, **kwargs):
    218     """
    219     Fill the masked token in the text(s) given as inputs.
    220
   (...)
    237         - **token** (`str`) -- The predicted token (to replace the masked one).
    238     """
--> 239     outputs = super().__call__(inputs, **kwargs)
    240     if isinstance(inputs, list) and len(inputs) == 1:
    241         return outputs[0]

File ~/skill_extraction/skill_extraction/lib/python3.10/site-packages/transformers/pipelines/base.py:1109, in Pipeline.__call__(self, inputs, num_workers, batch_size, *args, **kwargs)
   1101     return next(
   1102         iter(
   1103             self.get_iterator(
   (...)
   1106         )
   1107     )
   1108 else:
-> 1109     return self.run_single(inputs, preprocess_params, forward_params, postprocess_params)

File ~/skill_extraction/skill_extraction/lib/python3.10/site-packages/transformers/pipelines/base.py:1116, in Pipeline.run_single(self, inputs, preprocess_params, forward_params, postprocess_params)
   1114 def run_single(self, inputs, preprocess_params, forward_params, postprocess_params):
   1115     model_inputs = self.preprocess(inputs, **preprocess_params)
-> 1116     model_outputs = self.forward(model_inputs, **forward_params)
   1117     outputs = self.postprocess(model_outputs, **postprocess_params)
   1118     return outputs

File ~/skill_extraction/skill_extraction/lib/python3.10/site-packages/transformers/pipelines/base.py:1015, in Pipeline.forward(self, model_inputs, **forward_params)
   1013     with inference_context():
   1014         model_inputs = self._ensure_tensor_on_device(model_inputs, device=self.device)
-> 1015         model_outputs = self._forward(model_inputs, **forward_params)
   1016         model_outputs = self._ensure_tensor_on_device(model_outputs, device=torch.device("cpu"))
   1017 else:

File ~/skill_extraction/skill_extraction/lib/python3.10/site-packages/transformers/pipelines/fill_mask.py:101, in FillMaskPipeline._forward(self, model_inputs)
    100 def _forward(self, model_inputs):
--> 101     model_outputs = self.model(**model_inputs)
    102     model_outputs["input_ids"] = model_inputs["input_ids"]
    103     return model_outputs

File ~/skill_extraction/skill_extraction/lib/python3.10/site-packages/optimum/modeling_base.py:85, in OptimizedModel.__call__(self, *args, **kwargs)
     84 def __call__(self, *args, **kwargs):
---> 85     return self.forward(*args, **kwargs)

File ~/skill_extraction/skill_extraction/lib/python3.10/site-packages/optimum/onnxruntime/modeling_ort.py:1363, in ORTModelForTokenClassification.forward(self, input_ids, attention_mask, token_type_ids, **kwargs)
   1360     onnx_inputs["token_type_ids"] = token_type_ids
   1362 # run inference
-> 1363 outputs = self.model.run(None, onnx_inputs)
   1364 logits = outputs[self.output_names["logits"]]
   1366 if use_torch:

File ~/skill_extraction/skill_extraction/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:200, in Session.run(self, output_names, input_feed, run_options)
    198     output_names = [output.name for output in self._outputs_meta]
    199 try:
--> 200     return self._sess.run(output_names, input_feed, run_options)
    201 except C.EPFail as err:
    202     if self._enable_fallback:

InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid Feed Input Name:token_type_ids

Whatever I do I can't get rid of this error. I am a noob with HF. I have posted the issue in both forum and SO. But no replies. Here is my last hope. What am I doing wrong? I read the PR that adds the V2 support as well. I can't figure out what is wrong. Even when I use return_token_type_ids=False in my tokenizer call it does not solve the problem. My transformer version is 4.27.0 and my optimum version is also the latest.

@rcshubhadeep I moved your issue to https://github.com/huggingface/optimum/issues/968

hi ,  is optimum supports converting Llama (alpaca-lora) to onnx ? It would be great if i get some insights in this

hi , is optimum supports converting Llama (alpaca-lora) to onnx ? It would be great if i get some insights in this

Yes, this is supported and was introduced in https://github.com/huggingface/optimum/pull/975. You'll need to have Optimum v1.8 to do it.

Hello! I am sorry, can you please help in understanding the Registering the ONNX configuration in the TasksManager part of the guide?

The TasksManager allows to map model classes to export configuratons, here ONNX ones.
Registering your ONNX config will make it possible for you to use it with the CLI and everything else.
Are you doing a PR that will be merged on optimum?
If so, go to the optimum/exporters/tasks.py file and add an entry in the _SUPPORTED_MODEL_TYPE class attribute:
_SUPPORTED_MODEL_TYPE = {
    ....,
    "custom": supported_task_mapping("text-classification", ...., onnx="CustomOnnxConfig")
}

But if you are not doing a PR that will be merged in optimum, and want to dynamically register your class in your own library you can create a registering method:
register_for_onnx = TasksManager.create_register("onnx")

@register_for_onnx("model_type_here", "text-classification", ...)
class CustomOnnxConfig(TextEncoderOnnxConfig):
...

@michaelbenayoun
I am sorry, I know it's very stupid of me, but I still do not get it how to convert the model:
So
register_for_onnx = TasksManager.create_register("onnx")

@register_for_onnx("bert", "text-classification")
class CustomOnnxConfig(TextEncoderOnnxConfig):
    # Specifies how to normalize the BertConfig, this is needed to access common attributes
    # during dummy input generation.
    NORMALIZED_CONFIG_CLASS = NormalizedTextConfig
    # Sets the absolute tolerance to when validating the exported ONNX model against the
    # reference model.
    ATOL_FOR_VALIDATION = 1e-4

    def inputs(self) -> Dict[str, Dict[int, str]]:
        if self.task == "multiple-choice":
            dynamic_axis = {0: "batch_size", 1: "num_choices", 2: "sequence_length"}
        else:
            dynamic_axis = {0: "batch_size", 1: "sequence_length"}
        return {
            "input_ids": dynamic_axis,
            "attention_mask": dynamic_axis,
            "capitalization_ids": dynamic_axis,
            "token_type_ids": dynamic_axis,
        }

base_model = CustomBertForTokenClassification.from_pretrained("my-checkpoint")

onnx_path = Path("model.onnx")
onnx_config_constructor = TasksManager.get_exporter_config_constructor("onnx", base_model)
onnx_config = onnx_config_constructor(base_model.config)

onnx_inputs, onnx_outputs = export(base_model, onnx_config, onnx_path, onnx_config.DEFAULT_ONNX_OPSET)

I modified BERT architecture to accept capitalization information (does word contain capital letter or not) (it improved my metrics on NER task after further pretraining this new model on a large corpus of text)
class CustomBertForTokenClassification(CustomBertPreTrainedModel):

    _keys_to_ignore_on_load_unexpected = [r"pooler"]

    def __init__(self, config):
        super().__init__(config)
        self.num_labels = config.num_labels

        self.bert = CustomBertModel(config, add_pooling_layer=False)
        classifier_dropout = (
            config.classifier_dropout if config.classifier_dropout is not None else config.hidden_dropout_prob
        )
        self.dropout = nn.Dropout(classifier_dropout)
        self.classifier = nn.Linear(config.hidden_size, config.num_labels)

        # Initialize weights and apply final processing
        self.post_init()

    @add_start_docstrings_to_model_forward(BERT_INPUTS_DOCSTRING.format("batch_size, sequence_length"))
    @add_code_sample_docstrings(
        checkpoint=_CHECKPOINT_FOR_TOKEN_CLASSIFICATION,
        output_type=TokenClassifierOutput,
        config_class=_CONFIG_FOR_DOC,
        expected_output=_TOKEN_CLASS_EXPECTED_OUTPUT,
        expected_loss=_TOKEN_CLASS_EXPECTED_LOSS,
    )
    def forward(
        self,
        input_ids: Optional[torch.Tensor] = None,
        capitalization_ids: Optional[torch.Tensor] = None,
        attention_mask: Optional[torch.Tensor] = None,
        token_type_ids: Optional[torch.Tensor] = None,
        position_ids: Optional[torch.Tensor] = None,
        head_mask: Optional[torch.Tensor] = None,
        inputs_embeds: Optional[torch.Tensor] = None,
        labels: Optional[torch.Tensor] = None,
        output_attentions: Optional[bool] = None,
        output_hidden_states: Optional[bool] = None,
        return_dict: Optional[bool] = None,
    ) -> Union[Tuple[torch.Tensor], TokenClassifierOutput]:
        r"""
        labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*):
            Labels for computing the token classification loss. Indices should be in `[0, ..., config.num_labels - 1]`.
        """
        return_dict = return_dict if return_dict is not None else self.config.use_return_dict

        outputs = self.bert(
            input_ids,
            capitalization_ids=capitalization_ids,
            attention_mask=attention_mask,
            token_type_ids=token_type_ids,
            position_ids=position_ids,
            head_mask=head_mask,
            inputs_embeds=inputs_embeds,
            output_attentions=output_attentions,
            output_hidden_states=output_hidden_states,
            return_dict=return_dict,
        )

        sequence_output = outputs[0]

        sequence_output = self.dropout(sequence_output)
        logits = self.classifier(sequence_output)

        loss = None
        if labels is not None:
            loss_fct = CrossEntropyLoss()
            loss = loss_fct(logits.view(-1, self.num_labels), labels.view(-1))

        if not return_dict:
            output = (logits,) + outputs[2:]
            return ((loss,) + output) if loss is not None else output

        return TokenClassifierOutput(
            loss=loss,
            logits=logits,
            hidden_states=outputs.hidden_states,
            attentions=outputs.attentions,
        )

Now I am trying to export this model to ONNX so I can use optimum optimization (normal BERT got speed up ~2.5 on CPU, so I want to achieve same with this custom architecture)

Alright, could you open a PR for your issue please?
We will try to help you there.

Thank you for spending time on me! I think PR will be a difficult thing to do, since I am not that proficient and do not think many people will want to use my architecture anyway.
Maybe you can advice how to do it code just for my library?
base_model = CustomBertForTokenClassification.from_pretrained("my-checkpoint")
base_model.config returns BertConfig, which I think I need to overwrite with the custom config I created in the previous step...

Sorry I meant a separete issue...

Thank you a lot, I'll delete my comments here since they are unrelated to the discussion. I asked on discussion forum

I can work on CvT, if its open

Hi @rishabbala , sounds good, let us know if you encounter any help! A good reference is https://huggingface.co/docs/optimum/main/en/exporters/onnx/usage_guides/contribute

According to the above list, export of BLOOM models to ONNX is already supported, right?
Is export to ONNX already supposed to work for base models that have been finetuned with PEFT / LoRA?
Using the bigscience/bloom-560m base model and finetuning with PEFT / LoRA, I was able to perform inference after exporting to ONNX, but the model predictions are degraded 🤔 Details: https://github.com/huggingface/peft/issues/670

Hello, I would like to add onnx exporter support for GPT-NeoX.

Hello, I would like to add onnx exporter support for Funnel Transformer.

Hi @sidistic! Feel free to open a PR here and we'll help you if there is any issue :slightly_smiling_face:
This guide may be useful: https://huggingface.co/docs/optimum/main/en/exporters/onnx/usage_guides/contribute

Hello @regisss! I have opened a PR. This is my first ever PR on an open source project so looking forward to hearing your advice and learning from you.

Hello, is anyone working to implement this? If not then I might look into it
raise NotImplementedError( NotImplementedError: Tried to use ORTOptimizer for the model type mpnet, but it is not available yet.

Hi, I'm trying to export ChatGLM2 & Qwen models to onnx using hf optimum.

ChatGLM2: model-card-> https://huggingface.co/THUDM/chatglm2-6b
Qwen: model-card-> https://huggingface.co/Qwen/Qwen-7B-Chat

I'm using this code to export chatglm2: https://gist.github.com/manishghop/9be5aee6ed3d7551c751cc5d9f7eb8c3
While running the onnx export I faced: [UnsupportedOperatorError: Exporting the operator 'aten::scaled_dot_product_attention' to ONNX opset version 14 is not supported.](https://github.com/pytorch/pytorch/issues/97262#top) .
Fixed it by adding this code: https://github.com/pytorch/pytorch/issues/97262#issuecomment-1487141914 from the issue PR.

My question is, I do get 2 files(model.onnx & model.onnx_data, but it fails the onnx export validation stage. How do I check if the onnx model works?
Also for exporting qwen model: https://huggingface.co/Qwen/Qwen-7B-Chat, should I just make changes to the model_id to Qwen/Qwen-7B-Chat & hoping it should run the onnx export?

Thanks in advance

Hey all, Wanted to see if I could pick up doing the Canine implementation. I saw @RaghavPrabhakar66 was doing some work on it in the previous issue thread, but didn't see an official PR on it.

@mattsthilaire For sure, feel free to open a PR!

Could you please add support for Florence2 model?

Hey @ozancaglayan, unfortunately no. Last I left it off on my local branch, I was getting shape mismatches on the upsampling part. Tried to troubleshoot using Netron  to no avail. Happy to pass if off to you if you wanted to take a pass at it or try pairing up on it to see where we get.

@ozancaglayan Hi, I have opened a draft PR with some of the work done. When I try run model for QA tasks with ORTModelForQuestionAnswering, I get following error:
Some weights of CanineForQuestionAnswering were not initialized from the model checkpoint at google/canine-s and are newly initialized: ['qa_outputs.bias', 'qa_outputs.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Framework not specified. Using pt to export the model.
Some weights of CanineForQuestionAnswering were not initialized from the model checkpoint at google/canine-s and are newly initialized: ['qa_outputs.bias', 'qa_outputs.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Using the export variant default. Available variants are:
    - default: The default ONNX variant.

***** Exporting submodel 1/1: CanineForQuestionAnswering *****
Using framework PyTorch: 2.4.0+cu121
Overriding 1 configuration item(s)
        - use_cache -> False
/home/raghav/.micromamba/envs/optimum/lib/python3.10/site-packages/transformers/models/canine/modeling_canine.py:604: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  chunk_end = min(from_seq_length, chunk_start + self.attend_from_chunk_width)
/home/raghav/.micromamba/envs/optimum/lib/python3.10/site-packages/transformers/models/canine/modeling_canine.py:612: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  chunk_end = min(to_seq_length, chunk_start + self.attend_to_chunk_width)
/home/raghav/.micromamba/envs/optimum/lib/python3.10/site-packages/transformers/models/canine/modeling_canine.py:1073: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
  remainder_length = torch.fmod(torch.tensor(char_seq_length), torch.tensor(rate)).item()
/home/raghav/.micromamba/envs/optimum/lib/python3.10/site-packages/transformers/models/canine/modeling_canine.py:1073: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
  remainder_length = torch.fmod(torch.tensor(char_seq_length), torch.tensor(rate)).item()
/home/raghav/.micromamba/envs/optimum/lib/python3.10/site-packages/transformers/models/canine/modeling_canine.py:1073: TracerWarning: Converting a tensor to a Python number might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  remainder_length = torch.fmod(torch.tensor(char_seq_length), torch.tensor(rate)).item()
/home/raghav/.micromamba/envs/optimum/lib/python3.10/site-packages/torch/onnx/_internal/jit_utils.py:314: UserWarning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied. (Triggered internally at ../torch/csrc/jit/passes/onnx/constant_fold.cpp:179.)
  _C._jit_pass_onnx_node_shape_type_inference(node, params_dict, opset_version)
/home/raghav/.micromamba/envs/optimum/lib/python3.10/site-packages/torch/onnx/utils.py:739: UserWarning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied. (Triggered internally at ../torch/csrc/jit/passes/onnx/constant_fold.cpp:179.)
  _C._jit_pass_onnx_graph_shape_type_inference(
/home/raghav/.micromamba/envs/optimum/lib/python3.10/site-packages/torch/onnx/utils.py:1244: UserWarning: Constant folding - Only steps=1 can be constant folded for opset >= 10 onnx::Slice op. Constant folding not applied. (Triggered internally at ../torch/csrc/jit/passes/onnx/constant_fold.cpp:179.)
  _C._jit_pass_onnx_graph_shape_type_inference(
Traceback (most recent call last):
  File "/home/raghav/Dev/huggingface/optimum/test.py", line 25, in <module>
DEBUG: Input shapes: {'input_ids': torch.Size([1, 24]), 'token_type_ids': torch.Size([1, 24]), 'attention_mask': torch.Size([1, 24])}
DEBUG: PT Model Output: QuestionAnsweringModelOutput(loss=None, start_logits=tensor([[ 0.1626,  0.2406,  0.2992,  0.2548,  0.2493,  0.1242, -0.0853,  0.0104,
    model = ORTModelForQuestionAnswering.from_pretrained(
  File "/home/raghav/Dev/huggingface/optimum/optimum/onnxruntime/modeling_ort.py", line 738, in from_pretrained
    return super().from_pretrained(
  File "/home/raghav/Dev/huggingface/optimum/optimum/modeling_base.py", line 424, in from_pretrained
    return from_pretrained_method(
  File "/home/raghav/Dev/huggingface/optimum/optimum/onnxruntime/modeling_ort.py", line 599, in _from_transformers
    return cls._export(
  File "/home/raghav/Dev/huggingface/optimum/optimum/onnxruntime/modeling_ort.py", line 668, in _export
    return cls._from_pretrained(
  File "/home/raghav/Dev/huggingface/optimum/optimum/onnxruntime/modeling_ort.py", line 554, in _from_pretrained
    model = ORTModel.load_model(
  File "/home/raghav/Dev/huggingface/optimum/optimum/onnxruntime/modeling_ort.py", line 397, in load_model
    return ort.InferenceSession(
  File "/home/raghav/.micromamba/envs/optimum/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 419, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "/home/raghav/.micromamba/envs/optimum/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 483, in _create_inference_session
    sess.initialize_session(providers, provider_options, disabled_optimizers)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Node (/canine/final_char_encoder/layer.0/attention/self/MatMul) Op (MatMul) [ShapeInferenceError] Incompatible dimensions

Code to reproduce it:
from transformers import (
    AutoConfig,
    AutoModelForQuestionAnswering,
    AutoTokenizer,
)

from optimum.onnxruntime import ORTModelForQuestionAnswering

model_name = "google/canine-s"

config = AutoConfig.from_pretrained(model_name)

tokenizer = AutoTokenizer.from_pretrained(model_name)

dummy_inputs = tokenizer("This is a sample input", return_tensors="pt")
input_shapes = {k: v.shape for k, v in dummy_inputs.items()}

print(f"DEBUG: Input shapes: {input_shapes}")

pt_model = AutoModelForQuestionAnswering.from_pretrained(model_name)
outputs = pt_model(**dummy_inputs)
print(f"DEBUG: PT Model Output: {outputs}")

model = ORTModelForQuestionAnswering.from_pretrained(
    model_name,
    export=True,
)
outputs = model(**dummy_inputs)
print(f"DEBUG: ONNX Model Output: {outputs}")


Exit Gracefully on CTRL^C

Hook system even and exit gracefully.

Implemented exit gracefully for build and main. Build will finish writing the file. Pull etc will stop without printing a stack trace.


Upgrade shallow-compare to 1.2.2 for React 16 compatibility

The Problem
shallow-compare 1.2.1 has a peer dependency on React 15.  It is recently updated in 1.2.2 https://www.npmjs.com/package/shallow-compare which removes this dependency.
Currently RRF has an exact dependency on shallow-compare 1.2.1
Steps to Reproduce
npm install gives a warning:  peer invalid<EMAIL_ADDRESS>required by<EMAIL_ADDRESS>

@davidkpiano would love a new release with this change.  npm shrinkwrap breaks with any peerDependency warnings.  Thanks!

Will do first thing tomorrow.


fix(versions): update Activiti/activiti-cloud-full-chart-security versions

UpdateBot pushed helm dependency: activiti-cloud-full-example to: 7.1.104

UpdateBot commands:
updatebot push-version --kind helm activiti-cloud-full-example 7.1.104


Don't reverse the order of the dropdown options

is it possible not to reverse the order of the options whenver the options appears at the top of the dropdown select?

hi @finalight ,
Refer this example:
      <Dropdown
        ...
        flatListProps={{ inverted: false }}
      />

@hoaphantn7604 I tried and it's not working unfortunately

hi @Audrey-Ann,
Can you send a code example for me to support?

hi @Audrey-Ann ,
dropdownPosition is "bottom" then  flatListProps={{ inverted: true }}.
dropdownPosition is "top" then  flatListProps={{ inverted: false }}.

@hoaphantn7604
But when using dropdownPosition="top" and flatListProps={{ inverted: false }} with a large dataset (states for example) then it starts from the bottom (Wyoming). How can it work so that it starts at the top (Alabama)?
If I preselect Alabama and use the auto scroll feature there's a 1 second delay before it scrolls to the top.
Is there a better way?

hi @all,
Please install version 2.7.0. Use prop inverted set is "false".
   <Dropdown
      {...}
      inverted={false}
   />


Turning causes positional tracking of hmd to not work

Everything works correctly right when I open steamvr. I am using riftcat and freepie. Right away everything works great the controllers are tracked and I can move fine along the z,x, and y axis. The problems occur when I turn my head. The z and x axis seem to be locked to where I was first looking, so for example, if I turn right and walk forward I walk left because that's the direction I would be moving if I was looking forward still. Another example would be if I turn around completely everything is swapped, forward is backwards and right is left and so on. This also causes my controllers to be moving very awkwardly in any way I am facing other then forward. I hope this is a simple fix. Thank you for your time.

I learned what I did wrong. First off my phones gyroscope was drifting very badly so my controllers would get off centered fast then too "fix it" I would recalibrate riftcat and the controllers which caused them too be lopsided and backwards.