Chaincode install path is hard coded

The default chaincode installpath value is set to /opt/gopath/bin in the core.yaml file.  If one decides to change the installpath parameter to something other than the default value, the following error occurs when deploying chaincode:
12:14:29.626 [container] start -> ERRO 04b start-could not start container API error (500): Cannot start container dev-jdoe-3a46d4d629d8c1f931bb29c1b52ca9698cba5d75487771882068975b15f0440fb73bab315f5dad72eaaaeaead5ea6a211ac013cfb8d55c64547d348d51504e7d: [8] System error: exec: /opt/gopath/bin/3a46d4d629d8c1f931bb29c1b52ca9698cba5d75487771882068975b15f0440fb73bab315f5dad72eaaaeaead5ea6a211ac013cfb8d55c64547d348d51504e7d: stat /opt/gopath/bin/3a46d4d629d8c1f931bb29c1b52ca9698cba5d75487771882068975b15f0440fb73bab315f5dad72eaaaeaead5ea6a211ac013cfb8d55c64547d348d51504e7d : no such file or directory
When you change your default chaincode installpath within core.yaml, this also assumes that one has included the path within the Docker image used for chaincode operations.  When I built my base Docker image for chaincode use, I updated my PATH environment variable to include the chaincode installpath (/go/bin) set in my core.yaml file.
I took a look at the fabric core file core/chaincode/chaincode_support.go and noticed that on line 51 the constant chaincodeInstallPathDefault is set to /opt/gopath/bin.  Within the NewChaincodeSupport method on line 116 of chaincode_support.go, the s.chaincodeInstallPath is assigned the constant chaincodeInstallPathDefault (/opt/gopath/bin) rather than using the installpath value within the core.yaml file.
Just as a test, I went ahead and rebuilt the peer binary after updating  the chaindeInstallPathDefault constant in chaincode_support.go to match the installpath value used in my core.yaml file.  I then deployed chaincode and the error that I was experiencing previously did not occur.
Perhaps I'm missing something here... very possible. :-)

@muralisrini can you comment on this issue?

@harrijk : the installpath treatment is definitely incorrect. However I am not sure its the whole story. The chaincode is installed via "go install" which will put the chaincode in GOPATH in the image. By default this is /opt/gopath/bin. Did you copy the chaincode binary in the image to /go/bin in addition to " ...I updated my PATH environment variable to include the chaincode installpath (/go/bin) set in my core.yaml file." ?
In any case, given we don't expose setting GOPATH, I think we should just remove the "installpath" from core.yaml. As the fabric is fully managing the chaincode, I cannot think of a good reason why user may want it to run elsewhere. But perhaps there is one ?

@muralisrini, since the installpath parameter is being ignored in core.yaml, the ultimate placement of the chaincode is made in core/chaincode/chaincode_support.go outlined in my original post for this issue.

@harrijk : Right, that change - if I understood correctly - will launch the container from /go/bin/3a46d4d629d8c1f931bb29c1b52ca9698cba5d75487771882068975b15f0440fb73bab315f5dad72eaaaeaead5ea6a211ac013cfb8d55c64547d348d51504e7d. If GOPATH is /opt/gopath/bin, I'd expect to find the exe under /opt/gopath/bin. Just wondering how the exe found its way into /go/bin. I could easily be missing something ...

No worries.  If I didn't make a change to the core/chaincode/chaincode_support.go file on line 51 from /opt/gopath/bin to /go/bin, the chaincode exe would fail as my docker image listed in the chaincode: -> golang: -> Dockerfile parameter (FROM statement) of my core.yaml file does not contain a /opt/gopath/bin directory; it was setup with /go/bin.  Also, the GOPATH variable setup in that docker image is set to /go and not /opt/gopath/bin.
However, if I re-compile the peer exe after changing line 51 of core/chaincode/chaincode_support.go from /opt/gopath/bin to /go/bin then the chaincode exe runs without error.
I hope this helps.
Cheers.

Thanks. I think I see the problem... in my env GOPATH resolves to /opt/gopath in the image

And from a chaincode itself

There were some changes to the paths recently when we moved to hyperledger. Wonder if you are on the latest ?

Yeah, I switched over before I wrote up this issue.  Being curious, I was just wondering why I couldn't change the installpath without going through all of the above changes.  I have switched back to using /opt/gopath/bin as my install path in my Docker images.

Ok, that clears it up for me.
So given the fabric completely controls the generation of the exe and installation path exposing installpath does not make a lot of sense... would you agree ?

I agree.

hi, please take a look at #1103. Similar reason, but I guess we should allow people have the flexibility to control the installpath, as sometime they will have their own built image for testing or hacking.
So I think we should keep the installPath option in the core.yaml, while fix the code to let read the value before using the default one.
How do u think? Thanks1

Hi.  I am open to keeping the installpath option as long as it is read in and used as opposed to its current state.

@yeasy : I applied your patch and modified installpath to "/go/bin". When I deployed, it failed with this error
03:37:38.782 [container] start -> ERRO 040 start-could not start container API error (500): Container command not found or does not exist.
This is as expected as the exe is placed in $GOPATH/bin (which is /opt/gopath/bin) in the image. So just making this change (and I see you agree on this) is not enough.
A better approach, in my opinion, would be to remove install path completely from the picture..ie, both in core.yaml AND in chaincode_support.go and have $GOPATH/bin in PATH . Basically make the code independent of install path.
So this change works
//args = []string{chaincodeSupport.chaincodeInstallPath + cID.Name, fmt.Sprintf("-peer.address=%s", chaincodeSupport.peerAddress)}
args = []string{cID.Name, fmt.Sprintf("-peer.address=%s", chaincodeSupport.peerAddress)}
The only dependency is on GOPATH/bin being in PATH (which it is in the default image).
Seems simple and clean... what do you think ?

@murallsini IMHO, seems you need to update the GOPATH in your image?
InstallPath cannot be arbitrary, unless we put the full path to the chaincode exe.
As the exe should be started directly, it must be placed under the sys path, otherwise, it cannot be found.

@yeasy : correct. I found it it is in my path in my (default) image. The  point is, if you build your own image its easier to add GOPATH/bin to PATH in the image (in addition to the other changes you make there) than to have to change in the yaml too. But perhaps I'm missing something?

@muralisrini Yes, I agree with u for image and golang experts it might be easier (Actually I do believe we should put GOPATH/bin on PATH anytime after setup golang).
While we might keep the flexibility provided by the yaml way for customization usage. Think such scenario, we keep different version of chaincode, and want to testing... Then we can just change the installation path, instead of changing and rebuilding the image. The other consideration is for people who may be not that familiar with golang setups.

@yeasy :
Think such scenario, we keep different version of chaincode, and want to testing... Then we can just change the installation path, instead of changing and rebuilding the image.
Note that you'll have to make sure to copy the chaincode to the right install path... While you are at it you might as well update PATH to point to the new folder in the image... and you won't have to make a change in the yaml too.
I think removing the installpath is simple and will be enough 90% of the time. But I can see some benefit in calling it out explicitly (users don't want to launch  the wrong exe by mistake). If you feel strongly about having the installpath in the yaml, there's no harm in taking in your PR as is.

@muralisrini, thanks for your reply, and let's try to make hyperledger more mature and easier for usage.

Can you add some doc to installpath in core.yaml... something like this
  # typically install path should not be modified. If modified, user must ensure the chaincode
  # executable is placed in the directory specifed by installpath in the image
  installpath: /opt/gopath/bin/

@muralisrini Good idea, I will add it, thanks!

@muralisrini, adding the comment to the core.yaml file still does not resolve this issue.  Even if someone was to change the installpath and update their docker images to reflect this new path, the code in core/chaincode/chaincode_support.go does not read in the core.yaml installpath value, and as a result all chaincode will be installed at the location specified supplied by chaincodeInstallPathDefault in core/chaincode/chaincode_support.go at line 51, which is currently set to /opt/gopath/bin.  Perhaps I'm missing something. :)

@harrijk : the comment goes with PR  https://github.com/hyperledger/fabric/pull/1103 ... when that is merged we should good ?

Ha.  I knew I was missing something.  :)  Thanks for keeping me straight.  Yes, we should be good after the merge of the #1103 fix.
Cheers.


changeFeedQueryUpToAvailableNow - DRAFT

API change check
APIView has identified API level changes in this PR and created following API reviews.
com.azure:azure-cosmos

duplicate of https://github.com/Azure/azure-sdk-for-java/pull/42160


Remove Data will be available line

Merge this when the spatial transcriptomics data is available.

These data are on the portal now, so I'm going to go ahead and get this merged.


租的gpu,Running on local URL:  http://<IP_ADDRESS>:7860  To create a public link, set share=True in launch().  浏览器上访问不了？？？

(chatglm20230401) root@autodl-container-e82d11963c-10ece0d7:~/autodl-tmp/chatglm/langchain-ChatGLM-20230418# python3.9 webui.py
Explicitly passing a revision is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a revision is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision.
Explicitly passing a revision is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:10<00:00,  1.28s/it]
No sentence-transformers model found with name /root/.cache/torch/sentence_transformers/GanymedeNil_text2vec-large-chinese. Creating a new one with MEAN pooling.
No sentence-transformers model found with name /root/.cache/torch/sentence_transformers/GanymedeNil_text2vec-large-chinese. Creating a new one with MEAN pooling.
Running on local URL:  http://<IP_ADDRESS>:7860
To create a public link, set share=True in launch().

Share=Ture, server_port=6006 改用 autodl 自定义服务访问
cristianohello @.***>于2023年4月18日 周二11:40写道：

(chatglm20230401) @.***:~/autodl-tmp/chatglm/langchain-ChatGLM-20230418#
python3.9 webui.py
Explicitly passing a revision is encouraged when loading a model with
custom code to ensure no malicious code has been contributed in a newer
revision.
Explicitly passing a revision is encouraged when loading a configuration
with custom code to ensure no malicious code has been contributed in a
newer revision.
Explicitly passing a revision is encouraged when loading a model with
custom code to ensure no malicious code has been contributed in a newer
revision.
Loading checkpoint shards:
100%|██████████████████████████████████████████████████████████████████████████████████████████|
8/8 [00:10<00:00, 1.28s/it]
No sentence-transformers model found with name
/root/.cache/torch/sentence_transformers/GanymedeNil_text2vec-large-chinese.
Creating a new one with MEAN pooling.
No sentence-transformers model found with name
/root/.cache/torch/sentence_transformers/GanymedeNil_text2vec-large-chinese.
Creating a new one with MEAN pooling.
Running on local URL: http://<IP_ADDRESS>:7860
To create a public link, set share=True in launch().
—
Reply to this email directly, view it on GitHub
https://github.com/imClumsyPanda/langchain-ChatGLM/issues/122, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/ABLH5ESRWHCXONV5BGGMJQ3XBYEKXANCNFSM6AAAAAAXCAYFIA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

是不是要用服务器的IP加端口，因为<IP_ADDRESS>是本机，指的是服务器，大概应该也许可能是这样


CAMEL-14476: Able to extend workday component for futher entities support

@favalos could you please to also have a look at this

@dmvolod this breaks the build on master, do you mind checking

sorry, @davsclaus working to fix and regenerate this


Configure email alerts in alertmanager

Can someone please give an example on how to send email alerts in alertmanager ?

Check this answer: https://github.com/bosh-prometheus/prometheus-boshrelease/issues/208


Basket middleware takes a long time

Issue Summary
We profile oscar's home page loading with a profiling tool PyLive. And we find in src/oscar/apps/basket/middleware.py, BasketMiddleware.call unexpectedly takes a long time, summing up to over 90% of the total time. And things are the same for product page and even search result page.
So we wonder if there is anything slow down in this basket middleware.
Steps to Reproduce

Profile functions in oscar
Load the main page and get the profiling result, you will see BasketMiddleware.call takes up a lot of percentage.

Technical details

Python version: 3.6.9
Django version: 2.2.12
Oscar version: 2.0

Likely duplicate of #1329


Fix change detection issues on our search result page

is this still a problem?

@Xymmer I think this issue was fixed. Could you provide more details?

After some investigation this is fixed.


Add support for native Sketch plugin updates

Resolves #9
Further reading: https://docs.sketchpacks.com/developers/publishing/providing-plugin-updates.html

Thanks for the note Adam. Its done now!


✨[feature Request]: hover

Is there an existing issue for this?

[X] I have searched the existing issues

Feature Description
hovering the icons which looks better
Use Case
In a web application, hovering over icons can provide users with immediate feedbac
Benefits
Improved User Experience
Add ScreenShots

Priority
High
Record

[X] I have read the Contributing Guidelines
[X] I'm a GSSOC'24EXTD contributor
[X] I have starred the repository

@Anjaliavv51 Assign me this issue. I'd love to work on this.

@abhi200446 it is already done.


Fix icon in WPAppbox

The app icon is not loading, check if we have the correct URL.

It's now working properly. Must have been a problem of WPAppbox itself.
https://androidayuda.com/2018/04/05/aplicaciones-netflix-play-store/?server=https://localhost:3000


README: fix example port

Several places document how to do something and then follow with an example. However, the port in the documentation and following examples aren't consistent. Looks like convention is port 8027 for SSH and 8028 for HTTP, so I've adjusted them to be consistent with port 8028 as they are showing how to use the HTTP interface in these sections.

Thanks for taking time to fix this @JohnDMcMaster ! Much appreciated!


Editing entry on completed list fills "finish date" with current date

When editing any entry on the completed list which has no finish date, the finish date gets automatically filled with the current date.
Also, when editing any entry on the "Reading" or "Watching" lists which have no start date, that also gets filled in with the current date.

I'm going to take a look at this. Right now, I have AniDroid automatically filling in the dates when you edit the list item (end date for "completed" and start date for "current"). I'm going to add a setting that disables this functionality (so that the fields will remain blank when you set those statuses).
HOWEVER: I am fairly certain that AniList itself is assigning the end date automatically when adding a list item with the "completed" status (even if it appears blank in AniDroid). I will have to research this a bit more.

Fixed in 05af923e081024f71c71bda8d39e126e57885970


Improve: instance group updates

When updating the instance group template, it should not result in downtime of Nomad servers or clients. See update_policy attribute in Terraform.

Fixed in a2e9f6c10fd776ff53e6a9069eca3938c7a4b3fb


Use Azure DevOps REST API to Delete organizations / accounts?

Is it possible to Delete an organization / account within Azure DevOps using the REST API? You can delete other entities, but the documentation doesn't show a DELETE command for Accounts / Organizations.
Is this something that's possible?
I have a scenario where I need to automate the management of Azure DevOps Organizations across multiple users and Azure Subscriptions. I've also posted this question here: https://stackoverflow.com/questions/54793899/use-azure-devops-rest-api-to-delete-organizations-accounts

@crpietschmann Unfortunately we don't support deleting organizations via our REST APIs. There currently isn't a way to automate this operation.


i18n(ko-KR): create media.mdx

Description (required)

create media.mdx

Related issues & labels (optional)

Closes #
Suggested label:

Lunaria Status Overview
🌕 This pull request will trigger status changes.

Learn more
By default, every PR changing files present in the Lunaria configuration's files property will be considered and trigger status changes accordingly.
You can change this by adding one of the keywords present in the ignoreKeywords property in your Lunaria configuration file in the PR's title (ignoring all files) or by including a tracker directive in the merged commit's description.

Tracked Files

Locale
File
Note

ko
guides/media.mdx
Localization added, will be marked as complete.

Warnings reference

Icon
Description

🔄
The source for this localization has been updated since the creation of this pull request, make sure all changes in the source have been applied.


File does not work in subfolders.

Hello, currently it only works if you put it in the htdocs folder, is there any way to work in subfolders?

Hi RenanPiresMattos,
I had the same. You could try this, it worked for me ...

Place all the phprouter files (so also the .htaccess file) in your subfolder
Edit the router.php file accordingly:

comment out this line
// $request_url = filter_var($_SERVER['REQUEST_URI'], FILTER_SANITIZE_URL);
replace it with
// start add
$thisPart    = explode(basename(getcwd()), $_SERVER['REQUEST_URI']);
$fromSubBase = (isset($thisPart[1])) ? $thisPart[1] : $thisPart[0];
$request_url = filter_var($fromSubBase, FILTER_SANITIZE_URL);
// end add
Hope this works for you also.

Another way is just putting a basepath var or constant inside routes.php or inside router.php:
In routes.php would be the following:
<?php
require_once __DIR__.'/router.php';

$basePath='/app';
get("$basePath/", 'views/index.php');
get("$basePath/page1", 'views/page1.php');
any("$basePath/404", 'views/404.php');

But, it would be better placed inside router.php, to prevent writing '$basePath' many times.
There are some frameworks and libs that have configuration for setting the basePath. It would be nice to have something similar here.

I am still trying to get PhpRouter to work under any subfolder... any good idea, so it will work without having to set variables or configure anything? So, a piece of code that just does the job?

Hi,
Based on your comments, I have another solution:
comment out this line
// $request_url = filter_var($_SERVER['REQUEST_URI'], FILTER_SANITIZE_URL);
then add these lines to get the subdirectory and separate it from the real request_url
$api_root=str_replace($_SERVER['DOCUMENT_ROOT'],'',getcwd());
$request_url =str_replace($api_root,'',filter_var($_SERVER['REQUEST_URI'], FILTER_SANITIZE_URL));

I've added the following line to .htaccess that is placed in the subfolder.
RewriteBase /name_subfolder/
For me it worked with XAMPP on Mac

I have not tested @cancastilho solution, but it makes sence. Use it.

My 2 cents:
I've loaded into a folder called API so I strip out anything after it.   Works well enough so long as 'api' only occurs in the url once.
$request_url = filter_var($_SERVER['REQUEST_URI'], FILTER_SANITIZE_URL); $request_url = substr($request_url, strpos($request_url, "api") + 3);

Works under the root folder


feat: Add MangoHud support to Steam images running Sway

Since Mangoapp is already running in Gamescope, I figured MangoHud makes sense for Sway.
I have no idea where to document this. The current Wolf documenation doesn't mention MangoApp anywhere.
But in order for this to be useful, users need to know to press Right Shift + F12 to make the overlay show up, and Right Shift + F11 to change the position of the overlay (shows up in the top left by default).

Thanks for the PR, I like the idea of adding mangohud!
The good news is that mangoapp already brings in mangohud so the Steam image that we have already bundles that in, here's an example of running mangohud vkgears

This works when I open up a terminal using docker, but when I try to run it from inside the session it tries to parse /tmp/mangohud.uO7uX5um
control=mangohud
fsr_steam_sharpness=5
nis_steam_sharpness=10
no_display

if I comment out the no_display option I can successfully see the overlay

There are still these 3 errors that we probably should look into
[2024-08-25 10:17:18.690] [MANGOHUD] [error] [cpu.cpp:461] Could not find cpu temp sensor location
[2024-08-25 10:17:18.732] [MANGOHUD] [error] [loader_nvctrl.cpp:39] Failed to open 64bit libXNVCtrl.so.0: libXNVCtrl.so.0: cannot open shared object file: No such file or directory
[2024-08-25 10:17:18.732] [MANGOHUD] [error] [nvctrl.cpp:45] XNVCtrl loader failed to load

but we can leave that for later.
Where does that config file come from?
https://github.com/games-on-whales/gow/blob/b3ac0cc48331de1374a567183afc594a29610e2a/images/steam/scripts/startup.sh#L33-L37
This has been a long time ago, a time when we were using Gamescope instead of Sway and Steam wasn't even starting because of network issues. I think we should try removing STEAM_USE_MANGOAPP and that default mango config and see if everything still works.
TODO
To recap, I think we should:

[ ] Remove /tmp/mangohud.uO7uX5um and test it out
[ ] Check if we can fix the mangohud reported errors
[ ] Add this to the docs "somewhere"
[ ] Test it in games!

On a side note: the MANGOHUD=1 env variable only works for Vulkan apps, for Steam games we should generally ask users to add
mangohud %command%

to the launch options.

Thanks for the PR, I like the idea of adding mangohud!
The good news is that mangoapp already brings in mangohud so the Steam image that we have already bundles that in.

Damn, mangoapp was installed in the path, but not mangohud, which is what made me think they weren't dependencies. But I was clearly wrong and it works in a fresh image, so maybe I had just messed up my container from earlier experiments.

This works when I open up a terminal using docker, but when I try to run it from inside the session it tries to parse /tmp/mangohud.uO7uX5um
control=mangohud
fsr_steam_sharpness=5
nis_steam_sharpness=10
no_display

So, that actually works out really well for us, because no_display causes it to be hidden on start but the keyboard shortcut to show still works! This means we can leave this on by default and just document somewhere that users can get a frame counter on all proton games at any time by just pressing Right Shift + F12! :smile:

There are still these 3 errors that we probably should look into
[2024-08-25 10:17:18.690] [MANGOHUD] [error] [cpu.cpp:461] Could not find cpu temp sensor location

That one worked on my machine, I think it's hardware specific

[2024-08-25 10:17:18.732] [MANGOHUD] [error] [loader_nvctrl.cpp:39] Failed to open 64bit libXNVCtrl.so.0: libXNVCtrl.so.0: cannot open shared object file: No such file or directory
[2024-08-25 10:17:18.732] [MANGOHUD] [error] [nvctrl.cpp:45] XNVCtrl loader failed to load

libXNVCtrl looks to be part of nvidia-settings package. We can install it if you really want GPU fan speed info. Though I'm okay without it so that the overlay doesn't get too massive by default.
I did get:
[2024-08-25 18:05:21.180] [MANGOHUD] [error] [overlay_params.cpp:741] Unknown option 'nis_steam_sharpness'

And I see nis_steam_sharpness in the generated mango config, but  I can't for the life of me figure out where that comes from. I couldn't find a single reference to nis_steam_sharpness or even just sharpness in your entire repo. :confused:

On a side note: the MANGOHUD=1 env variable only works for Vulkan apps, for Steam games we should generally ask users to add
mangohud %command%

to the launch options.

Yep. I forgot to write that in the PR description, but that will also need to be included in the documentation.
Speaking of which, where should we put this? Shall we move the Steam page from GOW to Wolf documentation?
Long term, we should merge those - most users won't think to look in both places.


Region allocator should not be copyable

lodovico reported this on 2016-07-08T09:21:50Z
Transfered from https://issues.dlang.org/show_bug.cgi?id=16252
CC List

alphaglosined
lodovico
stojkovic.igor

Description
struct std.experimental.allocator.building_blocks.region.Region should have its postblit disabled, at least when it has a non-empty destructor, i.e. when it frees its store.

Consider the following code:

import std.experimental.allocator;
import std.experimental.allocator.mallocator;
import std.experimental.allocator.building_blocks.region;
import std.stdio: writeln;

void foo(Alloc)(Alloc alloc)
{
    writeln("alloc disposed...");
}

void main()
{
    auto alloc = Region!Mallocator(Mallocator.instance.allocate(128));
    auto ptr = alloc.make!int(3);
    writeln("*ptr = ", *ptr) // prints 3
    foo(alloc);
    writeln("*ptr = ", *ptr) // ops... invalid memory...
    // ops... alloc destructor trying to free again the block already freed by foo
}

alphaglosined commented on 2022-12-17T13:21:18Z
Alternatively, it should support moving rather than copying.

This allows other allocator data structures like free lists to move it into their own memory (which is quite a useful trick).

stojkovic.igor commented on 2017-09-03T14:12:54Z
Would it work if we allow copying in this case but reserve first few bytes from allocated memory to count references to it so we can only free it when count says there are no more references to it. I think we would just need to increment count on postBlit and decrement it in destructor and opAssign. Something like what I did here in my implementation of SharedRegion:

https://github.com/igor84/dngin/blob/master/source/util/allocators.d#L176


SpMV benchmark

What is the data_prep_param that we need to use for SpMV?
Can you explain in detail how can we run SpMV with different matrices and what is the format we can use for the matrices?

Hello!
We didn't include BFS, SpMV, and NW benchmarks in our open-sourced version of uPIMulator because we think the official PrIM version of those benchmarks has minor bugs or utilized undisclosed functions within the UPMEM SDK. When writing our HPCA'24 paper, for SpMV, seqread is the undisclosed function within the UPMEM SDK, so we modify seqread to mram_read function. In addition, we changed the data type to int32. Then, you will be able to run SpMV benchmark using uPIMulator.
As mentioned above, we applied those two changes from the original PrIM version of SpMV, so we use the format that the original PrIM's SpMV utilizes for matrices.
If you have any further questions, don't hesitate to reach out to us.
Thank you,
Bongjoon Hyun


"Missing newline after "("" triggered with parameter comments

I've enabled the --experimental flag on ktlint 0.32, and now we are seeing this error when there are comments on parameters, e.g.:
foo(
  /*param1=*/ param1,
  /*param2=*/ param2
)

If foo() is defined in Kotlin, we can fix the error by changing the comments to parameter names.  However, if foo() is defined in Java, there seems to be no way to name the parameters without triggering this erroneous message.

I think I see the issue. I'll put up a PR soon.

Thanks for the quick fix!


Why TPU uses more memory than GPU?

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Mostly stock, very little custom
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04 - Colab PRO
TensorFlow installed from (source or binary): Pre-installed
TensorFlow version (use command below): 2.3.0
Python version: 3.x
GPU model and memory: Using TPU

I was trying to train a model with around ~700 Million Parameters, however, when I switch to TPU with the correct strategy and proper initialization (code taken from Official TensorFlow site) I find that it gives me a resource error just before training (when I compile the model):-
Epoch 1/3
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/data/ops/multi_device_iterator_ops.py:601: get_next_as_optional (from tensorflow.python.data.ops.iterator_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Iterator.get_next_as_optional()` instead.

WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/data/ops/multi_device_iterator_ops.py:601: get_next_as_optional (from tensorflow.python.data.ops.iterator_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.data.Iterator.get_next_as_optional()` instead.

---------------------------------------------------------------------------

ResourceExhaustedError                    Traceback (most recent call last)

<ipython-input-15-331b28c97632> in <module>()
      8     filepath=checkpoint_prefix, monitor='loss', save_best_only=True)
      9
---> 10 history = model.fit(X_train, y_train, epochs=3, batch_size=4, verbose=1, callbacks=[checkpoint_callback])

10 frames

/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/func_graph.py in wrapper(*args, **kwargs)
    971           except Exception as e:  # pylint:disable=broad-except
    972             if hasattr(e, "ag_error_metadata"):
--> 973               raise e.ag_error_metadata.to_exception(e)
    974             else:
    975               raise

ResourceExhaustedError: in user code:

    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:806 train_function  *
        return step_function(self, iterator)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:796 step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/tpu_strategy.py:346 run
        return self.extended.tpu_run(fn, args, kwargs, options)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/tpu_strategy.py:1095 tpu_run
        return func(args, kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/tpu_strategy.py:1162 tpu_function
        padding_spec=padding_spec)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/tpu/tpu.py:915 replicate
        padding_spec=padding_spec)[1]
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/tpu/tpu.py:1380 split_compile_and_replicate
        outputs = computation(*computation_inputs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/tpu_strategy.py:1124 replicated_fn
        result[0] = fn(*replica_args, **replica_kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:789 run_step  **
        outputs = model.train_step(data)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:757 train_step
        self.trainable_variables)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:2745 _minimize
        experimental_aggregate_gradients=False)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py:519 apply_gradients
        self._create_all_weights(var_list)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py:704 _create_all_weights
        self._create_slots(var_list)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/optimizer_v2/adam.py:129 _create_slots
        self.add_slot(var, 'v')
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py:764 add_slot
        initial_value=initial_value)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/variables.py:262 __call__
        return cls._variable_v2_call(*args, **kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/variables.py:256 _variable_v2_call
        shape=shape)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/variables.py:67 getter
        return captured_getter(captured_previous, **kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2124 create_colocated_variable
        return next_creator(**kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/variables.py:67 getter
        return captured_getter(captured_previous, **kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_lib.py:2024 creator_with_resource_vars
        created = self._create_variable(next_creator, **kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/tpu_strategy.py:870 _create_variable
        **kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/distribute_utils.py:291 create_mirrored_variable
        value_list = real_mirrored_creator(**kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/distribute/tpu_strategy.py:861 _real_mirrored_creator
        v = next_creator(**kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/variables.py:67 getter
        return captured_getter(captured_previous, **kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py:685 variable_capturing_scope
        lifted_initializer_graph=lifted_initializer_graph, **kwds)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/variables.py:264 __call__
        return super(VariableMetaclass, cls).__call__(*args, **kwargs)
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py:233 __init__
        shape = initial_value.shape
    /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:1167 shape
        self._tensor_shape = tensor_shape.TensorShape(self._shape_tuple())

    ResourceExhaustedError: Failed to allocate request for 377.93MiB (396288000B) on device ordinal 0

This is extremely confusing, as I have thought that TPU's have wayy more memory than a GPU (Like at least 64GiB). Does this indicate a bug on my side? Is there any way to check the actual memory capacity and memory used of the TPU? please help
. I am using Colab Pro
TPU Initialization code:-

import tensorflow as tf
tpu = tf.distribute.cluster_resolver.TPUClusterResolver()
print(f"Running on TPU: {tpu.cluster_spec().as_dict()['worker']}")
tf.config.experimental_connect_to_cluster(tpu)
tf.tpu.experimental.initialize_tpu_system(tpu)
strategy = tf.distribute.TPUStrategy(tpu)

Reproduction Code:-
Lrelu = tf.keras.layers.LeakyReLU(alpha=0.1)
n_feautures = 40                 #Any number
def build_model():
  model = tf.keras.Sequential([
  tf.keras.layers.Dense(40, activation=Lrelu, input_shape=(n_features,)),
  tf.keras.layers.Reshape((n_features,1), input_shape=(n_features,)),
  tf.keras.layers.Dense(9000, activation=Lrelu),
  tf.keras.layers.Dense(7500, activation=Lrelu),
  tf.keras.layers.Conv1D(60, 6, 1, activation='relu'),
  tf.keras.layers.Dense(5500, activation=Lrelu),
  tf.keras.layers.Dense(8800, activation=Lrelu),
  tf.keras.layers.Conv1D(60, 6, 1, activation=Lrelu),
  tf.keras.layers.Dropout(0.15),

  tf.keras.layers.Dense(9000, activation=Lrelu),
  tf.keras.layers.Conv1D(60, 6, 1, activation=Lrelu),
  tf.keras.layers.Dense(10000, activation=Lrelu),
  tf.keras.layers.Dense(8000, activation=Lrelu),

  tf.keras.layers.Dense(9000, activation=Lrelu),
  tf.keras.layers.Conv1D(60, 6, 1, activation=Lrelu),
  tf.keras.layers.Dense(11000, activation=Lrelu),

  tf.keras.layers.Dense(9000, activation=Lrelu),
  tf.keras.layers.Dense(11000, activation=Lrelu),

  tf.keras.layers.Dense(8000, activation=Lrelu),
  tf.keras.layers.Dense(8100, activation=Lrelu),
  tf.keras.layers.Dense(8000, activation=Lrelu),
  tf.keras.layers.Dense(9000, activation=Lrelu),
  tf.keras.layers.Dense(1, activation='linear'),
  tf.keras.layers.BatchNormalization()
])
  return model

model = build_model()
adam = Adam(0.001)
model.compile(optimizer=adam, loss="mean_squared_logarithmic_error") #, metrics=['accuracy', 'acc']

#Let's see the model's organs!
model.summary()

Comment below for more information and I would update this code. Also, before this code cell, I also import a 600Mb Pandas Dataset, but since that would use System RAM, It would have a negligible effect and would be totally useless..

@neel04,
On running the code, I am facing an error stating NameError: name 'n_features' is not defined. Please find the gist of it here.
In order to expedite the trouble-shooting process, could you please provide the complete code to reproduce the issue reported here. Alternatively, you can also share the gist of the Colab notebook you are running. Thanks!

@amahendrakar Sorry, there was a typo. The line should be n_features = 40

Hi @neel04, when using TPUStrategy the model has to fit on each TPU core. So while the V2 TPU might have 64GB, each core only has 8GB.  TPUStrategy is a data parallelism strategy, and creates a model replica on each core, which you can verify by typing strategy.num_replicas_in_sync (this should be 8).
For a more in depth answer see #44689. However, that thread was about v3-8 TPUs, which have 16GB per core, and I think the default in Colab is the V2 TPUS, which have 8GB per core. If you want to double check, this Stack Overflow answer shows how you can figure out the TPU version and memory in Colab.
Hope this clears up any confusion!

@nikitamaia Thanx a lot for your reply. I understand the limit of each core in TPU, but perhaps is there a way to distrribute the model replica on every 2 cores rather than 1 core so as to fit all the parameters and train on it? Is there a way to adjust the strategy to accomodate something like that?

There is no out of the box way to do that currently, as the tf.distribute.Strategy API only supports data parallelism. Distributing the replica over two cores would require model parallelism, which is not supported right now. If 16GB is enough for you, you could consider upgrading to the V3-8 TPUs on GCP. Or if you're interested in doing some custom work  to distribute the model you can check out Mesh TensorFlow.
Closing this issue now as there is no bug.


[FLIP-934] Add InteractionTemplate support to FCL Dev Wallet

[FLIP-934] Add InteractionTemplate support to FCL Dev Wallet
Expected Outcomes

[ ] FCL Dev Wallet makes use of InteractionTemplate when rendering UI
[ ] FCL Dev Wallet queries auditors it trusts for InteractionTemplateAudit for an InteractionTemplate

@JeffreyDoyle is this still active?

Still active, though this is quite low priority at the moment.


Helium CSS/JS is not included in HTML in strict mode

I'm writing an application that accepts user input and then generates HTML docs from it. The HTML is then served via a web server to the users. I want to use the default Helium theme.
Because the user cannot be trusted, I have enabled the strict mode to disable all the complex extensions in Laika. However, I have noticed that this results in HTML files that have completely empty <head> elements, which means no CSS and JS files are included.
Is this even a bug? I can guess why this happens, but it's still counter-intuitive. Is there any workaround I could use?
Thanks! :)
PS: the JS and CSS files are correctly included in the output tree. It is only the HTML file that is missing references to them.

Ah, that is a somewhat exotic scenario that I have never thought of. And improving this would be a lot of work, so even for 0.19 which is fairly close to being finished, this would be too late.
The reason is that all extensions build around the core trait ExtensionBundle which has a property useInStrictMode so that when the user switches this flag on all bundles that have the flag set to false get removed. The problem with the default Helium theme is that it builds all extensions with the flag set to false and it's non-trivial to change that as Helium itself uses the public user-facing theme API for composing themes. To make this smarter those APIs would need to change so that a theme author can indicate whether an extension of their theme is compatible with strict mode or not. I'm happy to consider this for a future version, but it's too late for 0.19, so won't be this year. I have added this to the backlog for now.
I might find a quick win for 0.19 that at least sets the flag to true for all bundles that don't install parser extensions, but no promises right now, as I'm not sure how easy/feasible that is.
The only workaround for the current version I can think of is quite cumbersome: Configure a custom ThemeProvider that uses the Helium ThemeProvider but then translates the Theme that the latter produces by wrapping all ExtensionBundle instances with that flag set to true.

Thanks for the quick response and the suggested workaround! I think I will try something along these lines then.

After a lot of experimentation I have finally found a solution that I'll share here in case anyone else stumbles upon this. The solution was not to modify the bundles included with the Theme – that just reenabled very minor things, like heading anchors, but nothing else. What I really needed to do was to disable the strict mode, and modify some of the bundles that are included by default to remove "unsafe"-sounding features from them. So, I disabled a bunch of block and span directives, and turned off the config parser entirely (that one alone took me 3 hours to figure out, ugh).
I license the below code under the Apache License 2.0.

Code (Scala 3)
import cats.effect.{IO, Resource}
import laika.api.builder.{OperationConfig, TransformerBuilder}
import laika.ast.MessageFilter
import laika.bundle.{ConfigProvider, ExtensionBundle, ParserBundle}
import laika.directive.{DirectiveRegistry, DirectiveSupport}
import laika.directive.std.*
import laika.factory.{MarkupFormat, RenderFormat}
import laika.helium.Helium
import laika.io.api.TreeTransformer
import laika.io.implicits.*
import laika.parse.code.SyntaxHighlighting

object LaikaUtil:
  def makeTransformer[FMT](from: MarkupFormat, to: RenderFormat[FMT]):
  Resource[IO, TreeTransformer[IO]] =
    new TransformerBuilder(from, to, safeOpConfig.withBundlesFor(from))
      .parallel[IO]
      .withTheme(Helium.defaults.build)
      .build

  val safeOpConfig = OperationConfig(
    bundles = Seq(
      NoConfigProviderExtension,
      ExtensionBundle.LaikaDefaults,
      DirectiveSupport,
      SafeStandardDirectives,
    )
  )

  object NoConfigProviderExtension extends ExtensionBundle:
    override def description = "Registers a no-op config provider"
    override def parsers = ParserBundle(
      configProvider = Some(ConfigProvider.empty)
    )

  object SafeStandardDirectives extends DirectiveRegistry:
    def blockDirectives: Seq[laika.directive.Blocks.Directive] = List(
      BreadcrumbDirectives.forBlocks,
      NavigationTreeDirectives.forBlocks,
      ImageDirectives.forBlocks,
      StandardDirectives.callout,
      StandardDirectives.todoBlock,
    )
    def linkDirectives: Seq[laika.directive.Links.Directive] = List()
    def spanDirectives: Seq[laika.directive.Spans.Directive] = List(
      ImageDirectives.forSpans,
      StandardDirectives.todoSpan,
    )
    def templateDirectives: Seq[laika.directive.Templates.Directive] = StandardDirectives.templateDirectives

Ah, this is a very bespoke set of features somewhere halfway between strict and full mode. This is not how strict is defined  for Laika, which essentially means "nothing beyond the core spec of supported markup languages". So even after the planned improvements, you'd still need to do most of this.
The problem I discovered when you posted this was merely that in the case of Helium strict mode removes more things than just the parser extensions which is unnecessary and which I intend to improve for 0.19.
But congratulations for mastering all these internals, looks good! 🙂

@Ostrzyciel - just a little update on how you could simplify your logic now with version 0.19 (RC-1 is out now, the final release will follow in a few days).
The issues with strict mode aggressively removing too many extensions is fixed. However, you still have to deal with the fact that strict mode means no directives for markup. You can reduce your setup to just this (sorry, Scala 2 syntax):
object SafeMarkupDirectives extends ExtensionBundle {

  val description: String = "Safe markup directives for strict mode"

  private val blockDirectives = List(
    BreadcrumbDirectives.forBlocks,
    NavigationTreeDirectives.forBlocks,
    ImageDirectives.forBlocks,
    StandardDirectives.callout,
    StandardDirectives.todoBlock,
  )

  private val spanDirectives = List(
    ImageDirectives.forSpans,
    StandardDirectives.todoSpan,
  )

  override lazy val parsers: ParserBundle = ParserBundle(
    blockParsers = Seq(BlockDirectiveParsers.blockDirective(Blocks.toMap(blockDirectives))),
    spanParsers = Seq(
      SpanDirectiveParsers.spanDirective(Spans.toMap(spanDirectives)),
      SpanDirectiveParsers.contextRef
    )
  )

}

This just adds back the block and span directives you selected and the parsers for them (which get removed by strict mode). You do no longer have to deal with template directives (they now remain with strict mode) or config headers (always disabled in strict mode). Instead you can use standard APIs, switch strict mode on and use the above extension to get back your directives (this is untested, but should work as intended):
Transformer.from(Markdown).to(HTML).strict.using(SafeMarkupDirectives).parallel[IO].build

Thank you for your help! This will be very helpful when I upgrade to 0.19. :)


runtime scope dep as libraryjar?

Hi,
Not new to proguard, but new to maven. I have a project that I'm trying to run proguard on, and it has some dependencies which are scoped as runtime (I believe this is because they are deployed separately? it's hdfs infra.)
The runtime scope dependencies are not being passed to proguard as libraryjars (nor in any other capacity). Proguard complains vociferously (as it should) and fails.
I'm surprised the maven plugin does not pass the runtime scope deps onto proguard? Is this a conscious design decision or an oversight?  Is there any way to get it to?
Thanks!

This should be straightforward to fix by changing https://github.com/wvengen/proguard-maven-plugin/blob/master/src/main/java/com/github/wvengen/maven/proguard/ProGuardMojo.java#L515
We could just add a call to https://maven.apache.org/ref/3.2.3/apidocs/org/apache/maven/project/MavenProject.html#getRuntimeArtifacts() but both that and the existing getCompileArtifacts have been deprecated a long time. Instead we could use https://maven.apache.org/ref/3.2.3/apidocs/org/apache/maven/project/MavenProject.html#getDependencyArtifacts() and add only those with a proper scope. Test scope can be excluded at least, but seems like system and provided scope should be included.

@newobj do you have a simple test project for this you could share?

Here's a test project. Here, the artifact com.github.jnr:jffi:jar:native is a runtime dependency of jnr-jffi, which is reachable through a chain of compile dependencies. Consequently, the contents of the artifact (a bunch of shared libraries under a folder called jni) don't make it into the assembly. If I uncomment the commented-out dependency in the POM, the jar contents do make it in, because now it is a compile dependency.
proguard-test.zip


Support marshaling generic Godot.Object in C#

Fixes #47510
Allows using generic C# types in signals as long as they inherit from Godot.Object.

Thanks!


Add simplifierx switch

Fixes https://github.com/kframework/kore/issues/2939
See #2923 for simplifier discussion.
Scope:

Add --simplifierx switch

Review checklist
The author performs the actions on the checklist. The reviewer evaluates the work and checks the boxes as they are completed.

[ ] Summary. Write a summary of the changes. Explain what you did to fix the issue, and why you did it. Present the changes in a logical order. Instead of writing a summary in the pull request, you may push a clean Git history.
[ ] Documentation. Write documentation for new functions. Update documentation for functions that changed, or complete documentation where it is missing.
[ ] Tests. Write unit tests for every change. Write the unit tests that were missing before the changes. Include any examples from the reported issue as integration tests.
[ ] Clean up. The changes are already clean. Clean up anything near the changes that you noticed while working. This does not mean only spatially near the changes, but logically near: any code that interacts with the changes!

Since I need new work, I picked this up. I tried to work with having the new option added to GlobalOptions, but it seems to me like these options were added to only be used in GlobalMain, before calling the specific executable. Since I can't really use your changes, I'm going to open a new PR. That will also mean that you can review/approve it when it's ready.

Ok, thank you for the explanation!


[User] Logged users marked as non-authenticated

When you log in as a Customer or as an AdminUser, the Symfony ToolBar shows a yellow badge for the username and a red badge for authentication, when both should be green.
This is a sign that we're doing something wrong with credentials, but I can't find what is it.

Ok, I found it the very moment I sent the issue: we're missing to load the app/security/accessControl.yml file. The file is there, but not referenced anywhere. PR is coming.

Fixed in #283


feat: add fallback rate limiter and cache to fallback to memory if redis fails

Original discussion here: https://github.com/ethereum-optimism/infra/pull/90

Description
Adds a new config to proxyd: redis.fallback_to_memory.
This allows falling back to a memory implementation of Redis features if they fail instead of falling back to an error (in the case of rate limiting) or a no-op (in the case of caching).
I also moved redis connection testing which will warn if we're not able to connect with the new flag is enabled, but still error if the fallback is disabled.
If the context is cancelled for the cache, we don't retry with the secondary source because this indicates a
Tests
Added test for rate limiter fallback and cache fallback.

proxyd/v4.11.0


When calculate initial holdings only consider blocks deeper than 5

Description
When calculate initial holdings only consider blocks deeper than 5
https://github.com/statechannels/statechannels/issues/2927
Then, following a single blank line, a summary of the change and which issue is fixed, including

[ ] what is the problem being solved
To avoid including holdings that's not confirmed yet.
[ ] why this is a good approach
BlockTag give us data as of 6 blocks ago
[ ] what are the shortcomings of this approach, if any
Using BlockTag need provider support.

/********************************

These cases run good when alone, but unstable when run together with others, likely due to

there're still threads created by others advancing the block chain.

Removed
********************************/
it('Initial holding right when deposit not confirmed', async () => {
mineBlocks();
const channelId = randomChannelId();
await waitForChannelFunding(0, 2, channelId);
await mineBlocks();
await waitForChannelFunding(2, 3, channelId);
// When we query holdings we only get confirmed amount (6 blocks ago),
// at this time we have 5 latest but only 2 is confirmed
await new Promise(resolve =>
chainService.registerChannel(channelId, [ethAssetHolderAddress], {
...defaultNoopListeners,
holdingUpdated: arg => {
expect(arg).toMatchObject({
channelId,
assetHolderAddress: ethAssetHolderAddress,
amount: BN.from(2),
});
resolve(true);
},
})
);
});
it('Initial holding 0 when channel not confirmed', async () => {
await delay(5000);
const channelId = randomChannelId();
await waitForChannelFunding(0, 5, channelId);
// When we query holdings we only get confirmed amount (6 blocks ago),
// if channel creation is within 6 blocks then we should get 0
await new Promise(resolve =>
chainService.registerChannel(channelId, [ethAssetHolderAddress], {
...defaultNoopListeners,
holdingUpdated: arg => {
expect(arg).toMatchObject({
channelId,
assetHolderAddress: ethAssetHolderAddress,
amount: BN.from(0),
});
resolve(true);
},
})
);
});

Without diving into this too deeply, how does the chain service behave when:

Holdings are 0 in block 1
Holdings are updated to 10 in block 2
Wallet restarts. The latest block is block 3.

As of this change, the chain service would return the confirmed holdings. Will it later inform the wallet that the holdings are updated to 10? (I think this should happen when block 7 is mined, but only if it includes the block 2 that I noticed from before.)

No.
It seems when we subscribe to events we only  get the latest, there is no blockTag passed into this interface. https://docs.ethers.io/v5/api/contract/example/#erc20-events
So sounds like, when deposit/allocation/wallet restart happens, we should create an objective, and come back to check again after 6 blocks?

Without diving into this too deeply, how does the chain service behave when:

Holdings are 0 in block 1
Holdings are updated to 10 in block 2
Wallet restarts. The latest block is block 3.

As of this change, the chain service would return the confirmed holdings. Will it later inform the wallet that the holdings are updated to 10? (I think this should happen when block 7 is mined, but only if it includes the block 2 that I noticed from before.)

I took another

Without diving into this too deeply, how does the chain service behave when:

Holdings are 0 in block 1
Holdings are updated to 10 in block 2
Wallet restarts. The latest block is block 3.

As of this change, the chain service would return the confirmed holdings. Will it later inform the wallet that the holdings are updated to 10? (I think this should happen when block 7 is mined, but only if it includes the block 2 that I noticed from before.)

No.
It seems when we subscribe to events we only get the latest, there is no blockTag passed into this interface. https://docs.ethers.io/v5/api/contract/example/#erc20-events
So sounds like, when deposit/allocation/wallet restart happens, we should create an objective, and come back to check again after 6 blocks?

I took another look at Andrew's question, we seem already have some logic to handle confirmation on events.
registerChannel -> setUpAssetHolderListener -> addAssetHolderObservable -> waitForConfirmations
However, due to racing conditions there is still a slight chance that things can go wrong:
If the chain advances betweensetUpAssetHolderListener and getInitialHoldings, a deposit event can be double counted. The mitigate would be, always re-read and overwrite the holdings amount in event.
I'll think about this for a bit.
@kerzhner any suggestions?

Updated to wait on unconfirmed events instead.

My bad, cause getInitialHoldings is async already so it won't block registerChannel. registerChannel exist immediately, but by the time subscriber.holdingUpdated is called, it is 5 blocks away and all deposits included in the initial holdings are confirmed.
So the current version of the code should work.
@kerzhner pls help to do another round of review to see if this solution is good. Thank you!

Thanks for thinking this through and putting this work together. This is definitely a very tricky part of the chain service.
My interpretation of the work is that we are now doing the following:

When an event E is observed from the blockchain, we wait the number of confirmation blocks before considering E as valid.
Once E is valid, we inspect whether another more recent and valid event E' has been observed. If yes, E is ignored.
If not, subscribers are notified of E.

It seems to me that this is the right approach to think about events. One thing that would be nice to try is to the following:

A blockchain event can be observed from a contract listener or from looking at past blocks.
It seems that once an event is observed, the same 3 steps need to be taken (as outlined above). At the moment this logic is not shared. Is there a reasonable way to factor out/ share this logic?

Yes you understanding is correct.
For 1, yes, contract listener will play all events from "provider.blockNumber" to the latest block.  "provider.blockNumber" can be out of date at the time when we register channel. So we don't know which events are captured by listener and which are not. In theory we can move provider.blockNumber back and forth to replay events, but that's dangerous as it affects all contracts.
For 2, good suggestion. I'll do it and ping you for another review once ready.

Thanks for thinking this through and putting this work together. This is definitely a very tricky part of the chain service.
My interpretation of the work is that we are now doing the following:

When an event E is observed from the blockchain, we wait the number of confirmation blocks before considering E as valid.
Once E is valid, we inspect whether another more recent and valid event E' has been observed. If yes, E is ignored.
If not, subscribers are notified of E.

It seems to me that this is the right approach to think about events. One thing that would be nice to try is to the following:

A blockchain event can be observed from a contract listener or from looking at past blocks.
It seems that once an event is observed, the same 3 steps need to be taken (as outlined above). At the moment this logic is not shared. Is there a reasonable way to factor out/ share this logic?

Yes you understanding is correct.
For 1, yes, contract listener will play all events from "provider.blockNumber" to the latest block. "provider.blockNumber" can be out of date at the time when we register channel. So we don't know which events are captured by listener and which are not. In theory we can move provider.blockNumber back and forth to replay events, but that's dangerous as it affects all contracts.
For 2, good suggestion. I'll do it and ping you for another review once ready.

I added subscribeToConfirmedEvents and queryConfirmedEvents to generalize the logic to wait for confirmation. Now all events will need to be confirmed before passed to customer.
De-dupe logic cannot be further generalized - whether an asset holder event is duplicate, depends on the destination address it is passed to, for example:

A deposit to Ether assetholder happened at block #100, picked up at #120
A depost to ERC20 assetholder happened at block #95, picked up at #130
In this case, the later shouldn't be considered duplicate, as it has different destination.
It's better to leave this logic in EventTracker.

the event replay logic

Hi @kerzhner ,

Thanks for refactoring logic, things are further simplified. Looks good to me, thanks!
Yes we can add a UT case. Existing UT cases covered duplicate message got removed. We should have a case to check when event listener didn't play event, the query filter can catch it. I'll add such a case.

Thanks for refactoring logic, things are further simplified. Looks good to me, thanks!

Awesome! If those commits look good to you, feel free to cherry-pick those into the branch for this PR.

@isoosiss7 I would be curious to hear your feedback on 3b055411306721a9643870bd8963354a7ca82682...dcf9cfa1f3f7ab7c2130eed908a83b9818fcda23. The commit messages contain my reasoning for that work.
Specifically, it would be great to hear if dcf9cfa1f3f7ab7c2130eed908a83b9818fcda23 test refactor eliminates any test conditions that are needed to test this PR. In other words, does dcf9cfa1f3f7ab7c2130eed908a83b9818fcda23 make our tests less robust?
If you are happy with these commits, let's include them in this PR.

@isoosiss7 I would be curious to hear your feedback on 3b05541...3746e9b. The commit messages contain my reasoning for that work.
Specifically, it would be great to hear if dcf9cfa test refactor eliminates any test conditions that are needed to test this PR. In other words, does dcf9cfa make our tests less robust?
If you are happy with these commits, let's include them in this PR.

Merged, thanks!
This test case is to make sure the unconfirmed deposits (while getInitialHoldings) are played. Due to racing condition between ts code and the chain, it's hard to predict which deposit is unconfirmed when the test runs. So I had to run deposit 10 times to make sure one of them is in unconfirmed state while it runs.
Thanks for your optimization. I tried it a few times and it is stable.


Connect stumbles over two different robot models

mtc_demos/pa10: left-arm and right-arm task should be independent, but latter gets mixed up (RobotModel-wise) with former?!?

Probably was related to ce062235fbe232d5649f9c5780d90a2b243302d5.


SR-10689: Fix bugs of DataProtocol's firstRange(of:in:)/lastRange(of:in:).

DataProtocol's firstRange(of:in:) and lastRange(of:in:) return a wrong value or crash under some conditions because they handle indices incorrectly.
This PR fixes the issue.
Resolves SR-10689.

Note:
SR-10689 also says "lastRange(of:) returns firstRange(of:in:)", however, it has been already fixed by #2310.

@swift-ci test

Rebased to resolve conflicts.

@swift-ci test

The failures seem to be unrelated to this PR, don't they...?

@swift-ci test

@swift-ci test

@CodaFi
Thank you for running tests.
The error on macOS seems to be unrelated.

@swift-ci test macOS

@swift-ci test

The failure on Linux seems to be caused by LLDB...

I believe this is not stale... so rebased.
(Or we should wait for generalized "Collection Matchers" suggested by @milseman?)

@swift-ci test

@spevans Thank you for rerunning tests.

The error seems to be unrelated.

/home/buildnode/jenkins/workspace/swift-corelibs-foundation-PR-Linux/swift/validation-test/StdlibUnittest/RaceTest.swift:116:11: error: CHECK: expected string not found in input
16:55:33 // CHECK: [ OK ] Race.closure
16:55:33           ^
16:55:33 :17:1: note: scanning from here
16:55:33 stderr>>> CRASHED: SIGSEGV
16:55:33 ^
16:55:33 :19:3: note: possible intended match here
16:55:33 [ FAIL ] Race.closure
16:55:33   ^
16:55:33
16:55:33 --

@swift-ci test linux

@spevans Thank you again. At last tests have passed.

Finally PR for SDK Overlay (swift#28639) has been merged.
I expect this PR is also ready to be merged.

I've just discovered that the previous implementation also had bugs in the returned length of the range.
So it's probably a good idea to add tests for that too.
Here is a case that returns shorter range than the given input in Xcode 11.6 (11E708):
import Foundation
let data = Data([1, 2, 3, 4, 5, 6])
let search = Data([2, 3, 9])
data.firstRange(of: search, in: 0 ..< 5) // returns 1..<2, should be nil

@juozasvalancius
Thank you for the review.
The length is checked here https://github.com/apple/swift-corelibs-foundation/pull/2499/files#diff-593783312ec62e6e9299004a53468f83R865 and your case could be covered by https://github.com/apple/swift-corelibs-foundation/pull/2499/files#diff-593783312ec62e6e9299004a53468f83R892

No conflicts even after the base branch was changed from master to main.

Should this PR be alive? https://github.com/apple/swift-corelibs-foundation/pull/2915 was merged, though.

Confirmation

PR for SDK Overlay (swift#28639) was merged in Aug 2020. And now, the commit exists in this scf repository as a result of #2915. It is the same implementation with this PR.
SR-10689 is not yet resolved on non-Darwin and will be resolved by this PR.
All checks have passed, and this PR has no conflicts with main branch even now.
(#2993 would resolve the same issue in the different way.)

@millenomi Could you review this PR (and/or #2993) at your convenience?

@swift-ci test

@CodaFi
Thnak you for rerunning tests.
All tests have passed again.

I would like sign offs from some core maintainers here.
@millenomi @spevans

Done.

@millenomi Thank you for your approval.


run不了

D:\morecNew>flutter run
Using hardware rendering with device Android SDK built for x86. If you get graphics artifacts, consider enabling
software rendering with "--enable-software-rendering".
Launching lib/main.dart on Android SDK built for x86 in debug mode...
Initializing gradle...                                              1.7s
Resolving dependencies...

Error running Gradle:
ProcessException: Process "D:\morecNew\android\gradlew.bat" exited abnormally:

Configure project :app

Project evaluation failed including an error in afterEvaluate {}. Run with --stacktrace for details of the afterEvaluate{} error.
FAILURE: Build failed with an exception.

Where:
Build file 'D:\morecNew\android\app\build.gradle' line: 25

What went wrong:
A problem occurred evaluating project ':app'.

Could not resolve all artifacts for configuration 'classpath'.
Could not resolve com.android.tools.build:gradle:3.2.1.
Required by:
unspecified:unspecified:unspecified
Could not resolve com.android.tools.build:gradle:3.2.1.
Could not get resource
'https://maven.aliyun.com/repository/google/com/android/tools/build/gradle/3.2.1/gradle-3.2.1.pom'.
Could not HEAD
'https://maven.aliyun.com/repository/google/com/android/tools/build/gradle/3.2.1/gradle-3.2.1.pom'.
Connect to <IP_ADDRESS>:6600 [/<IP_ADDRESS>] failed: Connection refused: connect
Could not resolve com.android.tools.build:gradle:3.2.1.
Could not get resource
'https://maven.aliyun.com/repository/jcenter/com/android/tools/build/gradle/3.2.1/gradle-3.2.1.pom'.
Could not HEAD
'https://maven.aliyun.com/repository/jcenter/com/android/tools/build/gradle/3.2.1/gradle-3.2.1.pom'.
Connect to <IP_ADDRESS>:6600 [/<IP_ADDRESS>] failed: Connection refused: connect
Could not resolve com.android.tools.build:gradle:3.2.1.
Could not get resource
'http://maven.aliyun.com/nexus/content/groups/public/com/android/tools/build/gradle/3.2.1/gradle-3.2.1.pom'.
Could not HEAD
'http://maven.aliyun.com/nexus/content/groups/public/com/android/tools/build/gradle/3.2.1/gradle-3.2.1.pom'.               > Connect to <IP_ADDRESS>:6600 [/<IP_ADDRESS>] failed: Connection refused: connect

Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with
--scan to get full insights.

Get more help at https://help.gradle.org

BUILD FAILED in 9s
Command: D:\morecNew\android\gradlew.bat app:properties
Please review your Gradle project setup in the android/ folder.

请重新 pull 项目


Default to --access github

I've been using cargo-bisect-rustc for a while now, and --access github has never caused any problems. Maybe it can run into
ratelimiting in extreme cases, but the user should then just use git.
Meanwhile, git is a very confusing default as it will suddelny just
start cloning the repo unless you were very careful when installing.

I have not used the github access method much, but I'm game to try. It certainly will be a better default experience as long as it is not rate limited. @oli-obk what do you think?

I think it is rate limited, but I've never hit the limit in practice when doing one-off bisections (or even two or three).

The unauthenticated rate limit is 60 requests/hour/IP, which should be more than enough for bisecting, outside heavily NATted connections (I would assume it only does one request per run?).


[release]: Publish opensearch-cluster-cdk to NPM

Did you read the on-boarding document
Yes
What is the name of your component?
opensearch-cluster-cdk
What is the link to your GitHub repo?
https://github.com/opensearch-project/opensearch-cluster-cdk
Targeted release date
Jan 15th, 2024
Where should we publish this component?
npmjs.org
What type of artifact(s) will be generated for this component?
Can be published directly using repo
Have you completed the required reviews including security reviews, UX reviews?
In process. Will be updating this issue once we the review is complete.
Have you on-boarded automated security scanning for the GitHub repo associated with this component?
Yes
Additional context
We want to publish opensearch-cluster-cdk to npm so that the cdk stacks can be extended by other projects programmatically.

[Triage]
Assigning to @gaiksaya,

Proceeding with the on-boarding. cc: @rishabh6788


Import tools do not display cookbook name in error messages

When uploading data into Chef Server 12, error messages like this:
ERROR: /cookbooks failed to write: HTTP error writing: 400 "Bad Request" cause: {"error":["Field 'metadata.maintainer' invalid"]}
do not display the name of the cookbook which produced that error - would be nice if they did :)

I moved this since it's probably a knife issue, but in order to troubleshoot it we'd need to see the cookbook being uploaded and have it reproduced in knife 16.x. If anyone runs into this again open a new ticket and we'll track it down.


Test re-enabling depth camera integration test on mac

🦟 Bug fix
Summary
Testing fix from #333. This PR re-enables the depth camera integration test to see if it'll pass on our homebrew CI machine.
Checklist

[x] Signed all commits for DCO
[ ] Added tests
[ ] Updated documentation (as needed)
[ ] Updated migration guide (as needed)
[ ] codecheck passed (See contributing)
[ ] All tests passed (See test coverage)
[ ] While waiting for a review on your PR, please help review another open pull request to support the maintainers

Note to maintainers: Remember to use Squash-Merge

homebrew build is green:
https://build.osrfoundation.org/job/ignition_rendering-ci-pr_any-homebrew-amd64/1514/
I'll trigger one more build

@osrf-jenkins run tests please

the latest homebrew build failed to compile due to issue with ign-common4  package not being found. It's happening to other builds of ignition libraries so unrelated.
I'm going to merge this. If test fails again in the future, I'll disable it.


while simple to make, include a "bookmark" clipper

While simple to make the template, not every user will know how or think of how to create a template to clip the URL of a page they are viewing and save to read it later.  I think you should include in the base install a Book,ark template.  The template save the URL with page title to the end of a note.  The Note defaults to /Bookmarks or perhaps a bookmarks.md note in the default clipper folder.  Each bookmark would have title and the metadata as a callout below that, collapsed.  perhaps an option to be collapsed or not.
As always, user's can edit to place this at a different location, specify which metadata is saved, bottom or top of page and the like.
If possible allow a metadata field to specify a heading the bookmark is saved under.  If a header is specified that bottom/top of page mean top or bottom of items under that header
EDIT:  Perhaps a new behaviors would be top/bottom of bookmarks page

I am not sure I understand what you are saying. Do you mean that Web Clipper should support saving a page to the Obsidian Bookmarks plugin as opposed to saving it as a note?
If you mean saving the bookmark as a regular note, then this is already possibly by duplicating and/or modifying the default template. In the future we'll add more ways to explore and download community templates.


create_prompt_layout lacks argument for specifying editing mode

SearchToolbarControl has a vi_mode argument, but the default of vi_mode=False is effectively immutable for callers of create_prompt_layout, because create_prompt_layout lacks an argument for specifying the editing mode.  Consequently, vi mode users are shown an emacs prompt ("I-search: " or "I-search backward:") when searching their history.

True. This was actually done consiously. I thought that this would be the most user friendly visualisation if we have a multiline input. If there is some indentation in the left margin, like line numbers, then the "(reverse-i-search)" visualisation doesn't work that well. (Which is by the way exactly the same, regardless of whether the user is in Vi or Emacs mode. However, only for incremental search.)
I can always expose that argument, but I guess I need to rethink the search implementation, and have both incremental and non-incremental search. (If that's the correct name.) In Vi-mode, you can start searching by pressing "Control-R", or by pressing "/" in navigation mode.
Changing the visualisation to "/" for Vi mode would be confusing without first implementing the search accordingly.
Thanks for opening the issue!


deConz + rpi4 + raspbee = firmware update fails

Hi there,
I just want to update the firmware of my raspbee but I get following output. I followed every step like discribed.
Hardware:

Raspberry Pi 4b (hassio 32Bit)
RaspBee (current firmware version: 26050500)

I saw the update script here is version 0.4 and there is 0.5 allready out. Next thing I've read is that WiringPi 2.52 is needed for flashing the firmware correctly. But its all given in marthocs repository. Dont know if you merge from there.
Here is the whole log:
# docker run -it --rm --device=/dev/ttyAMA0 --privileged --cap-add=ALL -v /lib/m
odules:/lib/modules --entrypoint "/firmware-update.sh" marthoc/hassio-addon-deco
nz-armhf
-------------------------------------------------------------------

       marthoc/deconz Conbee/RaspBee Firmware Flashing Script

                       Version: 0.4

-------------------------------------------------------------------

Enter C for Conbee, R for RaspBee, or press Enter now to exit: R

-------------------------------------------------------------------

Firmware available for flashing:

deCONZ_ConBeeII_0x26480700.bin.GCF
deCONZ_ConBeeII_0x26490700.bin.GCF
deCONZ_Rpi_0x261f0500.bin.GCF
deCONZ_Rpi_0x26320500.bin.GCF
deCONZ_Rpi_0x26330500.bin.GCF

Enter the firmware file name from above, including extension,
or press Enter now to exit.

File Name : deCONZ_Rpi_0x26330500.bin.GCF

-------------------------------------------------------------------

Device: RaspBee

Firmware File: deCONZ_Rpi_0x26330500.bin.GCF

Are the above device and firmware values correct?
Enter Y to proceed, any other entry to exit: Y

Flashing...

GCFFlasher V3_05 (c) dresden elektronik ingenieurtechnik gmbh
Reboot device RaspBee (RaspBee)
deCONZ firmware version 26050500
action: reset device RaspBee
action: update firmware after 2 ms
could not sync with bootloader

Flashing Error! Please re-run this script...

marthoc/hassio-addon-deconz-armhf. That is not our add-on. Our add-on support firmware update over the deconz UI without any scripts or command line stuff.

marthoc/hassio-addon-deconz-armhf. That is not our add-on. Our add-on support firmware update over the deconz UI without any scripts or command line stuff.

Ok. I installed the official deconz addon using the add-on store. I have no other repos in my setup except the comunity repo. But when trying to upgrade the firmware over ui it says its been updated but after a minute it shows me the old firmware version and tells me there is a new one.

hmm, that is proper a very old issue. The UI should work, not sure if it is an RPi4 issue. I'm very happy that the new fast Conbee2 is USB


Item Shops Bugs Out After Heavy Usage

After heavy usage of an item shop, it can "bug out", causing you to be unable to buy/sell items from the menu until reopening the trade window. This is shown in the clicking on an item and it turns to a black glass pane.
No errors are shown if this happens.
https://i.imgur.com/7qLbWFE.gif

can you give me your shop.yml file?
I couldn't reproduce it.

Of course, it took quite a bit (~5m) for me to even get it to replicate once,
https://paste.gg/p/anonymous/76531b0787d74aae8efe3f995054b7e0

While trying to reproduce, I looked closely at the git file again, and it is strange that there is an item 'black_stained_glass_pane'.
DynamicShop plugin does not create 'black_stained_glass_pane' in that UI.
It is possible that another plugin on the server is causing the problem.

Hmm, I don't see why any plugin would do that on my server, however, I shall get back to you once I look through them all.

The way store data is handled is now improved.


fix(kafka): added strong ids

:tada: This PR is included in version 1.118.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Closes admiralpeds issue 40 Use pharmaversesdtm to host admiralpeds test data

Thank you for your Pull Request! We have developed this task checklist from the Development Process Guide to help with the final steps of the process. Completing the below tasks helps to ensure our reviewers can maximize their time on your code as well as making sure the admiral codebase remains robust and consistent.
Please check off each taskbox as an acknowledgment that you completed the task or check off that it is not relevant to your Pull Request. This checklist is part of the Github Action workflows and the Pull Request will not be merged into the devel branch until you have checked off each task.

[x] Place Closes #<insert_issue_number> into the beginning of your Pull Request Title (Use Edit button in top-right if you need to update)
[x] Code is formatted according to the tidyverse style guide. Run styler::style_file() to style R and Rmd files
[x] Updated relevant unit tests or have written new unit tests, which should consider realistic data scenarios and edge cases, e.g. empty datasets, errors, boundary cases etc. - See Unit Test Guide
[x] If you removed/replaced any function and/or function parameters, did you fully follow the deprecation guidance?
[x] Update to all relevant roxygen headers and examples, including keywords and families. Refer to the categorization of functions to tag appropriate keyword/family.
[x] Run devtools::document() so all .Rd files in the man folder and the NAMESPACE file in the project root are updated appropriately
[x] Address any updates needed for vignettes and/or templates
[x] Update NEWS.md if the changes pertain to a user-facing function (i.e. it has an @export tag) or documentation aimed at users (rather than developers)
[x] Build admiral site pkgdown::build_site() and check that all affected examples are displayed correctly and that all new functions occur on the "Reference" page.
[x] Address or fix all lintr warnings and errors - lintr::lint_package()
[x] Run R CMD check locally and address all errors and warnings - devtools::check()
[x] Link the issue in the Development Section on the right hand side.
[x] Address all merge conflicts and resolve appropriately
[x] Pat yourself on the back for a job well done! Much love to your accomplishment!

Do we hold off on this until first release of admiralpeds?

Do we hold off on this until first release of admiralpeds?

Yes, it should be aligned with this update: https://github.com/pharmaverse/admiralpeds/issues/40. I'm just catching up from last week and looks like we had more review comments for admiralpeds and the team are working through them still so no CRAN submission as of yet...

@zdz2101 please let us know if you think this one is now ready for review given we've released admiralpeds to CRAN, and me and @Fanny-Gautier could take a look

@zdz2101 please let us know if you think this one is now ready for review given we've released admiralpeds to CRAN, and me and @Fanny-Gautier could take a look

this is ready for review, the one on admiralpeds is to demonstrate it does work

@kaz462 @manciniedoardo @Fanny-Gautier appreciate if you could give this PR a sanity check too.

@rossfarrugia @Fanny-Gautier This can be reviewed once again


Some Char performance optimization

@CodiumAI-Agent /review

PR Reviewer Guide 🔍

⏱ Estimated effort to review: 4 🔵🔵🔵🔵⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Key issues to review
Performance ConcernThe use of Regex with a timeout specified as 100.Milliseconds() might not be performant for simple patterns and could be replaced with more efficient string operations or simpler regex patterns without timeouts.
Redundant CodeThe method ConcatenatedWith for ReadOnlySpan<char> is implemented multiple times with similar logic. Consider refactoring to reduce redundancy and improve maintainability.
Possible BugThe method ToLowerCase and ToUpperCase for char might not handle all Unicode characters correctly outside the basic Latin range. Consider using built-in methods like char.ToLowerInvariant for more comprehensive character handling.


Question about Camera and Cursor Parameters in Gaussian Splats Viewer

I hope this message finds you well. I'm currently using the Gaussian Splats Viewer and noticed the following parameters displayed during usage:
Camera position: 1.00591, 0.02695, -0.18080
Camera look-at: -0.38898, -0.46748, 7.90106
Camera up: 0.20539, -0.96532, -0.16119
Camera mode: Perspective
Cursor position: -0.39636, -0.49209, -7.91252
I’m curious about how these parameters are calculated, particularly the cursor position. Since I'm not very familiar with the inner workings of Three.js, I’d greatly appreciate it if you could explain or point me to relevant parts of the code or documentation for understanding this.

The camera position, camera-up, and camera look-at are all standard properties of a Camera object in three.js, and you can read about that more here: https://threejs.org/docs/#api/en/cameras/Camera. The cursor position is calculated by performing a raycast from from the camera position, using the direction the camera is facing to create a ray, and finding the first splat in the scene that ray intersects with.  The calculation starts here: https://github.com/mkkellogg/GaussianSplats3D/blob/f6f4adea2e0a5745f942767214c7d3b335a10751/src/Viewer.js#L561.

Thank you so much for your quick response! I really appreciate your help and the time you took to answer my question.


[Bug]: PaliGemma detection task is failing

Your current environment
Collecting environment information...
PyTorch version: 2.3.1+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A

OS: Ubuntu 20.04.6 LTS (x86_64)
GCC version: Could not collect
Clang version: Could not collect
CMake version: version 3.30.1
Libc version: glibc-2.31

Python version: 3.11.9 (main, Apr 19 2024, 16:48:06) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.15.0-1062-gcp-x86_64-with-glibc2.31
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA L4
Nvidia driver version: 535.183.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Byte Order:                         Little Endian
Address sizes:                      46 bits physical, 48 bits virtual
CPU(s):                             4
On-line CPU(s) list:                0-3
Thread(s) per core:                 2
Core(s) per socket:                 2
Socket(s):                          1
NUMA node(s):                       1
Vendor ID:                          GenuineIntel
CPU family:                         6
Model:                              85
Model name:                         Intel(R) Xeon(R) CPU @ 2.20GHz
Stepping:                           7
CPU MHz:                            2200.170
BogoMIPS:                           4400.34
Hypervisor vendor:                  KVM
Virtualization type:                full
L1d cache:                          64 KiB
L1i cache:                          64 KiB
L2 cache:                           2 MiB
L3 cache:                           38.5 MiB
NUMA node0 CPU(s):                  0-3
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit:        Not affected
Vulnerability L1tf:                 Not affected
Vulnerability Mds:                  Not affected
Vulnerability Meltdown:             Not affected
Vulnerability Mmio stale data:      Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown
Vulnerability Retbleed:             Mitigation; Enhanced IBRS
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass:    Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:           Mitigation; Enhanced IBRS; IBPB conditional; RSB filling; PBRSB-eIBRS SW sequence; BHI Syscall hardening, KVM SW loop
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Vulnerable: Clear CPU buffers attempted, no microcode; SMT Host state unknown
Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves arat avx512_vnni md_clear arch_capabilities

Versions of relevant libraries:
[pip3] numpy==1.26.4
[pip3] nvidia-nccl-cu12==2.20.5
[pip3] torch==2.3.1
[pip3] torchvision==0.18.1
[pip3] transformers==4.43.3
[pip3] triton==2.3.1
[conda] numpy                     1.26.4                   pypi_0    pypi
[conda] nvidia-nccl-cu12          2.20.5                   pypi_0    pypi
[conda] torch                     2.3.1                    pypi_0    pypi
[conda] torchvision               0.18.1                   pypi_0    pypi
[conda] transformers              4.43.3                   pypi_0    pypi
[conda] triton                    2.3.1                    pypi_0    pypi
ROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: 0.5.3.post1
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
GPU0    CPU Affinity    NUMA Affinity   GPU NUMA ID
GPU0     X      0-3     0               N/A

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

🐛 Describe the bug
I'm running the PaliGemma example code with a detection prompt:
from vllm import LLM
from vllm.assets.image import ImageAsset

def run_paligemma():
    llm = LLM(model="google/paligemma-3b-pt-448")

    prompt = "detect stop sign"

    image = ImageAsset("stop_sign").pil_image
    outputs = llm.generate({
        "prompt": prompt,
        "multi_modal_data": {
            "image": image
        },
    },
    )

    for o in outputs:
        generated_text = o.outputs[0].text
        print(generated_text)

if __name__ == "__main__":
    run_paligemma()

I ran this code with both the pt and mix models: google/paligemma-3b-pt-448 and google/paligemma-3b-mix-448 but it is either failing to generate any output or returning incorrect outputs:
Output for google/paligemma-3b-pt-448: ""
Output for google/paligemma-3b-mix-448: "GHFW"
In the HF implementation the extra loc tokens are added to the Gemma processor:
https://github.com/huggingface/transformers/blob/25245ec26dc29bcf6102e1b4ddd0dfd02e720cf5/src/transformers/models/paligemma/processing_paligemma.py#L38
I suspect it may have to do with this?

Thanks for reporting! I will take a look at this issue.

Hey! FWIW - I manually added the extra tokens and used the set_tokenizer method to update the tokenizer. The output changed, but it seems that there's still something wrong with it. It does not generate the extra detection tokens.

+1 for the same issue. I use paligemma-3b-mix-448 and it always predict empty results in detect and segment tasks.

Hey I just want to let you know this is still on my list of TODO but I simply had limited bandwidth with other priorities. Sorry for the inconvenience!

Hey @ywang96 - no worries :) I understand. Thanks for the heads-up.
I was actually taking some time today to debug this but I haven't found anything yet. I'll let you know if I do!

@ywang96 - I tried to debug the code a little further but had no luck.
As I said, I think special  tokens are an issue, but I believe there's something else.
FWIW - here're some thoughts:

the HF implementation uses GemmaForCausalLM as the language model, and I noticed that the vLLM implementation uses GemmaModel, not GemmaForCausalLM:

PaliGemmaForConditionalGeneration(
  (vision_tower): SiglipVisionModel(
    (vision_model): SiglipVisionTransformer(
      (embeddings): SiglipVisionEmbeddings(
        (patch_embedding): Conv2d(3, 1152, kernel_size=(14, 14), stride=(14, 14), padding=valid)
        (position_embedding): Embedding(256, 1152)
      )
      (encoder): SiglipEncoder(
        (layers): ModuleList(
          (0-26): 27 x SiglipEncoderLayer(
            (self_attn): SiglipSdpaAttention(
              (k_proj): Linear(in_features=1152, out_features=1152, bias=True)
              (v_proj): Linear(in_features=1152, out_features=1152, bias=True)
              (q_proj): Linear(in_features=1152, out_features=1152, bias=True)
              (out_proj): Linear(in_features=1152, out_features=1152, bias=True)
            )
            (layer_norm1): LayerNorm((1152,), eps=1e-06, elementwise_affine=True)
            (mlp): SiglipMLP(
              (activation_fn): PytorchGELUTanh()
              (fc1): Linear(in_features=1152, out_features=4304, bias=True)
              (fc2): Linear(in_features=4304, out_features=1152, bias=True)
            )
            (layer_norm2): LayerNorm((1152,), eps=1e-06, elementwise_affine=True)
          )
        )
      )
      (post_layernorm): LayerNorm((1152,), eps=1e-06, elementwise_affine=True)
    )
  )
  (multi_modal_projector): PaliGemmaMultiModalProjector(
    (linear): Linear(in_features=1152, out_features=2048, bias=True)
  )
  (language_model): GemmaForCausalLM(
    (model): GemmaModel(
      (embed_tokens): Embedding(257216, 2048, padding_idx=0)
      (layers): ModuleList(
        (0-17): 18 x GemmaDecoderLayer(
          (self_attn): GemmaSdpaAttention(
            (q_proj): Linear(in_features=2048, out_features=2048, bias=False)
            (k_proj): Linear(in_features=2048, out_features=256, bias=False)
            (v_proj): Linear(in_features=2048, out_features=256, bias=False)
            (o_proj): Linear(in_features=2048, out_features=2048, bias=False)
            (rotary_emb): GemmaRotaryEmbedding()
          )
          (mlp): GemmaMLP(
            (gate_proj): Linear(in_features=2048, out_features=16384, bias=False)
            (up_proj): Linear(in_features=2048, out_features=16384, bias=False)
            (down_proj): Linear(in_features=16384, out_features=2048, bias=False)
            (act_fn): PytorchGELUTanh()
          )
          (input_layernorm): GemmaRMSNorm((2048,), eps=1e-06)
          (post_attention_layernorm): GemmaRMSNorm((2048,), eps=1e-06)
        )
      )
      (norm): GemmaRMSNorm((2048,), eps=1e-06)
    )
    (lm_head): Linear(in_features=2048, out_features=257216, bias=False)
  )
)

I wonder if the hf_config.bos_token_id should be added to new_token_ids. Currently, the implementation has:

https://github.com/vllm-project/vllm/blob/6885fde317433eec52e00c14329270d742f0630d/vllm/model_executor/models/paligemma.py#L112
Perhaps it should be this:
new_token_ids = image_token_ids_pad + [hf_config.bos_token_id]+ orig_prompt_ids + [108]

Hope this effort to find something helps, somehow! :)

Hey @ywang96 - just wanted to leave a little bump here to see if there's been any progress?

@ywang96 - hey, left these comments on the discord:
https://discord.com/channels/1151009178698129408/1153200482140049498/1293351798093905971
https://discord.com/channels/1151009178698129408/1153200482140049498/1294026731107385506


swift generated code optimiziations / snapshot additional code bloat

when I generate code from querymodel
apollo-codegen generate corewatchlist.graphql --schema schema.json --output API.swift
All files are here ->
https://gist.github.com/johndpope/7cc331ad7182fb5ed112738500af5383
the resulting file is > 1MB.
I get snapshot code adding
public var snapshot: Snapshot
this happens in almost 600 occurrences

I'm thinking using the typed models maybe better.
Using the shopify code I get > 100 classes - but file size is tighter all up - 705KB
https://gist.github.com/wwe-johndpope/eff04d5297024f8570174a52a25e7c4f
I was thinking if the snapshot was for testing??? maybe it could be optionally added or excluded?
just a thought

I'm not entirely happy with the generated code either, but the snapshots are a necessary evil for allowing mutable data with type- or fragment-specific projections.
You can do something like:
hero.asDroid?.primaryFunction = "Foo"

or
hero.fragments.heroDetails.name = "Bla"

for example, and you expect that the updated values of primaryFunction or name are stored as part of hero. So that's why the generated code relies on a snapshot to store the actual values of fields, instead of using stored properties in the struct.

thanks for explanation @martijnwalraven

it's generally accepted that you can't / shouldn't store properties in extensions.
but if you could have the snapshot stored in the super
Graphql / this would alleviate having all the child nodes needing this property.
this would shed all this extra code.
public var snapshot: Snapshot
  public init(snapshot: Snapshot) {
    self.snapshot = snapshot
  }

a while back - there was some developers who did this via objc code.
the other idea is that for each model - this extra bloat could be saved separately as an extension.
eg.
AddToWatchlist.swift
would contain bare stuff
AddToWatchlist+Snapshot.swift
would contain extra code needed to do all snapshot stuff
extension AddToWatchlist{
}

I'm not sure I understand your proposals. The generated types are structs, so there is no superclass. And the snapshot is a stored property, so it can't be in an extension. All other properties are computed properties. The snapshot is no 'bloat', but a workaround to allow for consistent operations on type- or fragment projections.

Going to close this because it doesn't seem actionable, but feel free to reopen if you want to discuss further.

For reference/ https://marcosantadev.com/stored-properties-swift-extensions
In the > 10,000 lines of code there’s something like > 500 snapshot instantiations. This objc  hack could drastically simplify this. Consider.

Swift structs don't use the Objective-C runtime, so associated objects are not an option. But even if it was, it probably wouldn't make sense to use it because all getters and setters need access to snapshot, and making it an associated object would slow that down.
I don't really see the issue here. Code duplication isn't a problem because it's all generated code.


JSON_EXTRACT example not working on DBeaver mysql version 8.0.33

Description
I have tried executing a simple JSON command in DBeaver but it doesn't work. I have veriefed that it works in phpMyAdmin cPanel MySQL editor and it works. I have also reached out to the StackOverflow community who have helped me verify that it looks like a bug in DBeaver, plesae see https://stackoverflow.com/questions/76380997/json-extract-example-not-working-on-dbeaver-mysql-version-8-0-33
DBeaver Version
Community 8.0.33
Operating System
Ubuntu on Digital Ocean
Database and driver
com.mysql.cj.jdbc.Driver
Driver for MySQL 8 and later
mysql-connector-java  8.0.29
Steps to reproduce
I'm trying to get JSON_EXTRACT to work on a Digital Ocean droplet, executing
select version();
gives me 8.0.33-0ubuntu<IP_ADDRESS>
And I'm trying to use the example from the documentation that says it should work https://dev.mysql.com/doc/refman/8.0/en/json-search-functions.html#function_json-contains
but the code I pasted from the example:
SET @j = '{"a": 1, "b": 2, "c": {"d": 4}}';
SET @j2 = '1';
SELECT JSON_CONTAINS(@j, @j2, '$.a');
returns NULL
Additional context
No response

Hello @MarcusR1
Please, provide clear steps to reproduce your example.
For now, I can not reproduce it, unfortunately.

Hi Anastasiya,
Did you read the StackOverflow post? There are screen shots from my DBeaver, and RiggsFolly (https://stackoverflow.com/users/2310830/riggsfolly) also tried it and it failed for him too. If you need logs from DBeaver, please send me instructions as to how to get them.
Best regards, Marcus
From: Anastasiya @.>
Sent: Monday, June 5, 2023 3:51 PM
To: dbeaver/dbeaver @.>
Cc: Marcus Roskill @.>; Mention @.>
Subject: Re: [dbeaver/dbeaver] JSON_EXTRACT example not working on DBeaver mysql version 8.0.33 (Issue #20201)
Hello @MarcusR1https://github.com/MarcusR1
Please, provide clear steps to reproduce your example.
For now, I can not reproduce it, unfortunately.
[2023-06-05 17_49_40-Window]https://user-images.githubusercontent.com/45152336/243389561-dfc4c18c-1484-4d61-b4f8-bee1e8c63caa.png
—
Reply to this email directly, view it on GitHubhttps://github.com/dbeaver/dbeaver/issues/20201#issuecomment-1576949091, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ASPF7E6IRIM4STCAV7EDAITXJXW5NANCNFSM6AAAAAAY2VW47M.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>

Here's a video, the logs don't show anything.

But you didn't execute "set" statements first. Execute button only executes a current query.
You need to execute "set" statements before your select.

Please let me know if executing the following set of statements as a script with Execute Script button or sequantialy executing them one-by-one with Execute SQL query button helps.

Execute Script gives the correct response of 1
My apologies, I was not executing the whole script as one.

Thank you for update