Multiple images integration

Type of the feature

[ ] New function in sqduipy.im?
[ ] New function in squidpy.gr?
[ ] New function in squidpy.pl?
[ ] Change an existing functionality, such as default behavior?
[ x] Other?

Description

Different from scRNA data that multiple dataset can be analyzed and visualized at the same time. In imaging data, usually there will be multiple images from multiple patients/mice and there could be multiple duplicates for one case. It would be nice squidpy can account for that multiple FoV for feature enrichment and spatial analysis.

I would add my support for this request.
So far I work around the issue by merging segmented single cell data from different images for grouped analysis, splitting to adata image subsets for individual image analysis with cluster annotations from the large group analysis, then remerging to collate seperate spatial analyses

Hi @YubinXie and @mezwick
Thanks for your comments! I agree that it is important to consider analysing multiple images at once.
We have just finished implementing first support of image z-stacks in the ImageContainer (#329, now merged to dev). This means that images can now have dimensions x,y,z,channels. This is particularly useful if you have registered images that you would like to e.g. interactively visualise (which you can now do using Napari).
Processing and feature extraction functions have been updated to also consider the additional z dimension. As of now, we are limiting the feature extraction functions to only work on one z dimension at a time, but it is quite easy to extend this with a custom feature extraction function to also use multiple z dimensions.
We are now working on updating our tutorials to show how to work with these new changes (see also here: https://github.com/theislab/squidpy_notebooks/pull/64).
Please let me know if this z-stack extension already helps with your particular data, or if you need additional functionalities!
As we don't have a lot of 3D / z-stack spatial datasets that contain images, we are keen to hear from others what their needs are!
For spatial analysis using the spatial graph; if you have true 3D data (e.g. consecutive tissue sections), the graph functions should be able to work with 3D coordinates in adata.obsm['spatial']. Again, we will be adding an example for this very soon.

@hspitzer thanks for the update!
I'm working mainly with imaging mass cytometry type data, where we usually have 20-50 channels per image depending on the experiment, so this should be useful there. As an addition, it would be useful to have a way to refer to these channels by channel names (for instance the isotope linked to that channel, or the marker the isotope-tag in that channel is targeted to). I'd image channel_names might even be stored in adata.var[channel_name]. I usually achieve set this up for my own analysis with an xarray (perhaps there lies the answer).
A larger suggestion would be the 'multiple images' (and what i think @YubinXie was also getting at) mentioned above. For IMC and similar high plex technology studies where many individuals or many conditions are imaged, we end up with multiple images from which we would segment cells before pooling to an adata object for 'single-cell' analysis on the whole cohort. The way squidpy is set-up at the moment (maybe i'm mistaken), it seems only a single image stack can be stored in spatial. It would be useful to store all images in a study (perhaps prohibitively large for some studies though). I don't think the answer to this problem is to add a new dimension as rarely do all images have the same x, y or even c dimensions. Perhaps they could be stored within spatial under some image_name_id type set up where that name is also matched to events in adata.obs['image_name_id'] . In that way, one could do things like refer to individual images to identify neighbours in a network, but across all your images in one go.
As i said, i'm semi-achieving this:

by merging segmented single cell data from different images for grouped analysis, splitting to adata image subsets for individual image analysis with cluster annotations from the large group analysis, then remerging to collate seperate spatial analyses

But it would be nice to have a way to store multiple images within a single squidpy adata object, and refer to them by some id.

Thanks for the elaborations @mezwick!

I'm working mainly with imaging mass cytometry type data, where we usually have 20-50 channels per image depending on the experiment, so this should be useful there. As an addition, it would be useful to have a way to refer to these channels by channel names (for instance the isotope linked to that channel, or the marker the isotope-tag in that channel is targeted to). I'd imagine channel_names might even be stored in adata.var[channel_name]. I usually set this up for my own analysis with an xarray (perhaps there lies the answer).

Yes, when using adata, storing channel_name in adata.var is the preferred way of doing things. #343 proposes a similar schematic for the ImageContainer - lets continue this discussion in this issue.

A larger suggestion would be the 'multiple images' (and what i think @YubinXie was also getting at) mentioned above. For IMC and similar high plex studies where many individuals/conditions are imaged, we end up with multiple images from which we would segment cells before pooling to an adata object for 'single-cell' analysis on the whole cohort. The way squidpy is set-up at the moment (maybe i'm mistaken), it seems only a single image stack can be stored in spatial. It would be useful to store all images in a study (perhaps prohibitively large for some studies though). I don't think the answer to this problem is to add a new dimension as rarely do all images have the same x, y or even c dimensions. Perhaps they could be stored within spatial under some image_name_id type set up where that name is also matched to events in adata.obs['image_name_id'] . In that way, one could do things like refer to individual images to identify neighbours in a network, but across all your images in one go.

Ok, got it! Multiple images here means several images that have non-relating x and y coordinates.
We have designed the ImageContainer to only contain images with the same x,y coordinates. For utilising multiple images, we’d propose to use multiple ImageContainer. Theoretically, it should be no problem to extract “single-cell” measures from the images into the same adata object.
For the z-stack extension I mention above, we are storing a library_id column in adata.obs that allows relating observations (+ spatial coordinates) to specific images. This is essentially what you mention as well with image_name_id.
In your case, however, things might be a bit tricky, as the spatial coordinates from different images can’t be related to each other - meaning that you’d always have to subset to only one library_id before calculating spatial statistics.
It might be possible to make this a bit easier, by adding a library_id argument to the spatial graph function from Squidpy, which would subset the adata automatically, and save you the subsetting & merging that you currently do.
Pinging @giovp here, as I am not sure how feasible / sensible such an extension is.

Hi everyone! Thanks @mezwick for adding more explanation here. I am also analyzing imaging mass cytometry type data and usually the case is that, we have multiple FoV from the same and different patient. The analysis is usually on multiple images and single FoV usually does not provide enough information. I believe in general, in a context of most scientific paper, the needs for merging multiple conditions/batch for analysis are huge. It would be great to have a data system for imaging built for this. Thanks!

hi everyone, sorry for jumping late on this, very interesting discussion!
I'll give my take on these two bits (imho related) @YubinXie @mezwick and I believe is somewhat related to #318 .

It might be possible to make this a bit easier, by adding a library_id argument to the spatial graph function from Squidpy, which would subset the adata automatically, and save you the subsetting & merging that you currently do.
Pinging @giovp here, as I am not sure how feasible / sensible such an extension is.

The analysis is usually on multiple images and single FoV usually does not provide enough information. I believe in general, in a context of most scientific paper, the needs for merging multiple conditions/batch for analysis are huge. It would be great to have a data system for imaging built for this. Thanks!

As @hspitzer mentioned, so far the zstack is only for images with same y,x dim, in the case of different y,x(,c) then best would be to have a list of ImageContainer objects.
For the spatial graph functions (everything in squidpy.gr) this is not the case, as we treat "images" as "batches", and therefore can be stored in the same adata object. Notice that the low-res images (to be visualized statically with matplotlib) can also be stored in the same adata in adata.uns["spatial"] e.g.
adata.uns["spatial"]["img_1"] = ...
adata.uns["spatial"]["img_2"] = ...
...

however, I must say that the visualization would not work as smoothly, meaning that to visualize only one image you'd have to do this (note to self to modify this behaviour in scanpy).
sc.pl.spatial(adata[adata.obs.library_id=="img_1"], color="clusters", library_id = "img_1")

For the analysis side (again, everything in squidpy.gr), the only solution for now is to iterate over the library ids and store the results outside of anndata, see https://github.com/theislab/squidpy/issues/318#issuecomment-824319306
nhood_enrich_results = []
for batch in adata.obs.batch.unique():
	adata_copy = adata[adata.obs.batch == ... ]
	sq.gr.spatial_neighbors(adata_copy)
	res = sq.gr.nhood_enrichment(adata_copy, ..., copy=True)
	nhood_enrichment_result.append(res)

In principle, we could create a class/method that wraps this behaviour, i.e. that takes as input an Anndata and some arguments like the analysis to be performed and the analysis-specific args, and return aggregated results. something like
results = sq.gr.joint_spatial_analysis(adata, library_id = "library_id", tasks = [partial(sq.gr.spatial_neighbors), partial(sq.gr.nhood_enrichment, "clusters")], args)
results.shape
# (n_clusters, n_clusters, n_library_id)

and then users could decide how to aggregate them. But I am not super sold it is something really necessary. I wonder if maybe a tutorial showcasing this usage would be enough?
Really curious to hear your inputs

Hi @giovp Thanks for the thoughts!
If I think about the ideal situation, I would think that we have batch as required a col in obs for spatial data. And when we plot images, or do analysis, the users will have options to do on all images, or on one given batch. This is consistent with most main stream count matrix for multiple images data, where we usually pool everything together in one count matrix with batch annotated.
To link to adata, I would assume we can make batch info required in the adata if it is spatial info, and then add this 'batch' as a key in uns, and all other information as you indicated. To make things easier, the default plot, and computation can be on the first image, and the user will have the option to use specific image, and all images. For single image, the batch name will be something default and this new feature then will also work for single image case.
As for the ploting issue you mentioned, I usually plot them in a array. For example, if I have 30 images, I plot the results in a 5*6 subplots. It allows me to see all the information. I am currently putting all the adata into a list and plot one by one (actually I found a dictionary with key and adata is best). it works but not the most elegant way to do it I think.
The calculation of spatial_neighbors can be done on each image, but this loop process can be hidden in squidpy. Ideally the user only needs to give a list of batch name that they want to run.
An additional thing for this multiple image is that we could use a batch information table. For example, one could aggregate each image's immune cell count, tumor average KI67 expression et al... But of course, this is easier to do outside scanpy squidpy once the adata is mature on multiple images.
Thanks again for brainstorming and really appreciate your input. We have something to publish very soon and I found sq being great for quick spatial analysis!

hi @YubinXie ,
thanks a lot for the reply in detail. I think this is an interesting API suggestion that is worth considering. I agree that a "batch" key that would slice the anndata according to the "slides" would be very useful.
The biggest hurdle I see now is the plotting function, as we are still relying on the Scanpy one and therefore makes it a bit complicated to allow this flexibility. I would try to draft a new one in the coming future to allow for such behaviour and keep you posted here.

the plotting aspect of this will be available when #437 will be merged. I wont' link it though cause there is the larger issues on making all sq.gr methods working on multiple slides settings (probably not too difficult but for sure long).


Add 3rd party licenses

Mentions for used 3rd party libraries

Do this for main folder, front and back


Does anyone are having errors from Box API this February 2020?

I can't find the error, but seems like Box made an update or something that makes some requests break. Sometimes its like the generated token to authenticate to Box is not recognized as a recent and valid token. Anyone with this same issue?

Hello, we've been seeing intermittent token issues in the last couple weeks too. It seems to be a result of the getToken method failing to be issued a token in some cases. I debugged by just outputting the $result value to the screen. It was showing a 302 redirect so I updated the curl command with -L to have it follow the redirect which resulted in me seeing the actual error message.
Here it is:
{"error":"invalid_request","error_description":"Invalid grant_type parameter or parameter missing"}
I'm baffled at this point, but will keep debugging. Please let me know if you run across any leads.

@ecuation I got a suggestion from Box support to replace https://www.box.com/api/oauth2/token in BoxAppUser.php (set as $token_url) with https://api.box.com/oauth2/token.
I did this and then ran through about 20 requests and got a token every time and no errors.
https://community.box.com/t5/Platform-and-Development-Forum/Intermittent-quot-invalid-request-quot-when-requesting-token/m-p/81797/highlight/false#M7744
Followed up with Box support to see if there was a recent change that may have caused this. Anyway, hope this solves your issue as well.

Hi @kaypro4 I just read your last messages. Finally seems like that is the solution, I got but I don't know how... I was that day working more than 12 hours trying to find a fix and also Box didn't gave me a solution. So if someone else experiences that annoying issue, yes, that is the solution. Thanks!


switch to kdl's fork of expressiongraph

Pull request was merged. Now, we could switch to the original fork. Test if everything still behaves.

Done.


Verification

Hello, I am the author of the Core Lightning Plugin for Stable Channels, peer-to-peer US dollar balances on the Bitcoin Lightning Network.
I have spoken about this work at the Open Source Stage in Bitcoin Atlantis, and also at Nashville BitDevs.
https://github.com/toneloc/stable-channels
Thank you.

Here is some more info I put for speaker application :)
Hello, I am Tony Klausing and I am a software developer working on Stable Channels, peer-to-peer dollar balances on the Bitcoin Lightning Network.
This is an open-source project to do "stablecoins" better on Bitcoin -- with no token, no fiat, and no banks.
I have spoken about this work this year at Bitcoin Atlantis and Nashville BitDevs.
Given the tremendous growth of stablecoins, the audience may be interested to hear how we can use Bitcoin for the stablecoin use-case.
I would start my presentation with an overview of stablecoins and their different approaches.
After working in TradFi, but as a longtime Bitcoiner, I am pleased to now work on Bitcoin full-time.
Here is a Tweet thread that sums up Stable Channels - https://x.com/tonklaus/status/1729567459579945017
Thank you for your time.
Tony


[I256] Exponentiation. (5)

Implement exponentiation operations.
Test Plan
New unit tests

This looks awesome! Why was it closed!? I love exponentiation!

I merged all the 256-bit signed integer support in 1 PR, as I figured it didn’t make much sense to have partial support for it but it was too much code to review all at once.
Here it is: https://github.com/gnosis/ethcontract-rs/pull/231

Right, funny I was just looking at that one! So what does this mean for projects that already depend on this repo?

It means they can now do signed integer math! 🥳

Right, but wasn't this already a possibility with primitive types available in parity-ethereum?

Right, but wasn't this already a possibility with primitive types available in parity-ethereum?

No it only supported unsigned integers.


I replace 'i' with 'k', but using mini.ai, set inside = 'k', will overwrite my own key maps

Contributing guidelines

[X] I have read CONTRIBUTING.md
[X] I have read CODE_OF_CONDUCT.md
[X] I have updated 'mini.nvim' to latest version

Module(s)
Mini.ai
Description
I use Colemak keyboard layout, so I always map k to i when I use neovim/vim,
using k for insert and inside operator prepend.
Neovim version
0.82
Steps to reproduce

my keymaps

vim.keymap.set("", "k", "i")

set inside = "k" when I setup mini.ai

require("mini.ai").setup({
  mappings = {
    -- Main textobject prefixes
    around = 'a',
    inside = 'k',

    -- Next/last variants
    around_next = 'ae',
    inside_next = 'ke',
    around_last = 'au',
    inside_last = 'ku',
 }
})

Expected behavior
Works both mini.ai keymaps and my own keymaps
Actual behavior
It works when I type ckf, but not works when I type ckw.
I read Mini.ai source code, I think, mini.ai overwrote my mapping for 'k'.

Thanks for the issue! Never really looked into different keyboard layouts, so didn't think about this use case.

I read Mini.ai source code, I think, mini.ai overwrote my mapping for 'k'.

Overriding mappings for a and i is the whole point of this plugin. The issue here is that iw is not a 'mini.ai'-defined textobject, so it falls back to being equivalent to typing iw, which is not correct in your user case.
I'll see what I can do here.

Have you tried using 'langmap' instead of key mapping? It seems like a more appropriate solution for your use case.
So instead vim.keymap.set("", "k", "i") try vim.o.langmap = 'k;i,i;k'.

wow, Thanks so much, I dont even know about langmap, It's amazinig, I learned a lot

I've just added tests for this use case. Having set langmap=ki,ik seems to solve this use case with default 'mini.ai' config.
Thanks for the issue!

I tried this solution.

I mapped "i" to "l", because I need to use "i" to move the cursor to the right.

I use vim.o.langmap = 'k;i,i;k'
"i" will move the cursor up

Modify langmap to vim.o.langmap = 'k;i'
"i" moves the cursor to the right, but "k" doesn't go into insert mode, "k" also moves the cursor to the right

add keymap: vim.keymap.set({ "n", "x", "o" }, "k", "i")
"k" still can't get into insert mode

I tried this solution.
...
2. I use vim.o.langmap = 'k;i,i;k'
"i" will move the cursor up

Sorry, I do not understand the problem here. With vim.o.langmap = 'ki,ik' keys "i" and "k" are essentially swapped: use "i" to go up and "k" for all "regular" "inside" textobjects (like vk), dkw, etc.). This works and does not need any more additional mappings.

The problem is that colemak's i is qwerty's l, and I need to keep the neovim convention, so I need to map i to l.
At the same time, I needed a key instead of i, for entering insert mode and text objects, and I chose the k key.
The current mapping situation is as follows:
vim.keymap.set({ "n", "x" }, "i", "l")
vim.keymap.set({ "n", "x", "o" }, "k", "i")

So my scenario involves three keys, i, l, k, and I cannot simply swap k and i.
The implementation principle of mini.ai is to remap inside, which will theoretically conflict with my keys.
Maybe only compatible from within.

The problem is that colemak's i is qwerty's l, and I need to keep the neovim convention, so I need to map i to l.
At the same time, I needed a key instead of i, for entering insert mode and text objects, and I chose the k key.

I am not sure that manually choosing what keys to remap is a good idea here. I would assume that the better approach is to remap the whole keyboard layout altogether. For example, judging by Wikipedia article, a "key instead if i" is "u".
Here is an article with 'langmap' example. With those remappings, pressing "u" on Colemak will have the same effect as "i" in QWERTY. So, for example, to delete inside word ("diw" in QWERTY) press "suw".
This is the only way 'mini.ai' will support different keyboard layouts (outside of already exported config.mappings).

The problem is that colemak's i is qwerty's l, and I need to keep the neovim convention, so I need to map i to l. At the same time, I needed a key instead of i, for entering insert mode and text objects, and I chose the k key. The current mapping situation is as follows:
vim.keymap.set({ "n", "x" }, "i", "l")
vim.keymap.set({ "n", "x", "o" }, "k", "i")

So my scenario involves three keys, i, l, k, and I cannot simply swap k and i. The implementation principle of mini.ai is to remap inside, which will theoretically conflict with my keys. Maybe only compatible from within.

custom_textobjects = {
    w = { "()()%f[%w]%w+()[ \t]*()" },
},

this might help, only kw overwrite keymap

If it is a custom textobjects, there will not be fall back to builtin a/i textobjects

The problem is that colemak's i is qwerty's l, and I need to keep the neovim convention, so I need to map i to l. At the same time, I needed a key instead of i, for entering insert mode and text objects, and I chose the k key. The current mapping situation is as follows:
vim.keymap.set({ "n", "x" }, "i", "l")
vim.keymap.set({ "n", "x", "o" }, "k", "i")

So my scenario involves three keys, i, l, k, and I cannot simply swap k and i. The implementation principle of mini.ai is to remap inside, which will theoretically conflict with my keys. Maybe only compatible from within.

custom_textobjects = {
    w = { "()()%f[%w]%w+()[ \t]*()" },
},

this might help, only kw overwrite keymap

Yes, it works


Add this repo to the overlay list

Would be nice. I will attempt to make a PR once https://github.com/kir68k/xira/issues/4 is closed.

I think I'm gonna try now, despite #4 not being finished.

Done, in gentoo/api-gentoo-org@88e1f3b34e256e2aaead97cbc189fc36fe0c5c46.


Diagram stuck at 50%

When scaling up to a MG with multiple SUs, diagram run gets stuck at 50%. Works fine with fewer SUs.
Can't figure this one out. Any idea?

Hello,
Is this a big environment?
The diagram is done by a single powershell job, by my experience in bigger environments it takes a long time to finish, but in really large environments it might get stuck, I haven't figured why.
We are considering some options to work this out in the new version of the script, perhaps creating separate files during the diagram and then merging them at the end. But for now we don't have a workaround for that problem :(

Ok, thanks for the quick reply on this.
Not that big, some thousands of resources in total. I'll play around and see if I can find a limit.

Hello,
Just release the version 3.0, now the diagram should work in that environment.
Can you please try that version and let me know?
Thanks

Verified and functionality works now. Even with quite large environment.
However, I have encountered another issue with new Organization view.
Diagram doesn't seem to populate all subscriptions correctly if mgmt. group structure pass 4 levels of depth. Works only up to 3 levels.
Example 1
Root(0) -> LZ(1) -> Corp(2) -> PurposeY(3) -> SubscriptionY
SubscriptionY does not get populated on diagram.
Example 2
Root(0) -> LZ(1) -> Corp(2) -> SubscriptionX
Subscription X gets populated correctly.
Data do exists on report.

Hi, Glad it worked!
About the Organizational view, by now it only goes till level 3, you are correct.
I'll see If I can improve that in the following weeks.

Hello,
Organizational view should cover 4th depth level now.
Can you try?

Hello @cladaw,
Organizational view should cover 4th depth level now.
Can you try?

Subscriptions are being populated on the diagram, but error with properties while adding subscription name on container4

Hello @cladaw,
That was a bug, thanks for testing. It should be fixed now, can you try?

Verified, works now. Much appreciated for the quick fix!


RBAC migration ignores command line db argument

With commit b8cb5e2a28ccbd60235eeded97838cc75063a386, the RBAC migration script will always use the db configured in the authManager, ignoring any command line argument like yiic migrate --migrationPath=@yii/rbac/migrations/ --db=testdb.
This probably won't be a widespread issue, and a workaround to temporarily change the authManager config db is easy enough, but if this will continue to be the behavior of the script, a note in the documentation about the inconsistency from the regular behavior might save others some time down the road.

It uses db of the target. You can have multiple loggers configured and each could use its own connection so migration should be applied to connection specified in the target.

Why do you want to override it?

@samdark I ran into this while trying to apply the migration to my test database.  In the console configuration, I have my production database in 'db', test database in 'testdb', and have also included authManager, which doesn't define a db, and is therefore using the default db setting.  For any of my own migrations, I just append --db=testdb to the yii migrate command, and expected this particular migration to obey the db parameter.  When I saw that there was a db customization in authManager, it became clear why the db parameter was being ignored, but it was a little confusing at first since it completely disregarded my override.  Since the migration is strictly rbac-related, it would seem to make more sense to use the command line parameter if it was passed in instead of always using the one in the configuration.
On a more general note, if this isn't the suggested method for migrating non-production databases from the command line, please let me know, as this seemed to have been a fairly acceptable pattern with 1.1.  (If there is a more acceptable pattern, it may also be worth it to add to the docs, assuming I didn't miss it somewhere.  Off the top of my head, it almost seems like a split yii and yii-test for the console may be a better method?)
Thanks!

https://github.com/yiisoft/yii2-app-advanced/blob/master/tests/codeception/bin/yii
try /path/to/tests/codeception/bin/yii migrate

Sounds like a misuse of dev/production environments, otherwise why would you need to connect to a 'production' DB from a dev environment?

Agreed that the codeception/bin/yii is the way to go here - I just hadn't dug into that code deep enough when I submitted this issue.


Create LICENSE file

Would you please consider creating a LICENSE or LICENSE.md file with your copyright and the text of the MIT license?  The MIT license requires that the license text accompany the source code.
Also, could you also add this license file to your defence.js repo as well?  I would have opened a separate issue there, but issues appear to be disabled for that repo.

Are you repackaging something else that uses defence and replace-require-self?

Added in 6649daf076847f67fa8a0867d5968da85e3af9c9

Thank you.  I'm packaging up the "validate-npm-package-license" module for Fedora and CentOS as a dependency of npm, and it depends on defence and replace-require-self.

Does Fedora package development dependencies as well as functional dependencies as a matter of policy?

Yes, they do.

Thanks.


Exporting Jupyter notebook to python script includes notebook artifacts

Issue Type: Bug
Open a Jupyter notebook, choose "Export" -> "as Python script". The exported *.py script includes notebook specific artifacts like "Run Cell", "Debug Cell" etc. in the script. It should ignore such notebook commands and artifacts and export only the python command blocks plus the markdowns. It'll be nice to be able to toggle export of markdowns as a stand-alone setting though.
VS Code version: Code 1.58.1 (2aeda6b18e13c4f4f9edf6667158a6b8d408874b, 2021-07-13T06:58:28.115Z)
OS version: Windows_NT x64 10.0.19043
Restricted Mode: No

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-6600U CPU @ 2.60GHz (4 x 2808)

GPU Status
2d_canvas: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onoop_rasterization: enabledopengl: enabled_onrasterization: enabledskia_renderer: enabled_onvideo_decode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
15.92GB (9.10GB free)

Process Argv
--crash-reporter-id 5e461d3c-9837-4ac7-9413-c948a9378407

Screen Reader
no

VM
0%

Extensions (5)

Extension
Author (truncated)
Version

gc-excelviewer
Gra
3.0.42

python
ms-
2021.6.944021595

vscode-pylance
ms-
2021.7.3

jupyter
ms-
2021.8.1013163132

open-in-browser
tec
2.0.0

A/B Experiments
vsliv368:30146709
vsreu685:30147344
python383:30185418
pythonvspyt602:30300191
vspor879:30202332
vspor708:30202333
vspor363:30204092
vswsl492cf:30256860
pythonvspyt639:30300192
pythontb:30283811
pythonvspyt551:30311712
vspre833:30321513
pythonptprofiler:30281270
vshan820:30294714
vstes263cf:30335440
pythondataviewer:30285071
vscus158:30321503
pythonvsuse255cf:30335482
vscod805cf:30301675
pythonvspyt200cf:30331938
vscextlangct:30333562
binariesv615:30325510
bridge0708:30335490

Thanks for filing this issue, we'll discuss this in our triage meeting.

Had discussed this with @greazer sometime last year...
Idea: We plan to move the export functionality to the file menu items so I think it makes sense that that method of exporting does not include any of the above mentioned items. We could introduce a different interaction to get from nb -> interactive window

thanks for the suggestion, unfortunately this hasn't generated enough interest for us to prioritize this feature


Prevent MMD shapekeys being deleted

I think it will be good if MMD shapekeys could be kept after optimization.

If you can point me to a list of all MMD shapekeys I could add an option to keep them. A quick google search refused to give me any kind of list of MMD shapekey names.

You could get a full list here: https://www.deviantart.com/xoriu/art/MMD-Facial-Expressions-Chart-341504917
MMD shapekeys name will be something like "まばたき", "笑い" or "Blink", "Smile" or both of them
BLUE - EYES
まばたき - Blink
笑い - Smile
ウィンク - Wink
ウィンク右 - Wink-a
ウィンク２ - Wink-b
ｳｨﾝｸ２右 - Wink-c
なごみ - Howawa
はぅ - > <
びっくり - Ha!!!
じと目 - Jito-eye
ｷﾘｯ - Kiri-eye
はちゅ目 - O O
星目 - EyeStar
はぁと - EyeHeart
瞳小 - EyeSmall
瞳縦潰れ - EyeSmall-v
光下 - EyeUnderli
恐ろしい子！ - EyeFunky
ハイライト消 - EyeHi-off
映り込み消 - EyeRef-off
喜び - Joy
わぉ?! - Wao?!
なごみω - Howawa ω
悲しむ - Wail
敵意 - Hostility
PINK - MOUTH
あ - a
い - i
う - u
え - e
お - o
あ２ - a 2
ん - n
▲ - Mouse_1
∧ - Mouse_2
□ - □
ワ - Wa
ω - Omega
ω□ - ω□
にやり - Niyari
にやり２ - Niyari2
にっこり - Smile
ぺろっ - Pero
てへぺろ - Bero-tehe
てへぺろ２ - Bero-tehe2
口角上げ - MouseUP
口角下げ - MouseDW
口横広げ - MouseWD
歯無し上 - ToothAnon
歯無し下 - ToothBnon
GREEN - BROWS
真面目 - Serious
困る - Trouble
にこり - Smily
怒り - Get angry
上 - UP
下 - Down

Ok, so the option should keep both the JP and EN names, right?
Thanks for the list!

Yes, I think it would be great if both JP and EN blendshapekeys are kept. By the way, love your tool very much. It helps a lot.

This is now live in 2.1.0


Fix e2e test and add openshift-ci make target

Motivation

Upstream modelmesh fvt test is not running with odh-modelmesh.
Moreover, odh-modelmesh need to add this fvt test to openshift-ci as E2E test to check every PR validation.

Modifications

Add openshift-ci make targets
Increase timeout for ISVC deployment from 120s to 600s
Change ETCD image from coreos to registry.access.redhat.com.

Still, the image can be pulled without credential

Change --data-dir of etcd in order to avoid permission issues.

Test Steps
After you login to Openshift cluster, execute the following commands:
NAMESPACE=modelmesh-serving make e2e-test

Result
...
Ran 5 of 5 Specs in 79.927 seconds
SUCCESS! -- 5 Passed | 0 Failed | 0 Pending | 0 Skipped

Ginkgo ran 2 suites in 4m26.116938102s
Test Suite Passed

Merging manually as ci isn't currently functioning.


Refactor organization user sharing implementation and implement user sharing on sub-organization creation.

Purpose

$subject

Depends on

https://github.com/wso2/identity-organization-management-core/pull/86

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/6435267365

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/6435267365
Status: success

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/6621080982

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/6621440114

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/6621080982
Status: failure

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/6621440114
Status: failure


Can' t create volume snapshots

In an test environment using Trident 21.01.1 on k8s 1.18.14, we want to create volume snapshots on the k8s level. I followed the instructions here. Everything worked fine until I tried to create an actual snapshot from an existing PVC. Then the creation of the snapshot hangs indefinitely, the "ready to use" status is "false".
Depending on if I add a namespace at snapshot creation or not, I get the errors/events
Failed to create snapshot content with error snapshot controller failed to update data-bitbucket-0-snapshot on API server: cannot get claim from snapshot
or
Waiting for a snapshot bitbucket/data-bitbucket-0-snapshot to be created by the CSI driver.
I do also not find any other errors, except these messages mentioned above. Also the logs of the snapshot-controller pod don't tell me anything useful.
Environment
Provide accurate information about the environment to help us reproduce the issue.

Trident version: 21.01.1
Trident installation flags used: -n trident
Container runtime: RHEL Docker 1.13
Kubernetes version: 1.18.14
Kubernetes orchestrator: Rancher 2.5.5
Kubernetes enabled feature gates:
OS: RHEL 7.9
NetApp backend types: ONTAP 9.7P11
Other:

To Reproduce
Steps to reproduce the behavior:

Install everything needed to create Kubernetes volume snapshots (see https://netapp-trident.readthedocs.io/en/stable-v21.01/kubernetes/operations/tasks/volumes/snapshots.html)
Create snapshot class
Create snapshot from existing pvc

Expected behavior
Snapshot gets created and is in "ready to use" state.

Hi, @roycec.  The only time I've seen cannot get claim from snapshot is when the VolumeSnapshot was in a different namespace from the PVC.

Hi, @roycec. The only time I've seen cannot get claim from snapshot is when the VolumeSnapshot was in a different namespace from the PVC.

How can I influence this? Afaik the pvcs are namespaced, the examples regarding snapshots in the documentation are without any reference to namespaces. Now the case is, that both pvc and the possible snapshot are within the same namespace, so I get the second message.

@roycec  Yes, all PVCs are namespaced, as are VolumeSnapshot objects.  New namespaced objects such as snapshots may specify their namespace, otherwise they wind up in the namespace from your current context.  Specify the namespace like this, where the snapshot namespace must match that of the PVC.
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
  name: pvc1-snapshot
  namespace: default
spec:
  volumeSnapshotClassName: trident-csi
  source:
    persistentVolumeClaimName: pvc1

@clintonk yes, this makes this a lot more clear, the docs could be a little more precise on this. But if I apply the YAML like this, I'm still stuck in the creation process of the snapshot itself
Events:
  Type    Reason            Age   From                 Message
  ----    ------            ----  ----                 -------
  Normal  CreatingSnapshot  37s   snapshot-controller  Waiting for a snapshot bitbucket/data-bitbucket-0-snapshot to be created by the CSI driver.

Ensure that your VolumeSnapshotClass includes driver: csi.trident.netapp.io.  Then check the logs for the csi-snapshotter & trident-main containers in the Trident deployment pod to see what might be happening.

yes, it does
apiVersion: snapshot.storage.k8s.io/v1beta1
kind: VolumeSnapshotClass
metadata:
  name: csi-snapclass-test
driver: csi.trident.netapp.io
deletionPolicy: Delete

In my case it turned out, that the backend was offline. Since it is back online again, the snapshots can be created successfully.


Updated GTFS analysis to address stop duplication across modes and provide more informative feed output as per #385

This is a patch to address particular concerns raised by @marcdmallafre via e-mail ("problem 1").  Marc, if you could trial this code out to see if it addresses this specific issue that would be appreciated.

Updated GTFS analysis to address stop duplication across modes and provide more informative feed output as per #385; also,
pandas future warnings re pd.concat ('The behavior of DataFrame concatenation with empty or all-NA entries is deprecated') were specifically filtered/ignored as this pattern is used heavily and a way of refactoring this is not currently obvious, and the warnings impede readable output for users.
re-factored gtfs_analysis() function to reduce output and code complexity

improved user print out display to include more meaningful information and guidance for GTFS analysis
reduce some code repetition, pulling out some code as sub-functions (added r.bbox attribute; functions fro get_frequent_stop_stats, get_average_headway, gtfs_to_db, load_gtfs_feed, get_frequencies_df).

As per review by @marcdmallafre in #385, I'll merge these changes into the Enhancements branch now.  Once that branch is subsequently updated with other changes related to Docker environment and template updates, it can be merged into main and #385 can be closed.


Update example package to use forked path.

The example package was still referencing github.com/getsentry/raven-go.
This means when vendoring, we wound up accidentally vendoring
github.com/getsentry/raven-go in addition to
github.com/DramaFever/raven-go. And since we don't particularly need
two copies vendored, I figured it made sense to just update this and
call it a day.

Out of curiosity, why do we have this fork at all? If we're doing vendoring, do we even need to use this?

lgtm

GREEN GOOD LETS DO IT


is there a FutureObj equivalent for streams?

I would like to write an API where I return a struct with a stream value inside of it. Is there a FutureObj equivalent for Stream?
use futures::stream::{ repeat, Repeat };
use std::future::{FutureObj};

struct Response<'a> {
    stream: FutureObj<'a, Repeat<u8>>
}

pub fn main() {
    let mut s = repeat(1);
    let _r = Response {
        stream: FutureObj::new(&mut s)
    };
}

errors with:
error[E0277]: the trait bound `futures::stream::Repeat<{integer}>: futures::Future` is not satisfied
  --> src/boop.rs:11:17
   |
11 |         stream: FutureObj::new(&mut s)
   |                 ^^^^^^^^^^^^^^ the trait `futures::Future` is not implemented for `futures::stream::Repeat<{integer}>`
   |
   = note: required because of the requirements on the impl of `std::future::UnsafeFutureObj<'_, _>` for `&mut futures::stream::Repeat<{integer}>`
   = note: required by `<std::future::FutureObj<'a, T>>::new`

error: aborting due to 2 previous errors

I can satisfy the FutureObj by transforming the Stream in to a Future but that changes the code a lot:
use futures::stream::{ repeat, StreamExt, Repeat }; //, Stream };
use std::future::{FutureObj};

struct Response<'a> {
    stream: FutureObj<'a, (Option<u8>, Repeat<u8>)>
}

pub fn main() {
    let s = repeat(1);
    let mut sf = s.into_future();
    let _r = Response {
        stream: FutureObj::new(&mut sf)
    };
}

@MajorBreakfast I'm open to some mentorship on this!
My first guess is to take what's in libcore/future/future_obj.rs and put it in to the futures-core-preview/src/stream.rs and change references for Future to Stream. Does that sound OK?

@xrl That's correct. I recommend cloning the rust repo and to search for "FutureObj". Aside from in libcore there are some other occurrences (definitely in liballoc and just maybe somewhere else as well)


I can't choose language.

I can't choose language.
hello，I want to choose language,but It's grey.

The language has been greyed out and cannot be chosen on web UI to prevent end-users from changing it.
To change it, please create a file at ~/.kettle/.languageChoice with the following content:
LocaleDefault=en_US

then change en_US with your preferred locale (e.g., zh-CN).

I revised kettle's index.html in Chinese, but it was scrambled. How can I solve it?

I tried your method,  LocaleDefault=zh_CN ，but still showed English.

My previous reply contains a typo.
wrong: LocaleDefault=zh-CN (hyphen)
correct: LocaleDefault=zh_CN (underscore)
But I don't think this typo affects you because seems like you already noticed it.
To make the change effective, please restart the webSpoon server.
Also make sure to create ~/.kettle/.languageChoice in the server side. If webSpoon is deployed as a Docker container, ~/.kettle/.languageChoice within the container should be created.
The scrambled index.html is a different problem.
I can take a look if you can share the index.html.

Thank you very much. I've settled it.

You're very welcome :)

use docker ps command  to get container id;
use docker inspect 5bb04abad45d command to get MergedDir
enter MergedDir and get in home/tomcat/.kettle directory;
create .languageChoise
restart container


Add support for recurring events

This one might be a bit trickier, but definitely cool to have

That would be nice indeed.
Recurring events are retrieved from the calendars in iOS (tested on an iPad), but not in Android (tested on a Galaxy S21).


Adding an autocmd TermClose in init.vim breaks the stabilization

Setup information
Using stabilize.nvim 174dfcd
~/.l/s/n/p/s/lua $ nvim --version
NVIM v0.7.0
Build type: Release
LuaJIT 2.1.0-beta3
Compiled by builduser

Features: +acl +iconv +tui
See ":help feature-compile"

   system vimrc file: "$VIM/sysinit.vim"
  fall-back for $VIM: "/usr/share/nvim"

Run :checkhealth for more info

Stabilize setup config:
require("stabilize").setup({
    ignore = {
		filetype = { "help", "list", "Trouble" },
		buftype = { "quickfix", "loclist" }
	},
	nested = nil
})

Description
First of all I haven't tried this issue with a minimal init.vim, because the issue does not appear as soon as I remove TermClose autocmd from my init.vim.
Basicaly I use autocmd TermClose * :q to automatically close the terminal, because I don't want to see the 'process exited ...' text every time I exit the shell.
After messing around I realized that removing that autocmd makes everything work again.
Here is a video I recorded of the issue:
https://user-images.githubusercontent.com/83361786/174862590-e224c068-732b-4288-9ac6-61bf227a0d20.mp4
The smaller window is :CocDiagnostics mapped to <C-i><C-d>, the one with the shell is a terminal: nnoremap <silent> <C-d> :sp\|te<CR> As the video shows sometimes the stabilization works, sometimes not. After removing the TermClose autocmd everything works reliably.
Please let me know if some additional information is needed.

Adding nested = "TermClose" to the plugins config should fix the issue.

I have tried that before, but thought that shouldn't be used there, because I get the following error on TermClose:
Error detected while processing TermClose Autocommands for "*":
Error executing lua callback: ...ocal/share/nvim/plugged/stabilize.nvim/lua/stabilize.lua:112: attempt to call field 'nvim_exec_autocmd' (a nil value)
stack traceback:
        ...ocal/share/nvim/plugged/stabilize.nvim/lua/stabilize.lua:112: in function <...ocal/share/nvim/plugged/stabilize.nvim/lua/stabilize.lua:111>

Setup:
require("stabilize").setup({
    ignore = {
		filetype = { "help", "list", "Trouble" },
		buftype = { "quickfix", "loclist" }
	},
	nested = "TermClose"
})

Just realized that :help nvim_exec_autocmd gives help about 'nvim_exec_autocmds', and making these changes seem to fix the issue:
diff --git a/lua/stabilize.lua b/lua/stabilize.lua
index 86171a2..a7f9868 100644
--- a/lua/stabilize.lua
+++ b/lua/stabilize.lua
@@ -109,7 +109,7 @@ function M.setup(setup_cfg)

        if cfg.nested then
                api.nvim_create_autocmd(cfg.nested, { group = group, callback = function()
-                       api.nvim_exec_autocmd("User", { pattern = "StabilizeRestore" })
+                       api.nvim_exec_autocmds("User", { pattern = "StabilizeRestore" })
                end})
        end
 end

Looks like nvim_exec_autocmd got replaced with nvim_exec_autocmds

The change I suggested fixes this issue, using branch 0.8 with neovim 0.8 works as expected, so I guess this can be closed.
Some distributions already provide 0.8 neovim, so the 100% solution would be checking which function to use, but that might be an overkill.


Node list from zone

I want to get node list from selected zone. Any example to do it?

Try this:

use Dyn\TrafficManagement;

$tm = new TrafficManagement('customerName', 'username', 'password');
$tm->createSession();

$zone = $tm->getZone('example.com');
$records = $zone->getAllRecords();

var_dump($records);

That will give you an array of records on the zone, indexed by record type. E.g. $records['A'] will contain all the 'A' records from the zone.

Thanks for your reply @tfountain


[hi] Add content\hi\docs\setup\production-environment\turnkey-solutions.md

localizes file content\hi\docs\setup\production-environment\turnkey-solutions.md

Made the asked changes @mittalyashu

/retitle [hi] Add content\hi\docs\setup\production-environment\turnkey-solutions.md

/close
in favour of #29954


Question about c_str use

Hey, Sorry not an issue, just a question and Im not sure how else to ask.
Ive been trying to do the same thing you did here in this repo for that opengl tutorial. I was stuck with no real errors.
I was doing this  gl::GetUniformLocation(shader.id, "model".as_ptr() as *const i8); and it was working until 1_6_1.
If you look at some of your code like 1_6_1 you use a macro to create a CStr.
If i did something like
gl::GetUniformLocation(shader.id, "model".as_ptr() as *const i8);
gl::GetUniformLocation(shader.id, "view".as_ptr() as *const i8);

My square wouldn't render. I copied your macro for cstr and it worked.
How did you figure out that you needed to use a CStr in that function the way you did?

OpenGL is a C library. Rust strings are not C style strings but rather UTF-8 encoded strings which lack the terminating null byte marking the end of the string.
The below would have worked too without the use of cstr!:
gl::GetUniformLocation(id, b"model\0".as_ptr() as *const i8);

Yes, exactly. And I found it it better to do in in a macro instead of a helper function, because I didn't want any string operations (concatenating \0) to happen at runtime during the render loop.


Restore color mode fallback in useThemedColormap

Adds fallback to dark color mode in useThemedColormap that previously existed in useColormap before the refactor in https://github.com/carbonplan/colormaps/pull/7

Handles cases where an unexpected colormap is present (we may want to look further into how this happens, but because of connection to local storage, etc. it seems likely that we will encounter unexpected values on at least initial renders)

looks great, glad we caught this!


print help message

Problem
-h or --help doesn't print a help message
Modification
Added an option field for -h or --help so that it will print a help message
Result
prints help message with -h or --help

Nice!


Reorder TopicMessageQuery.subscribe method signature

Problem
const errorHandler = null;
const messageHandler = (message: TopicMessage) => {}
new TopicMessageQuery(configObj).subscribe(client, errorHandler, messageHandler)

The subscribe method requires an error handler which at this time might already be supplied via the constructor or dedicated setter methods.
In this case null has to explicitly be passed to the function.
Solution
Switch the position of the error and message handler and make the error handler optional.  This is a breaking change!
Alternatives
No response

As you said in the solution, it's a breaking change and it's not something we can do :( Especially since this API has been stable for awhile. If we were to do a v3 SDK then we'd definitely fix this.


Kindly request the inclusion

Thank you for this great paper collection! It will be my pleasure if my work can be included in the repo; thanks!

Title
Link
Code
Venue
Classification
Model
Comment

MetaCloak: Preventing Unauthorized Subject-driven Text-to-image Diffusion-based Synthesis via Meta-learning
https://arxiv.org/abs/2311.13127
https://github.com/liuyixin-louis/MetaCloak
CVPR'24 Oral
B1. Adversarial Examples
Diffusion
a more robust protective perturbation framework for safeguarding portrait against customized diffusion models training

Great work! We have added this paper, please check it, thanks!

Perfect! Thanks again for this great collection.


Flattened dependency format (OData, V2 feed) varies for the same package

Compare the Dependencies property in:
WindowsAzure.Storage 4.3.0 - from search service
WindowsAzure.Storage 4.3.0 - from DB
WindowsAzure.Storage 8.1.1 - from search service
WindowsAzure.Storage 8.1.1 - from DB
In every case that I found, a version range like [1.0.0, ) is materialized as 1.0.0 in search service. This uses S VersionRange format string (source). This is meant to be legacy compatible.
At some point in time, gallery switched from version range [1.0.0, ) as 1.0.0, to  [1.0.0, ). This is why 4.3.0 has the old format and 8.1.1 has the new format.

What is the customer impact of this discrepancy?


Please add time stamps.

Description of the new feature/enhancement

I want time stamps in the terminal for every command (although some would prefer for every line number, potentially).
I envision it like https://gitlab.com/gitlab-org/gitlab/-/blob/72e14aa6bb97c6e10870812ac9b521d41b8f890b/doc/ci/yaml/ci_job_log_timestamps.md#example-configuration-and-output depicts:

Example configuration and output
Example pipeline configuration with FF_TIMESTAMPS enabled:
variables:
  FF_TIMESTAMPS: true

Example log output with FF_TIMESTAMPS enabled:

...which happens to depict how this feature would work well with https://github.com/microsoft/terminal/issues/17490#issue-2382168026.
Proposed technical implementation details (optional)

https://gitlab.com/gitlab-org/gitlab/-/issues/463391 contains most of the useful discussions regarding implementation.

This might be a duplicate of https://github.com/microsoft/terminal/issues/9331#issue-819782127, in retrospect, although I think I've phrased it better if so.

Thanks for filing! Marking this as a /dup of #9331.


Case-insensitive query generation

I try to retrieve historical features from a Postgres batch source, using SQL as an entity_df parameter for get_historical_features.
Some of the column names have capital letters: EmployeeID, Date, etc.
    features = store.get_historical_features(
        features=feature_service,
        entity_df='select history."EmployeeID" as "EmployeeID", DATE(history."Date") as event_timestamp from public.history',
    ).to_df()

Usage of double quotes helps in vanilla PSQL, but in this case, the final join query is generated from a Jinja template that makes all columns lowercase. So, I receive errors such as this one:
  File "/home/k/venv/lib/python3.8/site-packages/feast_postgres/offline_stores/postgres.py", line 207, in _to_arrow_internal
    cur.execute(query)
psycopg2.errors.UndefinedColumn: column "employeeid" does not exist
LINE 13:                     CAST(EmployeeID as VARCHAR) ||
                                  ^
HINT:  Perhaps you meant to reference the column "sub.EmployeeID".

Seems to be a bug.

Hey @kirill-zaitsev, thanks for creating the issue.
I'll try to get this fixed over the weekend.

Quick and dirty. v0.2.3 should fix this issue.

@nossrannug I am using v0.2.4 and still get this error when I have uppercase characters in my feature name(column name). I am using feast v0.16.0
File "/opt/venv/lib/python3.9/site-packages/feast_postgres/offline_stores/postgres.py", line 214, in _to_arrow_internal
    cur.execute(query)

psycopg2.errors.UndefinedColumn: column a.testColumn does not exist

LINE 6:                 SELECT a."test_case_id", a."testColumn", a."d...
                                                 ^
HINT:  Perhaps you meant to reference the column "a.testcolumn".


Ingest clinical COVID-19 data from C3.ai's data lake (or their upstream source)

Name of the dataset
~Clinical data from C3.ai data lake, for example case (age/gender/location/symptoms/date of onset) from https://c3.ai/covid-19-api-documentation/~
Per Marcin's sharp eyes, we could just ingest clinical data upstream, from where C3.ai ingests it:
https://raw.githubusercontent.com/beoutbreakprepared/nCoV2019/master/latest_data/latestdata.csv
Also here:
https://docs.google.com/spreadsheets/d/e/2PACX-1vQU0SIALScXx8VXDX7yKNKWWPKE1YjFlWc6VTEVSN45CklWWf-uWmprQIyLtoPDA18tX9cFDr-aQ9S6/pubhtml
Mapping or relevant fields
TBD
If possible, highlight which fields map to nodes and which fields map to edges.
Refer to Data Preparation for guidelines on how the final transformed data should be represented.

Sounds like @realmarcin will implement this, and Bill can review his mapping/transform

Added a documentation directory for the C3.ai API.
We can place other documentation there as needed.
cc @deepakunni3

Note that if this ingest requires an API call, this ticket will then probably require a function in download() to call the API and emit a file

see quickstart.ipynb for examples of how to access data

Discussion with Bill - initial thought is to ingest the following for each case:
age    sex     location    symptoms    outcome       therapeutic intervention    covid genome sequence

Per Marcin's sharp eyes, we could just ingest clinical data upstream, from where C3.ai ingests it:
https://raw.githubusercontent.com/beoutbreakprepared/nCoV2019/master/latest_data/latestdata.csv

Also here:
https://docs.google.com/spreadsheets/d/e/2PACX-1vQU0SIALScXx8VXDX7yKNKWWPKE1YjFlWc6VTEVSN45CklWWf-uWmprQIyLtoPDA18tX9cFDr-aQ9S6/pubhtml

@justaddcoffee @realmarcin
Where did you find these URLs for the data? I can't find them.

These two links seem to be the upstream source of some of C3.ai data - are they not working for you?
https://raw.githubusercontent.com/beoutbreakprepared/nCoV2019/master/latest_data/latestdata.csv
https://docs.google.com/spreadsheets/d/e/2PACX-1vQU0SIALScXx8VXDX7yKNKWWPKE1YjFlWc6VTEVSN45CklWWf-uWmprQIyLtoPDA18tX9cFDr-aQ9S6/pubhtml

Yes. They work for me.
I was curious how you found them in the documenation.


Project unmaintained

@svartalf I believe the project needs another maintainer since the issues and pull requests have no one to tend to them. Probably would be good if there are other people to help maintain actions-rs, like outdated template (v2 instead of v3).
Not sure if author will reply here, so I will email him.

I tried contacting @svartalf via email from his GH profile on June 16th, however, haven't heard from him back.
Given his last contribution to actions-rs was in Nov 2020, I think the best course of action would be to fork the entire organization.
Personally, I'd love to see this go under the https://github.com/rust-unofficial umbrella since such a move makes it less likely for the project to go stale again, or even (ideally) https://github.com/orgs/rust-lang/repositories?q=action ... Perhaps we could try to ping someone from rust-lang, I saw @steveklabnik showing interest in one of the PRs here some time ago - maybe you could help us get the ball rolling?
Otherwise, we can create a new organization. Not gonna lie, actions-rs is I think a fantastic name, but maybe we can come up with something as good.

If any, I think https://github.com/actions/setup-rust is the best since most other popular languages are there.

that would also be fantastic! do you know how to get in?

I am no longer affiliated with the Rust Project, so if you're looking for an official fork, I can't be of help, sadly.

By the way, this 404s for me...

Yes, it's 404. I suggested that but don't have access to actions organization.

Let's not fork until we have an idea of where it would go at the very least.  Meaning an official organization if at all possible.
It is also possible that @svartalf may eventually respond and be willing to add new maintainers to the existing organization.

It is also possible that @svartalf may eventually respond and be willing to add new maintainers to the existing organization.

Such as perhaps @dtolnay?  https://github.com/dtolnay/rust-toolchain

Nope, I am not interested in being added as a maintainer.

From the reddit thread and here it looks to me like the most likely way forward is either to essentially fork the entire organization or start anew but make sure that the bus factor is greater than one. Would it be possible to perhaps create an action in rust-lang-nursery with the possibility of moving it to the rust-lang repo proper like haskell/actions / deno-land/setup-deno? I realize that this requires a fair amount of trust, but it is in my opinion the best option.
As an aside, something I really like with for example setup-node is that it has built-in caching so no messing about with action/cache, so maybe include @Swatinem in the conversation since he has written a great Rust caching action.

To be fair, I think just run: rustup install stable and variations of that one, and similarly run: cargo test is way simpler than something based on an action.
I believe for the toolchain, dtolnay/rust-toolchain may do better caching for toolchains that are not directly installed on the runners, though I haven’t used that myself so far.
Also, thanks for your kind words ;-)

@Swatinem do you know a way to install rust_version & components & targets comming fom rust-toolchain.tom in one command? Not sure if rustup show doesn't do that... (which is quite bizzare given the name of the command - https://github.com/rust-lang/rustup/issues/1397)
For my usecase, this whole Action would no longer had any use if this would land:

https://github.com/rust-lang/rustup/issues/2686

I’m not really familiar with rust-toolchain.toml and how to best use that. For the projects I help maintain, we test against our MSRV, stable and beta, and have no requirements for more specific versions.
I would imagine just updating your rustup invocation for your CI job alongside rust-toolchain.toml updates would suffice, but like I said I’m not really familiar with any of this.

The dtolnay/rust-toolchain action felt a bit too bare bones to me, so I created https://github.com/actions-rust-lang/setup-rust-toolchain based on it. It does some extra steps on top like disabling incremental compilation for faster building from scratch (CARGO_INCREMENTAL=0) and sets up problem matchers for cargo and rustfmt.

@jonasbb instead of doing that, why not just send a pull request to dtolnay repository?

@pickfire There are some maintenance concerns which make me convinced a separate project is more beneficial. However, this issue is not the place to discuss these concerns.

If you don't need lots of customizability, this works quite well for installation and caching of dependencies:
# File .github/composite/rust/action.yml

name: rust
description: "Install Rust toolchain, with caching"

inputs:
  rust:
    required: false
    description: "Rust toolchain, e.g. 'stable' or 'nightly'"
    default: stable
  cache-key:
    required: false
    description: "Extra key to resolve cache"
    default: ''

runs:
  using: "composite"
  steps:
    - name: "Install Rust"
      run: |
        rustup toolchain install ${{ inputs.rust }} --profile minimal --no-self-update
        rustup default ${{ inputs.rust }}
      shell: bash

    - uses: Swatinem/rust-cache@v2
      with:
        shared-key: ${{ inputs.cache-key }}

    - name: "Set environment variables used by toolchain"
      run: |
        echo CARGO_TERM_COLOR=always >> $GITHUB_ENV
        echo CARGO_INCREMENTAL=0 >> $GITHUB_ENV
        echo RUST_BACKTRACE=1 >> $GITHUB_ENV

        rustc --version --verbose
      shell: bash

I am interested in maintaining open source software though DAOs.  Basically, users get a say in how a project develops but maintainers also get paid.
https://www.youtube.com/watch?v=9VXWcLOsdBM
Note: I don't work for codebase and their solution is still in early stages but something like their governance system would help situations where a maintainer gets busy with other things.

I was also irritated by the loss of actions-rs, so I created this somewhat experimental replacement  to use with one of my Rust projects. I absolutely would not recommend for production use-cases, but for hobbyist-sized projects I think it might be useful.

For anyone visiting this thread, I've created a new action: https://github.com/moonrepo/setup-rust
This supports installing the toolchain, as well as caching ~/.cargo/target, and installing binaries.

After spending a day on fixing up our CI using GitHub Actions, turns out that their Linux, Mac, and Windows images all include Rust along with clippy and rustfmt. Therefore, for most GH Actions scenarios, you just need:
    - name: Update Rust
      run: rustup update

I also contacted the maintainer by email today. I let you know if I receive an answer.


Revert "jz - added bookFixtures.js"

This reverts ucsb-cs156-s23/team01-s23-6pm-1#27.
The field should be renamed "bookFixtures"

This PR is replaced by #29


server_setup creates bogus nodes

server_setup includes a call to Chef::Node.find_or_create.  This results in the creation of a bogus node if the FQDN of a node does not match the node name.  I have a potential fix, but have not yet submitted a pull request as I have not properly tested this branch yet.
https://github.com/jdblack/chef-gluster/tree/dont_create_nodes

*** Do not Merge Yet***
This is a work in progress. This branch intends to solve two problems:  This mostly works for me, but there is a bit of work to do involving races between the simultaneous provisioning of peer nodes.
I have continued this work in https://github.com/jdblack/chef-gluster/tree/gluster_based_on_nodenames
The good news:

Bogus peer nodes are no longer implicitly created.
Nodes are now referenced via attributes strictly by node name. The fqdn of peer nodes is looked up as necessary by the peer node.

The bad news:

We need introduce some configurable delays to make this work robust against provisioning race conditions (e.g. bringing up nodes A and B at the same time). This patch, in its current state, should eventually converge, but who wants to wait?

Fixed by https://github.com/shortdudey123/chef-gluster/commit/121994ae8c5c09884d295d765d683c2e916ae9cd


[hi] Localize content/hi/docs/contribute/new-content/new-features.md

Signed-off-by: amolmote<EMAIL_ADDRESS>Fixes subtopic issue #36468 [hi] Localize en/docs/contribute/new-content/new-features.md into  hi/docs/contribute/new-content/new-features.md
which is the sub-issue of #36455

@divya-mohan0209, please review I have made all the possible changes.

@bishal7679, have a look.

/remove-lifecycle rotten

Hi @divya-mohan0209 , due to change in head I am unable to update this branch. created new PR. could you pls review this: https://github.com/kubernetes/website/pull/39197. same issue happened with my other PR's also I am trying to update it one by one.

closing this PR now


add transparent logos

Add in transparent square logo in DemoNav
Add in transparent horizontal logo in LandingPageNav

Closing this because the transparent logos are on https://github.com/AbigailDawson/knownative/pull/85


[Epic] Support Bitcoin Cash

Introduce Client of xchain-bitcoincash DONE #940
Add Address to Settings DONE #940
Add balances to assets overview
Add AssetDetail
Add Send component
Support BCH for Swap #995
Support BCH for asym Deposit
Support BCH for sym Deposit
Support BCH for asym Withdraw
Support BCH for sym Withdraw
Add Ledger support

Except ledger support (it will be added later), everything is done


Remove column from tool panel column chooser

I am wondering if it is possible to have a column NOT appear in the column show/hide section of the tool panel? I want to be able to create a checkbox column and allow a user to use that for selections, but not allow them to re-position it within the grid. I was wondering if this was a possible feature?

already requested
https://github.com/ceolter/angular-grid/issues/340
it's all in the plan, but unfortunately i've a day job :(


changing code highlight mode to azurecli

Changing code highlight mode from Azure Power Shell (azurepowershell-interactive) to Azure Cloud shell (azurecli)

@shef : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#sign-off


Sphinx docs may not be compatible with html_context

🐛 Describe the bug
When building documentation with the template-repo-python, the docs/source/conf.py specifies a html_context. However, the html_context feature is available only with Sphinx ≥ 0.5. The setup.cfg doesn't give a version specification for sphinx. Depending on your virtual environment and the system Python from whence it came, it may use an older version of Sphinx that's incompatible.
The setup.cfg should stipulate a higher version of Sphinx.
(Actually, the fact there are no version specifications for any of the dependencies listed is  a version-maximizing dependency confusion attack just waiting to happen 😮)
📜 To Reproduce

Install sphinx on Windows:

pip install -U sphinx

run sphinx-quickstart to generate make files
copy make.bat to docs folder of a newly created repo (with "template-repo-python" template).
run make.bat html
open build/html/index.html
The page looks wrong. No CSS are applied.

🕵 Expected behavior
Good clean HTML docs.
📚 Version of Software Used
Windows:
C:\>sphinx-build --version
sphinx-build 4.4.0

C:\>sphinx-quickstart --version
sphinx-quickstart 4.4.0

pip list:
Sphinx       4.4.0

🩺 Test Data / Additional context
N/A.
🏞Screenshots

🖥 System Info
Windows 10 (21H2)

🦄 Related requirements
⚙ Engineering Details
This issue can be fixed by using html_css_files :
html_css_files = [
    'theme_overrides.css',
]

@nutjob4life is this still an issue?

@jordanpadams, afraid so.
In fact, none of these requirements have version specifications:
[options.extras_require]
dev =
    black
    flake8
    flake8-bugbear
    flake8-docstrings
    pep8-naming
    mypy
    pydocstyle
    coverage
    pytest
    pytest-cov
    pytest-watch
    pytest-xdist
    pre-commit
    sphinx
    sphinx-rtd-theme
    tox
    types-setuptools

which means they are all attack vectors for the version-maximizing dependency confusion attack.

@nutjob4life copy. how critical of an issue do you think this is? can it be tabled for a bit, or you think we should get on this pronto?

@jordanpadams I think we can sit on it for awhile 😁


Pydantic: 'allow_population_by_field_name' has been renamed to 'populate_by_name'

This issue still exists, from what I've noticed.
* 'allow_population_by_field_name' has been renamed to 'populate_by_name'
  warnings.warn(message, UserWarning)

Originally posted by @ferrari in https://github.com/AndreiDrang/python3-capsolver/issues/63#issuecomment-2035553170

Hi
Thanks for your report.
I didn't find any use of "allow_population_by_field_name", maybe you are using some old version or some other library that displays this warning.


input_number (slider) changes appearance in 0.76.0b5

Looks like i'm the Sliderman! :joy:
Opening a view, or refreshing a view shows the slider for a short time like this:

and after a second or so like this:

This happens not in all views, some show only the first appearance and i can't figure out what it depends on.
Like every issue i have with lovelace, only in Firefox, Chrome shows only the second appearance.
BTW, there's no chance to see what value the slider has, because of https://github.com/home-assistant/ui-schema/issues/150

At home in FF 61.0.1 on Ubuntu the issue is gone.
Sliders show normal.
In chrome they show without the value at the right.

They come and go for me on iOS.

Which version of Ubuntu and Chrome are you running? I'll see if I can reproduce.
The state text gets hidden if the overall width of the row is less than 350 px to make more room for the entity name, but it should happen faster than a second unless maybe the machine you are on is slow. And it could be that your firefox window is wider than your chrome window so the state is not hidden. Or Firefox is measuring the width of that div slightly larger

You're absolutely right. Making the chrome window wider shows the values at the end.
The issue showed on my laptop at work which is running ubuntu 14.04.
Will take a look tomorrow, but i think this can be closed.

Can we re-open this? Once the number disappears on iOS its gone for good. Shouldn't the name be moved to the top kinda like the input_select? Currently the only way I can see what I changed it to is create a sensor.

Open a new issue.


This table had some misaligned columns, so was missing from the website.

There should be a table after the first paragraph here: https://docs.flamegpu.com/guide/5-running-a-sim/5-configuring-execution.html
There isn't.
This PR fixes it:

Just fixed the bad -h visible in the screenshot too.


[SUPPORT] DMS with 2 key example

would u be able to add an example to https://cwiki.apache.org/confluence/display/HUDI/2020/01/20/Change+Capture+Using+AWS+Database+Migration+Service+and+Hudi using 2 column key?
can it just be done calling pre-built   'spark-submit --class org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer'  without writing any class ?

You would need to set the below configs for composite keys.
hoodie.datasource.write.keygenerator.class=org.apache.hudi.keygen.ComplexKeyGenerator
hoodie.datasource.write.recordkey.field=comma_seperated_list_of_primary_keys

@tooptoop4 : Did this resolve the issue ?

We are still facing this issue after setting the hoodie.datasource.write.keygenerator.class=org.apache.hudi.keygen.ComplexKeyGenerator
hoodie.datasource.write.recordkey.field=comma_seperated_list_of_primary_keys
The fist key is only considered as a recordkey.

@bvaradar pls reopen

Reopened.. and let me understand what's going on..

Okay.. DMS or not, this just seems like a DeltaStreamer config, that should work.. what error are you facing? can you give me more details?

@nandurj

I am using multiple keys to create CoW tables by using below properties hoodie.datasource.write.keygenerator.class=org.apache.hudi.keygen.ComplexKeyGenerator
hoodie.datasource.write.recordkey.field=customer_id,product_id
But the delta streamer is not picking up the second key, It is only picking up the first key customer_id. I have verified this by _hoodie_redord_key value on the table only shows customer_id

cc @pratyakshsharma  any ideas? (since you are actively looking at this code)

Hmm strange @nandurj , after setting the above 2 properties, things should work as expected. Can you help me with which version of Hudi you are using and how are you trying to run the job? Can you add logs in DeltaSync constructor to see which keyGenerator class is actually getting initialised. This looks like more of mis configuration issue than code issue. Can you confirm you are adding the properties in the right config file which is getting used by the job?

I am working with HUDI 0.5.2 on EMR 5.30. I am running the job using the Delta streamer. Below is how I am running the spark job.
spark-submit --class org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer
--jars /usr/lib/spark/external/lib/spark-avro_2.11-2.4.5-amzn-0.jar
--master yarn --deploy-mode client
--executor-memory 10G --executor-cores 4
file:///usr/lib/hudi/hudi-utilities-bundle_2.11-0.5.2-incubating.jar
--table-type COPY_ON_WRITE
--source-ordering-field TIMESTAMP
--continuous
--enable-hive-sync
--min-sync-interval-seconds 60
--source-class org.apache.hudi.utilities.sources.ParquetDFSSource
--transformer-class org.apache.hudi.utilities.transform.AWSDmsTransformer
--target-base-path s3://mybucket/CoWex --target-table table_test
--payload-class org.apache.hudi.payload.AWSDmsAvroPayload
--schemaprovider-class org.apache.hudi.utilities.schema.FilebasedSchemaProvider
--hoodie-conf hoodie.datasource.write.recordkey.field="Field1, Field2, Field3"
--hoodie-conf hoodie.datasource.write.keygenerator.class=org.apache.hudi.keygen.ComplexKeyGenerator
--hoodie-conf hoodie.datasource.write.partitionpath.field="Field1"
--hoodie-conf hoodie.datasource.hive_sync.database=testdb
--hoodie-conf hoodie.datasource.hive_sync.table=test_table
--hoodie-conf hoodie.datasource.hive_sync.partition_fields="datefield"
--hoodie-conf hoodie.datasource.hive_sync.partition_extractor_class=org.apache.hudi.hive.MultiPartKeysValueExtractor
--hoodie-conf hoodie.deltastreamer.source.dfs.root=s3://mybucket/input

@pratyakshsharma : Would you be able to debug  this issue ?

This issue got resolved by passing the configs through a props file, e.g:
cat test.properties
hoodie.datasource.write.recordkey.field=f1,f2,f3
hoodie.datasource.write.keygenerator.class=org.apache.hudi.keygen.ComplexKeyGenerator
hoodie.datasource.write.partitionpath.field=dt
hoodie.datasource.hive_sync.database=default
hoodie.datasource.hive_sync.table=hudi_table
hoodie.datasource.hive_sync.partition_fields=dt
hoodie.datasource.hive_sync.partition_extractor_class=org.apache.hudi.hive.MultiPartKeysValueExtractor
hoodie.deltastreamer.source.dfs.root=s3://test/hudi/data
spark-submit --class org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer
--jars /usr/lib/spark/external/lib/spark-avro_2.11-2.4.5-amzn-0.jar
--master yarn --deploy-mode client
~/hudi-utilities-bundle*
--table-type COPY_ON_WRITE
--source-ordering-field tp
--continuous
--enable-hive-sync
--min-sync-interval-seconds 30
--op UPSERT
--source-class org.apache.hudi.utilities.sources.ParquetDFSSource
--transformer-class org.apache.hudi.utilities.transform.AWSDmsTransformer
--target-base-path s3://test-bucket/hudi/tables/hudi_table--target-table hudi_table
--props file:////home/hadoop/test.properties

The properties should be passable in both ways (file or through command line params). @pratyakshsharma : Can you try and let us know if this is an issue ?

Will take a look at it.

I added a log print to checked the config read by JCommander:
20/07/25 00:50:06 INFO HoodieDeltaStreamer: debug => config is hoodie.datasource.write.recordkey.field=f1
20/07/25 00:50:06 INFO HoodieDeltaStreamer: debug => config is f2,,f3
20/07/25 00:50:06 INFO HoodieDeltaStreamer: debug => config is hoodie.datasource.write.keygenerator.class=org.apache.hudi.keygen.ComplexKeyGenerator
20/07/25 00:50:06 INFO HoodieDeltaStreamer: debug => config is hoodie.datasource.write.partitionpath.field=dt
20/07/25 00:50:06 INFO HoodieDeltaStreamer: debug => config is hoodie.datasource.hive_sync.database=default
20/07/25 00:50:06 INFO HoodieDeltaStreamer: debug => config is hoodie.datasource.hive_sync.table=hudi_table
20/07/25 00:50:06 INFO HoodieDeltaStreamer: debug => config is hoodie.datasource.hive_sync.partition_fields=dt
20/07/25 00:50:06 INFO HoodieDeltaStreamer: debug => config is hoodie.datasource.hive_sync.partition_extractor_class=org.apache.hudi.hive.MultiPartKeysValueExtractor
20/07/25 00:50:06 INFO HoodieDeltaStreamer: debug => config is hoodie.deltastreamer.source.dfs.root=s3://hudi/data

It shows that JCommander will automatically split the string by ",":
20/07/25 00:50:06 INFO HoodieDeltaStreamer: debug => config is hoodie.datasource.write.recordkey.field=f1
20/07/25 00:50:06 INFO HoodieDeltaStreamer: debug => config is f2,,f3

That's why it unable to read multiple fields as record key.

Opened https://issues.apache.org/jira/browse/HUDI-1140

@zhedoubushishi Interesting! Were you able to deep dive into how can we ensure JCommander does not split by comma or any work around we could do?

Sure. Will take a look later.

https://www.baeldung.com/jcommander-parsing-command-line-parameters#separated-lists might be useful.

Closing this as we have a PR.

https://github.com/apache/hudi/pull/1898

@bvaradar
is this issue resolved in any of Hudi release ?
I am currently using 0.6.0 with scala 2.11 and previously tried with 0.5.2-incubating/0.5.3 + Scala 2.11 as well the result is same Below is the description of the spark job that I wrote.
My usecase is somewhat same where I am using 2 columns of a dataset to make an upsert. but the comma separated values don't work. Below is the code I am using to make an upsert in a dataset where albumId and trackId are 2 keys which identifies a unique record :
albumDf.write
.format("hudi")
.option(DataSourceWriteOptions.TABLE_TYPE_OPT_KEY, DataSourceWriteOptions.COW_TABLE_TYPE_OPT_VAL)
.option(DataSourceWriteOptions.RECORDKEY_FIELD_OPT_KEY, "albumId, trackId")
.option(DataSourceWriteOptions.PRECOMBINE_FIELD_OPT_KEY, combineKey)
.option(HoodieWriteConfig.TABLE_NAME, tableName)
.option(DataSourceWriteOptions.OPERATION_OPT_KEY, DataSourceWriteOptions.UPSERT_OPERATION_OPT_VAL)
.option("hoodie.upsert.shuffle.parallelism", "2")
.mode(SaveMode.Append)
.save(s"$basePath/$tableName/")
And below is the error that I get :
... 5 more
Caused by: org.apache.hudi.exception.HoodieKeyException: recordKey value: "null" for field: "albumId, trackId" cannot be null or empty.
at org.apache.hudi.keygen.SimpleKeyGenerator.getKe
Can you please help ?

@noobarcitect : This is a different issue.  You need to use ComplexKeyGenerator instead of SimpleKeyGenerator if you are using more than 1 columns as record key.

@bvaradar why not point SimpleKeyGenerator class to ComplexKeyGenerator ? seems no need to maintain a limited 1 key class if ComplexKeyGenerator can do 1 or multi keys without the null bug

Thanks for replying @bvaradar and @tooptoop4 .
Though this solves the problem, the exception trace is quite confusing. Also keys such as these can not be handled through 1 class only ?

I have been facing 1 more problem. The hudi dataset generated by spark job contains a .hoodie and default directory. Now I want to view it in AWS data cataogue and crawling this data through glue crawler.
I have hit a roadblock now that my crawled table is having those extra metadata columns such as _hoodie_commit_time, _hoodie_coomit_seq_no, _hoodie_record_key etc.
Is there a way to just get the data which contains only data columns and not these metadata columns ? Although I can read the dataset as df and another spark job  and drop these extra columns but I want to avoid that.

@noobarcitect : SimpleKeyGenerator is intended to handle singe key column as the name suggests. To avoid confusion, it is best to retain the current behavior :)
Regarding data catalog, Hudi columns are needed for your incremental queries. If you want to omit them, I am not sure how the query integration would work. If this simply needed for tracking columns, the way you suggested is the one that works.

@noobarcitect : We would be happy to help you in answering your queries. Kindly open a new ticket as this is a closed ticket.

@bvaradar .. That was really helpful.
Will surely open a ticket If I need. I have got all the answers for now.


Consider replacing Bone mux with Chi

FEATURE REQUEST

Is there an open issue addressing this request? If it does, please add a "+1" reaction to the
existing issue, otherwise proceed to step 2.

Describe the feature you are requesting, as well as the possible use case(s) for it.
Currently Mainflux HTTP and WS use Bone mux.

New mux Chi that appeared in the meanwhile gives more modern impression and should be evaluated as a Bone replacement.

Indicate the importance of this feature to you (must-have, should-have, nice-to-have).
nice-to-have

Chi looks really interesting, light ( stdlib only), and performant. Yet I would mark this as a low priority, cause it does not bring real benefits (its control plane API, not performance-critical, bone served us quite well, except now with #1357 which will be fixed with #1358 and the issue will be opened (@mteodo you should probably raise the issue ).

This was addressed by this pr: https://github.com/absmach/magistrala-old/pull/47


Domain activation is Failed - [BRIEF DESCRIPTION]

App Id: d3e2xikkscic4d
Region: us-east-2
Step: Domain activation
Status: Failed

Note: Do not include information that is sensitive in nature such as your domain name, company etc.

Issue/question
A clear and concise description of what the issue/question is.
Error message
If there is an error message, please include it here.
Note: Be sure to check the message for sensitive information.
Additional information
Please add any other relevant information. Please feel free to include screenshots.

It looks like the custom domain has since been removed, there are a number of reasons for the DNS error you encountered: https://docs.aws.amazon.com/amplify/latest/userguide/understanding-dns-terminology-and-concepts.html Please feel free to reopen if you're still experiencing an issue.


Please consider to remove OV model cache enabling for model compiling

Currently, OV model cache is not actual bring benefit on CPU, and GPU also generates cache file larger than original weights. I recommend to remove model cache enabling at this moment in case bring negative feedback from users.
When we call "OVModelForCausalLM.from_pretrained" the model compiled by OV to generate IR, below source code required to compiling with cache. Please consider to modify it disable it when using CPU, enable it on GPU.
https://github.com/huggingface/optimum-intel/blob/main/optimum/intel/openvino/modeling_base.py#L318

Without model cache, it just require 40s to compile.

Hi Fiona, model caching is supported on CPU, and can help quite a bit with some large models. If loading from cache takes longer than without cache for certain models, that looks like an issue with OpenVINO. Could you raise that there?
I do agree with you that we should have an option to disable model caching, and to override the CACHE_DIR directory.  cc @echarlaix

Sorry Helena, my mistake, I used optimum-cli generated model to compare the loading time.
The optimum-intel generated llama 7b every time requires 260s for first compiling, by using cache, next time decrease to 47.5s. The inference perf looks also affected by compiling costs.

We also tried to load model generated by optimum-cli, it only take 45.8s for 1st time compiling. I think you may close this ticket, cache helps. But we should figure out the huge costs for 1st time compiling.

Sure @helena-intel, just opened https://github.com/huggingface/optimum-intel/issues/352


Cannot Bypass

Domain
https://adbs.uab.gov.tr
Link
https://adbs.uab.gov.tr
Version
0.0.0
What browsers are you seeing the problem on?
Firefox, Chrome
What OS are you seeing the problem on?
Windows
Anything else?
doesn't bypass

Please add an example link.

Inactive, closing this.


ToolTip Context Not Passing Context

Checking on the log below from Technici4n, it looks like ItemEmiStack (and maybe elsewhere? Didn't check) uses TooltipContext.DEFAULT in it's tooltip method. This short circuits any possibility of context in the context.
TooltipContext.create(client.world), which takes a nullable param, would build contextual Contexts as needed by users.
https://github.com/emilyploszaj/emi/blob/615d816a81de7627292934c1c3d99b4b6931cfd1/xplat/src/main/java/dev/emi/emi/api/stack/ItemEmiStack.java#L162
java.lang.NullPointerException: Cannot invoke "net.minecraft.core.HolderLookup$Provider.lookupOrThrow(net.minecraft.resources.ResourceKey)" because the return value of "net.minecraft.world.item.Item$TooltipContext.registries()" is null
    at TRANSFORMER/modern_industrialization@0.0.0-local/aztech.modern_industrialization.items.diesel_tools.DieselToolItem.appendHoverText(DieselToolItem.java:134)
    at TRANSFORMER/minecraft@1.21/net.minecraft.world.item.ItemStack.getTooltipLines(ItemStack.java:772)
    at TRANSFORMER/emi@1.1.7+1.21+neoforge/dev.emi.emi.api.stack.ItemEmiStack.getTooltipText(ItemEmiStack.java:162)
    at TRANSFORMER/emi@1.1.7+1.21+neoforge/dev.emi.emi.search.EmiSearch.bake(EmiSearch.java:62)
    at TRANSFORMER/emi@1.1.7+1.21+neoforge/dev.emi.emi.runtime.EmiReloadManager$ReloadWorker.run(EmiReloadManager.java:222)
    at java.base/java.lang.Thread.run(Thread.java:1583)
    ```

This was changed to provide context if possible in 1.1.8


Update log4j2.xml 设置日志切割时间为1h

What is the purpose of the change
(For example: EngineConn-Core defines the the abstractions and interfaces of the EngineConn core functions.
The Engine Service in Linkis 0.x is refactored, EngineConn will handle the engine connection and session management.
Related issues: #590. )
Brief change log
(for example:)

Define the core abstraction and interfaces of the EngineConn Factory;
Define the core abstraction and interfaces of Executor Manager.

Verifying this change
(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added tests for submit and execute all kinds of jobs to go through and verify the lifecycles of different EngineConns.

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
Anything that affects deployment: (yes / no / don't know)
The MGS(Microservice Governance Services), i.e., Spring Cloud Gateway, OpenFeign, Eureka.: (yes / no)

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

assembly-combined-package下设置日志切割时间为1h


VMU Screen corrupted in Soul Calibur

Disabling the SH4 JIT solves the issue.

Using the interpreter fallback for or Rm,Rn solves the issue.

This is actually an issue with the inline_small_forward_jumps optimization: Using the fallback for or Rm,Rn prevented it to be applied somewhere and fixed the issue by chance.

The issue might actually lies in the inline_jumps_to_start_of_block optimization. Disabling inline_small_forward_jumps also prevented this optimization in some cases. Only disabling inline_jumps_to_start_of_block also fixes the issue.

Fixed by 168596ccd21291e7f2037afb99505bc49a0a6669


Modules and NPM2

@mnmtanish I just implemented NPM2 support for storybooks.
See: #356
It'll work with current modules.
But I want to know about how it will work with custom modules (user supplied modules.)
Could you possible to do some review on this.

If there's an issue, we could alias core module repos with require.resolve

We can try a few open source repos with customised storybook and see whether it works with npm2. We can try repos with custom .babelrc files and webpack config files. If you know any, I can check them out.

I think this is working pretty greatly.