recognizing "species group" or "species complex" suffixes as indicators of infrageneric groupings

created by @mtholder at https://gitlab.com/gogna/gnparser/-/issues/55
(First off: thanks for all the work on gnparser! It is shockingly efficient, precise, and complete.)
This may well be out of scope, but I am working with some data that uses names like
"Pelomedusa subrufa species complex"

between the genus-level and the species level.
In gnparser "v0.7.4" (running on ubuntu 18.10), this " species complex" is just treated as a part of the unparsed tail, and the name info reflects just the info for "Pelomedusa subrufa"

Recommendation: Follow the ZooCode article 6.2 to the extent possible.
Article 6. Interpolated names.
6.2. Names of aggregates of species or subspecies. A specific name may be added in parentheses
after the genus-group name, or be interpolated in parentheses between the genus-group name and the
specific name, to denote an aggregate of species within a genus-group taxon; and a subspecific name
may be interpolated in parentheses between the specific and subspecific names to denote an aggregate
of subspecies within a species; such names, which must always begin with a lower-case letter and be
written in full, are not counted in the number of words in a binomen or trinomen. The Principle of
Priority applies to such names [Art. 23.3.3]; for their availability see Article <IP_ADDRESS>.

Recommendation 6B. Taxonomic meaning of interpolated names. An author who wishes to denote
an aggregate at either of the additional taxonomic levels mentioned in Article 6.2 should place a term
to indicate the taxonomic meaning of the aggregate in the same parentheses as its interpolated
species-group name on the first occasion that the notation is used in any work.

Example. In the butterfly genus _Ornithoptera_ Boisduval, 1832 the species _O. priamus_ (Linnaeus,
1758) is the earliest-named member of an aggregate of vicarious species that includes also _O. lydius_
Felder, 1865 and _O. croesus_ Wallace, 1865. The taxonomic meaning accorded to the _O. priamus_
aggregate may be expressed in the notation "_Ornithoptera_ (superspecies _priamus_)", and the members
of the aggregate by the notations "_O._ (_priamus_) _priamus_ (Linnaeus, 1758)", "_O._ (_priamus_) _lydius_
Felder, 1865", and "_O._ (_priamus_) _croesus_ Wallace, 1865".

Treat interpolated names of subgenera (Art. 6.1) in a consistent way to the extent possible.

The key question for Pelomedusa subrufa species complex is not how to handle the tail but how to handle the expression itself.
A species complex is:
In biology, a species complex is a group of closely related organisms that are so similar in appearance and other features that the boundaries between them are often unclear.
https://en.wikipedia.org/wiki/Species_complex
Which does not automatically equate to a name used for an aggregate of species of subspecies which is denoted by a term such as "superspecies" in the sense of the ZooCode.
aggregate, n.
A group of species, other than a subgenus, within a genus; or a group of species within a
subgenus; or a group of subspecies within a species. An aggregate may be denoted by a
species-group name interpolated in parentheses [Art. 6.2].

The authors referring to a "species complex" are not necessarily aware of the rules of the ZooCode governing the availability, priority, and correct formation of a name for an aggregate of species, e.g., 6.2, 10.4, <IP_ADDRESS>, 23.3.3, and the related definitions in the Glossary.
Therefore, "species complex" can be taken off of the name and just retrieving the stated name is a good result.

For the example for subgeneric names with two different formats:

Aus subgen. Bus
Aus (Bus)

Normalization to canonical form:
short version: Bus
full version: Aus (Bus) #Parentheses are important here as per article 6.1 of the ZooCode.
Aus bus species group becomes:
normalized: Aus (superspecies bus)
canonical simple: Aus (bus)
canonical full: Aus (superspecies bus)
in details rank: superspecies or aggregate of species #Avoid using species group, which is defined as:
species group, n.
In the zoological classification, the lowest-ranking group of taxa the names of which are
regulated by the Code. The species group includes all taxa at the ranks of species and
subspecies [Art. 45.1].

Thank you @Archilegt for detailed explanation!
Aus subgen. Bus and Aus (Bus) are already covered:
Id,Verbatim,Cardinality,CanonicalStem,CanonicalSimple,CanonicalFull,Authorship,Year,Quality
0f831955-2e8b-52f0-8b05-aa3a825c629e,Aus subgen. Bus,1,Bus,Bus,Aus subgen. Bus,,,2
784a3c10-e635-5cdb-a4bb-0f29c1a167ce,Aus (Bus),1,Bus,Bus,Aus subgen. Bus,,,2

@Archilegt and @proceps, do I understand it correctly that names like
Aus bus species complex
Aus bus species group
Aus bus superspecies
Aus (bus)

all can be converted to:
normalized:           Aus (superspecies bus)
canonical simple:  Aus (bus)
canonical full:       Aus (superspecies bus)

?
Can it be that for names iike Aus bus species group people actually do mean coordinated name according to ICZN?

@dimus, "species complex" should not be converted. See this comment.
The rest can be converted, but I have often found that the authors don't refer to ssuperspecies anywhere in the text, which is what should be happening after 2000. I have also found cases where the authors state that their species group corresponds to whatever subgenus but they still use "species group". In those cases, converting "species group" to superspecies instead of leaving it blank could result in confusion. But it may be that for most cases will be fine. You can set gnparser to convert "species group" to superspecies.

@dimus, "species complex" should not be converted.

Oups my bad. So I guess we can continue to treat species complex the same as before?
http://parser.globalnames.org/?format=html&names=Aus+bus+species+complex&with_details=on
Parser now ignores 'species complex' and creates a warning about an unparsed annotation.
And for all kind of superspecies, as I understand, no warning is needed, and we would trust the author of database or a checklist that they took care to write down the name as it supposed to be written, the same way as we trust a person that 'Aus bus' meant a binomia of genus 'Aus' and specific epithet 'bus'

@dimus I believe it should be treated as superspecies, but the problem is that authors not always follow the ICZN rules for coordinated names, and many taxonomists believe that 'species group' and 'species complex' are informal grouping. 'Superspecies' is not clearly defined by ICZN, neither do complexes and groups. The only thing that ICZN says, that different  ranks are permitted, and if the rank falls within species group rank, it should follow the rules for this group. So, I would probably agree, that it is safer not to treat complexes as superspecies, just because the taxonomists do not follow those regulations consistently. At the same time, I know that the new edition of ICZN will probably reduce the significance of species complexes even more.

Thanks @proceps, very helpful comment


Clarification on "interrupt"

What is the question:
Likely a very quick one:
The interrupt property defines if the currently running instance should be allowed to finish its current execution flow before it needs to be terminated. If set to true, the current instance execution should stop immediately.
Does current execution flow mean the active state (finish all actions, etc)?

@kevinteg To my understanding, yes, but I wasn't there when this was defined. WDYT @tsurdilo ?

meaning that its current execution flow / path should be able to finish (reach an end definition). maybe we should use "path" instead to be clear.

OK glad I asked.  Yes I think it would be helpful to add some wording there about what it means (and maybe an example use case).
Thanks!


Forgot to close http resource cause to ConnectionPoolTimeoutException

Bug detail:
org.apache.skywalking.oap.server.core.alarm.provider.dingtalk.DingtalkHookCallback -152207495 [pool-3-thread-1] ERROR [] - send dingtalk alarm to https://oapi.dingtalk.com/robot/send?access_token=xxx failure.
org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection from pool
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:314) ~[httpclient-4.5.7.jar:4.5.7]
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:280) ~[httpclient-4.5.7.jar:4.5.7]
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:190) ~[httpclient-4.5.7.jar:4.5.7]
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185) ~[httpclient-4.5.7.jar:4.5.7]
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[httpclient-4.5.7.jar:4.5.7]
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[httpclient-4.5.7.jar:4.5.7]
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[httpclient-4.5.7.jar:4.5.7]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[httpclient-4.5.7.jar:4.5.7]
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) ~[httpclient-4.5.7.jar:4.5.7]
at org.apache.skywalking.oap.server.core.alarm.provider.dingtalk.DingtalkHookCallback.sendAlarmMessage(DingtalkHookCallback.java:144) [server-alarm-plugin-8.2.0.jar:8.2.0]
at org.apache.skywalking.oap.server.core.alarm.provider.dingtalk.DingtalkHookCallback.lambda$null$0(DingtalkHookCallback.java:88) [server-alarm-plugin-8.2.0.jar:8.2.0]
at java.util.ArrayList.forEach(ArrayList.java:1541) [?:?]
at org.apache.skywalking.oap.server.core.alarm.provider.dingtalk.DingtalkHookCallback.lambda$doAlarm$1(DingtalkHookCallback.java:83) [server-alarm-plugin-8.2.0.jar:8.2.0]
at java.util.ArrayList.forEach(ArrayList.java:1541) [?:?]
at org.apache.skywalking.oap.server.core.alarm.provider.dingtalk.DingtalkHookCallback.doAlarm(DingtalkHookCallback.java:82) [server-alarm-plugin-8.2.0.jar:8.2.0]
at org.apache.skywalking.oap.server.core.alarm.provider.AlarmCore.lambda$null$3(AlarmCore.java:84) [server-alarm-plugin-8.2.0.jar:8.2.0]
at java.util.ArrayList.forEach(ArrayList.java:1541) [?:?]
at org.apache.skywalking.oap.server.core.alarm.provider.AlarmCore.lambda$start$4(AlarmCore.java:84) [server-alarm-plugin-8.2.0.jar:8.2.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?]
Reason:
Forgot to close http response resource. Following file line:
https://github.com/apache/skywalking/blob/master/oap-server/server-alarm-plugin/src/main/java/org/apache/skywalking/oap/server/core/alarm/provider/dingtalk/DingtalkHookCallback.java#L144
The "httpResponse" must be close to release the connection, otherwise it will hold the connection resource. When the numbers of connection over the default overhead, next request will throw ConnectionPoolTimeoutException.
All other webhook callback have the same problem.
Suggestion
User can define the connection numbers, e.g:
HTTP_CONNECTION_REQUEST_TIMEOUT
org.apache.http.impl.client.HttpClientBuilder#setMaxConnPerRoute
etc..
Using env variable is the best.

Sorry, I am not familiar with this part, aren't httpClient#close enough?

Sorry, I am not familiar with this part, aren't httpClient#close enough?

Sorry, I am not familiar with this part, aren't httpClient#close enough?

Yes, just do that.

Sorry, I am not familiar with this part, aren't httpClient#close enough?

Yes, just do that.

Are you saying, because there are many messages sent, so the httpClient opened for a long time?

Are you saying, because there are many messages sent, so the httpClient opened for a long time?

Are you saying, because there are many messages sent, so the httpClient opened for a long time?

No. each send action hold a http connection resource, if you don't close it, the connection pool will be full always, so next request can do nothing.
Infact that is an java.util.concurrent.Future.

Are you saying, because there are many messages sent, so the httpClient opened for a long time?

No. each send action hold a http connection resource, if you don't close it, the connection pool will be full always, so next request can do nothing.
Infact that is an java.util.concurrent.Future.

OK, get it. Then the next question is, why do you suggest to open HTTP_CONNECTION_REQUEST_TIMEOUT rather than closing the httpResponse in the try(...)?

OK, get it. Then the next question is, why do you suggest to open HTTP_CONNECTION_REQUEST_TIMEOUT rather than closing the httpResponse in the try(...)?

OK, get it. Then the next question is, why do you suggest to open HTTP_CONNECTION_REQUEST_TIMEOUT rather than closing the httpResponse in the try(...)?

For example, some network problem in IDC, 1 second mabey timeout easily, I hope to change it.

OK, get it. Then the next question is, why do you suggest to open HTTP_CONNECTION_REQUEST_TIMEOUT rather than closing the httpResponse in the try(...)?

For example, some network problem in IDC, 1 second mabey timeout easily, I hope to change it.

Agree, we could have both. But the system env (HTTP_CONNECTION_REQUEST_TIMEOUT) should not implement in the codes level, we provide a standard mode by application.yml to do this already, I think you have used it.
Recommend you to add AlarmSettings#httpTimeout, and make all notifiers to use it.
We are welcome you to submit pull request(1 or 2 up to you) to submit this. We are closing the 8.4.0, next 2-3 days, if you plan to include this in this release, you need to submit the pull request today.

Agree, we could have both. But the system env (HTTP_CONNECTION_REQUEST_TIMEOUT) should not implement in the codes level, we provide a standard mode by application.yml to do this already, I think you have used it.
Recommend you to add AlarmSettings#httpTimeout, and make all notifiers to use it.
We are welcome you to submit pull request(1 or 2 up to you) to submit this. We are closing the 8.4.0, next 2-3 days, if you plan to include this in this release, you need to submit the pull request today.

OK, I try my best.

OK, I try my best.

If you don't have enough time, it is fine. You could submit later, we will loop it in the 8.5.0.

If you don't have enough time, it is fine. You could submit later, we will loop it in the 8.5.0.

Nice, so wait for next version.

Nice, so wait for next version.


Allow privileged for fio server pods

Allow privileged fio server pods so we can drop caches using the fio parameter exec_prerun as mentioned at the updated documentation.

While this might work for Ben if -
a) The user runs FIO across all the nodes w/ OSDs
b) Are not working with distributed storage like Ceph.
Overall, I think it is a good change..

While this might work for Ben if -
a) The user runs FIO across all the nodes w/ OSDs
b) Are not working with distributed storage like Ceph.
Overall, I think it is a good change..

Yeah, I'm aware of that. It does not resolve Ben's issue completely because Fio severs usually don't run on OSD nodes.

While this might work for Ben if -
a) The user runs FIO across all the nodes w/ OSDs
b) Are not working with distributed storage like Ceph.
Overall, I think it is a good change..

Yeah, I'm aware of that. It does not resolve Ben's issue completely because Fio severs usually don't run on OSD nodes.

Right, you might be aware of that, however, people might think this does more than appears. :)

Results for crc_jjb

Test
Result
Retries
Duration (HH:MM:SS)

test_backpack
Pass
0
00:01:34

test_byowl
Pass
0
00:02:06

test_fiod
Pass
0
00:13:23

test_fs_drift
Pass
0
00:04:42

test_hammerdb
Pass
0
00:04:26

test_iperf3
Pass
0
00:03:28

test_pgbench
Pass
0
00:00:40

test_smallfile
Pass
0
00:05:25

test_sysbench
Pass
0
00:01:48

test_uperf
Pass
0
00:18:11

test_ycsb
Pass
0
00:01:39

I'm not pursuing this because I think it's better to have a separate cache dropping pod (or daemonset for kernel cache) that is privileged and does only that, and then invoke it via a URL.


Possible inversion of x-axis and y-axis in crop transform

I just realized that the crop transform in transform/functional.py with arguments i, j, w and h (using PIL.crop function) actually crops a rectangular portion of w width and h height, having the top left coordinate as (j, i). Wouldn't it be intuitive to have the top left corner coordinates as (i, j)?

Related to #1323

Hi @pmeier ! As pointed out by @fmassa, Pillow has different ordering for width and height but I think the position of i and j are still reversed. Please check out: https://github.com/python-pillow/Pillow/issues/2729#issuecomment-425388850 for the syntax for crop in Pillow. The first 2 arguments are for the (x, y) coordinates for the top left corner.
If the user input (i, j) is in the common format of (x, y), then  the current implementation of crop in Pytorch is setting (y, x) as coordinates for top left corner. Am I missing something?

IMO the current implementation is correct. I did the following to verify it:
from PIL import Image
import numpy as np
from torchvision.transforms import functional as F

np_image = np.random.rand(100, 100)
pil_image = Image.fromarray(np_image * 255.0)

left = 10
top = 20
width = 30
height = 40
right = left + width
bottom = top + height

crop1 = pil_image.crop((left, top, right, bottom))
crop2 = F.crop(pil_image, top, left, height, width)

abs_diff = np.sum(np.array(crop1) - np.array(crop2))
implementation_correct = abs_diff < 1e-6

However, we should definitely change the signature and docstring to avoid further confusion. I propose the following:
def crop(img, top, left, height, width):
    """Crop the given PIL Image.
    Args:
        img (PIL Image): Image to be cropped. (0,0) denotes the top left corner of the image.
        top (int): Vertical coordinate of the top left corner of the crop box.
        left (int): Horizontal coordinate of the top left corder of the crop box.
        height (int): Height of the crop box.
        width (int): Width of the crop box.
    Returns:
        PIL Image: Cropped image.
    """

If we follow the convention of (top, left, height, width) then the current implementation is indeed fine. However, we have 2 choices for the naming conventions:

(top, left, height, width)
(x, y, height, width)

Since top and left correspond to coordinates in y-axis  and x-axis respectively, both of them are equivalent. So the only question remaining is what convention do we follow? IMO (x, y, height, width) would be more clear than (top, left, height, width)  as even to explain what top and left mean, we need to use (x, y) coordinates, e.g., (0, 0) is used in the docs.
Is it a standard convention to use (top, left, height, width) in Pytorch? Have  we used this convention anywhere else?

IMO (x, y, height, width) would be more clear than (top, left, height, width) as even to explain what top and left mean, we need to use (x, y) coordinates, e.g., (0, 0) is used in the docs.

If we use x and y we need to state in which direction the box is expanding. This is clear for top and left. Thus, even for the amount of documentation both are equivalent. I think this comes down to preference.
If we go with option 2. the signature should be
(y, x, height, width)
to keep BC.

I guess we can leave it as it. You may proceed to change the documentation as you mentioned earlier or if you want I can do it in case you are busy at the moment.

I can send a PR, if the documentation change is OK for @fmassa

@pmeier the doc improvements that you proposed in https://github.com/pytorch/vision/issues/1380#issuecomment-535871094 are great, it would be great if you could send a PR improving it


Edit link broken in the Portray documentation

Edit on Github link does not work and give 404. Example screenshot attached.

I noticed this too. I think it only happens to the pdocs generated content (stuff pulled from code/docstrings) - I'm guessing portray has pdocs write the output markdown to a reference folder within the temporarily generated documentation.


Slope Blur Update

Hi,
I've recently been messing around with Material Maker as a substance alternative. I was checking out the slope blur and I think the implementation is at least partly wrong. (At least as far as I'm aware)
At the moment the node is doing a directional blur towards the white part of a greyscale texture. However the way it should be behaving is by warping in the direction and adding those all together. There is a very good video explaining how the slope blur node works in substance here:
https://www.youtube.com/watch?v=H0c1SffwllY&t=308s
I think making this change in Maker would be a good improvement to make. Currently the slope blur node isn't producing the result exactly how I would expect them to when compared to Substance.
Edit:
It seems like the MultiWarp is almost doing the right thing but just in the opposite direction. Is there a way to switch the direction it works? This might be a solution?
EditEdit:
Actually it seems as though the MultiWarp is doing the right thing. I'll just use that node!

Material Maker is not a SD clone (I absolutely never used SD), and its slope blur node is not supposed to have the exact same behavior (it's a gaussian blur guided by the slope of its input).
It is probably possible to create a node that behaves the same, but "fixing" that node would just break compatibility, which won't happen (does not mean the new node could not be named "Slope Blur 2" or something).

Yep, I think the MultiWarp is actually doing what I'm seeing in Substance!

Yeah, multiwarp is closer to SD's Slope Blur.


Drag drop placeholder in front of elements

Reproduction
Go on the document page for https://material.angular.io/cdk/drag-drop/overview#customizing-the-drag-placeholder
Drag an item from the end of the list, up.
Expected Behavior
The placholder should be animating "behind" the rows
Actual Behavior
When draging an item "up" (index going down), then the placeholder is shown in front of the cards:

(note that I added border to the items, and black background, for better visualization)
Thoughts

Playing with z-index has no effect
Only happens when dragging to a lower index from the initial drag position
Individual drag position is not relevant, just that it’s a transition at a lower index

Environment

Angular: ?
CDK/Material: 8.2.1
Browser(s):
Operating System (e.g. Windows, macOS, Ubuntu):

The placeholder element is just another element inside the list so changing the z-index does work. Here's an example that has it always appearing over the other elements: https://stackblitz.com/edit/angular-1yremt?file=app/cdk-drag-drop-custom-placeholder-example.css.

Oh. I had to add position: relative for it to work. I guess there’s a workaround then. Would be nice if this was included in the generated css though.

Including it in the generated CSS might be a little too opinionated since it depends very much on the setup of your app and how it's supposed to behave. Closing since it works as expected.

Closing since it works as expected.

Thanks.
Would it make sense to fix this by updating the example snippet? I know I implemented by basically copy pasting the snippet, and then tailoring it to my own needs.
Had the z-index been there to begin with, it would have helped.


Replacing spaces with %20 in VSTS host urls

In VSTS you can have spaces in the URL to the repository, however URI doesn't support this. The solution I have implemented is to simply replace these spaces with %20 which is what VSTS itself does.

          1 Warning

:warning:
Tests were not updated

  Generated by :no_entry_sign: Danger

Ill implement a test later this evening for this.


Fix encryption keys

Not sure, but I think, the problem is a misconfigured .travis.yml file not missing keys. Turns out that the keys are not set in the current script. This might fix it. Hard to test.

Try to merge it and see what happens...

merging, with fingers crossed that this will fix it (no need to worry about tests; this only affects structure of the .travis.yml, and we won't know if it works until it is merged to master anyway)

Excellent! This fixed it. Still have lots of warnings to fix in the docs build, but at least the docs are getting uploaded! (I really needed that, since I want the report I need to finish this week to point to the new docs on TIS analysis -- thanks for figuring out the problem and quickly fixing it!)

No way, really? Wow, that was a long shot... Great...

I needed this to exist: http://openpathsampling.org/latest/topics/tis_analysis.html
Now it does! :-)


Should we have an alumni channel or role?

What are people's thoughts on having an alumni channel/role? Someone who is an alumni may not want to get a whole lot of notifications about normal club chatter, but still be kept in the loop, so having a specific spot they could get check notifications for could keep them updated on club stuff. It would also give us a single spot to ask them for help (about an old project if they were a lead or if they could get their company to sponsor a meeting or do a workshop with us)

I created a channel and a role. Will need to add it to the bot to join the role.

OK cool!


Improve casting

Describe the problem or limitation you are having in your project:
Currently you solution for casting GodotObjects is very limiting and performance hungry.
We need to find a better way of casting objects.
Describe how this feature / enhancement will help you overcome this problem or limitation:
Currently a user cannot cast a node to a custom class which derives from node as it has no from function.
Show a mock up screenshots/video or a flow diagram explaining how your proposal will work:
None to provide
Describe implementation detail for your proposal (in code), if possible:
None to provide
If this enhancement will not be used often, can it be worked around with a few lines of code?:
Not really.
Is there a reason why this should be in this project and not individually solved?:
Yes the casting is implemented in the core project.

Closed in favor of the merge of the bindings.


DocsArchive-<IP_ADDRESS>Auto archive content

Please don't merge this PR until target archive repo PR https://github.com/OPS-E2E-PPE/docs-archive-test-target/pull/10 is merged into live branch.
Auto archive content to https://github.com/OPS-E2E-PPE/docs-archive-test-target.git

Docs Build status updates of commit 378d94e:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

.openpublishing.redirection.json
:white_check_mark:Succeeded

archvieSourceFolder/TOC.yml
:white_check_mark:Succeeded

archvieSourceFolder/content/someContent.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


HDFS-16194. Simplify the code with DatanodeID#getXferAddrWithHostname

JIRA: HDFS-16194
Simplify the code with DatanodeID#getXferAddrWithHostname.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 55s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+0 :ok:
mvndep
11m 43s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
25m 43s

trunk passed

+1 :green_heart:
compile
6m 20s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
5m 43s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
1m 21s

trunk passed

+1 :green_heart:
mvnsite
3m 20s

trunk passed

+1 :green_heart:
javadoc
2m 27s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
3m 18s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
8m  6s

trunk passed

+1 :green_heart:
shadedclient
18m 33s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 26s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 57s

the patch passed

+1 :green_heart:
compile
6m  6s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
6m  6s

the patch passed

+1 :green_heart:
compile
5m 39s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
5m 39s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m 14s

the patch passed

+1 :green_heart:
mvnsite
2m 59s

the patch passed

+1 :green_heart:
javadoc
2m 11s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
2m 55s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
8m 37s

the patch passed

+1 :green_heart:
shadedclient
18m 25s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 28s

hadoop-hdfs-client in the patch passed.

+1 :green_heart:
unit
337m 25s

hadoop-hdfs in the patch passed.

-1 :x:
unit
32m 37s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
hadoop-hdfs-rbf in the patch passed.

+1 :green_heart:
asflicense
0m 56s

The patch does not generate ASF License warnings.

514m 32s

Reason
Tests

Failed junit tests
hadoop.hdfs.rbfbalance.TestRouterDistCpProcedure

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3354/1/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3354

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux 024829d4f93d 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk /<PHONE_NUMBER>bd9eb0233e3d9031538f4a77b89e04

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3354/1/testReport/

Max. process+thread count
2286 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3354/1/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop.hdfs.rbfbalance.TestRouterDistCpProcedure
This failed unit test is unrelated to the change. @jojochuang @ayushtkn @ferhui Could you please help review the code. Thanks.

Hi @tasanuma @Hexiaoqiao , could you please review the code? Thanks.

LGTM. +1

Thanks @Hexiaoqiao for your review.

+1, pending Jenkins.

:broken_heart: -1 overall

Vote
Subsystem
Runtime
Logfile
Comment

+0 :ok:
reexec
0m 51s

Docker mode activated.

_ Prechecks _

+1 :green_heart:
dupname
0m  0s

No case conflicting files found.

+0 :ok:
codespell
0m  1s

codespell was not available.

+1 :green_heart:
@author
0m  0s

The patch does not contain any @author tags.

-1 :x:
test4tests
0m  0s

The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.

_ trunk Compile Tests _

+0 :ok:
mvndep
12m 35s

Maven dependency ordering for branch

+1 :green_heart:
mvninstall
23m  0s

trunk passed

+1 :green_heart:
compile
5m 23s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
compile
4m 47s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
checkstyle
1m 12s

trunk passed

+1 :green_heart:
mvnsite
2m 54s

trunk passed

+1 :green_heart:
javadoc
2m 13s

trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
2m 56s

trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
7m  6s

trunk passed

+1 :green_heart:
shadedclient
16m 53s

branch has no errors when building and testing our client artifacts.

_ Patch Compile Tests _

+0 :ok:
mvndep
0m 22s

Maven dependency ordering for patch

+1 :green_heart:
mvninstall
2m 35s

the patch passed

+1 :green_heart:
compile
5m  9s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javac
5m  9s

the patch passed

+1 :green_heart:
compile
4m 47s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
javac
4m 47s

the patch passed

+1 :green_heart:
blanks
0m  0s

The patch has no blanks issues.

+1 :green_heart:
checkstyle
1m  7s

the patch passed

+1 :green_heart:
mvnsite
2m 38s

the patch passed

+1 :green_heart:
javadoc
1m 53s

the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04

+1 :green_heart:
javadoc
2m 36s

the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

+1 :green_heart:
spotbugs
7m 19s

the patch passed

+1 :green_heart:
shadedclient
16m 54s

patch has no errors when building and testing our client artifacts.

_ Other Tests _

+1 :green_heart:
unit
2m 15s

hadoop-hdfs-client in the patch passed.

-1 :x:
unit
333m 37s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
hadoop-hdfs in the patch passed.

-1 :x:
unit
31m 33s
/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
hadoop-hdfs-rbf in the patch passed.

+1 :green_heart:
asflicense
0m 55s

The patch does not generate ASF License warnings.

495m 30s

Reason
Tests

Failed junit tests
hadoop.hdfs.server.mover.TestMover

hadoop.hdfs.rbfbalance.TestRouterDistCpProcedure

Subsystem
Report/Notes

Docker
ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3354/2/artifact/out/Dockerfile

GITHUB PR
https://github.com/apache/hadoop/pull/3354

Optional Tests
dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell

uname
Linux 81a761efa091 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Build tool
maven

Personality
dev-support/bin/hadoop.sh

git revision
trunk / 90e060c2d467bc9f500c6f0dc5ff17e90e16edc2

Default Java
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Multi-JDK versions
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10

Test Results
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3354/2/testReport/

Max. process+thread count
2283 (vs. ulimit of 5500)

modules
C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project

Console output
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3354/2/console

versions
git=2.25.1 maven=3.6.3 spotbugs=4.2.2

Powered by
Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Those failed unit tests are unrelated to the change.
@tasanuma Please take a look. Thank you.

@Hexiaoqiao Could you review it again, and merge it?


Runtime: Use dimension labels instead of dimension names for pivoted exports

Similar to https://github.com/rilldata/rill/issues/3878, we should use label not name for dimensions.

https://github.com/rilldata/rill/pull/4456


Retrieving system properties

First, thanks for the great work!
I'd like to be able to retrieve system properties like __createdAt
As far as I see, this is currently not possible.
Fixed it for myself by changing getTable as follows:
...
var data = client.getTable(tableName);
data.systemProperties = WindowsAzure.MobileServiceTable.SystemProperties.CreatedAt;
return data;
Is there a better way? Or, could this be added as a feature?

I found a better way to do this:  Check out this blog post http://www.brandonmartinez.com/2014/10/22/retrieve-system-properties-in-azure-mobile-services-javascript-backend/
You add this to the scripts section for that table on the Azure management site.   Works like a champ.
Let me know if you need anything else.

Ah?
Thanks for that - I found that article but misunderstood and
tried something from the client - and that didn't work :o]
But indeed now I notice it says "in the Azure Mobile Services JavaScript
backend".
That should indeed solve my problem, so I'm alright, but you may wonder if
it should be the server or the client to decide which system properties...?
On 8 June 2015 at 03:23, Terry Moore<EMAIL_ADDRESS>wrote:

I found a better way to do this: Check out this blog post
http://www.brandonmartinez.com/2014/10/22/retrieve-system-properties-in-azure-mobile-services-javascript-backend/
You add this to the scripts section for that table on the Azure management
site. Works like a champ.
Let me know if you need anything else.
—
Reply to this email directly or view it on GitHub
https://github.com/TerryMooreII/angular-azure-mobile-service/issues/32#issuecomment-109820748
.

--
@FreeWillaert
https://twitter.com/intent/follow?original_referer=https%3A%2F%2Fabout.twitter.com%2Fresources%2Fbuttons&region=follow_link&screen_name=FreeWillaert&tw_p=followbutton
on
Twitter
http://facebook.com/dekaasstolp - whether you like it or not :o)
+32 498 91 44 31
sitcom bvba
Diestsesteenweg 111
3010 Kessel-Lo
BE0895.067.696

The only reason i can see MS doing this is so that you, the admin, has to specifically allow the client to see these internal properties.


Evaluates Tiles of DataSources that are not attached to a MapView as NOT visible

Thank you for contributing to harp.gl!
Before requesting a pull request, please remember to check the following documents:

contribution guidelines
coding style

If you are adding new functionality we would highly appreciate if you can describe what is the capability you are adding and even better if you can add some examples. Please also remember to add tests for it.
CI Check
Our bots will check whether your PR can be directly integrated into the mainline. We have some internal integration tests running on the background, our bots will inform you of the next steps and someone from our team will take a look and help if needed!
And please do not forget to sign-off your commit! You can read more about DCO here. But, in short, you just need to use git commit -s or append --signoff when you are committing to the repo.
Happy contributing!

@nzjony  I reverted to the original version as the changing of the DataSource API would cause a lot of issues deep through the code, I created HARP-11190 for a more detailed look into this.


Mobile - Update web-runner 1.1.64

Describe the task
Update web-runner 1.1.64

Refer: https://github.com/Koniverse/SubWallet-Extension/releases/tag/v1.1.64

Version test: https://www.diawi.com/qrcode/link/hroE17
Device test: IP 11

[ ] Show incorrect explorer for Invarch network
Actual: Navigate to https://invarch.statescan.io/address/i4x4ekSCSgD8LkDLvpApdrGEDw4CYAtTVxirme5kRbpqScBhV

Expect: Navigate to https://invarch.statescan.io/#/accounts/i4x4ekSCSgD8LkDLvpApdrGEDw4CYAtTVxirme5kRbpqScBhV

Version test: https://www.diawi.com/qrcode/link/wgnj7z
Device test: IP 11
Validation passed.

Update chain-list

Add support for Curio Network
Add new blockchain explorer for InvArch
Add explorer for Mythos and Avail Turing
Update ED for Mythos
Update symbol for native token on Commune AI: C -> COMAI

Fixed bug enable chains when detect balance
Fixed bug Invalid recipient address when Dapp deploy smart contract


Roll Flutter Engine from 135acd5a689a to 22cafb9af87b (1 revision)

https://github.com/flutter/engine/compare/135acd5a689a...22cafb9af87b
2024-05-04<EMAIL_ADDRESS>Roll Fuchsia Linux SDK from QDv0tal7_BgoZx0ju... to PqcaFvWlaz5ZrFcpC... (flutter/engine#52560)
Also rolling transitive DEPS:
fuchsia/sdk/core/linux-amd64 from QDv0tal7_Bgo to PqcaFvWlaz5Z
If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-engine-flutter-autoroll
Please CC aaclarke@google.com,rmistry<EMAIL_ADDRESS>on the revert to ensure that a human
is aware of the problem.
To file a bug in Flutter: https://github.com/flutter/flutter/issues/new/choose
To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622
Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md

Commit queue failed; closing this roll.


fix(meta): fix search method of createIndexGroupIfNeeded to decide wh…

What problem does this PR solve?

Signed-off-by: zhenyuxie<EMAIL_ADDRESS>Issue Number: close #41
Problem Summary:
The index group in openGemini like shard group, it is responsible for dividing the index data into corresponding time range. So, an index group only corresponds to a time range，that's to say a time range only corresponds to a index. But actually I found multiple indexes on the same time range. This bug causes index bloat, wasting performance and storage.
What is changed and how it works?
I fixed the method createIndexGroupIfNeeded in data.go and avoid creating duplicate indexes on the same time range. It worked well.

thx @zhenyuxie


evaluate() needs optional delay to prevent rate limiting by API

Describe the Feature
see title
Why is the feature important for you?
avoid rate limiting
Additional context
no

we actually have something for that - could you check out https://docs.ragas.io/en/stable/howtos/customizations/_run_config/  and let me know if it is working 🙂

ah ok thanks. Are these ""Exception raised in Job[200]: TimeoutError()
Exception raised in Job[198]: TimeoutError()"" self recovering, or is this saying it will skip over the data?

skip over check the raise_exceptions flag if you have doubts

I just mean it as a question, do these try the call again, or just skip them?

edited

Ah ok. By default, do you think it should try again? Skipping over examples due to rate limiting seems to skew the results of the entire evaluation. Do you think it should wait to try again?


FYI on dirvish/netrw style nvim ./

If lir.nvim is loaded within the vim.defer_fn() than dirvish/netrw style loading will not work properly. I do not know why this is, but just wanted to make you aware. I do not know if this is something that can be fixed or if the answer is to either avoid defer_fn entirely or to just be aware that if you use it you'll need to than call lir outside of it.

I'm closing this issue. I swapped to packer from paq and I can now use the fn_defer and the other issues are all gone as well. I'm going to assume it was something to do with how packadd was being called...?

I don't know the cause, but I'm glad it seems to have been resolved anyway.
Thank you for the report.


dask_cuda memory error when calculating dask.array correlation

Describe the issue:
I am looking to run pearson correlation on a very large matrix of features (1575538 x 292) for creation of a network graph, and output the results to disk without loading them to memory. I was looking to use dask.array for this, as I am calculating correlation per row, not per column (loading the transposed matrix into dask_cudf also does not run), to output a final 1M x 1M matrix. However, when running this using the dask.array.corrcoef function, it seems that the memory requirement/chunking is not handled properly, causing the code to crash. Changing the memory limits in the local cuda cluster does has not mitigated this issue for me.
Minimal Complete Verifiable Example:
# Put your MCVE code here
from dask_cuda import LocalCUDACluster
from dask.distributed import Client
import cudf
import cupy
import dask_cudf
import dask.array as da
import tiledb

dda = cupy.random.random((1575538, 292))
df = da.from_array(dda)
res = da.corrcoef(df)
res.to_tiledb('tiles')

Error:
2023-04-18 13:15:14,941 - distributed.worker - WARNING - Compute Failed
Key:       ('sum-sum-aggregate-truediv-truediv-05744de3d956b8746945bbc779e50d30', 1, 1)
Function:  execute_task
args:      ((subgraph_callable-83f21de4-662c-442e-932f-3f06ffeee06f, (Compose(functools.partial(<function sum at 0x7fffec769bd0>, dtype=dtype('float64'), axis=(1,), keepdims=False), functools.partial(<function _concatenate2 at 0x7fff62c7c4c0>, axes=[1])), [(subgraph_callable-4e2c394e-b261-4e77-8a62-8a537598cd8a, array([[-0.49037911, -0.33841151, -0.29730389, ..., -0.19010068,
        -0.07755179,  0.23832127],
       [ 0.22536128,  0.13509355, -0.01844846, ..., -0.4230372 ,
        -0.35520002,  0.23595457],
       [ 0.40335423,  0.25512431,  0.07122093, ...,  0.48374337,
        -0.28509283,  0.36211846],
       ...,
       [-0.47958972, -0.09360203, -0.30290464, ..., -0.30419598,
        -0.04496128,  0.02401314],
       [ 0.24298105,  0.43314774,  0.24283649, ..., -0.378227  ,
         0.07893316,  0.00610615],
       [ 0.28571441, -0.07779414, -0.33461673, ..., -0.12132853,
         0.11584436, -0.47641358]]), array([[-0.49037911, -0.33841151, -0.29730389, ..., -0.19010068,
        -0.0775517
kwargs:    {}
Exception: "OutOfMemoryError('Out of memory allocating 26,409,535,488 bytes (allocated so far: 53,224,481,792 bytes).')"

2023-04-18 13:15:15,750 - distributed.worker - WARNING - Compute Failed
Key:       ('sum-sum-aggregate-truediv-truediv-05744de3d956b8746945bbc779e50d30', 0, 10)
Function:  execute_task
args:      ((subgraph_callable-83f21de4-662c-442e-932f-3f06ffeee06f, (Compose(functools.partial(<function sum at 0x7fffec769bd0>, dtype=dtype('float64'), axis=(1,), keepdims=False), functools.partial(<function _concatenate2 at 0x7fff62c7c4c0>, axes=[1])), [(subgraph_callable-4e2c394e-b261-4e77-8a62-8a537598cd8a, array([[-0.09245323, -0.44134584,  0.02786883, ..., -0.40654986,
         0.0754504 ,  0.17065629],
       [-0.33108703, -0.19285417,  0.17163849, ...,  0.12303687,
        -0.04800523, -0.27986298],
       [ 0.26232529,  0.3914969 ,  0.44120061, ..., -0.10972233,
        -0.39118952,  0.17157114],
       ...,
       [ 0.17873431, -0.0227869 , -0.00081084, ...,  0.11129026,
        -0.04977892, -0.1175841 ],
       [ 0.15734707,  0.505664  , -0.18658433, ..., -0.40455414,
        -0.20738407,  0.06418578],
       [ 0.32899475,  0.10454224, -0.35374564, ...,  0.27058079,
         0.2201003 ,  0.20427084]]), array([[ 0.47533733,  0.16476587, -0.19923557, ...,  0.45717506,
         0.4214467
kwargs:    {}
Exception: "OutOfMemoryError('Out of memory allocating 26,409,535,488 bytes (allocated so far: 79,768,234,496 bytes).')"

2023-04-18 13:15:15,909 - distributed.worker - WARNING - Compute Failed
Key:       ('sum-sum-aggregate-truediv-truediv-05744de3d956b8746945bbc779e50d30', 0, 1)
Function:  execute_task
args:      ((subgraph_callable-83f21de4-662c-442e-932f-3f06ffeee06f, (Compose(functools.partial(<function sum at 0x7fffec769bd0>, dtype=dtype('float64'), axis=(1,), keepdims=False), functools.partial(<function _concatenate2 at 0x7fff62c7c4c0>, axes=[1])), [(subgraph_callable-4e2c394e-b261-4e77-8a62-8a537598cd8a, array([[-0.09245323, -0.44134584,  0.02786883, ..., -0.40654986,
         0.0754504 ,  0.17065629],
       [-0.33108703, -0.19285417,  0.17163849, ...,  0.12303687,
        -0.04800523, -0.27986298],
       [ 0.26232529,  0.3914969 ,  0.44120061, ..., -0.10972233,
        -0.39118952,  0.17157114],
       ...,
       [ 0.17873431, -0.0227869 , -0.00081084, ...,  0.11129026,
        -0.04977892, -0.1175841 ],
       [ 0.15734707,  0.505664  , -0.18658433, ..., -0.40455414,
        -0.20738407,  0.06418578],
       [ 0.32899475,  0.10454224, -0.35374564, ...,  0.27058079,
         0.2201003 ,  0.20427084]]), array([[-0.49037911, -0.33841151, -0.29730389, ..., -0.19010068,
        -0.0775517
kwargs:    {}
Exception: "OutOfMemoryError('Out of memory allocating 26,409,535,488 bytes (allocated so far: 53,087,505,408 bytes).')"

---------------------------------------------------------------------------
OutOfMemoryError                          Traceback (most recent call last)
Cell In[10], line 1
----> 1 res.to_tiledb('tiles')

File ~/data/dir/anaconda3/envs/rapids-23.02/lib/python3.10/site-packages/dask/array/core.py:2957, in Array.to_tiledb(self, uri, *args, **kwargs)
   2945 """Save array to the TileDB storage manager
   2946
   2947 See https://docs.tiledb.io/ for details about the format and engine.
   (...)
   2953 dask.array.to_tiledb : equivalent function
   2954 """
   2955 from dask.array.tiledb_io import to_tiledb
-> 2957 return to_tiledb(self, uri, *args, **kwargs)

File ~/data/dir/anaconda3/envs/rapids-23.02/lib/python3.10/site-packages/dask/array/tiledb_io.py:165, in to_tiledb(darray, uri, compute, return_stored, storage_options, key, **kwargs)
    162 if not (tdb.isopen and tdb.iswritable):
    163     raise ValueError("Target TileDB array is not open and writable.")
--> 165 return darray.store(tdb, lock=False, compute=compute, return_stored=return_stored)

File ~/data/dir/anaconda3/envs/rapids-23.02/lib/python3.10/site-packages/dask/array/core.py:1767, in Array.store(self, target, **kwargs)
   1765 @wraps(store)
   1766 def store(self, target, **kwargs):
-> 1767     r = store([self], [target], **kwargs)
   1769     if kwargs.get("return_stored", False):
   1770         r = r[0]

File ~/data/dir/anaconda3/envs/rapids-23.02/lib/python3.10/site-packages/dask/array/core.py:1235, in store(***failed resolving arguments***)
   1233 elif compute:
   1234     store_dsk = HighLevelGraph(layers, dependencies)
-> 1235     compute_as_if_collection(Array, store_dsk, map_keys, **kwargs)
   1236     return None
   1238 else:

File ~/data/dir/anaconda3/envs/rapids-23.02/lib/python3.10/site-packages/dask/base.py:341, in compute_as_if_collection(cls, dsk, keys, scheduler, get, **kwargs)
    339 schedule = get_scheduler(scheduler=scheduler, cls=cls, get=get)
    340 dsk2 = optimization_function(cls)(dsk, keys, **kwargs)
--> 341 return schedule(dsk2, keys, **kwargs)

File ~/data/dir/anaconda3/envs/rapids-23.02/lib/python3.10/site-packages/distributed/client.py:3137, in Client.get(self, dsk, keys, workers, allow_other_workers, resources, sync, asynchronous, direct, retries, priority, fifo_timeout, actors, **kwargs)
   3135         should_rejoin = False
   3136 try:
-> 3137     results = self.gather(packed, asynchronous=asynchronous, direct=direct)
   3138 finally:
   3139     for f in futures.values():

File ~/data/dir/anaconda3/envs/rapids-23.02/lib/python3.10/site-packages/distributed/client.py:2306, in Client.gather(self, futures, errors, direct, asynchronous)
   2304 else:
   2305     local_worker = None
-> 2306 return self.sync(
   2307     self._gather,
   2308     futures,
   2309     errors=errors,
   2310     direct=direct,
   2311     local_worker=local_worker,
   2312     asynchronous=asynchronous,
   2313 )

File ~/data/dir/anaconda3/envs/rapids-23.02/lib/python3.10/site-packages/distributed/utils.py:338, in SyncMethodMixin.sync(self, func, asynchronous, callback_timeout, *args, **kwargs)
    336     return future
    337 else:
--> 338     return sync(
    339         self.loop, func, *args, callback_timeout=callback_timeout, **kwargs
    340     )

File ~/data/dir/anaconda3/envs/rapids-23.02/lib/python3.10/site-packages/distributed/utils.py:405, in sync(loop, func, callback_timeout, *args, **kwargs)
    403 if error:
    404     typ, exc, tb = error
--> 405     raise exc.with_traceback(tb)
    406 else:
    407     return result

File ~/data/dir/anaconda3/envs/rapids-23.02/lib/python3.10/site-packages/distributed/utils.py:378, in sync.<locals>.f()
    376         future = asyncio.wait_for(future, callback_timeout)
    377     future = asyncio.ensure_future(future)
--> 378     result = yield future
    379 except Exception:
    380     error = sys.exc_info()

File ~/data/dir/anaconda3/envs/rapids-23.02/lib/python3.10/site-packages/tornado/gen.py:769, in Runner.run(self)
    766 exc_info = None
    768 try:
--> 769     value = future.result()
    770 except Exception:
    771     exc_info = sys.exc_info()

File ~/data/dir/anaconda3/envs/rapids-23.02/lib/python3.10/site-packages/distributed/client.py:2169, in Client._gather(self, futures, errors, direct, local_worker)
   2167         exc = CancelledError(key)
   2168     else:
-> 2169         raise exception.with_traceback(traceback)
   2170     raise exc
   2171 if errors == "skip":

File ~/data/dir/anaconda3/envs/rapids-23.02/lib/python3.10/site-packages/cytoolz/functoolz.pyx:504, in cytoolz.functoolz.Compose.__call__()
    502 ret = PyObject_Call(self.first, args, kwargs)
    503 for func in self.funcs:
--> 504     ret = func(ret)
    505 return ret
    506

File <__array_function__ internals>:180, in sum()

File cupy/_core/core.pyx:1701, in cupy._core.core._ndarray_base.__array_function__()

File ~/data/dir/anaconda3/envs/rapids-23.02/lib/python3.10/site-packages/cupy/_math/sumprod.py:38, in sum()
     34     return _fusion_thread_local.call_reduction(
     35         func, a, axis=axis, dtype=dtype, out=out)
     37 # TODO(okuta): check type
---> 38 return a.sum(axis, dtype, out, keepdims)

File cupy/_core/core.pyx:1091, in cupy._core.core._ndarray_base.sum()

File cupy/_core/core.pyx:1100, in cupy._core.core._ndarray_base.sum()

File cupy/_core/_routines_math.pyx:126, in cupy._core._routines_math._ndarray_sum()

File cupy/_core/_reduction.pyx:568, in cupy._core._reduction._SimpleReductionKernel.__call__()

File cupy/_core/_reduction.pyx:327, in cupy._core._reduction._AbstractReductionKernel._call()

File cupy/_core/_reduction.pyx:609, in cupy._core._reduction._SimpleReductionKernel._get_out_args()

File cupy/_core/_kernel.pyx:644, in cupy._core._kernel._get_out_args_from_optionals()

File cupy/_core/core.pyx:2765, in cupy._core.core._ndarray_init()

File cupy/_core/core.pyx:241, in cupy._core.core._ndarray_base._init_fast()

File cupy/cuda/memory.pyx:742, in cupy.cuda.memory.alloc()

File cupy/cuda/memory.pyx:1419, in cupy.cuda.memory.MemoryPool.malloc()

File cupy/cuda/memory.pyx:1440, in cupy.cuda.memory.MemoryPool.malloc()

File cupy/cuda/memory.pyx:1120, in cupy.cuda.memory.SingleDeviceMemoryPool.malloc()

File cupy/cuda/memory.pyx:1141, in cupy.cuda.memory.SingleDeviceMemoryPool._malloc()

File cupy/cuda/memory.pyx:1379, in cupy.cuda.memory.SingleDeviceMemoryPool._try_malloc()

OutOfMemoryError: Out of memory allocating 26,409,535,488 bytes (allocated so far: 53,224,481,792 bytes).

Anything else we need to know?:
Environment:

Dask version:
dask - 2023.1.1
dask-cuda - 23.02.01
dask-cudf - 23.02.00
Python version: 3.10.10
Operating System: RHEL 7.9
Install method (conda, pip, source): conda (mamba)

I realized that this issue likely should be posted at the dask_cuda github page, someone may confirm and close this if that is the case.

responding in the dask-cuda repo


⚠ Processors has degraded performance

In 03d1316, Processors ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 46 ms

Resolved: Processors performance has improved in 260fe63.


update copa slack channel

updates copa slack channel to #copacetic

Hello! Thanks for the update, but this data is pulled in from CNCF landscape into this autogenerated file. You can submit a PR to the landscape repo to update your project data: https://github.com/cncf/landscape


[Feature Request] Support for gofr framework

Tell us about your feature request
I would like to add support for the gofr framework.
GoFr is an opinionated microservice development framework.
Here is the link to the github repo https://github.com/gofr-dev/gofr
Disclaimer

[X] I agree

We think that this framework would not fit into the current project structure. It is very opinionated. The goal is to provide an initial project structure as a starting point, not a full solution. Thanks for the suggestion.


Symmetric quantization with activations 16-bit and weights 8-bit: interface

In this PR we add a new option TFLITE_BUILTINS_ACTIVATIONS_INT16_WEIGHTS_INT8 to enable quantization with 16-bit activations and weights 8-bit.
When it is set, then we do post-training symmetric quantization with 16-bit activations and 8-bit weights. The bias is 64-bit in this case. It behaves the same way as TFLITE_BUILTINS_INT8.
Example of usage:
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_ACTIVATIONS_INT16_WEIGHTS_INT8]
The name is quite long even it is explanatory.
May be, we should use something like  TFLITE_BUILTINS_INT16x8.
Any suggestions are welcome.
Implementations of reference kernels are submitted in other PRs.

@suharshs Thanks for the review!
I will correct according to the suggestions and re-test internally.

Hi @suharshs, I added "non-strict" mode to this PR in the last commit. Now all changes to introduce 16x8 mode is here. Could you please take a look when you have a time ? Thanks

This overall looks good now.
Before submitting this change, we need to ensure that the kernels are in, and then in one change we need to bump the version number of all the 16 bit ops, and ensure that this tool uses that new version when applying 16 bit operations.
Let's first get the rest of the kernels submitted, then send a single PR that updates the version for all the kernels, and finally update this PR with those version numbers.

Thanks for the review! The plan sounds good to me.

@wwwind Can you please resolve conflicts? Thanks!

@gbaned conflicts are resolved

This overall looks good now.
Before submitting this change, we need to ensure that the kernels are in, and then in one change we need to bump the version number of all the 16 bit ops, and ensure that this tool uses that new version when applying 16 bit operations.
Let's first get the rest of the kernels submitted, then send a single PR that updates the version for all the kernels, and finally update this PR with those version numbers.

@suharshs dependent changes are submitted internally or there is a PR ?

Adding Feng to speak to how to add support in the mlir code path too.

Hi @liufengdb , I tested all my models with the flag experimental_new_converter = True (default)
and all models converted without problems to 16x8 (activations int16, weights int8) mode, with description "MLIR converted", etc.
I know that there is _experimental_new_quantizer flag, but this path is in the active development, so I keep an eye on it, but it looks that it is not needed for now.
All our essential changes are in quantize_model.cc.
Please let me know if something need to be done in addition.
*There is merge conflict on this PR - changes to resolve it are under testing right now.
Thanks!

Hi @wwwind, it is great to see the patch works well. Internally we have decided to migrate to the new quantizer, and, of course, we want to support this 16 bits activation feature in the new quantizer. My only concern is how to verify the results, because most of the tests in this patch seems tied to the old quantizer.

Thanks @liufengdb for the support in the new quantizer.

Yes, we have not added a lot of tests here, but I am ready to add unit tests with models that cover all our operators. Like in quantize_model_test.cc. Is it okay ? Perhaps, should I do this in the additional PR, because this one is too big already.

Internally we have a set of end2end dummy models for all reference kernels that we have implemented:
we create one layer model and quantize it to 16x8 and do some checks

proper accuracy testing on classic models with 16x8 mode.
In all these tests we use installed patched tf.
Is there any place in tensorflow where I can add similar tests?

What is the timeline for the migration to the new quantizer ? When does it make sense for us to start testing it ?

Thanks!

add unit tests with models that cover all our operators.
Internally we have a set of end2end dummy models for all reference kernels that we have >implemented. we create one layer model and quantize it to 16x8 and do some checks
A good place for these per-op tests can be extent from these tests:
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/testing/op_tests

Is there any place in tensorflow where I can add similar tests?
There are some tests are using import tf directly: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/testing/generate_examples.py#L32

What is the timeline for the migration to the new quantizer? When does it make sense for us to start testing it ?
My plan is to switch the flag internally by the end of this quarter and make it public externally in the middle of next quarter. We have some internal users with the new converter, and most of the features are ready, except this 16 bits. I can add the 16bits support next week, and later on, you add the tests to tensorflow/lite/testing/op_tests?

Thanks! This sounds good. I will prepare PR with these tests.

Hi @liufengdb ! I created a set of tests for 16x8 quantization in ops_test:
https://github.com/tensorflow/tensorflow/pull/39543
My concern is that these tests are for the old converter:
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/testing/toco_convert.py#L123
If the function QuantizeModel stays the same for the new quantizer, then
in this PR I have parametrized with int16 all tests here.
This should be a good cover for 16x8 case.
Thanks!

I will switch the zip tests to use the new converter. Then you can submit #39543.

@wwwind can you please resolve conflicts and check sanity errors

@liufengdb I updated - please take a look
There is a failure in CI with
//tensorflow/tools/api/tests:api_compatibility_test
File "/home/elezhe01/.local/lib/python3.6/site-packages/tensorflow_estimator/python/estimator/canned/dnn.py", line 23, in <module>
  from tensorflow.python.feature_column import dense_features
ImportError: cannot import name 'dense_features'

But I checked that the same error is in master, so it's not specific to this PR.

I tested this PR on our set of models - everything is working as should be when
experimental_new_converter = True
Inference is broken now, when experimental_new_converter = False.
Do we need to support this case ?

I tested this PR on our set of models - everything is working as should be when
experimental_new_converter = True
Inference is broken now, when experimental_new_converter = False.
Do we need to support this case ?

No, but we should:

Document this restriction in the API docs (and eventual guide)
Throw an exception if trying to use this flag when the new converter is disabled

Hi @rthadur Thanks! I pushed a fix for these errors.
@jdduke, the option is renamed as suggested.

Hi @rthadur
There is a failure of the test in Ubuntu CPU checks:
//tensorflow/tools/api/tests:api_compatibility_test
I checked master and this test fails their as well
Thanks

@wwwind  Can you please address Ubuntu Sanity errors? Thanks!

Hi @rthadur, Could you please re-approve this PR ?
I found a problem and fixed this CI failure finally.
Thanks!

Hi @jdduke I added description to API as requested.
Please take a look.
Thanks!

Hi @rthadur Sorry to bother you, but could you please re-approve this PR?
I had to push fixes to pylint errors.
Thanks!

Hi @jdduke ! Thanks for the review! Documentation is merged into this PR and comment is corrected, + small fix for pylint.
Could you please re-approve ?
Thanks!

For @tensorflow/api-owners, this looks good.

Hi @jdduke Could you please help with this PR ?
There are failures but they don't look relevant to my PR.
I tried these targets locally and they are green:
//tensorflow/lite/tools/optimize/calibration:logging_op_resolver (Linux GPU)
//tensorflow/tools/ci_build/builds:gen_win_out (Windows bazel)
Is it possible to re-run CI somehow ?

I run the failing target locally:
bazel test //tensorflow/lite/tools/optimize/calibration:logging_op_resolver
and it's green locally.

run the failing target locally:
bazel test //tensorflow/lite/tools/optimize/calibration:logging_op_resolver
and it's green locally.

Right, I think there was a broken build at head, so probably just needed a rebase.

Hi @jdduke I pushed a fix. Could you please re-approve ?
Thanks!

The build error in "MacOS CPU Python3" is reproducable in master branch

@rthadur can you help push this through internal migration?

Hi @rthadur I have corrected. Could you please re-approve ? Thanks!

Hi @jdduke Could you please re-approve this PR? I pushed a small change as requested.
Thanks!

Hi @rthadur there is again failure on this PR in "MacOS CPU Python3".
It is the same as before and it is reproducible in master branch for me locally.
thanks

@wwwind will you be able to fix the error ?

Hi, I will try to patch this internally to resolve the tests. Thanks.


Pagespeed is not rewriting my website

I use the Nginx 1.8.0 and WP Super Cache.
I have rebuilt Nginx and the module is included when I type the command
Nginx -V
but when i type
curl -D- http://mydomain.com | less
I don't see the X-Pagespeed there..
What is wrong and what do you suggest.
Thanks.

do you have pagespeed on in your configuration?
See this page for details: https://developers.google.com/speed/pagespeed/module/configuration

I did. but never mind. got a new server. will work on it later. thanks


Support running on single SVM

Does the plugin support logging in to a cluster/data vserver?

Currently trident requires a cluster-scoped user account to work, however we do have SVM-scoped account support on the roadmap

Tracked internally as US2167

I built docker images based off the latest code base. They are at https://hub.docker.com/r/carln/
Using the latest code trident container is now able to provision volumes on an SVM, using the SVM interface.
Thanks guys.

Fixed in 17.04.0


Do u plan to make a discord room like shiro?
Will we have subscribed anime shows?
Will there be a way to see downloading videos on the app? I know its on the notification bar but can we have?
Thumbnails and titles?
TV
Is there a way to refresh the app on tv? hopefully there is i cant play/find latest episodes without refreshing the app
Multiple accounts?

Do u plan to private the app? i dont want "no" to shut down again

I have no plans of making the app private.

Does gogoanime allow what ur doing?

Likely not, I have not notified nor talked to them.

Do u plan to make a discord room like shiro?

I am considering it.

Will we have subscribed anime shows?

Maybe.

Will there be a way to see downloading videos on the app? I know its on the notification bar but can we have?
Thumbnails and titles?

Likely, but not too soon.

Is there a way to refresh the app on tv?

No, but If you want to refresh the latest episode of an anime re-opening the anime info page is sufficient.
NOTE: Swiping down on the phone version does not refresh the home page, only the random poster.

Multiple accounts?

Maybe.


spec issue: DMA_MAP MAPPABLE flag

The MAPPABLE flag in 'flags' field of the DMA_MAP struct is vfio-user-specific, but currently we sort of re-use the VFIO flags.
Instead, now we accept only one map at a time, we can drop the flag and presume that if an fd is provided, it's mappable, and vice versa.

JJ pushed the fix here: https://github.com/oracle/qemu/commit/9baf27a8fb99392a4ff85071670cd40cec1ef4af
Need to adapt libvfio-user and test.

Fixed by #553


Upgrade to Svelte v4

Can this package be upgraded to support Svelte v4?
Currently it causes this string of errors from NPM:
npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR!
npm ERR! While resolving<EMAIL_ADDRESS>npm ERR! Found<EMAIL_ADDRESS>npm ERR! node_modules/svelte
npm ERR!   dev<EMAIL_ADDRESS>from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer<EMAIL_ADDRESS>from<EMAIL_ADDRESS>npm ERR! node_modules/svelte-local-storage-store
npm ERR<EMAIL_ADDRESS>from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
npm ERR!
npm ERR!
npm ERR! For a full report see:
npm ERR! /home/user/.npm/_logs/2023-08-08T11_14_36_997Z-eresolve-report.txt

npm ERR! A complete log of this run can be found in: /home/user/.npm/_logs/2023-08-08T11_14_36_997Z-debug-0.log

Thanks!

I opened a pull request for this issue, #183 I'll close this now


Converting sparse matrix errors

Am seeing the following error when trying to convert a csc_matrix to a coo_matrix. Though also see some variation of this when trying to go to a csr_matrix from a csc_matrix and other similar conversions.

Conditions (you can just paste the output of python -c 'import cupy; cupy.show_config()')

CuPy Version          : 7.2.0
CUDA Root             : /datasets/jkirkham/miniconda/envs/rapids13dev
CUDA Build Version    : 10010
CUDA Driver Version   : 10020
CUDA Runtime Version  : 10010
cuBLAS Version        : 10201
cuFFT Version         : 10101
cuRAND Version        : 10101
cuSOLVER Version      : (10, 2, 0)
cuSPARSE Version      : 10300
NVRTC Version         : (10, 1)
cuDNN Build Version   : 7605
cuDNN Version         : 7605
NCCL Build Version    : 2406
NCCL Runtime Version  : 2507

Code to reproduce

In [1]: import cupy

In [2]: a = cupy.array([[0, 1, 0],
   ...:                 [2, 0, 3],
   ...:                 [0, 4, 0]], dtype=float)

In [3]: a2 = cupy.sparse.csc_matrix(a)

In [4]: a3 = a2.tocoo()

Error messages, stack traces, or logs

---------------------------------------------------------------------------
CUDADriverError                           Traceback (most recent call last)
<ipython-input-4-14bf9613c858> in <module>
----> 1 a3 = a2.tocoo()

/datasets/jkirkham/miniconda/envs/rapids13dev/lib/python3.7/site-packages/cupyx/scipy/sparse/csc.py in tocoo(self, copy)
    179
    180         """
--> 181         return self.T.tocoo(copy).T
    182
    183     def tocsc(self, copy=None):

/datasets/jkirkham/miniconda/envs/rapids13dev/lib/python3.7/site-packages/cupyx/scipy/sparse/csr.py in tocoo(self, copy)
    239             indices = self.indices
    240
--> 241         return cusparse.csr2coo(self, data, indices)
    242
    243     def tocsc(self, copy=False):

/datasets/jkirkham/miniconda/envs/rapids13dev/lib/python3.7/site-packages/cupy/cusparse.py in csr2coo(x, data, indices)
    631     # data and indices did not need to be copied already
    632     return cupyx.scipy.sparse.coo_matrix(
--> 633         (data, (row, indices)), shape=x.shape)
    634
    635

/datasets/jkirkham/miniconda/envs/rapids13dev/lib/python3.7/site-packages/cupyx/scipy/sparse/coo.py in __init__(self, arg1, shape, dtype, copy)
    128
    129         if len(data) > 0:
--> 130             if row.max() >= shape[0]:
    131                 raise ValueError('row index exceeds matrix dimensions')
    132             if col.max() >= shape[1]:

cupy/core/core.pyx in cupy.core.core.ndarray.__richcmp__()

cupy/core/_kernel.pyx in cupy.core._kernel.ufunc.__call__()

cupy/core/_kernel.pyx in cupy.core._kernel.ufunc._get_ufunc_kernel()

cupy/core/_kernel.pyx in cupy.core._kernel._get_ufunc_kernel()

cupy/core/_kernel.pyx in cupy.core._kernel._get_simple_elementwise_kernel()

cupy/core/carray.pxi in cupy.core.core.compile_with_cache()

/datasets/jkirkham/miniconda/envs/rapids13dev/lib/python3.7/site-packages/cupy/cuda/compiler.py in compile_with_cache(source, options, arch, cache_dir, extra_source, backend)
    285     else:
    286         return _compile_with_cache_cuda(source, options, arch, cache_dir,
--> 287                                         extra_source, backend)
    288
    289

/datasets/jkirkham/miniconda/envs/rapids13dev/lib/python3.7/site-packages/cupy/cuda/compiler.py in _compile_with_cache_cuda(source, options, arch, cache_dir, extra_source, backend)
    333             cubin_hash = six.b(hashlib.md5(cubin).hexdigest())
    334             if hash == cubin_hash:
--> 335                 mod.load(cubin)
    336                 return mod
    337

cupy/cuda/function.pyx in cupy.cuda.function.Module.load()

cupy/cuda/function.pyx in cupy.cuda.function.Module.load()

cupy/cuda/driver.pyx in cupy.cuda.driver.moduleLoadData()

cupy/cuda/driver.pyx in cupy.cuda.driver.check_status()

CUDADriverError: CUDA_ERROR_ILLEGAL_ADDRESS: an illegal memory access was encountered

Let me guess: try setting cub_enabled=False?

I could have sworn I tried that already. Sorry. Yeah you are right another CUB issue.

Note max() is called prior to the rich comp. At that time the memory is already corrupted...Same bug as in #3092.

Thanks for spotting that 🙂 Yeah let's close. Sorry for the noise

No I think you should open it, John. It shows how widely impactful the CUB module can be, and how critical it is to get it bundled and tested.

Sure if it is helpful to have open. No objections. Reopening...
Thanks Leo 🙂

Confirmed this is fixed in the updated conda package (conda-forge/cupy-feedstock#39).

Thanks Leo! 😄

#3152 is merged but it didn't automatically close this issue?


Make the scala API more user-friendly

The scala DSL requires the import of various internal objects. This should be cleaner:

one import for all the transforms that mimic the official python API
one import to add all the implicits in scope

First cut mostly done. Missing tests and documentation.


Unable to upload images to own server

we are using "angular-froala": "2.3.1" ,
and Getting the following error  for imageUpload
" XMLHttpRequest cannot load http://localhost:8080/talent-wagon/api/v0/media?mediafor=statement&id=1. Response to preflight request doesn't pass access control check: The value of the 'Access-Control-Allow-Origin' header in the response must not be the wildcard '*' when the request's credentials mode is 'include'. Origin 'http://localhost:9000' is therefore not allowed access. The credentials mode of requests initiated by the XMLHttpRequest is controlled by the withCredentials attribute. "
froala options configured as follows in controller
$scope.froalaOptions = {
toolbarButtons: ["bold", "italic", "underline", "|", "align", "formatOL", "formatUL", "codePanel", "insertImage", "insertLink"],
height: 200,
html:{
'get': function(){
console.log("here")
}
},
imageUploadURL: AppConfig.API_URL + 'media?mediafor=statement&id=' + $scope.challengeId,
imageUploadMethod: 'POST'
};

@sandeep-purma that is because on the remote server where you're uploading the image, you should enable CORS: https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS.

@sandeep-purma that is because on the remote server where you're uploading the image, you should enable CORS: https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS.


Investigate Metamask Snap and create technical documentation

Description
This is my first time implementing Metamask snap, so I'm going to do

quick stark
look for the Account Abstraction example
create a technical document

Then get feedback from the Metamask team.
Also, I will crate a list of challenges or benefits when building snaps or using Truffle to apply bounty.

This is the tutorial
https://github.com/MetaMask/snap-box

Unbox like this.

I'm able to send custom message

It seems:

Frontend uses Gatsby and Styled Components.
It gives pre-customed Metamask Snap
It also gives a truffle contract directory
It configures monorepo

Pros

It makes me start development quickly

Cons

It is a bit different from my usual development setup, so I need to adjust a bit to develop


[python] Python doc-string audits

Two phases:

Make sure everything has something at all -- something that will appear when a user types help(soma.foo) as rendered by the Python interpreter itself
Tweaking style (single vs double backticks, etc etc etc) which control how docstrings are rendered by various systems including readthedocs

This issue is about the former; see #331 for the latter.

Re-opening as
https://github.com/single-cell-data/TileDB-SOMA/blob/a7b33be1c7f70bfc6c01fb6e9d902a0ab383a977/apis/python/src/tiledbsoma/options/tiledb_create_options.py#L51-L96
reveals a great many functions without docstrings.
Morever, I should probably make this a keep-open-until-release issue.

P.S. I am taking a look at
https://pypi.org/project/docstr-coverage/

P.S. I am taking a look at
https://pypi.org/project/docstr-coverage/

@johnkerl - regarding the above statement:

Tweaking style (single vs double backticks, etc etc etc) which control how docstrings are rendered by various systems including readthedocs

Is there a specific list of requirements?  Just double-backtick, or other baseline requirements for a docstring?

@bkmartinjr I had anecdotal observations & shared what I knew at the time -- I don't have encyclopedic knowledge to share.

I had anecdotal observations & shared what I knew at the time -- I don't have encyclopedic knowledge to share.

Oh, I was hoping we had a existing standard that you could point me at, rather than having us make a new one.  I've sent some options (pandas, numpy) via Slack. No reason to think those are right, but there needs to be a target for us to shoot at :-)


KeyError

Hi,
Thanks for the nice tool. Im running into this error which pops up straight away after running:
~/miniconda3/bin/savana --tumour COLO829/PAO32033.cram --normal COLO829BL/PAO33946.cram --outdir savana_out --ref GCA_000001405.15_GRCh38_no_alt_analysis_set.fna

Version 0.2.3 - beta
Source: /Users/kezcleal/miniconda3/lib/python3.10/site-packages/savana/savana.py

Running as sample PAO32033
Creating directory /Volumes/Kez6T/colo829/kit14/savana_out to store results
Using GCA_000001405.15_GRCh38_no_alt_analysis_set.fna.fai as reference fasta index
Using multiprocessing with 8 threads

Traceback (most recent call last):
  File "/Users/kezcleal/miniconda3/bin/savana", line 10, in <module>
    sys.exit(main())
  File "/Users/kezcleal/miniconda3/lib/python3.10/site-packages/savana/savana.py", line 281, in main
    consensus_clusters, checkpoints, time_str = spawn_processes(args, bam_files, checkpoints, time_str, outdir)
  File "/Users/kezcleal/miniconda3/lib/python3.10/site-packages/savana/savana.py", line 160, in spawn_processes
    validated_breakpoints = call_breakpoints(clusters, args.buffer)
  File "/Users/kezcleal/miniconda3/lib/python3.10/site-packages/savana/breakpoints.py", line 205, in call_breakpoints
    for insertion_cluster in clustered_breakpoints[bp_type]:
KeyError: '<INS>'

Hi there, thanks for raising this issue. Unfortunately at the moment SAVANA doesn't support the use of .cram files - only .bam. This is functionality I can look into adding though.
For now, if you convert your cram files to bam does this resolve the issue?

I converted a few Mb to bam and I think it works, thanks. I had a go at changing some of the source code to get it working but not luck yet. Thanks for the quick reply

Hi there, I've implemented CRAM support in the latest commit (v1.0.3 - I'll create a release for it soon). However, SAVANA will run much more slowly on CRAM files than BAM files. This is because I rely on the get_index_statistics functionality of pysam to optimise memory usage. This information isn't available for CRAM files though (see: https://github.com/pysam-developers/pysam/issues/1060).
So I'd still recommend converting to BAM if you'd like faster speeds, but if that's not a concern, CRAM files should now work. If you know your chromosomes of interest, it's highly recommended that you supply them via the --contigs argument. This will prevent worker threads being spawned for thousands of contigs which have no reads mapped to them.


[Enhancement] Fix udf-extensions security issue

What type of PR is this：

[ ] BugFix
[ ] Feature
[x] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Which issues of this PR fixes ：

Fixes #
Problem Summary(Required) ：

udf-extensions jackson-databind version is too low it will cause many of security problem if user use this version.
Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] I have added user document for my new feature or new function

run starrocks_admit_test


Refactor method names in message.py to be more descriptive

@rohe this looks good to me, what do you think?


React template

Hi all,
Is there a simple template for React? Trying to integrate it using useRefs and orb.view.render doesn't seem to be called from useLayoutEffect. Thank you

Hello @alexanderbsd, thank you for your interest! You can find a working example here. It uses useLayoutEffect along with useRef in the CustomOrbGraphImpl component to get it to work. If it does not seem useful, feel free to share your code so we can figure out how to make it work. I hope it helps!

Works well. Thanks a lot!

Glad I could help, feel free to ask if you have any further questions :)


Expandables have lost their internal bottom spacing

Our :last-child { margin-bottom: 0; } rule has eliminated the bottom spacing in all expandables, which were counting on their last child's bottom margin to be there.
Current behavior
Expandables have no spacing at the bottom of their content area.
Expected behavior
Expandables have 15px spacing at the bottom of their content area.
Steps to replicate behavior

Pull the cf-v4 branch of design-manual and run it locally
View the Expandables page

Screenshots
Current

Expected

@Scotchester this is a duplicate of #234

No, it's not. It's affecting all expandables, and the cause is different than the one identified in that issue, and its preceding issue, cfpb/cfgov-refresh#487.


JBS-87: CreateVolume from snapshot is not concurrent

Limitation:
This will not work if we have multiple cinder-volume processes on same / different nodes

+1. Looks good to me


update editor for comments

Currently this pr is blocked. As EmojiPlugin only show suggestion list below the text and it get partially hidden as comment form always sticks bottom of page. Should we disable suggestion for comments?

Sorry, I've been reconsidering this mission and actually we dont need editor on comment.


werf  helm upgrade fails with error processing rollout phase stage: error tracking resources: but normal helm doesn't

Before proceeding

[X] I didn't find a similar issue

Version
1.2.305
How to reproduce
This is a bit difficult to reproduce because it does not happen every time. However I get this error with werf helm upgrade where helm upgrade --wait never fails for this.
Result
Upgrading release=jenkins, chart=/home/felipecrs/.cache/helmfile/https_github_com/jenkinsci/helm-charts/releases/download/jenkins-5.1.5/jenkins-5.1.5.tgz

┌ Waiting for resources to become ready
│ sts/jenkins ERROR: RecreatingFailedPod: StatefulSet default/jenkins is recreating failed Pod jenkins-0
│ 1/1 allowed errors occurred for sts/jenkins: continue tracking
│ sts/jenkins ERROR: FailedDelete: delete Pod jenkins-0 in StatefulSet jenkins failed error: pods "jenkins-0" not found
│ Allowed failures count for sts/jenkins exceeded 1 errors: stop tracking immediately!
│
│ ┌ Failed resource sts/jenkins service messages
│ │ added
│ │ po/jenkins-0 added
│ │ po/jenkins-0 added
│ │ event: RecreatingFailedPod: StatefulSet default/jenkins is recreating failed Pod jenkins-0
│ │ event: FailedDelete: delete Pod jenkins-0 in StatefulSet jenkins failed error: pods "jenkins-0" not found
│ └ Failed resource sts/jenkins service messages
└ Waiting for resources to become ready (2.01 seconds) FAILED

Error: UPGRADE FAILED: error processing rollout phase stage: error tracking resources: sts/jenkins failed: FailedDelete: delete Pod jenkins-0 in StatefulSet jenkins failed error: pods "jenkins-0" not found

FAILED RELEASES:
NAME      CHART                                                                                VERSION   DURATION
jenkins   https://github.com/jenkinsci/helm-charts/releases/download/jenkins-5.1.5/jenkins-5.1.5.tgz                   4s

Expected result
It was supposed to retry until timeout, I believe. I believe that's what helm upgrade --wait does.
Additional information
If you need me to build a reproducible environment, please let me know.

When this happening can you check if the pod jenkins-0 has any issues? Like container failing to start or getting killed by a probe? We have our own engine to track resource statuses and errors, so we can react to resource failures faster and stop release/run auto-rollback without waiting for the timeout like Helm does.
By default we ignore the first error per pod, but fail the release if it happens again. This behavior can be configured with annotations:
https://werf.io/documentation/v1.2/reference/deploy_annotations.html#failures-allowed-per-replica
https://werf.io/documentation/v1.2/reference/deploy_annotations.html#fail-mode

I just caught the issue again. Then:
❯ k get pods
NAME        READY   STATUS    RESTARTS   AGE
jenkins-0   1/1     Running   0          39s
❯ k describe pod jenkins-0
Name:             jenkins-0
Namespace:        default
Priority:         0
Service Account:  jenkins
Node:             k3d-jenkins-agent-dind-test-server-0/<IP_ADDRESS>
Start Time:       Fri, 05 Apr 2024 17:42:25 -0300
Labels:           app.kubernetes.io/component=jenkins-controller
                  app.kubernetes.io/instance=jenkins
                  app.kubernetes.io/managed-by=Helm
                  app.kubernetes.io/name=jenkins
                  controller-revision-hash=jenkins-7dd6558c9
                  statefulset.kubernetes.io/pod-name=jenkins-0
Annotations:      checksum/config: 9cda187dbf3e8e406c63fb2fa0f8b9be1282a524afe08f969aaa13634095fca5
Status:           Running
IP:               <IP_ADDRESS>
IPs:
  IP:           <IP_ADDRESS>
Controlled By:  StatefulSet/jenkins
Init Containers:
  init:
    Container ID:  containerd://2b16a7b993d311fa92aea7c6bfbee146191391e21a52caf53c10b53453f46d73
    Image:         jenkins-agent-dind-test-registry:5000/jenkins:latest
    Image ID:      jenkins-agent-dind-test-registry:5000/jenkins@sha256:87909327ff3bea4bcf6067f5be6fa7cfa0f714ef5d4fd75b68987dc17e284396
    Port:          <none>
    Host Port:     <none>
    Command:
      sh
      /var/jenkins_config/apply_config.sh
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Fri, 05 Apr 2024 17:42:26 -0300
      Finished:     Fri, 05 Apr 2024 17:42:26 -0300
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     2
      memory:  4Gi
    Requests:
      cpu:        50m
      memory:     256Mi
    Environment:  <none>
    Mounts:
      /var/jenkins_config from jenkins-config (rw)
      /var/jenkins_home from jenkins-home (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-m2nfn (ro)
Containers:
  jenkins:
    Container ID:  containerd://8cd20a8929bf0b96285dd9db763367a677d8f14817271408881b3874db382108
    Image:         jenkins-agent-dind-test-registry:5000/jenkins:latest
    Image ID:      jenkins-agent-dind-test-registry:5000/jenkins@sha256:87909327ff3bea4bcf6067f5be6fa7cfa0f714ef5d4fd75b68987dc17e284396
    Ports:         8080/TCP, 50000/TCP
    Host Ports:    0/TCP, 0/TCP
    Args:
      --httpPort=8080
    State:          Running
      Started:      Fri, 05 Apr 2024 17:42:27 -0300
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     2
      memory:  4Gi
    Requests:
      cpu:      50m
      memory:   256Mi
    Liveness:   http-get http://:http/login delay=0s timeout=5s period=10s #success=1 #failure=5
    Readiness:  http-get http://:http/login delay=0s timeout=5s period=10s #success=1 #failure=3
    Startup:    http-get http://:http/login delay=0s timeout=5s period=10s #success=1 #failure=12
    Environment:
      SECRETS:                   /run/secrets/additional
      POD_NAME:                  jenkins-0 (v1:metadata.name)
      JAVA_OPTS:
      JENKINS_OPTS:              --webroot=/var/jenkins_cache/war
      JENKINS_SLAVE_AGENT_PORT:  50000
      CASC_JENKINS_CONFIG:       /var/jenkins_home/casc_configs
    Mounts:
      /run/secrets/additional from jenkins-secrets (ro)
      /tmp from tmp-volume (rw)
      /var/jenkins_cache from jenkins-cache (rw)
      /var/jenkins_config from jenkins-config (ro)
      /var/jenkins_home from jenkins-home (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-m2nfn (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  jenkins-config:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      jenkins
    Optional:  false
  jenkins-secrets:
    Type:                Projected (a volume that contains injected data from multiple sources)
    SecretName:          jenkins
    SecretOptionalName:  <nil>
  jenkins-cache:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
  jenkins-home:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  jenkins
    ReadOnly:   false
  sc-config-volume:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
  tmp-volume:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:
    SizeLimit:  <unset>
  kube-api-access-m2nfn:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason     Age   From               Message
  ----     ------     ----  ----               -------
  Normal   Scheduled  54s   default-scheduler  Successfully assigned default/jenkins-0 to k3d-jenkins-agent-dind-test-server-0
  Normal   Pulling    53s   kubelet            Pulling image "jenkins-agent-dind-test-registry:5000/jenkins:latest"
  Normal   Pulled     53s   kubelet            Successfully pulled image "jenkins-agent-dind-test-registry:5000/jenkins:latest" in 73.017453ms (73.025639ms including waiting)
  Normal   Created    53s   kubelet            Created container init
  Normal   Started    53s   kubelet            Started container init
  Normal   Pulling    52s   kubelet            Pulling image "jenkins-agent-dind-test-registry:5000/jenkins:latest"
  Normal   Pulled     52s   kubelet            Successfully pulled image "jenkins-agent-dind-test-registry:5000/jenkins:latest" in 57.934933ms (57.959749ms including waiting)
  Normal   Created    52s   kubelet            Created container jenkins
  Normal   Started    52s   kubelet            Started container jenkins
  Warning  Unhealthy  44s   kubelet            Startup probe failed: HTTP probe failed with statuscode: 503

Looks like the pod was indeed unhealthy for some time, probably during the time is was being terminated (as it takes some seconds to terminate, and healtcheck would fail during it).

I just wonder if werf should indeed be this agressive by default. My intention is just to replace helm calls in my pipelines to use werf helm instead, so if I could avoid making changes to chart templates, it'd be nicer.

Must be fixed in 2.6.3.


Question 5 : more than one answer are not valid

If answer A is not valid so the answer D that tell that all answer are valid is false, so not valid at all. ;)

Aha, I see your point. I hope that most people will understand the question either way, but it could be rephrased. It would be great if you could open a PR for this  :) Otherwise I'll change it in my next batch of questions.


Improve Pyarrow Scanner::TakeRows()

Problem Statement
When we try to do point query via scanner.take() in python, it calls ::arrow::dataset::AsyncScanner::TakeRows().
The upstream implement did not optimize for point query.
Desired behavior
We should override this TakeRows() behavior to take advantage of lance layout for fast point query.

This is done in Rust already


Use explicit name to build docker images with bazel

This is a test.

Jenkins job istio-testing/pr-master passed

This is a test.

Jenkins job istio-testing/pr-master passed


Scaffolding Identity in .Net 6 Core

The 'Scaffold Identity into a Razor project without existing authorization' instructions for  don't work for .Net 6.0 Core.
The following code was automatically added to my Program.cs file as a result of doing scaffolding for a Razor Pages app, using a context that did not previously exist:
builder.Services.AddDefaultIdentity(options => options.SignIn.RequireConfirmedAccount = true).
AddEntityFrameworkStores<ApplicationDbContext)<>();builder.Services.AddDbContext(options =>
options.UseSqlServier(connectionString));
I got two error messages:

IServiceCollection does not contain a definition for 'AddDefaultIdentity'.
The name 'connectionString' does not exist in the current context.

What do you suggest, please?
Jeff

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 804e24dd-7e30-8cf5-2c99-1ffacb4582f6
Version Independent ID: 3e25547b-e2c9-6634-2c38-2a77207f17d4
Content: Scaffold Identity in ASP.NET Core projects
Content Source: aspnetcore/security/authentication/scaffold-identity.md
Product: aspnet-core
Technology: aspnetcore-security
GitHub Login: @Rick-Anderson
Microsoft Alias: riande

Do you have      <ImplicitUsings>enable</ImplicitUsings> in you project file?
Can you post to a public GitHub repo the simplest possible sample that reproduces this problem?
File a bug with the repo. If you used the CLI, file in https://github.com/dotnet/Scaffolding
For Visual Studio, use the Report a Problem gesture or Suggest a Feature gesture within VS, which open internal issues for the VS team. For Visual Studio Code, you can open an issue at microsoft/vscode issues.


Silent tensorflow models output to stdout

thank you for your contribution
newer tf dumps progress bar and it is really annoying, i could not find a solution and recommended downgrade tf. it will help developers a lot.
i will validate the PR and merge soon. thank you.