Create initial version of 22.04 scripts.

Add this too: https://github.com/cli/cli/blob/trunk/docs/install_linux.md

Please check if you think that the configuration done to git as well as the added aliases are useful

Not found during installation:

ros-rolling-gz-ros2-control
kdesdk-kio-plugins


Wildcard gives key error

@seanpearsonuk @mkundu1
Following code gives KeyError -
session.setup.boundary_conditions.wall['wall-geom*'] = {
    "thermal" : {
        "free_stream_temp" : 300,
        "heat_transfer_coeff" : 100,
        "thermal_condition" : "Convection"}
    }

Traceback (most recent call last):
  File "D:\Repos\pyfluent-examples\examples\00-released_examples\01-brake.py", line 136, in <module>
    session.setup.boundary_conditions.wall['wall-geom*'] = {
  File "D:\Repos\pyfluent-examples\venv\lib\site-packages\ansys\fluent\core\solver\flobject.py", line 1784, in __setitem__
    raise KeyError(name)
KeyError: 'wall-geom*'

Equivalent tui code is as follows -
session.tui.define.boundary_conditions.set.wall(
    "wall-disc*",
    "wall-geom*",
    "()",
    "thermal-bc",
    "yes",
    "convection",
    "convective-heat-transfer-coefficient",
    "no",
    100,
    "q",
)

https://examples.fluent.docs.pyansys.com/version/dev/examples/00-released_examples/01-brake.html

There is a limitation in Fluent side to implement this, logged Fluent defect 1099706

There is a limitation in Fluent side to implement this, logged Fluent defect 1099706

Thank you.


[Extensibility Request] Page 397 "Sales Invoice Statistics" - Protected variable

Reference to the old issue marked as complete:
https://github.com/microsoft/ALAppExtensions/issues/16612
Not included in the Build ID 48357 (Update 1 for BC 2022 Wave 2
Could you please set the variable SalesInvLine in Page 397 to protected?
protected var
    SalesInvLine: Record "Sales Invoice Line";

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 49742.


Could you please add a new UI Framework for Angular4

a new UI Framework(FreeNG: https://ironpans.github.io/freeng) for Angular4

Closing as you should use devarchy. Any reason why you didn't use it?


ImportError: Error relocating .../_perlin.cpython-37m-x86_64-linux-gnu.so: grad2: symbol not found

Hello,
I'm trying to use the noise library from inside docker alpine.
Installing noise with pip is okay, but when I try to import it I get this error message :

Traceback (most recent call last):
  File "map_generator.py", line 1, in <module>
    import noise
  File "/usr/lib/python3.7/site-packages/noise/__init__.py", line 12, in <module>
    from . import _perlin, _simplex
ImportError: Error relocating /usr/lib/python3.7/site-packages/noise/_perlin.cpython-37m-x86_64-linux-gnu.so: grad2: symbol not found

Any idea on how to fix this ?

It got fixed when I used distutils.core instead of setuptools in the setup.py. Not sure if this is the solution.
before
try:     from setuptools import setup, Extension except ImportError:     from distutils.core import setup, Extension
after
from distutils.core import setup, Extension


fix(autoware_obstacle_stop_planner): fix funcArgNamesDifferent

Description
This is a fix based on cppcheck funcArgNamesDifferent warnings
planning/autoware_obstacle_stop_planner/src/adaptive_cruise_control.cpp:203:49: style: inconclusive: Function 'insertAdaptiveCruiseVelocity' argument 7 names different: declaration 'current_velocity_ptr' definition 'current_odometry_ptr'. [funcArgNamesDifferent]
  const nav_msgs::msg::Odometry::ConstSharedPtr current_odometry_ptr, bool * need_to_stop,
                                                ^

Related links
Parent Issue:

Link

How was this PR tested?
Notes for reviewers
None.
Interface changes
None.

Effects on system behavior
None.

@brkay54 @beyzanurkaya @satoshi-ota @shmpwk @taikitanaka3 @tkimura4
Please confirm the PR. Thank you very much.

Thanks @kobayu858 -san, do you think we should change the variable name in the same named other function here?

https://github.com/autowarefoundation/autoware.universe/pull/8018/files#diff-dcf194d2671df058128fd28886cb9184728818de13c9b2e8e043a553a591dda9L52-L58

@brkay54
Thanks for the confirmation.
As you said, I thought it would be better to change the names of the arguments in the above section, so I changed them.
The reason is that the argument name in the .cpp file is current_odometry_ptr.
Strangely enough, cppchecker did not warn me about this, so I overlooked it. Thank you very much.


Where is data base

Thanks for proving example Rest API. But where are the values or student data saving ? there is no db code for this project.
Also, when i tried with get method getting below response (I used post method before using this)
{
"timestamp": "2019-02-01T18:46:56.355+0000",
"status": 404,
"error": "Not Found",
"message": "No message available",
"path": "/student/allstudent"
}

No need of any perticular database. maybe u r not connected to internet in SoapUI/Postman testing tool. just make sure ur firewall is not blocking internet access for that testing tool. And update the maven project and tick on force update of snapshots/Releases and update the project. Then run As Maven Install. after build success run the server and then u will not get the error.


Where Is Linq?

The source code does not contain the LINQ samples, nor does it contain the LINQ assembly.
Has this been pulled from the open source version?

Hi @DAConsulting ,
Thanks for your comment. Yes, NCache Open Source edition has been updated recently and the LINQ features are now only supported with NCache Enterprise edition. This is based on a business decision that our management board has taken. Moreover, we plan to release a new open source version which will include some platform support (.NET Core Server etc) but may have some feature limitations.  You can view the Edition comparison document on the Alachisoft website at the following link.
http://www.alachisoft.com/ncache/edition-comparison.html
Hope this helps!


Fix spring-boot#481: add JRE_HOME

JRE_HOME is often used as a convention to convey the path of a JRE; in this case it's convenient to distinguish the JRE from a JDK (when both are required), since the PATH can insert the JRE_HOME/bin or JDK_HOME/bin in a non predictable order.
Thanks to JRE_HOME, subsequent build steps will be able to use JRE_HOME/bin/java to make sure they pick a JRE and not a JDK
in case there's just a JDK desired by the user, JRE_HOME will also be set, as the JDK_HOME

Checklist

[X] I have viewed, signed, and submitted the Contributor License Agreement.
[X] I have linked issue(s) that this PR should close using keywords or the Github UI (See docs)
[ ] I have added an integration test, if necessary.
[X] I have reviewed the styleguide for guidance on my code quality.
[X] I'm happy with the commit history on this PR (I have rebased/squashed as needed).

So if I have this correct.
At build time:

JRE_HOME -> the JRE
JDK_HOME -> the JDK
JAVA_HOME -> the JDK (cause JRE is set to default, and JDK is set to override)

I'm curious what the behavior of PATH is here, given that both the JRE & JDK will be on PATH. I guess it depends on how the lifecycle processes the layers and puts them on the PATH? What have you observed @anthonydahanne ?

Should we also update the JLink installer?
That presently sets JAVA_HOME at build time but not JRE_HOME or JDK_HOME. I'm not 100% sure what it should do though. Probably JRE_HOME cause we strip out the JDK part, at least by default, and you're guaranteed to have at least a JRE.

At build time:

JRE_HOME -> the JRE
JDK_HOME -> the JDK
JAVA_HOME -> the JDK (cause JRE is set to default, and JDK is set to override)

Almost: if the buildplan requires both JRE and JDK: JRE_HOME -> the JDK

I'm curious what the behavior of PATH is here, given that both the JRE & JDK will be on PATH. I guess it depends on how the lifecycle processes the layers and puts them on the PATH? What have you observed @anthonydahanne ?

Well, PATH is kind of... not predictable. What I have observed is JDK path sometimes before, sometimes after JRE path in the PATH env.
So, that was not deterministic to call java from a contribute method, so that's why it's fixed using JRE_HOME/bin/java

Should we also update the JLink installer?
That, I don't know. But... I supposed it's not required if nobody had any issues with it so far.

We had the issue with CDS training run: CDS MUST use the same JVM during training run and runtime; and because of PATH, it was not reliable

Well, PATH is kind of... not predictable. What I have observed is JDK path sometimes before, sometimes after JRE path in the PATH env.

Interesting 🤔 I'm going to mention this upstream and see what they say.

Almost: if the buildplan requires both JRE and JDK: JRE_HOME -> the JDK

Oh, that's not what I would have expected. Why would JRE_HOME point to the JDK? Isn't that what you're trying to avoid? Cause the JDK would run the CDS training, but then the JRE would run the app at runtime and they'd be different?

oh I'm sorry, I pressed enter too fast
I meant to say:

Almost: if the buildplan requires ONLY a JDK (BP_JVM_TYPE=jdk): JRE_HOME -> the JDK


Update raygui Rectangle to match raylib Rectangle

Rectangle in raygui should use a float.

@RobLoach thanks for the review!


Fix lock-ordering during init

Reorders how locking is handled in objc_send_initialize() to prevent a deadlock. Previously, contention on the low level spinlocks could cause a very intermittent deadlock:

Thread A : objc_send_initialize() holds the runtime lock, then tries to acquire the object lock on the metaclass, which needs to initialize the mutex for the new metaclass inside referenceListForObject(), so it tries to lock the lock_for_pointer() / lock_spinlock()
Thread B : referenceListForObject() holds a spinlock for an unrelated object while running initHiddenClassForObject() -> allocateHiddenClass(), which tries to acquire the runtime lock

If the metaclass object pointer in Thread A hashes to the same spinlock as the object in thread B, the runtime lock ends up deadlocked forever.
Potential fix for this issue

Thanks, I think we need to keep the current lock acquisition to avoid UB but then drop it before acquiring the second one and reacquire it lower down.

Thanks for the speedy response!   Im not 100% sure what you mean by "to avoid UB"?  So let me know if this update was the correct change you're looking for.

Looks great, thanks!


Don't dispose controllers passed from outside

This solves an issue where the CropController is disposed even if it is owned by another object.
Fixes #15

Thanks.

No problem. Glad I could help


Wrong this in the callback

TypeError: Cannot read property 'emit' of undefined
at lib/mailin.js:163:37

Thanks @placeba!


[bug] fix higher grad log

There is a bug in the implementation for higher order gradient of log.
https://github.com/apache/incubator-mxnet/blob/7b343d1fcde73b61322985580080333d9eee9e82/src/operator/tensor/elemwise_unary_op_basic.cc#L1077-L1079
We multiply  gx * gx where gx = ograd * f'(x), getting ograd^2 * f'(x)^2, however we want only ograd * f'(x)^2 which can be achieved in a similar fashion to the implementation of _backward_log10/2.
I have validated the expected results for grad on x.
from mxnet import nd, autograd
import numpy
import math

op = lambda x: nd.log(x)
grad_op = lambda x: 1/x
grad_grad_op = lambda x: (-1/x**2)

x = nd.random.normal(0,1,(3,3))
head_grads = nd.random.normal(0,1,(3,3))
head_grad_grads = nd.random.normal(0,1,(3,3))

x.attach_grad()
head_grads.attach_grad()

with autograd.record():
  y = nd.cos(x)
  y_grad = autograd.grad(y, x, head_grads=head_grads, create_graph=True, retain_graph=True)[0]
y_grad.backward(head_grad_grads)

expected_grad_grad_x = (grad_grad_op(x) * head_grad_grads * head_grads).asnumpy()
expected_head_grad = (grad_op(x) * head_grad_grads).asnumpy()

numpy.testing.assert_allclose(x.grad.asnumpy(), expected_grad_grad_x, rtol=1e-7, atol=1e-7)
numpy.testing.assert_allclose(head_grads.grad.asnumpy(), expected_head_grad, rtol=1e-7, atol=1e-7)

Which fails with current code.
Have confirmed the behaviour with Pytorch as well.
import torch
import numpy
import math

op = lambda x: torch.log(x)
grad_op = lambda x: 1/x
grad_grad_op = lambda x: (-1/x**2)

x = torch.randn(2,3)
head_grads = torch.randn(2,3)
head_grad_grads = torch.randn(2,3)
x.requires_grad = True
head_grads.requires_grad = True

y = op(x)
y_grad = torch.autograd.grad(y, x, grad_outputs= head_grads, create_graph=True, retain_graph=True)[0]
y_grad.backward(head_grad_grads)

expected_grad_grad_x = (grad_grad_op(x) * head_grad_grads * head_grads).detach().numpy()
expected_head_grad = (grad_op(x) * head_grad_grads).detach().numpy()

numpy.testing.assert_allclose(x.grad.detach().numpy(), expected_grad_grad_x, rtol=1e-7, atol=1e-7)
numpy.testing.assert_allclose(head_grads.grad.detach().numpy(), expected_head_grad, rtol=1e-7, atol=1e-7)```

@apeforest @larroy please review

Thanks for your contributions @kshitij12345
@mxnet-label-bot Add [pr-awaiting-review, Operator]

Hi @kshitij12345 thanks for looking into this.
I think we need to clarify what exactly we have in the first parameter of FGradient "node". We were a bit puzzled with @apeforest looking at your PR. I validated the results with the tests but I think I tried only one log, don't remember which base. But the result seemed correct to me, I guess I missed this problem.
Why do you say that node is ograd*f'(x)?     the node argument I understand is the node to calculate the gradient for, in this case we are calculating the gradient of the backward of the log. So are you saying that by chain rule, the node is ograd(of log) * d (log(x)) / dx  = ograd * reciprocal?
Would be great if we could add this to the documentation, either to the FGradient typedef or to new_op. Otherwise I always have to dig through the code to refresh this. I think is poorly documented and tricky.

So are you saying that by chain rule, the node is ograd(of log) * d (log(x)) / dx = ograd * reciprocal?

Following the unary_bwd
https://github.com/apache/incubator-mxnet/blob/b0be6c50ae686a59c340c56b9c11493a5c84ff72/src/operator/tensor/elemwise_unary_op.h#L400-L402
which leads here.
https://github.com/apache/incubator-mxnet/blob/2fd4720342f87be80f280176ffbecb8f37e49b73/src/operator/mxnet_op.h#L520-L538
which fuses the output_grad * func_grad where func_grad our current case is d (log(x))/ dx operation.
Thus the node is multiplication of ograd * d(log(x)) / dx, which also implies that for every second order gradient function we have to do some operation to get only func_grad ( used for the gradient of ograd).
Also I had observed some weird behaviour for functions wrapped with ElemGradUseOut. Will post about it later in the day.

Would be great if we could add this to the documentation, either to the FGradient typedef or to new_op. Otherwise I always have to dig through the code to refresh this. I think is poorly documented and tricky.

Yes it is. True there should be some documentation regarding the same.

@kshitij12345 I have some question about the equation expected_head_grad = (grad_op(x) * head_grad_grads).asnumpy() in your test.
My understanding from the chain rule:
Given y =f(x)
dL/dx = dL/dy * dy/dx -->  this is the first forward pass. Note dL/dy as y_grad, we get dL/dx (note as x_grad)

Now we rewrite the above the equation:

input0: y_grad
input1: x
output: x_grad = y_grad * f'(x)

Another backward pass for this would be:
dL/d y_grad = dL/d x_grad * f'(x)
dL/dx = dL/d x_grad * y_grad * f''(x)

What is the meaning of dL/d y_grad? Are we treating y_grad as another input variable here?

Thanks for your clarification.

I did some more probing. I think the reason that head_grads.grad being all zeros is the variable head_grads was not specified during the second backward pass. I updated the test as follows, but got an assertion error:
grad_grad_op = lambda x: -nd.sin(x)
grad_op = lambda x: nd.cos(x)

x = nd.array([1, 2, 3])
head_grads = nd.ones_like(x) * 0.5
head_grads_grads = nd.ones_like(x) * 0.6

x.attach_grad()
head_grads.attach_grad()

with autograd.record():
    y = nd.sin(x)
    xgrad = autograd.grad(heads=y, variables=x, head_grads=head_grads, create_graph=True, retain_graph=True)[0]
print('xgrad = {} exp_xgrad = {}'.format(xgrad.asnumpy(), grad_op(x) * head_grads))
grad_grad_out = autograd.grad(heads=xgrad, variables=[x, head_grads], head_grads=head_grads_grads, create_graph=False, retain_graph=True)
print(len(grad_grad_out))

It complains that the variable head_grads is unreachable from the output which I think is reasonable. This again comes back to my question above, what is the mathematical meaning of head_grads.grad and why do we need this value?

@apeforest see my reply above, head_grads is not in the graph so it's not updated during backward

@larroy Yes, I agree with your reply. Also, I don't understand the meaning (or need) to return dL/dy_grad. @kshitij12345 Please comment. Thanks

@kshitij12345 I have some question about the equation expected_head_grad = (grad_op(x) * head_grad_grads).asnumpy() in your test.
My understanding from the chain rule:
Given y =f(x)
dL/dx = dL/dy * dy/dx -->  this is the first forward pass. Let dL/dy be y_grad, we get dL/dx (noted as x_grad)

Now we rewrite the above the equation:

input0: y_grad
input1: x
output: x_grad = y_grad * f'(x)

Another backward pass for this would be:
dL/d y_grad = dL/d x_grad * f'(x)
dL/dx = dL/d x_grad * y_grad * f''(x)

What is the meaning of dL/d y_grad? Are we treating y_grad as another input variable here?
Many thanks for your clarification.

@kshitij12345 I have some question about the equation expected_head_grad = (grad_op(x) * head_grad_grads).asnumpy() in your test.
My understanding from the chain rule:
Given y =f(x)
dL/dx = dL/dy * dy/dx -->  this is the first forward pass. Let dL/dy be y_grad, we get dL/dx (noted as x_grad)

Now we rewrite the above the equation:

input0: y_grad
input1: x
output: x_grad = y_grad * f'(x)

Another backward pass for this would be:
dL/d y_grad = dL/d x_grad * f'(x)
dL/dx = dL/d x_grad * y_grad * f''(x)

What is the meaning of dL/d y_grad? Are we treating y_grad as another input variable here?
Many thanks for your clarification.

I think the introduction of L (loss) is confusing here. As per back accumulation of gradients and chain rule we always have the incoming gradient (also called head gradient or output gradient). So the second backward pass should calculate:

I'm thinking that maybe the problem is that we should not reuse the head gradient from the first gradient in the second gradient. Shouldn't the two head gradients be independent variables?

G

cluster_1

Fwd

cluster_2

bwd

cluster_3

bwd

x

x

log

log

x->log

log_bwd

log_bwd

x->log_bwd

log_bwd_bwd

log_bwd_bwd

x->log_bwd_bwd

y

y

log->y

ograd

ograd

ograd->log_bwd

x_grad

x_grad

log_bwd->x_grad

ograd2

ograd2

ograd2->log_bwd_bwd

x_grad_grad

x_grad_grad

log_bwd_bwd->x_grad_grad

@kshitij12345 I have some question about the equation expected_head_grad = (grad_op(x) * head_grad_grads).asnumpy() in your test.
My understanding from the chain rule:
Given y =f(x)
dL/dx = dL/dy * dy/dx -->  this is the first backward pass. Let dL/dy be y_grad, we get dL/dx (noted as x_grad)

Now we rewrite the above the equation:

input0: y_grad
input1: x
output: x_grad = y_grad * f'(x)

Another backward pass for this would be:
dL/d y_grad = dL/d x_grad * f'(x)
dL/dx = dL/d x_grad * y_grad * f''(x)

What is the meaning of dL/d y_grad? Are we treating y_grad as another input variable here?
Many thanks for your clarification.

I was following on the basis of this graph that I had in my mind.

Even I am not sure about the mathematical meaning of it. Also in usual scenario it would essentially be an intermediate node. Like in the picture.
However since we are returning it might as well test for it. And in case of the current test, head_grads as depicted by dL/dy is an input leaf node in the second graph. So I expect that we will be able to accumulate gradients in it (similar to Pytorch).
I hope it makes sense. Thank You.

@kshitij12345 The computation graph for the second backward pass makes sense to me. As you can see there is only one output from the graph, that is x_grad_grad. It is not clear to me where the output dL/dy_grad is

@kshitij12345 The computation graph for the second backward pass makes sense to me. As you can see there is only one output from the graph, that is x_grad_grad. It is not clear to me where the output dL/dy_grad is

Actually I haven't shown the other graph for ograd_grad. If you take a look at the x_grad, it is computed using ograd and x. So x_grad will change not only with x but also with ograd.
If you focus only on * (multiplication node) while computing x_grad.

x_grad = x_grad_mid * y_grad # Note this part.

I don't think this part is correct.
Note: your x_grad_mid is dL/dx.
So x_grad_mid = y_grad * grad_op(x)
Do you meant to write x_grad = grad_op(x) * y_grad?

@kshitij12345 I think it's because of the design of backward computation graph in MXNet. In C++ implementation, when you specify variables=x, it will compute gradients for the input variables.
As in your case 2:
x_grad = autograd.grad(heads=y, variables=x, head_grads=y_grad, create_graph=True, retain_graph=True)[0]

If you perform another backward on x_grad as x_grad.backward(out_grad=head_grads_grads), y_grad is not listed as input variable and therefore it's gradient is zero
As in your case 1:
x_grad = x_grad_mid * y_grad # Note
x_grad.backward(out_grad=head_grad_grads)

You implicitly made y_grad an input variable when calling backward on x_grad. And that is why you will get values in y_grad.grad.
I replaced the backward() method with an explicit autograd.grad() call, which should call the same C++ backend function and result is different.
case 1.1: if I do the following, I again don't get any values for y_grad because the output only contains one gradient variable
out_grad = autograd.grad(heads=x_grad, variables=x, head_grads=head_grad_grads, create_graph=False, retain_graph=False)
print(out_grad[0])   # values equals to expected_grad_grad

case 1.2: I explicitly set y_grad as input variable, I then get the expected result as in your case 1
out_grad = autograd.grad(heads=x_grad, variables=[x, y_grad], head_grads=head_grad_grads, create_graph=False, retain_graph=False)
print(out_grad[0])   # value equals to expected_grad_grad
print(out_grad[1])   # value equals to expected_heads_grad

At this point, I am not sure if this is a bug because the backward API is designed differently from PyTorch. If y_grad is not specified as part of the input variables that need to perform gradient on, it will not get values assigned even if you write y_grad.attach_grad() to it. This seems to be consistent from API spec. Also, given that the value y_grad does not have real useful values, I also don't feel the necessity to store it. Please let me know if this makes sense. Thanks a lot for your careful drawing and insightful discussion.

As a follow up, I just dumped out the computation graph in case 2. Indeed, the node that used to calculate y_grad.grad is not even in the final symbolic graph because there is no input dependency.

Oh thank you very much for explaining what is happening. I get it now.
It makes sense, as passing argument as head_grads doesn't make any explicit computation on the graph.
y_grad may not have any useful meaning as of now, however I believe that we should test for the value using case 1.2, just to verify and catch any mistakes there. Rather than the same thing biting us in the future. Your thoughts? @larroy @apeforest
Also it would be really great if you can tell me how to get the dump of the graph that would be really helpful.
Thank you again for your time and efforts.

@apeforest can you paste your dump of the graph? and share where did you dump it? I was working on a utility to dump the graph.

Also it would be really great if you can tell me how to get the dump of the graph that would be really helpful.

@kshitij12345 There is a utility function inline void LogMemoryPlan(const nnvm::Graph& g) in exec_utils.h that can dump out the computation graph. However it only works in symbol mode currently and you need to manually copy paste it to imperative.cc to dump out graph in NDArray APIs. Besides, you also need to comment out the const auto& vstorage = g.GetAttr<nnvm::StorageVector>("storage_id") line otherwise you will get a core dump.
In one word, it's not an easy to use debugging utility and @larroy is working to improve it and create a PR soon. Stay tuned :)

y_grad may not have any useful meaning as of now, however I believe that we should test for the value using case 1.2, just to verify and catch any mistakes there. Rather than the same thing biting us in the future. Your thoughts? @larroy @apeforest

I don't have a clear answer to how to test this value now. Definitely not through a python unit test because the value is not getting output as NDArray. Besides, as I commented earlier, that computation node was even in nnvm graph maybe it was later deleted by graph optimization because it is a dangling node with no output.

@apeforest , Thank You for explaining how to get the dump of the graph. Waiting the PR which simplifies that.

@apeforest , Thank You for explaining how to get the dump of the graph. Waiting the PR which simplifies that.
Also could you tell me if it is possible to print the Node and it's value (for the array) via C++ and if yes then how .

We don't yet have a utility function to do that. What do you mean the value of the node? A node only represent a computation in the graph. The values are passed in/out when the graph is traversed and each node invokes the FCompute/FGradient functions registered. I think we can only track values in the corresponding NDArray outputs. Please let me know if I misunderstand your question.

Oops, sorry for confusion. I meant the value of a NDArray.
auto gx = nnvm::NodeEntry{n};, can I get/print the value of the NDArray corresponding to that node.
Thank You.

Right now is very difficult to print the value of the NDArray. I will work on a utility to dump the graph this week, first the graph itself, and if possible the NDArray values.

There is no available utility to print out value from NDArray. I used to write a for loop to iterate the dptr pointer and dump out values. Besides, we need extra synchronization handling if the NDArray is stored in GPU. I agree with @larroy that this is something very important for debugging MXNet backend. I have created an issue https://github.com/apache/incubator-mxnet/issues/15198 to ask for contributions.

@kshitij12345 Could you please rebase and retrigger CI again? Thanks!

@kshitij12345 still one GPU test failed. I looked at the log and don't find it related to your change. Could you please rebase and trigger CI one more time. The master branch was broken last weekend. Thanks again.

@apeforest Note that the pending job has succeeded but for some reason it isn't updated here.

@kshitij12345 There was some issue with CI recently. Could you please re-trigger it one more time? Sorry for the inconvenience.


test key

Description
Infrastructure as code config
Related Issue
#627

Fixes # (issue)
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist before merging

[ ] Tests have been added/updated as necessary (add the needs-tests label)
[ ] Documentation has been updated as necessary (add the needs-docs label)
[ ] An ADR has been written as necessary (add the needs-adr label) [ 1 2 3 ]
[ ] (Optional) Changes have been linted locally with golangci-lint. (NOTE: We haven't turned on lint checks in the pipeline yet so linting may be hard if it shows a lot of lint errors in places that weren't touched by changes. Thus, linting is optional right now.)

I needed to validate GPG signing -- holding this as a draft with config changes re: #627

Close for now, feel free to re-open once you have something for us to review.


Recommended way to add configuration blocks to cluster

I'm relatively new to Terraform, sorry in advance if the question is newbiish...
I would like to add an istio_config block to the cluster. It currently requires the google-beta provider, but if we leave that aside, is there a way to extend the cluster configuration from the module section in the parent project without modifying the generated/imported module?
Thanks

Hi! No worries 😄 I don't think there is a way to add blocks to modules, but I will look into it.
I would like to add an istio_config so am open to any PRs, though will also need to look into the google-beta provider and what effect switching to that will have. There are a few other features I'd like to add that also need it so would be good to have!


Allow mut args

(in the most annoying way)
This is useful for when you want to do more complicated manual defaults for arguments, such as trying to read from an environment variable if an argument is not provided.

Here's the current (working) main! I have in the crate that caused me to submit this:
main!(|args: Args, log_level: verbosity| {
    let args = Args {
        token: args.token.or(option_env!("GH_TOKEN").map(ToOwned::to_owned)),
        ..args
    };
    ghp_upload(args)?;
});

I don't see why you need the mutable bit – just use args.clone(). It's totally irrelevant for performance unless your CLI arguments are megabytes of data (and even then LLVM can probably optimize this).

Additionally, the proper solution for what your example shows is to have a better story around configuration management. See #21 for some ideas how to use config files, env vars, and cli args.

Makes sense.
The error message rustc spits out when putting a mut there is .... confusing though; I think it gets grabbed by the $expr case.

(Just realized that I meant to use env::var("GH_TOKEN") and not option_env!("GH_TOKEN") in the example, because I just fixed that in the example it came from)


Android cannot scroll when touch starts on element outside of specified width

https://snack.expo.dev/dXmUBT65P
On android, if you Carousel's width smaller than window's width, the PanGestureHandler will not work when you starts dragging from outside of the view
https://user-images.githubusercontent.com/27767989/139675846-6a469e7f-0479-4a8d-9014-f607c8921a3b.mp4
https://github.com/dohooo/react-native-reanimated-carousel/blob/03177f787a34b227908482a553e1844246b30051/src/Carousel.tsx#L368
Is it better to spread styles on the bottom of the object? By allowing user to set their container width, they can set the container's width to be 100% while the content still retain it's width

I have been busy recently, so I will come back to this issue in a few days.

I am sorry that I have been busy recently. I am still paying attention to the problem and I will fix it as soon as possible, maybe this week.

I think you should use style={{ overflow: 'hidden' }} prop to unify.It has nothing to do with the package itself.. :D
For the problem of disunity,I think it has something to do with PangestureHandler ...
If you have more specific problems and desired solutions, I will open this issues again,Thank you for the isseues

v1.0.5
THX! 🍺！！～


prepare_mkl.sh script issue report

Description
Build MXNet with Intel MKLML under Linux will link with incorrect MKL library instead of MKLML library due to a bug in prepare_mkl.sh.
The root cause is in prepare_mkl.sh USE_MKLML flag is wrongly set to 0 even when compiling with MKLML which make libmxnet.so linked with unexpected MKL library.
To be more accurate, there is issue in below find statement (in prepare_mkl.sh line 118) that would return empty even libmklml_gnu.so exists under MKLROOT directory.
if [ -z find $MKLROOT -name libmklml_gnu.so -o -name libmklml.dylib -print -quit ]
The fix should be something like below (like add bracket to embrace the "or" logic):
diff --git a/prepare_mkl.sh b/prepare_mkl.sh
index 97a1e49..e048f59 100755
--- a/prepare_mkl.sh
+++ b/prepare_mkl.sh
@@ -115,7 +115,7 @@ if [ -z $MKLROOT ]; then
fi
\# Check what MKL lib we have in MKLROOT
-if [ -z find $MKLROOT -name libmklml_gnu.so -o -name libmklml.dylib -print -quit ]; then
+if [ -z find $MKLROOT \( -name libmklml_gnu.so -o -name libmklml.dylib \) -print -quit ]; then
USE_MKLML=0
elif [ -z find $MKLROOT -name libmkl_core.so -print -quit ]; then
USE_MKLML=1
Build info (Required if built from source)
Compiler: GCC
HW:  Intel CPU
MXNet commit hash:
ae0fe77c25c51f2afecb178c610cfcd0b8a18aac
Build command:
make -j 12 USE_OPENCV=1 USE_MKL2017=1 USE_MKL2017_EXPERIMENTAL=1 USE_BLAS=mkl USE_PROFILER=1 MKLML_ROOT=/home/jinhuang/downloads/mklml_lnx_2<IP_ADDRESS>71007
What have you tried to solve it?

Applied a fix in prepare_mkl.sh to fix this issue.
Added a test script to check if MKLML is installed correctly (linked with expected MKLML library) after
build/install.
After the fix,  MXNet with MKLML could be compiled successfully, and the image training performance is about 42.66 images/sec on skylake 8180 using bs64 (use train_imagenet.py script)

Other info:  similar issue is also reported in another thread https://github.com/apache/incubator-mxnet/issues/8881

@jinhuang415 thanks for reporting. Would you propose a pull request for the fix?


mpfs_ethernet.c: increase GMAC_RX_UNITSIZE

Set RX buffer size to max frame length to be able to receive MC side MTU size frames

LGTM, although the suggestion about having a separate kconfig variable does sound tempting as well.

Now when you say it, I'll think I'll do that instead, so it would be consistent with the TX buffer.

I was thinking is there any reason to use different packet sizes for RX and TX, but just use the same size/config for both?

I was thinking is there any reason to use different packet sizes for RX and TX, but just use the same size/config for both?

To me it would make sense to just use the MTU for both, shouldn't that be enough ?


Add Ability to Render Multiple Queries to Multiple Graphs

On the graphing page we support multiple queries, but they all go to the same graph. This often doesn't work well because the y scale values are often vastly different, making one of the series illegible on the graph.
I think we should have an option to render each query to its own graph. For example, see the Pandas subplots option: http://pandas.pydata.org/pandas-docs/stable/visualization.html#subplots

3rd, and probably easier option, is just to support a conversion to log scale when graphing this stuff.

Can do repeating panels in Grafana, not going to try to tackle this in Bosun's UI.


Conversion issues in frequent tokens from Wikipedia

Inspired by @ssvb's suggestion, I downloaded recent dumps of both Belarusian Wikipedias, bewiki (modern orthography) and be_x_oldwiki (classical orthography), and processed them as follows:

Extract article texts from each dump using WikiExtractor, then tokenize the texts, lowercase each token and count token frequencies.
Process all ~940K distinct tokens in bewiki with a local instance of taraskevizatar, converting them into the classical orthography.
Select only those tokens which have frequency ≥10 in bewiki but their converted counterparts have zero frequency in be_x_oldwiki, meaning that the conversion is likely wrong.

Please find here the resulting file which has three columns: frequency in bewiki, token, and the output of its conversion with taraskevizatar. It may require manual review, as it uncovers quite a few issues in taraskevizatar. In particular:

абласцей should be вобласьцяў, not абласьцяў;
медыцынскі (in all genders, numbers and cases) should be мэдычны, not мэдычні;
вакзал, фінал should not be altered;
герцагства should not be altered;
the patronymic name Дзмітрыевіч should be Дзьмітравіч or Зьмітравіч but not Дзьмітаравіч;
the abbreviation ГДР should probably be НДР (from Нямеччына) but not ГДАР;
the toponym Каліфорнія should not be altered;
Еўрасаюз should be Эўразьвяз, not Эўрасаюз;
Венгрыя should be Вугоршчына, not Вэнгрыя;
some regex replacements are unexpected, such as найчасцей → найчасьцяў, прывілей → прывялей, выканала → выканалу, etc.

Thanks

Thank you, I'll review and fix this later
Country names like Вэнгрыя/Вугоршчына, Літва/Летува, Расія/Расея etc. shouldn't be replaced because it's not about orthography but lexis. On the tarask wiki that some country names are traditionally written a certain way, but both versions in both orthographies are correct


update module version

https://github.com/PowerShell/platyPS/blob/d9029dc74acedd40f1ce1092820eee1dcf300edb/build.ps1#L69
you can use Update-ModuleManifest here

@timoline care to send a PR for that?

The current build script updates the version in the output. This has been fixed.


Add dependency on React pod

With use_frameworks! enabled in my PodFile, I get a linker error:
Undefined symbols for architecture x86_64:
"_RCTRegisterModule", referenced from:
+[RNDeviceInfo load] in RNDeviceInfo.o
ld: symbol(s) not found for architecture x86_64
This is fixed by adding React as a dependency in the podspec

Thanks! I've pushed this in 0.5.1 to npm just now :)


Advanced hunting

[Enter feedback here]
Hi, I have problem with my advanced hunting quires.
For example quire:
FileCreationEvents
| where EventTime > ago(40d)
| where MD5 == "9655d95b611bff033d3360899bf7cdad77c3af43"
| project ComputerName, FileName, MachineId, ReportId
| top 100 by ComputerName
not giving any result and I am searching for hash from one of the alerts.
Can you help ?
Michal
Document details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 88b10778-0203-5e62-911a-62781a5147ab
Version Independent ID: 55213ba9-a050-5b98-4453-57258b1df352
Content: Overview of advanced hunting capabilities
Content Source: windows/security/threat-protection/microsoft-defender-atp/overview-hunting.md
Product: w10
Technology: windows
GitHub Login: @mjcaparas
Microsoft Alias: macapara

Sorry should be:
ProcessCreationEvents
| where EventTime > ago(40d)
| where MD5 == "9655d95b611bff033d3360899bf7cdad77c3af43"
| project ComputerName, FileName
| top 100 by ComputerName

Sorry, but this is not a doc issue. The query appears valid, so it's possible you have the wrong MD5 or the file really does not exist.
Please contact your support provider if you need further assistance. You can contact Microsoft Support from the Help (?) link in Microsoft Defender Security Center.


Feat: Add login control for inactive users - 191

This PR introduces a new feature that allows the application to control login access for users marked as inactive.
Closes #191

Code looks good but we need to present an error to the user who is trying to login. Currently the app will just return to the login screen but it would be nice to show an error alert on that page when login fails.

"Given I am an inactive BCeID user, When I am attempting to log in, then I am presented with an error message and will not be able to log in"

Thank you, Alex, for your feedback. I'll create a new PR for the requested changes.


Enhance HTTP/2 Support in Helidon

Current support is experimental and there is no automated testing for it. This issue will keep track of HTTP/2 related work going forward:

[ ] Support for TLS with ALPN extension for protocol negotiation. See #762.
[ ] Explore suitable HTTP/2 client library with content negotiation with and without TLS
[ ] Develop a test suite for HTTP/2 testing content negotiation and client connection with HTTP/2 prior knowledge
[ ] Consider making HTTP/2 support no longer experimental

JDK 8 does not support TLS with ALPN without an external library in bootclasspath (alpn-boot-8.1.11.v20170118.jar).
JDK 9 and later support TLS with ALPN
Only JDK 11 supports the latest TLS 1.3

HTTP/2 Clients:

okhttp: Supports 1.1 and 2.0 but content negotiation is only supported via TLS/ALPN. Tools like curl support negotiation with clear text as well.
JDK 11 client: TODO

The current HTTP/2 implementation based on Netty still uses an adapter to convert an HTTP/2 stream into an HTTP/1 message. As a result, there is buffering (limited by maxContentLength) and some streaming limitations.
Given these limitations and the fact that very few users inquires about HTTP/2 support, it seems reasonable to leave this feature as experimental for now. I'm closing this issue as complete.


Cut 2.49.0-rc.0

Some epic changes in this release. Amazing community work! 💪🏽

/prombench v2.48.0

/prombench v2.48.0

So prombench results are generally healthy. Over long periods I see:

~5% less (heap) memory use 👍🏽
~5% more CPU used 👎🏽
~5%  increase in query_range latency 👎🏽

Mem
go_memstats_alloc_bytes and RSS looks generally lower than v2.48.0, sometimes up to 30%:

Over days, it uses 5% less memory:

CPU
CPU use, I would say is slightly worse (up to 10%):

There were also periods when it was a bit smaller usage too (generally after Sunday midday, which is interesting):

Generally over longer period we use more CPU (5% in worst case):

Query latency
Does not look very good, but the difference is also small (e.g. 890ms vs 850ms)

LGTM

The CPU is higher on new test for 2.49, but load is lower on 2.48, unsure why:

Adding issue to prombench https://github.com/prometheus/test-infra/issues/629
There might be still some small overhead (however better memory use) but agree with @roidelapluie , let's take it to production and see if blocking

Copied profiles we found comparing 5% more on New vs Old for the reference.
/prombench cancel

/prombench cancel


Add PiiEntities to Azure.AI.TextAnalytics

Contributing to the Azure SDK
Please see our CONTRIBUTING.md if you are not familiar with contributing to this repository or have questions.
For specific information about pull request etiquette and best practices, see this section.

API change check for Azure.AI.TextAnalytics
API changes have been detected in Azure.AI.TextAnalytics. You can review API changes here
API changes
+             V2022_03_01_Preview = 4,
-         public TextAnalyticsClientOptions(ServiceVersion version = V3_2_Preview_2);
+         public TextAnalyticsClientOptions(ServiceVersion version = V2022_03_01_Preview);
+         public static TextAnalyticsOperationStatus PartiallySucceeded { get; }

closing due to too many file changes
re-opened here: https://github.com/Azure/azure-sdk-for-net/pull/28470


Support to saving cloned deployed contract

For abracadabra, we are deploying cauldron's from bentobox deploy function, which deploy a minimal proxy for a given mastercontract.
Ref: https://github.com/boringcrypto/BoringSolidity/blob/78f4817d9c0d95fe9c45cd42e307ccd22cf5f4fc/contracts/BoringFactory.sol#L32
But I'm getting this error after deploying and syncing:
➜  ./forge-deploy sync
could not find tx for in-memory deployed contract YearnPrivateMIM3PoolCauldron2 at 0xe61519F163cB2515199ed6de656Cc7E5A50EEF0D

I'm using the code below, so that we can deploy with deployer.deployCauldronV4(...) and avoid redeploying if ran again using has. Back when we were using hardhat-deploy we could save this deployment like this: https://github.com/Abracadabra-money/magic-internet-money/blob/23266d17969a95e69199670cba9d0060bff33340/archive/deploy/PopsicleUsdcWavax.ts#L51
This wasn't storing all the info but at least you get a deployment file and the address / abi for reference and loading it back.
I was trying to achieve the same with foundry forge-deploy but couldn't.
Here are the details and the run-latest.json file causing the issue. I had to create the deployment file myself with just the address and it works, but would be nice if that was automated just like the rest.
In case it's difficult to achieve, I think it would be nice to have a raw save function where forge-deploy would not try to resolve more out of it.
function deployCauldronV4(
        Deployer deployer,
        string memory deploymentName,
        IBentoBoxV1 degenBox,
        address masterContract,
        IERC20 collateral,
        IOracle oracle,
        bytes memory oracleData,
        uint256 ltvBips,
        uint256 interestBips,
        uint256 borrowFeeBips,
        uint256 liquidationFeeBips
    ) internal returns (ICauldronV4 cauldron) {
        if (deployer.has(deploymentName)) {
            return ICauldronV4(deployer.getAddress(deploymentName));
        }

        bytes memory data = getCauldronParameters(collateral, oracle, oracleData, ltvBips, interestBips, borrowFeeBips, liquidationFeeBips);
        cauldron = ICauldronV4(IBentoBoxV1(degenBox).deploy(masterContract, data, true));

        deployer.save(deploymentName, address(cauldron), "", "", "");
    }

usage example:
deployer.deployCauldronV4(
            "YearnPrivateMIM3PoolCauldron2",
            IBentoBoxV1(constants.getAddress("mainnet.degenBox")),
            constants.getAddress("mainnet.cauldronV4Whitelisted"),
            IERC20(constants.getAddress("mainnet.yearn.mim3crv")),
            oracle,
            "",
            9800, // 98% ltv
            100, // 1% interests
            0, // 0% opening
            50 // 0.5% liquidation
        );

run-latest.json
{
  "transactions": [
    {
      "hash": "0x85109a418c8090b9fa4a18edf6737f31c95b6bbcc461aa8e5cc71e14fc301fae",
      "transactionType": "CALL",
      "contractName": "DegenBox",
      "contractAddress": "0xd96f48665a1410C0cd669A88898ecA36B9Fc2cce",
      "function": "deploy(address,bytes,bool):(address)",
      "arguments": [
        "0x369d81cf263aBC7EE567d8836A39234141D4dA07",
        "0x000000000000000000000000a540744dedbda9ef64cf753f0e851efe4a419ea9000000000000000000000000be47fe5215945dfd4aa2c6b5e0e4074e7b96da1400000000000000000000000000000000000000000000000000000000000000e00000000000000000000000000000000000000000000000000000000012e337ee0000000000000000000000000000000000000000000000000000000000018894000000000000000000000000000000000000000000000000000000000001770000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
        "true"
      ],
      "transaction": {
        "type": "0x02",
        "from": "0xfb3485c2e209a5cfbdc1447674256578f1a80ee3",
        "to": "0xd96f48665a1410c0cd669a88898eca36b9fc2cce",
        "gas": "0xa7381",
        "value": "0x0",
        "data": "0x1f54245b000000000000000000000000369d81cf263abc7ee567d8836a39234141d4da07000000000000000000000000000000000000000000000000000000000000006000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000100000000000000000000000000a540744dedbda9ef64cf753f0e851efe4a419ea9000000000000000000000000be47fe5215945dfd4aa2c6b5e0e4074e7b96da1400000000000000000000000000000000000000000000000000000000000000e00000000000000000000000000000000000000000000000000000000012e337ee0000000000000000000000000000000000000000000000000000000000018894000000000000000000000000000000000000000000000000000000000001770000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
        "nonce": "0x4ed",
        "accessList": []
      },
      "additionalContracts": [
        {
          "transactionType": "CREATE2",
          "address": "0xe61519F163cB2515199ed6de656Cc7E5A50EEF0D",
          "initCode": "3d602d80600a3d3981f3363d3d373d3d3d363d73369d81cf263abc7ee567d8836a39234141d4da075af43d82803e903d91602b57fd5bf3"
        }
      ],
      "isFixedGasLimit": false
    }
  ],
  "receipts": [
    {
      "transactionHash": "0x85109a418c8090b9fa4a18edf6737f31c95b6bbcc461aa8e5cc71e14fc301fae",
      "transactionIndex": "0x10",
      "blockHash": "0x990f4c0f7b50df3d6daf3ddbe2bd182b684be78c758868b22ff34db7fe53c97c",
      "blockNumber": "0x10a11fa",
      "from": "0xfB3485c2e209A5cfBDC1447674256578f1A80eE3",
      "to": "0xd96f48665a1410C0cd669A88898ecA36B9Fc2cce",
      "cumulativeGasUsed": "0x2a474f",
      "gasUsed": "0x79109",
      "contractAddress": null,
      "logs": [
        {
          "address": "0x99D8a9C45b2ecA8864373A26D1459e3Dff1e17F3",
          "topics": [
            "0x8c5be1e5ebec7d5bd14f71427d1e84f3dd0314c0f7b2291e5b200ac8c7c3b925",
            "0x000000000000000000000000e61519f163cb2515199ed6de656cc7e5a50eef0d",
            "0x000000000000000000000000d96f48665a1410c0cd669a88898eca36b9fc2cce"
          ],
          "data": "0xffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
          "blockHash": "0x990f4c0f7b50df3d6daf3ddbe2bd182b684be78c758868b22ff34db7fe53c97c",
          "blockNumber": "0x10a11fa",
          "transactionHash": "0x85109a418c8090b9fa4a18edf6737f31c95b6bbcc461aa8e5cc71e14fc301fae",
          "transactionIndex": "0x10",
          "logIndex": "0x5a",
          "removed": false
        },
        {
          "address": "0xd96f48665a1410C0cd669A88898ecA36B9Fc2cce",
          "topics": [
            "0xd62166f3c2149208e51788b1401cc356bf5da1fc6c7886a32e18570f57d88b3b",
            "0x000000000000000000000000369d81cf263abc7ee567d8836a39234141d4da07",
            "0x000000000000000000000000e61519f163cb2515199ed6de656cc7e5a50eef0d"
          ],
          "data": "0x00000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000100000000000000000000000000a540744dedbda9ef64cf753f0e851efe4a419ea9000000000000000000000000be47fe5215945dfd4aa2c6b5e0e4074e7b96da1400000000000000000000000000000000000000000000000000000000000000e00000000000000000000000000000000000000000000000000000000012e337ee0000000000000000000000000000000000000000000000000000000000018894000000000000000000000000000000000000000000000000000000000001770000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
          "blockHash": "0x990f4c0f7b50df3d6daf3ddbe2bd182b684be78c758868b22ff34db7fe53c97c",
          "blockNumber": "0x10a11fa",
          "transactionHash": "0x85109a418c8090b9fa4a18edf6737f31c95b6bbcc461aa8e5cc71e14fc301fae",
          "transactionIndex": "0x10",
          "logIndex": "0x5b",
          "removed": false
        }
      ],
      "status": "0x1",
      "logsBloom": "0x00000000000000400000000000000020000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000200000000000000400000000000000000000000000000000000000000000000000000000000080000000000000000001000000000000000000001000000000800000000000000000000000000000400000000000000000000004000000000000000000020000000000000000000000000800000000000000000000000000000000000000000000000000100001400004000000000000000000000004000000000000010010000000000000000080004000000000000000000000000000000000000000",
      "type": "0x2",
      "effectiveGasPrice": "0x6b54bc33d"
    }
  ],
  "libraries": [],
  "pending": [],
  "returns": {
    "newDeployments": {
      "internal_type": "struct DeployerDeployment[]",
      "value": "[(YearnPrivateMIM3PoolCauldron2, 0xe61519F163cB2515199ed6de656Cc7E5A50EEF0D, 0x, 0x, , 1, 1)]"
    }
  },
  "timestamp":<PHONE_NUMBER>,
  "chain": 1,
  "multi": false,
  "commit": "9b808b9"
}

Thanks @0xCalibur for the report,
I ll look into it, I think forge-deploy assume there is tx associated to each deployment but this should obviously not always be the case.
I ll investigate that later


Support GraphQL Config in graphql-codegen plugin

Added support for GraphQL Config in graphql-codegen plugin config file.
https://the-guild.dev/graphql/config

Thanks for the PR, @taro-28! I'm wondering, what is the reason to extend the graphql-codegen plugin instead of creating a new plugin?
Their docs show the graphql-config package should enable this plugin, and so on. LEt me know if you have any questions, happy to answer them.

@webpro
Thanks for your comment!
Indeed, it seems better to add the graphql-config plugin...!
(I added it to the graphql-codegen plugin without much thought...🙏)
I haven't fully understood how to add the plugin yet, but would it be okay if I tried it while looking at the existing code?
I am thinking of the following flow:

If there is a graphql-config config file (e.g., graphql-config.ts), enable the graphql-config plugin.
Enable the plugin (e.g., graphql-codegen) corresponding to the key (e.g., graphql-codegen) in the extensions of the config file.

I haven't fully understood how to add the plugin yet, but would it be okay if I tried it while looking at the existing code?

In case you've missed it, here's a guide to create the plugin including boilerplate generation: https://knip.dev/guides/writing-a-plugin
That said, of course you can also copy-paste an existing similar plugin.

The reason I thought it would be better to enable the graphql-config plugin based on the existence of a config file rather than whether graphql-config is included in the dependencies is that the graphql-config file can be used without being installed.

Do you mean that the config file is useful without the dependency/ies being installed? If so, do you maybe have an explanation or link about this?

In case you've missed it, here's a guide to create the plugin including boilerplate generation: https://knip.dev/guides/writing-a-plugin
That said, of course you can also copy-paste an existing similar plugin.

Thank you very much. I will try with reference to those.

Do you mean that the config file is useful without the dependency/ies being installed? If so, do you maybe have an explanation or link about this?

Libraries like graphql-codegen will find the graphql-config configuration file.
https://github.com/dotansimha/graphql-code-generator/blob/21fbf0db2ba7a560aeb0aa52e9b9bf792ac94227/packages/graphql-codegen-cli/src/graphql-config.ts#L37-L49
So for users (not library developers), I think installing graphql-config is necessary only when they want to use types in a TypeScript configuration file like graphql.config.ts.
If they don't need to use types, they can use it without installation.

Libraries like graphql-codegen will find the graphql-config configuration file.

Thanks for clarifying, now I see why you've extended the existing plugin. Having one plugin makes the implementation more complex, but I guess it's the way to go :)

I am thinking of the following flow:

If there is a graphql-config config file (e.g., graphql-config.ts) or graphql-config is included in the dependencies, enable the graphql-config plugin.
Enable the plugin (e.g., graphql-codegen) corresponding to the key (e.g., graphql-codegen) in the extensions of the config file.

How about using const enablers = [/^@graphql-codegen\//, "graphql-config"] and take it from there? I'm asking, because hasDependency is cheap, no disk reads for users without GQL.
Feel free to create multiple fixture folders to test the various use cases.

hasDependency is cheap, no disk reads for users without GQL.

I see. You're right.
I'll try it that way!

Feel free to create multiple fixture folders to test the various use cases.

Thanks, all right :)

The CI is failing on integration with InvokeAI, and it seems to be caused by recent changes in invoke-ai/InvokeAI. How should I address this issue?

Yeah this sometimes happens. I've "fixed" it in main, so if you rebase it should the lights should go green again.

Pinging @beaussan, as you're the original author of the plugin. Any chance there's public repos using this plugin you could point us to so we can verify there are no regressions?

Yeah this sometimes happens. I've "fixed" it in main, so if you rebase the lights should go green again.

Thanks, I rebased and it fixed it.

Excellent stuff, @taro-28! Going to merge it now and it'll be part of a bigger release, expected later this week.

@webpro
Thank you very much!
Your consistent and courteous support was very helpful!
Looking forward to the release!🤩

I've check everything and it looks great to me!

I've check everything and it looks great to me!

Thank you so much, Nicolas! Didn't want to give you any more work so asked for repos, but this is even better! 🙏 Should go out later this week.

:rocket: This pull request is included in v5.17.0-canary.0. See Release 5.17.0-canary.0 for release notes.
Using Knip in a commercial project? Please consider sponsoring me.


issue with debug and make_bcs for ease grid

When running boundary conditions package if we compile GEOSldas with debug flag
./parallel_build.csh -debug
Package compiles successfully, but during make_bcs run I get an error and abort for ease m9 grid.

I do not have any issues with normal compilation so we never noticed this before.

What is the bcs version selected when you get the floating point error?   NLv3 or NLv4?  NLv5?
What tag GEOSgcm_GridComp version (tag or branch)?

I ran from GEOSldas Develop NLv3 with ease M09.

Will be addressed with https://github.com/GEOS-ESM/GEOSgcm_GridComp/pull/601
See also https://github.com/GEOS-ESM/GEOSgcm_GridComp/issues/596


add detailed documents for beam search implementation

The SequenceGenerator class is so hard to understand, can someone provide a detailed document?
e.g.
        # get the top beam_size active hypotheses, which are just the hypos
        # with the smallest values in active_mask
        active_hypos, _ignore = buffer('active_hypos'), buffer('_ignore')  # [b, k]
        torch.topk(
            active_mask, k=beam_size, dim=1, largest=False,
            out=(_ignore, active_hypos)
        )

        active_bbsz_idx = buffer('active_bbsz_idx')
        torch.gather(
            cand_bbsz_idx, dim=1, index=active_hypos,
            out=active_bbsz_idx,
        )
        active_scores = torch.gather(
            cand_scores, dim=1, index=active_hypos,
            out=scores[:, step].view(bsz, beam_size),
        )

        active_bbsz_idx = active_bbsz_idx.view(-1)
        active_scores = active_scores.view(-1)

        # copy tokens and scores for active hypotheses
        torch.index_select(
            tokens[:, :step + 1], dim=0, index=active_bbsz_idx,
            out=tokens_buf[:, :step + 1],
        )
        torch.gather(
            cand_indices, dim=1, index=active_hypos,
            out=tokens_buf.view(bsz, beam_size, -1)[:, :, step + 1],
        )
        if step > 0:
            torch.index_select(
                scores[:, :step], dim=0, index=active_bbsz_idx,
                out=scores_buf[:, :step],
            )
        torch.gather(
            cand_scores, dim=1, index=active_hypos,
            out=scores_buf.view(bsz, beam_size, -1)[:, :, step],`

The above vectorized code makes me almost crazy. I know it helps speeding computation, but at the cost of understanding.

Hi, could you give me a contact information to discuss the beam search in fairseq ？

We plan to release a simpler (albeit much slower) implementation as an alternative soon. If you want to modify the search code, the simplified version may be a bit easier to work with.
Or, are you trying to understand the fast version? If so, I highly recommend putting a break point (pdb) and stepping through it line by line with a batch of data to see what’s happening. The code you referenced is simply selecting the topk hypotheses that don’t end in eos, and adding the selected tokens/scores the final tokens/scores tensors.

Hi, Myle. Have you implemented the Bleu validation in the training ?

Hi, could you give me a contact information to discuss the beam search in fairseq ？

wx: AsahiNever

We plan to release a simpler (albeit much slower) implementation as an alternative soon. If you want to modify the search code, the simplified version may be a bit easier to work with.
Or, are you trying to understand the fast version? If so, I highly recommend putting a break point (pdb) and stepping through it line by line with a batch of data to see what’s happening. The code you referenced is simply selecting the topk hypotheses that don’t end in eos, and adding the selected tokens/scores to the final tokens/scores tensors.

thx O(∩_∩)O

Hi, has there been any further update on the more-understandable beam search that the one from SequenceGenerator?
Also, what is SequenceScorer doing exactly? From the code if i'm not wrong, is it just a normal forward pass inference equivalent to beam_size=1?

Someone wrote a really nice tutorial about the beam search implementation in fairseq: http://www.telesens.co/2019/04/21/understanding-incremental-decoding-in-fairseq/
We have a slightly simpler beam search implementation, but we'd like to simplify it even further (by removing all batching) before releasing it. In any case this will be much slower than the current vectorized implementation.

Any updates on the release of the simpler beam search implementation?

@villmow, take a look here: https://github.com/pytorch/fairseq/commit/20bbbdc068dabef1a3a4689dd5a9ccffcc8297c5
It should be a drop-in replacement for SequenceGenerator. It's still batched, but removes a lot of the other complexity. Happy to take a PR if you want to take a stab at integrating this more cleanly or simplifying it further.

I echo @shawnthu's sentiment, the current SequenceGenerator class and especially its generate method is very hard to dissect for an outsider (like me).


fix(deps): update images to latest

This PR contains the following updates:

Package
Type
Update
Change

icr.io/ibm-iac-charts/sysdig-deploy
helm_release
minor
1.48.1 -> 1.49.5

icr.io/ibm-iac/kspm-analyzer

patch
1.42.6 -> 1.42.7

icr.io/ibm-iac/vuln-host-scanner

patch
0.8.0 -> 0.8.1

icr.io/ibm-iac/vuln-runtime-scanner

patch
1.6.10 -> 1.6.11

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
👻 Immortal: This PR will be recreated if closed unmerged. Get config help if that's undesired.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

/run pipeline

/run pipeline

/run pipeline

/run pipeline

/run pipeline

/run pipeline

/run pipeline

/run pipeline

/run pipeline

/run pipeline

/run pipeline

/run pipeline

:tada: This PR is included in version 1.1.10 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


I could not connect to a repository

In Pentaho 7, the connection to the repository has been removed from the Tools> Repository> Connect menu. Now this option is in the upper right corner of the screen. I could not find any of the options on Spoon Web. Am I doing something wrong? Or is it really a bug for version 7?

From 7.0, the repository is provided as a plugin, meaning webSpoon should be deployed with plugins.
How did you deploy webSpoon?

If it was deployed as a Docker container, please use the image "with plugins" (tag: latest-full).
If it was deployed to (bare) Tomcat, download and unzip pdi-ce-<IP_ADDRESS>-25.zip, then copy the system and plugins folders to tomcat/system and tomcat/plugins, respectively.(https://github.com/HiromuHota/pentaho-kettle#deploy-to-bare-tomcat)
If it was deployed to Pentaho server, deploying webSpoon with plugins does not work, hence is not currently supported.

I has deployed webSpoon to Tomcat with Pentaho CE <IP_ADDRESS>-12, i can configure connection to Repository but when i try to connect get message erro: login failure, but if i try to connect with spoon i can connect normaly.
I did copy plugins repositories-plugin-core-<IP_ADDRESS>-12.jar and repositories-plugin-core-<IP_ADDRESS>-12-source.jar to system/karaf/system/org/pentaho/repositories-plugin-core/<IP_ADDRESS>-12/ and others, like the recomendation on https://github.com/HiromuHota/pentaho-kettle#plugins, if i doesnt this, webSpoon can't get work correctly the Repositorie connection.
Here the steps i did:

Download spoon.war https://github.com/HiromuHota/pentaho-kettle/releases/download/webspoon%2F<IP_ADDRESS>/spoon.war
Download and unzip apache-tomcat-9.0.0.M21
Copy spoon.war to webapps/ in apache-tomcat root directory
Copy plugins and system from pdi-ce-71.0.0-12 to apache-tomcat root directory
Copy pdi-platform-utils-plugin, big-data-plugin, repositories-plugin for determinated location, only repositories-plugin is diferent becouse of Pentaho CE.
Copy JDBCs to apache-tomcat/lib
Start apache-tomcat

Hi @marcellobs,
Thank for reporting.
I was able to reproduce the issue with webSpoon/<IP_ADDRESS>, so nothing wrong with your configurations.
I'm currently trying to figure out what is wrong.
In the meantime, you can try with webSpoon/<IP_ADDRESS>, which I tested again but didn't find this issue.

I fixed the issue by 71b441c0307cd112682e562832daf2099a8fdf76.
This was caused by the fact the repositories-plugin has changed the way how it works in 7.1.
You can build a spoon.war yourself using the latest commit, otherwise please wait for the next release.

Close this issue until it should be re-opened.


superfluous response.WriteHeader call from code.gitea.io

Gitea version (or commit ref): 1.14.0+dev-642-g0e0424c8e
Database (use [x]):

[x] PostgreSQL
[ ] MySQL
[ ] MSSQL
[ ] SQLite

Can you reproduce the bug at https://try.gitea.io:

[ ] Yes (provide example URL)
[x] No

Log gist:

2021/01/30 11:16:48 ...els/issue_comment.go:382:LoadLabel() [W] Commit 922 cannot load label 0
2021/01/30 11:16:48 ...els/issue_comment.go:382:LoadLabel() [W] Commit 958 cannot load label 0
2021/01/30 11:16:48 ...els/issue_comment.go:382:LoadLabel() [W] Commit 961 cannot load label 0
2021/01/30 11:16:48 ...s/context/context.go:191:HTML() [E] Render failed: template: repo/issue/view_content/comments:188:51: executing "repo/issue/view_content/comments" at <RenderLabels .AddedLabels>: error calling RenderLabels: runtime error: invalid memory address or nil pointer dereference
2021/01/30 11:16:48 ...c/net/http/server.go:3095:logf() [I] http: superfluous response.WriteHeader call from code.gitea.io/gitea/modules/context.(*Response).WriteHeader (response.go:64)

Description
Template render errors cause HTML content of Internal Server Error page being displayed as text/plain along with full panic message.

Related to #14466

Related to #14466

This will be due to the panic writing a header and the original template renderer writing a header.
Unfortunately I don't think there is likely to be a solution here. You have to write the header before you start writing the template but if there is a panic in the template then the panic handler will attempt to write the header again.
So I think we're just gonna have to accept this one as a wontfix

Are you sure that is correct? Do we really want that if panic occurs during template rendering, the error page's HTML content is dumped as text/plain (so not rendered as HTML)? That seems kind of a weird. Also I really don't think panic message should be exposed to user in production mode.

@CirnoT your original message was about the writeheader log message - not about it being dumped as plain text. WriteHeader will always have to happen.
I'll reopen. It's really really helpful if the issue is clear from the start as to what the real problem is.


Feature/equal merit tie breakers

PREVIEW:DEVELOP

@HalcyonJAC, sorry to ask again, but can you please regenerate the PR preview URL?

@HalcyonJAC I tested this PR and had one intial suggestion and then ran into an issue early on.

teams will have a dry run for their EMZ tie-break, but they will also need to run a main test. We should have some kind of differentiation between the main test and a dry run. We need an option for people to select the scenario test (main test) and the dry run:

I was told that I couldn't create the test becuase a QT was open for the exercise, but I don't think there was.

I thought this issue could be because there was QTs open across the develop app, as we had discussed not allowing QTs and EMZ test to run at the same time????
I can retest tomorrow as well, just in case.

[ ] Update the 'Create an EM Tie-breaker test' page so that Scenario Test is visible and chosen automatically, so that it is clear to the user that the test being created is a scenario test. Note: The user will still be able to choose whether it is a dry run or not.

Sent to the UTG a second time, after last week's testers did not complete.

Have fixed the merge conflicts however there are a few fixes still to make

@warrensearle @tomlovesgithub Is there anything needed from this ticket?

Hey @londonprojects, @tomlovesgithub and I are pairing today to sort out these merge conflicts

@Halcyon can you regenerate the PR pls.

Merge conflicts resolved and router bugs fixed. PR url regenerated.
It would be good to get a re-review of this and sent for testing again.

@HalcyonJAC when I click on 'Equal merit tie-breakers' link it takes me back to Exercises list -

Thanks @mbrookeswebdev it looks like the routes were merged into the wrong place after staged applications maybe? I've edited and it appears to work now. Please could you try again?

I don't think I am doing anything wrong but the Create tests button is disabled -

@HalcyonJAC @JeremykJAC Thanks for resolving the merged conflict however I got this error message while l was creating a Equal Merit Tie broke test. I know Kate flagged  this issues up sometime ago. Can we look this pls. I should be able to input and date without having to change the date in my timeline. The scenario for qualifying test should be mirrored into Equal Merit Tie broke test.
Capture.PNG99.PNG

Hi @joy-ade
The reason this is happening is because there is an EMT second stage date set for the 22/07/21 and therefore the system requires the first EMT to be completed.
This was included on the development plan:

[ ] 'When creating a EM tie-breaker the start and end dates for the test should default to the Equal Merit Second Phase start and end dates for the exercise'.

Would you like us to ignore the second stage and allow the end date to be set afterwards?

@KateMJAC FYI
JeremykJAC
commented 20 minutes ago
Hi @joy-ade
The reason this is happening is because there is an EMT second stage date set for the 22/07/21 and therefore the system requires the first EMT to be completed.
This was included on the development plan:
'When creating a EM tie-breaker the start and end dates for the test should default to the Equal Merit Second Phase start and end dates for the exercise'.
Would you like us to ignore the second stage and allow the end date to be set afterwards?

@joy-ade, yesterday I outlined to Jeremy that there may be an outlier where a candidate needs to sit the test at a different date, after the intial test.

@KateMJAC We will remove the constraint that stops the tie breaker end date being forced to be before the second stage date. If this is ok, then we will do this first thing and hopefully we can get this ticket moving again :)

I have done that now. Darrel.


chore: update tests to new react and cleanup

This supports yarn for the project and fixes local RN testing issues. Needed for #240


Bounding box visibility for small/many bounding boxes

When working on https://github.com/scenerygraphics/sciview/issues/241
I was trying to turn on bounding boxes for all meshes in the segmentation demo, and it looks like bounding boxes can disappear pretty easily, see

All of these sphere should be showing bounding boxes.

@skalarproduktraum i think this one is upstream


Add build/release process

Current process is manual, and I often goof it up and end up publishing a minor version (I often forget to do npm run build):
// Manually update `package.json` version
git commit -m "Bumping to vX.X.X"
npm run build
git tag -a vX.X.X -m "vX.X.X"
git push origin master --follow-tags
npm publish

Maybe move the "contributors" field in package.json to a separate file as well, and let that be auto-generated during build as well?


[SPARK-43300][CORE] NonFateSharingCache wrapper for Guava Cache

What changes were proposed in this pull request?
Create NonFateSharingCache to wrap around Guava cache with a KeyLock to synchronize all requests with the same key, so they will run individually and fail as if they come one at a time.
Wrap cache in CodeGenerator with NonFateSharingCache to protect it from unexpected cascade failure due to cancellation from irrelevant queries that loading the same key. Feel free to use this in other places where we used Guava cache and don't want fate-sharing behavior.
Also, instead of implementing Guava Cache and LoadingCache interface, I define a subset of it so that we can control at compile time what cache operations are allowed and make sure all cache loading action go through our narrow waist code path with key lock. Feel free to add new APIs when needed.
Why are the changes needed?
Guava cache is widely used in spark, however, it suffers from fate-sharing behavior: If there are multiple requests trying to access the same key in the cache at the same time when the key is not in the cache, Guava cache will block all requests and create the object only once. If the creation fails, all requests will fail immediately without retry. So we might see task failure due to irrelevant failure in other queries due to fate sharing.
This fate sharing behavior leads to unexpected results in some situation(for example, in code gen).
Does this PR introduce any user-facing change?
No
How was this patch tested?
UT

PR description seems a bit vague, given that actual code comments give pretty clear reasoning for why this change is needed?

Wrap cache in CodeGenerator as an example. Feel free to use this in other places where we used Guava cache and don't want fate-sharing behavior.

It's not just an "example" -- the code comments detail a specific bad behavior we're trying to avoid with this change.

This fate sharing behavior might lead to unexpected results in some situation.

Again, we can be specific -- we know it does lead to unexpected results (query canceled during codegen)

Hi @ryan-johnson-databricks would you mind triggering the merge for this PR?

Hi @ryan-johnson-databricks would you mind triggering the merge for this PR?

Sorry, I'm not a spark committer.

Hi @JoshRosen would you mind merging this PR? Or maybe @cloud-fan could help?

I've merged this to master (Spark 3.5.0). Thanks @liuzqt!

@liuzqt @JoshRosen Report an issue about maven test related to this pr.
The CodeGenerator in the catalyst module uses the NonFateSharingCache in the core module, and the input parameter of NonFateSharingCache#apply is com.google.common.cache.Cache.
The catalyst module may use shaded spark-core jar when we do maven test, the Guava related classes in the core module will be relocated from com.google.common to org.sparkproject.guava, so the input parameter of NonFateSharingCache#apply will change to org.sparkproject.guava.cache.Cache, but the catalyst module has not been shaded yet when do maven test, so CodeGenerator will still use com.google.common.cache.Cache to call the NonFateSharingCache#apply function, this will result in a mismatch of input types when do maven test and maven test will failed.
The steps to reproduce are as follows:
 build/mvn clean install -DskipTests -pl sql/catalyst -am
 build/mvn test -pl sql/catalyst

the catalyst module maven test aborted as follows:
ProductAggSuite:
*** RUN ABORTED ***
  java.lang.NoClassDefFoundError: Could not initialize class org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator$
  at org.apache.spark.sql.catalyst.expressions.codegen.JavaCode$.variable(javaCode.scala:64)
  at org.apache.spark.sql.catalyst.expressions.codegen.JavaCode$.isNullVariable(javaCode.scala:77)
  at org.apache.spark.sql.catalyst.expressions.Expression.$anonfun$genCode$3(Expression.scala:200)
  at scala.Option.getOrElse(Option.scala:189)
  at org.apache.spark.sql.catalyst.expressions.Expression.genCode(Expression.scala:196)
  at org.apache.spark.sql.catalyst.expressions.codegen.GenerateSafeProjection$.$anonfun$create$1(GenerateSafeProjection.scala:156)
  at scala.collection.immutable.List.map(List.scala:293)
  at org.apache.spark.sql.catalyst.expressions.codegen.GenerateSafeProjection$.create(GenerateSafeProjection.scala:153)
  at org.apache.spark.sql.catalyst.expressions.codegen.GenerateSafeProjection$.create(GenerateSafeProjection.scala:39)
  at org.apache.spark.sql.catalyst.expressions.codegen.CodeGenerator.generate(CodeGenerator.scala:1369)

I revert this patch locally, and the above maven test can pass.
also cc @HyukjinKwon to know


Server doesn’t always terminate its process

On using ctrl + c to close server, it can fail a few seconds later, emitting the following error:
npmnode:events:492
      throw er; // Unhandled 'error' event
      ^

Error: read EIO
    at TTY.onStreamRead (node:internal/stream_base_commons:217:20)
Emitted 'error' event on ReadStream instance at:
    at emitErrorNT (node:internal/streams/destroy:151:8)
    at emitErrorCloseNT (node:internal/streams/destroy:116:3)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
  errno: -5,
  code: 'EIO',
  syscall: 'read'
}

Node.js v18.17.0

I think I’ve fixed this in 7bb6ade826a6363bf359f16b31864d755c18cacc.


Sort order not ideal for feature in admin

Hi,
When affecting entities to buckets, the sort order is from least recent to most recent, in the "Available items" sections.
This is not ideal, since you likely will follow the following path when editing :

create a new item (let's say, an article)
go to the features page
search for your item in the 'available items' on the left
feature it

Right now, you must go to the last page, to the bottom, to find your article. If you search for it, it's the same, your newly created article will be the last in the list.
I've checked in FeaturedController and I haven't find a way to change this behaviour. Or maybe I missed something ?
Thanks !

Ok, nevermind, I did miss something. In fact you can do :
'bucketables' => [
                    [
                        'module' => 'articles',
                        'name' => 'Articles',
                        'scopes' => ['published' => true],
                        'orders' => ['publish_start_date' => 'DESC'],
                    ],
    ],

I'm leaving this here for future reference since it's not in the docs


huggingface model how to convert to SAT model

Feature request  / 功能建议
how to convert huggingface model convert to SAT model?
Motivation / 动机
So can use sat to train V2 faster.
Your contribution / 您的贡献
NA

Here are the SAT version of CogVLM2: https://github.com/THUDM/CogVLM/blob/cogvlm2_dev/utils/models/cogvlm2_model.py
You can train the model with this branch:
https://github.com/THUDM/CogVLM/tree/cogvlm2_dev

Great，thank you so much!

Do I just need to modify this file finetune_demo/finetune_cogvlm_demo.py from FineTuneTrainCogVLMModel to FineTuneTrainCogVLM2Model?
And I wonder how to intial the tokenizer?

Here are the SAT models of CogVLM2: https://github.com/THUDM/CogVLM/blob/cogvlm2_dev/utils/models/cogvlm2_model.py
You can train the model with this branch: https://github.com/THUDM/CogVLM/tree/cogvlm2_dev

It seems can't load directly current open-source CogVLM2 model in CogVLM finetune code, which still need to convert hf checkpoint to sat format for using.


Switch to ui_test in compile fail tests.

Objective
Make compile fail tests less likely to break with new Rust versions. Closes #12627
Solution
Switch from trybuild to ui_test.
TODO

[x] Update bevy_ecs_compile_fail_tests
[x] Update bevy_macros_compile_fail_tests
[x] Update bevy_reflect_compile_fail_tests

The pr is smaller that it seems.

I'd suggest reviewing this commit by commit. They all touch different parts of the project.


Add a warning callout in browser bugs page

In v4 we still have the Browser Bugs page, but it's outdated for sure. We should add a callout or something. We could also remove the page but maybe it makes more sense to keep it with a warning at the top?
/CC @mdo @MartijnCuppens

A warning feels more appropriate.


Have an ffi c example that actually reads data

Flesh out what a more full FFI API for data reading would look like. We can certainly iterate on this.
Lots more into the example for using the c-ffi (sorry for another big PR. It's mostly example code at least):

Actually read the data
Include partition columns
Print out the table
Use clang-format to get consistent c style
Add all the gcc warnings, make them errors, and fix them

Unfortunately the glib arrow api doesn't seem to allow passing options to how we print things, so this will print a max of 4 rows out the table. To print more we'd need to write our own printing function.
This also adds engine_funcs.rs which currently only exposes read_parquet_files. This is more of a POC, but seems like potentially the right way forward for exposing the various functions available to rust when it has an Engine.
A couple of example runs:
Reading the basic_partitioned table:
Reading table at file:///home/nick/databricks/delta-kernel-rs/acceptance/tests/dat/out/reader_tests/generated/basic_partitioned/delta/
version: 1

Building schema
Making a list of lenth 3 with id 0
Asked to visit string named letter belonging to list 0.
Asked to visit long named number belonging to list 0.
Asked to visit double named a_float belonging to list 0.
Schema returned in list 0
Done building schema
Schema:
├─ letter: string
├─ number: long
└─ a_float: double

Starting table scan

Building list of partition columns
Done iterating partition columns
Partition columns are:
  - letter

Iterating scan data

Scan iterator found some data to read
  Of this data, here is a selection vector
    sel[0] = 0
    sel[1] = 1
    sel[2] = 1
    sel[3] = 1
Asking kernel to call us back for each scan row (file to read)
Called back to read file: letter=a/part-00000-b17b8597-4c08-4867-b882-2249d604cbc2.c000.snappy.parquet
  No selection vector for this file
  partition 'letter' here: a
  Reading parquet file at file:///home/nick/databricks/delta-kernel-rs/acceptance/tests/dat/out/reader_tests/generated/basic_partitioned/delta/letter=a/part-00000-b17b8597-4c08-4867-b882-2249d604cbc2.c000.snappy.parquet
  Converting read data to arrow
  Adding partition column 'letter' with value 'a' at column 2
  Added batch to arrow context, have 1 batches in context now
  Done reading parquet file
Called back to read file: letter=e/part-00000-90f8c03f-e005-4714-a8ac-55bcf6d94d7d.c000.snappy.parquet
  No selection vector for this file
  partition 'letter' here: e
  Reading parquet file at file:///home/nick/databricks/delta-kernel-rs/acceptance/tests/dat/out/reader_tests/generated/basic_partitioned/delta/letter=e/part-00000-90f8c03f-e005-4714-a8ac-55bcf6d94d7d.c000.snappy.parquet
  Converting read data to arrow
  Adding partition column 'letter' with value 'e' at column 2
  Added batch to arrow context, have 2 batches in context now
  Done reading parquet file
Called back to read file: letter=f/part-00000-4da5e756-3c84-4b40-b483-ebcf3c4ed6ce.c000.snappy.parquet
  No selection vector for this file
  partition 'letter' here: f
  Reading parquet file at file:///home/nick/databricks/delta-kernel-rs/acceptance/tests/dat/out/reader_tests/generated/basic_partitioned/delta/letter=f/part-00000-4da5e756-3c84-4b40-b483-ebcf3c4ed6ce.c000.snappy.parquet
  Converting read data to arrow
  Adding partition column 'letter' with value 'f' at column 2
  Added batch to arrow context, have 3 batches in context now
  Done reading parquet file

Scan iterator found some data to read
  Of this data, here is a selection vector
    sel[0] = 0
    sel[1] = 0
    sel[2] = 0
    sel[3] = 1
    sel[4] = 1
    sel[5] = 1
Asking kernel to call us back for each scan row (file to read)
Called back to read file: letter=a/part-00000-8fe836ee-ec38-4388-91a1-5c3eff176e84.c000.snappy.parquet
  No selection vector for this file
  partition 'letter' here: a
  Reading parquet file at file:///home/nick/databricks/delta-kernel-rs/acceptance/tests/dat/out/reader_tests/generated/basic_partitioned/delta/letter=a/part-00000-8fe836ee-ec38-4388-91a1-5c3eff176e84.c000.snappy.parquet
  Converting read data to arrow
  Adding partition column 'letter' with value 'a' at column 2
  Added batch to arrow context, have 4 batches in context now
  Done reading parquet file
Called back to read file: letter=b/part-00000-2465ef7c-268e-43be-98fe-c9c37a2a6482.c000.snappy.parquet
  No selection vector for this file
  partition 'letter' here: b
  Reading parquet file at file:///home/nick/databricks/delta-kernel-rs/acceptance/tests/dat/out/reader_tests/generated/basic_partitioned/delta/letter=b/part-00000-2465ef7c-268e-43be-98fe-c9c37a2a6482.c000.snappy.parquet
  Converting read data to arrow
  Adding partition column 'letter' with value 'b' at column 2
  Added batch to arrow context, have 5 batches in context now
  Done reading parquet file
Called back to read file: letter=c/part-00000-28c8b160-ee9c-407d-93f5-ff7d8e2a8d3f.c000.snappy.parquet
  No selection vector for this file
  partition 'letter' here: c
  Reading parquet file at file:///home/nick/databricks/delta-kernel-rs/acceptance/tests/dat/out/reader_tests/generated/basic_partitioned/delta/letter=c/part-00000-28c8b160-ee9c-407d-93f5-ff7d8e2a8d3f.c000.snappy.parquet
  Converting read data to arrow
  Adding partition column 'letter' with value 'c' at column 2
  Added batch to arrow context, have 6 batches in context now
  Done reading parquet file
Scan data iterator done
All done reading table data

Table Data:
-----------

number: int64
a_float: double
letter: string
----
number:
  [
    [
      4
    ],
    [
      5
    ],
  ...,
    [
      2
    ],
    [
      3
    ]
  ]
a_float:
  [
    [
      4.4
    ],
    [
      5.5
    ],
  ...,
    [
      2.2
    ],
    [
      3.3
    ]
  ]
letter:
  [
    [
      "a"
    ],
    [
      "e"
    ],
  ...,
    [
      "b"
    ],
    [
      "c"
    ]
  ]

Reading the nested_types table
Reading table at file:///home/nick/databricks/delta-kernel-rs/acceptance/tests/dat/out/reader_tests/generated/nested_types/delta/
version: 0

Building schema
Making a list of lenth 4 with id 0
Asked to visit integer named pk belonging to list 0.
Making a list of lenth 2 with id 1
Asked to visit double named float64 belonging to list 1.
Asked to visit boolean named bool belonging to list 1.
Asked to visit struct named struct belonging to list 0. Children are in 1.
Making a list of lenth 1 with id 2
Asked to visit short named array_element belonging to list 2.
Asked to visit array named array (contains null: true) belonging to list 0. Types are in 2.
Making a list of lenth 2 with id 3
Asked to visit string named map_key belonging to list 3.
Asked to visit integer named map_value belonging to list 3.
Asked to visit map named map (contains null: true) belonging to list 0. Types are in 3.
Schema returned in list 0
Done building schema
Schema:
├─ pk: integer
├─ struct: struct
│  ├─ float64: double
│  └─ bool: boolean
├─ array (contains null: true): array
│  └─ array_element: short
└─ map (contains null: true): map
   ├─ map_key: string
   └─ map_value: integer

Starting table scan

Building list of partition columns
Done iterating partition columns
Table has no partition columns

Iterating scan data

Scan iterator found some data to read
  Of this data, here is a selection vector
    sel[0] = 0
    sel[1] = 0
    sel[2] = 0
    sel[3] = 1
Asking kernel to call us back for each scan row (file to read)
Called back to read file: part-00000-3b8ca3f0-6444-4ed5-8961-c605cba95bf1-c000.snappy.parquet
  No selection vector for this file
  Reading parquet file at file:///home/nick/databricks/delta-kernel-rs/acceptance/tests/dat/out/reader_tests/generated/nested_types/delta/part-00000-3b8ca3f0-6444-4ed5-8961-c605cba95bf1-c000.snappy.parquet
  Converting read data to arrow
  Added batch to arrow context, have 1 batches in context now
  Done reading parquet file
Scan data iterator done
All done reading table data

Table Data:
-----------

pk: int32
struct: struct<float64: double, bool: bool>
  child 0, float64: double
  child 1, bool: bool
array: list<element: int16>
  child 0, element: int16
map: map<string, int32>
  child 0, entries: struct<key: string not null, value: int32> not null
      child 0, key: string not null
      child 1, value: int32
----
pk:
  [
    [
      0,
      1,
      2,
      3,
      4
    ]
  ]
struct:
  [
    -- is_valid: all not null
    -- child 0 type: double
      [
        0,
        1,
        2,
        3,
        4
      ]
    -- child 1 type: bool
      [
        true,
        false,
        true,
        false,
        true
      ]
  ]
array:
  [
    [
      [
        0
      ],
      [
        0,
        1
      ],
      [
        0,
        1,
        2
      ],
      [
        0,
        1,
        2,
        3
      ],
      [
        0,
        1,
        2,
        3,
        4
      ]
    ]
  ]
map:
  [
    [
      keys:
      []
      values:
      [],
      keys:
      [
        "0"
      ]
      values:
      [
        0
      ],
      keys:
      [
        "0",
        "1"
      ]
      values:
      [
        0,
        1
      ],
      keys:
      [
        "0",
        "1",
        "2"
      ]
      values:
      [
        0,
        1,
        2
      ],
      keys:
      [
        "0",
        "1",
        "2",
        "3"
      ]
      values:
      [
        0,
        1,
        2,
        3
      ]
    ]
  ]

@scovich if you have time, I think this is ready for review (but no worries if not!)

@nicklan -- Anything blocking merge on this PR?

Nope, merged! Thanks for the reviews, I know it was a lot of code.


[BUG] Request with payload of BinaryData from InputStream fails due to invalid content length

Describe the bug
Request fails due to invalid content length.
code
        ByteArrayInputStream messageStream = new ByteArrayInputStream(message.toBytes());
        client.sendToUserWithResponse("test_user",
            BinaryData.fromStream(messageStream),
            WebPubSubContentType.APPLICATION_OCTET_STREAM, message.getLength(),
            null, Context.NONE);

internally it already included the "Content-Length"
        requestOptions.addRequestCallback(httpRequest -> {
            if (httpRequest.getHeaders().stream()
                .noneMatch(header -> header.getName().equalsIgnoreCase("Content-Length"))) {
                httpRequest.setHeader("Content-Length", String.valueOf(contentLength));
            }
        });

Possible cause:
InputStreamContent return null in getLength, which is kind of reasonable.
https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/core/azure-core/src/main/java/com/azure/core/implementation/util/InputStreamContent.java#L40-L46
Code provided Content-Length header in RequestOptions. However due to the order of validateLength and options.getRequestCallback().accept(request), the set of Content-Length header in RequestOptions is after validateLength.
https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/core/azure-core/src/main/java/com/azure/core/http/rest/RestProxy.java#L134-L140
Exception or Stack Trace
Add the exception log and stack trace if available
stack
java.lang.NumberFormatException: null

	at java.base/java.lang.Long.parseLong(Long.java:670)
	at java.base/java.lang.Long.parseLong(Long.java:832)
	at com.azure.core@1.22.0-beta.1/com.azure.core.http.rest.RestProxy.validateLength(RestProxy.java:164)
	at com.azure.core@1.22.0-beta.1/com.azure.core.http.rest.RestProxy.invoke(RestProxy.java:130)

To Reproduce
Steps to reproduce the behavior:
Code Snippet
Add the code snippet that causes the issue.
Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Setup (please complete the following information):

OS: [e.g. iOS]
IDE: [e.g. IntelliJ]
Library/Libraries: [e.g. com.azure:azure-core:1.16.0 (groupId:artifactId:version)] 1.22.0-beta.1 (should be same as the unreleased 1.22.0)
Java version: [e.g. 8] 15
App Server/Environment: [e.g. Tomcat, WildFly, Azure Function, Apache Spark, Databricks, IDE plugin or anything special]
Frameworks: [e.g. Spring Boot, Micronaut, Quarkus, etc]

If you suspect a dependency version mismatch (e.g. you see NoClassDefFoundError, NoSuchMethodError or similar), please provide

verbose dependency tree (mvn dependency:tree -Dverbose)

Additional context
Add any other context about the problem here.
Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

[x] Bug Description Added
[x] Repro Steps Added
[x] Setup information Added

@srnagar @alzimmermsft

@alzimmermsft can you please fix this and do a scrub for other potential unhandled exceptions?
https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/core/azure-core/src/main/java/com/azure/core/http/rest/RestProxy.java#L186

@weidongxu-microsoft do you think that RequestOptions handling should occur before any validation?

@alzimmermsft
The other "validate" in RestProxy is the deprecated validateResumeOperationIsNotPresent, so I assume this is not relevent.
So far validateLength seems will work fine after the PR.

If proxy method does not have @BodyParam (happen when body is optional in swagger), and user provide it via RequestOptions.setBody(). When accept of RequestOptions get called, it will set the body and the content length. Then validation happens.
If proxy method have @BodyParam, and user also provide RequestOptions.setBody(), current core seems override the @BodyParam by the content of RequestOptions. This appears to be reasonable. And then validation happens to the overriden body and content length.

There could be a (unrelated) performance issue (only happen to stream data etc.), for the difference of handling of BinaryData in RestProxy and in RequestOptions.
 I am not exactly sure but it seems will load every bytes for steam data like FluxByteBufferContent and InputStreamContent.
https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/core/azure-core/src/main/java/com/azure/core/http/rest/RequestOptions.java#L238-L242
Though I guess we might not want to touch this in such a late stage (as it might affect 2nd case of the above discussion, when it cannot set content length to stream data).


JVM-target compatibility issue

Some packages I used or my build environment makes me use java 21 and I get this error, as a workaround I forked the project and changed requirements and got fixed it.
But I guess you need to fix it in a more generic way.
Thanks.
My error:
FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':myket_iap:compileReleaseKotlin'.

Inconsistent JVM-target compatibility detected for tasks 'compileReleaseJavaWithJavac' (17) and 'compileReleaseKotlin' (21).

Consider using JVM Toolchain: https://kotl.in/gradle/jvm/toolchain
Learn more about JVM-target validation: https://kotl.in/gradle/jvm/target-validation

Try:

Run with --stacktrace option to get the stack trace.
Run with --info or --debug option to get more log output.
Run with --scan to get full insights.
Get more help at https://help.gradle.org.

BUILD FAILED in 18s

Add in your gradle.properties:
kotlin.jvm.target.validation.mode = IGNORE


Tiny failure case

Triangle is failing to triangulate the convex hull with these vertices
6.899643741648033 10.556739733611963
6.8743893086546723 10.577559204153792
6.8990280198173055 10.557055643048765
6.8994694525740767 10.556817827203695
6.8996252478389311 10.556759464230709
6.899537086138448 10.5566972172105

and these constraint edges
1 2
3 4
4 5
5 4

Just to double check, are the indices starting from 0 in your example?  (I set the z flag for triangle implicitly to support 0-based index.)

Sorry for the confusion. They start at 1 (vertex 1 is [6.899643741648033 10.556739733611963])

This example (and lots others like it) are especially painful because it seems to send triangle into an infinite loop.

My debug traced to this loop: https://github.com/PyMesh/triangle/blob/afc07fb324307dd485b0873bf355ee791be846cc/triangle.c#L11659-L11673
Basically, triangle is looking for the first triangle on a path.  It is looping through 4 vertices and turns out all 4 vertices are all on the right of the search path.
#810: 6.899643741648033  10.556739733611963
#8b0: 6.899537086138448  10.5566972172105
#8d0: 6.8996152772410877 10.55676319934893
#890: 6.8996252478389311 10.556759464230709

The interesting observation is that point #8d0 is not from the input, thus is inaccurate.  Not sure how it got created.

The input segments actually has a tiny intersection.  That explains the extra vertex.  I consider triangle's intersection resolution not robust, however, if preprocessing can resolve these intersections triangle should be pretty good.

Oh nice! I got to the same place :-) the libigl triangle fork will now at
least throw an except rather than loop forever
On Mon, May 24, 2021, 12:59 PM Qingnan Zhou @.***>
wrote:

The input segments actually has a tiny intersection. That explains the
extra vertex. I consider triangle's intersection resolution not robust,
however, if preprocessing can resolve these intersections triangle should
be pretty good.
[image: image]
https://user-images.githubusercontent.com/3606672/119381198-24e0f580-bc8f-11eb-94c9-7b4ef061f0d5.png
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/qnzhou/trianglelite/issues/1#issuecomment-847194245,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AARDJGK23ULDHRS4EMFFD23TPKAVVANCNFSM45LFYODA
.

Great!  I probably have to sync up my repo with libigl's triangle.

Updated to libigl in 8a36714ca3eb40117620792b99327c0aeafd0b62.