Unable to quantize network which includes Normalization layer

Hi ,
I have created a sample network which includes Normalization Layer. I was able to do the FP32 inference but unable to create a quantized armnn model . Below is the error which that I faced during the quantization.
terminate called after throwing an instance of 'armnn::Exception'
  what():  Have no entry for layer GUID [6]

Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51	../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00007ffff57e642a in __GI_abort () at abort.c:89
#2  0x00007ffff60fd0ad in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x00007ffff60fb066 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007ffff60fb0b1 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007ffff60fb2c9 in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x00007ffff71fa1e6 in armnn::RangeTracker::GetRange (this=0x55555579eb50, guid=..., idx=0) at /local-ssd/pgajula/Project/DeepLearning/ARM/Intel1911/armnn/src/armnn/RangeTracker.cpp:36
#7  0x00007ffff71ee3d7 in armnn::QuantizerVisitor::CreateQuantizedBias (this=0x7fffffffd5a0, srcLayer=0x555555799980, weights=..., biases=..., backing=...)
    at /local-ssd/pgajula/Project/DeepLearning/ARM/Intel1911/armnn/src/armnn/QuantizerVisitor.cpp:90
#8  0x00007ffff71eedfc in armnn::QuantizerVisitor::VisitConvolution2dLayer (this=0x7fffffffd5a0, layer=0x555555799980, convolution2dDescriptor=..., weights=..., biases=..., name=0x5555557999b0 "convLayer")
    at /local-ssd/pgajula/Project/DeepLearning/ARM/Intel1911/armnn/src/armnn/QuantizerVisitor.cpp:230
#9  0x00007ffff7111ba5 in armnn::Convolution2dLayer::Accept (this=0x555555799980, visitor=...) at /local-ssd/pgajula/Project/DeepLearning/ARM/Intel1911/armnn/src/armnn/layers/Convolution2dLayer.cpp:147
#10 0x00007ffff71d18b3 in armnn::VisitLayers<armnn::Graph> (layerContainer=..., visitor=...) at /local-ssd/pgajula/Project/DeepLearning/ARM/Intel1911/armnn/src/armnn/NetworkQuantizerUtils.hpp:59
#11 0x00007ffff71cfb95 in armnn::NetworkQuantizer::ExportNetwork (this=0x55555579eaa0) at /local-ssd/pgajula/Project/DeepLearning/ARM/Intel1911/armnn/src/armnn/NetworkQuantizer.cpp:175
#12 0x00005555555664ef in main (argc=10, argv=0x7fffffffe128) at /local-ssd/pgajula/Project/DeepLearning/ARM/Intel1911/armnn/src/armnnQuantizer/ArmNNQuantizerMain.cpp:117

Could you look into this issue.
I have attached the below files for your reference:

GoogleNet.cpp - for FP32 inference and armnn model creation
img.txt for FP32 inference
img.raw , csv file used for quantization
FP32 models

GoogleNet.zip
Thank you,
Praveen.

Hi @uu-praveeng
Could you please let us know the following:

armnn version your are working on
the command you type when you run the tool

Hi @uu-praveeng
We're working on a patch to fix this problem, in the meantime you could use the solution shown below:
diff --git a/src/armnn/DynamicQuantizationVisitor.cpp b/src/armnn/DynamicQuantizationVisitor.cpp
index 4b1dce0b..c8100bc2 100644
--- a/src/armnn/DynamicQuantizationVisitor.cpp
+++ b/src/armnn/DynamicQuantizationVisitor.cpp
@@ -108,6 +108,16 @@ void DynamicQuantizationVisitor::VisitBatchNormalizationLayer(const IConnectable
     AddToCalibratedLayers(layer);
 }

+void DynamicQuantizationVisitor::VisitNormalizationLayer(const IConnectableLayer* layer,
+                                 const NormalizationDescriptor& desc,
+                                 const char* name)
+{
+    boost::ignore_unused(desc);
+    boost::ignore_unused(name);
+    SetRange(layer, 0, -15.0f, 15.0f);
+    AddToCalibratedLayers(layer);
+}
+
 void DynamicQuantizationVisitor::VisitConvolution2dLayer(const IConnectableLayer* layer,
                                                          const Convolution2dDescriptor& convolution2dDescriptor,
                                                          const ConstTensor& weights,
diff --git a/src/armnn/DynamicQuantizationVisitor.hpp b/src/armnn/DynamicQuantizationVisitor.hpp
index 43768fd0..16f826eb 100644
--- a/src/armnn/DynamicQuantizationVisitor.hpp
+++ b/src/armnn/DynamicQuantizationVisitor.hpp
@@ -25,6 +25,10 @@ public:
     /// Functions to set the Range on a per-layer-type basis
     void VisitAdditionLayer(const IConnectableLayer* layer, const char* name = nullptr) override;

+    void VisitNormalizationLayer(const IConnectableLayer* layer,
+                                 const NormalizationDescriptor& desc,
+                                 const char* name = nullptr) override ;
+
     void VisitBatchNormalizationLayer(const IConnectableLayer* layer,
                                       const BatchNormalizationDescriptor& desc,
                                       const ConstTensor& mean,
@@ -134,4 +138,4 @@ private:
     void RemoveDebugLayers();
 };

-} //namespace armnn
\ No newline at end of file
+} //namespace armnn

Hope this helps.

Hi @morgolock ,
Thanks for the patch. I was able to do the quantization now and inference as well.
I found huge difference wrt FP32 output vs INT8 output.
Could you cross verify the same and let me know how you fare.
Thank you,
Praveen.

Hi @morgolock
Any update on this. Let me know if you need any inputs.
Thank you,
Praveen

Hi @uu-praveeng
Could you please provide some numbers showing the differences you mentioned?
ArmNN quantisation is experimental, I'd suggest using tensorflow quantisation tools instead.

Closing due to inactivity.


Introduce const for a more consistent formatting

The token method, and related routines can do quirky things with strings, which might be debatable.
Introducing 'constant like' formatting makes it all simpler, but uglier to read.

only use capitals
only use digits
replace anything else with one single underscore _

Arguably, these should be prepended with SQLSTATE_. (this should have been done with HTTP::Status).
Suggestion, use our $CONST_PREFIX = 'SQL_STATE' sort can be adjusted at will.

Can be closed, as of v0.04


M3-1801: Fix tags error for restricted users w/ Linode creation permissions

Description
Currently, restricted users with Linode creation permissions who go to create a Linode see this:

This is because calling the getTags method returns an error. Based on the API, these users are able to create Linodes with (new) tags but can't interact with existing tags on other entities.
I added an optional fromLinodeCreateFlow prop that gets passed from LinodeCreate --> LabelAndTagsPanel --> TagsInput, where the defaultError is not set if the error reason is "Unauthorized" and fromLinodeCreateFlow is true.
Type of Change

Bug fix ('fix', 'repair', 'bug')

Have you reached out to Matthew about this one? Hiding the error works and is probably an acceptable workaround, but the UX isn't great--the dropdown says "Type to choose or create a tag" but a restricted user can never choose, the list is empty. If they know the name of an existing tag they can type it themselves, but it's not a great experience.

I hadn't but I just reached out to get his perspective

I had another (optional) thought though, which is that while intercepting the error works well, we know in advance the request will fail for any restricted user (const { isRestrictedUser } = useAccountManagement();), so we could save that round trip by not making the request in the first place.

This sounds good. For now I'll merge this PR but I made M3-5081 for this


bug: Filter with "{}" operator error

I use 4.7.3 version of dgs and my query is running without any problem. But after update to 4.7.4 version I get this error. My query is:
{
  students(filter: {}) {
    status
    currentGradeLevel
    gradeLevel {
      shortName
      order
    }
    profile {
      firstName
      dateOfBirth
    }
  }
}

Expected behavior
Query run
Actual behavior
Error Message
com.netflix.graphql.dgs.exceptions.DgsInvalidInputArgumentException: Input argument type 'class x.xxxx.xxxxxx.service.analysis.documents.FilterInput' doesn't match input {}
	at com.netflix.graphql.dgs.internal.InputObjectMapper.mapToJavaObject(InputObjectMapper.kt:109)
	at com.netflix.graphql.dgs.internal.DataFetcherInvoker.convertValue(DataFetcherInvoker.kt:240)
	at com.netflix.graphql.dgs.internal.DataFetcherInvoker.processInputArgument(DataFetcherInvoker.kt:207)
	at com.netflix.graphql.dgs.internal.DataFetcherInvoker.invokeDataFetcher(DataFetcherInvoker.kt:62)
	at com.netflix.graphql.dgs.internal.DgsSchemaProvider.createBasicDataFetcher$lambda-19(DgsSchemaProvider.kt:310)
	at com.netflix.graphql.dgs.internal.DgsSchemaProvider$$Lambda$670/0x0000000000000000.get(Unknown Source)
	at graphql.execution.ExecutionStrategy.fetchField(ExecutionStrategy.java:270)
	at graphql.execution.ExecutionStrategy.resolveFieldWithInfo(ExecutionStrategy.java:203)
	at graphql.execution.AsyncExecutionStrategy.execute(AsyncExecutionStrategy.java:60)
	at graphql.execution.Execution.executeOperation(Execution.java:165)
	at graphql.execution.Execution.execute(Execution.java:104)
	at graphql.GraphQL.execute(GraphQL.java:557)
	at graphql.GraphQL.parseValidateAndExecute(GraphQL.java:482)
	at graphql.GraphQL.executeAsync(GraphQL.java:446)

Steps to reproduce
Note: A test case would be highly appreciated, but we understand that's not always possible

Could you please update to the latest version, 4.7.6?
Could you also share the method signature of your DataFetcher that has the InputArgument that fails?

Nothing changed.
	@DgsData(parentType = "Query", field = "students")
	public List<Student> students(@InputArgument("filter") FilterInput filter)
	{
		if (filter == null)
		{
			return Collections.emptyList();
		}

		List<Student> result = repository.findAll(filter);
		return result;
	}

public class FilterInput
{
	private FilterField expression;
	private List<FilterField> and;
	private List<FilterField> or;

Thanks @blueiceprj , this is fixed in v4.8.0.


How to pass CRLF boundry last line in MultipartFormData Request

Hi All, I am using AFNetworking library in all my apps. And also used it MultipartFormData Task to upload picture on server. That is working everytime on linux server. Right now i am using backend with windows server in .Net. However I am facing one issue like image not getting uploaded because in boundry last line CRLF (Carriage Return and Line Feed --> \r\n) is not passing. I checked with Header request with many clients on browser and finally got know that just that last CRLF line is missing when I call API from my app. Is there any way I could pass CRLF as a last boundry line to API call?

There hasn't been any activity on this issue recently. I'm closing this now to clean up some old issues. If this issue persists with the latest version of AFNetworking, or you have a reproducible test case, please open a new issue and reference this issue.
Thanks! 🍻


Update staticify.

@rakeshpai: any idea why whatever maxAge I set, I always end up with 1 year? etag has an effect, I can see it being removed.
You can try /showcase/ images to see their headers.

On my phone, so I’m limited. But IIRC, maxage is 1yr, if it’s a url with a
hash.
On Fri, 22 Dec 2017 at 7:13 PM, XhmikosR<EMAIL_ADDRESS>wrote:

@rakeshpai https://github.com/rakeshpai: any idea why whatever maxAge I
set, I always end up with 1 year? etag has an effect, I can see it being
removed.
You can try /showcase/ images to see their headers.
You can view, comment on, or merge this pull request online at:
https://github.com/MaxCDN/bootstrap-cdn/pull/942
Commit Summary

Update staticify.

File Changes

M app.js
https://github.com/MaxCDN/bootstrap-cdn/pull/942/files#diff-0 (7)
M package-lock.json
https://github.com/MaxCDN/bootstrap-cdn/pull/942/files#diff-1 (10)
M package.json
https://github.com/MaxCDN/bootstrap-cdn/pull/942/files#diff-2 (2)

Patch Links:

https://github.com/MaxCDN/bootstrap-cdn/pull/942.patch
https://github.com/MaxCDN/bootstrap-cdn/pull/942.diff

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/MaxCDN/bootstrap-cdn/pull/942, or mute the thread
https://github.com/notifications/unsubscribe-auth/AAFQwBz2TPYfq7GA-fbnVA08HtRPCHhCks5tC7HkgaJpZM4RLDQr
.

--
Rakesh Pai
Mumbai, India.
http://rakeshpai.me/
http://twitter.com/rakesh314
http://www.google.com/profiles/rakeshpai

@XhmikosR merge when ready

@rakeshpai: so... any ideas why whatever I set for sendOptions's maxAge, we always end up with 1 year? I tried setting it to '1d which should work in theory.

@jmervine: you OK with the small JS refactoring?

@XhmikosR Not sure. Could you try replicating this in a test in staticify, and checking if the test fails?

@rakeshpai; it doesn't seem to happen upstream, but I just don't get why it happens here :/

I think there's a conflict with serve-static somewhere...
It's not a big deal since we want to serve the hashed assets with 1 year, but it worries me that the setting has no effect.


Update KnowledgeBasePanel.php

Change sort comparison for Documentable to use spaceship operator instead of a boolean result
Fixes #28

Hi Travis, thanks for the fix! merging


What is the scaling_factor?

We have latent_shift and latent_magnitude values here:
https://github.com/madebyollin/taesd/blob/main/taesd.py#L44C1-L45C23
But is there a scaling_factor as well or is it just one?
scaling_factor as observed in https://github.com/huggingface/diffusers/blob/ea5b0575f8f91b76f32fb6f6930c0bc30e42865e/src/diffusers/models/autoencoder_kl.py#L61.

There is no scaling_factor for TAESD - TAESD directly converts SD(XL) latents into RGB images in [0, 1] (see the usage in the example notebook). So if you need to specify a value you can probably set it to 1.0.
(The latent_shift and latent_magnitude values in taesd.py are only relevant if you want to store latents into RGBA PNG files - sorry for the confusion.)

Thanks for your reply!
I am trying to integrate your work in diffusers so that users can use it very easily (of course, crediting this repository).
With the following code (diffusers was installed using pip install git+https://github.com/huggingface/diffusers@feat/tiny-autoenc):
import torch
from diffusers import DiffusionPipeline, TinyAutoencoder

pipe = DiffusionPipeline.from_pretrained(
    "stabilityai/stable-diffusion-2-1-base", torch_dtype=torch.float16
)
pipe.vae = TinyAutoencoder.from_pretrained("sayakpaul/taesd-diffusers", torch_dtype=torch.float16)
pipe = pipe.to("cuda")

prompt = "slice of delicious New York-style berry cheesecake"
image = pipe(prompt, num_inference_steps=25, height=512, width=512, guidance_scale=3.0).images[0]
image

I am getting:

Is the quality somewhat expected?
To give you some more context here's what we do in the standard pipeline settings.
After we get the latents from the UNet,

We first decode it: https://github.com/huggingface/diffusers/blob/ea5b0575f8f91b76f32fb6f6930c0bc30e42865e/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py#L697.
And then run it through the postprocessor: https://github.com/huggingface/diffusers/blob/ea5b0575f8f91b76f32fb6f6930c0bc30e42865e/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py#L708.
Then we denormalize the image: https://github.com/huggingface/diffusers/blob/ea5b0575f8f91b76f32fb6f6930c0bc30e42865e/src/diffusers/image_processor.py#L240.
And then generate the final PIL image.

From your [example notebook](https://github.com/madebyollin/taesd/blob/ma, in/examples/Previewing_During_Image_Generation.ipynb), from this line:
res_taesd = taesd_dec(latents).cpu().permute(0, 2, 3, 1).float().clamp(0, 1).numpy()

to this one in diffusers, it feels like that the additional (images / 2 + 0.5) is not required for the tiny autoencoder?
Would be amazing to get your thoughts here.

to this one in diffusers, it feels like that the additional (images / 2 + 0.5) is not required for the tiny autoencoder?

Seems like it's indeed the case.
When I do:
import PIL

pipe.vae = TinyAutoencoder.from_pretrained(
    "sayakpaul/taesd-diffusers", torch_dtype=torch.float16
).to("cuda")
latents = pipe(
    prompt, num_inference_steps=25, height=512, width=512, guidance_scale=3.0,
    generator=torch.manual_seed(0), output_type="latent"
)

decoded_image = pipe.vae.decode(
    latents / pipe.vae.config.scaling_factor, return_dict=False
)[0]
decoded_image = decoded_image.permute(0, 2, 3, 1).float().clamp(0, 1).cpu().detach().numpy().squeeze(0)

PIL.Image.fromarray((decoded_image * 255).round().astype("uint8"))

With this, I am getting:

When I use the original VAE, I get:
from diffusers import AutoencoderKL

original_vae = AutoencoderKL.from_pretrained(
    "stabilityai/stable-diffusion-2-1-base", subfolder="vae", torch_dtype=torch.float16
).to("cuda")
pipe.vae = original_vae

prompt = "slice of delicious New York-style berry cheesecake"
image = pipe(
    prompt, num_inference_steps=25, height=512, width=512, guidance_scale=3.0,
    generator=torch.manual_seed(0)
).images[0]
image

Closing the issue.

Yup, TAESD directly predicts values in [0, 1] so you don't need the additional denormalization step (though clamping is still recommended). The image here look correct to me 👍


Link With Unclear Purpose fails even when aria-label is defined

Hi all,
I've noticed that the LinkWithUnclearPurpose test fails if there is no text inside the anchor element and if aria-label has been specified.
I believe that this test should pass since the label has been defined.
If you agree, I'd be more than happy to provide a PR?
Thanks!

How can close my issues


Key navigation in component tree is buggy

Nav key such as left, right, up or down doesn't work in the component tree of the chrome extension.
devtools ver 2.3.1 from chrome webstore | extension (2016-11-14)
chrome ver 55.0.2883.95 (mac OS 64-bit)
Only left and right keys work if built and installed locally.

tested with v3 and up/down still doesn't work


[GraalVM 21.0.0] Hazelcast extension : using HazelcastSedaProducer with transferExchange=true

On native mode, it's impossible use to write HazelcastSedaProducer with transferExchange=true.
The reason is there an issue while writing object of type DefaultExchangeHolder. It's related to this issue graal#460

Support for JDK serialization/deserialization https://github.com/oracle/graal/pull/2730 was merged recently and should be delivered with GraalVM 21.0.0

Support for JDK serialization/deserialization https://github.com/oracle/graal/pull/2730 was merged recently and should be delivered with GraalVM 21.0.0

I'll add the integration test when we move to GraalVm 21.0.0

I'll add the integration test when we move to GraalVm 21.0.0

integration test for Query on Map, using a Serializable object.

integration test for Query on Map, using a Serializable object.

@davsclaus I wonder if we should start a stream to investigate a  pluggable strategy for this and other similar cases i.e. allowing to use json, protobuf & what not

@davsclaus I wonder if we should start a stream to investigate a  pluggable strategy for this and other similar cases i.e. allowing to use json, protobuf & what not

The transferExchange is a bad design as it enforces to use Camel on both sides, and gives the impression that you can transfer the current state of the exchange to some cache / storage, and then retrieve it later, and then continue routing it.
Its for historical use we had it, and when Java objects wasn't yet a big big security vulnerability and warning.
Instead what is a more general concept would be to have an API for serializing / de-serializing a Camel message with its body and header (eg user payload) and not to include internal exchange state that may be stored as exchange properties and whatnot.
Such a API / contract can also be used by EIPs and components to store user payload more easily. And then we can have implementations for common types, such as you refer to with json/xml payloads. And for binary then avro/protobuf/what not

The transferExchange is a bad design as it enforces to use Camel on both sides, and gives the impression that you can transfer the current state of the exchange to some cache / storage, and then retrieve it later, and then continue routing it.
Its for historical use we had it, and when Java objects wasn't yet a big big security vulnerability and warning.
Instead what is a more general concept would be to have an API for serializing / de-serializing a Camel message with its body and header (eg user payload) and not to include internal exchange state that may be stored as exchange properties and whatnot.
Such a API / contract can also be used by EIPs and components to store user payload more easily. And then we can have implementations for common types, such as you refer to with json/xml payloads. And for binary then avro/protobuf/what not

Blocked by https://github.com/quarkusio/quarkus/pull/15380


fixed-Project-TeckStack-not-showing

Targets #2816

Description

Changes were made because Project TeckStack not showing the data.
Checklist:

[x] I have performed a self-review of my own code.

Take latest pull from master and push it again


PlugY stashes are lost when converting S7 to S8

Hey,
It seems like all stash pages but the first one are lost when converting this save: Vadmin.zip. It's got plenty of items on the 2nd and 3rd stash pages if you open it in the S7, but only the 1st stash page is available in the converted save.

Oh dang. That was working earlier this season. I used it to convert all my s7 open wounds items and had 3 pages of them in a character's stash which transferred over fine. It seems one of the later updates broke it for d2x files. Anyway, it's working again in the most recent update. Tried it with your files and was able to see multiple stash pages of items.

Thanks a lot!


Getting "The network connection was lost." when uploading image file

Hi!
I'm getting "The network connection was lost." when uploading file. No matter how file size is(tested on 128kb and 2mb). Here are logs:
Opera Build[4820:554763] Task <0C57E5BB-7A5F-4D18-8391-3A6043AEF13E>.<14> HTTP load failed (error code: -1005 [1:57])
Opera Build[4820:554415] Task <0C57E5BB-7A5F-4D18-8391-3A6043AEF13E>.<14> finished with error - code: -1005
Opera Build[4820:554279] {"error":"The network connection was lost.","underlyingErrorCode":-1005,"id":"a50c-0.870c710c88a06"}
Do I need to fix something on server side? Right now nginx is configured to keepalive 65s.
P.s. Using iOS 12.4.

I get the same error, and it looks like the problem is the Content-Length header.

I opened an issue a few days ago because I noticed that we were having problems using multi-part file upload in AngularJS (missing Content-type and boundary). The issue was due to some missing properties that were preventing Object.toString from identifying FormData properly.
It sounds like what you're describing could be caused by that...
You can try the fix I have here and see if solves your issue.
https://github.com/oracle/cordova-plugin-wkwebview-file-xhr/issues/39

I opened an issue a few days ago because I noticed that we were having problems using multi-part file upload in AngularJS (missing Content-type and boundary). The issue was due to some missing properties that were preventing Object.toString from identifying FormData properly.
It sounds like what you're describing could be caused by that...
You can try the fix I have here and see if solves your issue.
#39

I'm not sure if @yoldar is experiencing the same issue as me, but I get "The network connection was lost." even when using raw XHR and not the AngularJS $http service. As best I can tell, the issue is that the Content-Length header is incorrect.

I just updated the reference to this plugin in my app from:
<plugin name="cordova-plugin-wkwebview-file-xhr" spec="~2.1.1" />

to:
<plugin name="cordova-plugin-wkwebview-file-xhr" spec="https://github.com/oracle/cordova-plugin-wkwebview-file-xhr.git" />

With this change, this issue was resolved for me. It appears that there have been a number of changes made here in Git that are not on npm. (The last version on npm is 2 years old, whereas the most recent changes in this repo are from 3 months ago.)

Please verify with latest version of plugin.


remove redundant ILP HPWL constraints

Use the sequence pair information to reduce the number of constraints for HPWL in ILP.
e.g. Net N connecting pins of blocks A and B. if posPair : A < B and negPair A < B, then the left coordinate of N comes from A and right coordinate of N from B.

Use the sequence pair information to reduce the number of constraints for HPWL in ILP. e.g. Net N connecting pins of blocks A and B. if posPair : A < B and negPair A < B, then the left coordinate of N comes from A and right coordinate of N from B.

This is similar to my previous implementation. In this example, the vertical HPWL is hard to determine. I think #858  is better.

Use the sequence pair information to reduce the number of constraints for HPWL in ILP. e.g. Net N connecting pins of blocks A and B. if posPair : A < B and negPair A < B, then the left coordinate of N comes from A and right coordinate of N from B.

This is similar to my previous implementation. In this example, the vertical HPWL is hard to determine. I think #858 is better.

@854768750 The output of this one matches with the one shown in #858. The constraints for ILP in this branch are fewer than #858. But this branch did not providing any runtime improvement. I'll use this branch for experiments.

Tried replacing lp_solve with
(a) GLPK (feature/glpk_ilp): This improves runtime significantly but is incompatible since its under GLP v3.
(b) CBC (feature/cbc_ilp): Benchmarksref indicate this is one of the fastest open-source MILP solvers. But this provides no runtime benefit compared to lpsolve in the default mode. There might be some runtime tuning parameters/cut generators that help our placer.
In this branch, right HPWL is calculated in ILP if the number of blocks is less than 20.
We have to look at the long-running tests in the integration regression to see if those are needed.
e.g. examples/modified_USC_UW_UT_testcases/TEST. This tests has 74 blocks takes ~2 hours with the right HPWL calculation in ILP.


DBT Features List

Implemented features for DBT:

Showing DBT queries on UI
DBT node/table lineage
DBT column level lineage (only possible if query is parsable by lineage parser)
DBT Tests and test results
Adding owner/team from DBT into OpenMetadata
Ingesting table and column level tags from DBT into OpenMetadata
DBT descriptions (toggle is added if descriptions should be overridden from DBT)
DBT Tag Category name
Handle multiple DBT projects for a single service

To be considered Feature List:

Exposures (example in metabase) - Exposures make it possible to define and describe a downstream use of your dbt project, such as in a dashboard, application, or data science pipeline
DBT Lineage when there are intermediate not-materialized models
Add support for test_suite_name meta defined in model schemal.yaml file

Issues/Bug List:

[x] DBT errors if tests are present in manifest but run_results file is not passed
[x] DBT - Issue while adding descriptions

https://github.com/open-metadata/OpenMetadata/issues/9011

column level lineage for DBT projects with macros https://github.com/open-metadata/OpenMetadata/issues/9011
tier sync https://github.com/open-metadata/OpenMetadata/issues/9031
arbitrary key/value sync as extra properties https://github.com/open-metadata/OpenMetadata/issues/9637

https://github.com/open-metadata/OpenMetadata/issues/10199
https://github.com/open-metadata/OpenMetadata/issues/10184
https://github.com/open-metadata/OpenMetadata/issues/8561

@OnkarVO7 can you update the ticket with what are we solved so far and what is remaining. Looking at the issue I am not able to tell if shipped some of these already

@harshach the list mentioned under Implemented section here are resolved. Rest others are remaining

Show dbt fields in the UI https://github.com/open-metadata/OpenMetadata/issues/10502

Synchronizing back to metadata to dbt would be crucial as we import metadata from dbt, I suppose.
https://www.castordoc.com/blog/now-live-sync-back-to-dbt
https://openmetadata.slack.com/archives/C02DZK3QH8Q/p1678973195319499

Background:
English is not my native language. So, my level of English may lead to communication barriers. If there are any unclear expressions, please let me know.
We are only at the documentation stage with OMD, so we cannot provide recommendations from a design perspective, but I can give you my requirements:

Metric sharing.
Basic metric search functionality, similar to that of a data table, where metrics can be tagged and associated with a glossary. Additionally, descriptions of the metric's dimensions, statistical granularity, business limitations, and derived relationships between metrics should be provided to allow users to perceive subtle differences between different metrics.

Metric query agent.
Based on managing the physical tables on which the metrics depend, as well as their dimensions and granularity, the agent can generate query SQL based on user parameters and submit the SQL to the data warehouse to obtain data samples.

It would be best to implement the aforementioned requirements using dbt as the data source.

Reference materials:
https://docs.transform.co/docs/metricflow/metricflow-tutorial
https://docs.getdbt.com/docs/build/metrics

Hi, guys.  i updated new tag in dbt and ingest it into openmetadata again.
but i found that old tag still remained. is it possible to add this feature in enhancement(remove the old metadata automatically)?

Closing the issue and opening separate issues for the remaining features
https://github.com/open-metadata/OpenMetadata/issues/12909
https://github.com/open-metadata/OpenMetadata/issues/12910
https://github.com/open-metadata/OpenMetadata/issues/12911


feat: Implement llm system evaluation

Implementation of llm system evaluation.
Usage example:
If would like to output detailed metric table (row-specific scores), add the following env var export EXPORT_CSV=True
Run the following: python run_evaluation.py

/gcbrun


Autocomplete doesn't work in Firefox

Are you submitting a bug report or a feature request?
This is a bug report
What is the current behavior?
Sandbox: https://codesandbox.io/s/mZRjw05yp
How to reproduce:
0. open Firefox

input first name
click submit
try to input first name again

What is the expected behavior?
autocomplete should work, but in Firefox it doesn't work, in Chrome works.
Sandbox Link
https://codesandbox.io/s/mZRjw05yp
What's your environment?
windows 10, Firefox 59.0.2 (64-bit)

Still not fixed. I have the same problem.

Same thing but with password saving prompt.

Autocomplete is not related with the library, since we don't implemente autocomplete attribute in any part of the code. Please check if your browser doesn't turn off this feature.


Access to global definitions masked when adding fileClasses

Enjoying the plugin a lot, thank you.
I have a Global Variable, X, of date type. When I apply my global fields, I can use the date picker to choose a date, i.e. it is recognized as a date.
If I add a fileClass with its fields (no name overlap) and I try to update the global date field I am prompted to enter any value, i.e. it is no longer recognized as a date.
If I remove the fileClass declaration from the frontmatter the global date is recognized again as a date variable.
Should I move all of my global variables to a "base" fileClass and then use "extends: " syntax? Is there an easier way?
Thanks
PS. What is the future of this plug-in given the upcoming 1.4 release with the "Properties" functionality?

I created a default fileClass and all the other classes extend the default. Works fine.


Radio, checkbox and chips in the same modal

Hi all!
Is it possible to have multi selects? (radio, chips and checkbox) Same as Uber Eats.
Example:

I released https://pub.dev/packages/flutter_awesome_select with fixed null safety.
Could you check that everything works for you?

Hi everyone,
I'm sorry to announce that I'm no longer maintaining the smart_select package. It's been a great project, but it's become too difficult to maintain.
In its place, I've released a new package called choice. The combination to smart_select and chips_choice with cleaner, more flexible, and composable API for creating inline or prompted choice widgets with single or multiple selection.
I hope you'll check out choice. I think you'll find it to be a great replacement for smart_select.
Thanks for your understanding.


Travis: Add 2.4.0, use jruby-<IP_ADDRESS>

This PR

adds 2.4.0 to the build matrix
updates to latest JRuby

Looks like tests hang on JRuby now, presumably because SIGQUIT is not handled as it used to be.

@michaelklishin That message about SIGQUIT has been in JRuby a long time. Should Hutch change its tune about that signal?

@olleolleolle on JRuby only, I think so.

I was able to get the hang on QUIT with "Randomized with seed 60614".

@olleolleolle 👍  let's drop QUIT support on JRuby then.

By the way, we can do it in this very PR.

@michaelklishin I'll see if I can get a fix on the thing, so that we don't... mess things up.

@michaelklishin I began by unwinding a loop, so I could be remove a weirdly-placed next.

@olleolleolle I'm not sure what you mean. But thank you.

@michaelklishin The next move would be to never mention the QUIT signal to JRuby in the implementation of Waiter, so that no warnings are spouted.

Makes sense, thank you!

Now, the test output of WaiterSpec is
Hutch::Waiter
  ["TERM", "INT"]
    includes only things in Signal.list.keys
  .wait_until_signaled
    a TERM signal is received
      logs that hutch is stopping
    a INT signal is received
      logs that hutch is stopping

@michaelklishin Can you kill this build? https://travis-ci.org/gocardless/hutch/jobs/193808578

@michaelklishin I'm more interested in the next build, which included the "skip QUIT on JRuby".

@olleolleolle thank you!

@michaelklishin Happy weekend!


SIM109—potential false positive

Hi there, I have this code this:
import abc

def x(annotation_origin):
    return (
        annotation_origin == list
        or annotation_origin == tuple
        or annotation_origin is abc.Sequence
    )

that gets converted to
import abc

def x(annotation_origin):
    return (
        annotation_origin in (list, tuple)
    )

I'm not 100% sure this is a bug, but the previous code we had is valid. The same behaviour happens with Flake8
Playground link

It looks like a bug to me.

By the way, my “workaround” was:
import abc

def x(annotation_origin):
    return (
        annotation_origin in (list, tuple)
    ) or annotation_origin is abc.Sequence


Change concepts/ to redirect to reference/concepts/ instead of tour/

Having concepts/ redirect to reference/concepts/ is better than tour/.

Minor cleanup of this redirect within the old site, as it'll make the redirect from the old site to new site cleaner as well.


Improve README with a better example and description

Change-type: patch
Signed-off-by: Lucian Buzzo<EMAIL_ADDRESS>

@balena-ci I self-certify!


mdot connect issue

Hello folks i tried to connect my mdot online with the example in following link:
"https://thethingsnetwork.org/labs/story/getting-started-with-the-multitech-mdot/step/arm-mbed#step-anchor"
but the problem is something going wrong when i try to do this example and nothing appears in serial monitoring scene.
"Warning: Incompatible redefinition of macro "__MBED_CMSIS_RTOS_CM" in "mbed-rtos/rtx/TARGET_CORTEX_M/cmsis_os.h", Line: 60, Col: 10
Warning: Incompatible redefinition of macro "__CMSIS_RTOS" in "mbed-rtos/rtx/TARGET_CORTEX_M/cmsis_os.h", Line: 62, Col: 10
Warning: Statement is unreachable in "main.cpp", Line: 57, Col: 6"
Those are the errors i saw when i tried to compile my code  or arm mbed compile.
Is there anyone have any advice to help me?
Thanks in Advice

@eren23 please open a support ticket at http://support.multitech.com

ARM Internal Ref: IOTMORF-158


⚠ Bofrost.es has degraded performance

In 7fa9af4, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1642 ms

Resolved: Bofrost.es performance has improved in 94f07b0 after 22 minutes.


Deleting remainder of banner...

Change-type: patch

@balena-ci rebase

@balena-ci I self-certify!


Error in building ML-containers with L4T R32.7.2

Hi,
I'm trying to build ml-containers using ./scripts/docker_build_ml.sh all and have R32.7.2 in my Jetson Nano.
I added the line BASE_IMAGE="nvcr.io/nvidia/l4t-base:r32.7.1" in jetson-containers/scripts/docker_base.sh as you suggested.  I managed to proceed to Step 14/30 in order to build the ML-containers. However, in this step I get the following error:
`Step 14/30 : RUN git clone https://github.com/pytorch/vision torchvision &&     cd torchvision &&     git checkout ${TORCHVISION_VERSION} &&     python3 setup.py install &&     cd ../ &&     rm -rf torchvision
---> Running in d1a19ebfadc7
Cloning into 'torchvision'...
Note: checking out 'v0.10.0'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b
HEAD is now at 300a8a401 backport fixes to build/push docs on tags (#4060)
Traceback (most recent call last):
File "setup.py", line 12, in
import torch
File "/usr/local/lib/python3.6/dist-packages/torch/init.py", line 196, in
_load_global_deps()
File "/usr/local/lib/python3.6/dist-packages/torch/init.py", line 149, in _load_global_deps
ctypes.CDLL(lib_path, mode=ctypes.RTLD_GLOBAL)
File "/usr/lib/python3.6/ctypes/init.py", line 348, in init
self._handle = _dlopen(self._name, mode)
OSError: libcurand.so.10: cannot open shared object file: No such file or directory
The command '/bin/sh -c git clone https://github.com/pytorch/vision torchvision &&     cd torchvision &&     git checkout ${TORCHVISION_VERSION} &&     python3 setup.py install &&     cd ../ &&     rm -rf torchvision' returned a non-zero code: 1
`
I don't know how to proceed from here. Could you help me with this?
Here's nv_tegra_release:
cat /etc/nv_tegra_release R32 (release), REVISION: 7.2, GCID: 30192233, BOARD: t210ref, EABI: aarch64, DATE: Wed Apr 20 21:34:48 UTC 2022
Originally posted by @jarvinenemma in https://github.com/dusty-nv/jetson-containers/issues/177#issuecomment-1291703942

Resolved the error using https://github.com/dusty-nv/jetson-containers/issues/108#issuecomment-995090398
Remember to restart docker after performing the aforementioned steps!
sudo systemctl daemon-reload
sudo systemctl restart docker
I'm trying to build the ML-containers again, and the error in Step 14 disappeared. I'll update this thread after the build completes.

Hi @jarvinenemma, can you try adding a RUN pip3 install --verbose scikit-build before in the dockerfile?

Thanks for the tip. I'll try that later today.

Hi, im getting the same issue now.
ive tried this which still didnt resolve it.

Hi @jarvinenemma, can you try adding a RUN pip3 install --verbose scikit-build before in the dockerfile?

Will try to reinstall docker as mentioned here and will let you know if that resolves it

Resolved the error using #108 (comment)
Remember to restart docker after performing the aforementioned steps! sudo systemctl daemon-reload sudo systemctl restart docker
I'm trying to build the ML-containers again, and the error in Step 14 disappeared. I'll update this thread after the build completes.

Ive tried the solution above with still no success.
I already had the latest docker installed and nvidia is my default runtime and ive restarted
Please help

If you already tried adding RUN pip3 install --verbose scikit-build earlier in the dockerfile (before cmake gets upgraded), then I would recommend uncommenting this block of code:
https://github.com/dusty-nv/jetson-containers/blob/eb2307d40f0884d66310e9ac34633a4c5ef2e083/Dockerfile.pytorch#L108
and then commenting out this line:
https://github.com/dusty-nv/jetson-containers/blob/eb2307d40f0884d66310e9ac34633a4c5ef2e083/Dockerfile.pytorch#L118

Im failing here, before even getting to cmake

https://github.com/dusty-nv/jetson-containers/blob/eb2307d40f0884d66310e9ac34633a4c5ef2e083/Dockerfile.pytorch#L79

My Issue is slightly different then the above issue ive realised, I get a segmentation fault.
Cloning into 'torchvision'...
Note: checking out 'v0.10.0'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 300a8a4013 backport fixes to build/push docs on tags (#4060)
Segmentation fault (core dumped)

When I look at my usage at the time im not at 60% memory usage

@CodiTrixta if you check the dmesg kernel log after this occurs, does it mention anything about being out of memory (OOM) or otherwise?  If so, you may want to mount additional swap memory.

i'm using this BASE_IMAGE="nvcr.io/nvidia/l4t-base:r32.7.1. By running the following command ./scripts/docker_build_ml.sh pytorch i received this error in step 9: ARCH:  aarch64
reading L4T version from /etc/nv_tegra_release
L4T BSP Version:  L4T R32.7.2
L4T Base Image:   nvcr.io/nvidia/l4t-base:r32.7.1
selecting OpenCV for L4T R32.7.2...
OPENCV_URL=https://nvidia.box.com/shared/static/5v89u6g5rb62fpz4lh0rz531ajo2t5ef.gz
OPENCV_DEB=OpenCV-4.5.0-aarch64.tar.gz
Python3 version:  3.6
building PyTorch torch-1.9.0-cp36-cp36m-linux_aarch64.whl, torchvision v0.10.0, torchaudio v0.9.0, cuda arch 5.3;6.2;7.2
Building l4t-pytorch:r32.7.2-pth1.9-py3 container...
[sudo] password for velox:
Sending build context to Docker daemon  179.7kB
Step 1/30 : ARG BASE_IMAGE=nvcr.io/nvidia/l4t-base:r32.4.4
Step 2/30 : FROM ${BASE_IMAGE}
---> 7a58a57ab551
Step 3/30 : ENV DEBIAN_FRONTEND=noninteractive
---> Using cache
---> 61db3c30c3d1
Step 4/30 : RUN apt-get update &&     apt-get install -y --no-install-recommends             python3-pip 		  python3-dev 		  libopenblas-dev 		  libopenmpi-dev             openmpi-bin             openmpi-common 		  gfortran 		  libomp-dev     && rm -rf /var/lib/apt/lists/*     && apt-get clean
---> Using cache
---> fc4afd7976b9
Step 5/30 : RUN pip3 install --no-cache-dir setuptools Cython wheel
---> Using cache
---> decb5dbf1045
Step 6/30 : RUN pip3 install --no-cache-dir --verbose numpy
---> Using cache
---> abfe5fb114e4
Step 7/30 : ARG PYTORCH_URL=https://nvidia.box.com/shared/static/lufbgr3xu2uha40cs9ryq1zn4kxsnogl.whl
---> Using cache
---> 2648c66700b5
Step 8/30 : ARG PYTORCH_WHL=torch-1.2.0-cp36-cp36m-linux_aarch64.whl
---> Using cache
---> 0b1cc0d07682
Step 9/30 : RUN wget --quiet --show-progress --progress=bar:force:noscroll --no-check-certificate ${PYTORCH_URL} -O ${PYTORCH_WHL} &&     pip3 install --no-cache-dir --verbose ${PYTORCH_WHL} &&     rm ${PYTORCH_WHL}
---> Running in 93f82c7f42a4
failed to create shim: OCI runtime create failed: runc create failed: unable to start container process: exec: "/bin/sh": stat /bin/sh: no such file or directory: unknown

I also had a similar problem by executing the following command.
./scripts/docker_build_ml.sh tensorflow :
Testsuite summary for Protocol Buffers 3.19.4
TOTAL: 7
PASS:  6
SKIP:  0
XFAIL: 0
FAIL:  1
XPASS: 0
ERROR: 0
The command '/bin/sh -c cd /tmp &&     wget --quiet --show-progress --progress=bar:force:noscroll --no-check-certificate ${PROTOBUF_URL}/$PROTOBUF_DIR.zip &&     wget --quiet --show-progress --progress=bar:force:noscroll --no-check-certificate ${PROTOBUF_URL}/$PROTOC_DIR.zip &&     unzip ${PROTOBUF_DIR}.zip -d ${PROTOBUF_DIR} &&     unzip ${PROTOC_DIR}.zip -d ${PROTOC_DIR} &&     cp ${PROTOC_DIR}/bin/protoc /usr/local/bin/protoc &&     cd ${PROTOBUF_DIR}/protobuf-${PROTOBUF_VERSION} &&     ./autogen.sh &&     ./configure --prefix=/usr/local &&     make -j$(nproc) &&     make check -j4 &&     make install &&     ldconfig &&     cd python &&     python3 setup.py build --cpp_implementation &&     python3 setup.py test --cpp_implementation &&     python3 setup.py bdist_wheel --cpp_implementation &&     cp dist/.whl /opt &&     pip3 install dist/.whl &&     cd ../../../ &&     rm ${PROTOBUF_DIR}.zip &&     rm ${PROTOC_DIR}.zip &&     rm -rf ${PROTOBUF_DIR} &&     rm -rf ${PROTOC_DIR}' returned a non-zero code: 2

failed to create shim: OCI runtime create failed: runc create failed: unable to start container process: exec: "/bin/sh": stat /bin/sh: no such file or directory: unknown

@Feki-Tech as a test, are you able to run the nvcr.io/nvidia/l4t-base:r32.7.1 container with GPU enabled?  Like this:
scripts/docker_run.sh -c nvcr.io/nvidia/l4t-base:r32.7.1

@CodiTrixta if you check the dmesg kernel log after this occurs, does it mention anything about being out of memory (OOM) or otherwise? If so, you may want to mount additional swap memory.

Hi Dusty, sorry for not replying, yea adding additional swap memory would of helped but I ended up just building everything that I needed locally and not using docker. Funny enough I did add swap memory anyways as I needed to use more

failed to create shim: OCI runtime create failed: runc create failed: unable to start container process: exec: "/bin/sh": stat /bin/sh: no such file or directory: unknown

@Feki-Tech as a test, are you able to run the nvcr.io/nvidia/l4t-base:r32.7.1 container with GPU enabled? Like this:
scripts/docker_run.sh -c nvcr.io/nvidia/l4t-base:r32.7.1

this working without any problems


Support for HCL

Describe the bug
Feature request to have HCL supported as a language option for things like Terraform snippets
To Reproduce
N/A
Expected behavior
Highlighting for HCL examples such as:
terraform {
  required_version = "0.12.25"

  backend "s3" {
    acl     = "private"
    encrypt = true
    key     = "terraform.tfstate"
  }
}

Screenshots
N/A
Info (please complete the following information):
N/A

@dannyleesmith do you know if there is a CodeMirror mode for Terraform/HCL?

@mfix22 wasn't aware of the term but I think if it was it would appear at this link and it does not: https://codemirror.net/mode/

Please reopen this if anyones finds an HCL/Terraform mode for CodeMirror


Remove gemm size checkes from ConvMlirIgemmBwd

conv_mlir_igemm_bwd_xdlops.cpp doesn't have the same checks. It does call IsValidXdlopsGemm(). Do you mean IsValidXdlopsGemm() should be removed?

@yiqian1 Correct, any non-trivial checks in any solver needs to be removed and make its way into MLIR. Right now there is all except the two bwd solvers that have those checks.

Got it. I'll work on it.

Converted to draft.

jenkins retest this please

jenkins: retest this please

How this affects performance of IsApplicable()? It must be pretty fast, otherwise the performance of the library will suffer. Measurements? I can recommend collecting logs with export MIOPEN_ENABLE_LOGGING_ELAPSED_TIME=1 and comparing them.
Or you can temporarily instrument code with with MIOPEN_LOG_SCOPE_TIME, see #622.

@yiqian1

yiqian1 force-pushed the cleanupMLIRBwdSolver branch from 3625018 to 37fef14 23 minutes ago

This is NOT recommended due to several reasons. Force-pushing can be more or less safely used only if the branch has been published for review yet.

@yiqian1

jenkins: retest this please

Hmm. Jenkins automatically begins a new job after branch is updated and pushed to the repo.
In order to restart the job after accidental failure (without branch update), you need to visit micimaster.amd.com, login and click here:

How this affects performance of IsApplicable()? It must be pretty fast, otherwise the performance of the library will suffer. Measurements? I can recommend collecting logs with export MIOPEN_ENABLE_LOGGING_ELAPSED_TIME=1 and comparing them.

The changes are removing code from IsApplicable(), so they would make IsApplicable() faster, though the real impact of the change might be too trivial to have any meaningful impact on the performance of kernels. For example, I tested one config:
The original code gave:
MIOpen Backward Data Conv. Algorithm: 5, Solution: 99/ConvMlirIgemmBwd
GPU Kernel Time Backward Data Conv. Elapsed: 2.264169 ms (average)
stats: name, n, c, ho, wo, x, y, k, flopCnt, bytesRead, bytesWritten, GFLOPs, GB/s, timeMs
stats: bwdd-conv1x1u2, 256, 128, 1, 1, 2048, 7, 7,<PHONE_NUMBER>, 269484032, 102760448, 2905, 164, 2.264169
Backward Convolution Data Verifies OK on GPU reference (9.19717e-08)
The new code gave:
MIOpen Backward Data Conv. Algorithm: 5, Solution: 99/ConvMlirIgemmBwd
GPU Kernel Time Backward Data Conv. Elapsed: 2.260250 ms (average)
stats: name, n, c, ho, wo, x, y, k, flopCnt, bytesRead, bytesWritten, GFLOPs, GB/s, timeMs
stats: bwdd-conv1x1u2, 256, 128, 1, 1, 2048, 7, 7,<PHONE_NUMBER>, 269484032, 102760448, 2910, 165, 2.260250
Backward Convolution Data Verifies OK on GPU reference (9.19717e-08)
I don't think it's necessary to collect and compare the performance for all test cases, as it's obvious that there won't be negative impact on performance. However if it is the standard way for merging, I will definitely follow it. Please let me know.

@yiqian1

The changes are removing code from IsApplicable(), so they would make IsApplicable() faster, though the real impact of the change might be too trivial to have any meaningful impact on the performance of kernels.

This is not so.

Before this change, the IsApplicable was able to return false without need to call MiirIsConfigApplicable(). How MIOpen does not have this shortcut.
This change includes update of MLIR_COMMIT. If I understand correctly, the updated MLIR performs some additional job in MiirIsConfigApplicable().

For example, I tested one config:...

Please note that the logs do not include information I've asked for. I need to see how the performance of IsApplicable() changed, taken in isolation.

The checks that were removed from IsApplicable() were added in MiirIsConfigApplicable(). So instead of returning false from IsApplicable(), now MiirIsConfigApplicable() would return false. These changes are conceptually same as PR#1026. Let me try to get the logs of the isolated performance of IsApplicable().

Here is what I got.
These changes would degrade the performance of IsApplicable() by 20x for an invalid configuration. For example, I tested an invalid configuration which would fail the checks that were removed by this PR. The original IsApplicable() takes 12us due to the short-cut check. With the changes, IsApplicable() takes 2397us, as it needs to call the library.
For applicable configurations (I used the configurations in test_conv_igemm_mlir, and each config ran twice) on gfx900,  there was no significant performance change of IsApplicable(). Without the changes, the average time of IsApplicable() is 941us. With the changes, the average time is 845us.

@yiqian1 To add in a few background information: Very soon when the padding kernel becomes available, we'd no longer need to call into the API of miirLowerTuningParams(), but instead only leave the handle creation in the applicability checks. I believe 90% of time in that 2397 us is because of the tuning parameter lowering, which is known to be more expensive. Would you be able to deduct the time of the tuning paramter lowering API call and see the fair comparison in this context?

I believe 90% of time in that 2397 us is because of the tuning parameter lowering, which is known to be more expensive. Would you be able to deduct the time of the tuning paramter lowering API call and see the fair comparison in this context?

Yes. My experiment shows miirLowerTuningParams() takes 60% execution time of IsApplicable().

@atamazov Please let me know if you think the performance of IsApplicable() is acceptable. Jerry and I had a discussion and believe we can improve the performance of IsApplicable(). This needs some investigation and will be in another PR.

@atamazov Please let me know if you think the performance of IsApplicable() is acceptable.

Unfortunately, not. IsApplicable is used in many contexts. It must be really really fast.

Jerry and I had a discussion and believe we can improve the performance of IsApplicable(). This needs some investigation and will be in another PR.

We can merge this as is, provided that a performance regression ticket is created in this repo. If you want to go that way, then please go ahead and create a ticket, and let me know.


Add plot export button that closes the dialog

But keep a button for apply style export
Add error message if exported file is missing
Resolves #1676

Hmm that sounds really annoying, I will check if some flag might be helpful. The problem is likely that the dialog signals the plottab to open the export file select dialog. This dialog will be a sibling to the export dialog itself. Needs redesign but that will have to wait until some other time.

Could not reproduce in Kubuntu but set export dialog as parent to file dialog (instead of null). Will likely help.


Add github.com/bufbuild/buf

bufbuild/buf has already been supported.
https://github.com/aquaproj/aqua-registry/blob/main/pkgs/bufbuild/buf/registry.yaml


regression: fork in test doesn't work anymore

steps
fork in test := true

problem
Test does not fork
expectations
Test forks.
notes
fork in Test works. Also see Enable forking:

To enable forking all test tasks only, set fork to true in the test scope:
fork in test := true

Closing this since we likely won't fix this.


Consider using releases

Using GitHub releases makes it easier to download specific "blessed" versions of the source than it is checking out commits.

I agree, I haven't done much development on this recently, but I'll consider using the github releases next time.


[App Service] az functionapp create: Remove unnecessary app settings for flex function apps

Related command
az functionapp create
Description

The backend logic has blocked the creation of flex function apps with the FUNCTIONS_EXTENSION_VERSION app setting. Therefore, we need to make sure that this app setting is not added in the creation flow.
Additionally, we should not be setting distributed tracing on by default for flex function apps.

Testing Guide

az functionapp create for a flex app no longer adds the FUNCTIONS_EXTENSION_VERSION app setting
az functionapp create for a java flex app no longer adds the APPLICATIONINSIGHTS_ENABLE_AGENT app setting

History Notes

[AppService] az functionapp create: Remove FUNCTIONS_EXTENSION_VERSION app setting for flex function apps
[AppService] az functionapp create: Remove APPLICATIONINSIGHTS_ENABLE_AGENT app setting for flex function apps
[AppService] az functionapp create: Remove AzureWebJobsDashboard app setting for all function app SKUs
[AppService] az functionapp create and az functionapp list-flexconsumption-locations: Make request to geo regions API
[AppService] az functionapp create and az functionapp vnet-integration add: Register subscription to Microsoft.App RP for flex function app vnet integration

This checklist is used to make sure that common guidelines for a pull request are followed.

[x] The PR title and description has followed the guideline in Submitting Pull Requests.

[x] I adhere to the Command Guidelines.

[x] I adhere to the Error Handling Guidelines.

AppService

[AppService] az functionapp create: Remove FUNCTIONS_EXTENSION_VERSION app setting for flex function apps
[AppService] az functionapp create: Remove APPLICATIONINSIGHTS_ENABLE_AGENT app setting for flex function apps
[AppService] az functionapp create: Remove AzureWebJobsDashboard app setting for all function app SKUs

I would like to confirm if these changes will cause a breaking change for customers?

[AppService] az functionapp create: Remove FUNCTIONS_EXTENSION_VERSION app setting for flex function apps
[AppService] az functionapp create: Remove APPLICATIONINSIGHTS_ENABLE_AGENT app setting for flex function apps
[AppService] az functionapp create: Remove AzureWebJobsDashboard app setting for all function app SKUs

I would like to confirm if these changes will cause a breaking change for customers?

These changes would not cause a breaking change for customers.


Feature: Documentation around @lexical/code

Would it be possible to have more documentation around @lexical/code? I'm very interested to understand how this is designed to be used, and I'm inferring usage based on the code at the moment.

Hi @rememberlenny,
As of now that package exports registerCodeHighlighting that can be used to mount its features to the editor. If you're using react, then here's an example from our playground on how to call it from plugin. There's some ongoing work to split registerCodeHighlighting into code highlighting and code typing enhancements (#2382), after that we'll update docs to reflect new structure

Covered here: https://github.com/facebook/lexical/issues/2845


[ai-form-recognizer] Expose 2.1-preview.3 API Features

This branch supports the new 2.1-preview.3 service API, which will be released as SDK version 3.1.0-beta.3. There are a few changes to the API surface:

Added support for identity document recognition

[x] mentioned in the README
[x] added sample programs
[x] added tests

new form field value types: "gender" and "country"
all recognition endpoints now support pages (layout recognition already did)
layout recognition now supports readingOrder, which describes the algorithm used to sort lines of text for recognition on the service. It has two values: "basic" (default, uses a simple top-to-bottom right-to-left sort) and "natural" (tries to emulate the way a human would read the page)
custom model recognition supports bitmap images

In addition, I made a few cleanliness improvements to the code:

Used KnownValue enums (this is a JS convention) where appropriate in the tests
Added a basic exhaustiveness check to FieldValue transforms (to make sure that we never forget to cover "phoneNumber" or "gender", etc.)
Relaxed the constraints for expectedFormType on prebuilts, so the validator will allow a model that begins with the expected prefix instead of requiring it to match exactly (identity document support requires this, as the identity document model can possibly return more than one unique form type, all beginning with prebuilt:idDocument).

I've also made minor changes to the tests in addition to what's mentioned above in order to re-enable codepaths that were disabled due to known service regressions in 2.1-preview.2. All tests were re-recorded to make them work with 2.1-preview.3.


Add network manager support and remove dependency on netifd manager

Signed-off-by: aparna.salian<EMAIL_ADDRESS>

Hi,
thanks for the patch, there are a few formal errors and if I am reading the code correctly, merging this PR will cause a regression and make ApNos behave differently.
*) please make sure that the patch is properly annotated. the prefix and secription are mssing, the SoB is incorrectly formatted
*) when merging the patch, the ApNos images would be built with NM2 enabled as opposed to the netifd manager.
John


Remove dea_logging_agent concerns from loggregator

The logger package is used only in the dea_logging_agent, which vendors
it. This commit removes logger from loggregator and instead pulls in an
updated dea_logging_agent submodule which includes it.
This commit also moves the integration tests for the dea agent into the
dea_logging_agent package.
This PR resolves the test failures in a previous attempt.

@jasonkeene The previous PR was caught up by a lingering integration test around the dea_logging_agent which should have been a part of that submodule. I've moved that last test to where it belongs, and now all loggregator tests are green on my machine. Sorry for the trouble.

Note this PR shouldn't be merged until we update the spec for Bosh here.

I've made a PR here to resolve the package problems and tested this on a bosh-lite. This PR should be ready to merge in unison with the work in dea_logging_agent.
/cc @jasonkeene


Run functional tests in AppVeyor

Issue #, if available:
Description of changes:
Checklist:

[ ] Write Design Document (Do I need to write a design document?)
[ ] Write unit tests
[ ] Write/update functional tests
[ ] Write/update integration tests
[ ] make pr passes
[ ] Write documentation

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Several functional tests fail. WE need to investigate them separately. Closing this PR


[GUI] Notifications: API not found

Affected Version: 58.1-20190321

Test OK.

Tested on Version: 62.3-20190603


Check that the ipv4 address is actually v4

This fixes a bug where the autojoin API would fail creating the right hostname and DNS records when the registration request comes over IPv6, treating it as an IPv4. e.g.:
2024-08-21 21:04:49 autojoin[20240821t195345]  /workspace/handler/handler.go:227: dns register failure: googleapi: Error 400: Invalid value for 'entity.change.additions[ndt-oma0-.mlab.autojoin.measurement-lab.org.][A].rrdata[0]': '2600:1900:4000:7927::', invalid
Possible scenarios:

The request comes over IPv4, without ipv4 in the querystring: the client's IP address is only accepted if it's a valid v4 address
The request comes over IPv4, with ipv4 in the querystring: the ipv4 in the querystring takes precedence. For example, the default route on the node might not use the same interface the host wants to use to serve NDT
The request comes over IPv6, without ipv4 in the querystring: this fails, since there is no way for Autojoin API to know the node's IPv4
The request comes over IPv6, with ipv4 in the querystring: the ipv4 in the querystring is used, no ipv6 is set. Noting that this makes it mandatory for the host to provide an ipv6 if they want to serve tests over ipv6.

This change is

Pull Request Test Coverage Report for Build<PHONE_NUMBER>4
Details

2 of 2 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 98.212%

Totals

Change from base Build<PHONE_NUMBER>0:
0.0%

Covered Lines:
714

Relevant Lines:
727

💛  - Coveralls


Mobile

Checklist

[x] commit
[x] push
[x] pull
[x] status
[x] error handling
[x] validation check
[x] Repository init
[x] settings pane with configurations
[x] disable status bar on mobile
[x] port simple-git to new plugin structure
[x] documentation
[ ] community security check (I want to check the secure use of the personal access-token)

I want install plugin version on my phone from this branch for tests, how can i do this?
Build and replace files main.js and manifest.json at .obsidian folder on my android smartphone not working for me.

@hardenchant You need to remove any node imports. So you have to delete the simple git file. Just ignore the import errors now. I think there is also a child process import in main.ts. this import needs to be deleted too. A bit tricky I know. The only import/require at the top of main.js should be obsidian.

Is there anything we can do to help with this? I am looking forward to this being implemented; this plugin is awesome!!

See merged #270


Sam at luther/use buf

This PR should be reviewed after https://github.com/luthersystems/sandbox/pull/42 is merged.


fixes: Template can't be imported into Zabbix 5.4.3

Hi, the Template can't be imported into Zabbix 5.4.3. :-/ It seems that the “Template Module ICMP Ping (5.x)” (which the Template is linking to) was renamed into “ICMP Ping” there. This patch fixes this and works fine for me.
s3h10r@milliways zabbix-qnap-template % diff zabbix-5.x/template.xml zabbix-5.4.3/template.xml
16c16
<                     <name>Template Module ICMP Ping</name>
---
>                     <name>ICMP Ping</name>

Greetings, Sven

Thanks!
Correct, they seem to have renamed the default template name yet again. 😄


chore(release): v3.13.3

Issue being fixed or feature implemented
This PR update wallet-lib to 7.13.2 which brings multiples fixes and an additional features.
Especially it adds fix for identity listing and therefore fix a usage issue on registering documents.
As well as an initial sync up fix so we do ensure to sync ALL addresses immediately before releasing (which in the previous build were done later during the app execution).
What was done?

chore: update wallet-lib to 7.13.2

How Has This Been Tested?

Breaking Changes

Checklist:

[X] I have performed a self-review of my own code
[X] I have commented my code, particularly in hard-to-understand areas
[X] I have added or updated relevant unit/integration/functional/e2e tests
[X] I have made corresponding changes to the documentation

Additional fix incoming, adding new tags in the meantime.

Rebased on top of master


[BUG] SKRegion new/dispose memory leak

Description
SKRegion constructor leaks memory. The memory is not reclaimed after calling SKRegion.Dispose().
Code
const int n = 1000000;

System.GC.GetTotalMemory(true);
long mem_start = System.Diagnostics.Process.GetCurrentProcess().PrivateMemorySize64;

for (var idx = 0; idx < n; idx++)
{
    using (var r = new SkiaSharp.SKRegion())
    {
        //using r or not doesn't seem to matter
    }
}

//does not show leak, leaked memory is unmanaged
long mem_end_manged = System.GC.GetTotalMemory(true);

long mem_end = System.Diagnostics.Process.GetCurrentProcess().PrivateMemorySize64;

Console.WriteLine(string.Format("N: {0}", n));
Console.WriteLine(string.Format("Memory total (managed): {0} kb", mem_end_manged / 1024d));
Console.WriteLine(string.Format("Memory total: {0} kb", mem_end / 1024d));
Console.WriteLine(string.Format("Memory delta: {0} b/iter", ((double)mem_end - mem_start) / n));
Console.WriteLine(string.Format("Memory delta: {0} kb", ((double)mem_end - mem_start)/1024));

Outputs:
N: 1000000
Memory total (managed): 1617.04296875 kb
Memory total: 110620 kb
Memory delta: 35.65568 b/iter
Memory delta: 34820 kb

With increased N:
N: 40000000
Memory total (managed): 1590.5 kb
Memory total: 1342928 kb
Memory delta: 33.3193216 b/iter
Memory delta: 1301536 kb

With decreased N:
N: 10000
Memory total (managed): 1619.0625 kb
Memory total: 433892 kb
Memory delta: 26.2144 b/iter
Memory delta: 256 kb

Expected Behavior
Memory delta to be zero or at least small, and not increase with increasing n.
Actual Behavior
Memory delta increases without bound with increasing n.
Basic Information

Version with issue: Skiasharp <IP_ADDRESS>
IDE:  Visual Studio 2013
Platform Target Frameworks:  Win10  .net framework 4.5 (32bit)

Thanks for reporting this, I will investigate.

I am having a look at this now, and I can't find any leaks. I tried a massive amount of instances (50M) and there was basically a 5mb increment. But that was the same for all number 1M, 10M, 50M. I also got that for SKMemoryStream and SKData.
.NET Core 3.1
N: 50,000,000
Memory delta (managed): 608.51 kb
Memory total (managed): 681.22 kb
Memory total: 12,576.00 kb
Memory delta: 0.10 b/iter
Memory delta: 5,052.00 kb

.NET Framework 4.7.1:
N: 50,000,000
Memory delta (managed): 654.24 kb
Memory total (managed): 675.89 kb
Memory total: 14,012.00 kb
Memory delta: 0.09 b/iter
Memory delta: 4,548.00 kb

I am able to reproduce your results with <IP_ADDRESS> (no leak) and have re-checked <IP_ADDRESS> and confirmed it does have the leak.
Using this repro code (updated to include version info):
static void Main(string[] args)
{

const int n = 10000000;

System.GC.GetTotalMemory(true);
long mem_start = System.Diagnostics.Process.GetCurrentProcess().PrivateMemorySize64;

for (var idx = 0; idx < n; idx++)
{
    using (var r = new SkiaSharp.SKRegion())
    {
        //using r or not doesn't seem to matter
    }
}

long mem_end_manged = System.GC.GetTotalMemory(true); //does not show leak, leaked memory is unmanaged
long mem_end = System.Diagnostics.Process.GetCurrentProcess().PrivateMemorySize64;

Console.WriteLine(typeof(SkiaSharp.SKRegion).Assembly.GetName().ToString());
Console.WriteLine(typeof(SkiaSharp.SKRegion).Assembly.Location);
Console.Write(System.Diagnostics.FileVersionInfo.GetVersionInfo(typeof(SkiaSharp.SKRegion).Assembly.Location));
Console.WriteLine(string.Format("N: {0}", n));
Console.WriteLine(string.Format("Memory total (managed): {0} kb", mem_end_manged / 1024d));
Console.WriteLine(string.Format("Memory total: {0} kb", mem_end / 1024d));
Console.WriteLine(string.Format("Memory delta: {0} b/iter", ((double)mem_end - mem_start) / n));
Console.WriteLine(string.Format("Memory delta: {0} kb", ((double)mem_end - mem_start) / 1024));
Console.ReadKey();
}

Installed older broken version via the nuget package console:
Install-Package SkiaSharp -Version <IP_ADDRESS>
SkiaSharp, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=0738eb9f132ed756
C:\vbproj\sksharp_leak_repro\ConsoleApplication1\bin\Debug\SkiaSharp.dll
File:             C:\vbproj\sksharp_leak_repro\ConsoleApplication1\bin\Debug\SkiaSharp.dll
InternalName:     SkiaSharp.dll
OriginalFilename: SkiaSharp.dll
FileVersion:      <IP_ADDRESS>
FileDescription:  SkiaSharp
Product:          SkiaSharp
ProductVersion:   <IP_ADDRESS>-92a3ad87cc24727048d6f80810d109fbca0b3a51
Debug:            False
Patched:          False
PreRelease:       False
PrivateBuild:     False
SpecialBuild:     False
Language:         Language Neutral
N: 10000000
Memory total (managed): 582.5625 kb
Memory total: 344248 kb
Memory delta: 33.5179776 b/iter
Memory delta: 327324 kb

Installed new working version via the nuget package console:
Install-Package SkiaSharp -Version <IP_ADDRESS>
SkiaSharp, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=0738eb9f132ed756
C:\vbproj\sksharp_leak_repro\ConsoleApplication1\bin\Debug\SkiaSharp.dll
File:             C:\vbproj\sksharp_leak_repro\ConsoleApplication1\bin\Debug\SkiaSharp.dll
InternalName:     SkiaSharp.dll
OriginalFilename: SkiaSharp.dll
FileVersion:      <IP_ADDRESS>
FileDescription:  SkiaSharp
Product:          SkiaSharp
ProductVersion:   <IP_ADDRESS>-7fe55eb3ee6e2dfad48a03851314de8420109dce
Debug:            False
Patched:          False
PreRelease:       False
PrivateBuild:     False
SpecialBuild:     False
Language:         Language Neutral
N: 10000000
Memory total (managed):<PHONE_NUMBER>5 kb
Memory total: 19008 kb
Memory delta: 0.2174976 b/iter
Memory delta: 2124 kb

The assembly version reported by Assembly.GetName().ToString() is confusing, but it appears that it is actually running a different skiasharp assembly.
I can switch between the broken and fixed cases at will using the nuget commands.
I suppose its fixed in the new version, although it would be nice to know what was broken.

The old version had a leak with 'SKObject' due to a dictionary holding up to objects forever. This is no longer the case and therefore you don't have a leak anymore.


Auto focus on pane create

I'd like to request (or at least get some pointers on contributing to) a setting that will auto focus when creating a new pane (so, create pane to the right, then auto focus there). Something likeorigami_create_auto_focus_on_create.

Yeah, you could add the setting and if it's true move to a pane after you create it. It should be reasonably straightforward. How can I help you?

By default, create_new_pane_if_necessary is set to true, so you can just travel in a direction without a pane and it'll automatically make a pane and move you there.


Sub-minute scheduling does not work

Describe the bug
Having a schedule of every 55 seconds results in this execution plan:
2021-10-21T14:53:55.945Z
2021-10-21T14:54:00.945Z
2021-10-21T14:54:55.945Z
2021-10-21T14:55:00.945Z
2021-10-21T14:55:55.945Z
2021-10-21T14:56:00.945Z
2021-10-21T14:56:55.945Z
2021-10-21T14:57:00.945Z
2021-10-21T14:57:55.945Z
2021-10-21T14:58:00.945Z

Which is not what's expected.
This is however happening due to how the underlying scheduling library works (discussion here).
For now this will not be fixed in Butler CW. A proper/better implementation may be added in future versions of Butler CW.

Update doc file wrt to the above!


Clean up certain tests containing multiple assertions in one function

Avoid tests like these: test_worker_request_resolution_paths

I think this was done


System spec 006 sometimes fails

System spec 006 sometimes fails.
How to reproduce is unclear.
The failing expectation is line 45 searching the tooltip tags.

Depending on the faker-generated names.
With debugging output, sometimes family name phonetic can have a single quotation mark (').
e.g. O'Kon
This can break the HTML attributes.

fixed 6ce8673bd95816a21edd3a4ef535ac7e2e9e55f9


spring-cloud-config-monitor Gitee webhook not work

hi,
This is Gitee webhook request header (use /config because my gateway config)
Request URL: http://xxx.xxx.xxx.xxx/config/monitor?path=*
Request Method: POST
X-Gitee-Token:
X-Gitee-Event: push_hooks
User-Agent: git-oschina-hook
X-Gitee-Timestamp:<PHONE_NUMBER>493
X-Gitee-Ping: true
Content-Type: application/json
X-Git-Oschina-Event: push_hooks

This is GiteePropertyPathNotificationExtractor.java
latest version
private static final String HEADERS_KEY = "x-git-oschina-event";

private static final String HEADERS_VALUE = "Push Hook";

@Override
protected boolean requestBelongsToGitRepoManager(MultiValueMap<String, String> headers) {
return HEADERS_VALUE.equals(headers.getFirst(HEADERS_KEY));
}

my version (2.2.2.RELEASE)
protected boolean requestBelongsToGitRepoManager(MultiValueMap<String, String> headers) {
    return "Push Hook".equals(headers.getFirst("x-git-oschina-event"));
}

maybe here's the problem
I think the Push Hook maybe should change to push_hooks
sorry, my english is low.
thanks

It is hard for me to tell since the documentation is in Chinese, can you point me to any Gitee documentation that shows the headers in the request.
Also if you are using versions 2.2.2.RELEASE you need to upgrade, its no longer supported

I can't find the detail document for gitee webhook,  only this
https://help.gitee.com/webhook/gitee-webhook-push-data-format

I assume you opened this issue?
https://gitee.com/oschina/git-osc/issues/I8U3K7?from=project-issue
It would be great if they respond to that an clarify what the header value should be.

Yep

Please let me know if you hear back from them

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.


[BUG] Canvas reset after brush color/thickness / Reset button Bug / Increase height bug

Describe the bug

The canvas is getting reset after the brush color/thickness is changed
The reset button isn't working as intended not resetting the background color
When increasing the height of the canvas, the newly added area is being filled with black color instead of remaining transparent or matching the existing background color.

To Reproduce
Steps to reproduce the behavior:

Draw something on canvas, change the color/thickness  you will see its getting reset
Select a background color , click on the X (RESET BUTTON) the bg wont be reset
Click on the + to increase the height , you will the new area is filled with black color

@singodiyashubham87 Can you assign this issue to me , i would like to work under GSSoC'24 .

@singodiyashubham87 I have created the PR , Kindly review it ,
Thank you


make forward slash in output path consistent

Noticed that a trailing forward slash (or lack thereof?) on an output directory can cause funny results for output names.
Example without this change (from my unit test):
You're trying to obfuscate the code in src/cli/ and output it to src/cli/dist but some of the files end up in src/cli/dist/src/cli/ instead.

Thank you!


error when  do this command "./opam install sawja.1.5 atdgen.1.5.0 javalib.2.3a extlib.1.5.4"

The following actions will be performed:
∗  install ocamlfind   1.5.5                [required by javalib, atdgen, extlib, sawja]
∗  install menhir      20140422             [required by atd]
∗  install extlib      1.5.4
∗  install easy-format 1.0.2                [required by biniou, atd, yojson]
∗  install cppo        1.1.2                [required by yojson]
∗  install camlzip     1.05                 [required by javalib]
∗  install biniou      1.0.9                [required by atdgen]
∗  install atd         1.1.2                [required by atdgen]
∗  install javalib     2.3a
∗  install yojson      1.2.1                [required by atdgen]
∗  install sawja       1.5
∗  install atdgen      1.5.0
===== ∗  12 =====
Do you want to continue ? [Y/n] Y
=-=- Gathering sources =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
[atd] Archive in cache
[atdgen] Archive in cache
[biniou] Archive in cache
[camlzip] Archive in cache
[cppo] Archive in cache
[easy-format] Archive in cache
[extlib] Archive in cache
[javalib] Archive in cache
[menhir] Archive in cache
[ocamlfind] Archive in cache
[sawja] Archive in cache
[yojson] Archive in cache
=-=- Processing actions -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
[ERROR] The compilation of ocamlfind failed at "./configure -bindir /home/cjy/.opam/4.01.0/bin -sitelib /home/cjy/.opam/4.01.0/lib -mandir /home/cjy/.opam/4.01.0/man -config
/home/cjy/.opam/4.01.0/lib/findlib.conf".
Processing  1/12: [ocamlfind: make uninstall]
#=== ERROR while installing ocamlfind.1.5.5 ===================================#
opam-version 1.2.2
os           linux
command      ./configure -bindir /home/cjy/.opam/4.01.0/bin -sitelib /home/cjy/.opam/4.01.0/lib -mandir /home/cjy/.opam/4.01.0/man -config /home/cjy/.opam/4.01.0/lib/findlib.conf
path         /home/cjy/.opam/4.01.0/build/ocamlfind.1.5.5
compiler     4.01.0
exit-code    1
env-file     /home/cjy/.opam/4.01.0/build/ocamlfind.1.5.5/ocamlfind-29279-d8a4ae.env
stdout-file  /home/cjy/.opam/4.01.0/build/ocamlfind.1.5.5/ocamlfind-29279-d8a4ae.out
stderr-file  /home/cjy/.opam/4.01.0/build/ocamlfind.1.5.5/ocamlfind-29279-d8a4ae.err
stdout
Welcome to findlib version 1.5.5
Configuring core...
stderr
configure: ocaml not in PATH; this is required
=-=- Error report -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
The following actions were aborted
∗  install atd         1.1.2
∗  install atdgen      1.5.0
∗  install biniou      1.0.9
∗  install camlzip     1.05
∗  install cppo        1.1.2
∗  install easy-format 1.0.2
∗  install extlib      1.5.4
∗  install javalib     2.3a
∗  install menhir      20140422
∗  install sawja       1.5
∗  install yojson      1.2.1
The following actions failed
∗  install ocamlfind 1.5.5
No changes have been performed
=-=- ocamlfind.1.5.5 troubleshooting =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
=> Could not build ocamlfind. The most common reason for that is a missing 'm4' system package.

Hello @chenjinying,
Do you have m4 installed on your machine? Please run m4 --version and check if you get version information about it.
Thanks.

Hi @chenjinying, did @martinoluca's suggestion solve your issue?

Please reopen this issue if needed.


Swapped findProgramLocation with findProgramOnSearchPath

This should make it compile with GHC 8. I've only changed the deprecated function but since I don't have mysql installed I can't check if there are any more errors after that.

a release for 8 would be great.

@bos @MaxGabriel @Daniel-Diaz @yihuang can we get this merged for GHC 8?

@gregwebs I'd gladly merge, but I don't have write access!

findProgramOnSearchPath has been around for awhile, since at least Cabal 1.18 I think, so you could use it on more versions of GHC if you wanted. Probably unnecessary though.

I'm not sure if #if __GLASGOW_HASKELL__ < 800 is correct. findProgramLocation is depecrated in cabal 1.24.0. My system needs findProgramOnSearchPath but it is running ghc-7.10.3 and cabal 1.24.0 so that will try to make it use findProgramLocation. Can you do an if directive on the cabal version?

In that case you couldn't compile mysql even before the pull request since it used findProgramLocation unconditionally. I'm up for fixing this but I don't think this is the PR to do it. Then again, I'll add it to the fix if someone figures out what needs to be done.

@LukaHorvat that's correct. mysql was not installing. I think something like this would work, but I'm not sure how to get the cabal macros running in with runhaskell.
#if MIN_VERSION_Cabal(1,24,0)
        (findProgramOnSearchPath verbosity [ProgramSearchPathDefault] "mysql_config")
        (findProgramOnSearchPath verbosity [ProgramSearchPathDefault] "mysql_config5")
#else
        (findProgramLocation verbosity "mysql_config")
        (findProgramLocation verbosity "mysql_config5")
#endif

I made a new pull request with my fix. I believe it should work for GHC 7, GHC 8, and Cabal Versions 1.24.0 and below. findProgramOnSearchPath for Cabal 1.24.0, findProgramLocation for anything below #22

Closed via #22.  (I am now maintaining this package.)


Update scalacheck-1-14_2.11, ... to <IP_ADDRESS>

Updates

org.scalatestplus:scalacheck-1-14_2.11
org.scalatestplus:scalacheck-1-14_2.12

from <IP_ADDRESS> to <IP_ADDRESS>.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scalatestplus" } ]

labels: library-update

Skipping


App engine application resource

Just a quick question - I'm been trying to figure out what the app engine application is for, but can't see any obvious purpose it serves or anything it would do
Would you be able to share a bit on what it's for, and in what sort of situations it should be enabled?
Many thanks! :)

Hi @billinghamj, this is required for Cloud Scheduler as you can see in the official documentation

To use Cloud Scheduler your Cloud project must contain an App Engine app that is located in one of the supported regions.

Best,

App Engine application is no longer required for Cloud Scheduler
https://cloud.google.com/scheduler/docs/release-notes#February_11_2022
https://cloud.google.com/scheduler/docs/overview#supported_regions_by_target

If you are not using App Engine as a target, you do not need to deploy an App Engine app, and you can disable any existing App Engine app.


Could not find the required files

I've been trying to set up a libGDX project but i keep getting the same errors.
I tried googling but found nothing close to this. I tried reinstalling gradle, java (17,21,22), downloading the missing files (they seem to no longer be named as below) but nothing works.
I'm using 1.11.0 libGDX and if you dont already have the files in your maven repository then it probably wont work for you either.
This is the output when I try running the gradle desktop-other-run command on vscode.

3 actionable tasks: 1 executed, 2 up-to-date
<-------------> 0% WAITING
IDLE
FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':desktop:compileJava'.
Could not resolve all files for configuration ':desktop:compileClasspath'.
Could not find lwjgl-glfw-3.3.3-natives-linux.jar (org.lwjgl:lwjgl-glfw:3.3.3).
Searched in the following locations:
file:/C:/Users/USER/.m2/repository/org/lwjgl/lwjgl-glfw/3.3.3/lwjgl-glfw-3.3.3-natives-linux.jar
Could not find lwjgl-glfw-3.3.3-natives-linux-arm32.jar (org.lwjgl:lwjgl-glfw:3.3.3).
Searched in the following locations:
file:/C:/Users/USER/.m2/repository/org/lwjgl/lwjgl-glfw/3.3.3/lwjgl-glfw-3.3.3-natives-linux-arm32.jar
Could not find lwjgl-glfw-3.3.3-natives-linux-arm64.jar (org.lwjgl:lwjgl-glfw:3.3.3).
Searched in the following locations:
file:/C:/Users/USER/.m2/repository/org/lwjgl/lwjgl-glfw/3.3.3/lwjgl-glfw-3.3.3-natives-linux-arm64.jar
(etc...)

.
.
.
I also recently got this error:
<-------------> 0% WAITING

IDLE
C:\Users\USER\Desktop\Coding\Extensions\test\core\src\com\mygdx\game\MyGdxGame.java:3: error: package com.badlogic.gdx does not exist
import com.badlogic.gdx.ApplicationAdapter;
^
(etc...)
13 errors

FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':core:compileJava'.

So, gdx-setup is still (for now) linked in the documentation. But what was its competitor, gdx-liftoff, is now part of the libgdx organization. Hooray, I can officially recommend gdx-liftoff as the better option! I'm very close to releasing a new stable version, but for now there's a preview release of the new UI (and newest Gradle version) here: https://github.com/tommyettinger/gdx-liftoff-ui-redesign/releases/tag/v<IP_ADDRESS>-PREVIEW-A . This is for libGDX version 1.12.1; I don't know why you want the older 1.11.0 or if you do at all. If the issue is what I think it is, you will be able to generate a project just fine with Liftoff and may already be generating a project with gdx-setup, but the initial sync may fail on either or both. Steps to fix:
You should remove the lines in build.gradle that consist entirely of mavenLocal() (in repositories blocks). It is not at all rare for one failed download to the local Maven cache in C:\Users\USER\.m2 to stop the whole Gradle build. Avoiding the local Maven cache is one option (it will put libGDX in the Gradle cache instead, which doesn't have this problem). The other option is to remove the problem files from .m2, such as any files inside .m2/repository/com/badlogicgames/ , and let Gradle try to download them again.
If you use IDEA or Android Studio, after you remove either the mavenLocal() lines or the files from .m2, you should re-import the Gradle project, or whatever the term JetBrains uses today for "try to open the Gradle project again." The button is on the Gradle taskbar (to get the taskbar open, either click Gradle on the right edge of the window, or use View -> Tool Windows -> Gradle), and it looks like two circling arrows. It is usually at the top left of the Gradle tool window. This may start the download of all dependencies, potentially from nothing if you deleted files from .m2 . Once it completes, DesktopLauncher or Lwjgl3Launcher should have no red lines. You can launch the app using the Gradle task gradlew desktop:run (which you can launch in the Gradle tool window, potentially requiring some Settings wrangling to get it to show Gradle tasks, or you can launch the task by typing it in to Execute Gradle Task, or you can enter it at the command line). You can also launch using the green triangle next to main() in the code's margin, but I don't recommend this because it needs you to make a Run Configuration that has the right working directory (assets/).
I hope this helps!

 Thank you so much for your help!
The problem seemed to be a mixture of my firewall settings and the maven repository, so I deleted the .m2 repository folder and it redownloaded everything! Also doing this in vscode was a bad idea and switching to IDEA made it work almost instantly.
.
.
I also tried the liftoff version but I was getting the same error that it couldnt find a package, specifically com.badlogicgames.gdx:gdx:17. The error gave some links to repos where it would download, and I tried to do it manually but all of the links (like https://jitpack.io/com/badlogicgames/gdx/gdx/17/gdx-17.pom) didnt work. But its no matter since the original version works beautifully!
I appreciate your help so very much! Thank you again!

Yeah, the latest version of gdx-liftoff won't work (see https://github.com/libgdx/gdx-liftoff/issues/163) but you can use an older version of it or just stick to gdx-setup as you've done.

Yeah, the latest version of gdx-liftoff won't work (see libgdx/gdx-liftoff#163) but you can use an older version of it or just stick to gdx-setup as you've done.

I'd appreciate if you'd try the latest version of gdx-liftoff, since I can't reproduce the issue with versions getting mixed up.

Looks like you've got it fixed now in https://github.com/libgdx/gdx-liftoff/commit/c2f2f99ce77c7bcada8fd4b1c7b855e718044d45

Huh??? Who did that? Me???
...Yes, there were some stripe-y copy-paste-y types of errors.


Jikkou API server - kafka-connect support

Hey Jikkout team,
I want to use Jikkou API Server in our EKS cluster in-order to manage multiple kafka-connectors configurations.
i have read the instructions in the readme and in this medium article
Here is an output of the jikkou API version(latest 0.34):
curl -sX GET http://localhost:28082/  | jq .
{
  "version": "0.34.0",
  "build_time": "2024-05-01T22:00:55+0000",
  "commit_id": "1d3829c69adfb6b4ccbd23ba0ded1788946ed513",
  "_links": {
    "self": {
      "href": "/",
      "templated": false
    },
    "get-apis": {
      "href": "/apis",
      "templated": false
    }
  }
}

i've used the /apis endpoint to get whats available:
{
  "kind": "ApiGroupList",
  "apiVersion": "v1",
  "groups": [
    {
      "name": "kafka.jikkou.io",
      "versions": [
        {
          "groupVersion": "kafka.jikkou.io/v1beta1",
          "version": "v1beta1",
          "metadata": {
            "_links": {
              "self": {
                "href": "/apis/kafka.jikkou.io/v1beta1",
                "templated": false
              }
            }
          }
        },
        {
          "groupVersion": "kafka.jikkou.io/v1beta2",
          "version": "v1beta2",
          "metadata": {
            "_links": {
              "self": {
                "href": "/apis/kafka.jikkou.io/v1beta2",
                "templated": false
              }
            }
          }
        }
      ]
    }
  ],
  "_links": {
    "self": {
      "href": "/apis",
      "templated": false
    }
  }
}

after checking both groups with:
curl -sX GET http://localhost:28082/apis/kafka.jikkou.io/v1beta2
curl -sX GET http://localhost:28082/apis/kafka.jikkou.io/v1beta1

(output is too long)
i've noticed there is no KafkaConnector or anything related to kafka-connect at all.
so i have a couple of questions:
Does the API Server support kafka-connect?
Is there a plan to make it support kafka-connect? (if needed i can open an issue)
Thanks a lot!

Hi @arnoldyahad, did you enable the KafkaConnector provider ? See doc: https://www.jikkou.io/docs/jikkou-api-server/configuration/server_configuration/#enabling-specific-extension-providers

thanks a lot @fhussonnois
im using the kafkaConnector through the dockerized version of it, taken from the documentation
$ docker run -it streamthoughts/jikkou-api-server:main

i see in the doc you provided that there is a way to enable it through application.conf file, but i don't see it in my docker
is there any method to enable it through api request? or to pass an environment variable to enable it?
something like:
$ docker run -it streamthoughts/jikkou-api-server:main -e ENABLE_INTEGRATIONS=true

or by running an api command?
curl -sX GET http://localhost:28082/enable_extentions -H "Accept: application/json"  | jq .

(although a variable is much better way and can be persisted in code)
if those 2 options dont exist at the moment, what is the best way for me to proceed? is it adding a /etc/application.conf on my own (to use docker-compose for that)?
thanks a lot for your help!

Hi @arnoldyahad, sorry for my late reply:
You need to create an application.yaml file with :
micronaut:
  server:
    port: 8080
jikkou:
  extension.providers:
    default.enabled: true # -> enable all extensions by default

Then:
docker pull streamthoughts/jikkou-api-server:latest
docker run -it \
    -p 8080:8080 \
    -v /your-path/application.yaml:/app/resources/application.yaml
    streamthoughts/jikkou-api-server:latest


Logs of barong mailer are not displayed properly

That is true, I can confirm since version 2.4.0.
I guess the issue #1068 have something related.

Looks like log are buffering.
I can see logs after kill the service.

Seems to be related to devops side of the question. I will re-open if we have any new info on that


Improve/fix URI utils (3.0)

This is the same #17583 did for 3.1, but since one of the commits there has been reverted, it's no longer cherry-pickable as a whole. So I'm submitting here for 3.0 excluding the bad commit.

I will test this soon and this can probably go into 3.0.3rc2

Merged for 3.0.3rc2


podinfo: Try first in-cluster to get kubeconfig

In order to get a kubeconfig to talk to a server API in podinfo command
three elements are tried

KUBECONFIG env variable
~/.kube/config
in-cluster config

Better to check "in-cluster config" before the one in the user home to
avoid conflicts.
pod-info command info: #64

/lgtm

/hold

/unhold

/cherry-pick release-4.10
As #64  has been only cherry-picked to 4.10 this change need to be also cherry-picked to the same release.


Added XCode project configuration for use with Carthage

Looks good thanks!
I will move the tests to this Xcode Project  (and keep the Demo as a real Demo)


Events attached to the input element are removed

What happened?
Events added to the input element before using typeahead() are removed because the input element is cloned. Cloning does not copy events:
https://github.com/digitalfortress-tech/typeahead-standalone/blob/master/src/typeahead-standalone.ts#L93
Events added to the input field after using the typeahead() work fine.
Solution: Use event delegation on the parent span element instead of cloning the input element?
Affected version
v4.7.0
Reproduction URL
https://jsfiddle.net/r4f0h9yn/2/
How to reproduce?

Open console to see output
type something in the input field
console log should show 'added before' and 'added after', but only shows 'added after'

Browsers
No response
Relevant log output
No response
Additional context
No response

No cloning the input would also solve another problem: When I set the required attribute on the original input field, its never resolved, since it gets cloned into the search input...

Hello, thank you for your valuable feedback. Will resolve it with the next release.


[WIP] Tutorials refresh

Description forthcoming

I need a couple more hours with this.

Working through some screenshotting stability issues

Ok, this is ready.
In general, I'd like to make a more serious pass through the App user guide. Proposing I do that in a new PR.
As an aside, the server/web API is due for an upgrade. Making sure Colab is fully functional is becoming tricky.


Potential deadlock

By looking at the code, I suspect there's a potential for dead locks. I will try to come up with a test case that shows that and will let you know if I succeed. Until then, you might want to think about it and see if you come to the same conclusion.
Sequence leading to a dead lock (assume 4 tasks that take turns, e.g., only ever await asyncio.sleep(0) and the event loop is round robin):

Some task 1 acquires lock
There is a task 2 doing some (so far unrelated) things
Some task 3 tries to acquire the lock and ends up in the queue L39
Task 3 awaits the event L43
Some task 4 tries to acquire the lock (analog to the 2 steps above)
Task 1 releases the lock and sets the event which task 3 is waiting on L57
Now it's taks 2's turn again and it decides to cancel task 3 for whatever reason and then task 2 ends
Task 3's turn: it was cancelled and removes itself from the queue L47 :warning:  even though the event was set and task 3 was scheduled to take the lock
Now it's task 4's turn... It will never be notified that the lock is actually free => :zap: dead lock

@robsdedude Thanks kindly for your inspection, I'm back from holidays and am looking at this.
Hmm, let me see if I get it. In your example if Task 1 releases the lock then Task 3 is freed on it's next round, but we get to Task 2 first, and it cancels Task 3. So when we get to Task 3, we get the cancel error before it takes ownership, queue removal, and Task 4 is left hanging.
I was able to create the scenario (I think) but actually this revealed a different bug! Event 3 is not in the queue when it tries to remove itself from it.
@pytest.mark.asyncio
async def test_fair_async_rlock_deadlock_scenario():
    lock = FairAsyncRLock()

    # Use events to control the order of execution
    task1_aquired = asyncio.Event()
    task2_started = asyncio.Event()
    task3_started = asyncio.Event()
    task3_acquired = asyncio.Event()
    task4_acquired = asyncio.Event()
    task4_started = asyncio.Event()
    task1_done = asyncio.Event()

    async def task1():
        async with lock:
            task1_aquired.set()
            await task2_started.wait()
            await task3_started.wait() # wait until Task 3 in queue too
            await task4_started.wait() # wait until Task 4 in queue too
            await asyncio.sleep(0.1)  # make sure Task 4 gets in queue
            task1_done.set() # signal done before release, so Task 2 can cancel Task 3

    async def task2():
        await task1_aquired.wait()
        task2_started.set()
        await task1_done.wait() # Wait until Task 1 done then cancel Task 3
        t3.cancel()

    async def task3():
        await task2_started.wait()
        task3_started.set()
        async with lock: # now in queue, waiting
            task3_acquired.set() # Should not get reached, because Task 3 will be cancelled always

    async def task4():
        await task3_started.wait()
        await asyncio.sleep(0.1) # make sure Task 3 gets in queue first
        task4_started.set()
        async with lock: # it's not in queue, just after Task 3, and waiting
            task4_acquired.set()

    t1 = asyncio.create_task(task1())
    t2 = asyncio.create_task(task2())
    t3 = asyncio.create_task(task3())
    t4 = asyncio.create_task(task4())

    await t1
    await t2
    await t3 # Here we get ValueError: <asyncio.locks.Event object at 0x7f419f7dba90 [set]> is not in deque

    # Task 3 would never acquire
    assert not task3_acquired.is_set()
    # Task 4 should not deadlock. It should be able to acquire the locks
    await asyncio.wait([t4], timeout=1)
    assert task4_acquired.is_set()

Solution (to both bugs) is to replace the queue removal at https://github.com/Joshuaalbert/FairAsyncRLock/blob/main/fair_async_rlock/fair_async_rlock.py#L47 with
self._owner = me
self._count = 1
self._current_task_release()