Creating a layer heat map to better understand the layers?

When using convis on a model's higher level layers, a large amount of individual images are produced. This makes trying to view the image as the model sees it, very impractical.
So I was wondering about the practicality of having a heat map that combines all of the images, into a single false color image?

Each feature map of the model sees the image differently. One sees horizontal lines, another vertical lines, others see diagonal lines, circles, boxes, windows, eyes and so on. A layer may consist of as many as 512 feature maps, which respond to different features. Combining them does not sound a good idea to me, just like I wouldn't put 512 photos from London on top of each other to show what London is like.
My main idea in making convis was to be able to check, when training a model, how the training is succeeding. One can also use it to gain some understanding of what a model sees. But the feature maps respond of thousands of different features, and I don't see how one could compress that into a heat map in a meaningful way. To understand the layers, one would have to feed the model different kinds of images and the examine all the feature maps to determine what features exactly each feature map is seeing. But convis is probably too simplistic for that kind of work.

I recently noticed that MIT's Places 365 models were used to generate saliency maps: http://cnnlocalization.csail.mit.edu/
That is exactly what I was trying to do here with convis. I wonder if we can apply this to other models or if it's specific to the Places 365 project?

I found an implementation of CAM that works on the regular caffemodels that Neural-Style uses: https://github.com/ramprs/grad-cam

This is what I have so far for Neural-Style: https://gist.github.com/ProGamerGov/013349db88ff4df4bd7e0a647160a6ce
Iterations 50, 100:

Iterations 150, 200:

Iterations 50, 100:

Iterations 150, 200:

So classification.lua contains the code, along with: utils.lua
Specifically these two functions are used to create the heatmap:
https://github.com/ramprs/grad-cam/blob/master/misc/utils.lua#L84-L128
https://github.com/ramprs/grad-cam/blob/master/misc/utils.lua#L154-L176
Edit:
@htoyryla I can't seem to figure out how to get the code working in Neural-Style. I've been trying to place it all in the feval(x) function. Maybe it needs to be implemented like a loss function to work correctly?

It seems to me that you are trying to achieve both
a) get a heatmap that somehow combines all the featuremap activations from a given layer
b) monitor how this heatmap changed during iterations
I don't think there is any major difficulty doing this in neural-style, one simply needs to find a good way to combine, say, 128 feature maps into a single heatmap.
I did something related in this in one of our earlier threads here. This does not display feature maps though, but the gradients at each iteration, as if to indicate which part of the image is now changing and how much.

This gist contains modified neural-style to display the total gradient which is converted to an image by normalizing to range 0 .. 255. It just gives a rough visual indication how and weher the image is changing during the iteration process.
https://gist.github.com/htoyryla/445e2649293f702a940c58a8a3cef472

Convis was made simple to just map the activations. The more sophisticated visualization methods attempt to follow gradients to indicate exactly which areas in the image caused those activations. I guess your difficulties arise from the need to make neural-style do both the usual iterations and to trace the gradients for visualization. Good luck.

I made a quick test modifying convis to save a single combined activation map from a layer. Easy to do, it is only how meaningful such a map is as the different channels respond to different features, so it is quite natural that the combined activations from a layer cover most of the image.

Another thought: you cannot do this inside feval, because the output of each layer is not available there. However, one could calculate the activation map inside each style (and content) loss module and store the results inside feval. So when using the simple activations like in convis, no additional loss modules are needed. And I don't have time or interest to start looking into this following the gradients thing.

OK, I can see how the simple combined activation maps could be useful.
Relu1_1

Relu2_1

Relu3_1

Relu5_3

Just as a sidetrack... so my convis shows me the activations from each individual filter in a VGG network. Now I noticed that the second filter of relu1_1 of the usual VGG19 reacts mainly on the sky in the default Thuringen image. So taking the output from that feature map and adding some postprocessing I can get masks like these. The point here is that the filters in relu1_1 act directly on the image and therefore can also be used as ordinary image filters (if they happen to produce useful output, that is).

Could you please share the convis modifications for creating simple combined activation maps?

Could you please share the convis modifications for creating simple combined activation maps?
I am also wondering how torch.max can be used to get a predicted class value, when no classification list is provided. The code lines here seem to do this and I can't seem to recreate it in convis or Neural-Style.
Like for example:
local y = net:forward(img)
local score, pred_label = torch.max(y,1)
label = pred_label[1]
print("Predicted label: ", pred_label)

T
Another idea I just had was what if instead of arbitrary restricting Neural-Style's layer channels to specific values, we could instead restrict it to what matches the most likely label. Is is possible to get a list of layers and their filters using the above code?

@htoyryla For the mask image you created with relu1_1, I guess that particular filter was looking for a "sky texture"?
And in the context of our conversations here, "filter" and "layer channel", are the same thing, right?

For the mask image you created with relu1_1, I guess that particular filter was looking for a "sky texture"?

Not really, the lowest levels cannot detect complex entities like "sky", they simply act as basic convolutional filters. It could be that it detects a certain color.

And in the context of our conversations here, "filter" and "layer channel", are the same thing, right?

Yes. Functionally they are filters. In neural-style/torch terms, a channel in a layer.

Is the above method basically the same as yours, only it uses torch.max to get the label with the highest accuracy prediction, whereas your code checks every label and creates a top 5 set of labels.

Yes. Also when neural-mirage creates a new image, the target is the complete set classification probabilities, not only a single class with the highest probability.

This is the (quick & dirty) code I used to make an average activation map from a given layer. I am simply taking the CxHxW output from the layer and summing the different channels which gives a HxW tensor, then normalizing it to 0...255 value range for display.
require 'torch'
require 'nn'
require 'image'
require 'loadcaffe'

function preprocess(img)
   local mean_pixel = torch.DoubleTensor({103.939, 116.779, 123.68})
   local perm = torch.LongTensor{3, 2, 1}
   img = img:index(1, perm):mul(256.0)
   mean_pixel = mean_pixel:view(3, 1, 1):expandAs(img)
   img:add(-1, mean_pixel)
   return img
end

function deprocess(img)
  local mean_pixel = torch.DoubleTensor({103.939, 116.779, 123.68})
  mean_pixel = mean_pixel:view(3, 1, 1):expandAs(img)
  img = img + mean_pixel
  local perm = torch.LongTensor{3, 2, 1}
  img = img:index(1, perm):div(256.0)
  return img
end

local cmd = torch.CmdLine()

cmd:option('-image', 'examples/inputs/tubingen.jpg')
cmd:option('-output_dir', 'convis', 'directory where to place images')
cmd:option('-image_size', 800, 'output image size')
cmd:option('-proto', 'models/VGG_ILSVRC_19_layers_deploy.prototxt')
cmd:option('-model', 'models/VGG_ILSVRC_19_layers.caffemodel')
cmd:option('-layer', 'relu4_2', 'layer for examine')

local params = cmd:parse(arg)

local content_image = image.load(params.image, 3)
content_image = image.scale(content_image, params.image_size, 'bilinear')
local content_image_caffe = preprocess(content_image):float()
local img = content_image_caffe:clone():float()

local cnn = loadcaffe.load(params.proto, params.model, "nn"):float()

local net = nn.Sequential()

for i = 1, #cnn do
      local layer = cnn:get(i)
      local typ = torch.type(layer)
      local name = layer.name
      print(name, typ)
      net:add(layer)
      if (name == params.layer) then break end
      if (i == #cnn) then
        print("No such layer: "..params.layer)
        return
      end
end

local fmaps = net:forward(img)

local n = fmaps:size(1)
local filename = "c9out.png" --params.output_dir .. "/" .. string.sub(params.image:match("[^/]+$"), 1, -5) .. "-" .. params.layer

local y = torch.sum(fmaps, 1)
local m = y:max()
y = y:mul(255):div(m)

local y3 = torch.Tensor(3,y:size(2),y:size(3))
local y1 = y[1]
y3[1] = y1
y3[2] = y1
y3[3] = y1
local disp = deprocess(y3:double())
disp = image.minmax{tensor=disp, min=0, max=1}
disp = image.scale(disp, content_image:size(3), content_image:size(2))
image.save(filename, disp)
print("saving image ",filename)

In this comment here, I noted that the FCN-32s PASCAL model creates grey rectangle artifacts.
Image size 512:

Image size 1536:

I used a modified version of your convis.lua: https://gist.github.com/ProGamerGov/8f0560d8aea77c8c39c4d694b711e123
Then I just averaged all the layer output together with:
convert <layer1.png layer2.png layer3.png layer4.png> -average average_layers.png

Do you think that this has something to do with the artifacts?

You mean the added frame around the image. I think that comes from the 100 pixel padding used in the model, see https://github.com/shelhamer/fcn.berkeleyvision.org/blob/master/voc-fcn32s/val.prototxt#L27
I think there are ways to modify the model to remove the padding, I haven't done exactly this kind of operation thought. It is probably easier to try modifying style loss modules to remove the padding before calculating the Gram matrix.

In fact it is quite easy to remove the padding. Load the model into th, take the first layer and set padH and padW to zero (for instance). But one cannot save into a caffemodel from torch. I guess there are tools in caffe to do this though, but I haven't used them.
But one can do this at runtime like this:
   if next_content_idx <= #content_layers or next_style_idx <= #style_layers then
      local layer = cnn:get(i)
      local name = layer.name
      local layer_type = torch.type(layer)
      local is_pooling = (layer_type == 'cudnn.SpatialMaxPooling' or layer_type == 'nn.SpatialMaxPooling')
      --remove extra padding in fcn32s model
      if i==0 then
	    layer.padH = 0
	    layer.padW = 0
	  end
      if is_pooling and params.pooling == 'avg' then

@htoyryla I'm just curious if the padding is somehow the cause of the artifacts I experience.
Also, do you have any idea where I should start if I want to record information about the individual filters and their activations so that I can generate a list of usable layer channels?

Just modify neural-style  to remove the padding by adding these lines
  if i==0 then
	    layer.padH = 0
	    layer.padW = 0
	  end

and see if it makes a difference.

Also, do you have any idea where I should start if I want to record information about the individual filters and their activations so that I can generate a list of usable layer channels?

You seem to be asking for a simple way to do something which is quite complex.
Yet, as a second thought,  the most relevant channels are probably those with the strongest activations for the relevant images (both content and style). We could feed in an image and calculate some statistics on each channel, and then list the channels with the strongest activations.
Mere average would be too crude: it dismisses channels with strong activations within a smaller area. But it could be a way to start. Or taking the maximum. One can then try to find a better formula to measure the activations.
Let's see if I can try this approach, seems interesting to try.

Try this https://gist.github.com/htoyryla/49cb3ab0864d2a12f558631c7b3d87a3
Give an image (for use with neural-channels probably should be the style image) and you get a list of channels which might be the best ones suitable. Param nc specifies how many are listed.

Here's hopefully a working version, that tests the model during style capture and selects, per style layers, nc channels with the strongest activations (as measured during torch.norm of the channel output). These channels are then favored during iterations similar to how the earlier neural_channel worked. The rest the channels is not ignored totally (as this would stop the iterations from working) but given lower weight.
Remember that when decreasing nc, you need to increase style_weight yourself to keep the same content-style balance.
https://gist.github.com/htoyryla/b7940d31d329ee6ffb67b3185f414b8e

I'm noticing that the loss values with 10 of the best channels for each layer, resemble the loss values you see in the later stages of multiscale resolution.
I had a theory that channels/filters with strong activations result in a high degree of stylization while channels/filters with weak activations result in a low degree of stylization.
I first noticed this in my Protobuf-Dreamer project. For example, you can see that different channels of the mixed5a_1x1 layer, have different intensities of activations: https://i.imgur.com/icJjqm9.png

The difference is especially apparent on channel 106 (left), and channel 184 (right), where this was the input image:

While the inception5h model used in Protobuf Dreamer uses the inception architecture and not the VGG architecture that Neural-Style uses, I have suspect that the two are similar in regards to these high and low activation channels. Playing around with neural-channels.lua, it looked like I could influence the degree of stylization with by only changing the channel values. While testing my fine-tuned models, I noticed what appeared to be a similar effect:

The output from the Places 365 Hybrid model (left), and the fine-tuned version of the model (right).

The output from the Places 365 Hybrid model (left), and the fine-tuned version of the model (right).

Full album: https://imgur.com/a/TDiWU
What's interesting here, is that the degree of stylization is less with one style image, and more with another style image. The parameters never changed, but the channels/filters in the model did. I think this also backs up my theory.

For convis, I noticed that the Illustration2vec model's activations, resemble the model's "style". Compared to other models, the Illustration2vec model transfers styles with a very distinct anime style of it's own.

It seems to "see" every input image in an anime style.

I wonder how well placing an emphasis on the best content layer channels in addition to the best style layer channels would work? How would just placing an emphasis on the content layers compare to just placing an emphasis on the style layers?

I tried using:
  if i==0 then
	    layer.padH = 0
	    layer.padW = 0
  end

And it did not stop the artifacts from the FCN-32s PASCAL.

These are the results from my experiments with bestchannels.lua, style channels only, and the default channel weighting: https://imgur.com/a/yxnZm
This is the result from using style and content channels, in addition to the default channel weighting: https://i.imgur.com/5bFqeX9.png
These are the results from using style and content channels, and custom channel weighting values: https://imgur.com/a/LVekL
And this was the control test: https://i.imgur.com/kFUEZK0.png

Of course, the same parameters were used for each test, and only the -nc parameter along with the channel weighting value, was changed.

These results are certainly interesting, but I am having a hard time quantifying the differences in a meaningful way that makes sense based on the chosen parameters.

Changing the channel weighting was done by changing the value in this line of code: local m = torch.Tensor(C,H,W):fill(0.2), from the inputMask function.

This line defines the default weight of the channels. The weight of the selected channels is set here:
     m[sch] = 5

Ideally, I think, one would set the default weight to zero. When I was testing the original neural_channels, however, the iterations failed if the default weight was zero. The matrix became too sparse, I guess. But then I was testing with a single channel. With nc=10 I guess the default weight could be much smaller, like your experiment shows.

Because different channels have different activation strengths, I wonder what would happen if instead of giving the strongest channels a higher weighting, we instead tried to make every channel equal to every other channel regardless of activation intensity. Like for example, we gave the weakest channels higher weights relative to the strongest channels.

This could be an interesting experiment, but the results could be quite erratic: we would be emphasising features NOT found in the images!

I think I got neural_bestchannels.lua to do the same thing with the content layer(s): https://gist.github.com/ProGamerGov/ef79cc3d47f6647f8f5a1582a657ce3d

I had to add the mode captureS to make sure that the styleLoss module captures the best channels from the style image, not from the content image. I think in the contentLoss module this danger does not exist.
But interesting idea... ignoring all but the strongest content features.

Ouch... there is bug in neural_bestchannels.lua so that no channels actually get emphasis. So the only thing that happens that the style weight is decreased.
https://gist.github.com/htoyryla/b7940d31d329ee6ffb67b3185f414b8e#file-neural_bestchannels-lua-L530
This line should be
 if channels ~= nil then

Yea, I was looking over that part of the code earlier and wondering if it was indeed a bug. It looks like I did fix it myself, but that fix wasn't actually in the script I used to create the above examples...

One observation: this method of using mainly nc channels per layer now appears to favor relu1_x layers, which now have the highest loss values, while previously I think relu3_x was the strongest.
This is probably because relu1_x has fewest channels, so dropping most channels off has a smaller effect than on higher levels.

I couldn't find the style_channels variable anywhere else in the code,

In neural_channels style_channels contained the layers given in the parameter style_channels, which was now replaced by the automatically detected best channels. I had only overlooked this if statement.

Because different channels have different activation strengths, I wonder what would happen if instead of giving the strongest channels a higher weighting, we instead tried to make every channel equal to every other channel regardless of activation intensity. Like for example, we gave the weakest channels higher weights relative to the strongest channels.

One might calculate the average norm for the channels, and then populate the channel mask with multipliers: (average norm / channel norm). This would in effect make all channels equally strong. Should be easily to implement, although the effect could be strange: we would be favoring features  not present in the style model.
Meanwhile I made a more simple test: by just adjusting the code for inputMask as follows:
function inputMask(C, H, W, channels)
  local t = torch.Tensor(C,H,W):fill(1)
  local m = torch.Tensor(C,H,W):fill(2)
  if channels ~= nil then
    for i=1,#channels do
      local sch = channels[i]
      --print(i, sch)
      if sch > C then
        print("skipping non-existent channel ",sch)
      end
      m[sch] = 0.2
    end
  end
  return t:cmul(m):cuda()
end

we can suppress a few of the strongest channels, while still keeping close to the original style. For instance I like this result using the defaults but suppressing 8 strongest channels: simple, not too much detail.

This https://gist.github.com/htoyryla/072e1f0475eebc9a4dfc0c011498da9c
implements weighting each channel by (average norm of channels / norm of this channel).

Makes nothing dramatical, as far as I can see. It does not (as I may have thought) bring out features which are not in the style. I was thinking wrong: the process is still moving towards the style target. But what this may do is make finding the target more difficult, as those channels which contribute more to this style are attenuated.
Which makes me think: could be make the search faster by doing the reverse, amplifying the already strong channels. I guess not... like when you increase the learning rate, you are likely to miss the target.

I'm getting NANs from neural-equalchannels.lua.

Using the modified bestchannels.lua that supports both content and style layer channels, the results seem to make more sense now:

The weighting works correctly now as well it seems:

The control test:

Using different amounts of channels with the -nc parameter results in a focus on different aspects of the style image and content image. For example, the control image has vivid spirals, while -nc 50 created some really nice wave like patterns instead. I also think that the style "flows" better in relation to the content image with -nc 50, compared to the control image and other -nc values.

I meanwhile have come to like more the approach of suppressing the strongest channels. Produces simpler, less detailed image (kind of follows better the large forms of the style).
Suppressing strong channels:

Suppressing weaker channels

We can only use the maximum number of channels in the lowest layer right now. But maybe we could counteract some of this favoritism by treating the layer normally when the number of channels is larger than what the layer has.

Don't immediately see how that would work, but never mind. You are free to try it. I was thinking rather that, to make the effect similar in all layers, one would suppress a given proportion of channels. E.g. nc would be given 1..64, and then it would be multiplied by C/64.
I'll have a look at neural-equalchannels in a moment.

I download neural-equalchannels from the gist and give the command
th neural-equalchannels-gist.lua -print_iter 1 -backend cudnn

and it iterates nicely. Using adam works too. But it can well be that it will not work with all models or in all cases. After all, equalizing the activations from all channels is a quite extreme idea.

Suppressing the weaker channels creates a result that looks a bit more like fast style transfer, especially in that last example you posted. I've only been messing around giving emphasis to the strongest layers. How well do the values work in your suppression code that was shared earlier in an above comment?

Personally, I feel I never got fast-neural-style to give anything this close to my style. I am often after styles that are not too detailed, even towards abstract, and neural-style is not so good at it, and fast-neural-style was much worse.
Here's my inputMask() for suppressing strongest channels:
function inputMask(C, H, W, channels)
  local t = torch.Tensor(C,H,W):fill(1)
  local m = torch.Tensor(C,H,W):fill(2)
  if channels ~= nil then
    for i=1,#channels do
      local sch = channels[i]
      --print(i, sch)
      if sch > C then
        print("skipping non-existent channel ",sch)
      end
      m[sch] = 0.2
    end
  end
  return t:cmul(m):cuda()
end

Suppressing the weaker channels creates a result that looks a bit more like fast style transfer

Just to make sure... "suppressing weaker channels" is neural-bestchannels.lua as it is now in gist. The reverse approach would be suppressing stronger channels, with inputMask as in the comment above.

Just to make sure... "suppressing weaker channels" is neural-bestchannels.lua as it is now in gist. The reverse approach would be suppressing stronger channels, with inputMask as in the comment above.

I meant suppressing stronger channels.

I meant suppressing stronger channels.

The last example with suppressing stronger channels is with the lowest style weight. I think that makes it similar to fast-neural-style (with which one gets mainly color and texture effects while the shapes are not much affected... at least my impression of it).
I guess I did not try suppress weak channels with as low style weight at all. So to make a comparison ignore that example.

Here are the equalized content and style layer channel results:

I'm not sure what to say about the equalization results, but they are certainly different than all the previous tests.
And here's what happened when I suppressed the top 50 strongest channels on each layer for both the content and style layers:

I find it interesting that suppressed the top 50 strongest channels, helped the moon be transferred from the style image in a more complete form, than in the previous experiments.
(I hope that posting the images in this way, where you can click on them in order to get the full size, is better than creating really long/large comments filled with image)

Some experiments with different amount of channels for style and content layers, and experiments with equalizing either the content or style layer channels:

So all the "equalized" results that I have created seem to be have a flaw in the code that allowed for the loss values to not be NANs. I don't know what it is, but the equalization does not seem to work for me.
I'll have to play around with the parameters and see if that's the cause.

PS. noticed that you actually wrote "to not be NANs", which I do not understand, but anyway, pushing up even the channel that see nothing of interest is not very good for optimization. Maybe there is indeed a bug, that allows it to work at all.

I wrote "not to be NANs" because when I fixed the issue, my parameters resulted in NANs.

A while back I was using equal content and style weights in order to see what artifacts a particular content or style layer would produce. I found that for the VGG-16 SOD Finetune model, two style layers in particular: relu2_1, and relu3_1, were responsible for almost all of the "artifacts". The relu3_2
content layer produced the least amount of artifacts compared to the other content layers.
Some examples:
https://i.imgur.com/wQlvFml.jpg
https://i.imgur.com/YbQrwXj.png
For the NIN model, I found that using -content_layers relu1,relu2,relu7 -style_layers relu1,relu2,relu3,relu5,relu7, produced the least amount of artifacts. This worked really well in my tiling experiments, in addition to a -tv_weight of 0.000001 (just high enough to destroy the remaining artifacts, but not high enough to affect the output that much).
For your goal of having less detail and a "simpler" look to your outputs, it might be useful to try and eliminate the "high noise" layers from the -content_layers and -style_layers input values. That is, if my idea of using equal content and style weight values, works for this sort of task. If this does work, then I wonder what the effects of using "low noise" layers together with channel manipulation are?

PS. noticed that you actually wrote "to not be NANs", which I do not understand, but anyway, pushing up even the channel that see nothing of interest is not very good for optimization. Maybe there is indeed a bug, that allows it to work at all.

I wrote "not to be NANs" because when I fixed the issue, my parameters resulted in NANs.
This was the flaw in my code:
  if self.mode == 'captureS' then
		  print(self.name)
	      self.channelNorms = channelNorms(input)
    end
	self.mask = inputMaskStyle(input:size(1), input:size(2), input:size(3), self.bestChannels):cuda()
  end

And it seems that the following examples had this flaw:

Equalized Style -Content 50

Equalized Style - Suppressed Content 50

Equal Content & Style Channels

I think what happened was I copied the original flawed code for style channel equalization, for subsequent experiments using style channel equalization. All of the tests that did not try to equalize the style channels, don't have the flaw in their code.

Also, using multiple style images with channel prominence, and equalized content channels results in a familiar gray goo:

So the optimal parameters have changed with respect to Adam (I was using the default parameters, not the better ones I discovered). I'm not sure if this was just the the results of the input images like it has been in the past, or if using multiple style images had something to with it.

For your goal of having less detail and a "simpler" look to your outputs, it might be useful to try and eliminate the "high noise" layers from the -content_layers and -style_layers input values.

When I say "simple", I am actually thinking using a Finnish work "pelkistää", which the dictionary translates "simplify" but which actually means "reduce to the bare essentials". I don't believe that can be done through layer selection. Channel selection, maybe, not sure even about that.

It is curious that style channel equalization is resulting in NANs for me, while content channel equalization is not.

Just something that comes to my mind. I have not tried content channel equalization, so I do not really know, but there was something tricky about capturing the channel norms in style modules, as during the capture phases, the module gets both content and style images, one by one, as input. That's why I had to introduce the mode captureS, in order to capture the channel norms using the style image. Otherwise one also would run into size mismatches.
For content loss modules, captureS mode should be irrelevant.


Params is empty inside a middleware not added on a route .use

the params object should be filled for middleware that are not on a route. For example
router
  .use(someMiddleware) // in this middleware params is empty
  .use('/:id', routeHandler) // in this handler params has the id

Ideally all middleware should have the params regardless of where it lies in the use structure.

Parameters are determined when the path is matched. The url cannot been matched against all of the possible patterns in all nested routes before running any middleware.


Defaulting fields with kube builder

We have a workaround on this one so mark it as important-longterm.

xref: https://github.com/kubernetes/cel-admission-webhook/blob/main/hack/update-codegen.sh#L89-L92


Question: argument {vimrcs} of dein#begin

Warning:  I will close the bug issue without the minimal init.vim and the reproduce ways.
Problems summary
I can't understand the argument {vimrcs} of dein#begin() ：
{vimrcs} is a list of compared .vimrc and/or other configuration files.

I'm reading your vim config, why are you use dein.rc.vim to the argument.

It is used for updated vimrcs detection to update the cache automatically.

Got it


Removing URI.js

Removing the URI.js dependency, which was a bit heavier weight than necessary.

Although we aren't using URI.js - I would still like to see a class called URI, with the same API as URI.js (containing only the minimal features an implementation.)

@danmarshall Why? That will inevitably end up being bigger than what's here.

@billba I'm not saying the entrire API should be implemented, I'm suggesting that:

only implement the stuff we actually need
use the same "API surface" signatures for the stuff we implemented

Why?

The template string https://${YOUTUBE_DOMAIN}/embed/${props.embedId} is nice, but it also might be misused such that a developer might skip the encoding of parameters if they https://${YOUTUBE_DOMAIN}/embed/${props.embedId}?foo=${barValue}

@danmarshall is that any different from doing this as a [...].join('')?

@billba in the array.join approach, the developer will be adding https:// and slashes surrounding segements such as /embed/, I was thinking that a small class could do this job, as well as the encoding of parameters. I do like the idea of having a testable URI rather that ad-hoc string building. I think it can prevent a lot of bugs.

I agree that you could eliminate a little duplication (and thus bugs) with a function. I think trying to implement a subset of the URI API is overkill. But I'm open to the idea. My proposal is to have @bnookala implement this using the simple string interpolation as I proposed. Then if you want to take a swing at extracting the URI logic into a URI subset, we can see how much extra code it adds.

@billba, @danmarshall: once more, with feeling 🙃


Include the DB availability check we have on our k8s scripts into our standard startup scripts

Problem
Requesting DB availability check we have in our k8s scripts be included in our standard startup script. This will be helpful in case they start the server before the DB server has started.
Solution
Add logic similar to our K8s startup script
Reference to the k8s startup script - https://github.com/wso2/kubernetes-apim/blob/9faab3a92189f10f9d299d7b055efd08560c7040/simple/am-single/templates/am/instance/wso2am-deployment.yaml#L40
Affected Component
APIM
Version
No response
Implementation
No response
Related Issues
No response
Suggested Labels
No response

@Arshardh - Have you made a decision to include this for APIM 4.3.0? If so the customer would like to know what technique we'll use for this so that they can modify their startup scripts on the current version accordingly.


⚠ Mobitopup has degraded performance

In 8a3f454, Mobitopup (https://napcard.voz.tech) experienced degraded performance:

HTTP code: 200
Response time: 6057 ms

Resolved: Mobitopup performance has improved in 94d72a6 after 16 minutes.


Create Documentation Team

Create Documentation Team
Due Date: 1 June
Description:
We want to create a group for folks that are wanting to get into Documentation. This would be our documentation group and would focus on giving folks the ability to contribute to the community while learning valuable documenation coordination skills around open source.
Goal: Create a group lead by a BPD Council Leader. They are responsible for updating documenation across the website as needed.
Tasks:

[ ] Identify Documentation Leader
[ ] Create Documentation team on GitHub
[ ] Create role documentation for documentation team
[ ] Create a yagpbd command for documentation team
[X] create label for documenation
[ ] documentation role
[ ] create GitHub webhook for documenation labels to ping documenation role

It would be good to find someone else in the community to lead this but if we don't have anyone willing to take this role then I'm happy to run with this.


'_debugUltimatePreviousSiblingOf(after, equals: _firstChild)': is not true.

Closing as dup of #11895


how to enable t_debug?

I am seeing lot of T_Debug which enables number of debug prints. However I am at loss to how to enable it from cmdline param?
I see there is rv=kernelArguments(T_debug) but it is unclear how to pass this onto API.
There are other cmdline args i.e. TENSILE_DB which I can easily figure out and enable add'l debug prints but not with T_debug.

@jdgh000 Apologies for the lack of response.  Do you still need assistance with this ticket? Thanks!

Hi @jdgh000. Closing ticket due to lack of activity. Please re-open ticket if you still need assistance. Thanks!


RecordRTC , type:MediaStreamRecorder, timeSlice: 5000 calling onDataAvailable 2 times on each timeslice causing corrupted data

let recordrtc = require('recordrtc');
this.recorder = recordrtc(this.stream, {
recorderType: MediaStreamRecorder,
type: 'audio',
mimeType: 'audio/webm',
bitsPerSecond: 256 * 8 * 1024,
checkForInactiveTracks: false,
timeSlice: 5000,
ondataavailable: this.onChunkAvailable
});
This code behaving differently for users, few users getting data on each 5 sec timeslice, but few getting data 2 times per 5 sec timeslice which is corrupted and duplicate data.

Hi. I am facing the same issue. If you find any solution do let me know as well and If i find anything I'll keep you posted.

@nileshsargar All users are using Chrome/Firefox?

Yes.
On Sun, 7 Jul 2019 at 16:52, Muaz Khan<EMAIL_ADDRESS>wrote:

@nileshsargar https://github.com/nileshsargar All users are using
Chrome/Firefox?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/muaz-khan/RecordRTC/issues/540?email_source=notifications&email_token=ACAGTTEZIYWC6YE7J6EDUM3P6HKH7A5CNFSM4H324CKKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZLJ7LA#issuecomment-508993452,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACAGTTHVK7ONLGBJQ3WTJS3P6HKH7ANCNFSM4H324CKA
.

Apologies from my part, my code was calling the "startRecording()" api twice for one user, like user who is sending the command "startRecording" for each participant, getting called twice, I have resolved it, there was no bug for atleast me.


Add microseconds

Proposed Changes

Add microseconds (closes #45).
Adjust types and signedness.

:x: Build concoct 0.1.3 failed (commit https://github.com/Concoctist/concoct/commit/1aed7dabec by @ldilley)

:x: Build concoct 0.1.104 failed (commit https://github.com/Concoctist/concoct/commit/056c04c889 by @ldilley)

:x: Build concoct 0.1.4 failed (commit https://github.com/Concoctist/concoct/commit/056c04c889 by @ldilley)

:x: Build concoct 0.1.105 failed (commit https://github.com/Concoctist/concoct/commit/d20088495e by @ldilley)

:x: Build concoct 0.1.5 failed (commit https://github.com/Concoctist/concoct/commit/d20088495e by @ldilley)

:x: Build concoct 0.1.106 failed (commit https://github.com/Concoctist/concoct/commit/e9fcd45545 by @ldilley)

:x: Build concoct 0.1.6 failed (commit https://github.com/Concoctist/concoct/commit/e9fcd45545 by @ldilley)

:x: Build concoct 0.1.107 failed (commit https://github.com/Concoctist/concoct/commit/347ec1d568 by @ldilley)

:x: Build concoct 0.1.7 failed (commit https://github.com/Concoctist/concoct/commit/347ec1d568 by @ldilley)

:white_check_mark: Build concoct 0.1.108 completed (commit https://github.com/Concoctist/concoct/commit/9a33df8ba6 by @ldilley)

:white_check_mark: Build concoct 0.1.8 completed (commit https://github.com/Concoctist/concoct/commit/9a33df8ba6 by @ldilley)


useQuery not updating / showing correct result

Pretty simple setup:

create a world with an entity
query for that entity inside useQuery
result stays empty, while manual query in effect shows entity to exist

Managed to isolate it in this csb – open up the console to see the difference (if you're searching for it like me because of the new design – click the terminal icon on the top right):
https://codesandbox.io/p/sandbox/strange-poitras-2wl5l8?file=%2Fsrc%2FApp.js%3A14%2C35

Addressed by https://github.com/pmndrs/koota/pull/10


Is this possible on Visual Studio for Mac?

Is there a Visual Studio for Mac version of this page?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 3e8df1d5-dc7d-52fd-a7ef-130fae8dc67f
Version Independent ID: 331b4c43-96fb-3799-2646-82d2a681d18b
Content: Connect to Azure Government with Visual Studio - Azure Government
Content Source: articles/azure-government/documentation-government-connect-vs.md
Service: azure-government
GitHub Login: @stevevi
Microsoft Alias: stevevi

Discussed offline.  Customer request routed to VS for Mac product team.

#please-close


fix: only dismiss the report attachment modal once

Details

Fixed Issues

$ https://github.com/Expensify/App/issues/46388
PROPOSAL: https://github.com/Expensify/App/issues/46388#issuecomment-2255158354
Tests

Note: Only test in Desktop app/Desktop browser web app

Upload a pdf in a chat (chat 1) and then open another chat (chat 2)
Open chat 1 again using LHN
Open the pdf attachment
Click Cmd/Ctrl + K

Verify that: the first chat is still shown in the background while the chat search left modal open

[x] Verify that no errors appear in the JS console

Offline tests

QA Steps

Note: Only test in Desktop app/Desktop browser web app

Upload a pdf in a chat (chat 1) and then open another chat (chat 2)
Open chat 1 again using LHN
Open the pdf attachment
Click Cmd/Ctrl + K

Verify that: the first chat is still shown in the background while the chat search left modal open

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is either coming verbatim from figma or has been approved by marketing (in order to get marketing approval, ask the Bug Zero team member to add the Waiting for copy label to the issue)
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the UI (e.g. new buttons, new UI components, changing the padding/spacing/sizing, moving components, etc) or modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label and/or tagged @Expensify/design so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

https://github.com/user-attachments/assets/2ceb1b9e-ac4c-4138-9d3f-3865aeae896e

Android: mWeb Chrome

aweb.webm

iOS: Native

https://github.com/user-attachments/assets/dfb5d37c-64f8-4ac1-984a-c78c581fd0b7

iOS: mWeb Safari

https://github.com/user-attachments/assets/7ab3a9ec-75bd-4349-bbdf-de30b5c3ea5e

MacOS: Chrome / Safari

https://github.com/user-attachments/assets/dd7cdb44-56e4-4319-b459-97dd83dcc096

MacOS: Desktop

https://github.com/user-attachments/assets/a8764e83-5c4a-4a37-844d-a396983f1b22

🚀 Deployed to staging by https://github.com/thienlnam in version: 9.0.26-1 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/puneetlath in version: 9.0.26-6 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


coredump: adjust mem_region find in gdbstub

Adjust get_mem_region to not return region when address == end
as there will be nothing to read there. Also, a subsequent region
may have that address as a start address and would be a more appropriate
selection.
Signed-off-by: Mark Holden<EMAIL_ADDRESS>

I'm adding a comment saying that I would like to have the "stale" label removed.
Just a one line change, in need of a quick review.


Diffs consumption re boundaries

Expanding on the work done in #13 in regards to boundaries...
We've been following the openmaptiles method, in which boundaries are loaded. This is less than optimally, as it takes a half-hour to tun and is "just a second thing" Ideally, we would modify the mapping.yaml so that the boundaries were imported via imposm3 like everything else.
See what we can do about this. Ideal targets would include:

[ ] remove the ohmborders tasks and documentation, as they would not be used anymore

[ ] adjust the mapping.yaml to import boundaries. refer to the ohmborder source code to find the filtering and features

[ ] purge the DB and reload, confirm presence of boundaries

[ ] make some boundary-related change to OHM data, watch it propagate into vector tiles in our usual timeframe

Shifting temporarily to @jeffreyameyer so he can include thoughts and sketch on a broader solution to the low zoom border problem.

I[;m going to close this out and make a new issue based on latest approach


feat: inlay hints

Hi guys,
inlay_hints support for flutter and dart.
this pr just contains the implementation code. I will update README later :D
Demo:

Please take a look when you have any time. @akinsho

I tested the code, it does not work yet in flutter stable branch, but in Flutter master hints are visible.
Not sure if they should be enabled by default until anticonceal is done. They do not add much value at the end of the line.
Example in nvim:

Example in VS Code:

I tested the code, it does not work yet in flutter stable branch, but in Flutter master hints are visible.
Not sure if they should be enabled by default until anticonceal is done. They do not add much value at the end of the line.
Example in nvim:
Example in VS Code:

thanks for review
I agree that inlay hints should be placed in exact characters that lsp return.
So I will disable inlay hints by default.

Neovim nightly added support to inlay hints, so there is no need to add additional setup in this plugin anymore.
https://twitter.com/Neovim/status/1671046461071523843
inlay_hint({bufnr}, {enable}) vim.lsp.buf.inlay_hint()
Enable/disable/toggle inlay hints for a buffer
Parameters:
{bufnr} (integer) Buffer handle, or 0 for current
{enable} (boolean|nil) true/false to enable/disable, nil to toggle


  It makes the OTLP HTTP and gRPC exporters optional peer dependencies.  The dependencies are only required if you configure the tasks runner to use those specific exporters.
Breaking Changes
This PR changes the default exporter from otlp-grpc to console.  This way if you do not specify a specific exporter, no additional exporter packages will be required to be installed.

@Djiit Let me know if this does what you were looking for!

Yup, something like this :) Thank you
Reading the doc, I think you have to declare the 2 dependencies in peerDependencies and in peerDepencenciesMeta (see https://docs.npmjs.com/cli/v8/configuring-npm/package-json#peerdependenciesmeta)

@Djiit  Nx automatically defines those for us when we build the package with the buildableProjectDepsInPackageJsonType option.
"build": {
      "executor": "@nrwl/js:swc",
      ...
        "buildableProjectDepsInPackageJsonType": "peerDependencies"
      }
    }

This is what the generated package json looks like when built:
"peerDependencies": {
    "@nrwl/devkit": "^13.10.0",
    "nx": "^13.10.0",
    "@grpc/grpc-js": "^1.3.7",
    "@opentelemetry/exporter-trace-otlp-grpc": "^0.27.0",
    "@opentelemetry/sdk-node": "^0.26.0",
    "@opentelemetry/sdk-trace-base": "^1.0.0",
    "@opentelemetry/api": "^1.0.3",
    "@opentelemetry/core": "^1.0.0",
    "@opentelemetry/resources": "^1.0.0",
    "@opentelemetry/semantic-conventions": "^1.0.0",
    "@opentelemetry/exporter-trace-otlp-http": "^0.27.0",
    "rxjs": "^6.6.7"
  },
  "dependencies": {
    "@swc/helpers": "~0.3.3"
  },
  "devDependencies": {},
  "peerDependenciesMeta": {
    "@opentelemetry/exporter-trace-otlp-grpc": {
      "optional": true
    },
    "@opentelemetry/exporter-trace-otlp-http": {
      "optional": true
    }
  },

I don't think we need to manually define those dependencies.  The NX packages are manually defined so that we can make their version requirements more flexible than what we've defined in the root package.json, but no changes are needed for those packages.

Alright, lgtm :)
On a side note, may I ask why the swc/helpers is a direct dependency ?

Good question.  Looks like the SWC compiler was configured to use externally packaged helpers rather than inline them.  Looking at their docs, that's really only helpful in bundled apps.
I've updated the compiler settings to use inline helpers and removed the dependency.


feat(vitest/prefer-concurrent): add rule

This rule would guide test authors to prefer using vitest's concurrent mode (either at the suite or test level).

@jroru  when do you want the plugin to get triggered? Share more.


It does not work

I have tried the demo version and it does not work. Are you sure the library works? I have used an iPad and safari browser for trying it.

really? well it had been tested on android, sadly i did not have an ios to test it on. Could you show me the behavior?

I have found the issue. It seems that the algorithm used to check for mobile does not include safari on iPads specifically. I'll get this fixed as soon as possible.

Thanks so much, I saw your reply a bit late. Thanks so much for showing attention to, and developing this extension. I really appreciate it.

Alright, in the newest release I had fixed the issue. The issue was that the userAgent in the browser for apple devices have been changed to look identical to the Mac's, causing the browser not to recognize it as mobile.


No tooltip shown when hovering over a terminal status

Testing #122241
There is no tooltip over the terminal status e.g. , the other items have a tooltip. The tooltip could say 'Relaunch Needed'.
There also is no tooltip on the bell icon.

This also happens in SCM and explorer decorations, looks like it happens because the hover listener is on .monaco-icon-label-container and not .monaco-icon-label

Changing the container to its parent will end up breaking the actions:

https://github.com/microsoft/vscode/blob/61364b049268df5d4fc01b32602e7b7fd7d67203/src/vs/base/browser/ui/iconLabel/iconLabel.ts#L155-L155

I want to know how toggle  attach.😢

Anybody's working on this? If not, I will try.

Anybody's working on this? If not, I will try.
@khoaleanhvn  may not

Hello @jrieken, I would love to work on this issue if no one is working on it, could you provide a guide on what is to be done.

I would be happy to take this up. Looks like @Ashray123 has already requested to take up #124419, so this one should be available.

This also happens in SCM and explorer decorations, looks like it happens because the hover listener is on .monaco-icon-label-container and not .monaco-icon-label

Changing the container to its parent will end up breaking the actions:

https://github.com/microsoft/vscode/blob/61364b049268df5d4fc01b32602e7b7fd7d67203/src/vs/base/browser/ui/iconLabel/iconLabel.ts#L155-L155

Hey @Tyriar, I created the PR above, and didn't notice any broken actions on my branch. Do you mean that the other buttons won't work, e.g. for a Terminal session the :wastebasket: ?

I forgot why we added that for quick input, probably to avoid a hover when hovering over the label of a quick pick item but still show it when hovering over description which can be cropped.
Can we not put the hover over the entire thing if there is no description title otherwise?

Should the title be set up for the whole element when there is no description or no description title?

This seems reasonable to me.

/verified


Remote Tunnel: Failed to connect to the remote extension host server (Error: WebSocket close with status code 1006)

Does this issue occur when all extensions are disabled?: Yes/No

VS Code Version: 1.85.0-insider
OS Version: Windows_NT x64 10.0.22621

Steps to Reproduce: Cannot be systematically reproduced
Hello everyone,
For the past few days, I've encountered an error while attempting to connect to the VS Code server hosted on a remote Linux VM. This issue occurred a few weeks ago, and it was resolved the following day without any explanation.
Actions taken at that time:

Reloaded VS Code
Restarted Windows 11
Uninstalled and reinstalled the service
Recreated a new GitHub token

The current error message is: "Failed to connect to the remote extension host server (Error: WebSocket close with status code 1006)."
What I didn't do during the previous occurrence was take a closer look at the Output trace:
2023-11-28 17:35:07.988 [trace] [server] Installing and setting up Visual Studio Code Server...
2023-11-28 17:35:07.990 [error] [proxy] [connection.0] error setting up websocket: Error: WrappedError(WrappedError { message: "error creating server directory", original: "No space left on device (os error 28)" })

However, I want to emphasize that there is sufficient disk space, and the VS Code server has been installed and used daily for a few months now.
Could you please assist me in figuring out the root cause of this issue?

"No space left on device" is an error from your operating system. By default, the CLI will install servers in the ~/.vscode-server directory in your home folder. Please make sure there's sufficient space there and that the CLI has permissions to create more files in that location.


Program braks after finding match for vehicle

Hello,
This is everything I have done.

cloned reop
Implemented Traccar configuration to the necessary files. I can see the vehicle in real-time, no noticeable latency in-vehicle refresh.
Enable vehicle auto-assigner
I can see when 2 or more routes are a match and none is selected. (Program keeps running)
When one route is selected and is a temporary match the program breaks

For agencyId=1 Exception null for avlReport=AvlReport [vehicleId=1697, time=07-22-2020 10:44:54.000 AST, timeProcessed=07-22-2020 10:44:55.212 AST, location=[#####, #######], speed=NaN, heading=NaN, source=This is th, assignmentId=null, assignmentType=UNSET]. java.lang.NullPointerException: null 	at org.transitclock.core.RealTimeSchedAdhProcessor.generateEffectiveScheduleDifference(RealTimeSchedAdhProcessor.java:185) ~[Core.jar:na] 	at org.transitclock.core.PredictionGeneratorDefaultImpl.generate(PredictionGeneratorDefaultImpl.java:423) ~[Core.jar:na] 	at org.transitclock.core.MatchProcessor.processPredictions(MatchProcessor.java:80) ~[Core.jar:na] 	at org.transitclock.core.MatchProcessor.generateResultsOfMatch(MatchProcessor.java:200) ~[Core.jar:na] 	at org.transitclock.core.AvlProcessor.lowLevelProcessAvlReport(AvlProcessor.java:1425) ~[Core.jar:na] 	at org.transitclock.core.AvlProcessor.processAvlReport(AvlProcessor.java:1619) ~[Core.jar:na] 	at org.transitclock.avl.AvlClient.run(AvlClient.java:163) ~[Core.jar:na] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_262] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_262] 	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_262] 	at org.transitclock.utils.threading.NamedThread.run(NamedThread.java:100) [Core.jar:na]


bad wallet path causes "[Fatal] Unhandled top-level exception: $exn"

I started the daemon from the .coda-config folder and so the path to the private key was wrong - it caused the daemon to crash with unhandled exception.

Closing this issue as it is stale for a long time.


feat(Segmentation): Add contour representation for segmentations

Continuing on the plan to add multiple representations for a segmentation, this PR adds the contour representation (read about representations here)
This PR add new geometryLoader which is supposed to deal with loading geometries and cache them. As the first geometry, this PR introduces contourSets and contours.
Since usually a set of contours represents an entity, a contourSet exists

Adding a contour representation is similar to adding a labelmap

First the geometry (in labelmap, the volume) should get added. For each contourSet a separate geometry is cached. This is a design decision that is intended to enabled converting of the geometry types to each other (contour -> closed surface and vice versa). As seen below each contourSet will get a geometryId that can later be used to construct a segmentation.

const promises = contourSets.map((contourSet) => {
  return geometryLoader.createAndCacheGeometry(contourSet.id, {
    type: GeometryType.CONTOUR,
    geometryData: contourSet as Types.PublicContourSetData,
  });
});

await Promise.all(promises);

A segmentation can get created

// Add the segmentations to state
  segmentation.addSegmentations([
    {
      segmentationId,
      representation: {
        // The type of segmentation
        type: csToolsEnums.SegmentationRepresentations.Contour,
        // The actual segmentation data, in the case of contour geometry
        //  this is a reference to the geometry data
        data: {
          geometryIds: contourSets.map({id}=>id)
        },
      },
    },
  ]);

A segmentation representation should get added

 await segmentation.addSegmentationRepresentations(toolGroupId, [
    {
      segmentationId,
      type: csToolsEnums.SegmentationRepresentations.Contour,
    },
  ]);

Demo
https://deploy-preview-384--cornerstone-3d-docs.netlify.app/live-examples/contoursegmentationrepresentation

Hey not sure if this is appropriate forum for this, but just wanted to say that I really appreciate the level of documentation for this PR as it made it quite straightforward understand the design choice.
Awesome work on this!

How did the contourSets data come from ?


Manage permissions to have access to Sanity

Check why @marinalinn doesn’t have access to the EDS Mobile documentation project in Sanity

Note: check if is possible to have a final access this week

@vnys I think you said there is a last access isn't?

Added @marinalinn to the project. Log in with SSO on https://www.sanity.io and choose the equinor-eds project.

Tusen takk! I will check it


Convert Sorted Array to Binary Search Tree

problem link: https://leetcode.com/problems/convert-sorted-array-to-binary-search-tree/

@aritra-tech may I work on this issue?

Ok sure
Make a folder under the tree folder named "Leetcode Problems" and then create another folder named as the problem statement (i.e Sorted_array_to_bst) and then add the java/c++ file over there.
And create a README and add the question link over there.

Okay, I will do the same.

Ok sure
Make a folder under the tree folder named "Leetcode Problems" and then create another folder named as the problem statement (i.e Sorted_array_to_bst) and then add the java/c++ file over there. And create a README and add the question link over there.

Is Python also allowed?

Ok sure
Make a folder under the tree folder named "Leetcode Problems" and then create another folder named as the problem statement (i.e Sorted_array_to_bst) and then add the java/c++ file over there. And create a README and add the question link over there.

Is Python also allowed?

No Python
Just Java/C++/C


exitAfterDays

Make it easy to follow up for a fixed amount of time
exitAfterDays(
cohort,
cohortId = NULL,
days = 365, #reset cohort end to be 365 after cohort start
requireDays= FALSE, # if TRUE, keep only the people that had 365 days (drop people with exit before then)
name = tableName(cohort)
)

duplicate of #132


Upgrade Prometheus & Alertmanager

[ ] I added CHANGELOG entry for this change.
[ ] Change is not relevant to the end user.

Changes
We've an use case in Cortex for which we need to upgrade Prometheus and the Alertmanager. Some breaking changes were introduced in the HTTP client (from prometheus/common) so we need to upgrade it in Thanos too.
What's the sentiment if we do such upgrade in Thanos as well?
Verification
Existing tests

The Github CI seems like broken, failures are not related, so mering this one.

The Github CI seems broken and failures are not related, in any case I'm going to run the tests locally.


middleware: Allow HTTP from localhost, not through a reverse proxy.

In servers with application_server.http_only = true and loadbalancer.ips set, the DetectProxyMisconfiguration middleware prevents access over HTTP from IP addresses other than the loadbalancer.
However, this misses the case of access from localhost over HTTP, which is safe and expected -- for instance, the email-mirror-postfix script used in the email gateway^1 will post to http://localhost/ by default in such configurations.  With the
DetectProxyMisconfiguration installed, this will result in a 403 response.
Make an exception for requests from <IP_ADDRESS> and ::1 from proxy-misconfiguration rejections.

Self-review checklist

[ ] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[ ] Explains differences from previous plans (e.g., issue description).
[ ] Highlights technical choices and bugs encountered.
[ ] Calls out remaining decisions and concerns.
[ ] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[ ] Each commit is a coherent idea.
[ ] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[ ] Visual appearance of the changes.
[ ] Responsiveness and internationalization.
[ ] Strings and tooltips.
[ ] End-to-end functionality of buttons, interactions and flows.
[ ] Corner cases, error conditions, and easily imagined bugs.

Nicely debugged, merged, thanks @alexmv!

Backported to 7.x as 7482a7e61fc502564657060cb6e2346593bf1e06.


What about upload images?

is there an example to share an image with WhatsApp as its website
WhatsApp Messenger can handle various types of media:
images of any type that conforms to public.image (for example, PNG and JPEG)

images - «.wai» which is of type net.whatsapp.image

+1

+1


PERF/COMPAT: use kahan summation in .rolling_sum

from mailing list
to avoid imprecision errors as the rolling computations are evaluated marginally (sliding the window and adding new / subtracting old). We tend to accumulate floating point errors.
I looks like just an extra sum and subtract so I think the perf impact would be minimal.
This might be able to be applied to .rolling_mean as well. Note that .rolling_std/var already use Welford's algo for better precision.
https://en.wikipedia.org/wiki/Kahan_summation_algorithm
implementation from numpy
cdef double kahan_sum(double *darr, npy_intp n):
    cdef double c, y, t, sum
    cdef npy_intp i
    sum = darr[0]
    c = 0.0
    for i from 1 <= i < n:
        y = darr[i] - c
        t = sum + y
        c = (t-sum) - y
        sum = t
    return sum

cc @jaimefrio


[Mint] Implement real BIP-32 key derivation

Right now we use this simple function in cashu/core/crypto/keys.py to derive the keys for the next keyset. The derivation path is the string 0/0/0/0.
def derive_keys(master_key: str, derivation_path: str = ""):
    """
    Deterministic derivation of keys for 2^n values.
    TODO: Implement BIP32.
    """
    return {
        2
        ** i: PrivateKey(
            hashlib.sha256(
                (master_key + derivation_path + str(i)).encode("utf-8")
            ).digest()[:32],
            raw=True,
        )
        for i in range(settings.max_order)
    }

Seems to be implemented already.


Type inference fails with overloading, for variables and wildcards

I thought type inference was supposed to give variables the most general types that let a rule parse, and then assign other labels and terms the least sorts after that.
This doesn't seem to always work, and also I get different errors about variables and wildcards. Here's a sample definition, to be saved as test.k
require "domains.k"
module TEST-SYNTAX
syntax A ::= B | "a"
syntax B ::= "b"
syntax TA ::= TB | t(A) [klabel(t)]
syntax TB ::= t(B) [klabel(t)]
endmodule
module TEST
imports MAP
imports TEST-SYNTAX
configuration <k>$PGM:K</k>
rule t(X)::TA => .K
rule t(Y) => .K
rule t(_) => .K
endmodule

Compiling it produces as kompile --kore test.k produces this output:
[Error] Compiler: Had 1 parsing errors.
[Error] Critical: Could not infer a unique sort for variable 'Y'. Possible
sorts: A, B
	Source(/home/brandon/test/k/wildcard/././test.k)
	Location(13,6,13,16)
[Warning] Inner Parser: Parsing ambiguity. Arbitrarily choosing the first.
1: syntax TB ::= "t" "(" B ")"
[Location(#token("6",Int),#token("15",Int),#token("6",Int),#token("30",Int))
Source(#token("/home/brandon/test/k/wildcard/././test.k",KString))
klabel(#token("t",AttributeValue)) klabel(#token("t",KString))]
    t(#SemanticCastToB(#token(#KVariable,"_")))
2: syntax TA ::= "t" "(" A ")"
[Location(#token("5",Int),#token("20",Int),#token("5",Int),#token("35",Int))
Source(#token("/home/brandon/test/k/wildcard/././test.k",KString))
klabel(#token("t",AttributeValue)) klabel(#token("t",KString))]
    t(#SemanticCastToA(#token(#KVariable,"_")))
	Source(/home/brandon/test/k/wildcard/././test.k)
	Location(12,6,12,10)

Note that there's an error about Y but not X even though Y could be typed just like X, and that the wildcard gets only a warning even though the ambiguity is equivalent to the ambiguity about Y (and it just happens to default to the less general type).
I expected all of X, Y, and _ to end up having inferred type A.

This looks indeed like a bug to me.  @radumereuta ?

@bmmoore:

I thought type inference was supposed to give variables the most general types that let a rule parse, > and then assign other labels and terms the least sorts after that.

Yes, variables should be inferred the largest sorts, but so should be the terms!  The rationale is that one always has the option to use Term::Sort to enforce a smaller sort to a term if one wants to, while it is impossible to do the same for a larger sort.
I'm adding some notes for @radumereuta to the manual, to clarify these better.


RuntimeError: Default config is missing

When I try to run (sudo or not) I receive the message:
Traceback (most recent call last):
File "./main.py", line 213, in
main()
File "./main.py", line 37, in main
raise RuntimeError("Default config is missing")
RuntimeError: Default config is missing

If you look at the last two lines you can see the error, plain as day.

What should I do now?

What should I do now?

remove all drivers with driver store explorer and only use the ones bundled with the exploit

The Solution to : Default config is missing :
The Problem is that u have not downloaded the payloads required as mentioned in README.md file
Now Open Terminal where  bypass_utility is located in your system
And then  run:
git clone https://github.com/MTK-bypass/exploits_collection.git
cd exploits_collection
cp default_config.json5 ../
cp payloads ../
cd ../
And now run
sudo python3 main.py
Hope these helps!

The Solution to : Default config is missing :
The Problem is that u have not downloaded the payloads required as mentioned in README.md file
Now Open Terminal where bypass_utility is located in your system
And then run: git clone https://github.com/MTK-bypass/exploits_collection.git  cd exploits_collection cp default_config.json5 ../ cp payloads ../ cd ../ And now run sudo python3 main.py Hope these helps!

Can you please tell me how to execute these commands through mtkclients for Windows? 1 and 2 commands are executed, the rest is not clear how to start.

The Solution to : Default config is missing :
The Problem is that u have not downloaded the payloads required as mentioned in README.md file
Now Open Terminal where bypass_utility is located in your system
And then run: git clone https://github.com/MTK-bypass/exploits_collection.git  cd exploits_collection cp default_config.json5 ../ cp payloads ../ cd ../ And now run sudo python3 main.py Hope these helps!

Hi Sir, Thanks for you reply. I followed your advise and solved the error. Now while running the main.py , it is stuck at found device and nothing is happening. I'm not a tech person. I'm just following your code and running it. Can you please help?

Hi Sir, Thanks for you reply. I followed your advise and solved the error. Now while running the main.py , it is stuck at found device and nothing is happening. I'm not a tech person. I'm just following your code and running it. Can you please help?


Ci improvements

Description

Fixes collecting test coverage
Removes code climate scripts (probably it would need to remove from codeclimate service? @martinpovolny )
introduce codecov for collecting coverage in PRs

@miq-bot add_label bug, enhancement
@Hyperkid123

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@9ff5505). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #86   +/-   ##
=========================================
  Coverage          ?   40.24%
=========================================
  Files             ?       56
  Lines             ?      492
  Branches          ?       53
=========================================
  Hits              ?      198
  Misses            ?      262
  Partials          ?       32

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9ff5505...4a79221. Read the comment docs.


Make sure progressive enhancement works with aliases

Description:
The function render of class Alias does not take occupants as an argument. In order to make progressive enhancement work, it needs to do so. (I hope I'm not missing some bigger picture.)
Is breaking:
Should not break anything.

Nope, that's it. Thanks for the fix!


docker can not work on ubuntu 16.04

docker version
Client:
Version:      1.10.3
API version:  1.22
Go version:   go1.6.1
Git commit:   20f81dd
Built:        Wed, 20 Apr 2016 14:19:16 -0700
OS/Arch:      linux/amd64
Cannot connect to the Docker daemon. Is the docker daemon running on this host?
ps -ef|grep docker
root      7299     1  0 17:38 ?        00:00:00 /usr/bin/docker daemon -H fd://
Steps to reproduce the issue:

Install Docker via "apt install docker.io" on Ubuntu 16.04
Just run any docker command, docker will fail to start with the error:

your user is running with the right permission ?

can you try using sudo docker version?
Also note that you're installing a version of docker that is maintained and supported by Ubuntu, and not the official docker packages we provide. To install the official docker packages from this repository, follow the instructions here; https://docs.docker.com/engine/installation/linux/ubuntulinux/

@wenchma @thaJeztah it work well after I reboot my server but not work only restart docker service

@YuleiXiao I'm not sure if the ubuntu packages also modify the user to be in the docker group, which requires you to log out and back in again; (we mention this in our installer script)
I'm going to close this issue, because I think it's resolved, and the package you installed is maintained by Ubuntu, so we cannot provide support for that.
Feel free to comment here after I closed 👍


Update hpu-tests.yml

Add the latest docker container image used to run hpu tests
What does this PR do?

Fixes #<issue_number>
Does your PR introduce any breaking changes? If yes, please list them.

Before submitting

[ ] Was this discussed/approved via a GitHub issue? (not for typos and docs)
[ ] Did you read the contributor guideline, Pull Request section?
[ ] Did you make sure your PR does only one thing, instead of bundling different changes together?
[ ] Did you make sure to update the documentation with your changes? (if necessary)
[ ] Did you write any new necessary tests? (not for typos and docs)
[ ] Did you verify new and existing tests pass locally with your changes?
[ ] Did you list all the breaking changes introduced by this pull request?
[ ] Did you update the CHANGELOG? (not for typos, docs, test updates, or minor internal changes/refactors)

PR review
Anyone in the community is welcome to review the PR.
Before you start reviewing, make sure you have read the review guidelines. In short, see the following bullet-list:

[ ] Is this pull request ready for review? (if not, please submit in draft mode)
[ ] Check that all items from Before submitting are resolved
[ ] Make sure the title is self-explanatory and the description concisely explains the PR
[ ] Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?
Make sure you had fun coding 🙃

@raoakarsha You might be already working on a fix for the following error, but just in case...
It seems docker isn't available on the host https://dev.azure.com/PytorchLightning/pytorch-lightning/_build/results?buildId=69018&view=logs&j=4d03c832-dbbd-5507-ad7a-9e2939b77985&t=ca57f4b3-95be-41fd-a882-0a5d9d095e29&l=11:
##[error]File not found: 'docker'

Related doc page is here: https://docs.microsoft.com/en-us/azure/devops/pipelines/process/container-phases?view=azure-devops

Linux-based containers
The Azure Pipelines system requires a few things in Linux-based containers:

Bash
glibc-based
Can run Node.js (which the agent provides)
Does not define an ENTRYPOINT
USER has access to groupadd and other privileges commands without sudo

And on your agent host:

Ensure Docker is installed
The agent must have permission to access the Docker daemon

Hi @raoakarsha! What's the status of this PR? 🐶

Converting this PR to draft as it's not ready yet.

@raoakarsha You might be already working on a fix for the following error, but just in case...
It seems docker isn't available on the host of the agent pool https://dev.azure.com/PytorchLightning/pytorch-lightning/_build/results?buildId=69018&view=logs&j=4d03c832-dbbd-5507-ad7a-9e2939b77985&t=ca57f4b3-95be-41fd-a882-0a5d9d095e29&l=11:
##[error]File not found: 'docker'

Related doc page is here: https://docs.microsoft.com/en-us/azure/devops/pipelines/process/container-phases?view=azure-devops

Linux-based containers
The Azure Pipelines system requires a few things in Linux-based containers:

Bash
glibc-based
Can run Node.js (which the agent provides)
Does not define an ENTRYPOINT
USER has access to groupadd and other privileges commands without sudo

And on your agent host:

Ensure Docker is installed
The agent must have permission to access the Docker daemon

Hello, thank you @akihironitta  for sharing the document. The agent on which the hpu tests are running is already running on a docker hence, it cannot identify the docker. @Borda could you please suggest here?

@akihironitta  HPU test run on intel-hpu agent, which is a docker hosted agent. Hence, we cannot see the docker tool available.
CC: @Borda @kaushikb11  Please comment.

@akihironitta @carmocca @tchaton  Could you please, review this change

@raoakarsha I'm sorry for the delay. I'll come back to this PR and look into it this weekend!

@akihironitta @carmocca @tchaton Could you please, review this change

@akihironitta  Hello, Did you get a chance to review this change? We need to update the latest docker version too.

@akihironitta I see some test being done on this patch. Could you please, help me understand what is missing here? We need this patch for updating later docker versions.
Note: intel-hpu node is already running on docker hence there are docker related errors
@Borda  Please, correct me if wrong.

@raoakarsha for reproducibility we would need to include also the docker image for building the agent... I have the local copy, so can we add it as part of this PR? 🐰

Sure @Borda anything that works.

Please, help review this change.

New commit done https://github.com/Lightning-AI/lightning/pull/13398/.

closing in favor of newer image #13398


Pipewire

It is now possible to ask turbo audio to connect to specific streams with pipewire.
It is also possible to specify which ports connect with which ports.
In this example, we ask that all spotify output ports should be connected in order with turbo audio's input ports
    let connections = vec![StreamConnections {
        output_stream: "spotify".to_string(),
        input_stream: "ALSA plug-in [turbo_audio]".to_string(),
        port_connections: PortConnections::AllInOrder,
    }];
    start_pipewire_listener(connections);

In this example, we ask that that all spotify output ports should be connected in order with turbo audio's input ports and that Firefox's output_FL should be connected to turbo audio's input_FR
    let connections = vec![
        StreamConnections {
            output_stream: "spotify".to_string(),
            input_stream: "ALSA plug-in [turbo_audio]".to_string(),
            port_connections: PortConnections::AllInOrder,
        },
        StreamConnections {
            output_stream: "Firefox".to_string(),
            input_stream: "ALSA plug-in [turbo_audio]".to_string(),
            port_connections: PortConnections::Only(vec![(
                "output_FL".to_string(),
                "input_FR".to_string(),
            )]),
        },
    ];
    start_pipewire_listener(connections);

LGTM


Stop depending on | Upgrade Qiskit-Terra Dependency Cap the Qiskit-Terra version

What is the expected enhancement?
Running tests yields a deprecation warning for PauliTable:
Test table initialization.  DeprecationWarning: The class ``qiskit.quantum_info.operators.symplectic.pauli_table.PauliTable`` is deprecated as of qiskit-terra 0.24.0. It will be removed no earlier than 3 months after the release date. Instead, use the class PauliList
In order to stay with the most recent versions of the qiskit-ecosystem, we need to find an alternative to PauliTable.

Also handled in #385


Support builds for CLI + platform

Case:
A request to create a compose-file/preview is sent from CLI to platform. The compose file has context that points to a github repo. The core calls the platform parser for the build option, but later it fails in ContainerBuilderService because we don't call a platform method from there.
We need to call the platform's ContainerBuilderService if dependency injection exists.

Not supported in the current scope


Different improvements

Hello,
I have provided several improvements and would be happy if they are included in the next release.

First I have overriden the doSetLastModifiedTime because the parent one simply throws an Exception  and vfs is calling this method at some point so I am unable to upload new files.
Second I have made getDefaultFileSystemOptions method to be static so that we can access it when creating the authentication object:

            StaticUserAuthenticator auth = new StaticUserAuthenticator(null, azureProperties.getAccountName(), azureProperties.getAccountKey());
            FileSystemOptions opts = AzFileProvider.getDefaultFileSystemOptions();
            DefaultFileSystemConfigBuilder.getInstance().setUserAuthenticator(opts, auth);

And finally I have added the vfs-providers.xml so that the azure provider will be registered automatically when detected in the classpath.

Any news on this one?


Update CONTRIBUTING

View the preview here: https://github.com/ChainSafe/lodestar/blob/cayman/style-guide/CONTRIBUTING.md
Suggestions welcome

damm, review was pending :( @wemeetagain

I mean it's nothing major, we can fix it in subsequent PRs

I'll make another one now :)


Explain Listen All, multiple ports, and addresses

@stummsft : Thanks for your contribution! The author, @MikeRayMSFT, has been notified to review your proposed change.

Validating.

@stummsft - thank you for your submission. #sign-off


[ Layer ] Draft of Conv2D Layer

This is the first draft of 2d convolution layer
Self evaluation:

Build test:	 [X]Passed [ ]Failed [ ]Skipped
Run test:	 [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon<EMAIL_ADDRESS>

:memo: TAOS-CI Version: 1.4.20191203. Thank you for submitting PR #121. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://nnsuite.mooo.com/.

:octocat: cibot: @jijoongmoon, A builder checker could not be completed because one of the checkers is not completed. In order to find out a reason, please go to http://nnsuite.mooo.com/nntrainer/ci/repo-workers/pr-checker/121-202006021347300.62391710281372-06450acab1f911c34a6d8e33c3b13847cc49b1c0/.

:octocat: cibot: @jijoongmoon, A builder checker could not be completed because one of the checkers is not completed. In order to find out a reason, please go to http://nnsuite.mooo.com/nntrainer/ci/repo-workers/pr-checker/121-202006021401210.21146607398987-a3a2e954ad7c842923434cfe95cedb1c1ceda68e/.


update faraday

Update faraday :)

@jodyheavener I've fixed the feed state it should've been available and also created the find operation 👍

Hi @acrogenesis! Thanks for taking the time to open a PR. I'm no longer part of this project but perhaps @ryanbrunner or @pauloancheta can help with this.

@acrogenesis thanks for the contribution!


Create a callback function for "Verse of the Day"

Create a callback function that queries the vese_of_the_day table at let's say 6 am IST every day and notifies all the subscribed applications of the latest verse of the day.
We could later use this for our own bots. So, when the API notifies our bots of the latest "Verse of the Day", our bots will parse and post it to the relevant social media channel.
Reference guide: https://fastapi.tiangolo.com/advanced/openapi-callbacks/

Depends on #43

Hi @samanyougarg ,
I want to work on this issue

Hey
So basically we've a cron job that posts a verse of the day daily to our Twitter Handle
and now we have to make a callback function so that people can subscribe to the daily verse and we send them daily verse, something like a push notification
Please find the reference here :- https://fastapi.tiangolo.com/advanced/openapi-callbacks/


Documentation

Is there any documentation available on how to use this ETL tool?

Hi @iramedh,
Once you are able to launch the Hydrograph UI, you can navigate to the Help contents (by pressing the F1 key or via the About -> Help menu). There is pretty extensive documentation on how to use the tool, explanation regarding the available components, how they work with each other and all the features the tool provides. Let us know if you have any specific questions.

@alpeshk, How do I launch the Hydrograph?


Old-style text figures (hanging numbers) not working for certain fonts.

Description
Using #set text(font: "Palatino", number-type: "old-style", fallback: false) at the top of the document, the text In order to<PHONE_NUMBER> should render with old-style numerals, but instead they look the same as if number-type: "lining" had been set. There are no compiler errors, and there is certainly some version of Palatino that is being loaded, since all of the characters render.
I have a system-installed version of Palatino (macOS Ventura 13.4), and I verified that it has support for old-style figures (U+0030 through U+0039). I also tried copying the Palatino.ttf file to a local directory for my project and running typst --font-path ./local_font_dir watch mydocument.typ, but the document renders the same.
Please see the screenshot below: the top line is rendered with LaTeX using \usepackage{mathpazo}[osf] and is the style I'm trying to replicate, while the bottom line is rendered with Typst.

This was discussed today on the Discord. It also seems like there is no way to detect which version of a font Typst is loading (e.g., local, system, Typst-included). I should also note that old-style numerals do seem to be working for the default Typst font.
Reproduction URL
No response
Operating system
macOS
Typst version

[X] I am using the latest version of Typst

Thanks for investigating @mkorje! Based on this, I'll close as it's not a Typst issue.

thanks so much for looking into this @mkorje, this is great information

yeah thank you!


[SUPPORT] With HoodieROTablePathFilter is too slow load normal parquets  in hudi release

Describe the problem you faced
with the HoodieROTablePathFilter  load normal parquet file, it will be too slow when  reaches a certain order of magnitude
For example：500 partitions and 50000 data files
data path: s3://bucket1/{baseDir}/{partitionDir}/{partitionDir}/{data file}
To Reproduce
Steps to reproduce the behavior:

submit spark application

spark-sql --master yarn \
--conf spark.hadoop.mapreduce.input.pathFilter.class=org.apache.hudi.hadoop.HoodieROTablePathFilter

create temp view

create or replace temporary view {user_view} using parquet options (path "s3://bucket1/{baseDir}/");

Then slow load occurs
Environment Description

Hudi version : 0.10.0

Spark version : 3.1.1

Hive version : 3.1.2

Hadoop version : 3.2.1

Storage (HDFS/S3/GCS..) :  S3

Running on Docker? (yes/no) : no

Additional context
use  the PR [https://github.com/apache/hudi/pull/3719] will  mitigate this problem，again run
create or replace temporary view {user_view} using parquet options (path "s3://bucket1/{baseDir}/");

can finished in about 60 seconds
22/12/09 14:01:41 WARN SharedInMemoryCache: Evicting cached table partition metadata from memory due to size constraints (spark.sql.hive.filesourcePartitionFileCacheSize = 262144000 bytes). This may impact query planning performance.

Time taken: 61.771 seconds

At the same time，we have not repeated the problem [https://github.com/apache/hudi/issues/4188].  In our spark cluster，[HUDI-3719] this PR has used to query partition tables for half a year，such as：
==create table==
CREATE EXTERNAL TABLE `pickinglogs`(
  `_hoodie_commit_time` string COMMENT '',
  `_hoodie_commit_seqno` string COMMENT '',
  `_hoodie_record_key` string COMMENT '',
  `_hoodie_partition_path` string COMMENT '',
  `_hoodie_file_name` string COMMENT '',
  `id` string COMMENT 'ID',

.......

  `meta_es_offset` string COMMENT '',
  `meta_type` string COMMENT '',
  `meta_status` int COMMENT '',
  `meta_md5` string COMMENT '',
  `ptk_time_create` string COMMENT '')
PARTITIONED BY (
  `year` string COMMENT '',
  `month` string COMMENT '',
  `day` string COMMENT '')
ROW FORMAT SERDE
  'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe'
STORED AS INPUTFORMAT
  'org.apache.hudi.hadoop.HoodieParquetInputFormat'
OUTPUTFORMAT
  'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'
LOCATION

==query for sparksql==
spark-sql> select count(id) from pickinglogs where year=2022 and month between '08' and '10';
441834287
Time taken: 22.095 seconds, Fetched 1 row(s)

Stacktrace

@zhangyue19921010

Introduced the following PR in hudi-0.11.0-rc1 will again causes slow load issues.
[HUDI-2779] Cache BaseDir if HudiTableNotFound Exception thrown (https://github.com/apache/hudi/pull/4014)
When i removed that code in hudi-0.12.1，the slow load problem was alleviated.
      if (baseDir != null) {
        // Check whether baseDir in nonHoodiePathCache
//        if (nonHoodiePathCache.contains(baseDir.toString())) {
//          if (LOG.isDebugEnabled()) {
//            LOG.debug("Accepting non-hoodie path from cache: " + path);
//          }
//          return true;
//        }

Example:
==submit application==
# sudo -u hive spark-sql --master yarn --conf spark.hadoop.mapreduce.input.pathFilter.class=org.apache.hudi.hadoop.HoodieROTablePathFilter --jars s3://bucket1/hudi-spark3.1-bundle_2.12-0.12.1.jar
Setting default log level to "WARN".
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
22/12/12 16:02:04 WARN HiveConf: HiveConf of name hive.server2.thrift.url does not exist
22/12/12 16:02:05 WARN Utils: spark.executor.instances less than spark.dynamicAllocation.minExecutors is invalid, ignoring its setting, please update your configs.
22/12/12 16:02:05 WARN Client: Neither spark.yarn.jars nor spark.yarn.archive is set, falling back to uploading libraries under SPARK_HOME.
22/12/12 16:02:14 WARN Utils: spark.executor.instances less than spark.dynamicAllocation.minExecutors is invalid, ignoring its setting, please update your configs.
22/12/12 16:02:14 WARN YarnSchedulerBackend$YarnSchedulerEndpoint: Attempted to request executors before the AM has registered!
Spark master: yarn, Application Id: application_1660903282590_9234

==run load==
spark-sql> create or replace temporary view src_order_cate_query using parquet options(path 's3://bucket1/search_offline/src_order_cate_query/');    //this path have 570 partitions, 5.4w parquet files
22/12/12 16:04:02 WARN SharedInMemoryCache: Evicting cached table partition metadata from memory due to size constraints (spark.sql.hive.filesourcePartitionFileCacheSize = 262144000 bytes). This may impact query planning performance.
Time taken: 97.353 seconds

Actually, for legacy MapReduce, This patch is very important. Without this patch, HoodiROTablePathFilter will be thousands times slower.
Can we just brinig back https://github.com/apache/hudi/pull/3719 and fix the NPE?

Should we open an Apache JIRA for this?


[BUG] setuptools module version issue

setuptools version
71.0.1
Python version
Python 3.9
OS
Linux
Additional environment information
No response
Description
We have an application where we use multiple Python packages to run some tests. Recently, we observed some anomalies when rerunning those tests after using a new Docker image. In Docker, we used the setuptools module along with other modules, pointing to the latest version of setuptools. The issues arose when it installed the new version of setuptools 71.0.1. I will share some logs of the error.
Expected behavior
it successfully installed the package without any anomalies.
How to Reproduce
you can run some pytest test cases using the latest  version of setuptools.
Output

  File "/usr/local/lib/python3.9/site-packages/importlib_metadata/__init__.py", line 183, in load
    module = import_module(match.group('module'))
  File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/usr/local/lib/python3.9/site-packages/setuptools/_vendor/typeguard/__init__.py", line 21, in <module>
    from ._importhook import ImportHookManager as ImportHookManager
  File "/usr/local/lib/python3.9/site-packages/setuptools/_vendor/typeguard/_importhook.py", line 35, in <module>
    OPTIMIZATION = "typeguard" + "".join(version("typeguard").split(".")[:3])
  File "/usr/local/lib/python3.9/site-packages/importlib_metadata/__init__.py", line 977, in version
    return distribution(distribution_name).version
  File "/usr/local/lib/python3.9/site-packages/importlib_metadata/__init__.py", line 470, in version
    return self.metadata['Version']
  File "/usr/local/lib/python3.9/site-packages/importlib_metadata/_adapters.py", line 54, in __getitem__
    raise KeyError(item)
KeyError: 'Version'

@DevCompanionReviewer Setuptools 71.0.1 was yanked because of a similar bug, which I think was fixed in version 71.0.2, can you try upgrading ?

This issue is a duplicate of #4489.

yes @neutrinoceros, we pointed here the older version to fix the issue for the time being. Also with the 71.0.2 version we have seen it is fine thanks.


Testing in relx fails

Rebar3 can't be used to test relx because it doesn't use the new relx code during the tests. This happens even though the common test provider prepends the code paths.
Not sure if there is a solution besides adding a code:del_path/1 or code:replace_path/2 where code paths are added, like in common test provider.

The code path being there doesn't mean the new modules will be loaded unless they're specifically unloaded first and need to be re-fetched dynamically. You will in fact be likely to end up with mixed versions if starting the dependency doesn't load all its modules right away;  old modules stay in place, new ones get loaded from the new path.

Ugh, and because it is an escript it loads them even though they haven't been used?

Didn't think I'd need the purge unless it was actually used or was running in embedded mode.
Well ok, replace_path it is.

maybe related, but to test rebar itself i think the escript has to be rebuilt to ensure the correct versions of the module are loaded

@talentdeficit yeah. I either need to rebootstrap it and/or to specifically call ./rebar3 ct for it to run.

I fixed this for relx (and any other project) but it completely fucks running ct on rebar3 itself. Even if I ensure rebar and meck aren't unloaded I get:
{undef, [{rebar_log_meck_original,default_level,[],[]},

in the deps suite and some others. So still investigating.
Not sure if we'll just have to say screw it and leave it blowing up when run against rebar3 and use ct_run directly or use an escript like the boostrap one that calls rebar3:run for running ct so it uses the latest code.

Wait, nm, it works. I'm not 100% about it, but will explain that in the PR.


Feature Request: format elapsed time for running queries

It would be nice if the elapsed time displayed for running queries was formatted as HH:MM:SS instead of just displaying seconds:

I use dbeaver for a lot of long-running analytics queries and this would be useful for monitoring run times.

Yes, second this request. I don't want to have to keep dividing by 60 to get a real sense of time elapsed.
Thanks.

I second this request as well.

@serge-rider Is there any updates you could provide related this feature request? I believe this would be something quick and easy to provide. Any reason why this couldn't or wouldn't be implemented?

You can vote for the issue with 👍 under the issue description to promote it to a higher priority.


Display Homebrew analytics data

Make use of the command or code of https://github.com/homebrew/homebrew-formula-analytics to query Homebrew's analytics data and display that on each formula's page. I can provide you with a token that will get you access to query this data on your Heroku servers.

Sounds amazing. Thanks.
Do you have some example data so I can experiment a bit?

@koraktor Sure thing. I'll send you an invite to our Slack.

Moved to https://github.com/Homebrew/formulae.brew.sh/issues/10