Convolutional layers for 3D

This PR includes the following convolution layers:

Convolution3D
MaxPooling3D
AveragePooling3D
UpSampling3D
ZeroPadding3D

And also the corresponding test functions for these layers.
You can use it like this:
Convolution3D(nb_filters, nb_time, nb_row, nb_col)
See the convolution 3D example in examples/shapes_3d_cnn.py by @MinhazPalasara.
The shape of the underlaying 5D tensor:
(samples, channels, time, rows, cols) if dim_ordering='th'
(samples, time, rows, cols, channels) if dim_ordering='tf'.
Here I used "time" for the case of movies, of course it can be depth of a 3D volume. I didn't use the notion "depth" because some people use "depth" to stand for filter or channel, the word "depth" sometimes appears in the docstring of Keras when it means channel.
The following functions are added into theano_backend.py:

conv3d
pool3d
resize_volumes
spatial_3d_padding

But notice that, currently, these layers won't work in tensorflow backend, due the fact that there is no convolution3d ops implemented in tensorflow. It would be great if anyone can implement it.
Details about convolutional 3D in theano:
There are two implementations available in theano, I used conv3d2d.conv3d for both GPU and CPU, because it's faster:

on CPU:

        *  conv3d2d.conv3d:    160 ms per loop
output:
[  2.30778933   2.12729692  -3.06885958  -4.90572691  10.89875412
   3.7447443   -4.60463953   8.3501606    3.69148254  -0.93439424]

        *  nnet.conv3D: 1.19 s per loop
output:
[  2.30778956   2.12729645  -3.0688591   -4.90572643  10.89875507
   3.74474525  -4.60463953   8.35016346   3.69148254  -0.93439448]

on GPU (k40):

        *  conv3d2d.conv3d:    36.2 ms per loop
output:
[  2.30778933   2.12729692  -3.06885958  -4.90572691  10.89875412
   3.7447443   -4.60463953   8.3501606    3.69148254  -0.93439424]

       *  nnet.conv3D: 941 ms per loop
output:
[  2.30778956   2.12729645  -3.0688591   -4.90572643  10.89875507
   3.74474525  -4.60463953   8.35016346   3.69148254  -0.93439448]

Notice that there are precision differences with conv3d2d.conv3d on GPU and CPU. The testing code can be found in this discussion: https://groups.google.com/d/msg/theano-users/1S9_bZgHxVw/0cQR9a4riFUJ.
Another PR(#718) was created by @MinhazPalasara  but it's out dated. Many functions, the test example and test dataset in this PR are taken from #718, thanks @MinhazPalasara, @rbharath, @fchollet and others for improving that codes and giving suggestions.
Be careful that, these codes are not tested restrictedly, please report bugs if you encountered any.

By running the test code, I notice that, the loss value from each epoch is different when I run on GPU and CPU, I think it could possibly because the precision difference of conv3d2d.conv3d, but not sure if it's normal for a layer to give different result on GPU and CPU.
In my test, I tried to fix the random seed by np.random.seed(0), but maybe it's not working correctly because of theano. But it still can't explain why the CPU tend to have higher loss value with the same epoch number, I tried many times and it behaves similar.

Run on GPU

Train on 4000 samples, validate on 1000 samples
Epoch 1/100
32s - loss: 0.7605 - acc: 0.7418 - val_loss: 0.1312 - val_acc: 1.0000
Epoch 2/100
32s - loss: 0.0711 - acc: 0.9910 - val_loss: 0.0062 - val_acc: 1.0000
Epoch 3/100
32s - loss: 0.1353 - acc: 0.9808 - val_loss: 0.0029 - val_acc: 1.0000
Epoch 4/100
32s - loss: 0.0045 - acc: 0.9995 - val_loss: 9.2182e-04 - val_acc: 1.0000
Epoch 5/100
32s - loss: 0.0019 - acc: 0.9995 - val_loss: 0.9107 - val_acc: 0.8010
Epoch 6/100
32s - loss: 0.0465 - acc: 0.9910 - val_loss: 1.4903e-04 - val_acc: 1.0000

Run on CPU

Train on 4000 samples, validate on 1000 samples
Epoch 1/100
223s - loss: 0.9434 - acc: 0.5200 - val_loss: 0.6909 - val_acc: 0.6500
Epoch 2/100
245s - loss: 0.5292 - acc: 0.7597 - val_loss: 0.1592 - val_acc: 0.9680
Epoch 3/100
248s - loss: 0.1219 - acc: 0.9738 - val_loss: 0.0613 - val_acc: 1.0000
Epoch 4/100
249s - loss: 0.1347 - acc: 0.9698 - val_loss: 0.0566 - val_acc: 1.0000
Epoch 5/100
249s - loss: 0.0099 - acc: 0.9988 - val_loss: 0.0095 - val_acc: 1.0000
Epoch 6/100
249s - loss: 0.0039 - acc: 0.9990 - val_loss: 8.9441e-04 - val_acc: 1.0000

Any idea about this difference?

I think 3D operators will be interesting to have in Keras. I will do a detailed code review in the near future. For now, please:

drop the dataset and the associated example
scrub references to "time", "rows" and "cols" from the added layers (including the docstrings). "rows" and "cols" (and channels) make sense for arbitrary 4D tensors, but "time" does not. I'd replace them with "first_spatial_dim", "second_spatial_dim", "third_spatial_dim", "channels" (if you can think of something more concise while still abstract and explicit, that's fine too).
Add exceptions in all added layers to output a helpful error message in case the layers are instantiated with the TensorFlow backend.

Thanks for the response.
For the references to "time", I have the similar concern as you, but it's not easy to find one which is abstract and explicit.
In real world, and in the context of CNN, 3D data usually means 3D volume or 2D movie. For 3D volume, it's spatial signal. And for movie, it's spatio-temporal signal. For  "first_spatial_dim", "second_spatial_dim" etc., it's still not precise, because, time dimension for a movie is not the first spatial dim, it's a temporal dim.
Maybe other people have better ideas about how to define it, but before we find a good one, perhaps, it would still be reasonable to use "time":

In the real world, movie-like data have much more applications, simply because we have more cameras which can take videos, and much less device to capture 3D volume.
In 2D, most application are working on static image, and it's very natural to treat 3D as movie by default.
For 1D signal, Keras say it's a temporal signal, and for 2D signal, it's spatial signal. And these assumptions are not always true, it's still useful because it matches the most common task in real-life and it's very intuitive to understand how layers work.
Theano should be more general than Keras, but it still uses "time", "movie" and "spatio-temporal" for 3D signal. For example, in conv3d2d.conv3d and nnet.conv3D, they both used "time" in the docstring.
When the user understand how Keras process a movie, and it would be very easy for them to replace the time dimension into z dimension. Plus, we could just say it in the docstring about that.

So, I would suggest we use a more specific word to make it more easier to understand.
If other people have better idea on this, please discuss here.

It's precise because "space" does not mean physical 3D space, it means
space. As in, a vector space in linear algebra. In the context of
convolution, "space" means that which on which you convolve (as opposed to
"channels").
On 2 February 2016 at 15:15, Will Ouyang<EMAIL_ADDRESS>wrote:

Thanks for the response.
For the references to "time", I have the similar concern as you, but it's
not easy to find one which is abstract and explicit.
In real world, and in the context of CNN, 3D data usually means 3D volume
or 2D movie. For 3D volume, it's spatial signal. And for movie, it's
spatio-temporal signal. For "first_spatial_dim", "second_spatial_dim" etc.,
it's still not precise, because, time dimension for a movie is not the
first spatial dim, it's a temporal dim.
Maybe other people have better ideas about how to define it, but before we
find a good one, perhaps, it would still be reasonable to use "time":

In the real world, movie-like data have much more applications,
simply because we have more cameras which can take videos, and much less
device to capture 3D volume.
In 2D, most application are working on static image, and it's very
natural to treat 3D as movie by default.
For 1D signal, Keras say it's a temporal signal, and for 2D signal,
it's spatial signal. And these assumptions are not always true, it's still
useful because it matches the most common task in real-life and it's very
intuitive to understand how layers work.
Theano should be more general than Keras, but it still uses "time",
"movie" and "spatio-temporal" for 3D signal. For example, in
conv3d2d.conv3d and nnet.conv3D, they both used "time" in the docstring.
When the user understand how Keras process a movie, and it would be
very easy for them to replace the time dimension into z dimension. Plus, we
could just say it in the docstring about that.

So, I would suggest we use a more specific word to make it more easier to
understand.
If other people have better idea on this, please discuss here.
—
Reply to this email directly or view it on GitHub
https://github.com/fchollet/keras/pull/1623#issuecomment-178879472.

+1 for @fchollet's notation. I use 3D CNNs on 3D renderings where the 3rd dimension isn't time.

+1 for the "vector space" notation.
This notation would be compatible with hypothetical N-d convolutions.
We also have 3 spatial dimensions in medical imaging btw.

OK, let's use the vector space notation then. Thanks for everyone.

Didn't you guys think "first_spatial_dim" is too long?
Then I will have this in the docstring:
    # Input shape
        5D tensor with shape:
        `(samples, channels, first_spatial_dim, second_spatial_dim, third_spatial_dim)` if dim_ordering='th'
        or 5D tensor with shape:
        `(samples, first_spatial_dim, second_spatial_dim, third_spatial_dim, channels)` if dim_ordering='tf'.

In the arguments we have "nb_row", "nb_filter", and now we need to add  "nb_first_spatial_dim"?
Arguments
        nb_filter: Number of convolution filters to use.
        nb_first_spatial_dim: Number of first_spatial_dim in the covolution kernel.

Alternatives are to use "spatial_dim1", "nb_spatial_dim1", etc., or use "dim1", "nb_dim1" as a simpler notation.
What do you think?

How about keep the "rows" and "cols", and add another notation for the extra dimension? That would be more consistent with 1D and 2D case. Examples are "z_dim" if we think "rows" and "cols" are from XY plane, then the extra dimension is Z.
What do you think?

There is a reddit thread on exactly this topic:  If a 2d matrix has rows and columns, what is the third dimension in a 3d matrix?.
I like slices.

+1 for slices.

+1 for slices since they could mean time-slices or 2D slices of a 3D scan

What distinguishes the row, column, and new dimension from the batch and filter dimensions is that the convolution filter slides across these dimensions, so wouldn't it make sense to call these dimensions conv_dim1, conv_dim2, and conv_dim3?

wouldn't it make sense to call these dimensions conv_dim1, conv_dim2, and conv_dim3?

1

I have changed it to "conv_dim1" and "len_conv_dim1", if you guys have better idea, or more people vote for "slices, rows, cols", we could make another change to it.
@fchollet I think I am done, could you please do a code review if you have time, and merge it? Otherwise, I would need to follow your master repository all the time. Thanks.

I'm using this PR as a part of a project and it doesn't seem to be working with a fairly straightforward model design. Are you sure it's working as intended?

Thanks for reporting the problem. So strides are not fully tested. I just checked, I think strides is not implemented, since we used conv3d2d.conv3d.
You can see the document here: http://deeplearning.net/software/theano/library/tensor/nnet/conv.html#theano.tensor.nnet.conv3d2d.conv3d
I will put in the docstring then, sorry for the misleading.

Right that makes sense. You should probably note that this parameter is ignored and add in a check so that this doesn't happen to others.
Thanks!

@mdering I just added the support for strides by slicing the output of conv3d, it won't change the real computation, but you should get the result you want. Tell me if you have any problem, thank you.
Here is what I did:
conv_out = conv_out[:, :, ::strides[0], ::strides[1], ::strides[2]]

Will that new statement be optimized away when strides are (1, 1, 1)? Otherwise it might be good to add an if statement for that case.
conv3d2d uses 2d convs to create a 3d conv, so I wonder if it would be possible to implement strides efficiently inside conv3d2d by using strides in the conv2d calls. That would be a Theano PR though

@oeway I've tried it and it seems to be working correctly. Thanks. It would be nice to get a better fix for this upstream, since I'm running on some not so fantastic hardware, but in the meantime this is working great. Thanks again.

Hi, me again. These layers are not currently serializable to either json or yaml, because the relevant part of the get_config method is missing. Specifically you'll need to add
base_config = super(Convolution3D, self).get_config()  return dict(list(base_config.items()) + list(config.items()))
at the end of the method for this to work properly.

I think I found the problem, after deserialising, strings from the json code are becomes unicode string:
json_string = model.to_json()
model2 = model_from_json(json_string)
print model2.layers[0].border_mode, model.layers[0].border_mode

output: (u'valid', 'valid')
And in conv3d  https://github.com/Theano/Theano/blob/master/theano/tensor/nnet/conv3d2d.py#L209
isinstance(u'valid', str), isinstance('valid', str)

output: (False, True)
Maybe we should fix model_from_json() function in Keras.
However, I didn't see any problem with model_from_yaml(), do you have any problem yaml?

Would anybody like to review this PR within the next few days?

See also https://github.com/tensorflow/tensorflow/issues/150

I just tried serializing to and from yaml and received this
Exception: Layer is not connected. Did you forget to set "input_shape"

so the yaml is still not working.

Both are working for my test now, I think maybe there are other problems, could a minimum code which can reproduce the error?

@fchollet I have reviewed the code and @oeway has made the corresponding changes. Would you have time this coming week to give the code a final review and pull the PR if it is fine?

I will do my best to review and merge this week.

Looks good to me. Thanks to everyone involved!
Please address the last small comments and squash your commits into a single commit. I will merge.

Has regularization been build into this? My filter came out looking like this after I applied l2 regularization:
http://imgur.com/c6M9HLS
TL = first and second dimension
TR = second and third dimension
BL = first and third dimension
The regularization seem to force every layer to be same in the third dimension, or am I doing something wrong?

@MathiasPolfliet can you verify that your third dimension doesn't have length 1? Could you provide the relevant experiment file?

The filter have length 7x7x7. If no regularization is applied all slices look different. It is only after regularization that they look the same. I hope you don't mind the messy code:
c_Filter = [32,64]
c_Kernel = [7,7]
c_Nodes = [128,256]
c_Patch = [7,13]
c_Bias = 0.5
c_SamplesPerPatient = 200
c_Iterations = [1000,200]
model_p1 = Sequential()
model_p1.add(Convolution3D(nb_filter=c_Filter[0], len_conv_dim1=c_Kernel[0], len_conv_dim2=c_Kernel[0], len_conv_dim3=c_Kernel[0], init='normal', W_regularizer=l2(0.4), border_mode='valid', input_shape=(1,c_Patch[0],c_Patch[0],c_Patch[0])))
model_p1.add(Activation('relu'))
model_p1.add(Dropout(0.5))
model_p1.add(Flatten())
model_p1.add(Dense(c_Nodes[0],init='normal'))
model_p1.add(Activation('relu'))
model_p1.add(Dropout(0.5))
model_p1.add(Dense(1))
model_p1.add(Activation('sigmoid'))
model_p1.compile(loss='binary_crossentropy', optimizer=RMSprop(lr=0.0001, rho=0.9, epsilon=1e-06))
model_p1.fit_generator(construct_biased_dataset_generator(c_SamplesPerPatient,c_Bias,c_Patch[0],c_Images[0:2],c_Segs[0:2]),samples_per_epoch=4000,nb_epoch=c_Iterations[0],verbose=1,nb_worker=4)

For verification reasons, I have run the exact same experiment without the use of the regularizer.
http://imgur.com/j4b0rNB
As you can see, the weights of the filter seem a bit random and are not identical slice-wise.

I did a quick check, I didn't find any reason for this, I used exactly the same implementation as Convolution2D in terms of regularization.

@MathiasPolfliet make sure that your input_shape is 5-dimensional

@jruales You mean here?
model_p1.add(Convolution3D(nb_filter=c_Filter[0], len_conv_dim1=c_Kernel[0], len_conv_dim2=c_Kernel[0], len_conv_dim3=c_Kernel[0], init='normal', W_regularizer=l2(0.4), border_mode='valid', input_shape=(1,c_Patch[0],c_Patch[0],c_Patch[0])))
According to the comments in the code it should only be 4-D:
provide the keyword argument input_shape
(tuple of integers, does not include the sample axis),
e.g. input_shape=(3, 10, 128, 128) for 10 frames of 128x128 RGB pictures.

Sorry, my bad, I was thinking of the actual shape of the input tensor to the network and forgot the sample dimension is excluded from the constructor argument

As a sanity check, could you please check the following:

What is (W_conv1[0]).shape ?
Are all the values along kernel_dim3 exactly the same, or just very close to each other?
If you use W_regularizer=l2(0.0) do you obtain the same result as with W_regularizer=None?

Also please make sure you are using the latest version of the pull request's code

@fchollet I think now it's ready to merge. The ongoing discussion shouldn't be a problem of Convolution3D.

Agreed. LGTM.

32x1x7x7x7
Very close to each other after 100 iterations, exactly the same after 1000
yes

Is someone able to reproduce these findings? If not, it'll be just a bug in my internal code...

Okay, whne using two layers of convolution this problem arises in the second layer and not in the first layer. Can it have something to do with Flatten()?

I have determined what causes the behaviour. When convolving a nxnxn patch with an nxnxn filter the output is a 1x1x1, which leads to the strange behaviour. When using larger patches or smaller kernels 'fixes' it.

Convolving an nxnxn patch with an nxnxn kernel is equivalent to using a Dense layer with output size nb_filter . We could add a test to make sure these two layers output the same exact output (upon flattening) when under the same conditions.
I can't think of why the strange behavior you see would only appear when using regularization. I'm wondering if it has to do with the fact that you regularize your convolutional weights but not your dense weights? Or maybe somehow your data is repeated along the third convolutional dimension? I haven't had the chance to experiment with this in code, so I'm just throwing out ideas.

I try to insert a 3D Conv layer to a Graph model but meet some errors. Is it because something not compatible with Graph?
The code I used is shown below:
graph = Graph()
graph.add_input(name='input1', input_shape=(1, 8, 100, 100))
graph.add_node(Convolution3D(nb_filter=32, nb_depth=3, nb_row=3, nb_col=3, border_mode='same'), name='conv1', input='input1')
graph.add_node(Flatten(), name='flatten', input='conv1')
graph.add_output(name='output', input='flatten')
graph.compile(optimizer='rmsprop', loss={'output':'categorical_crossentropy'})

the error occurs when compile the graph
ValueError: get_scalar_constant_value detected deterministic IndexError: x.shape[2] when x.ndim=2. x=Subtensor{int64}.0

Hi @DeeperCS, you are using the wrong argument name, you should use:
Convolution3D(nb_filter=32, kernel_dim1 =3, kernel_dim2 =3,
kernel_dim3 =3, border_mode='same')
for simplicity, I would just write as:
Convolution3D(32, 3, 3, 3, border_mode='same')
On Wed, Mar 9, 2016 at 2:29 PM, DeeperCS<EMAIL_ADDRESS>wrote:

I try to insert a 3D Conv layer to a Graph model but meet some errors. Is
it because something not compatible with Graph?
The code I used is shown below:
graph = Graph()
graph.add_input(name='input1', input_shape=(1, 8, 100, 100))
graph.add_node(Convolution3D(nb_filter=32, nb_depth=3, nb_row=3, nb_col=3, border_mode='same'), name='conv1', input='input1')
graph.add_node(Flatten(), name='flatten', input='conv1')
graph.add_output(name='output', input='flatten')
graph.compile(optimizer='rmsprop', loss={'output':'categorical_crossentropy'})
the error occurs when compile the graph
ValueError: get_scalar_constant_value detected deterministic IndexError: x.shape[2] when x.ndim=2. x=Subtensor{int64}.0
—
Reply to this email directly or view it on GitHub
https://github.com/fchollet/keras/pull/1623#issuecomment-194296533.

@oeway
Hi~oeway~
Thank you for your reply~ I have tried your suggestions, but there are still some troubles.

For the code below:
graph = Graph()
graph.add_input(name='input1', input_shape=(3, 8, 100, 100))
graph.add_node(Convolution3D(nb_filter=32, kernel_dim1=3, kernel_dim2=3, kernel_dim3=3, border_mode='same'), name='conv1', input='input1')
graph.add_node(Flatten(), name='flatten', input='conv1')
graph.add_output(name='output', input='flatten')
graph.compile(optimizer='rmsprop', loss={'output':'categorical_crossentropy'})

I got the response below:
<ipython-input-76-7951aa957c60> in <module>()
      5
      6 #graph.add_node(Convolution3D(32, 3, 3, 3, border_mode='same'), name='conv1', input='input1')
----> 7 graph.add_node(Convolution3D(nb_filter=32, kernel_dim1=3, kernel_dim2=3, kernel_dim3=3, border_mode='same'), name='conv1', input='input1')
      8 graph.add_node(Flatten(), name='flatten', input='conv1')
      9 graph.add_output(name='output', input='flatten')

TypeError: __init__() takes at least 5 arguments (3 given)

For the other code below:
graph = Graph()
graph.add_input(name='input1', input_shape=(3, 8, 100, 100))
graph.add_node(Convolution3D(32, 3, 3, 3, border_mode='same'), name='conv1', input='input1')
graph.add_node(Flatten(), name='flatten', input='conv1')
graph.add_output(name='output', input='flatten')
graph.compile(optimizer='rmsprop', loss={'output':'categorical_crossentropy'})

I got the response below(seems the same error as what I met few minutes ago):
<ipython-input-78-ba79d46c5110> in <module>()
      8 graph.add_node(Flatten(), name='flatten', input='conv1')
      9 graph.add_output(name='output', input='flatten')
---> 10 graph.compile(optimizer='rmsprop', loss={'output':'categorical_crossentropy'})

/home/xxx/anaconda/lib/python2.7/site-packages/Keras-0.2.0-py2.7.egg/keras/models.pyc in compile(self, optimizer, loss, theano_mode)
    579             loss_fn = loss[output_name]
    580             output = self.outputs[output_name]
--> 581             y_train = output.get_output(True)
    582             y_test = output.get_output(False)
    583             y = T.zeros_like(y_test)

/home/xxx/anaconda/lib/python2.7/site-packages/Keras-0.2.0-py2.7.egg/keras/layers/core.pyc in get_output(self, train)
    488
    489     def get_output(self, train=False):
--> 490         X = self.get_input(train)
    491         size = theano.tensor.prod(X.shape) // X.shape[0]
    492         nshape = (X.shape[0], size)

/home/xxx/anaconda/lib/python2.7/site-packages/Keras-0.2.0-py2.7.egg/keras/layers/core.pyc in get_input(self, train)
     87     def get_input(self, train=False):
     88         if hasattr(self, 'previous'):
---> 89             return self.previous.get_output(train=train)
     90         elif hasattr(self, 'input'):
     91             return self.input

/home/xxx/anaconda/lib/python2.7/site-packages/Keras-0.2.0-py2.7.egg/keras/layers/convolutional.pyc in get_output(self, train)
    375                             input_shape[3] + pad_x,
    376                             input_shape[4] + pad_y)
--> 377             output = T.zeros(output_shape)
    378             indices = (slice(None), slice(None),
    379                        slice(pad_z//2, input_shape[2] + pad_z//2),

/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/tensor/basic.pyc in zeros(shape, dtype)
   2284     if dtype is None:
   2285         dtype = config.floatX
-> 2286     return alloc(numpy.array(0, dtype=dtype), *shape)
   2287
   2288

/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/tensor/basic.pyc in __call__(self, val, *shapes, **kwargs)
   2808
   2809         """
-> 2810         ret = super(Alloc, self).__call__(val, *shapes, **kwargs)
   2811         try:
   2812             # It makes optimization difficult when useless allocs are thrown

/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/gof/op.pyc in __call__(self, *inputs, **kwargs)
    609         """
    610         return_list = kwargs.pop('return_list', False)
--> 611         node = self.make_node(*inputs, **kwargs)
    612
    613         if config.compute_test_value != 'off':

/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/tensor/basic.pyc in make_node(self, value, *shape)
   2689     def make_node(self, value, *shape):
   2690         v = as_tensor_variable(value)
-> 2691         sh, bcast = self.validate_shape(shape)
   2692         if v.ndim > len(sh):
   2693             raise TypeError("The Alloc value to use has more dimensions"

/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/tensor/basic.pyc in validate_shape(self, shape)
   2681             # if s is constant 1, then we're broadcastable in that dim
   2682             try:
-> 2683                 const_shp = get_scalar_constant_value(s)
   2684             except NotScalarConstantError:
   2685                 const_shp = None

/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/tensor/basic.pyc in get_scalar_constant_value(orig_v, elemwise, only_process_constants)
    658                                 get_scalar_constant_value_elemwises):
    659                     const = [get_scalar_constant_value(i)
--> 660                              for i in v.owner.inputs]
    661                     ret = [[None]]
    662                     v.owner.op.perform(v.owner, const, ret)

/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/tensor/basic.pyc in get_scalar_constant_value(orig_v, elemwise, only_process_constants)
    770                         else:
    771                             msg += ' x=%s' % str(v)
--> 772                         raise ValueError(msg)
    773
    774                     if gp_broadcastable[idx]:

ValueError: get_scalar_constant_value detected deterministic IndexError: x.shape[2] when x.ndim=2. x=Subtensor{int64}.0

Is there some other things that I did wrong?

Not sure what's the problem, but I just tried your code, for me it can get
through the compilation without any error.
Sorry I am quite busy at the moment. Good luck.
On Wed, Mar 9, 2016 at 3:09 PM, DeeperCS<EMAIL_ADDRESS>wrote:

@oeway https://github.com/oeway
Hi~oeway~
Thank you for your reply~ I have tried your suggestions, but there are
still some troubles.
For the code below:
graph = Graph()
graph.add_input(name='input1', input_shape=(3, 8, 100, 100))
graph.add_node(Convolution3D(nb_filter=32, kernel_dim1=3, kernel_dim2=3, kernel_dim3=3, border_mode='same'), name='conv1', input='input1')
graph.add_node(Flatten(), name='flatten', input='conv1')
graph.add_output(name='output', input='flatten')
graph.compile(optimizer='rmsprop', loss={'output':'categorical_crossentropy'})
I got the response below:
 in ()
5
6 #graph.add_node(Convolution3D(32, 3, 3, 3, border_mode='same'), name='conv1', input='input1')
----> 7 graph.add_node(Convolution3D(nb_filter=32, kernel_dim1=3, kernel_dim2=3, kernel_dim3=3, border_mode='same'), name='conv1', input='input1')
8 graph.add_node(Flatten(), name='flatten', input='conv1')
9 graph.add_output(name='output', input='flatten')
TypeError: init() takes at least 5 arguments (3 given)

For the other code below:
graph = Graph()
graph.add_input(name='input1', input_shape=(3, 8, 100, 100))
graph.add_node(Convolution3D(32, 3, 3, 3, border_mode='same'), name='conv1', input='input1')
graph.add_node(Flatten(), name='flatten', input='conv1')
graph.add_output(name='output', input='flatten')
graph.compile(optimizer='rmsprop', loss={'output':'categorical_crossentropy'})
I got the response below(seems the same error as what I met few minutes
ago):
 in ()
8 graph.add_node(Flatten(), name='flatten', input='conv1')
9 graph.add_output(name='output', input='flatten')
---> 10 graph.compile(optimizer='rmsprop', loss={'output':'categorical_crossentropy'})
/home/xxx/anaconda/lib/python2.7/site-packages/Keras-0.2.0-py2.7.egg/keras/models.pyc in compile(self, optimizer, loss, theano_mode)
579             loss_fn = loss[output_name]
580             output = self.outputs[output_name]
--> 581             y_train = output.get_output(True)
582             y_test = output.get_output(False)
583             y = T.zeros_like(y_test)
/home/xxx/anaconda/lib/python2.7/site-packages/Keras-0.2.0-py2.7.egg/keras/layers/core.pyc in get_output(self, train)
488
489     def get_output(self, train=False):
--> 490         X = self.get_input(train)
491         size = theano.tensor.prod(X.shape) // X.shape[0]
492         nshape = (X.shape[0], size)
/home/xxx/anaconda/lib/python2.7/site-packages/Keras-0.2.0-py2.7.egg/keras/layers/core.pyc in get_input(self, train)
87     def get_input(self, train=False):
88         if hasattr(self, 'previous'):
---> 89             return self.previous.get_output(train=train)
90         elif hasattr(self, 'input'):
91             return self.input
/home/xxx/anaconda/lib/python2.7/site-packages/Keras-0.2.0-py2.7.egg/keras/layers/convolutional.pyc in get_output(self, train)
375                             input_shape[3] + pad_x,
376                             input_shape[4] + pad_y)
--> 377             output = T.zeros(output_shape)
378             indices = (slice(None), slice(None),
379                        slice(pad_z//2, input_shape[2] + pad_z//2),
/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/tensor/basic.pyc in zeros(shape, dtype)
2284     if dtype is None:
2285         dtype = config.floatX
-> 2286     return alloc(numpy.array(0, dtype=dtype), *shape)
2287
2288
/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/tensor/basic.pyc in call(self, val, *shapes, **kwargs)
2808
2809         """
-> 2810         ret = super(Alloc, self).call(val, *shapes, **kwargs)
2811         try:
2812             # It makes optimization difficult when useless allocs are thrown
/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/gof/op.pyc in call(self, *inputs, **kwargs)
609         """
610         return_list = kwargs.pop('return_list', False)
--> 611         node = self.make_node(*inputs, **kwargs)
612
613         if config.compute_test_value != 'off':
/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/tensor/basic.pyc in make_node(self, value, *shape)
2689     def make_node(self, value, *shape):
2690         v = as_tensor_variable(value)
-> 2691         sh, bcast = self.validate_shape(shape)
2692         if v.ndim > len(sh):
2693             raise TypeError("The Alloc value to use has more dimensions"
/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/tensor/basic.pyc in validate_shape(self, shape)
2681             # if s is constant 1, then we're broadcastable in that dim
2682             try:
-> 2683                 const_shp = get_scalar_constant_value(s)
2684             except NotScalarConstantError:
2685                 const_shp = None
/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/tensor/basic.pyc in get_scalar_constant_value(orig_v, elemwise, only_process_constants)
658                                 get_scalar_constant_value_elemwises):
659                     const = [get_scalar_constant_value(i)
--> 660                              for i in v.owner.inputs]
661                     ret = [[None]]
662                     v.owner.op.perform(v.owner, const, ret)
/home/xxx/anaconda/lib/python2.7/site-packages/Theano-0.8.0.dev0-py2.7.egg/theano/tensor/basic.pyc in get_scalar_constant_value(orig_v, elemwise, only_process_constants)
770                         else:
771                             msg += ' x=%s' % str(v)
--> 772                         raise ValueError(msg)
773
774                     if gp_broadcastable[idx]:
ValueError: get_scalar_constant_value detected deterministic IndexError: x.shape[2] when x.ndim=2. x=Subtensor{int64}.0

Is there some other things that I did wrong?
—
Reply to this email directly or view it on GitHub
https://github.com/fchollet/keras/pull/1623#issuecomment-194310463.

@oeway Thank you so much. I used the #MinhazPalasara/keras at the beginning, and I just checked that 3D convolution stuffs have merged to the master version, so I installed the latest keras 0.3.2 and tried again, unfortunately, these errors still around me - -！
Besides, if I changed the 3D conv layer to 2D conv and decrease the input dim, it can get through the compilation, so I doubted that there is some thing wrong between Graph and 3D conv layer.
Can you give me a example code that works well or list your platform information (e.g. versions of os, python, theano and keras) at your convenience ^ ^

Hi, I am using Ubuntu 14.04LTS, here some other information:
import sys
print sys.version
import keras, theano
print keras.version
print theano.version
2.7.6 (default, Jun 22 2015, 17:58:13)
[GCC 4.8.2]
0.3.2
0.8.0.dev0.dev-4800a51cf2c1e5504bfeddb25c980c61bba90809
On Thu, Mar 10, 2016 at 6:07 AM, DeeperCS<EMAIL_ADDRESS>wrote:

@oeway https://github.com/oeway Thank you so much. I used the
#MinhazPalasara/keras at the beginning, and I just checked that 3D
convolution stuffs have merged to the master version, so I installed the
latest keras 0.3.2 and tried again, unfortunately, these errors still
around me - -！
Besides, if I changed the 3D conv layer to 2D conv and decrease the input
dim, it can get through the compilation, so I doubted that there is some
thing wrong between Graph and 3D conv layer.
Can you give me a example code that works well or list your platform
information (e.g. versions of os, python, theano and keras) at your
convenience ^ ^
—
Reply to this email directly or view it on GitHub
https://github.com/fchollet/keras/pull/1623#issuecomment-194668446.

@oeway Thanks a million~!~!~!~ I use 'conda create' to set a new environment and reinstalled theano-0.8.0 and keras-0.3.2 which make the graph with 3d conv layers works now, I thought the main reason is that I was using a wrong version of keras...

Convolutions are arriving in TF with https://github.com/tensorflow/tensorflow/pull/1571/commits/784778e76d3f5fd75b9ec99138770795907f1616

3d convolution and pooling are available now in Tensorflow with https://github.com/tensorflow/tensorflow/commit/6a187ccddaebb741ea77fc3201c6e36625f0aadb. Do you want update this?

Thanks for the reminder, I would look into that later.
On Thu, May 5, 2016 at 8:05 AM, bhack<EMAIL_ADDRESS>wrote:

3d convolution and pooling are available now in Tensorflow with
tensorflow/tensorflow@6a187cc
https://github.com/tensorflow/tensorflow/commit/6a187ccddaebb741ea77fc3201c6e36625f0aadb.
Do you want update this?
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/fchollet/keras/pull/1623#issuecomment-217082476


DSND-2646: Persist capital date as an ISO_INSTANT

Describe the changes

Persists capital date fields as an Instant instead of String.
Added capital mapping to GET response.

Related Jira tickets
DSND-2464
Developer check list
General

[ ] Is the PR as small as it can be?
[ ] Has the Jira ticket been updated with any relevant comments?
[ ] Is the README up to date?
[ ] Has the POM been updated for the latest versions of dependencies?
[ ] Has the code been double-checked against main?
[ ] Does the code adhere standards in this repository, including SonarQube checks?

Testing

[ ] Do the code changes have unit and integration tests with code coverage > 80%?
[ ] Has the code been tested locally and/or passed the API Karate tests on Tilt?
[ ] Have mandatory manual test cases been run?
[ ] Are extra Karate tests required?
[ ] Are all the test data resources up to date?

Release preparation
Where possible, add links to the respective Jira tickets when an item is checked

[ ] Have these changes been included in a release ticket?
[ ] Are changes required to the docker-chs-development deployment or test data?
[ ] Are any changes required to the environment added to deployment repositories?

Notes to the tester
Add any testing hints or instructions here.

CI: No security warnings found


chore: add tests for function deploy

What kind of change does this PR introduce?
tests
What is the new behavior?

refactored deno path to utils
pass in cancellation context
extracted string and regex constants
switch to codegen api client

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 68 (0.0%)  changed or added relevant lines in 3 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.4%) to 33.253%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

cmd/functions.go
0
2
0.0%

internal/utils/misc.go
0
25
0.0%

internal/functions/deploy/deploy.go
0
41
0.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.4%

Covered Lines:
1662

Relevant Lines:
4998

💛  - Coveralls


Support auth migration using port alias

What this PR does / why we need it:
This PR provides mechanism to create 'alias' port with different auth policy via service annotation. This is mainly used to upgrade Istio security (to mTLS) without downtime (assuming "envoy multiplex", i.e a feature that allows envoy to handle both mTLS and plaintext on the same port, is not available).
For more details, see doc
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
Special notes for your reviewer:
Release note:

@diemtvu: Adding do-not-merge/release-note-label-needed because the release note process has not been followed.

One of the following labels is required "release-note", "release-note-action-required", or "release-note-none".
Please see: https://github.com/kubernetes/community/blob/master/contributors/devel/pull-requests.md#write-release-notes-if-needed.
Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@diemtvu we should have this as WIP and enable this when @PiotrSikora completes the fallback option in Envoy. So that we can actually test this.

I suggest keeping this open with a [WIP] and [skip ci]

This actually doesn't require multiplex. This is the super slow approach as discussed in the doc. I can add e2e test once we agree this approach (create alias port) is reasonable and doesn't create any side effect.

I didn't realize I closed this by accident. Reopen.


[DataFactory] Enable Avro Dataset in SDK

Can one of the admins verify this patch?

@yuzhangyi  , please:

Provide the swagger spec PR link related to this PR
Please follow the instruction

@yuzhangyi  , please:
Provide the swagger spec PR link related to this PR
Please follow the instruction

The public swagger is here: https://github.com/Azure/azure-rest-api-specs/pull/6560

@yuzhangyi , is this PR still valid?

I do not see any update for this comment:

Please follow the instruction

@yuzhangyi , is this PR still valid?
I do not see any update for this comment:
Please follow the instruction

Hi, @erich-wang , this PR is still valid, for related change in public swagger, I have added the link in comment. For the link "instruction", I cannot open it with 404 error, so can you help give the guidance about what I need to do? Thanks

@yuzhangyi , is this PR still valid?
I do not see any update for this comment:
Please follow the instruction

Hi, @erich-wang , this PR is still valid, for related change in public swagger, I have added the link in comment. For the link "instruction", I cannot open it with 404 error, so can you help give the guidance about what I need to do? Thanks

@yuzhangyi , the link points to private repo, you should login first and you github account should  have joined Azure organization.

Close this PR firstly and will use newest public swagger to generate SDK in next week.


works with ipywidgets7rc0

This may break some themes, like the default Jupyter theme that ipywidgets uses. In that case, we were using the default embed url, which does not require requirejs, since requirejs broke the page.
Can we add a config option for this plugin to choose between the requirejs and standard embedding, and switch the includes based on that? For the standard widget docs, I'd set that config to not be the requirejs option.

@maartenbreddels  and @jasongrout what are status of this PR?

I believe stylesheets can be added in sphinx, I don't think this is the place to add this.
Apart from that I'm ok with someone merging this ;-)
I've been using it for ipyvolume succesfully.

I've been using it for ipyvolume succesfully.

yes, I tried your branch and work nicely with nglview too.

This looks good to me. Any blocker on your side @jasongrout?

I don't recall any outstanding objection.

If everyone is OK with this, would it be OK to merge and release? :)

better squash this

@jasongrout / @SylvainCorlay I think only owners have merge right on this repo. Would you consider merging as both of you seem to think it is good to go?

Wow, I thought that this was merged. Doing this now.

Any block for a 0.1.2 release?
@maartenbreddels

The only issue I know of is a css issue. If you open http://ipyvolume.readthedocs.io/en/latest/ you can see the text starts grey, then when the css is loaded turns blue. I've seen this also on other rtd pages, so it's not ipyvolume specific, but i'm not sure it has anything to do with jupyter-sphinx.
Otherwise, I'd say go for it.

@maartenbreddels I think that is unrelated to this repo: https://github.com/jupyter-widgets/ipywidgets/issues/1673 Once that gets fixed it should propagate directly, right?

Ah yes, lost track of that, so unrelated.

@SylvainCorlay Considering that the issue Maarten mentioned was unrelated to this package and that he has used this in production by linking his branch I think it should be ready for a 0.1.2 release.

ok, making the release now.


Update TeX-FPC and give credit to CTAN resp. Wolfgang Helbig

Hi Alexey,
the version on CTAN has been updated since creation of your repository here, see files on CTAN. So it would be nice, if you could update your repo to the current state. Already knowing about TeX-GPX and having had contact with Wolfgang Helbig some years ago, I checked CTAN after stumbling over your repo and having seen that the Readme.md was "signed" by Wolfgang Helbig  ...
regards,  Bernhard

we have a problem to update it. I cannot download one file with all content (so I must download all small files 1 by 1).
give me the entire TeX-FPC in one file, pls?

Head over to https://ctan.org/pkg/tex-fpc and click on "Herunterladen"/"Download".

I append it here:
tex-fpc.zip

Updated. if ok, close the issue.


Fix datatables overflow on mobile resolutions. Issue #42

Thanks for your supporting, but don't need to make like this. #42


Strip excess newlines

Running tl on arbitrary lua files produced a large number of excess
newlines.
This commit runs the generated lua string though a regex substitution
that will replace occurrences of two or more newlines with two newlines.
The output is a bit denser and looks closer to human-written code.

@ashn-github the new lines are kept on purpose (in fact, the pretty-printer goes through some pains to preserve them). The goal is to keep the line numbers of the generated .lua file to match your input .tl file. This is done so that runtime errors produce backtraces with the correct line numbers.

Since I expect the programmer writing a .tl to maintain that one as their point of reference (and not the generated .lua file, which is more of a build artifact), I think it's more important to keep the backtrace truthful to the input than to compress the output .lua file (there are projects out there to minify Lua code if one doesn't care about preserving backtraces).
So, I'm closing this PR for now since we won't be merging this, but thank you for the contribution! Do keep providing feedback on tl, it's much appreciated!


Ixia Issue : Context deadline exceeded error if StartProtcols called immediately after PushConfig.

Hi,
If StartProtocols is called immediately after PushConfig , context deadline exceeded error is found.
Currently we are providing 30 sec sleep after Pushconfig and StartProtocols to avoid this issue.
We need fix for this issue to avoid sleep time.
actions_MED_LocPref_prepend_flow_control_test.go:441:
*** Pushing config to <IP_ADDRESS>:40051...
actions_MED_LocPref_prepend_flow_control_test.go:442:
*** Starting protocols on <IP_ADDRESS>:40051...
actions_MED_LocPref_prepend_flow_control_test.go:442: StartProtocols(t) on ATE{Dims:Dims{Name:<IP_ADDRESS>:40051 Vendor:VENDOR_UNSPECIFIED HardwareModel: SoftwareVersion: Ports:map[port1:Port{Name:<IP_ADDRESS>;1;5 Speed:SPEED_UNSPECIFIED CardModel: PMD:PMD_UNSPECIFIED} port2:Port{Name:<IP_ADDRESS>;1;1 Speed:SPEED_UNSPECIFIED CardModel: PMD:PMD_UNSPECIFIED} port3:Port{Name:<IP_ADDRESS>;7;7 Speed:SPEED_UNSPECIFIED CardModel: PMD:PMD_UNSPECIFIED} port4:Port{Name:<IP_ADDRESS>;7;8 Speed:SPEED_UNSPECIFIED CardModel: PMD:PMD_UNSPECIFIED}]}}: {
"code":  4,
"errors":  [
"error on request {\ncontrol_state:  {\n  choice:  protocol\n  protocol:  {\n    choice:  all\n    all:  {\n      state:  start\n    }\n  }\n}\n}: context deadline exceeded"
]
}
Thanks,
Prabha

@cprabha can you help with providing the containers logs for this issue? Please also share the test link.
Instructions on how to collect it is here:

https://github.com/open-traffic-generator/otg-examples/tree/main/hw/ixhw-b2b#diagnostics

Also which DUT are you running your test against?

we're not able to reproduce this while trying to run that test (RT-1.32). Our test fails during dut configuration phase

Hi,
Issue is seen even with 30 seconds  delay.
please try running below script.
https://github.com/openconfig/featureprofiles/tree/main/feature/experimental/gribi/otg_tests/vrf_policy_driven_te
Thanks,
Prabha.

Issue is resolved after updating binding file otg timeout value.


Quick fix for bender et al contamination issue - files will be removed next week.

OAS will be changed as bender_2020 heavy chains are contaminated.
This will break KA search when these files are deleted.
The solution was to read in a blank csv and provide a warning.
If you have a better idea let me know. My only concern is when the data frames with NaNs are concatenated with rows that do not contain NaN values.
Let me know.

FYI - users will get this warning:
"FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation.
test = pd.concat([results, df])"


python3Packages.pymazda: 0.1.5 -> 0.1.6

Motivation for this change
Update to latest upstream release 0.1.6
Change log: https://github.com/bdr99/pymazda/compare/0.1.5...0.1.6

Related Home Assistant change: -
Target Home Assistant release: current (only dependency update, no code changes)

Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Added a release notes entry if the change is major or breaking
[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 124475 run on x86_64-linux 1

3 packages built:

home-assistant
python38Packages.pymazda
python39Packages.pymazda

Result of nixpkgs-review pr 124475 at 9fa8d245 run on aarch64-linux 1

3 packages built successfully:

home-assistant
python38Packages.pymazda
python39Packages.pymazda


Just to let you know

I installed the software, and it works fine with SteelSeries Apex 5 Keyboard and with Rival 710 mouse.
Is there any chance for a future update allowing full control of all rgb supported devices like Ram, AIO, Fans ?

Glad to hear it works for you and it would be great with added support for more devices.
Ideally someone who owns them, would fork this repo, make it work and merge back.
If I had the hardware I would look definately look into it.


Don't call on_disconnect if connlib can't reconnect to the portal

For explanation see this comment. Otherwise a client that loses internet connectivity for a few minutes will be signed out. This will cause problems with clients operating headlessly.
Instead, we should only call on_disconnect(errorString) if there's been a fatal error that absolutely requires signing out and clearing the token.

I think if we just keep retrying every 5 seconds, we don't need to detect internet connectivity changes in the clients (except the Apple client, which requires this because of its DNS limitations).
The end-user experience here would then be:

The Firezone client is always "Signed in". Only on_disconnect(errorString) is called for 4xx errors and non-recoverable errors, like if the tunnel interface can't be created. For those cases, we sign the user out.
If internet connectivity for a client drops temporarily (switching networks), Firezone will remain "Signed in". When connlib reconnects, it calls getSystemDefaultResolvers again and re-initializes the interface.
If internet connectivity drops completely for a long time (use-case: clients with spotty cellular connections), Firezone is still signed in with connlib retrying in the background (every 5 seconds)

The edge case here is if the DNS servers change for an interface without any other network connectivity changes. If connlib doesn't detect portal connectivity loss, it won't retry getting the DNS servers. For this edge case, we can handle it one of two ways:

Either the OS detects these "network changes" and disconnects and reconnects connlib, keeping the token, OR
Connlib occasionally calls getSystemDefaultResolvers in the background on, say a 5-second timer. If they're different, it re-initializes the resolver to use these new ones.

Option (2) I think this eliminates the need (for quite some time) for clients to rely on OS-level APIs to "detect network connectivity" and respond, because that requires a complex state machine like the one we have in the Apple client. It means the client will need to handle a "Signed in but not connected" state which we probably need to reflect in the UI as well.
cc @conectado @ReactorScram

https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/
Just wanted to say if the portal goes down or for some reason a customer loses Internet and like 500 clients try to reconnect all at once, some jitter might help reduce the thundering herd effect on the portal server
Had it happen at a previous job where a customer would lose power and 100-ish IoT devices would all boot up exactly X seconds later and then all hammer the server at the same time.


What should the application do if a message can't be sent?

Currently, if redis is down, the mail that is supposed to go out won't go out. We will log in honeybadger the fact that redis was down, but we will lose the message and someone won't get an email. How do we want to handle this?

A first step could be to make sure we log enough information to know what project the error was for.

Short-term: log errors in Honeybadger with a custom error type and enough information to know what project the error was raised for, and check on these as part of runner tasks in RDSS, especially if/when Redis goes down.
Keep in mind how often Redis goes down.

It looks like this has been done already: https://github.com/pulibrary/tiger-data-app/blob/main/app/controllers/projects_controller.rb#L21C5-L44C35


Configure Backstage Github Events Support

https://backstage.io/docs/integrations/github/discovery#events-support

Closing to focus on the tooling instead of the platform.


Add crowd.Bible tests to workflow

Piyush is writing tests in crowd.Bible.

We need those tests run on merges to dev branch using GitHub Actions. I assume we should create a docker env in GA and run the tests on the newly created env in GA.
We also need to run the tests on the actual AWS dev env. After GA runs the tests in the temporary GA env, run the tests against the AWS dev env.

Latest PR from Piyush: https://github.com/etenlab/crowd.Bible/pull/60


test(client): test JS PlanetScale connector

WIP
Based on https://github.com/prisma/prisma/pull/20507
Related
https://github.com/prisma/team-orm/issues/275
https://github.com/prisma/team-orm/issues/258

Would it make sense to extract the refactored helpers and _schema.ts into a separate PR for easier review?

@millsp 🤔 it could be tricky and I need to focus on more testing, so I won't try anything on that side

The story continues here https://github.com/prisma/prisma/pull/20991


Optimize flash patterns

Just as a record.
We found the flash pattern might randomly got doubled duration when the frequency is approaching the limit(ex. 500Hz with 512 oversampling for 7 sources and 8 LED channels) on Revision 1, but the firmware won't fail out nor warn the user. We suspect it was because the CPU missed interrupt when it's close to the limitation, so some led_loop was skipped which results a doubled duration.
This symptom is not shown on Revision 2, as it's uses hardware oversampling which significantly reduced the CPU load, and the maximum frequency for ADC will be reached before the CPU gets to its limitation.
We don't have plan to optimize this unless it shows affects on further revisions(>=2).

In current flash mode, we flash the chosen LEDs in a fixed sequence and sample them interspersely, so compare with continuously mode, the maxmium frequency allowed for flash mode will be inversely proportional to number of LEDs enabled. A typical maximum value will be: 2760Hz with 512 oversample and 2 LEDs enabled, or 1380Hz with 512 oversample and 4 LEDs enabled, etc
Reflective/transmissive mode doesn't impact the configurations, and there is a roughly 6-10us delay for SPI transmission

which is about 1-2 CPU cycles, and it shouldn't affect the whole sampling period.

It seems there is no further issue with flash pattern now, so close it


Transitions with multiple inputs are not activated

When a transition has inputs from more than one place, it is not activated even if there are tokens in each place.
Please see attached .work file for an example

andrikos at 2010-07-09T17:52:28Z:

rotsor at 2010-07-10T16:19:15Z:
Hehe, you have two overlapping places there, one containing two tokens, and the other containing none.
All the arcs are connected to the place without tokens.

andrikos at 2010-07-11T06:48:10Z:
Oops, you are right, sorry for the mess :$


Replace UglifyJS with Terser

Detailed description
UglifyJS is unmaintained. This PR replaces the uglify_js filter with a terser filter, which does essentially the same.
The new filter should be backwards compatible. The old filter identifier, uglify_js, is kept for that reason.
Additionally, this PR has some development improvements regarding the JavaScript implementation. This PR switches the JavaScript implementation to use mini_racer rather than libv8, which makes it work on macOS by default. It also makes the duktape gem obsolete.
To do

[x] Tests
[x] Verify backwards compatibility

Related issues
Fixes #1578.

@ddfreyne I incorrectly assumed such update was available in last nanoc release.
I checked too quickly when updating my gems, saw an update of nanoc and considered it was the one I wanted 😅.
Do you have an ETA for the next release to include the :terser filter availability?
(Obviously, no pressure at all!)

@opatry I’ll get a release out today!

Done. This is released in 4.12.7.


Compatibility with BitBucket (Stash) version 4.0

Make compatible SonarQube Stash plugin with Stash (BitBucket) version 4.0.
Current plugin supports version 3.x.
We remind Stash has been renamed as BitBucket with version 4.0.

Tested through issue #38.
We need to update README file according to this outcome.


feat: adds escaping variables by default

Codecov Report

Merging #7 (263f563) into master (8def5a1) will increase coverage by 0.02%.
The diff coverage is 75.00%.

:exclamation: Current head 263f563 differs from pull request most recent head 11ed145. Consider uploading reports for the commit 11ed145 to get more accurate results

@@            Coverage Diff             @@
##           master       #7      +/-   ##
==========================================
+ Coverage   75.57%   75.60%   +0.02%
==========================================
  Files          11       11
  Lines         524      541      +17
==========================================
+ Hits          396      409      +13
- Misses        128      132       +4

Impacted Files
Coverage Δ

lib/src/options.dart
61.78% <58.33%> (-0.16%)
:arrow_down:

lib/interpolator.dart
92.15% <100.00%> (+1.24%)
:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


EntityEquality generates type mismatch

Test: Contains_over_entityType_with_null_should_rewrite_to_identity_equality
                var query
                    = context.Orders.Where(o => o.CustomerID == "VINET")
                        .Contains(null);

bool Contains(
|__ source: IQueryable<Order> Where(
|   |__ source: DbSet<Order>,
|   |__ predicate: (Order o) => o.CustomerID == "VINET"),
|__ item: (Unhandled parameter: __p_0))

// Gets rewritten into

bool Contains(
|__ source: IQueryable<int> Select(
|   |__ source: IQueryable<Order> Where(
|   |   |__ source: DbSet<Order>,
|   |   |__ predicate: (Order Param_0) => Param_0.CustomerID == "VINET"),
|   |__ selector: (Order Param_0) => Property(Param_0, "OrderID")),
|__ item: (Unhandled parameter: __entity_equality_p_0_OrderID))

here p0 is null so runtime generated parameter is also null. But value is being cast to int which is NRE. When rewriting key access on potentially null expression, it should convert the types to nullable.

Another case
IQueryable<Customer> Where(
|__ source: DbSet<Customer>,
|__ predicate: (Customer c) => Order FirstOrDefault(IOrderedEnumerable<Order> OrderBy(
    |__ source: c.Orders,
    |__ keySelector: (Order o) => o.OrderID)) == Order 10243)

Sorry, my github email notifications are clearly not working well - missed this. Will check it out as soon as the preview7 port is done.

@smitpatel I see the issue you're describing. However, the first test (Contains_over_entityType_with_null_should_rewrite_to_identity_equality) passes correctly - do you have another repro in mind for relational?
Will investigate the second failure.
Regardless, will wait on #16791 to get merged to avoid conflicts.

InMemory failure. The test is disabled for InMemory provider with this bug number.


[ENHANCEMENT] Adding user type in frontend

Description
Right now frontend does not have a user type in the types folder. User types need to be added in the types folder for better type checking. and prevent future bugs.
Current Behavior
No user type in frontend types folderthe folder
Proposed Enhancement
add a user type on frontend type folder and use it everywhere.
Use Case
This will prevent future bugs and clean code format.
Blockers
No response
Additional Information/ Attachments
No response

Hello @krishnaacharyaa 🤗
Please do assign me this issue under gssoc'24
Thank you

@krishnaacharyaa I have made the PR
Please do review it.

@krishnaacharyaa  I have made the PR
Please do review it.
Sorry 😅 for the delay, I was doubtful as to if the type should be applied for the mongo schema as well.

@krishnaacharyaa I have made the fix, please do review it


Fixed hang in Device Manager on unexpected connection close

-- Fixed a logic bug in the WeaveDeviceManager C++ class that resulted in the application’s on-error callback never being called when a connection to a device closed unexpectedly during a connect or rendezvous operation.  In the observed scenario, a spontaneous close of a newly established Weave connection just as a security session was starting would result in the Device Manager ignoring the HandleSessionError() callback from the WeaveSecurityManager, but only in the case where it had received a HandleConnectionClosed() callback prior to the HandleSessionError().
The ordering of the two events depends very much on the behavior of the underlying TCP stack.  Indeed, the situation may only be prevalent on mobile devices, where the network stack immediately invalidates TCP sockets when there is a change in WiFi/Cellular connectivity.
The fix was loosen a state check such that the session establishment error is process correctly regardless of the ordering of the two events.
This fix relates to the problem reported in buganizer issue 135032367.

@mburshteyn1 Actually, no.  The root problem in OBS-5981 was that the code in HandleSessionError()
called con->Closed() in the case where mConState == kConnectionState_NotConnected.  Since con->Close() had already been called in HandleConnectionClosed(), this caused the connection reference count to go below zero.
Note that, even though HandleSessionError() can now go on to call devMgr->Close() in the case where mConState == kConnectionState_NotConnected, that path will never result in the connection object being double-closed because mDeviceCon has been set to NULL in HandleConnectionClosed().

@jaylogue got it, thanks for the explanation!


Logo design proposal

Hi, my name is Valeria. I'm a graphic designer and I'd like to collaborate on your open source project and propose free logo/icon design for your app. If it's something you're interested in, please, let me know!
Best,
Valeria

@teneresa
Hey, Valeria
I‘m so pleased that we can collaborate on the project. It would be appreciated if you could design a logo for our project.
Feel free to make any pull requests.

Zan means 👍 (like) in Chinese. No special requirements at all!

Ok, I'll prepare some options soon :)

Hi Mickey, here is an option for the logo. This is a combination of "like" symbol and server icon. Please, let me know what do you think?

Hi Mickey, here are a color option and icon variations.

Wow~ That's amazing! You've done a great job! I'd like to use it in the README and the website!

Mickey, I'm happy to hear this, thank you! I uploaded all the required files and created a pull-request. Please merge it if everything is Ok :)


Refined Principals

@JensGroth and @robin-kunzler have noticed that users often fail to consider the different sorts of principals when doing authentication, e.g. unintentionally creating accounts for anonymous principals.
Perhaps we should consider projecting the IC principals into a more refined Motoko (variant?) type, forcing users to consider all the possible cases when handling principals, much as option types force users to explicitly deal with nulls.
One backwards compatible way to do this might be to extend the context type (with the current caller : Principal field) to have a second field of the more refined caller type and deprecate the old caller field, or just refine Principal into a more concrete type and perhaps warn when testing equality at type Principal.

Well, Principal is already a tagged type, right? So what we need is something like
type PrincipalRole = Anonymous | User | ...
type ICPrincipal (role : PrincipalRole) = <magic>
type UserPrincipal = ICPrincipal User

and convince developers to only open accounts for UserPrincipals :-)
Then we would also need some switch caller { case user : UserPrincipal; _ } GADT-like pattern matching to make this work.

It goes against the (well, my) philosophy of a “mostly opaque” principal (and what's so wrong if a canister, or “the general public” a.k.a. the anonymous user can open an account). So I'll state my opposition here for the record, but otherwise will shut up :-)

The problem is that some person A, accidentally logged in as anonymous and creates an account with sensitive information (e.g. telephone number or birthday). Then another user B makes the same mistake and suddenly sees the profile of A. Leaking out information is bad.

But isn't that a frontend problem? Why can you even create an account when you aren’t logged in?

Thanks for creating the issue and for the discussion!

The problem is that some person A, accidentally logged in as anonymous and creates an account with sensitive information (e.g. telephone number or birthday). Then another user B makes the same mistake and suddenly sees the profile of A. Leaking out information is bad.

I agree with @ggreif.

But isn't that a frontend problem? Why can you even create an account when you aren’t logged in?

@nomeata : I agree that one would expect a frontend not to create an account unless the user has logged in (e.g. using Internet Identity). However, from a security perspective IMO such a property should be enforced also in the canister. The canisters are not necessarily always used with a front-end. And essentially in every canister we've reviewed, the developers do not restrict account creation to self-authenticating principals. So I think it would be good if the dev is forced to choose what kind of principal they want when reading the caller.
Please see also the security best practice for context where this discussion comes from.

And essentially in every canister we've reviewed, the developers do not restrict account creation to self-authenticating principals

But that’s a good thing, at least partly! One of the (many) visions for the Internet Computer is the “strong canister principle”. Anything that you can do from the outside with the IC you should also be able to do from a canister. For automation, mash-ups, etc.
If we now encourage developers to restrict access to those principals that correspond to external users, they will (intentionally or accidentially) prevent canisters from using their services.
The fact that principals even have structure, and are not completely opaque, was a hacky work-around the fact that we didn’t want to store the data elsewhere (for decentralization etc.), now we face the consequences. Hence my resistence. . Maybe @rossberg  can phrase these concerns more convincingly. Or maybe that ship has sailed anyways…

@nomeata : Thanks, see your point. But there are security issues around this, such as what @ggreif describes:

The problem is that some person A, accidentally logged in as anonymous and creates an account with sensitive information (e.g. telephone number or birthday). Then another user B makes the same mistake and suddenly sees the profile of A. Leaking out information is bad.

Is this no issue from your perspective? If you agree this is an issue, do you have suggestions on how one could address it in a different way than proposed here?

I could accede that the “anonymous” principal is more special, and expose the caller as ? principal – this preserves the idea that a canister should not care about who is invoking it, but may care about if there is someone.
But I indeed do not think it’s a serious issue. The premise of “logged in as the anonymous user” is already flawed – if your front-end has a notion of “logged in”, but doesn’t actually log you in or fails to sign the requests, then… it just seems buggy. But I understand the sentiment to make such bugs easier to detect.
(There was a time before we had anonymous requests, and one possible design then would have been to keep the system simple, and leave it completely to agents to still sign such requests, possibly with a random, or a particular hard-coded key. If we’d taken that route, probably nobody would have thought of bothering the canisters with the concept.)

I could accede that the “anonymous” principal is more special, and expose the caller as ? principal – this preserves the idea that a canister should not care about who is invoking it, but may care about if there is someone.

@nomeata : I like your suggestion (just addressing the special case of the anonymous principal), since it solves the concern from my perspective. WDYT @crusso ?

But I indeed do not think it’s a serious issue. The premise of “logged in as the anonymous user” is already flawed – if your front-end has a notion of “logged in”, but doesn’t actually log you in or fails to sign the requests, then… it just seems buggy. But I understand the sentiment to make such bugs easier to detect.

I agree, the issues we saw around this were usually 'low' risk or sometimes 'medium' risk. The reason for bringing this up is not the severity of the individual issues, but the fact that we have seen issues around this multiple times. I also agree that such a frontend would be buggy (and we have seen such bugs!). I think the above solution improves the situation as a 'defense in depth' mechanism:

It helps to guarantee that such a buggy frontend would not work as the developer would be nudged to forbid using the anonymous principal to e.g. create accounts on the canister side.
This will also hold if other canisters or systems (no web frontend) interacts with the canister.


remove l8 breaking code

WHY
BEFORE - What was wrong? What was happening before this PR?
Reported here, stackoverflow etc.
Str::lcfirst() is not available at L9.
AFTER - What is happening after this PR?
we use the native PHP function.
Is it a breaking change or non-breaking change?
nop
How can we test the before & after?
try building a CRUD in L8 project.

I still have this error when using command line custom operation builder: https://backpackforlaravel.com/docs/5.x/crud-operations#creating-a-custom-operation-1
I am using backpack/crud 5.4.16 + backpack/generators 3.3.13 / laravel/framework v8.83.27
php artisan backpack:crud-operation myOperation

   BadMethodCallException

  Method Illuminate\Support\Stringable::lcfirst does not exist.

  at vendor/laravel/framework/src/Illuminate/Macroable/Traits/Macroable.php:113

The bug is saved. I am using laravel 8.12 and backpack/crud 5.0
root@c29d571eee5b:/var/www# php artisan backpack:crud-operation OperationName
 BadMethodCallException

 Method Illuminate\Support\Stringable::lcfirst does not exist.

 at vendor/laravel/framework/src/Illuminate/Macroable/Traits/Macroable.php:113
   109▕      */
   110▕     public function __call($method, $parameters)
   111▕     {
   112▕         if (! static::hasMacro($method)) {
 ➜ 113▕             throw new BadMethodCallException(sprintf(
   114▕                 'Method %s::%s does not exist.', static::class, $method
   115▕             ));
   116▕         }
   117▕

 • Bad Method Call: Did you mean Illuminate\Support\Stringable::ucfirst() ?

     +16 vendor frames
 17  artisan:37
     Illuminate\Foundation\Console\Kernel::handle(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))


Flow sensitive assumptions on ivars can be invalidated

Input
→ View on sorbet.run
# typed: strict
class A
  extend T::Sig

  sig {void}
  def initialize
    @foo = T.let(nil, T.nilable(String))
  end

  sig {void}
  def set_nil
    @foo = nil
  end

  sig {void}
  def set_str
    @foo = "not nil"
  end

  sig {void}
  def wrong_error
    if @foo
      set_nil # in the real world, this could be something that sometimes set @foo to nil,
              # which warrants a check below.
      unless @foo
        puts "reachable in real life, unreachable in Sorbet"
      end
    end
  end

  sig {void}
  def missing_error
    if @foo
      set_nil
      puts @foo.succ # in the vm, this calls `succ` on `nil`
    end
  end
end

Observed output
editor.rb:26: This code is unreachable https://srb.help/7006
    26 |        puts "reachable in real life, unreachable in Sorbet"
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Errors: 1

Expected behavior
No method error on the succ call.
This issue was discussed on Slack a while ago but I don't see a ticket for it so I thought I would make one.

SEO instance variable variables alias


As a developer, I want to create a formula for determining a disadvantaged community, so that this data can be visualized on a map and included in score download files.

Description
The system needs to have logic for identifying disadvantaged communities.
Formula:

Calculate the total population across all census block groups.
Calculate 25% of the total population across all census block groups.
Rank the census block groups in descending order based on their score.
Calculate the cumulative sum of the population in the census block groups.
For all census block groups in the top 25% of the total population with the highest scores, mark these census block groups as "disadvantaged communities".
For all census block groups in the top 25.1% to 40% of the total population with the highest scores, mark these census block groups as "on the cusp of disadvantaged communities"
For the purposes of iterating on this score formula, it should be easy to change these thresholds (mentioned in #5 and #6 above) so that we could change the thresholds and regenerate the results.

Note that this work is dependent on having the census block group population data which is in card #158
Solution
A clear and concise description of the solution.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Links to user research or other resources
Tasks
Definition of "Done"

[ ] other USDS team members and CEQ can see the results of this designation in some way (on the map is ideal, in a data export of some kind is OK)
[ ] we can change the threshold % ranges for determining disadvantages and cusp communities and regenerate the data

This will be tackled likely as part of the PR that tackles #155

@lucasmbrown-usds can you clarify in the card how this relates to the other score-related cards?

@switzersc-usds I created this because I thought there would need to be a place where we specifically detail the threshold for being considered prioritized or not, and to explain that we might have to play around with the % thresholds before we land on a true definition of what makes a census block group in or out. And finally, to call out that this data has to be available to be seen in the UI of the map but also exported as part of the list.
If that is properly described elsewhere then we can certainly call this a duplicate.


Tree View component is not accessible in Android

The Tree View component does not work in Android when used with Talkback enabled. When Talkback (screen reader for Android) is enabled, the nodes in the Tree View are not working.

[x] The issue is present in the latest release.
[x] I have searched the issues of this repository and believe that this is not a duplicate.

Current Behavior 😯
All the below list issues occur when Talkback is enabled.

When Talkback is enabled, the tree nodes get focused by the swipe right gesture. But when double tapped, it does not expand or collapsed and instead only announces the current state.
Similar issue occurs on node selection as well. With Talkback enabled and when we use the double-tap gesture to select the node, it does not get selected.
The name of the node is not announced when a node is focused

Expected Behavior 🤔

While using Talkback the nodes should expand or collapse when the double tap gesture is used.
Tree nodes should be selectable when double tag gesture is used
The name of the tree node should be announced when it is focused.

Steps to Reproduce 🕹
Enable Talkback on an Android mobile and try to interact with the Tree nodes give in the example page
Context 🔦
I am trying to use the Tree view component in Android.
Your Environment 🌎

`npx @material-ui/envinfo`
    System:
    OS: macOS 10.15.7
  Binaries:
    Node: 14.15.5 - /usr/local/bin/node
    Yarn: 1.22.10 - /usr/local/bin/yarn
    npm: 6.14.11 - /usr/local/bin/npm
  Browsers:
    Chrome: 90.0.4430.93
    Edge: 90.0.818.49
    Firefox: 88.0
    Safari: 14.1
  npmPackages:
    @material-ui/core: ^4.11.4 => 4.11.4
    @material-ui/icons: ^4.11.2 => 4.11.2
    @material-ui/lab: ^4.0.0-alpha.58 => 4.0.0-alpha.58
    @material-ui/styles:  4.11.4
    @material-ui/system:  4.11.3
    @material-ui/types:  5.1.0
    @material-ui/utils:  4.11.2
    @types/react:  17.0.4
    react: ^17.0.2 => 17.0.2
    react-dom: ^17.0.2 => 17.0.2

Could you test https://next.material-ui.com/components/tree-view/ out? The focus is now handled with a aria-activedescendant (however, it creates other issues #24480)

@oliviertassinari  Unfortunately that did not fix the issue. I am getting the same issues I originally reported.
While testing this on two different devices I observed the following:
Device 1 - Android version 7.0

I am able to expand a tree node, select a child node without any issue.
Only issue is I am NOT able to collapse an already expanded node.

Device 2 - Android version 10

All the issues I originally reported are observed here.

Note: Both the above tests were done with Talkback (version 9.1) enabled and tested on both the main Material UI documentation site as well as next.material UI documentation site.

@oliviertassinari Any update on this? Could you point me where the issue might be occurring from so I can attempt a fix?

@krv5vignesh I don't have a single clue about what's going wrong. It's also unlikely this issue will be worked on before many months. If it's important for your organization, feel free to invest time in it.

Using Android 7.0 NRD90M on a Redmi Note 4 on https://6093d26ee081cf00074f9d5d--material-ui.netlify.app/components/tree-view#FileSystemNavigator.
Checked means working, unchecked means not working:

[x] announcing name when focused
[x] selection
[x] announcing selected state
[x] announcing expanded state
[x] expansion when double tapped
[ ] collapsing when double tapped

Same behavior with https://www.w3.org/TR/wai-aria-practices/examples/treeview/treeview-2/treeview-2a.html
Ignore the display :smile:
https://user-images.githubusercontent.com/12292047/117434150-98af9e00-af2c-11eb-8ab2-b3a13092baba.mp4
Will check if there are bug reports for Android 10. But I'd be surprised if this were an issue on our part.


A generic list sorting mechanism for the listing views

This applies to listing views of each subsystem as well as views within containers. Containers here includes the listing views of the vocabulary concept hierarchy.
Example functionality:

By default, items in listing view is sorted by default label predicate and default literal language tag
This sorting can be customised in the profiles to sort by another predicate and also by the type of value (e.g., alphabetical order versus a numerical value)

In BGS' case where they want to sort by time in the geologic timescale vocabulary, they can add a predicate with a value in the profiles. Will also require some other way to target that sorting behaviour for just that vocab.

commenting to track progress, as I am also interested in this.

I think first step is to confirm / document the existing behaviour (maybe some test cases) that use these query params and see if there's any gaps with e.g. numerical datatypes; it may already work.
order_by
order_by_direction


Hydra node cannot handle time conversion once 3k slots have passed

fix #487

Sorry for the commit mess. I'll post another branch on top of the current master.

Superseded by https://github.com/input-output-hk/hydra-poc/pull/523


Poor SEO

This isn't very crawler friendly at the moment. Google and others have some trouble with SPAs. Searching something like "one hour one life craft door" has the site show up pretty far down, and doesn't go directly to the page. Some resources:

https://moz.com/blog/optimizing-angularjs-single-page-applications-googlebot-crawlers
https://vuejsdevelopers.com/2018/04/09/single-page-app-seo/
https://vuejsdevelopers.com/2017/04/01/vue-js-prerendering-node-laravel/
https://github.com/rafrex/spa-github-pages
https://github.com/isaacs/github/issues/408

Looking into this a bit more, I think the best solution is to use the Vue Router with History Mode along with this GitHub pages workaround. We can then add a sitemap and ensure there are crawlable links around the whole app. This should resolve the majority of SEO issues.
Alternatively we could mess with prerendering or move away from GitHub Pages to do server-side rendering. Both of these are heavy solutions which I don't think are necessary.

Despite these changes, the pages still aren't showing up in Google. I need to look into this more.

The custom domain is all setup now which will probably improve SEO. I've also setup the site with Google's Search Console which will show me what's wrong once it indexes the site (in a day or two).

Looks like Google is reporting a 404 error on a number of pages even though those pages work when I visit. I don't think Google likes the 404.html hack that is described here.
I might put this on a cheap VPS in the near future so we can have true route redirects.

Google Search Console is reporting this about the indexing:

It is intentionally excluding the URLs because the server reports a 404 status, even though we redirect on the client side and the SPA hack was supposed to work with Google indexing.
I recreated the sitemap to use the onetech.info?p=... URL format which will result in no 404s. Let's see if Google likes that.
If this doesn't work, setting up a separate server seems like the only other option.

Using # URLs should still be viable and fine for SEO
On Wed, 20 Jun 2018, 07:29 Ryan Bates<EMAIL_ADDRESS>wrote:

Google Search Console is reporting this about the indexing:
[image: screen shot 2018-06-19 at 12 26 18 pm]
https://user-images.githubusercontent.com/161/41619713-10e104fe-73bc-11e8-8cec-c03a89a1c413.png
It is intentionally excluding the URLs because the server reports a 404
status, even though we redirect on the client side and the SPA hack
https://github.com/rafrex/spa-github-pages was supposed to work with
Google indexing.
I recreated the sitemap to use the onetech.info?p=... URL format which
will result in no 404s. Let's see if Google likes that.
If this doesn't work, setting up a separate server seems like the only
other option.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/Kazetsukai/onetech/issues/78#issuecomment-398517367,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA5GRv9Lskk3qxx_LVd_oCSfKu-ljocJks5t-VEXgaJpZM4T3Chh
.

Yeah, have run into that. In practice at my workplace it hasn't caused us
SEO problems, although we are moving away from it longer term.
Something cheeky like cloudfront is also possible:
https://keita.blog/2015/11/24/hosting-a-single-page-app-on-s3-with-proper-urls/
On 20 June 2018 at 20:25, Ryan Bates<EMAIL_ADDRESS>wrote:

Really? Everything I’ve read says Google doesn’t like URL hashes to
represent separate pages. See: https://www.google.com/amp/s/
www.seroundtable.com/amp/google-no-url-fragments-23449.html
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/Kazetsukai/onetech/issues/78#issuecomment-398667148,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA5GRiSJJw3OKMXau29KehQ7yN3SDROgks5t-gcRgaJpZM4T3Chh
.

Something cheeky like cloudfront is also possible:

Yeah, Cloudfront would be cheeper than a dedicated VPS, although it would be nice to have a persisted local Git repo + cron job that we can do git pull every few minutes and auto-update the site with new content.
We could get fancy and use something like Amazon Lambda and communicate with GitHub through the API to check for changes and request just what we need, however not sure about sprite processing and memory usage. Also sounds like a lot of work to setup. ;)

Again cheeky, but you could probably get away with free tier codebuild.
On Thu, 21 Jun 2018, 07:48 Ryan Bates<EMAIL_ADDRESS>wrote:

Something cheeky like cloudfront is also possible:
Yeah, Cloudfront would be cheeper than a dedicated VPS, although it would
be nice to have a persisted local Git repo + cron job that we can do git
pull every few minutes and auto-update the site with new content.
We could get fancy and use something like Amazon Lambda and communicate
with GitHub through the API to check for changes and request just what we
need, however not sure about sprite processing and memory usage. Also
sounds like a lot of work to setup. ;)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/Kazetsukai/onetech/issues/78#issuecomment-398874051,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AA5GRikpudIroKdPiF25_V73W6-2AKS2ks5t-qcYgaJpZM4T3Chh
.

Interesting. I didn't consider a build service but that might just work.

The app is now fully hosted on a VPS so the 404 hack is no longer necessary. I think this will help with SEO but time will tell.

Moving to a VPS hosted page has helped SEO significantly. Since I have control over the Nginx config I no longer need to do the 404 redirect work-around. Google is still excluding a lot of the URLs in the sitemap, but I don't think there is much more I can do about that for now.


Features for chimps

@santisiri @bondsbw @vjrj @brylie
Implements #103
This PR is based on master, and was merged without too much friction.
The ids of DOM elements (titleContent and editor) were left untouched in the end, except for tags. Time to refactor this later, as ids should be unique.
We fixed some issues introduced in recent commits for non-phone users.
We noticed that there are still critical issues remaining, but they're not introduced by these commits.

Roughly, the features tested :

[x] Authoring

[x] Ideas without ballots
[x] Ideas with ballots

[x] Voting

[x] Receiving votes upon account creation
[x] Using votes on ideas

[x] Debating

[x] Using a chronological tree

Delegation is awaiting feedback in #103

Awww... Seems like recent commit 7d9b756a413b7380a3ea47216566112a0e08fa50 created a conflict. I do not really know what I am doing with git and how to solve this now. Help ?

Never mind, doing the obvious thing worked. :tada:

This is ready for review ; an early merge (before the major refacto to React, for instance) is recommended, as a test suite such as this one proves very useful during refactorizations.

There's a massive amount of changes here and the codebase has significantly changed on some files. Maybe split the feature related part into on PR and the rest on another one?

I'll rebase this on the new master ; it's mostly all for the features, there were some hotfixes that are meaningless now, so I'll just remove them.

Hi there @domi41: I've just discover this PR after coming back to sovereign today. Can you resolve the conflicts? Good job!

A lot of conflcts. Likely to be working with very old code by now.


Two feature requests for rowing integration

First of all, thank you for your great work. I am using your NPM for adding fit-files export to OpenRowingMonitor (I'm the package maintainer for that), it is a breeze to create files this way.
I'm missing a field which is used in rowing machines (in the "record" message): the stroke_distance. I think it has ID 87, and seems to be in centimeters (values typically range between 900 cm and 1200 cm. When using cycle length it doesn't get picked up, and when I record on a Garmin, it is stored in the 87 field.
An additional request is to be able to reset the recorded file and start again. I know I can clear the entire variable, but a nice reset function would be cleaner I think.
Thanks again for the great work, and I hope you can help me.

Can you point me to the stroke_distance documentation? I'm not seeing it anywhere in the fit sdk.
You mention cycle_length - I'm not sure how that's connected to stroke_distance, but I do see a field named cycle_length16 with id 87. It appears to be identical to cycle_length, except it's a uint16 rather than a uint8. If the values are typically 900 to 1200cm that would make sense...
Is that what you're looking for?

You mention cycle_length - I'm not sure how that's connected to stroke_distance, but I do see a field named cycle_length16 with id 87. It appears to be identical to cycle_length, except it's a uint16 rather than a uint8. If the values are typically 900 to 1200cm that would make sense...
Is that what you're looking for?

Yep,
That is the one. One of my fellow developers on OpenRowingMonitor discovered the field two years ago, see his bugreport: https://forums.garmin.com/developer/fit-sdk/f/discussion/323579/missing-stroke_distance-field-on-the-record-message
He confirmed it is indeed called 'cycle_length16', but the averages are stored in 'avg_stroke_distance' on the lap and session level.

test1.fit.zip
Can you confirm this is what you're expecting? The field is defined with units of m with a scaling factor of 100. So the raw values are in cm. I modified my test case to include cycle_length16 fields in the range 9 to 12m (so 900 to 1200cm). The resulting fit file can be decoded at https://www.fitfileviewer.com, and shows the values I was trying to write. I just want to make sure I've got the scaling correct.

v0.1.3 should fix the cycle_length16 issue.
I'm not sure I want to add a reset() method. What's the advantage vs throwing the old one away and creating a new one?

v0.1.3 should fix the cycle_length16 issue.

Thanks, I'll test tomorrow! That will make a lot of indoor rowers happy!

I'm not sure I want to add a reset() method. What's the advantage vs throwing the old one away and creating a new one?

Yeah, I solved this differently now, by limiting the object scope to our file generation function. We noticed the Javascript garbage collector has some issues, which sometimes results in a memory leak. So we try to clear memory as explicit as we can (solved a lot of our memory issues). But our current approach works well enough


How come the theme selected no more?

@pojokcodeid Sorry for asking here, why after using your custom folder, the theme selected with SPC T H, no more functioning?
Or may i know how to get it back?

@pojokcodeid This is nothing to do with Nvchad, your custome modification change the key press, i have make change on that, and it now show up with SPC t e

Ok, thanks for the clarification.
On Tue, Jun 6, 2023 at 5:45 PM Thomas Yeun @.***> wrote:

@pojokcodeid https://github.com/pojokcodeid This is nothing to do with
Nvchad, your custome modification change the key press, i have make change
on that, and it now show up with SPC t e
—
Reply to this email directly, view it on GitHub
https://github.com/pojokcodeid/NvChad-v2-config/issues/1#issuecomment-1578431511,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQNF3RJEHJWP2UKGUSNKAKDXJ4C43ANCNFSM6AAAAAAYVUTTEI
.
You are receiving this because you were mentioned.Message ID:
@.***>


Update toolchain to 1.34.0

Codecov Report

Merging #263 into develop will increase coverage by 0.08%.
The diff coverage is 100%.

@@             Coverage Diff             @@
##           develop     #263      +/-   ##
===========================================
+ Coverage    48.99%   49.07%   +0.08%
===========================================
  Files          116      116
  Lines        17352    17354       +2
===========================================
+ Hits          8501     8516      +15
+ Misses        8851     8838      -13

Impacted Files
Coverage Δ

db/src/trie/triedbmut.rs
92.26% <ø> (+0.36%)
:arrow_up:

db/src/journaldb/earlymergedb.rs
93.35% <ø> (+0.99%)
:arrow_up:

db/src/journaldb/archivedb.rs
98.92% <ø> (+0.71%)
:arrow_up:

db/src/journaldb/refcounteddb.rs
97.82% <ø> (-0.55%)
:arrow_down:

db/src/journaldb/overlayrecentdb.rs
99.39% <100%> (+0.75%)
:arrow_up:

logger/src/lib.rs
95.16% <100%> (ø)
:arrow_up:

blake2b/src/blake2b.c
14.28% <0%> (-19.05%)
:arrow_down:

cita-sm2/src/signer.rs
92.3% <0%> (-7.7%)
:arrow_down:

util/src/cache.rs
73.8% <0%> (-7.15%)
:arrow_down:

util/src/bytes.rs
73.33% <0%> (-6.67%)
:arrow_down:

... and 50 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cde49e7...97fa3bf. Read the comment docs.


[Feature Request] Improve diff2html

Checklist

[X] I searched exising issues and no one else requests similar feature.
[X] I think that 25%+ users are positive towards this feature.

Describe the feature
<link
  rel="stylesheet"
  href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.8.0/styles/github.min.css"
  media="screen and (prefers-color-scheme: light)"
/>
<link
  rel="stylesheet"
  href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.8.0/styles/github-dark.min.css"
  media="screen and (prefers-color-scheme: dark)"
/>

Additional context
No response

diff2html支持离线模式

https://diff2html.xyz/demo.html?matching=lines&matchWordsThreshold=0.25&maxLineLengthHighlight=10000&diffStyle=word&colorScheme=auto&renderNothingWhenEmpty=0&matchingMaxComparisons=2500&maxLineSizeInBlockForComparison=200&outputFormat=side-by-side&drawFileList=true&synchronisedScroll=1&highlight=1&fileListToggle=1&fileListStartVisible=0&highlightLanguages=[object Map]&smartSelection=1&fileContentToggle=1&stickyFileHeaders=1&diff=https://github.com/rtfpessoa/diff2html/pull/106


Bug report

Describe the bug
[RhttpTimeoutException] Request timed out. URL: https://<IP_ADDRESS>:53317/api/localsend/v1/info
To reproduce

Go to "Send"
Click on the heart who is used to add this contact as a favorite
Click on the i near the "Error message"
See error

Expected behavior
When I'am on my pc, I just wanna add my phone to favorite and it's doesn't work,
It's written :
[RhttpTimeoutException] Request timed out. URL: https://<IP_ADDRESS>:53317/api/localsend/v1/info
Screenshots

Desktop (please complete the following information)

OS Windows 11

Smartphone (please complete the following information)

iPhone 12 Pro (ios 18.1.1)

Additional context
No response

Same happens on Android v1.16.1.
Also, I cannot send files from affected Android device
Interestingly, sending to that Android device still works

Similar thing happens on Ubuntu 22.04. Cant detect my Windows machine and when I try to add to favorites via IP address or hashtag, I get the same error.

Usually, Windows Firewall is blocking those requests.
Make sure to use "private" as the network setting. On "public" networks, the firewall rejects any incoming data from peers.
Alternatively, adjust the firewall according to the README.

My wifi was set to PUBLIC . Set it to private and it works after configuring the firewall . Thanks

I tracked it down on my side. It is the macOS Firewall that is blocking my Android device. As soon as I disable the macOS Firewall, Android can send to macOS again.
But if I activate the Firewall on macOS and allow LocalSend to receive incoming traffic, it does not work.

I guess there is a bug in the macOS client that opens the port in a way that macOS does not conceive it's traffic connected to LocalSend.


Current main requires julia 1.9

I overlooked that the filter expressions
https://github.com/trixi-framework/libtrixi/blob/50c60c997439a516da0e3f1e8d4786d3b5443c43/LibTrixi.jl/src/LibTrixi.jl#L67
require julia 1.9 at least.
One could rewrite this (non-critical) piece of code or bump the the compat entry.

If (trivially) possible, I'd try to continue to support Julia v1.8 for now. It's just the filter expression that does not work, right? Something like
filter(p -> p.name == "whatever", Pkg.dependencies() |> values |> collect)

should still work


support pre-commit hook

as it is possible to run with pre-commit as one of hooks
Example:
https://github.com/pre-commit/pre-commit-hooks/blob/master/.pre-commit-hooks.yaml

See e.g. https://github.com/dhatim/python-license-check/issues/14


[Bug]: Closing Publish Modal Redirects To Previous URL

🐛 Describe the bug:
The close button in the modal has the functionality to redirect the user to the last visited URL. This functionality works fine with auth modals, but due to this, when we close the publish modal, we are redirected to the previous URL, which is not suitable for a good user experience.
🔃 Steps To Reproduce

Go to the settings page.
Go to the create page.
Click on the Publish button; a publish modal will open.
When we close this modal, we will be redirected to the settings page.

📃 Expected behavior
When we close the modal, the user should stay on the create page.
📸 Screenshots
NIL
👂 Additional context
NIL

@piyusharmap If I understand this very well, the [x] on the model, when clicked will reroute the page to a previous route, right? because when I checked the code, I saw that router.back() is making the page go back to the previous route instead of only setting the setModal(false)


Add support for 1024x1024 icon in .unoproj

Affects iOS release builds. Reported on Slack.

AppStore now recommends adding a 1024x1024 application icon that is shown on the AppStore for retina devices. Fuse should have an .unoproj setting that allows to specify an icon of that size.
Since recently this was just recommended, but now iOS apps cannot be submitted for review without one, judging from the screenshot above.

This isn't a fuselibs issue, it's a uno-issue.

logged where appropriate, closing ticket