Implemention of cupyx.scipy.ndimage.filters

I have implemented nearly all of cupyx.scipy.ndimage.filters matching scipy.ndimage.filters exactly. However, before I finish it I would like to confirm that I am developing it correctly. I have the file currently in a gist: https://gist.github.com/coderforlife/d953303da4bb7d8d28e49a568cb107b2
This works towards https://github.com/cupy/cupy/issues/2099. One of their requests was one of the missing features though.
Current Lacking Features:

Does not have the implementations of generic_filter() or generic_filter1d().
I want to make some changes to how rank filters work (includes rank_filter, median_filter, percentile_filter) based on the answers below.
There are no tests.
There are no function docs (but most would just refer to the scipy docs).

Changes From Current cupyx.scipy.ndimage.filters:

Implements tons of new functions (and they re-use slightly adjusted code that was already there).
convolution() and correlate() were adjusted to support non-contiguous input arrays without copying them (weights are still forced to be contiguous, but they are expected to be quite small).
convolution() and correlate() are slightly (~5%) faster now.
The ElementwiseKernels generated are now more runtime adjustable.

Open Questions:

The ElementwiseKernels are only used internally and are never returned. Should they have as much as possible compiled into them? The current version of the code compiles the number of dimensions, input shape, weights shape, border mode, constant value, and origin values. My version moves input shape (and actually strides), constant value, and origin values to be runtime values. Is it better to have more compile time values and thus need to compile new ElementwiseKernels more frequently or have more things moved to runtime values and need fewer compiles?
To implement generic_filter() and generic_filter1d() I think they should take a ReductionKernel (or a Python function that can be fused into a ReductionKernel). However, Cuda does not let you call functions in other modules. So to accomplish this I was thinking about re-creating the reduction kernel code in the convolution kernel like at https://github.com/cupy/cupy/blob/de3d32536c39eefea1805f4a7a76a14356eb3bff/cupy/core/_reduction.pyx#L36 however it would actually be simpler since the entire reduction would happen in a single thread and not need synchronization. Is this a reasonable approach?
Is it okay that numpy is used to create some of the filters internally? It is several orders of magnitude faster than doing it in cupy, even at the large kernels of 25.
Is the quality of the code okay?

Thanks for your consideration.

I have updated the code fixing some issues including adding all documentation. I also added a partially working generic_filter() and a 'working' generic_filter1d() however it turns out that in Scipy the 1d version doesn't behave like any of the other 1d versions so it needs to be updated to be inconsistent...
generic_filter() as written now supports 'functions' that are ReductionKernels, simple reduction kernels, Fusion objects that result in reduction, and regular Python callables that can be fused. There are several issues with the implementation, primarily with handling data types and possible name clashes.
I compared some timings with this generic filter. I used a 1024x1024 image stored as uint8 and did an L2-norm over a 5x5 region.

Scipy with a Python function using Numpy function: 4.72 s
Scipy with a LowLevelCallback around a Cython function: 47.1 ms (100x faster)
Cupy using ReductionKernel: 855 µs (55x to 5500x faster)

However, for the Cupy function to accept a fusable function I had to make the output a float64 which increased the time to 2.42 ms. This was comparable to using the ReductionKernel with a float64 output.
I have another question. For the "no shortcuts" rules, does it mean that I need to fully spell out the following:

cupy.fuse -> cupy.core.fusion.fuse
cupy.ndarray -> cupy.core.core.ndarray
cupy.ReductionKernel -> cupy.core._kernel.ReductionKernel
cupy.ElementwiseKernel -> cupy.core._kernel.ElementwiseKernel
cupy.ones -> cupy.creation.basic.ones
cupy.zeros -> cupy.creation.basic.zeros
cupy.array -> cupy.creation.basic.array
cupy.asarray -> cupy.creation.basic.asarray
cupy.float64 -> numpy.float64
cupy.sqrt -> ?

Some of those seem weird because they end up being in protected modules or switch to numpy.

Thanks for working on this @coderforlife!
I had also done a similar thing in January of this year, although I did not attempt generic_filter or generic_filter1d and hadn't posted the code yet. I will try to take a look at this version as well as make the code I have worked on available over the next week. Most likely we can collaborate on getting these into CuPy proper.
I had implemented tests by basically just adapting the existing test suite from scipy.ndimage, but had not made the tests consistent with the CuPy style or converted docstrings to the CuPy format.
I also have implementations for many of the ndimage.morphology operations and ndimage.fourier_shift as well as quite a few functions from scikit-image.
I was hoping to look at refactoring/implementing some things from ndimage.interpolation, but so far only made a first-order interpolation kernel for map_coordinates.
The other function I would really like to have is ndimage.label as requested in #3119, but I have not tried to implement it myself.

Interesting work!
I've recently implemented gaussian_filter and gaussian_filter1d on top of cupyx.scipy.ndimage.filters.correlate, based on the SciPy implementation. As I have no experience writing CUDA code it's just Python. My functions produce results comparable to SciPy, but are much faster with a proper GPU. The code is available here if you want to have look:
https://github.com/Skielex/structure-tensor/blob/master/structure_tensor/cp/filters.py
@coderforlife I tried running your implementation of  gaussian_filter and gaussian_filter1d. Here are a few things i noticed.

Both functions perform really well, much faster than mine. Nice work!
gaussian_filter does not appear the produce correct results. I tried comparing to the results from SciPy and my implementation and values we quite a bit off. Also even though data was float64, there we no decimals on the returned values.
Results of gaussian_filter1d are comparable to SciPy for the data I tested with.
cupyx.scipy.ndimage.filters.correlate suffers significantly from more dimensions, even if the kernel has a size of 1 for the dimensions. Your implementation of gaussian_filter1d appears to suffer at little from more dimensions even though I believe the computations should be identical.

# d.shape is (130, 150, 100)
%timeit filters2.gaussian_filter1d(d, 1, -1, mode='nearest'); cp.cuda.Stream.null.synchronize()
# 3.79 ms ± 11.2 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
%timeit filters2.gaussian_filter1d(d.reshape(-1, 100), 1, -1, mode='nearest'); cp.cuda.Stream.null.synchronize()
# 3.33 ms ± 6.18 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

If you can fix the gaussian_filter I'd very much like to try it for my structure-tensor package.

It is fixed now. I actually discovered that bug yesterday just hadn't uploaded my solution. If the image had an even number of dimensions it would return the last off-numbered dimension result I believe. Now I am getting np.allclose() to pass.
For the other issue about speed, this is related to my question about making the internal kernels more flexible or more often compiled. My code makes them compiled less often, but my guess this ends up having a larger overhead especially with larger number of dimensions. I will play around with timing the results of how long things actually take to compile,. I was already leaning towards making more things "compile time" constants.

I found small bug in line 312.
P = numpy.diag(cupy.ones(order) * sigma2i, -1)  # P @ q(x) = q(x) * p'(x)

should be
P = numpy.diag(numpy.ones(order) * sigma2i, -1)  # P @ q(x) = q(x) * p'(x)

otherwise it'll throw an exception when calling with order > 0

@coderforlife I found another issue in the _gaussian_kernel1d function. If order > 0, the weights are calculated wrong, as the value of x is incorrect.
Is there any reason not just to copy the _gaussian_kernel1d function directly from SciPy? I can see you opted for in-place ops, which I understand, but I don't think it makes a difference given the relatively small size of the weights. I did a small benchmark and the SciPy one was actually slightly faster.
In this gist I've changed to use an exact copy of the SciPy _gaussian_kernel1d function and changed the places it is used to be more aligned with the SciPy implementation:
https://gist.github.com/Skielex/14c060b962e1cfb64fe98de9e9dc7cbe
The gist produce the same results on my own implementation, which are close to the SciPy ones. After fixing the issues with _gaussian_kernel1d I compared time for both gaussian_filter1d and gaussian_filter from the gist to my own implementation. Below filters_old is mine, while filters is the gist one (yours modified).
%timeit Vx = filters.gaussian_filter1d(d, sigma, axis=0, order=1, mode='nearest'); cp.cuda.Stream.null.synchronize()
15 ms ± 34.7 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

%timeit Vx2 = filters_old.gaussian_filter1d(d, sigma, axis=0, order=1, mode='nearest'); cp.cuda.Stream.null.synchronize()
34.6 ms ± 74.7 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

%timeit Vx = filters.gaussian_filter(d, sigma, order=[0, 0, 1], mode='nearest'); cp.cuda.Stream.null.synchronize()
44.6 ms ± 268 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

%timeit Vx2 = filters_old.gaussian_filter(d, sigma, order=[0, 0, 1], mode='nearest'); cp.cuda.Stream.null.synchronize()
45.1 ms ± 219 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

I'm a bit surprised that while your gaussian_filter1d is about twice as fast, gaussian_filter performs the same. As gaussian_filter uses gaussian_filter1d, I would've thought it to be faster too. The GPU used is a RTX2070 Super and d is (128, 128, 128) float64.

This is great! Thanks for driving this work forward @coderforlife and everyone for chipping in 😄
Would it make sense to move this work to a PR for further discussion, refinement, and eventually integration? 😉

Would it make sense to move this work to a PR for further discussion, refinement, and eventually integration?

To ease review, a potential first step PR from the code proposed by @coderforlife would be to integrate the refactored CUDA kernels for convolution/correlation in place of those currently in CuPy (i.e. a PR focusing on using your refactored _get_nd_kernel with the functions already in cupy.ndimage.filters).
I tried your kernels on Friday and did find that they are substantially faster (by about a factor of two in some fully dense cases I tested), but I found some existing test cases with even sized kernels and/or non-zero origin did not give matching results. Making a PR updating the existing correlate/convolve would let you reuse the existing CuPy tests to work out any potential bugs in the refactored nd kernels.

cc @jni @GenevieveBuckley (for general awareness 😉)

I will work on the PR now. I was on vacation over the last week.

I will work on the PR now. I was on vacation over the last week.

Sounds good to me, but it would be nice if any core devs here could provide guidance on whether they agree with the general approach in your gist. Having separate 1d/nd cases seems reasonable to me, however if it is possible to just use nd for everything without substantial overhead that would probably ease maintenance.
You may also want to take a look at #3179 which was just recently opened and improves performance for the existing nd convolve/correlate.

The 1D and ND are actually fairly different beasts. One applies along an axis and the other doesn't.
I am close to submitting. I found two bugs in my code causing failures that I have now fixed (I added origins instead of subtracting and didn't handle 0-sized dimensions exactly like scipy). I am doing some speed tests on some final changes and then submitting the PR.

FWIW would check against the latest SciPy release. Recall there were a few bugs they fixed. Some of which relate to these edge cases (like 0-sized).

@Skilex I was attempting to optimize _gaussian_kernel1d() to work as fast with cupy as it did with numpy, and thus trying to use in-place everywhere, but yeah, it could just be Scipy's function.
The (non) speed difference is definitely interesting. I will look into that more.
At the moment my goal is to get a PR with convolve, correlate, convolve1d, and correlate1d including tests. I have the current tests passing with convolve and correlate and have started writing the 1d tests.

@grlee77 I have added the PR with just convolve, correlate, convovle1d, and correlate1d. All other functions use the same core of _get_nd_kernel and _get_1d_kernel so once those are seen and approved then I can easily add the rest of the functions (besides the generic_filter functions which will likely need some debating). I did fix the even-number-of-dimensions issue (I always returned the last odd dimension result) and the origin issue (a double-negative issue). I have added tests for the 1D versions and this actually caused me to find a bug in scipy, so I fail 8 tests right now because scipy gives the wrong result.

Did you raise an issue with SciPy already?

Yes, scipy/scipy#11661

I have made the code mentioned previously ( https://github.com/cupy/cupy/issues/3111#issuecomment-591052725) available at cupyimg.
See a more extended comment related to this in https://github.com/cupy/cupy/issues/2099#issuecomment-598381598.

@grlee77 You originally requested to see if 1d and nd could be combined, and the answer is yes. With a few tweaks to nd I was able to make 1d work with the nd kernel generation but maintain its speed. I have updating the gist with this and will be updating the PR shortly. At this point all kernels in ndimage.filters are the same base kernel with a few substitutions. I am also curious about making the shared memory kernel like you suggested somewhere and will be playing with that soon.
@Skielex the most recent gist has addressed the speed issues you were seeing (along with the bugs). Length-1 dimensions do not impact significantly speed any more and the speed is more reasonable in general.

@grlee77 You originally requested to see if 1d and nd could be combined, and the answer is yes. With a few tweaks to nd I was able to make 1d work with the nd kernel generation but maintain its speed.

Okay, I will take a look. When I implemented the 1d convolutions, I had just reused the same nd kernels and just broadcasted the 1d kernel to be size 1 along any non-filtered axes.

@grlee77 I did that too, but added in a little if that says if a dimension's length is 1 don't produce the loop or boundary condition checking in CUDA for that dimension. That was necessary to maintain the same speed with the original 1D code.

@grlee77 Using shared memory for the weights was actually fairly easy using the loop_prep= extra argument cupy.ElementwiseKernel. It gave marginal improvements. For a 4000-by-4000 image, using various sized kernels:

3x3 goes from 750 to 739 µs (1.5% faster)
7x7 goes from 1.52 to 1.49 ms (2% faster)
11x11 goes from 3.16 to 2.9 ms (9% faster)
21x21 goes from 27.8 to 25.3 ms (10% faster)
25x25 goes from 37.9 to 34.6 ms (10% faster)
29x29 goes from 55.8 to 47.5 ms (17% faster)
31x31 goes from 63.6 to 54.6 ms (17% faster)
33x33 goes from 66.7 to 41.3 ms (62% faster)
35x35 goes from 74.4 to 70 ms (6% faster)
37x37 goes from 89.9 to 84.6 ms (6% faster)
39x39 goes from 92.5 to 23.1 ms (300% faster)
41x41 goes from 110 to 99.4 ms (11% faster)
43x43 goes from 121 to 135 ms (12% slower)
45x45 goes from 120 to 33 ms (264% faster)
47x47 goes from 144 to 177 ms (23% slower)

I do find the ups and downs in the speed interesting. Like why are 33x33, 39x39, and 45x45 so much faster than the neighboring sizes? How to predict the slower ones?
I played with a few of the shared memory settings. Changing the device cache config with cudaDeviceSetCacheConfig() (default was cudaFuncCachePreferNone) always caused things to be slower. I could not change the device shared mem config with cudaDeviceSetSharedMemConfig(). It default to cudaSharedMemBankSizeFourByte and changing to eight-byte had no effect (no error, but cudaDeviceGetSharedMemConfig() still returned cudaSharedMemBankSizeFourByte). If it matters, this is on a Titan V.

A few questions about the desired level of compatibility between numpy/scipy and cupy. I am finding that there are some edges cases in which the filters in scipy are very hard to reproduce in cupy. A few examples:

Handling of the UB of casting a negative float to an unsigned int. CUDA has this as UB, although it seems that the behavior is to clip to 0 instead of wrap around. In C/numpy I am not sure if this is defined behavior, but it usually seems to wrap around. This is coming up when testing uint8 inputs with mode='constant' and cval=-1 (an invalid uint8 value). The point in time in which that -1 resolves to a uint8 is highly variable in all of the filters meaning there are lots of places that have inconsistent behavior between the two. I try to move some casting around and it fixes one but breaks another. Should we just write this whole situation off as cval=-1 makes no sense for uint8 so it is okay if there is inconsistent behavior? Or should we try to make all possible situations match?

Integer outputs with floating-point intermediates. I found this peculiar result in scipy's ndimage (imported as ndi):

ndi.gaussian_filter1d(np.array([5, 5, 5]), 2.5) # returns [4, 4, 4], should be all 5's

There are numerous way to avoid that particular output, all of the following return all integer 5's:
ndi.gaussian_filter1d(np.array([5, 5, 5]), 2.49)
ndi.gaussian_filter1d(np.array([5, 5, 5]), 2.51)
(ndi.gaussian_filter1d(np.array([5, 5, 5], float), 2.5) + 1e-14).astype(int)
(ndi.gaussian_filter1d(np.array([5, 5, 5]), 2.5, output=float) + 1e-14).astype(int)

Additionally, the implementation of ndi.gaussian_fitler1d I am writing also happens to return all 5's in the regular case which is making a test fail. The easiest solution would be to just change the 2.5 to something else like 2.25 and call it a day. Do we care?

Actually, I found that any number I would pick for sigma, either numpy or cupy would round down some of the numbers. Increasing the value of the truncate value can help. The 1e-14 solution only helped for small enough sigma values, eventually, I needed 1e-12 to deal with it. But that would only apply to cupy, not scipy. If both sides implemented that, then maybe we would have consistent results.

I actually see that the first issue is with wrapping vs clipping is actually not the same in numpy vs cupy already with the astype() method for casting:
((cp.random.rand(10, 10) * 2 - 1)*100).astype(cp.uint8) # contains lots of 0s
((np.random.rand(10, 10) * 2 - 1)*100).astype(np.uint8) # contains lots of >200

So apparently this behavior is expected (I found the documenation https://docs-cupy.chainer.org/en/stable/reference/difference.html#cast-behavior-from-float-to-integer that says exactly that).

Thanks for putting so much effort into this @coderforlife! I'm looking forward to replacing my hacky code with a proper generic_filter1d implementation. Curious to see how much faster it'll be.
While I appreciate as much compatibility as possible, I think we'll just have to live with small variations due to (lack of) floating point precision. If you're using gaussian_fitler1d with integer input and output, precision is clearly not a great concern.
In my case, I will be using float32 or float64, so as long as the results are "reasonably" close to SciPy I'm happy.

regarding casting float to unsigned:
At least as of 1.5.0, ndimage does the following in the C code.
https://github.com/scipy/scipy/blob/e4238172b1925e1bf6a554678fd765553f0e069c/scipy/ndimage/src/ni_filters.c#L159-L163
I think if you add a boolean unsigned_output to _get_correlate_kernel you can modify set post argument to _get_correlate_kernel using
if unsigned_output:
    post = 'y = (sum > -1) ? (Y)sum : -(Y)(-sum);'
else:
    post = 'y = (Y)sum;'

Integer outputs with floating-point intermediates. I found this peculiar result in scipy's ndimage (imported as ndi):
ndi.gaussian_filter1d(np.array([5, 5, 5]), 2.5) # returns [4, 4, 4], should be all 5's

I don't think we can ensure identical output in such cases where perhaps one implementation gives a float of 4.999999999999999 while another might give 5.00000000001, so that the cast to integer gives different values.
In principle one could argue whether integers should be rounded instead in SciPy, but that would still have the same potential differences due to precision, just for values on either side of the midpoint instead.
I have also seen similar differences in rounding for cupyx.scipy.ndimage vs scipy.ndimage when using interpolation functions like rotate with integer output type. I can't remember the exact case offhand, but it was the same type of  issue.

Based on the above how does the following sound?
1.) cast float to unsigned using the same convention from SciPy noted above
2.) add a note to the docstrings indicating potential non-identical results when casting intermediate floats to integer output types

Sounds good. For the casting bit, I made it based on type_traits abilities:
// Implements a casting function to make it compatible with scipy
// Use like cast<to_type>(value)

template <typename B, typename A>
__device__
typename enable_if<!is_floating_point<A>::value||is_signed<B>::value, B>::type
cast(A a) { return (B)a; }

template <typename B, typename A>
__device__
typename enable_if<is_floating_point<A>::value&&!is_signed<B>::value, B>::type
cast(A a) { return (a >= 0) ? (B)a : -(B)(-a); }

However, I then discovered you can't import <type_traits> in NVRTC (even though there is a CUDA-specific version in <cuda/std/type_traits> - they didn't mark the static constexpr functions as __device__ functions so it errors out...). So I need to provide a few <type_traits> pieces in addition to the 2 functions above:
// Small bit of <type_traits> which cannot be imported in NVRTC
// Requires compiling with --std=c++11 or higher
template<bool B, class T=void> struct enable_if {};
template<class T> struct enable_if<true, T> { typedef T type; };
template<class T> struct remove_const          { typedef T type; };
template<class T> struct remove_const<const T> { typedef T type; };
template<class T> struct remove_volatile             { typedef T type; };
template<class T> struct remove_volatile<volatile T> { typedef T type; };
template<class T> struct remove_cv {
  typedef typename remove_volatile<typename remove_const<T>::type>::type type;
};
template<class T, T v>
struct integral_constant { static constexpr T value = v; };
//struct true_type { static constexpr bool value = true; };
//struct false_type { static constexpr bool value = false; };
typedef integral_constant<bool, true> true_type;
typedef integral_constant<bool, false> false_type;
template<class T> struct __is_fp : public false_type {};
template<>        struct __is_fp<float> : public true_type {};
template<>        struct __is_fp<double> : public true_type {};
template<>        struct __is_fp<long double> : public true_type {};
template<class T> struct is_floating_point
    : public __is_fp<typename remove_cv<T>::type> {};
template<class T> struct is_signed : integral_constant<bool, (T)(-1)<0> {};


[17.10] Increase gRPC request timeout to 20 seconds when sending snapshots

Cherry pick #2391.
git cherry-pick -s -x e3e2821fe3eae707915b78215526da078d2d75a7

Ping @andrewhsu

@anshulpundir which other branches does this commit need to be cherry picked onto? I want to make sure that #2391 and #2375 go together everywhere.


v6.3.0 can't view pod logs

Describe the bug
The menu option to view logs for a pod is not available. It was in the previous version
Also refresh (cmd-R) from pods tab loses the namespace selections and reverts back to all namespaces.
To Reproduce
Steps to reproduce the behavior:

Goto to pods tab
click on the ... and logs is missing

Expected behavior
logs should be availavble
Screenshots
If applicable, add screenshots to help explain your problem.
Environment (please complete the following information):

Lens Version: 6.3.0
OS: OSX
Installation method (e.g. snap or AppImage in Linux):

Logs:
When you run the application executable from command line you will see some logging output. Please paste them here:
Your logs go here...

Kubeconfig:
Quite often the problems are caused by malformed kubeconfig which the application tries to load. Please share your kubeconfig, remember to remove any secret and sensitive information.
your kubeconfig here

Additional context
Add any other context about the problem here.

I've also lost the pod shells in 6.3.0, in addition to logs. Rolling back to 6.2.5 resolved both.

How did you roll back?

It seems to be intentional. See this comment:
https://github.com/lensapp/lens/issues/6819#issuecomment-1363751421

duplicate of #6823

I'd have a lot more respect for you if you admitted you'd taken the decision to take core functionality closed source for commerical reasons.


wip: fix incorrect date/time order in rtl (pt 2)

Closes
✅ Pull Request Checklist:

[ ] Included link to corresponding React Spectrum GitHub Issue.
[ ] Added/updated unit tests and storybook for this change (for new code or code which already has tests).
[ ] Filled out test instructions.
[ ] Updated documentation (if it already exists for this component).
[ ] Looked at the Accessibility Practices for this feature - Aria Practices

📝 Test Instructions:

🧢 Your Project:

Hey, Thanks for working on this PR
Is this also going to fix keyboard focus management on RTL too?
For example DateField component in the storybook link, if you pick Farsi locale, in order to focus DateSegments with arrow key you need to do it opposite direction to focus next date segment

DateField with Farsi locale:

yyy/mm/dd, format is right ✅
focus management is not right ❌

DatePicker with Farsi locale:

dd/mm/yyy, format is not right ❌
focus management is right ✅

@sadeghbarati Yes, the plan is to also correct the keyboard navigation for RTL as well. The PR is still in the early stages so there's a lot that has yet to be implemented.


Is possible improve the performance by a clip window?

Hi,
My question is I have a object but it just a small area in the screen, I only need render the green area is ok, but the full framebuffer is large, it will take me a lot of CPU resource to rendering full view(most is black). does ospray have any way to improve the performance by this use case?
Thanks,
Chengliang

The imageStart and imageEnd options in camera can change the rendering region to any region we want. I feel that's exactly what you want :)

imageStart/End can indeed be used to restrict rendering to e.g. the green area, however, the complete framebuffer will be filled. So, you would need to create a new, smaller framebuffer and later manually copy its content into the blue framebuffer.
Alternatively, if you actually accumulate many frames for the final rendering you can also try using the varianceThreshold renderer parameter together with a OSP_FB_VARIANCE framebuffer, then OSPray will automatically not compute the black (marked blue) regions, because its variance should be zero. See also the documentation, Sections Renderer and Rendering

Please reopen if you have further questions.


Project Dependencies Not Installing

I'm trying to follow the tutorial in order to build the starter app, but when i reach the step pip install -r requirements.txt, my terminal outputs a lot of error messages:
Collecting flask (from -r requirements.txt (line 1))
Using cached https://files.pythonhosted.org/packages/f2/28/2a03252dfb9ebf377f40fba6a7841b47083260bf8bd8e737b0c6952df83f/Flask-1.1.2-py2.py3-none-any.whl
Collecting Flask-Session from git+https://github.com/SqrtMinusOne/flask-session.git@560d00f1a84a9924d788a4f4e1ef35c5cf94c76d#egg=Flask-Session (from -r requirements.txt (line 5))
Cloning https://github.com/SqrtMinusOne/flask-session.git (to revision 560d00f1a84a9924d788a4f4e1ef35c5cf94c76d) to /private/var/folders/bg/y3y5xyh53w5f0b_4y9mrjyd40000gn/T/pip-install-8_kw6l5_/Flask-Session
<EMAIL_ADDRESS>Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@github.com:mitsuhiko/flask-sphinx-themes.git' into submodule path '/private/var/folders/bg/y3y5xyh53w5f0b_4y9mrjyd40000gn/T/pip-install-8_kw6l5_/Flask-Session/docs/_themes' failed
Failed to clone 'docs/_themes'. Retry scheduled
<EMAIL_ADDRESS>Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
fatal: clone of 'git@github.com:mitsuhiko/flask-sphinx-themes.git' into submodule path '/private/var/folders/bg/y3y5xyh53w5f0b_4y9mrjyd40000gn/T/pip-install-8_kw6l5_/Flask-Session/docs/_themes' failed
Failed to clone 'docs/themes' a second time, aborting
Command "git submodule update --init --recursive -q" failed with error code 1 in /private/var/folders/bg/y3y5xyh53w5f0b_4y9mrjyd40000gn/T/pip-install-8_kw6l5/Flask-Session
However, according to my IDE (vs code, running inside the venv), none of the dependencies have been installed. What could be the issue?
I'm running macOS Catalina v 10.15.5

@24anisha - Are you still having issues?
I'm not 100% sure why you're getting this error.  Have you check that you SSH keys are setup for your github profile?
https://github.com/settings/keys
We ran into some issues with the current version of Flask-Session and had to point to a specific commit and download that code into our project.  But in order to do that your SSH keys need to be config.
https://docs.github.com/en/github/authenticating-to-github/connecting-to-github-with-ssh
Let me know if this helps.

Above worked for me

closing this one off as resolved.


Can u gave some model  to test?

Sure. I will release the trained model soon.

Hi,
I have two questions and hope you can help me:

In SRGAN, the G network is supervised by the high-resolution images. Do I understand correctly?
How many epochs does this network need to generate good super-resolved images?

BTW, there's a typo in line 41 of run_test.lua, where a comma is missed.

Yes it is correct.
For me, I trained 3 epochs on ImageNet and it looks good already.

@leehomyc can you gave some models to test and compare?

@leehomyc - 3 epochs on image net, but how total images were you training on?


Improve forgot password error

Forgot password should indicate the reset code is consumed.

I'm not sure what does this issue meant. Need better description / explanation.

@chpapa updated.

@kiootic Thanks. Clearer now. Maybe we can cross check some OWASP best practices on should we indicate the code is used? Just my 2 cents :)

I have read https://cheatsheetseries.owasp.org/cheatsheets/Forgot_Password_Cheat_Sheet.html
but it doesn't mention whether we should display expired token error to the user. However, it does mention that in auditing the system should log failed attempted user of expired tokens. So we should keep the consumed token until it is expired.


FormModel::attributes renamed FormModel::attributeTypes

Until I looked at the initialization $attributes, I thought it was an array of attribute names
FormModel::attributes is [name attribute => type attribute]

👍


QnA knowledge base cannot be created from authoring canvas

Describe the bug
On Electron,

created QnA not be imported by dialog
design page create met error

Version

Browser

[ ] Electron distribution
[ ] Chrome
[ ] Safari
[ ] Firefox
[ ] Edge

OS

[ ] macOS
[ ] Windows
[ ] Ubuntu

To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior

Screenshots

Additional context

Can't repro. Close it for now


Don't Merge! PR real time test for webhook case

Docs Build status updates of commit a329638:
:clock10: Pending: waiting for processors (24 builds ahead of you)
:warning: Docs Build is busy, currently there are 24 builds ahead of this one, for more information you can view the Build queue graph on the Docs Portal.

Docs Build status updates of commit a329638:
:clock10: Full build: the average full build time is 6 min(s) 18 sec(s), this is based on the last 30 build(s) for this repository.

Docs Build status updates of commit a329638:
:warning: Validation status: warnings

File
Status
Preview URL
Details

docs-ref-conceptual/testing.md
:warning:Warning
View (azure-cli-latest) View (azure-cli-2017-03-09-profile) View (azure-cli-2018-03-01-hybrid)
Details

docs-ref-conceptual/testing.md

[Warning] Missing attribute: either ms.prod or ms.service is required. Use ms.prod for on-premise products, or ms.service for cloud services.
[Warning] Missing attribute: title.
[Warning] Missing attribute: either ms.prod or ms.service is required. Use ms.prod for on-premise products, or ms.service for cloud services.
[Warning] Missing attribute: title.
[Warning] Missing attribute: either ms.prod or ms.service is required. Use ms.prod for on-premise products, or ms.service for cloud services.
[Warning] Missing attribute: title.

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Prep work for conda install

This tiny PR is just some preparation for the conda install that I am working on.
Conda allows testing as part of the build.
So this tiny PR adds the test folder to the pypi distribution so that the conda build can make use of the tests.
Also removed myself and the old repository from the setup.py

no changes to the actual code, just renaming things now that it is in the Svalinn repo


Feature Gate: extends Turbine fanout experiment to wider fanout values

SIMD

Description
https://github.com/anza-xyz/agave/pull/2373
Based on previous Turbine fanout experiment, larger fanouts are more effective in propagating shreds and reducing repairs: https://discord.com/channels/428295358100013066/478692221441409024/1265782094211321897
However the optimal fanout value cannot be inferred from the experiment.
In order to identify optimal fanout value, this commit extends the experiment with wider fanout values.
Feature ID
BZn14Liea52wtBwrXUxTv6vojuTTmfc7XGEDTXrvMD7b
Activation Method
Single Core Contributor
Deployment Considerations

Minimum Beta Version
v2.0.10
Minimum Stable Version
v1.18.24
Testnet Activation Epoch
No response
Devnet Activation Epoch
No response
Mainnet-Beta Activation Epoch
No response

Replacing this with:
https://github.com/orgs/anza-xyz/projects/9/views/1?pane=issue&itemId=86808518&issue=anza-xyz|feature-gate-tracker|63
Creating the feature gates in the anza-xyz/feature-gate-tracker makes labels accessible and keeps permissions consistent


GREAT WORK

So I have struggled with this build for one year. It seems to be very hard....
And HA is constantly changing. Now the new version doesnt support UART?? Is that a problem?
Anyway i am again stuck.
I see that

Also the readme i got stuck at

I dont know how how to do no. 6 and 7.
and also i got now two red nodes??

And I still dont get any readings from EZo circuits. I Dont know how to enable i2C mode....
and also
what is this?

what is sparkly local i dont understans??
PLEASE HELP

To set EZO sensors to I2C mode, you can follow this tutorial. My version does not support UART mode, you would have to make it work if you really wanted to (the code is there, but commented out). I highly recommend using I2C mode though, it is easier to use and more reliable.
HA is indeed constantly changing (and not for the better IMO). I assume you installed the supervised version of HA, seeing the screenshots - and that is OK. Steps 6 & 7 are irrelevant to you, NR is running and connected to HA apparently, and the second NR entry is useless - just ignore it or remove it from your configuration file.
As I mentioned in the readme, the installation is very much a work in progress and I use HA Core, where some if this stuff is more complicated - but it runs much better on my RPi Zero W 2. I'm actually happy you made it that far, if you can contribute back with Install steps for HA Supervised, that would be nice.
Finally, my installation is called sparkly.local instead of homeassistant.local, so you already have that last point (Start HA and access) working :)

OK KUULL
Getting there!
So I enabled I2C mode manually on the EZO circuits now there is the Blue Light!
But still no communication with raspberry pi.
I know how to do it in raspbian OS
https://files.atlas-scientific.com/pi_sample_code.pdf
But how to do that in HA OS??
I found this should i do it?
https://www.home-assistant.io/common-tasks/os/#enable-i2c
I will gladly make a tutorial if I am Successful!

So I did this also..
https://www.home-assistant.io/common-tasks/os/#enable-i2c-via-home-assistant-operating-system-terminal
And nothing from the pH sensor...

Should sensors still be connected as segalion did??

Yeah, that won't work with I2C. So I guess you have a choice to make:

either figure out how to re-enable serial in sparkly (you should be able to by changing the ph.yaml and orp.yaml (the beginning of the file).
or figure out how to connect your sensors via I2C. Not sure if that Carter board works in I2C code, I am using the tentacle T3 carrier board.

Should sensors still be connected as segalion did??

have finally got this working after much trial and error, the sensor boards need to be connected as per the pi_sample.pdf file not as per the image in the repo, so RX on both boards need connecting to the SCL pin on the PI and the TX on both boards need to be connected to the SDA pin on the PI
then HA correctly finds them and reads :)

Hello I am Back!
Now i finally got the Whitebox T3 carrier board for RPI. Now still nothing in HA no readings from sensors...
What to do. Can someone write a quick tutorial.??
THANK YOU

Hi @bbombek, how did you get on or have you quit?
I received my Tentacle T3 today and have hooked up the EZO pH and ORP circuits and it appears to be working.

Although there is a weird difference between the 2 pH values being presented which I couldn't be bothered thinking about right now.
I'm happy for us to compare notes if you are still having issues.


Gruntfile.js: Use single quotes in htmlmin config to make JSCS happy

FYI @XhmikosR.

Tests passed. Automated cross-browser testing via Sauce Labs and Travis CI shows that the JavaScript changes in this pull request are: CONFIRMED
Commit: d567893f65fc23d7d28c2c49a4d1b4cb62c5d72c
Build details: https://travis-ci.org/twbs-savage/bootstrap/builds/145304843
Docs preview: http://preview.twbsapps.com/c/${commitSha.sha}
(Please note that this is a fully automated comment.)

Thanks, bad copy paste from my end.


add linear tag to bug_report.md

bug reports created using templates will have linear tag applied

Good watch


Build Errors? "error: no match for ‘operator=’"

Here is the errors from my terminal:
Pastebin
Ubuntu 16.10
Here are my files just in case. (I didn't edit any code)
my nihstro files.tar.gz

Boost ticket #12802

Haven't had issues with this recently, so presumably this was fixed somewhere along the way.


0840 bankswitching support

The "Toyshop Trouble" PAL60 rom has just been posted on AA (http://atariage.com/forums/topic/273291-pal-version-of-toyshop-trouble/#entry3916621) and I noticed that stellerator doesn't support its bankswitching scheme (0840).
I don't know of other games utilizing that (The NTSC rom that was released a few years ago uses the classic F8 scheme)
Here are some info:
http://atariage.com/forums/blog/142/entry-2282-econobanking/

Thanks a lot, Alex, I'll add support these days --- suits the holiday theme :)

Fixed and published :) Thanks again.


Test against PHP 7.2

With the release of PHP 7.2, would be interesting to test it.

The test currently fail because the required extensions can not be installed on PHP 7.2. Can you look into updating the travis-init.sh?

@clue Done!


Add support for Payment Handler API

The Payment Handler API is missing. It is having support from Chrome and Opera, Mozilla is working on it.
https://www.w3.org/TR/payment-handler/

Duplicate of #4416.


Add do/ssh module

The question is; should we still implement taking ssh-key as alternative of:

explicite provided key,
file

Or, we can stay with file only.
Signed-off-by: Zabielski, Kamil<EMAIL_ADDRESS>

@kamilzabielski are you asking if we should follow the pattern from the AWS module?

@kwilczynski Exacly.


Release to main from develop

:tada: This PR is included in version 4.1.11 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Add YEGSec to the Meetups List

What issue is this referencing?
<!--
Reference an issue by typing # (outside of this comment) and then searching whatever key words

This PR fixes

## Do these code changes work locally and have you tested that they fix the issue yourself?

-   [ ] yes!

## Does the following command run without warnings or errors?

-   [*] yes! `npm run pr-checks`

## Have you taken a look at our [contributing guidelines](https://github.com/devedmonton/DES-Website/blob/main/.github/CONTRIBUTING.md)?

-   [*] yes!

## My node version matches the one suggested when running `nvm use`?

-   [*] yes!

Hi @deejayjay. I do not have an option to assign a reviewer. May I am missing something.

Hi @deejayjay. I do not have an option to assign a reviewer. May I am missing something.

Do you have this gear icon to the right of this line?

If yes, click on it and you can add reviewers from there. I can add reviewers for now.

I do not have the gear...

Thanks for adding reviewers.

Thanks for the Git history feedback, Mandy. I will look into that in my next commit. I was annoyed by how I could see my previous PRs...lol. Let me pull master. Oh, by the way, I can now see the gear, I got the triage access invite. Thanks so much.


Add warning for servers potentially blocking CLIENT commands

This bite me pretty hard today https://github.com/whatyouhide/redix/issues/192 so I thought it would be nice to mention this issue in the doc, especially given that the error's trace is not immediately obvious:
** (ArgumentError) argument error
 (stdlib <IP_ADDRESS>) :ets.lookup_element(#Reference<0.3067823838.3890610178.94977>, :"$end_of_table", 3)
 (redix 1.1.2) lib/redix/socket_owner.ex:114: Redix.SocketOwner.peek_element_in_queue/2
 (redix 1.1.2) lib/redix/socket_owner.ex:90: Redix.SocketOwner.new_data/2
 (redix 1.1.2) lib/redix/socket_owner.ex:56: Redix.SocketOwner.handle_info/2
 (stdlib <IP_ADDRESS>) gen_server.erl:689: :gen_server.try_dispatch/4
 (stdlib <IP_ADDRESS>) gen_server.erl:765: :gen_server.handle_msg/6
 (stdlib <IP_ADDRESS>) proc_lib.erl:226: :proc_lib.init_p_do_apply/3
Last message: {:tcp, #Port<0.46>, "-ERR unknown command `CLIENT`, with args beginning with: `REPLY`, `OFF`, \r\n:1\r\n-ERR unknown command `CLIENT`, with args beginning with: `REPLY`, `ON`, \r\n"}

I pushed a commit to make this easier to debug so that the error won't be as obscure (https://github.com/whatyouhide/redix/commit/10b3193f9cb21338202fb26d6d70e940ef5ba308).
Thanks @martosaur! 💟


Remove a duplicated sentence

Just a small doc fix.

Maybe the emphasis was needed? 😄

@smoothdeveloper Well obviously, the 3 repetitions means we're only looking for 3x developers... That's a low goal. We should repeat 7 more times to get the 10x developers contributing!


Question: Should PollRecords create batches close to the max records argument if a consumer is lagging

Recently saw behavior in one of our applications where a consumer that was very behind on a topic was getting record sizes well below the max record parameter in the Client.PollRecord method.  Is that expected?  If so for what reason?

PollRecord only drains from the currently buffered queue, up to the requested record limit. The client buffers up to MaxConcurrentFetches of fetches at once (by default, one fetch per broker). Each fetch itself is controlled in size by FetchMaxBytes (by default, 50MiB per fetch) and FetchMaxPartitionBytes (by default, 1MiB per partition within each fetch).

How many partitions are being fetched?
Are any of the given limits overrode?

15 (spread 10/5 across 2 topics).
No overrides.

Might want to increase the FexMatchPartitionBytes as well, but 1MiB should be enough for the default small-records case. Anyway, lmk if tuning any settings fixes things, thanks!

Our records are a fairly big (1.5K-75K) so I think the partition limit is precisely the problem.  Have upped it and am running experiments now.

I've also always considered if a second fetch should be buffered for a single broker in some circumstances, but I don't think the ramifications for that are great (nor do I know what that'd affect inside the code)

I agree that would be harder to reason about.  For us it was simply hitting the partition maximum.  Changing that to a more appropriate value fixed the issue.
If we had more partitions its possible we'd hit the fetch max but currently we are off of that a bit.


Fix: Change -p option to reference the directory path of compile_commands.json instead of the file path

To fix issue 211

@JonathanFoo0523 can you rebase this over main and attend to the comment? Then I can take a final look and hopefully merge.

@JonathanFoo0523 Can you take another look at this PR please?

@afd Done. Sorry for the late response. I overlooked this thread.


performance?

I'm curious to know if you did any benchmarking or if you have any idea how much does this method save over the existing method of compiling to state machines ala regenerator.

No idea, it would be nice to do a few tests on this & and the results to the README file, though.


Custom icons doesn't work when updating

I am not able to edit existing application and replace the mdi icon.
Edit an application,  click on "switch to custom icon",  select a file,  save.  Message at bottom of the screen indicates success.  However,  the icon doesn't actually change.
Custom icons works fine on new entries.

Same problem here, I have to create new one if I have to edit one app


remove boost dependencies for the library

boost program options
boost pp (for f2f)
https://github.com/noma/ham/blob/7466a17f8d34ae107cb88bcb8b2a84da25a1b23d/include/ham/functor/function.hpp#L76-L122

Would shipping all boost dependencies in thirdparty and integrating them directly into the CMake build be an option for you?
Boost PP is header only, so shouldn't matter anyways.

We could change f2f in the following ways:

We use the original definition of f2f
#define f2f(function_ptr, ...) ham::function<decltype(function_ptr), function_ptr>(__VA_ARGS__)

and add f2f_zero for zero arguments
#define f2f_zero(function_ptr) ham::function<decltype(function_ptr), function_ptr>()

:+1: f2f stays the same
:+1: f2f can be used with zero arguments if the user does not bother about the warning
:-1: Introduces a new "name"

We change the definition of f2f
#define f2f(function_ptr) ham::function<decltype(function_ptr), function_ptr>

invocation would change to
f2f(&function)(arg0, arg1, arg2);

:+1: f2f now clearly differentiate between type and variables
:-1: breaks backwards compatibility

Note: as discussed yesterday for C++ 17:
template<auto function_ptr>
using f2f = ham::function<decltype(function_ptr), function_ptr>;

f2f<function>(args...)

So we have:
f2f(function, args...); // original syntax: BOOST_PP_OVERLOAD for portability and no compiler warnings
f2f(function)(args...); // C++11, still Macros, but nor variadic macro necessary
f2f<function>(args...); // C++17, no macros :-)

TODO:
decide what to offer, naming to avoid conflicts, and whether or not to drop backward compatibility

With respect to the standard library's std::functionand std::bind are closest relatives, f2f is very similar to bind, while the result is close to std::function:
For C++17 I'd propose the following:

remove f2f alltogether to get rid of the macro and not overload the old construct with something new

maybe add a dummy function instead of the macro and mark it deprecated to generate useful compiler errors for old code

introduce ham::function instead

ham::sync(ham::function<function_ptr>(arg1, arg2, ...)) // 3rd from above

@marehr opinions?

sounds good.

Removed Boost Program Options in favor of header only CLI11 lib. Only Boost PP remains, but is header only and shipped in thirdparty folder, for C++11/14 compatibility.


Add missing material

The indexes of the Potion list seem to have shifted by 1, and this Vine material seems to be responsible. This still needs to be properly verified however

... This seems to actually not be a material in the potion list... i'm unsure why the index is 1 higher than it should be (search monstrous powder -> get rat powder for example)

i was wrong, the atlas works as intended (tested by spawning potions at the chest coordinates, which worked flawlessly), just something was up with Dunk's game i guess? works consistently, as if there was no issue in the first place


Cannot import css from dropzone package when build

I tried to use vue2-dropzone with Nuxt.js (with SSR)
When i imported source file src/index.vue, and started build, i got error:
Cannot find module "-!../../../../../../../usr/local/lib/node_modules/nuxt/node_modules/css-loader/index.js

I try to change import dropzone.css in src/index.vue from ../node_modules/dropzone/dist/dropzone.css to ~dropzone/dist/dropzone.css as it says documentation for webpack less-loader and it works

Resolved by pull request #51 , released to npm as v 2.2.4


Implement User Authentication with JWT for Sign-up, Verification, and Login Endpoints #

What does this PR do?
This PR implements backend functionality for user login with email and password verification. It ensures secure authentication by hashing passwords using the bcrypt library in Python. Please note that verification setup should be customized according to your specific requirements, as I am not aware of the preferred methods you may use. For more information, please refer to the attached demonstration video.
What issues does this PR fix or reference?
This PR addresses the issue of securely logging in users with an email and password. It hashes user passwords to enhance security. For further reference on password hashing, you can visit the bcrypt library documentation: https://www.npmjs.com/package/bcrypt
https://github.com/user-attachments/assets/c9ffcfbc-994a-4db6-b846-4eedc4eb6453

@jayzalani There's been some changes added to master since you started development (see #236). You'll want to merge in current master and resolve the conflicts/re-test some of your signup/login flows.
There's more covered in the PR #236 comment but main thing is that the User model covers the 3 sign up/login flows. Creation of account (via email/password, Google, and Github) should all create a row in the same User table (normalized model), if using a OAuth provider hashed_password is left empty, and a provider is set (with the OAuth token).
Hopefully you can modify what you've already done to work with the new changes. Let me know if you have any questions also happy to work with you on finishing this up. It looks great so far!

I see that this branch is very WIP.
For now, I have rebased master into this branch. You should see the jayzalani-rebased branch. It has a few fixes as well addressing Ben's comments

A few comments.
You do not need to add # Add this line to indicate your diffs. Git does that for you. In particular, it is ideal that you make smaller commits rather than putting all your changes into one big commit.
I have no idea why you merged your branch into itself.

Also, because Ben put some other TO-DO stuff in other parts of the code, just a note to everyone: we haven't actuall implemented token verification logic yet. The relevant comment is
# TODO: Implement token verification logic
so you can sed for it.

Understood. I see the issues now. I'll modify the code and retest everything soon. Thanks for the feedback and support

Rebased on master again.


(GH) Only the last miniature is displayed in product addition

What

Only the last miniature is displayed in product addition

@teolemon
Since we delete the cropped image from the cache when it's done uploading, the flutter engine doesn't find any file at that location to render that image, and also the cache file needs to be deleted explicitly when done uploading ,
Possible solutions:

Don't show the miniature pictures at all, rather use a green tick ✅ to show that it's done.

Looks like it's already been coded.


release 0.1.1

Closed by https://github.com/nacos-cli/ncli/commit/6e41c0b3d0467212c8f1bd3a1f7b4064cfe2a660


Add priority settings to advanced firewall rule options

This will add 2 additional advanced options for firewall rules. One to set the priority, and one to match based on the priority. This should be useful for Google Fiber users to unlock full upload speed.

After testing the functionality in pf itself, i realized that "set prio" is not functioning, or at the very least, it is not setting the vlan pcp value, which makes it kind of useless.
After realizing this, I did some searching and realized that the vlanpcp setting in ifconfig was also brought into FreeBSD 11, and it does work to set a static outgoing pcp for the vlan, which I have created a (much smaller) patch for in #1394. This new patch has been tested by a user on google fiber, and it does work as expected, unlocking the full upload speed when the VLAN Priority is set to 3.
Based on this, I am wondering if we should just close this pull request, and perhaps re-visit this once the bugs are worked out of the pf set prio rules.

I have pushed fixes for everything mentioned (i think), figured out how to use 0 directly in the rules, and also added support for dual priorities in set prio.
Keep in mind that as far as i can tell, set prio does not work (a FreeBSD issue), I have no way to to test the match priority, but the rules are generated correctly.

After testing this on stock FreeBSD 11, I can definitely confirm that set prio does not work, even a rule as simple as:
pass out quick on vtnet0.6 set prio 2
Then using:
tcpdump -eni vtnet0 'vlan'
The pcp value (show as simply p) stays 0 on every vlan packet, yet when changing the vlanpcp setting with ifconfig, it is immediately seen in the dump.

merged, thanks! some more changes follow in a bit


Try sharing convolution filters between populations

Separate filters seem to work fine, but pg-gan shares the filters between populations, so it might be worth testing this.

Added discriminator that matches pg-gan.


FunctionClauseError

I am a Rails dev just getting started with Elixir. Thanks for your patience.
In a Phoenix app I am fetching a list of objects from MongoDB and rendering as json, like so:
def index(conn, _params) do
  mongo = Mongo.connect!
  db = mongo |> Mongo.db("my_db")
  collection = db |> Mongo.Db.collection("my_collection")
  items = collection |> Mongo.Collection.find() |> Enum.to_list
  json conn, items
end

I get this error:
** (exit) an exception was raised:
    ** (FunctionClauseError) no function clause matching in Poison.Encoder.BitString.chunk_size/3
        (poison) lib/poison/encoder.ex:127: Poison.Encoder.BitString.chunk_size(<<176, 87, 109, 84, 105, 109, 111, 60, 0, 0, 18>>, nil, 1)
        (poison) lib/poison/encoder.ex:122: Poison.Encoder.BitString.escape/2
        (poison) lib/poison/encoder.ex:84: Poison.Encoder.BitString.encode/2
        (poison) lib/poison/encoder.ex:213: anonymous fn/4 in Poison.Encoder.Map.encode/3
        (poison) lib/poison/encoder.ex:214: Poison.Encoder.Map."-encode/3-lists^foldl/2-0-"/3
        (poison) lib/poison/encoder.ex:214: Poison.Encoder.Map.encode/3
        (poison) lib/poison/encoder.ex:213: anonymous fn/4 in Poison.Encoder.Map.encode/3
        (poison) lib/poison/encoder.ex:214: Poison.Encoder.Map."-encode/3-lists^foldl/2-0-"/3
        (poison) lib/poison/encoder.ex:214: Poison.Encoder.Map.encode/3
        (poison) lib/poison/encoder.ex:232: anonymous fn/3 in Poison.Encoder.List.encode/3
        (poison) lib/poison/encoder.ex:233: Poison.Encoder.List."-encode/3-lists^foldr/2-1-"/3
        (poison) lib/poison/encoder.ex:233: Poison.Encoder.List.encode/3
        (poison) lib/poison.ex:41: Poison.encode!/2
        (phoenix) lib/phoenix/controller.ex:186: Phoenix.Controller.json/2
        (my_app) web/controllers/items_controller.ex:2: MyApp.ItemsController.action/2
        (my_app) web/controllers/items_controller.ex:2: MyApp.ItemsController.phoenix_controller_pipeline/2
        (my_app) lib/phoenix/router.ex:261: MyApp.Router.dispatch/2
        (my_app) web/router.ex:1: MyApp.Router.do_call/2
        (my_app) lib/my_app/endpoint.ex:1: MyApp.Endpoint.phoenix_pipeline/1
        (my_app) lib/plug/debugger.ex:93: MyApp.Endpoint."call (overridable 3)"/2

items is a list of maps. Each map represents a MongoDB document. Each document has a MongoDB assigned id like this _id: ObjectId(56b0576d54696d6f3c000011). Some of the values in each document are lists of string and others are lists of maps. Some of values are strings with escaped double quotes in them.
Any idea what might be wrong?

JSON does not support binary data so if you have structs that have binaries (for example ObjectId) you need to implement JSON.Encoder for them and convert the binaries to utf8 encoded strings.

Correct, it looks like <<176, 87, 109, 84, 105, 109, 111, 60, 0, 0, 18>> is a non utf-8 string.

Thanks. In case anyone else encounters this, I solved it by adding this penultimate line:
items = for item <- items, do: Map.put item, :_id, to_string(item._id)

Being an Elixir newb, I suppose this is not perfectly elegant or idiomatic, but it does what I want.

Hi there,
I added the line below but it is still not working,
items = for item <- items, do: Map.put item, :_id, to_string(item._id)
I get:
key :_id not found in: %{"_id" => #BSON.ObjectId<59e94002a28729527df848c1>
Could you please help me? Thank you!

Hello, I fixed by adding this in my application.ex in the lib folder
defimpl Poison.Encoder, for: BSON.ObjectId do
def encode(id, options) do
BSON.ObjectId.encode!(id) |> Poison.Encoder.encode(options)
end
end


add hostname to output of check nodes command

Current coverage is 79.73% (diff: 100%)

Merging #147 into master will not change coverage

@@             master       #147   diff @@
==========================================
  Files            16         16
  Lines          1653       1653
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
  Hits           1318       1318
  Misses          335        335
  Partials          0          0

Powered by Codecov. Last update a534371...d25b8a2

wow, this is awesome.
ACK


Problem with own task

Hello,
i write two tasks and wan't to use it in a laravel Projekt:
namespace Deployer;

desc('Run PHP Unit local');
task('local:phpunit', function () {
    $output = run("cd {{local_release_path}} && phpunit");
    writeln('<info>' . $output . '</info>');
});

desc('Run PHP Unit remote');
task('remote:phpunit', function () {
    $output = run("cd {{release_path}} && phpunit");
    writeln('<info>' . $output . '</info>');
});

When i run deployer i get the error message:
In functions.php line 160:
Call to a member function set() on null

What do i wrong?
Cheers
Ralf

Please, show full file.

What php version? Can you show full error cs.

Hi,
i use php 7.1.1, the full error is:
Notice: Trying to get property of non-object in /var/www/api/vendor/deployer/deployer/src/functions.php on line 160

Call Stack:
    0.0108     359680   1. {main}() /var/www/api/artisan:0
    0.1178    3626368   2. Illuminate\Foundation\Console\Kernel->handle() /var/www/api/artisan:37
    0.1178    3626368   3. Illuminate\Foundation\Console\Kernel->bootstrap() /var/www/api/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php:119
    0.1179    3626368   4. Illuminate\Foundation\Application->bootstrapWith() /var/www/api/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php:296
    0.1357    3735312   5. Illuminate\Foundation\Bootstrap\LoadConfiguration->bootstrap() /var/www/api/vendor/laravel/framework/src/Illuminate/Foundation/Application.php:213
    0.1378    3753560   6. Illuminate\Foundation\Bootstrap\LoadConfiguration->loadConfigurationFiles() /var/www/api/vendor/laravel/framework/src/Illuminate/Foundation/Bootstrap/LoadConfiguration.php:39
    0.1665    4136760   7. require('/var/www/api/config/recipe/tasks.php') /var/www/api/vendor/laravel/framework/src/Illuminate/Foundation/Bootstrap/LoadConfiguration.php:71
    0.1665    4137104   8. Deployer\task() /var/www/api/config/recipe/tasks.php:9

In functions.php line 160:
Call to a member function set() on null

Cheers
Ralf

Looks, like your are using old version of deployer.

Part of my composer.json:
"require": {
        "php": ">=7.0.0",
        "fideloper/proxy": "~4.0",
        "guzzlehttp/guzzle": "~6.0",
        "laravel/framework": "5.5.*",
        "laravel/tinker": "~1.0",
        "lorisleiva/laravel-deployer": "^0.2.2",
        "sentry/sentry-laravel": "^0.9.0",
        "tymon/jwt-auth": "^1.0"
    },
    "require-dev": {
        "deployer/recipes": "^6.1",
        "filp/whoops": "~2.0",
        "fzaninotto/faker": "~1.4",
        "mockery/mockery": "~1.0",
        "nunomaduro/collision": "~2.0",
        "phpunit/phpunit": "~7.0",
        "symfony/thanks": "^1.0"
    },

Run dep --version

MacBook-Pro-4:bin Ralf$ ./dep --version
Deployer master

Try to use not master, but latest phar.

i use https://github.com/lorisleiva/laravel-deployer, how can i use phar?

Can't tell. Looks like you using old version of deployer where what bug with conflicting set() function from laravel.
In your stack trase I see laravel called set. not deployer.

Ok, i ask in an issue by laravel-developer. Thx for your support!


Error when running tests with karma and phantomjs

Hello.
I'm tying to write tests on the project I work and I can't, because I get next error:
PhantomJS 2.1.1 (Linux 0.0.0) ERROR
  SyntaxError: Use of reserved word 'class'
  at http://localhost:8081webpack:///node_modules/vuejs-datatable/src/classes/settings.js:3:0 <- index.js:15069

Is it possible somehow to fix or to avoid it?
Thanks.

Seems to be related to a version problem with node. Phantomjs used to rely on a quite old & outdated version of the js engine. Have you tried updating both karma & phantomjs to the latest versions?

Even if I guess you don't work on the project anymore, and the thread is old, I'd like to let you know that I'm totally rewriting this part of the package.

@GerkinDev I don't have this issue for now, but I use this package in my projects. So, I would like to see updates :+1: Thank you very much for your work!

Hey there,
I'm currently catching up with the BrowserStack team to integrate E2E tests as part of the Continuous Integration process I'm setting up on the package.
Have you tried the solution I've suggested above, concerning transcripting your code to pre-ES6 before executing tests with Karma?
Cheers

@GerkinDev sorry but I finished the project some months ago and I don't have a chance to test it for now :slightly_frowning_face:

E2E testing was delegated to Cypress, which will soon include adapter for browserstack for IEx testing


Bump up Yorkie to 0.4.1

What this PR does / why we need it?
Bump up Yorkie to 0.4.1.
Any background context you want to provide?
What are the relevant tickets?

Fixes #
Checklist

[x] Added relevant tests or not required
[x] Didn't break anything

@krapie Thanks for your contribution.
But after merging this PR, there was an issue that the initial content of the document was not displayed.
Currently, I've redeployed with a previous commit. 🙏


Allow client_id and client_secret to taken from Authorization header.

Currently, Google action's account linking allows the client to let "Google to transmit clientID and secret via HTTP basic auth header".

If the option's checked, when the user makes a SYNC request, Google'll return client_id and client_secret via neither request's query string nor request's POST body. But those value with return via Authorization request's header as a base64 encoded string.  So, the application can't find those value and return 400 error because 'missing required parameter'.
I'd like to make a PR a patch to allow using those values from the request's header as I mentioned. I've tested my code and it works as I intended.  So, I can make PR as soon as I approved.
If you have any questions regarding this issue. Please reply to this issue.
Regards,
Edit: Just to clarify, the following functions are affected by this issue:

smart-home-provider/cloud/auth-provider.js

handleAuthCode
app.all('/token')

How do your auth changes work? Does the sample still work whether or not the box is checked?

How do your auth changes work?
I just create a function to decode Authorization string (if it was submitted). Then, allow clientId & clientSecret to take value from that header if the value submit via neither query string nor request body. So, if Google submits a value from either one of those, it should work just like the good old day!
Does the sample still work whether or not the box is checked?
I tried to uncheck and save the config. Unfortunately, Google still submits those value via header. So, I don't think unchecking work.

You may need to re-enable testing to verify that unchecking stops submitting the value (or maybe there's a bug). It looks like you may be handling both cases already though, which is good.
The PR can be useful in the short-term. We're going to be refactoring the sample soon, which will have some implications for this change, so it depends if you'd want to do it now or rebase it later.

Yeah, as I mentioned earlier, even when I uncheck that option, Google still submits value via header. 🤔

Since I already finish the code patching, I'll make a PR just to make sure it won't make some headache in the future.
Thank you for your attention.


Documentation on mobile devices bogus

Former versions of the docs worked okay on small screens, but the current version has a flaw: the toolbar buttons such as hamburger icon, search etc. do not react to clicks, possibly because they are overlapped by other elements that don't fit on narrow screens. It is barely usable like this on my smartphone.

Fixed in the current and upcoming versions.

Thank you so much, it works far better than ever before on both desktop and mobile!
There's no search in case of the desktop version however, whereas there's a custom Google search (which kinda fixes the former search problem) on my smartphone in the header next to the ArangoDB logo.

There is also a search box for the desktop version. You maybe have to delete your cache. If u have installed an ad blocker, this could also be the reason (because its a google based search).

Your right, it was Ghostery that blocked Google AJAX Search API and Google Custom Search, thanks again!


inappwebview can't listen touchmove

flutter_inappwebview 2.1.0+1
android
"touchstart" "touchend", both are ok, but it can't enter "touchmove" function.
No one keep to update this plugin anymore?

i fix it now...
if i set disableVerticalScroll=true & disableHorizontalScroll=true, the webview  can't listen "touchmove". but when they equals 'false', it works.
i think it's kinda rediculous ,, 'pagescroll' and 'touchmove' are different things,  i think


Fuzzilli: use libsocket.socket_recv instead of read

This is a cleanup change that uses socket_recv instead of a raw read
from the libc.  THe intent here is to be able to support Windows which
has a separate sockets library  which is not part of libc.

Ah great, thanks!


The commands for New-TenantAllowBlockListItems don't work. They say Allow and ListType aren't valid

[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b17ce75e-79ba-0acd-066c-a0120a29d65d
Version Independent ID: 14673b1c-8489-0abc-90a0-42f35d516424
Content: Configure the delivery of third-party phishing simulations to users and unfiltered messages to SecOps mailboxes - Office 365
Content Source: microsoft-365/security/office-365-security/configure-advanced-delivery.md
Product: m365-security
Technology: mdo
GitHub Login: @chrisda
Microsoft Alias: chrisda

@zinkco22, I copied the New-TenantAllowBlockListItems commands from Step 3 in an E5 nothing special test tenant as a Global admin, and it worked fine for me.

What version of the EXO V2 PowerShell module are you using? v2.0.5 is the latest general release, and v2.0.6-Preview6 is the latest Preview release. Make sure you're up to date. If you're using v2.0.6, try using the -UseRPPSSession switch in the connect command as described here.
Also, make sure you're connected to Exchange Online PowerShell before you run the command.

I’m running it with Business Premium and trying to run it in the Security and Compliance PowerShell since I want to add a set of URLs to the PhishingSimulation “Simulation URLs to allow”
For reference this is the script I’m trying to use.
New-TenantAllowBlockListItems -Allow -ListType Url -ListSubType AdvancedDelivery -Entries ".com-token-auth.com/",".com-onlinebanking.com/",".password.land/",".2o2.lol/",".protected-forms.com/",".net-login.com/",".ancillarycheese.com/",".bloemlight.com/",".comano.us/",".msftemail.com/" -NoExpiration
Cody Zink
M365CSAA
Security Implementation Engineer
worksighted | Professional Services
Need support?
phone:<PHONE_NUMBER>tel:616.546.2691 x1 |<PHONE_NUMBER>tel:888.978.7768
email<EMAIL_ADDRESS>
Customer Feedback | How am I doing?
Please contact my manager<EMAIL_ADDRESS>From: Chris Davis @.>
Sent: Tuesday, August 2, 2022 1:19 PM
To: MicrosoftDocs/microsoft-365-docs @.>
Cc: Cody Zink @.>; Mention @.>
Subject: Re: [MicrosoftDocs/microsoft-365-docs] The commands for New-TenantAllowBlockListItems don't work. They say Allow and ListType aren't valid (Issue #9304)
You don't often get email from<EMAIL_ADDRESS>Learn why this is importanthttps://aka.ms/LearnAboutSenderIdentification
@zinkco22https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fzinkco22&data=05|01|cody.zink%40worksighted.com|7abd89d7ac3948e92e1f08da74ab1fac|2b7b8007223d4e3d85331aa1bff3d819|0|0|637950575571635080|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=dKLzCnCQQunIndGuz2lFj%2BNXsuJO5o7KUnal8w%2BYUHI%3D&reserved=0, I copied the New-TenantAllowBlockListItems commands from Step 3 in an E5 nothing special test tenant as a Global admin, and it worked fine for me.

What version of the EXO V2 PowerShell module are you using? v2.0.5 is the latest general release, and v2.0.6-Preview6 is the latest Preview release. Make sure you're up to date. If you're using v2.0.6, try using the -UseRPPSSession switch in the connect command as described herehttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.microsoft.com%2Fpowershell%2Fexchange%2Fexchange-online-powershell-v2%23updates-for-version-206&data=05|01|cody.zink%40worksighted.com|7abd89d7ac3948e92e1f08da74ab1fac|2b7b8007223d4e3d85331aa1bff3d819|0|0|637950575571791295|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=fkysuKXnU9jiY2kD%2BPX3icMyEUmyUxdF52hwoW0hNFg%3D&reserved=0.
Also, make sure you're connected to Exchange Online PowerShellhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.microsoft.com%2Fpowershell%2Fexchange%2Fconnect-to-exchange-online-powershell&data=05|01|cody.zink%40worksighted.com|7abd89d7ac3948e92e1f08da74ab1fac|2b7b8007223d4e3d85331aa1bff3d819|0|0|637950575571791295|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=U6UtIfGq7DV%2FuFgApA6bbjnoGDXw58HiwF2WSmlmmJ0%3D&reserved=0 before you run the command.

—
Reply to this email directly, view it on GitHubhttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FMicrosoftDocs%2Fmicrosoft-365-docs%2Fissues%2F9304%23issuecomment-1203008662&data=05|01|cody.zink%40worksighted.com|7abd89d7ac3948e92e1f08da74ab1fac|2b7b8007223d4e3d85331aa1bff3d819|0|0|637950575571791295|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=dQ2KudrkVKbDIzURZbeGXXC%2FKoOtDImwKVhqbf4KKPQ%3D&reserved=0, or unsubscribehttps://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAPLQTWOVQZDN7PY2PSHI3EDVXFKBFANCNFSM55LHJQSA&data=05|01|cody.zink%40worksighted.com|7abd89d7ac3948e92e1f08da74ab1fac|2b7b8007223d4e3d85331aa1bff3d819|0|0|637950575571791295|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=iWQOS18R4dh5%2FGSiDp0eKYQL9Tz7KzoNRXqkhmh08sQ%3D&reserved=0.
You are receiving this because you were mentioned.Message ID<EMAIL_ADDRESS>

@zinkco22, I updated the topic to use Exchange Online PowerShell.
Thanks for the feedback.
Closing


Add modern __cplusplus behaviour to MSVC target compile options

Addition to further fix #475 without end-user CMake intervention
Adds /Zc:__cplusplus to MSVC compile options (as a hana target INTERFACE parameter) to force correct C++ standard reporting in __cplusplus variable.

Close for more fine-grained solution


Some refactor to enable to use createdisk.sh for microshift preset also

Initial refactor which help us to allow use of same script for microshift preset. Some more changes required but that will happen as part of #592 . So this initial patch is to make sure everything still works and able to generate the bundle.

/retest

/cherry-pick release-4.13


Getting new pages indexed quickly in Google

New cases take some time to get indexed in Google, which sometimes hinders our ability to be found via search.
For example, several days after the incidents, these pages are not indexed based on a check using the site: search operator:
http://ebwiki.org/articles/philando-castile
http://ebwiki.org/articles/alton-sterling

This one has been indexed within the past 2 minutes as I write this issue:

What may have helped is having Google fetch the pages individually and submitting them one by one to the index.

both pges are indexed at last check


feat(sdk): Remove unused files, corrections to docs

Codecov Report
Patch and project coverage have no change.

Comparison is base (7557cd7) 90.10% compared to head (2b0038f) 90.10%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #379   +/-   ##
=======================================
  Coverage   90.10%   90.10%
=======================================
  Files          74       74
  Lines        2608     2608
=======================================
  Hits         2350     2350
  Misses        181      181
  Partials       77       77

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Add override of changelogTemplate with link on entry

Following #163 on Ubuntu page it would be nice if we can override the changelog link of the changelogTemplate with a link on the entry itself. See link on entry for Ubuntu 20.10.
@captn3m0 please review

@captn3m0 Could do take a look of my proposed change?