Multiple numpy tests fail with numpy 1.19

The following four numpy op tests fails with numpy==1.19

test_np_delete
test_np_random_beta
test_np_random_f
test_np_random_chisquare

The failure happens on the master branch CI http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fwindows-cpu/detail/master/2034/pipeline and blocks recent PRs http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fwindows-cpu/detail/PR-18598/7/pipeline
It only happens recently, and I noticed that the installed numpy version on CI has changed from numpy-1.18.5 to numpy-1.19.0 - see installation log in previous commit and master head.
 For a one
[2020-06-20T23:26:16.963Z]         dimensional array, this returns those entries not returned by
[2020-06-20T23:26:16.963Z]         `arr[obj]`.
[2020-06-20T23:26:16.963Z]
[2020-06-20T23:26:16.963Z]         Parameters
[2020-06-20T23:26:16.963Z]         ----------
[2020-06-20T23:26:16.963Z]         arr : array_like
[2020-06-20T23:26:16.963Z]             Input array.
[2020-06-20T23:26:16.963Z]         obj : slice, int or array of ints
[2020-06-20T23:26:16.963Z]             Indicate indices of sub-arrays to remove along the specified axis.
[2020-06-20T23:26:16.963Z]
[2020-06-20T23:26:16.963Z]             .. versionchanged:: 1.19.0
[2020-06-20T23:26:16.963Z]                 Boolean indices are now treated as a mask of elements to remove,
[2020-06-20T23:26:16.963Z]                 rather than being cast to the integers 0 and 1.
[2020-06-20T23:26:16.963Z]
[2020-06-20T23:26:16.963Z]         axis : int, optional
[2020-06-20T23:26:16.963Z]             The axis along which to delete the subarray defined by `obj`.
[2020-06-20T23:26:16.963Z]             If `axis` is None, `obj` is applied to the flattened array.
[2020-06-20T23:26:16.963Z]
[2020-06-20T23:26:16.963Z]         Returns
[2020-06-20T23:26:16.963Z]         -------
[2020-06-20T23:26:16.963Z]         out : ndarray
[2020-06-20T23:26:16.963Z]             A copy of `arr` with the elements specified by `obj` removed. Note
[2020-06-20T23:26:16.963Z]             that `delete` does not occur in-place. If `axis` is None, `out` is
[2020-06-20T23:26:16.963Z]             a flattened array.
[2020-06-20T23:26:16.963Z]
[2020-06-20T23:26:16.963Z]         See Also
[2020-06-20T23:26:16.963Z]         --------
[2020-06-20T23:26:16.963Z]         insert : Insert elements into an array.
[2020-06-20T23:26:16.963Z]         append : Append elements at the end of an array.
[2020-06-20T23:26:16.963Z]
[2020-06-20T23:26:16.963Z]         Notes
[2020-06-20T23:26:16.963Z]         -----
[2020-06-20T23:26:16.963Z]         Often it is preferable to use a boolean mask.  For a one
[2020-06-22T00:36:45.758Z]         dimensional array, this returns those entries not returned by
[2020-06-22T00:36:45.758Z]         `arr[obj]`.
[2020-06-22T00:36:45.758Z]
[2020-06-22T00:36:45.758Z]         Parameters
[2020-06-22T00:36:45.758Z]         ----------
[2020-06-22T00:36:45.758Z]         arr : array_like
[2020-06-22T00:36:45.758Z]             Input array.
[2020-06-22T00:36:45.758Z]         obj : slice, int or array of ints
[2020-06-22T00:36:45.758Z]             Indicate indices of sub-arrays to remove along the specified axis.
[2020-06-22T00:36:45.758Z]
[2020-06-22T00:36:45.758Z]             .. versionchanged:: 1.19.0
[2020-06-22T00:36:45.758Z]                 Boolean indices are now treated as a mask of elements to remove,
[2020-06-22T00:36:45.758Z]                 rather than being cast to the integers 0 and 1.
[2020-06-22T00:36:45.758Z]
[2020-06-22T00:36:45.758Z]         axis : int, optional
[2020-06-22T00:36:45.758Z]             The axis along which to delete the subarray defined by `obj`.
[2020-06-22T00:36:45.758Z]             If `axis` is None, `obj` is applied to the flattened array.
[2020-06-22T00:36:45.758Z]
[2020-06-22T00:36:45.758Z]         Returns
[2020-06-22T00:36:45.758Z]         -------
[2020-06-22T00:36:45.758Z]         out : ndarray
[2020-06-22T00:36:45.758Z]             A copy of `arr` with the elements specified by `obj` removed. Note
[2020-06-22T00:36:45.758Z]             that `delete` does not occur in-place. If `axis` is None, `out` is
[2020-06-22T00:36:45.758Z]             a flattened array.
[2020-06-22T00:36:45.758Z]
[2020-06-22T00:36:45.758Z]         See Also
[2020-06-22T00:36:45.758Z]         --------
[2020-06-22T00:36:45.758Z]         insert : Insert elements into an array.
[2020-06-22T00:36:45.758Z]         append : Append elements at the end of an array.
[2020-06-22T00:36:45.758Z]
[2020-06-22T00:36:45.758Z]         Notes
[2020-06-22T00:36:45.758Z]         -----
[2020-06-22T00:36:45.758Z]         Often it is preferable to use a boolean mask. For example:
[2020-06-22T00:36:45.758Z]
[2020-06-22T00:36:45.758Z]         >>> arr = np.arange(12) + 1
[2020-06-22T00:36:45.758Z]         >>> mask = np.ones(len(arr), dtype=bool)
[2020-06-22T00:36:45.758Z]         >>> mask[[0,2,4]] = False
[2020-06-22T00:36:45.758Z]         >>> result = arr[mask,...]
[2020-06-22T00:36:45.758Z]
[2020-06-22T00:36:45.758Z]         Is equivalent to `np.delete(arr, [0,2,4], axis=0)`, but allows further
[2020-06-22T00:36:45.758Z]         use of `mask`.
[2020-06-22T00:36:45.758Z]
[2020-06-22T00:36:45.758Z]         Examples
[2020-06-22T00:36:45.758Z]         --------
[2020-06-22T00:36:45.758Z]         >>> arr = np.array([[1,2,3,4], [5,6,7,8], [9,10,11,12]])
[2020-06-22T00:36:45.758Z]         >>> arr
[2020-06-22T00:36:45.758Z]         array([[ 1,  2,  3,  4],
[2020-06-22T00:36:45.758Z]                [ 5,  6,  7,  8],
[2020-06-22T00:36:45.758Z]                [ 9, 10, 11, 12]])
[2020-06-22T00:36:45.758Z]         >>> np.delete(arr, 1, 0)
[2020-06-22T00:36:45.758Z]         array([[ 1,  2,  3,  4],
[2020-06-22T00:36:45.758Z]                [ 9, 10, 11, 12]])
[2020-06-22T00:36:45.758Z]
[2020-06-22T00:36:45.758Z]         >>> np.delete(arr, np.s_[::2], 1)
[2020-06-22T00:36:45.758Z]         array([[ 2,  4],
[2020-06-22T00:36:45.758Z]                [ 6,  8],
[2020-06-22T00:36:45.758Z]                [10, 12]])
[2020-06-22T00:36:45.758Z]         >>> np.delete(arr, [1,3,5], None)
[2020-06-22T00:36:45.758Z]         array([ 1,  3,  5,  7,  8,  9, 10, 11, 12])
[2020-06-22T00:36:45.758Z]
[2020-06-22T00:36:45.758Z]         """
[2020-06-22T00:36:45.758Z]         wrap = None
[2020-06-22T00:36:45.758Z]         if type(arr) is not ndarray:
[2020-06-22T00:36:45.758Z]             try:
[2020-06-22T00:36:45.758Z]                 wrap = arr.__array_wrap__
[2020-06-22T00:36:45.758Z]             except AttributeError:
[2020-06-22T00:36:45.758Z]                 pass
[2020-06-22T00:36:45.758Z]
[2020-06-22T00:36:45.758Z]         arr = asarray(arr)
[2020-06-22T00:36:45.758Z]         ndim = arr.ndim
[2020-06-22T00:36:45.758Z]         arrorder = 'F' if arr.flags.fnc else 'C'
[2020-06-22T00:36:45.758Z]         if axis is None:
[2020-06-22T00:36:45.758Z]             if ndim != 1:
[2020-06-22T00:36:45.758Z]                 arr = arr.ravel()
[2020-06-22T00:36:45.758Z]             # needed for np.matrix, which is still not 1d after being ravelled
[2020-06-22T00:36:45.758Z]             ndim = arr.ndim
[2020-06-22T00:36:45.758Z]             axis = ndim - 1
[2020-06-22T00:36:45.758Z]         else:
[2020-06-22T00:36:45.758Z]             axis = normalize_axis_index(axis, ndim)
[2020-06-22T00:36:45.758Z]
[2020-06-22T00:36:45.758Z]         slobj = [slice(None)]*ndim
[2020-06-22T00:36:45.758Z]         N = arr.shape[axis]
[2020-06-22T00:36:45.758Z]         newshape = list(arr.shape)
[2020-06-22T00:36:45.758Z]
[2020-06-22T00:36:45.758Z]         if isinstance(obj, slice):
[2020-06-22T00:36:45.758Z]             start, stop, step = obj.indices(N)
[2020-06-22T00:36:45.758Z]             xr = range(start, stop, step)
[2020-06-22T00:36:45.758Z]             numtodel = len(xr)
[2020-06-22T00:36:45.758Z]
[2020-06-22T00:36:45.758Z]             if numtodel <= 0:
[2020-06-22T00:36:45.758Z]                 if wrap:
[2020-06-22T00:36:45.758Z]                     return wrap(arr.copy(order=arrorder))
[2020-06-22T00:36:45.758Z]                 else:
[2020-06-22T00:36:45.758Z]                     return arr.copy(order=arrorder)
[2020-06-22T00:36:45.758Z]
[2020-06-22T00:36:45.758Z]             # Invert if step is negative:
[2020-06-22T00:36:45.758Z]             if step < 0:
[2020-06-22T00:36:45.758Z]                 step = -step
[2020-06-22T00:36:45.758Z]                 start = xr[-1]
[2020-06-22T00:36:45.758Z]                 stop = xr[0] + 1
[2020-06-22T00:36:45.758Z]
[2020-06-22T00:36:45.758Z]             newshape[axis] -= numtodel
[2020-06-22T00:36:45.758Z]             new = empty(newshape, arr.dtype, arrorder)
[2020-06-22T00:36:45.758Z]             # copy initial chunk
[2020-06-22T00:36:45.758Z]             if start == 0:
[2020-06-22T00:36:45.758Z]                 pass
[2020-06-22T00:36:45.758Z]             else:
[2020-06-22T00:36:45.758Z]                 slobj[axis] = slice(None, start)
[2020-06-22T00:36:45.758Z]                 new[tuple(slobj)] = arr[tuple(slobj)]
[2020-06-22T00:36:45.758Z]             # copy end chunk
[2020-06-22T00:36:45.758Z]             if stop == N:
[2020-06-22T00:36:45.758Z]                 pass
[2020-06-22T00:36:45.759Z]             else:
[2020-06-22T00:36:45.759Z]                 slobj[axis] = slice(stop-numtodel, None)
[2020-06-22T00:36:45.759Z]                 slobj2 = [slice(None)]*ndim
[2020-06-22T00:36:45.759Z]                 slobj2[axis] = slice(stop, None)
[2020-06-22T00:36:45.759Z]                 new[tuple(slobj)] = arr[tuple(slobj2)]
[2020-06-22T00:36:45.759Z]             # copy middle pieces
[2020-06-22T00:36:45.759Z]             if step == 1:
[2020-06-22T00:36:45.759Z]                 pass
[2020-06-22T00:36:45.759Z]             else:  # use array indexing.
[2020-06-22T00:36:45.759Z]                 keep = ones(stop-start, dtype=bool)
[2020-06-22T00:36:45.759Z]                 keep[:stop-start:step] = False
[2020-06-22T00:36:45.759Z]                 slobj[axis] = slice(start, stop-numtodel)
[2020-06-22T00:36:45.759Z]                 slobj2 = [slice(None)]*ndim
[2020-06-22T00:36:45.759Z]                 slobj2[axis] = slice(start, stop)
[2020-06-22T00:36:45.759Z]                 arr = arr[tuple(slobj2)]
[2020-06-22T00:36:45.759Z]                 slobj2[axis] = keep
[2020-06-22T00:36:45.759Z]                 new[tuple(slobj)] = arr[tuple(slobj2)]
[2020-06-22T00:36:45.759Z]             if wrap:
[2020-06-22T00:36:45.759Z]                 return wrap(new)
[2020-06-22T00:36:45.759Z]             else:
[2020-06-22T00:36:45.759Z]                 return new
[2020-06-22T00:36:45.759Z]
[2020-06-22T00:36:45.759Z]         if isinstance(obj, (int, integer)) and not isinstance(obj, bool):
[2020-06-22T00:36:45.759Z]             # optimization for a single value
[2020-06-22T00:36:45.759Z]             if (obj < -N or obj >= N):
[2020-06-22T00:36:45.759Z]                 raise IndexError(
[2020-06-22T00:36:45.759Z]                     "index %i is out of bounds for axis %i with "
[2020-06-22T00:36:45.759Z]                     "size %i" % (obj, axis, N))
[2020-06-22T00:36:45.759Z]             if (obj < 0):
[2020-06-22T00:36:45.759Z]                 obj += N
[2020-06-22T00:36:45.759Z]             newshape[axis] -= 1
[2020-06-22T00:36:45.759Z]             new = empty(newshape, arr.dtype, arrorder)
[2020-06-22T00:36:45.759Z]             slobj[axis] = slice(None, obj)
[2020-06-22T00:36:45.759Z]             new[tuple(slobj)] = arr[tuple(slobj)]
[2020-06-22T00:36:45.759Z]             slobj[axis] = slice(obj, None)
[2020-06-22T00:36:45.759Z]             slobj2 = [slice(None)]*ndim
[2020-06-22T00:36:45.759Z]             slobj2[axis] = slice(obj+1, None)
[2020-06-22T00:36:45.759Z]             new[tuple(slobj)] = arr[tuple(slobj2)]
[2020-06-22T00:36:45.759Z]         else:
[2020-06-22T00:36:45.759Z]             _obj = obj
[2020-06-22T00:36:45.759Z]             obj = np.asarray(obj)
[2020-06-22T00:36:45.759Z]             if obj.size == 0 and not isinstance(_obj, np.ndarray):
[2020-06-22T00:36:45.759Z]                 obj = obj.astype(intp)
[2020-06-22T00:36:45.759Z]
[2020-06-22T00:36:45.759Z]             if obj.dtype == bool:
[2020-06-22T00:36:45.759Z]                 if obj.shape != (N,):
[2020-06-22T00:36:45.759Z]                     raise ValueError('boolean array argument obj to delete '
[2020-06-22T00:36:45.759Z]                                      'must be one dimensional and match the axis '
[2020-06-22T00:36:45.759Z]                                      'length of {}'.format(N))
[2020-06-22T00:36:45.759Z]
[2020-06-22T00:36:45.759Z]                 # optimization, the other branch is slower
[2020-06-22T00:36:45.759Z]                 keep = ~obj
[2020-06-22T00:36:45.759Z]             else:
[2020-06-22T00:36:45.759Z]                 keep = ones(N, dtype=bool)
[2020-06-22T00:36:45.759Z] >               keep[obj,] = False
[2020-06-22T00:36:45.759Z] E               IndexError: arrays used as indices must be of integer (or boolean) type
[2020-06-22T00:36:45.759Z]
[2020-06-22T00:36:45.759Z] C:\Python37\lib\site-packages\numpy\lib\function_base.py:4406: IndexError
[2020-06-22T00:36:45.759Z]
[2020-06-22T00:36:45.759Z] During handling of the above exception, another exception occurred:
[2020-06-22T00:36:45.759Z]
[2020-06-22T00:36:45.759Z] args = (), kwargs = {}, test_count = 1, env_seed_str = None, i = 0
[2020-06-22T00:36:45.759Z] this_test_seed = 810322850, log_level = 10
[2020-06-22T00:36:45.759Z] post_test_state = ('MT19937', array([1938474601,<PHONE_NUMBER>,<PHONE_NUMBER>,   19860296,<PHONE_NUMBER>,
[2020-06-22T00:36:45.759Z]<PHONE_NUMBER>,  725127881, 21902135...,<PHONE_NUMBER>,  653485402,<PHONE_NUMBER>,
[2020-06-22T00:36:45.759Z]<PHONE_NUMBER>,  675050936,<PHONE_NUMBER>,<PHONE_NUMBER>], dtype=uint32), 1, 0, 0.0)
[2020-06-22T00:36:45.759Z] logger = <Logger common (DEBUG)>, test_count_msg = ''
[2020-06-22T00:36:45.759Z] test_msg = 'Setting test np/mx/python random seeds, use MXNET_TEST_SEED=810322850 to reproduce.'
[2020-06-22T00:36:45.759Z]
[2020-06-22T00:36:45.759Z]     @functools.wraps(orig_test)
[2020-06-22T00:36:45.759Z]     def test_new(*args, **kwargs):
[2020-06-22T00:36:45.759Z]         test_count = int(os.getenv('MXNET_TEST_COUNT', '1'))
[2020-06-22T00:36:45.759Z]         env_seed_str = os.getenv('MXNET_TEST_SEED')
[2020-06-22T00:36:45.759Z]         for i in range(test_count):
[2020-06-22T00:36:45.759Z]             if seed is not None:
[2020-06-22T00:36:45.759Z]                 this_test_seed = seed
[2020-06-22T00:36:45.759Z]                 log_level = logging.INFO
[2020-06-22T00:36:45.759Z]             elif env_seed_str is not None:
[2020-06-22T00:36:45.759Z]                 this_test_seed = int(env_seed_str)
[2020-06-22T00:36:45.759Z]                 log_level = logging.INFO
[2020-06-22T00:36:45.759Z]             else:
[2020-06-22T00:36:45.759Z]                 this_test_seed = np.random.randint(0, np.iinfo(np.int32).max)
[2020-06-22T00:36:45.759Z]                 log_level = logging.DEBUG
[2020-06-22T00:36:45.759Z]             post_test_state = np.random.get_state()
[2020-06-22T00:36:45.759Z]             np.random.seed(this_test_seed)
[2020-06-22T00:36:45.759Z]             mx.random.seed(this_test_seed)
[2020-06-22T00:36:45.759Z]             random.seed(this_test_seed)
[2020-06-22T00:36:45.759Z]             logger = default_logger()
[2020-06-22T00:36:45.759Z]             # 'pytest --logging-level=DEBUG' shows this msg even with an ensuing core dump.
[2020-06-22T00:36:45.759Z]             test_count_msg = '{} of {}: '.format(i+1,test_count) if test_count > 1 else ''
[2020-06-22T00:36:45.759Z]             test_msg = ('{}Setting test np/mx/python random seeds, use MXNET_TEST_SEED={}'
[2020-06-22T00:36:45.759Z]                         ' to reproduce.').format(test_count_msg, this_test_seed)
[2020-06-22T00:36:45.759Z]             logger.log(log_level, test_msg)
[2020-06-22T00:36:45.759Z]             try:
[2020-06-22T00:36:45.759Z] >               orig_test(*args, **kwargs)
[2020-06-22T00:36:45.759Z]
[2020-06-22T00:36:45.759Z] tests\python\unittest\common.py:223:
[2020-06-22T00:36:45.759Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[2020-06-22T00:36:45.759Z] windows_package\python\mxnet\util.py:298: in _with_np_shape
[2020-06-22T00:36:45.759Z]     return func(*args, **kwargs)
[2020-06-22T00:36:45.759Z] windows_package\python\mxnet\util.py:482: in _with_np_array
[2020-06-22T00:36:45.759Z]     return func(*args, **kwargs)
[2020-06-22T00:36:45.759Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[2020-06-22T00:36:45.759Z]
[2020-06-22T00:36:45.759Z] args = (), kwargs = {}
[2020-06-22T00:36:45.759Z]
[2020-06-22T00:36:45.759Z]     @functools.wraps(func)
[2020-06-22T00:36:45.759Z]     def _run_with_array_func_proto(*args, **kwargs):
[2020-06-22T00:36:45.759Z]         if cur_np_ver >= np_1_17_ver:
[2020-06-22T00:36:45.759Z]             try:
[2020-06-22T00:36:45.759Z]                 func(*args, **kwargs)
[2020-06-22T00:36:45.759Z]             except Exception as e:
[2020-06-22T00:36:45.759Z]                 raise RuntimeError('Running function {} with NumPy array function protocol failed'
[2020-06-22T00:36:45.759Z]                                    ' with exception {}'
[2020-06-22T00:36:45.759Z] >                                  .format(func.__name__, str(e)))
[2020-06-22T00:36:45.759Z] E               RuntimeError: Running function test_np_array_function_protocol with NumPy array function protocol failed with exception arrays used as indices must be of integer (or boolean) type

Temporarily disabled this test in #18598

The failure is observed in v1.7.x branch as well, check log http://jenkins.mxnet-ci.amazon-ml.com/blue/rest/organizations/jenkins/pipelines/mxnet-validation/pipelines/centos-cpu/branches/PR-18478/runs/2/nodes/110/steps/159/log/?start=0 from PR https://github.com/apache/incubator-mxnet/pull/18478.

@eric-haibin-lin actually we should have put constraint on the numpy version first before we can verify that the integration with the new version is functional, especially since numpy doesn't follow semantic versioning. Could you post a change to reflect this and to re-enable the test?

@ciyongch should this be fixed for 1.7 release? I'd expect that users would like to use latest MXNet version with latest numpy.

it's very common to pin package

@ciyongch that's not true. In fact it's easy to pip install --upgrade numpy on a system and get into a broken state. Users may also install other packages requiring a new version of numpy.

BTW, we're planing to cut the rc0 in the following days and will consider whether to include the fix after rc0, what do you think?

If we consider the numpy API experimental in MXNet 1.7, it's ok but certainly not ideal.

Yes, it could be a broken issue unless there's a constraint for the numpy version for MXNet's installation. I'm not sure whether we can state numpy API as an experimental feature or not since it's already introduced in 1.6.0 release.
If this is a mandatory fix for 1.7.0, may I know if there's anyone can help on this? @szha @leezu @sandeep-krishnamurthy
Thanks.

Is it feasible to make mxnet compatible with all versions above 1.17? If not then we definitely would need to make a choice on the numpy version to support.

Only deprecated APIs are removed. There's no need to support the deprecated APIs in older releases. So generally this should be possible. What do you think @yzhliu

Hi @leezu @szha @yzhliu , is there any updates for this topic, we're pending on this to drop the rc tag for 1.7 release. Based on the previous discussion, seems there're two options to go,

Make MXNet compatible with all numpy versions or versions above 1.17.
Add a constraint to numpy version that MXNet supports.

Or is it necessary to raise it on dev@ ? Thanks!

As discussed here, we'll mark numpy operator as experimental in v1.7 release and probably will move forward by taking this as an known issue (some cases are broken with latest numpy version 1.19.0). Please let me know if you have any further concerns or suggestions!


Set upper bound for angstrom

0.14.0 introduced a breaking change

I'd rather start using the new version. See #118


Add files via upload

file forked

not sure if done correctly

@CyberOnechow WindowsPCHealthCheckSetup.msi I am sorry, I have to close the request without merging, because you made a mistake in a request and did not specify any filepath. Please ensure that you are familiar with this repository rules located at the bottom of this page and make a new correct request. Thank you for your understanding!

@CyberOnechow There are several files in the request, along with file.zip. It must not be there. Please remove file.zip and make pull request again.  I have to close this request without merging. Thank you for your understanding.


app.css missing

Module not found: Error: Can't resolve './app.css' in 'E:\Temp_delete\angular2webpack2-starter\src\app'
@ ./src/app/app.ts 40:21-41
@ ./src/app/app.module.ts
@ ./src/main.browser.ts
@ multi main
Renamed app.scss to app.css and works fine!

If you run 'npm run sass' it will compile it and most of the scripts should compile it. When I get back in front my computer, I'll update it to do it on install and make sure all the scripts compile it when ran.

@jennyraj I updated the postinstall script to run npm run scss postinstall and I updated the readme.

thank you !!


Fix standard eslint config link

Hi, just a minor tweak. The JavaScript Standard Style doc was moved so I updated the link accordingly.
Thanks for your work.

Thanks!


add an example of how to reference a pipeline variable vs repeating the same expression

I'd like to have an example added that shows how to reference a pipeline level variable expression
IE
in pipeline top something like it
variables:

name: isMaster
value: eq(variables['Build.SourceBranch'], 'refs/heads/master')

then for a stage condition uses that isMaster:
condition: and(variables.isMaster, succeeded())

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 3f151218-9a11-0078-e038-f96198a76143
Version Independent ID: 09c4d032-62f3-d97c-79d7-6fbfd89910e9
Content: Conditions - Azure Pipelines
Content Source: docs/pipelines/process/conditions.md
Product: devops
Technology: devops-cicd
GitHub Login: @juliakm
Microsoft Alias: jukullam

@WilliamAntonRohm how do i find out the changeset related to this

@drdamour -- the updated article is live.  Here is the new YAML code example:
variables:
  isMain: $[eq(variables['Build.SourceBranch'], 'refs/heads/master')]
steps:
  - script: echo Hello!
    condition: and(variables.isMain, succeeded())

@WilliamAntonRohm  yeah i saw that,  BUT that's not what was asked for.  that is a JOB condition and the request was a STAGE condition. they seem to work differently.  can we re-open?
also just wondering where we can see the commit changeset for this ticket...somehow pr merger knew to close this...but it doesn't link the commits and i wanted to see if there was a way to find em

@drdamour -- thank you for clarifying.  The PRs are created in a private internal document repo, then merged into master (this repo) which is periodically published live.
@juliakm -- please look into this issue.

Here's an updated example that will be live shortly.
variables:
  isMain: $[eq(variables['Build.SourceBranch'], 'refs/heads/master')]

stages:
- stage: A
  jobs:
  - job: A1
    steps:
      - script: echo Hello Stage A!

- stage: B
  condition: eq(variables.isMain, true)
  jobs:
  - job: B1
    steps:
      - script: echo Hello Stage B!
      - script: echo $(isMain)


[Bug] Could not resolve reference to 'System.Drawing.Color'

With the latest version(s), it throws a build error but only in Release mode in Android. In Debug it works fine.
Please check all of the platforms you are having the issue on (if platform is not listed, it is not supported)

[ ] iOS
[X] Android
[ ] UWP
[ ] .NET Standard

Version of Library
7.0.35
Xamarin.Forms 4.2 (latest)
VS 2019 and 2017
Last working version
7.0.4
Version of OS(s) listed above with issue
Android 9.1
Steps to Reproduce

I just have the package installed, and in the code i have references to the Loading() and Toast()
Build in RELEASE mode

Expected Behavior
Solution builds
Actual Behavior

Severity	Code	Description	Project	File	Line	Suppression State
Error		The "LinkAssemblies" task failed unexpectedly.
Java.Interop.Tools.Diagnostics.XamarinAndroidException: error XA2006: Could not resolve reference to 'System.Drawing.Color' (defined in assembly 'Acr.UserDialogs, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null') with scope 'System.Drawing.Common, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'. When the scope is different from the defining assembly, it usually means that the type is forwarded. ---> Mono.Cecil.ResolutionException: Failed to resolve System.Drawing.Color
at Mono.Linker.Steps.MarkStep.HandleUnresolvedType(TypeReference reference)
at Mono.Linker.Steps.MarkStep.MarkType(TypeReference reference)
at MonoDroid.Tuner.MonoDroidMarkStep.MarkType(TypeReference reference)
at Mono.Linker.Steps.MarkStep.MarkGenericArguments(IGenericInstance instance)
at Mono.Linker.Steps.MarkStep.GetOriginalType(TypeReference type)
at Mono.Linker.Steps.MarkStep.MarkType(TypeReference reference)
at MonoDroid.Tuner.MonoDroidMarkStep.MarkType(TypeReference reference)
at Mono.Linker.Steps.MarkStep.MarkField(FieldReference reference)
at Mono.Linker.Steps.MarkStep.InitializeFields(TypeDefinition type)
at Mono.Linker.Steps.MarkStep.InitializeType(TypeDefinition type)
at Mono.Linker.Steps.MarkStep.InitializeAssembly(AssemblyDefinition assembly)
at Mono.Linker.Steps.MarkStep.Initialize()
at MonoDroid.Tuner.MonoDroidMarkStep.Process(LinkContext context)
at Mono.Linker.Pipeline.Process(LinkContext context)
at MonoDroid.Tuner.Linker.Process(LinkerOptions options, ILogger logger, LinkContext& context)
at Xamarin.Android.Tasks.LinkAssemblies.Execute(DirectoryAssemblyResolver res)
--- End of inner exception stack trace ---
at Java.Interop.Tools.Diagnostics.Diagnostic.Error(Int32 code, Exception innerException, String message, Object[] args)
at Xamarin.Android.Tasks.LinkAssemblies.Execute(DirectoryAssemblyResolver res)
at Xamarin.Android.Tasks.LinkAssemblies.Execute()
at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
at Microsoft.Build.BackEnd.TaskBuilder.d__26.MoveNext()	LiveDispatch.Mobile.Android

https://github.com/aritchie/userdialogs/issues/654 This was your issue and it has been pinned with who to contact.  DO NOT FILE THIS AGAIN


Look into Memory problem when analyzing multimillion line project

"the memory requirements of the plugin on large analyses are impractical and the user had given up on the plugin and instead set up Jenkins to run codesonar manually.  The user suspected a memory leak, however I suspect it may simply be an issue where it attempts to keep too much data in memory at once."

I contacted Grammatech related big project testing approaches.

Proposed tool from @kryptag http://www.oracle.com/technetwork/java/javaseproducts/mission-control/java-mission-control-1998576.html

We're running such tests with test setup available after closing #39 and with analysis from #43 - will close this old issue.


Add "--disallow-untyped-defs" option to mypy

This will enforce that all functions have type defs. As a prerequisite, someone will need to go through the codebase and add types to everything. We should definitely do this sooner rather than later so it's less burdensome to go through and add types and so we get the typechecking benefits sooner, before the codebase becomes large.

@jenniebrown FYI


Allow a maximum of 3 concurrent uv_write operations per connection

Keep logic to prevent unnecessary calls to KestrelThread.Post
This partially reverts commit 4809964.

#691

:ship:


Default Security List usage should be avoided -> need a lockdown feature

The problem with default Security List and Default Route Table :

they are created by default with every VCN and cannot be deleted
they are assigned to your subnet if you don’t specify one at creation creation time

That may lead to unexpected behavior and should be avoided past the "quick demo vcn" step.
This is a "sane defaults" design proposition for this module:

empty Default Security List,
guide user to create a dedicated SL/RT per subnet

That way, if someone create a subnet without specifying SL, it is just unreachable 🤷♂
Ideally, this should be a feature like "enable/disable Default Security List Lockdown" so the module user still keeps control and flexibility over how his vcn is configured.

:tada: This issue has been resolved in version v2.1.0 :tada:
The release is available here


minor changes

added loopComplete=false; so the menu can be reused.
uncommented a line to put Invoked output below the menu.
rest ist untabify and remove unused usings.

looks good thanks ..
next time please use tabs ... it is c# after all :)


customer portal UI 4

http://order-organic.com/category/subcategorys/Mzc=
Customer portal--->image is not displayed

not design issue (image path is not correrct or no image)


[WIP] Use set for worker addresses

Explicitly using the set() for worker addresses.

Unnecessary anymore.


Only set pragueTime if electra_fork_epoch is set

As the latest fork is Deneb, it shouldn't be assumed that test chains are already configuring Electra.
Currently the code will break if electra_fork_epoch isn't set, effectively forcing the EL config to set pragueTime.

This PR only considers geth's genesis.json, but doesn't do the same change for other ELs.
Could you please either update your PR so that it would consider for all ELs (chainspec.json and besu.json) as well as consider what changes might need to happen on the CL side?
Alternatively, if you only care about genesis.json, you are free to maintain a fork of this repo.

Okay I've updated the other EL config generators. On the CL side, I don't see anything that should be affected, which is confirmed by running this on a full testnet.
Side note: the code assumes the Verkle EIP 6800 will be on the Prague/Electra fork, has that been confirmed?

Okay I've updated the other EL config generators. On the CL side, I don't see anything that should be affected, which is confirmed by running this on a full testnet.
Side note: the code seems to assume that the Verkle trie update EIP 6800 will be on the Prague/Electra fork, has that been confirmed?

It hasn't, we're just using it as a placeholder for verkle tests and will re-evaluate once things are final


Stabilize master

hybrid option added to nextjs app & runs to completion - reading from both docs & web :)


If you delete it, I won't bother you anymore.  If you delete it, I won't bother you anymore. Actually, that's what I expect from you


ARROW-12166: [C++][Gandiva] Implements CONVERT_TO(value, type) function

9861

 Quality gate: Passed
34 Issues

 34 Code issues

Details


Prune Alert

Adding two exceptions on when we register an alert in case pruning failed. Judging by the integrity node this should be the lost of it.

actually I'm going to add this to https://github.com/SiaFoundation/renterd/pull/805


Fix that auto-Workshop script

We make 50 commits and then push it to workshop after a month. Then people start reporting that X is broken. Then we fix it and don't publish it to workshop again.
That auto-publishing script/webhook seriously needs to be fixed so that we don't have to depend on Faron.

Problem is, he keep tight control over the means of production.
The way I understood it, no one but him can publish wiremod.
I might be wrong though, don't quote me on it.

They've been working on it now


Added support for Java & C++

Added the Code highlight of Java and C++ programming lang.

@vignesh-gupta This looks fine to me, but would you mind fixing the lint warnings first, by running npm run ci-checks


fix: use in memory pubsub when no MQTT_HOST set

Description

I noticed npm run start was using nodemon. The showcase container image was running the app using nodemon (not very good)
I changed npm run start to start with node
I added npm run dev to start with MQTT_HOST=localhost nodemon
The server no longer defaults to localhost when no MQTT_HOST is set. Instead it will default to an in memory pubsub
While running locally, you can start with npm start or node src/index.js and by default you will get in memory pubsub
While running locally, you can start with npm run dev or MQTT_HOST=localhost node src/index.js to get MQTT Pubsub

Checklist

[ ] npm test passes
[ ] npm run build works
[ ] tests are included
[ ] documentation is changed or added

Thank you for the very fast fix! I have build container and run it separately without docker compose. Everything seems to be working. Passing localhost to env variable connected to MQTT broker

Nice one @darahayes - I forgot to mention the nodemon yesterday, I had noticed it too!
🎉


Human learning loop / compilation times

Heya!
@frankmcsherry I've been noodling on timely-dataflow and differential-dataflow for months, and this week I finally got a chance to read the mdbook and go through the examples. Very fun, mind-expanding stuff! The runtime speed numbers are impressive, and I like how much TLC is put into these libraries. Thank you for all of your hard work bringing these projects to the world.
As I learn the idioms of TD/DD, I've noticed that I want to tweak the provided examples (maybe by adding a function to a pipeline, or changing the input generator to use stdin, and so on). This kind of fiddling is part of my normal hands-on tinkering-to-learn process.
Unfortunately, I haven't been able to get compile times low enough to make this learning process pleasant. I've found that compiling one of the first examples from the mdbook, with a hot incremental build cache (Rust 1.33.0), and debug mode, still takes 10+ seconds.
I looked through the verbose cargo logs, but nothing stood out to me as being a problem I could fix to speed up compilation.
I know these libraries are often described as working in "mysterious ways" -- are there techniques we could employ to speed up the compile-test-edit cycle, so that fans like me can improve our intuition of these libraries?
Thanks!

I agree! I'm a bit tight on time right at the moment, but I'll try and explain more and sketch some alternatives later on today. But, you are right to be frustrated! :D

My experience has been that

timely and differential dataflow builds take a while, and
the vast majority of the time is spent in LLVM, compiling the code that Rust hands it.

My understanding is that Rust provides a fairly massive volume of code, at one point surprising some of the core devs: we had one example where a fraction of a gigabyte of intermediate representation was handed to LLVM, with the instructions "optimize this". I have a differential dataflow program that takes 20 minutes to build in release mode.
Release and debug builds of the simple example, whose source is literally
extern crate timely;

use timely::dataflow::operators::*;

fn main() {
    timely::example(|scope| {
        (0..10).to_stream(scope)
               .inspect(|x| println!("seen: {:?}", x));
    });
}

takes
echidnatron% cargo build --release --example simple
   Compiling timely v0.8.0 (/Users/mcsherry/Projects/timely-dataflow)
    Finished release [optimized + debuginfo] target(s) in 26.44s
echidnatron% cargo build --example simple
   Compiling timely v0.8.0 (/Users/mcsherry/Projects/timely-dataflow)
    Finished dev [unoptimized + debuginfo] target(s) in 6.39s
echidnatron%

which I agree is deeply objectionable.
I don't have a lot of visibility into why Rust takes so long to compile things. There is the cargo bloat subcommand which attempts to account for the space in binaries, which may be an indicator of what took so long to compile. Unfortunately, its output appears to be relatively unhelpful:
echidnatron% cargo bloat --example simple
Compiling ...

There is also the cargo llvm-lines subcommand, which when pointed at a new src/bin/simple.rs produces
echidnatron% cargo llvm-lines --bin simple | head -n20
   Compiling timely v0.8.0 (/Users/mcsherry/Projects/timely-dataflow)
    Finished dev [unoptimized + debuginfo] target(s) in 7.35s
  17942   25  <core::slice::Iter<'a, T> as core::iter::iterator::Iterator>::try_fold
  13511  641  core::ptr::real_drop_in_place
  12250   35  <alloc::raw_vec::RawVec<T, A>>::reserve_internal
   6701   72  <core::option::Option<T>>::map
   6239   55  <core::result::Result<T, E>>::map
   5272   46  <core::result::Result<T, E>>::map_err
   4028   43  <core::result::Result<T, E>>::unwrap_or_else
   3755   16  std::collections::hash::map::search_hashed_nonempty
   3534    8  alloc::slice::merge_sort
   3460   28  core::iter::iterator::Iterator::try_fold
   3146   26  <alloc::raw_vec::RawVec<T, A>>::allocate_in
   2898    4  <core::slice::IterMut<'a, T> as core::iter::iterator::Iterator>::try_fold
   2625   35  <alloc::raw_vec::RawVec<T, A>>::amortized_new_size
   2520    8  alloc::slice::insert_head
   2508   64  <alloc::vec::Vec<T> as core::ops::index::IndexMut<I>>::index_mut
   2479    7  <std::collections::hash::map::HashMap<K, V, S>>::try_resize
   2403   55  core::ptr::read
   2172    8  alloc::slice::merge
   2141   26  <alloc::vec::Vec<T>>::push
   2062   34  core::ptr::swap_nonoverlapping_one
echidnatron%

which appears to be telling us that there is a massive amount of random Rust cruft. If you add up all the lines for simple.rs, you get 295,988 lines.
My sense is that what this means is that Rust's "zero cost abstractions" are very much not zero cost to compile, and the encouragement to use them is perhaps an error for compilation-minded users.
I'm about to head out for some vacation, but I'll try and look into things and see what I can learn. For example, it may make sense to #ifdef out much of the networking code for "learning", because even though it is dead code for timely::example, Rust may force LLVM to rediscover this on each build. I'll see what I can do about pointing serious Rust folks at this problem, as it worries me too.

One thing that helped a little (not greatly, but ...) is to replace timely example code with
extern crate timely;

use timely::dataflow::operators::*;

fn main() {

    let alloc = timely::communication::allocator::thread::Thread::new();
    let mut worker = timely::worker::Worker::new(alloc);

    worker.dataflow::<u64,_,_>(|scope|  {
        (0..10).to_stream(scope)
               .inspect(|x| println!("seen: {:?}", x));
    });
}

which removes the timely::example() and timely::execute() calls, making much of the communication crate dead code. Instead, it uses only the single-threaded allocator and directly employs the main thread as the compute thread.
It still has lots of llvm-lines, and I'll try looking in to what could possibly be the source of all of them.

I just pushed https://github.com/TimelyDataflow/timely-dataflow/commit/d65e96d4bc85f1b48ed878641314c54691d76d24 which adds the timely::execute_directly(..) method. It bypasses much of the communication construction logic, taking no arguments from the command line and just using the main thread.
This also improved cargo test by about 20s (from 220s to 200s) as most of the doctest examples use timely::example() which was ported over to use this too.

@frankmcsherry I just verified that timely::execute_directly shaves multiple seconds off of my debug compilation loop -- thanks! Do you know of other avenues for improving these numbers further? From your explanations above, it sounds like this problem might fundamentally be a rustc issue.

Great to hear! I'm not immediately aware of other issues, but I'm going to try exploring a bit to find out. Timely dataflow aggressively monomorphises, meaning it creates new types whenever it can tell a difference between things, but perhaps it should do less of that and more "boxing" of types behind abstract trait interfaces.
I'll explore a bit. I've also asked some Rust core dev folks, who seem receptive to thinking about it (esp if folks show up with specific data, so I'm looking into that first).


Required Property Deserialization Error

I'm getting an error on a Required Property of a class that it wasn't present in the json though it is. Below is unit test code to reproduce.
public void FooTest()
{
    var foo1 = new Foo(new[] { "A", "B", "C" });
    var json = JsonConvert.SerializeObject(foo1);
    var foo2 = JsonConvert.DeserializeObject<Foo>(json); // Throws here.
    Assert.AreEqual(foo1.Bars.Count, foo2.Bars.Count);
    Assert.AreEqual(foo1.Bars[0], foo2.Bars[0]);
    Assert.AreEqual(foo1.Bars[1], foo2.Bars[1]);
    Assert.AreEqual(foo1.Bars[2], foo2.Bars[2]);
}

public class Foo
{
    [JsonProperty(Required = Required.Always)]
    public List<string> Bars { get; private set; }

    public Foo(IEnumerable<string> bars)
    {
        Bars = new List<string>(bars);
    }
}

It appears the issue is in the CreateObjectUsingCreatorWithParameters method in the JsonSerializerInternalReader class. I was able to get it to work if I changed
// map from creator property to normal property
JsonProperty presenceProperty = propertiesPresence.Keys.FirstOrDefault(p => p.PropertyName == property.PropertyName);

to
// map from creator property to normal property
JsonProperty presenceProperty = propertiesPresence.Keys.FirstOrDefault(p => string.Equals(p.PropertyName, property.PropertyName, StringComparison.OrdinalIgnoreCase));

Fixed https://github.com/JamesNK/Newtonsoft.Json/commit/6173ca1574de47075630f0ce52314778ab60e5bb

Thank you!


Wrap codec exceptions with additional context

These kind of errors are difficult to decypher:
chairman: DeserialiseFailure 4 "expected word"

It would be good to log protocol id, state singleton (which would give us which message we expect), peer id could also be useufl.
Related issue #778

The additional context is already provided by Mux, it logs all miniprotocols that fails due to exception with involved peers, specific miniprotocol and if it was the initiator or responder side.


Values not showing in namespace total cluster CPU & RAM usage

Describe the bug
Values not showing in namespace total cluster CPU & RAM usage  (Kubernetes/Views/Namespaces)

How to reproduce?
No response
Expected behavior
No response
Additional context
No response

Hi @noor-muradi,
Can you check if you have any results for kube_pod_container_resource_requests and machine_cpu_cores in Grafana explore?
Did you check if you have a relabeling configuration that drop one or the other?
If not, could you share more context on your setup, version of k8s and deployments related to this issue...

@dotdc
I have not configured any custom relabel, I have deployed the Prometheus & Grafana cluster using the official helm chart, my K8s version is "1.26.6" . Below is the values.yaml, other dashboards are showing values correctly as below screenshot.
### Grafana Section ###

grafana:
  enabled: true
  namespaceOverride: ""

  grafana.ini:
    smtp:
      enabled: true
      host: smtp.gmail.com:465
      user: xxx
      password: xxx
      from_address: xxx
      from_name: xxx
    server:
      root_url: https://xxx
    wall: true

  persistence:
    enabled: true
    existingClaim: "grafana-pvc"
    accessModes:
      - ReadWriteOnce

  initChownData:
   enabled: false

  resources:
    requests:
      memory: 256Mi
      cpu: 128m
    limits:
      memory: 1Gi
      cpu: 840m

  dashboardProviders:
    dashboardproviders.yaml:
      apiVersion: 1
      providers:
      - name: 'grafana-dashboards-kubernetes'
        orgId: 1
        folder: 'Kubernetes'
        type: file
        disableDeletion: true
        editable: true
        options:
          path: /var/lib/grafana/dashboards/grafana-dashboards-kubernetes
  dashboards:
    grafana-dashboards-kubernetes:
      k8s-system-api-server:
        url: https://raw.githubusercontent.com/dotdc/grafana-dashboards-kubernetes/master/dashboards/k8s-system-api-server.json
        token: ''
      k8s-system-coredns:
        url: https://raw.githubusercontent.com/dotdc/grafana-dashboards-kubernetes/master/dashboards/k8s-system-coredns.json
        token: ''
      k8s-views-global:
        url: https://raw.githubusercontent.com/dotdc/grafana-dashboards-kubernetes/master/dashboards/k8s-views-global.json
        token: ''
      k8s-views-namespaces:
        url: https://raw.githubusercontent.com/dotdc/grafana-dashboards-kubernetes/master/dashboards/k8s-views-namespaces.json
        token: ''
      k8s-views-nodes:
        url: https://raw.githubusercontent.com/dotdc/grafana-dashboards-kubernetes/master/dashboards/k8s-views-nodes.json
        token: ''
      k8s-views-pods:
        url: https://raw.githubusercontent.com/dotdc/grafana-dashboards-kubernetes/master/dashboards/k8s-views-pods.json
        token: ''

  #default passowrd
  adminPassword: prom-operator

  #disable default dashboard
  sidecar:
    dashboards:
      enabled: false
    datasources:
      enabled: true
      defaultDatasourceEnabled: true
      isDefaultDatasource: true
      uid: prometheus

defaultRules:
 create: false

#### Prometheus Section ###

prometheus:
  enabled: true

  prometheusSpec:
    image:
      registry: quay.io
      repository: prometheus/prometheus
      tag: v2.45.0
      sha: ""
    ## How long to retain metrics
    retention: "10d"
    ## Maximum size of metrics
    retentionSize: "30GB"
    storageSpec:
      volumeClaimTemplate:
        metadata:
          name: prom-pvc
        spec:
         #storageClassName: nfs-retain
          accessModes: ["ReadWriteOnce"]
          volumeName: prom-pv
          resources:
            requests:
              storage: 5Gi
          selector:
            matchLabels:
              app: prometheus

    resources:
      requests:
        memory: 512Mi
        cpu: 128m
      limits:
        memory: 2Gi
        cpu: 840m

    additionalScrapeConfigs:

      - job_name: 'pfsense'
        static_configs:
        - targets: ['<IP_ADDRESS>:9100']

alertmanager:
  ## Deploy alertmanager
  enabled: false

@noor-muradi
On the broken panel, could you try relacing image="" with image!="" and see if it works better?

@dotdc ,
Thanks alot, I have changed image="" with image!="" and it resolved the issue.

That's great, will make a patch to fix it!

@noor-muradi
https://github.com/dotdc/grafana-dashboards-kubernetes/commit/dc8c2f86cd537be8383e37c0be51b4ae71c9bffe should fix it!

@noor-muradi I did notice you have the following config:
defaultRules:
 create: false

Is there any reason for this? I imagine this could be why the time series wasn't in the expected format?

@dotdc
Thanks for your effort, yes it has been resolved, but some of the panels like "Persistent Volumes - Capacity and usage in %" is not showing data.

$ k get pvc -n monitoring
NAME                                                          STATUS   VOLUME       CAPACITY   ACCESS MODES   STORAGECLASS   AGE
grafana-pvc                                                   Bound    grafana-pv   1Gi        RWO                           43d
prom-pvc-prometheus-monitoring-kube-prometheus-prometheus-0   Bound    prom-pv      5Gi        RWO                           43d

@noor-muradi
On your screenshot, the dashboard is set to the istio-system namespace, and you're apparently looking for PVCs in the monitoring namespace. I assume you just need to select the correct namespace to fix the issue.
Keep in mind that No data can be a good thing, or at least not a problem in several dashboards.


[python-package] Pandas memory overhead leads to OOM

Description
I'm not sure if this is a bug or expected behaviour or place to improve, but I've noticed weird thing, namely when I provide data with pandas dataframe directly:
model = lgb.LGBMRegressor(
    n_estimators=5_000,
    learning_rate=0.01,
    max_depth=5,
    num_leaves=2**5-1,
    colsample_bytree=0.1,
    data_sample_strategy='GOSS',
    random_state=0,
)
model.fit(
    df[f_all], #provide pandas df directly
    df["target"]
)

it crashes notebook with OOM, but when I provide it converted to numpy with .values then there is also memory peak usage but smaller so it doesn't lead to OOM:
model = lgb.LGBMRegressor(
    n_estimators=5_000,
    learning_rate=0.01,
    max_depth=5,
    num_leaves=2**5-1,
    colsample_bytree=0.1,
    data_sample_strategy='GOSS',
    random_state=0,
)
model.fit(
    df[f_all].values, #provide pandas dataframe converted to numpy
    df["target"]
)

In both cases memory surges at initial stage of fitting, even before log [LightGBM] [Info] Start training from score ...  appears.
One peculiar thing about my data is that all columns are of type int8.
I've also looked at https://lightgbm.readthedocs.io/en/latest/FAQ.html#when-running-lightgbm-on-a-large-dataset-my-computer-runs-out-of-ram but it doesn't help.
Reproducible example
Here is google colab notebook with data:
https://colab.research.google.com/drive/1W4ohwLPP76rpvxRXJlziQ9NxZVtbrfbp?usp=sharing
Environment info

google colab
lightgbm 4.1.0

Hey @mglowacki100, thanks for using LightGBM. The core library uses floats (either float32 or float64), so if you provide ints they need to be converted to float first, which creates a copy and also increases memory usage (from int8 to float32 you'd see a 4x increase). You can try casting your DataFrame to a floating type first and in that case no copies will be made when building the LightGBM dataset (so no memory peak). Here's an example:
# cast all of your columns to float32
df = df.astype('float32')
model.fit(
    df[f_all],
    df["target"]
)

Please let us know if this helps.

Thanks @jmoralez for looking into this. Your explanation makes perfect sense, but I've checked df = df.astype('float32')  and it doesn't fix problem with pandas (still OOM), worse after this type casting numpy workaround also doesn't work (OOM).
Just from curiousity, lightGBM introduced 'histogram binning', how it works with feature that has only 0,1,2,3,4 values, it is also converted to float and "re-binned" in some sense ?

Is your target column in between the dataframe? We have a test https://github.com/microsoft/LightGBM/blob/252828fd86627d7405021c3377534d6a8239dd69/tests/python_package_test/test_basic.py#L810
to check that the df is not copied when it's of a single float dtype, but that's only for the data and I'm guessing it has to be contiguous. So maybe you can try instead: X_df = df[f_all].astype('float32') instead and then provide that to fit.

@jmoralez I've tried in this way:
model.fit(
    df[f_all].astype('float32'),
    df["target"]
)

and now it works without OOM, but peak memory usage is higher than for
model.fit(
    df[f_all].values,
    df["target"]
)

Here is notebook with sample data: https://colab.research.google.com/drive/1W4ohwLPP76rpvxRXJlziQ9NxZVtbrfbp?usp=sharing


added global.json to enforce an older .net sdk version

At the moment, looks like a lot of people are already on preview3 or preview4 which causes a failure when loading OmniSharp in VS.

this is just a quick fix until https://github.com/OmniSharp/omnisharp-roslyn/issues/695


Deployment

Deployment
User Story
As a boot camp student,
I want the prework notes to be structured on a webpage,
So that I can easily find and read the information
Acceptance Criteria
GIVEN a Prework Study Guide website,
WHEN I visit the website using the URL,
THEN I can access my website from any browser

Website Deployed 4/14/2023 URL:  https://meanbean87.github.io/prework-study-guide/


"Enable Instancing" in StandardFast doesn't enable GPU Instancing

Hello all-
I'm building a HoloLens app, and GPU instancing would improve performance significantly in some cases. The issue is that checking "Enable Instancing" in a material using HoloToolkit/StandardFast doesn't enable instancing, whereas checking the same box for Unity's Standard does. I really prefer to use HoloToolkit/FastConfigurable, but "Enable Instancing" isn't exposed (enhancement request, please!).
To reproduce, create a new project set up with HoloToolkit. Create 2 materials, one with Standard and one with HoloToolkit/StandardFast, both with "Enable Instancing" checked. Add a GameObject with this script:
public Material Material;

void Start () {

	var gameObj = new GameObject("TRIANGLE");
	gameObj.transform.position = new Vector3(-1, 0, 2);

	var mesh = new Mesh();
	mesh.vertices = new[] { new Vector3(0, 0, 0), new Vector3(1, 0, 0), new Vector3(0, 1, 0) };
	mesh.triangles = new[] { 2, 1, 0 };
	mesh.RecalculateNormals();

	gameObj.AddComponent<MeshFilter>().sharedMesh = mesh;
	gameObj.AddComponent<MeshRenderer>().sharedMaterial = Material;

	Instantiate(gameObj, new Vector3(0, 0, 2), Quaternion.identity);
	Instantiate(gameObj, new Vector3(1, 0, 2), Quaternion.identity);
}

Drag the Standard material to the script field.
Play. In the Frame Debug window you'll see "Draw Mesh (instanced) TRIANGLE". That's what I'm looking for.
Switch the field to StandardFast and repeat. You'll see "Dynamic Batch".
Unfortunately, most of my instance-able meshes are oversize for dynamic batching. If we disable batching in the player settings we see that this results in more draw call overhead.
Thanks everyone!

@dbastienMS any idea?

Everything should be migrated to FastConfigurable.  I haven't done any testing on the earlier shaders since their release.  FastConfigurable is a superset of existing functionality and is faster and easier to use as well.
StandardFast in particular is a dangerous red herring - at the time Unity tied Standard shader perf tradeoffs to the shader model supported and special cases for specific consoles, etc.
This is no longer the case, so StandardFast should be removed.  Additionally, it isn't performant enough for full screen rendering - it clocks in somewhere around 70 ALU ops.
We've got an open issue to remove this following shaders if someone can help out:
BlinnPhongConfigurable
BlinnPhongConfigurableTransparent
LambertianConfigurable
LambertianConfigurableTransparent
StandardFast
UnlitConfigurable
UnlitConfigurableTransparent
VertexLitConfigurable
VertexLitConfigurableTransparent

@dbastienMS do you remember which issue we had open for this?  I might go ahead and take care of it now.

It was surprisingly hard to find: https://github.com/Microsoft/HoloToolkit-Unity/issues/628

I guess I'm missing something, but I have the same question about GPU Instancing on the FastConfigurable shader.

@bkfichter the FastConfigurable shader is being replaced by a new shading system which you can find in the Dev_Working_Branch. here is the specific shader and material editor.
Using this new shader may help with your scenario if you are still experiencing issues with the FastConfigurable shader.


Please invite me to the GitHub Community Organization

Name
André Bianchi
Discord Username (if applicable)
No response
Additional Context
Learning a new skill may take time, but in the end, it can also be rewarding in so many ways.

Thanks


A round of doc fixes

What type of PR is this?

Uncomment only one  /kind line, and delete the rest.
For example, > /kind bug would simply become: /kind bug

/kind api-change
/kind bug
/kind cleanup
/kind deprecation
/kind design
/kind failing-test
/kind feature
/kind flake
/kind code-refactoring

What does this PR do / why we need it:
Have you updated the necessary documentation?

[x] Documentation update is required by this PR.
[x] Documentation has been updated.

Which issue(s) this PR fixes:
Fixes #?
How to test changes / Special notes to the reviewer:

/assign @chetan-rns
/assign @bigkevmcd

Codecov Report

Merging #42 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #42   +/-   ##
=======================================
  Coverage   61.41%   61.41%
=======================================
  Files          52       52
  Lines        2462     2462
=======================================
  Hits         1512     1512
  Misses        816      816
  Partials      134      134

Flag
Coverage Δ

#unittests
61.41% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

pkg/cmd/bootstrap.go
32.00% <ø> (ø)

pkg/cmd/kam.go
0.00% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4a7d6f0...6e7356c. Read the comment docs.

Other than a few minor typos. Everything looks good to me. I'll raise a separate PR to fix them

/lgtm

/approve


Broken resume due to an inappropriate equal checking

Warning: I will close the issue without the minimal init.vim and the
reproduction instructions.
Problems summary

Related: https://github.com/Shougo/ddu.vim/issues/25

https://github.com/Shougo/ddu.vim/commit/8d1af8a36eba41dcd3c29a3329c573dacff1ec69 introduced a Strict Equality Comparison for sources, which always becomes false unless an array is reused to point the same instance in the memory.
Expected
Either one of the following.

When name is specified, the difference of sources should be ignored.
When comparing sources, use a deep equal algorithm.

Environment Information

Same as https://github.com/Shougo/ddu.vim/issues/25

Provide a minimal init.vim/vimrc without plugin managers (Required!)

Same as https://github.com/Shougo/ddu.vim/issues/25

How to reproduce the problem from neovim/Vim startup (Required!)

:call ddu#start({'name': 'file', 'sources':[{'name':'file'}], 'resume': v:true})<CR>
:close<CR>
:call ddu#start({'name': 'file', 'sources':[{'name':'file'}], 'resume': v:true})<CR> -- doesn't reuse the 1. buffer

Screenshot (if possible)
Upload the log messages by :redir and :message (if errored)

Confirmed. Thank you!


[ci skip] Add RaulPPelaez as maintainer

Checklist

[ ] Used a personal fork of the feedstock to propose changes
[ ] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[ ] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[ ] Ensured the license file is being packaged.

@conda-forge-admin, please rerender

It wont be able to rerender without the changes to the migrator I added to #19

Does it need to be rerendered, though? It probably wont build successfully anyhow.

Oh, and welcome! 🥳


[Question] Any examples of processing video frames of a user uploaded video (specifically for depth estimation)?

Hi there, I'm wondering if there are any examples of processing video frames of a user uploaded video? I'm specifically looking to run depth estimation on each frame of a short video, but any similar example would be useful.
If not, does this approach seem correct?

Use one of the approaches described here to draw each frame of the video to a canvas
Call HTMLCanvasElement.toBlob() on the canvas to get a Blob
Pass N (10?) of those Blobs to a worker at a time
For each of those Blobs call const image = await RawImage.fromBlob(blob) to get a RawImage
Run depth estimation on the list of images with await classifier([rawImage1, rawImage2, etc.])

Thanks for any help!

Ive had /some/ success with opencv.js, it ultimately uses canvas, however it supports a slew of image manipulation that doesnt come standard with Canvas.

Thanks @matbee-eth, curious if you have any code you could share showing this?

Thanks @matbee-eth, curious if you have any code you could share showing this (i.e. opencv.js + transformers.js together)? Took a look at your profile but didn't see anything, perhaps I missed it?

https://docs.opencv.org/4.x/dd/d00/tutorial_js_video_display.html
Ultimately interfacing with a transformers.js model just means using the expected data/tensor shape. I have no public examples at the moment, working on something privately and tackling this shortly


Exclude optional Category X dependency

Here is how I verify the license issue:
$ mvn license:add-third-party -Dlicense.useMissingFile
$  cat dubbo-admin-server/target/generated-sources/license/THIRD-PARTY.txt | grep "General Public License"

and the output:
     (Eclipse Public License - v 1.0) (GNU Lesser General Public License) Logback Classic Module (ch.qos.logback:logback-classic:1.2.3 - http://logback.qos.ch/logback-classic)
     (Eclipse Public License - v 1.0) (GNU Lesser General Public License) Logback Core Module (ch.qos.logback:logback-core:1.2.3 - http://logback.qos.ch/logback-core)

also, I've tested the funcationality(pagination) works well in #324

Codecov Report

Merging #371 into develop will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             develop     #371   +/-   ##
==========================================
  Coverage      22.37%   22.37%
- Complexity       289      290    +1
==========================================
  Files             76       76
  Lines           3142     3142
  Branches         526      526
==========================================
  Hits             703      703
  Misses          2364     2364
  Partials          75       75

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6342bb3...6eaf93c. Read the comment docs.


Allow rollbar_context to be added to StandardError

Background
Currently the Ruby SDK (Rollbar-gem) allows context for a Rollbar event to be set at the time the event is sent (via “extra” data or custom_data_method), or globally (config.payload_options), or within a scope.
Scopes are blocks that can execute arbitrary code with modified notifier state. Any Rollbar event sent within a scoped block will use the context data defined for that scope. Any exception that bubbles up out of the scope will lose access to the scoped notifier and its data. This is by design. If the context were allowed to remain active, it would effectively become global, which is not intended. The current scoping semantics assume the relevant Rollbar events will be sent from within the scope block.
No current methods allow for setting context or extra data for a specific exception that will be caught and reported later either by the uncaught exception handler or by an outer layer of the application code.
Motivation
There are multiple real world use cases where context needs to be set for an error that occurs within a certain context, but cannot be reported to Rollbar until it exits that context. In each of these cases, attaching the context to the exception object itself would solve the issue by making the context available for exactly the correct Rollbar event without resorting to various workarounds. In general, workarounds that avoid attaching the context to the exception will instead rely on unreliable hacks such as using global state.
Specific examples are provided here that seek to establish a broader pattern of need for a context that is bound to the exception object.
Sidekiq exception handlers
In the Sidekiq job queue architecture, Rollbar registers a Sidekiq error handler. This handler is called outside the execution of the job worker. When users define a Rollbar scope in their worker, it is no longer present when the error handler runs. Letting the user attach the context to the error itself would solve this.
Re-raised exceptions
In the case where an exception is rescued and reported to Rollbar with context, sometimes the exception is re-raised so that an outer layer can then handle it. (For example, to prepare a 5xx HTTP response.) This requires the outer layer to then know that the exception was already reported, otherwise it will be reported twice. If the inner layer doesn’t report the error and lets the outer layer handle it, the outer layer won’t have the context data to add to it.
Allowing context on the exception provides multiple ways to handle this situation. Either the inner layer can add the context and allow the outer layer to report it, or the inner layer can report it and add an unambiguous context for the before_process handler to ignore it. Both options ensure the event is sent only once and with the proper context.
Error context supported by frameworks
The react_on_rails framework supports using exception context from Sentry (ex.raven_context) and Honeybadger (ex.to_honeybadger_context), and has requested a similar capability from Rollbar. Providing this allows reliable access to exception context by frameworks and libraries that wouldn’t be possible using Rollbar-gem’s current modes.
Implementation
This proposal recommends optionally adding attr_accessor :rollbar_context to StandardError.
module RollbarErrorContext
  attr_accessor :rollbar_context
end

StandardError.include RollbarErrorContext

The context data is merged with trace.extra in the payload since it is expected to be related to the exception.
Precedent in Rollbar-gem
Rollbar-gem already optionally modifies (“monkey patches”) core classes in cases where it makes configuration significantly easier for the user.
Low impact
An accessor named rollbar_context wouldn’t be expected to conflict with anything else, and the feature can be enabled/disabled as needed.
Ease of use
The recommended implementation only requires that the user set the value of rollbar_context on the existing exception object. The alternate implementation would require the user to define subclasses of the expected runtime errors, which then add the accessor. This would require housekeeping for each of these classes, and would cause the modified errors to no longer be reported with the expected class name. (NB: If Rollbar-gem didn’t patch StandardError, many users would do so anyway for the same reasons as listed here.)
Ensures correct exception class
As noted above, the recommended implementation helps ensure that Rollbar receives the intended exception class name, since it is not necessary to create a wrapper class.
Example application code
A minimal application example that sets the context data for an exception and reraises it.
class ExampleClass
  def any_method(params)
    // do work…
  rescue => e
    // Let the uncaught exception handler or other application code
    // report this later with the context set here.
    e.rollbar_context = { foo: ‘bar’ }
    raise
  end
end

Related:
https://github.com/rollbar/rollbar-gem/issues/893
https://github.com/rollbar/rollbar-gem/issues/875

Documented here: https://docs.rollbar.com/docs/ruby#section-error-contexts


Problem with a custom widget that wraps a TextInput

Is there an existing issue for this?

[X] I have searched the existing issues.

Is this issue related to iced?

[X] My hardware is compatible and my graphics drivers are up-to-date.

What happened?
I just ran into this while upgrading from 0.10.0 to 0.12.1. I think this is related to #2318, but I have some additional symptoms that may be related to the wrapper widget.
I have a custom widget used for wrapping TextInputs. All of the Widget functions forward to the wrapped widget, except that on_event adds some hooks for ctrl+z/etc.
On 0.12.1, I see these issues:

The initial text ("!") is not shown on app startup.
The text is shifted down so that you mostly can't see it.
The visible text doesn't change after an undo/redo event, even after multiple. The visible text only changes after an input event.

Gist: https://gist.github.com/mtkennerly/edea51b6c35b1cb85421107070cc7569

https://github.com/iced-rs/iced/assets/4934192/2534dece-300d-4281-ab2f-77b80700b16b

What is the expected behavior?

Text is fully visible
Initial text is visible immediately on startup
Visible text changes immediately after undo/redo event

Version
crates.io release
Operating System
Windows
Do you have any log output?
No response

It seems your widget is "state-transparent" but it's including the content state in both Widget::state and Widget::children. Then, you call layout directly with tree, but call the other methods with tree.children[0].
The fix is to remove the children implementation and directly use tree everywhere: https://gist.github.com/hecrj/a7c538d9d0ee37f90a77715f06bb095b/revisions

Thanks! That works perfectly :)


WIP: keymgr_sideload_otbn_test OTBN hangs during sideloading

Reproducer: ./bazelisk.sh test //sw/device/tests:keymgr_sideload_otbn_test_fpga_cw310_sival_rom_ext --test_output=streamed
E00006 otbn_testutils.c:40] CHECK-fail: OTBN is locked. Error bits: 0x00800020

I believe you have to downgrade the OTBN fatal errors to be able to execute the KEY_INVALID flow in this test. See:
https://github.com/lowRISC/opentitan/blob/4555303fd75a55fcd7888ca9737df2f4e2358384/sw/device/tests/keymgr_sideload_otbn_test.c#L70-L73

Indeed, that was needed at the time, but it's no longer needed since your 7d34f1d68cb932aaae3e1b821d14dba67e09e0ad commit, so I'm closing this PR.


Add MonthPickerInput & YearPickerInput

I think these components will be useful in many cases.
https://mantine.dev/dates/month-picker-input/
https://mantine.dev/dates/year-picker-input/

This will be in the next release 0.14.5  :woman_dancing:
Thanks again @stpnvkirill


Pecl install requires libpq-fe.h (Postgres) even though --with-postgres=n

Please answer these questions before submitting your issue. Thanks!

What did you do? If possible, provide a simple script for reproducing the error.
pecl install openswoole-4.8.1

Full output at the bottom of this issue.
Also note that this is a brand new installation of ubuntu 18.04, just upgraded to 20.04. This is a bare metal install, and not a virtualized install. I have installed some things necessary for php8.1, and that is it. (my php module list is below)
This may also be important: the first time I tried to install openswoole on this machine, I selected 'y' for postgres support. When it failed, I tried setting it to 'n'. When that also failed, I tried deleting pear's cache and selecting 'n' again. When that failed, I rebooted and selected 'n' again. I rebooted a couple more times, and the install failed every time in the same was as what I've pasted at the bottom of this issue.

What did you expect to see?

I expected the installation to run successfully

What did you see instead?

That it failed requiring Postgres, even though I selected 'n' for Postgres support.

What version of Open Swoole are you using (show your php --ri openswoole)?

Attempting to install openswoole-4.8.1 from pecl, on php8.1

What is your machine environment used (show your uname -a & php -v & gcc -v) ?

uname -a
Linux s187667 5.4.0-91-generic #102-Ubuntu SMP Fri Nov 5 16:31:28 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

php -v
PHP 8.1.0 (cli) (built: Nov 25 2021 20:22:03) (NTS)
Copyright (c) The PHP Group
Zend Engine v4.1.0, Copyright (c) Zend Technologies
    with Zend OPcache v8.1.0, Copyright (c), by Zend Technologies

gcc -v
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 9.3.0-17ubuntu1~20.04' --with-bugurl=file:///usr/share/doc/gcc-9/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,gm2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-9 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-9-HskZEa/gcc-9-9.3.0/debian/tmp-nvptx/usr,hsa --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)

php -m
[PHP Modules]
calendar
Core
ctype
curl
date
dba
dom
exif
FFI
fileinfo
filter
ftp
gd
gettext
hash
iconv
igbinary
imap
intl
json
libxml
mbstring
mysqli
mysqlnd
openssl
pcntl
pcre
PDO
pdo_mysql
pdo_pgsql
pdo_sqlite
pgsql
Phar
posix
readline
Reflection
session
shmop
SimpleXML
sockets
sodium
SPL
sqlite3
standard
sysvmsg
sysvsem
sysvshm
tidy
tokenizer
xml
xmlreader
xmlwriter
xsl
Zend OPcache
zip
zlib

[Zend Modules]
Zend OPcache

full install output:
# pecl install openswoole-4.8.1
WARNING: channel "pecl.php.net" has updated its protocols, use "pecl channel-update pecl.php.net" to update
downloading openswoole-4.8.1.tgz ...
Starting to download openswoole-4.8.1.tgz (1,623,495 bytes)
.................................................................................................................................................................................................................................................................................................................................done: 1,623,495 bytes
447 source files, building
running: phpize
Configuring for:
PHP Api Version:         20210902
Zend Module Api No:      20210902
Zend Extension Api No:   420210902
enable sockets supports? [no] : y
enable openssl support? [no] : y
enable http2 support? [no] : y
enable mysqlnd support? [no] : y
enable json support? [no] : y
enable curl support? [no] : y
enable postgres support? [no] : n
building in /tmp/pear/temp/pear-build-customeriMr8YV/openswoole-4.8.1
running: /tmp/pear/temp/openswoole/configure --with-php-config=/usr/bin/php-config --enable-sockets=y --enable-openssl=y --enable-http2=y --enable-mysqlnd=y --enable-swoole-json=y --enable-swoole-curl=y --with-postgres=n
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for a sed that does not truncate output... /bin/sed
checking for pkg-config... /usr/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
checking for cc... cc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether cc accepts -g... yes
checking for cc option to accept ISO C89... none needed
checking how to run the C preprocessor... cc -E
checking for icc... no
checking for suncc... no
checking for system library directory... lib
checking if compiler supports -R... no
checking if compiler supports -Wl,-rpath,... yes
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking target system type... x86_64-pc-linux-gnu
checking for PHP prefix... /usr
checking for PHP includes... -I/usr/include/php/20210902 -I/usr/include/php/20210902/main -I/usr/include/php/20210902/TSRM -I/usr/include/php/20210902/Zend -I/usr/include/php/20210902/ext -I/usr/include/php/20210902/ext/date/lib
checking for PHP extension directory... /usr/lib/php/20210902
checking for PHP installed headers prefix... /usr/include/php/20210902
checking if debug is enabled... no
checking if zts is enabled... no
checking for gawk... no
checking for nawk... nawk
checking if nawk is broken... no
checking enable debug log... no
checking enable trace log... no
checking enable sockets support... y
checking enable openssl support... y
checking enable http2.0 support... y
checking swoole support... yes, shared
checking enable mysqlnd support... y
checking enable postgreSQL support... n
checking enable c-ares support... no
checking dir of openssl... no
checking dir of jemalloc... no
checking enable asan... no
checking whether to enable swoole coverage support... no
checking whether to enable Swoole developer build flags... no
checking whether to enable Swoole JSON build flags... y
checking whether to enable Swoole CURL build flags... y
checking whether to enable thread context... no
checking if compiling with clang... no
checking for cc option to accept ISO C99... none needed
checking for accept4 in -lc... yes
checking for signalfd in -lc... yes
checking for eventfd in -lc... yes
checking for epoll_create in -lc... yes
checking for poll in -lc... yes
checking for sendfile in -lc... yes
checking for kqueue in -lc... no
checking for backtrace in -lc... yes
checking for daemon in -lc... yes
checking for mkostemp in -lc... yes
checking for inotify_init in -lc... yes
checking for malloc_trim in -lc... yes
checking for inotify_init1 in -lc... yes
checking for ptrace in -lc... yes
checking for getrandom in -lc... yes
checking for arc4random in -lc... no
checking for pthread_rwlock_init in -lpthread... yes
checking for pthread_spin_lock in -lpthread... yes
checking for pthread_mutex_timedlock in -lpthread... yes
checking for pthread_barrier_init in -lpthread... yes
checking for pthread_mutexattr_setpshared in -lpthread... yes
checking for pthread_mutexattr_setrobust in -lpthread... yes
checking for pthread_mutex_consistent in -lpthread... yes
checking for pcre_compile in -lpcre... no
checking for ares_gethostbyname in -lcares... yes
checking for gzgets in -lz... yes
checking for BrotliEncoderCreateInstance in -lbrotlienc... yes
checking for BrotliDecoderCreateInstance in -lbrotlidec... yes
checking for cpu affinity... yes
checking for socket REUSEPORT... yes
checking for futex... yes
checking for ucontext... yes
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking for valgrind... no
checking how to run the C++ preprocessor... g++ -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for struct cmsghdr... yes
checking for hstrerror... yes
checking for socketpair... yes
checking for if_nametoindex... yes
checking for if_indextoname... yes
checking netdb.h usability... yes
checking netdb.h presence... yes
checking for netdb.h... yes
checking netinet/tcp.h usability... yes
checking netinet/tcp.h presence... yes
checking for netinet/tcp.h... yes
checking sys/un.h usability... yes
checking sys/un.h presence... yes
checking for sys/un.h... yes
checking sys/sockio.h usability... no
checking sys/sockio.h presence... no
checking for sys/sockio.h... no
checking for field ss_family in struct sockaddr_storage... yes
checking if getaddrinfo supports AI_V4MAPPED... yes
checking if getaddrinfo supports AI_ALL... yes
checking if getaddrinfo supports AI_IDN... yes
checking if gethostbyname2_r is supported... yes
checking for valgrind... no
checking for clock_gettime in -lrt... yes
checking openssl fallback dir... checking for SSL_connect in -lssl... yes
checking for pg_config... not found
configure: error: Cannot find libpq-fe.h. Please specify correct PostgreSQL installation path
ERROR: `/tmp/pear/temp/openswoole/configure --with-php-config=/usr/bin/php-config --enable-sockets=y --enable-openssl=y --enable-http2=y --enable-mysqlnd=y --enable-swoole-json=y --enable-swoole-curl=y --with-postgres=n' failed

Note that I have just installed libpq-dev via apt, and the installation of openswoole-4.8.1 via pecl was successful.
Though I'd imagine if postgres support is optional and selected by the user, you may not intend for libpq-dev to be a dependency.

--with-postgres=n and --with-postgres should be removed if you don't like to support postgres

Installing from pecl, I'm prompted if I would like to add Postgres support. This isn't a configure option passed to the compilation process.
enable sockets supports? [no] : y
enable openssl support? [no] : y
enable http2 support? [no] : y
enable mysqlnd support? [no] : y
enable json support? [no] : y
enable curl support? [no] : y
enable postgres support? [no] : n

Considering that reponding with 'y' is accepted as yes, I believe the installer should be changed to accept 'n' as no.

Will look into improving this, thanks @benwills .

Thanks. I definitely appreciate all your work on OpenSwoole. With a background in C, I love having such a close interface to the event loop, networking, etc.

It will be available in the next version.


Question on how to define queries on Domain types?

I am new to GraphQL so any help would be appreciated. Currently I understand if I have to define queries on Domain types, I will need to add annotations in the same file, something like this:
public class A {
    private String id;
    private List<B> bList;

    @GraphQLQuery(name = "bList")
    public List<B> getBListByX(@GraphQLArgument(name = "x") final String x)
   }

This generates schema like:
 type A {
   id: String
   bList(x: String): [B]
 }

We have existing POJOs which we want to use for generating the schema but I do not want to add the query annotations on the POJO itself. Is there any way in which I can add the queries separately in a different file, without having to handwrite all POJOs

Certainly! Fields (queries) can be attached to types from the outside and, in all cases, annotations are optional. In fact, I'd argue if you need to modify your POJOs to fit a library/framework - something is wrong.
Let's say you have these classes:
public class Book { //POJO

    private String title;
    private String isbn;

   public getTitle() {...} //no annotations needed on getters here by default
   public getIsbn() {...}
}

public class BookService { //top-level stateless singleton

    @GraphQLQuery //here you need the annotation by default, but can be changed
    public List<Book> books(String title) {
        return ...; //find all books with the given title
    }
}

Here's some things to be aware of:

You can attach a field (query) to any type from the outside. You don't want to nest logic into POJOs, so any nested queries that are not simple getters can be attached from the outside:

public class BookService { //top-level stateless singleton

     //notice @GraphQLContext
    public List<Review> reviews(@GraphQLContext Book book, LocalDate publishedAfter) {
        return ...; //find review by book
    }

    public List<Book> books(String title) { // same as before
        return ...; //find all books with the given title
    }
}

Now, the type Book will have the reviews field, even though the Book POJO itself was unchanged. The extra field was attached via @GraphQLContext.
 type Book {
   title:  String
   isbn: String
   reviews(publishedAfter: LocalDate): [Review]
 }

Top-level queries are normally methods on singletons (like BookService above). By default, they need @GraphQLQuery to be exposed, but this is configurable. Lower-level queries are normally POJO fields. They do not need the annotation to be exposed by default, but again, this is configurable.

So, by default, with the situation from the start, you'll end up with:
 type Book {
   title:  String
   isbn: String
 }

type Query {
   books(title: String): [Book]
}

The way you configure what methods are exposed as queries is via choosing a ResolverBuilder implementation. You can configure different ResolverBuilders to be used for top-level and lower-levels, and also per type. There's 3 built-in implementations: AnnotatedResolverBuilder which exposes only the annotated methods, PublicResolverBuilder which exposes all public methods, and BeanResolverBuilder which exposes only public getters. By default, AnnotatedResolverBuilder is used for top-level, and both AnnotatedResolverBuilder and BeanResolverBuilder for deeper levels. This is why POJOs typically don't need explicit annotations - their getters will get exposed anyway. You can also implement your own ResolverBuilder to perform custom selection, or modify the provided ones, e.g. to change the way the queries they produce are named.

Let;s now say you BookService without annotations:
public class BookService { //top-level stateless singleton

    public List<Book> books(String title) { //no annotations any more
        return ...; //find all books with the given title
    }
}

You would register PublicResolverBuilder and get the same behavior as above:
generator
    .withResolverBuilder(new PublicResolverBuilder()) //globally use PublicResolverBuilder for top-level
    .withOperationsFromSingleton(new BookService())

You could also only use PublicResolverBuilder for a specific bean:
generator
    .withOperationsFromSingleton(new BookService(), new PublicResolverBuilder())

ResolverBuilders also give names to operations, and this is done via OperationNameGenerator. You can change the name generator a builder is using:

generator
    .withResolverBuilder(new PublicResolverBuilder()
        .withOperationNameGenerator(...)) //existing or custom implementation
    .withOperationsFromSingleton(new BookService())

Does this clear things up for you?
It's probably more info than you wanted, but my goal was to arm you with enough knowledge to handle most cases. I hope you're not overwhelmed :)

Thanks for the suggestion I really  appreciate the details you provided. But this is what keeps on happening. Taking the BookService example, This is my code structure:

public class Book { //POJO

    private String title;
    private String isbn;
    private Author author;

    public getTitle() {...} //no annotations needed on getters here by default
    public getIsbn() {...}
}

public class BookService {
   @GraphQLQuery
    public List<Book> books(String title) { //no annotations any more
        return ...; //find all books with the given titl
    }
}

Now I want to filter by author name, so I add the the following to the BookService

public class BookService {
   @GraphQLQuery
    public List<Book> books(String title) { //no annotations any more
        return ...; //find all books with the given title
    }

   @GraphQLQuery
   public Author getAuthor(
                 @GraphQLContext Author author,
                 @GraphQLArgument String authornameFilter) {
       ...
   }
}

The generated schema is updated as:
type QUERY_ROOT {
  author(arg0: AuthorInput, authornameFilter: String): Author
}

type Book {
   title:  String
   isbn: String
   author(arg0: AuthorInput, authornameFilter: String): Author
}

I have removed the mutations from the schema, still for some reason I am getting AuthorInput as the first argument for the query

Oh, just by the name of QUERY_ROOT I can tell you're using an old version of SPQR. Earlier implementations used to behave differently in that nested queries would also be exposed as top level (taking the context argument explicitly). Plus, there may have been a bug making the nested one also expect an argument.
Please try using the latest (0.9.6 at the moment, and 0.9.7 coming out by the end of the week).

Yes we were using the 0.9.3 version of SPQR library I have updated to the latest and it works as expected. Thanks.
Just another question on the latest version. I saw that both queries and mutations are showing up under the type Query. Whereas in the previous version they had both QUERY_ROOT and MUTATION_ROOT
Is that expected?

Hmm, no, it is not. It should have appeared under Mutation. Something must have gone wrong with the discovery.
What ResolverBuilders do you have configured? Certain combinations of builders can lead to this issue. E.g. AnnotatedResolverBuilder could decide something is a mutation, while PublicResolverBuilder coild decide the same method is a query. But even then, you should at least see it duplicated under both Mutation and Query roots, not only under Query... Is this the case?
If you could exemplify your schema generator config, it would help me troubleshoot this.

Yes I figured that one out. The PublicResolveBuilder was also being used for the Mutation. I have fixed that and it works as expected. Thanks!!

Great!
v0.9.7 will have deduping logic for methods between queries and mutations (the current logic dedupes only with lin one type), so these errors should go away permanently.
Closing this one.


scenes transitions' animation  about 3.0.11 is slower then 2.x

scenes transitions' animation  about 3.0.11 is slower then 2.x

It is fully configurable by using duration property.

I'm fine to change 'default' duration if you can say exact value for native iOS animation duration. Now it is equal to 500ms.

after test,  I think 300ms is ok


Support for loading 3D Tiles via CZML and Entity API

This change enables CZML and the Entity API to load 3D Tiles tilesets. Since Cesium3DTileset has dozens of options, I only exposed uri/show/maximumScreenSpaceError to start but I'm open to adding more if we think they're needed.  We also use the entity position/orientation (if specified) to compute the tileset.modelMatrix allowing for moving tilesets that can be tracked with the camera. There seems to be an issue with 3D Tiles here where a moving tileset never actually gets loaded unless you pause animation to the camera stops.  @loshjawrence seems to know how to fix this. (Also CC @lilleyse)

Details:

New Sandcastle example, CZML 3D Tiles.html
New class, Cesium3DTilesetGraphics for representing a tileset via the Entity API.
New class, Cesium3DTilesetVisualizer for creating/managing the primitives.
Add Entity.tileset which is an instance of Cesium3DTilesetGraphics
Specs for everything

This can wait until after Monday's release to merge so we aren't stuffing in a feature at the last moment.

@lilleyse just a reminder to look at the 3D Tiles loading issue in this branch.  Thanks!

For whoever has the time to look at the tile loading issue:
it will require having the tilesets track their frame position delta and subtract it from the camera delta before finding the magnitude of change this will give a relative-to-tileset magnitude of change. This camera  delta stuff occurs in Cesium3DTilesetTraversal.js

@mramato have you tried turning cullRequestsWhileMoving to false?
https://cesium.com/docs/cesiumjs-ref-doc/Cesium3DTileset.html#cullRequestsWhileMoving

No, but if that is the case it seems like something that shouldn't default to true since that means broken by default (or it needs to be made smarter)

cullRequestsWhileMoving turns the code path I mentioned on and off, This code path needs to be aware of the tilesets movement which it is not right now.
There is also a "multipler" flag with a similar name and if you set it to 0 it will turn the code path off.

To reproduce the problem:

Download FlyingHouse.zip and unpack it into the Cesium directory so you have a /FlyingHouse directory at the top level.

Run this Sandcastle using this branch: http://localhost:8080/Apps/Sandcastle/index.html#c=ZZBPT8MwDMW/StRLW2lKxBW6CRggJiFxqMQpl6zxtojUQYmzqUN8d9xtMP7kZvu951+8NVFsHewgiqlA2Ik5JJd7+XLoVWV3KOcByTiEWE7Eu0bBL21C9vYGXW8IxKWgmEHjR32lUeMxUVpDpg05cog01lan7Pm+93ffI+mDsVWpHvzgcP0YcgKVXP/mQXYsLOtao6QNYLXK2JELWJ1z6xPNaSFF072CvUdyNPCHzkIJY88xyBrodljwxpbBvXcEatG25Qh+oC8mRZNo8DA7Rl8zS4gkcvSVlIqA0diZ1DLzLpJdSqOtUV+mxrqtcHaqiz/H04XovEmJJ6vsfev2oItZo1j/yzYehE/xvIXozTBKNhezp2NTStkoLv+7KAS/NPFHosZP

You'll see a satellite marker flying like the typical simple.czml, but it's not until you hit pause that the house comes in.

And just to confirm (though @loshjawrence already mentioned it) setting cullRequestsWhileMoving = true doesn't fix this problem.

@mramato actually it should be cullRequestsWhileMoving = false. That fixed it for me, for debugging purposes at least.
I'm going to go with a simple solution where it treats cullRequestsWhileMoving as false if the tileset is determined to be moving.

@lilleyse  while your at it we should lower the cullRequestWhileMovingMultiplier because its too aggressive for static tilesets.

@loshjawrence I kept it out of https://github.com/AnalyticalGraphicsInc/cesium/pull/8598 but you can open a separate PR. 60 to 0.1 is a huge difference though...

Adding the 3D Tileset to the Entity API is kind of big news! It solves this 3D Tiles tracking/camera flying issue for example: https://github.com/AnalyticalGraphicsInc/cesium/issues/7086 since you can now add a 3D Tileset like this:
var viewer = new Cesium.Viewer('cesiumContainer');
var tilesetEntity = viewer.entities.add({
    tileset: {
        uri: "../../SampleData/Cesium3DTiles/Batched/BatchedColors/tileset.json"
    }
});
viewer.flyTo(tilesetEntity);

Or from ion:
var viewer = new Cesium.Viewer('cesiumContainer');
var tilesetEntity = viewer.entities.add({
    tileset: {
        uri: Cesium.IonResource.fromAssetId(5741)
    }
});
viewer.flyTo(tilesetEntity);

The code, docs, specs and Sandcastle example all look good to me. My only quandary here is: inheriting the modelMatrix from the entity's position & orientation seems like the wrong approach. This would only produce the expected result when the 3D Tileset is located at the center of the Earth, right?
I almost feel like Entity.tileset needs to be treated more like Entity.polygon, which is positioned independently of Entity.position. But then to create the flying house example you have, you would need to expose a position and orientation on Entity.tileset that would follow the Entity.position & orientation?
I would be fine with merging this as-is assuming the common approach would be users creating a tileset on an entity with no position, and only using position when that tileset is located at the center of the Earth and is meant to be time dynamic.

We should also remember to update https://github.com/AnalyticalGraphicsInc/czml-writer/wiki/CZML-Guide

I would be fine with merging this as-is assuming the common approach would be users creating a tileset on an entity with no position, and only using position when that tileset is located at the center of the Earth and is meant to be time dynamic.

This is exactly what they should do.

@OmarShehata ready

Thanks @mramato . I just merged in master and resolved the conflict in CHANGES.md. Will merge when green.

Awesome, thanks @OmarShehata!


chore: rename sidecli to deku-cli

Problem
The name of Deku's CLI is outdated.
Solution
This follows the name from deku-node and renames sidecli to deku-cli.

It makes more sense. LGTM.


Add functionality to download analysis on startup

If the sarif-viewer.githubCodeScanningInitialAlert setting has a
value, use the contents of the value as the alert to apply. This is
meant as a way to automatically apply fixes from a codescanning alert.
Additionally, if this alert has any fixes in it, they are automatically
applied. We also need to handle some slight differences in the SARIF
format. The fix objects that we are receiving are non-standard. We
expect that for each fix, there is a diff in its property bag that
contains a git-patch formatted version of the diff to apply.
Lastly, this adds some simple telemetry. It collects that the
autofix was applied, whether or not it was successful, and (if
unsuccessful) a failure message.

@nickfyson, this should work for downloading autofixes on startup, but of course I can't really test this until we have some sample autofix data to play with.

This PR is ready for review now. I've confirmed that it is working. However, there may need to be some tweaks as we formalize the feature on the code scanning side.

@EasyRhinoMSFT @50Wliu would one of you be able to take a look at this PR? I'm happy to chat about this to explain some of the changes.

I just added two more commits. When reviewing this PR, it will be helpful to review commit-by-commit.


Breaks on Ascii Characters

My HTML contains a registered symbol ® which is breaking the conversion. Any way to fix to allow html entities?

DOMDocument has issues with utf-8 encoded strings. Your class may work well with english text but with other utf-8 chars like umlaute in other languages you get screwed chars.
I fixed it with
static function convert($html) {
$html = static::fixNewlines($html);
$html = mb_convert_encoding($html, 'HTML-ENTITIES', 'UTF-8');
...

thanks @somatonic converting non-latin beforehand did the trick for me
$html = mb_convert_encoding($html, 'HTML-ENTITIES', 'UTF-8');
$text = Html2Text\Html2Text::convert($html);

+1 on this issue!
Thanks @somatonic for the workaround!

Thanks for the convert trick! :D!

@somatonic
Thank you, this resolved it.
It broken German Umlaut characters

I use utf8_decode
$text = utf8_decode(Html2Text\Html2Text::convert($html));

Can anyone attach a sample HTML file that is causing these issues? I'd love to fix this or at least write up an official workaround :)

Something like this?
utf8-example.html.txt

The workaround looks good, it doesn't change the behaviour of any of the other tests. I've merged this in and released 0.3.2 - can anyone test this out and check that it works better?

Maybe convert() could take an optional $encoding parameter that defaults to 'UTF-8' and is used as the third parameter to mb_convert_encoding? So that the application developer can use mb_detect_encoding if she wants to, otherwise just specify a known incoding.

From what I understand, the behaviour in 0.3.3 will now change the encoding from UTF-8 to HTML-ENTITIES ONLY if the document encoding is strictly UTF-8 (the true parameter to mb_detect_encoding). So, if a developer wants to use a different & known encoding, they can do this before they pass the string into #convert(), by calling mb_convert_encoding themselves.
If not - can you provide a use case/test case so I can understand the issue better?

Sounds reasonable, thanks!


Some suggestions

First I have to say that this package is very well designed, the levels use and the hierarchical inheritance of permissions between user permissions and his role permissions makes it unique and more efficient than other Laravel roles and permissions packages. Thank you :+1:
Attach Permissions and Roles by their slugs
I have found that if I could attach permissions and roles by slug would be very useful as I don't have to find out every time what was the permission id or to create an object for it. That would save some time and make it easier by attaching them by slug.
Multiple conditions (OR + AND) in permissions check
By checking permissions by multiple conditions, I can check user permissions with OR + AND in same check.
Ex. I want to check if user can 'view' OR 'edit' and can 'forum' OR 'all'
current solution
if ($user->can('view|edit')) {
     if($user->can('forum|all') {
          give access
     }
}

If there is a solution to make this check in just one line, that would be awesome
Thank you again for this great package :+1:

Hi @thekordy :-)
What about this? Its only one line
if ($user->can('view|edit') && $user->can('forum|all')) {
          give access
}

Thanks @balping , nice to see you here :) ... and yes that would work very well.

but sadly this project doesn't seem to be alive

This is sad, I've found this package the perfect one for me to manage roles and permissions.

I'm glad you find this package helpful, but I have been fighting with disease I have for the past 6 months and I really can't work on this project anymore.

Sorry to hear that. Get better

@romanbican you did a great job in this package :+1:  and I hope you get well soon.


[REQUEST] Extensions support for selenium

requesting for extensions support for selenium

Duplicate of https://github.com/openbullet/OpenBullet2/issues/762