Forwarding values to custom loss function for semiparametric estimation.

Summary
Forward columns of your dataset directly to a custom objective function.
Motivation
This is useful for semi-parametric models like poisson process regression where y | x, t ~ Poisson(mu(x) * t) and we wish to learn mu(), or a variance regression where mean is known, ex: y | x, m ~ Logistic(m, exp(s(x)) and we wish to learn s().
It's possible that lightgbm would pick up on that model if you were to just feed it those parameters as features with no additional context, but consider the case where you're using the poisson process regression for causal inference, and the the time intervals are correlated with treatment status. Including them naively could lead to unpredictable biases.
Description
My optimal intferface would look something like this
reg = CustomReg(objective=obj_func_with_param_named_time)
reg.fit(x, y, custom_objective_train_params={"time":t})

I recognize that this is probably not widely useful enough to add as a feature to main, and might need some custom work for each use case, but I'm really struggling to make it work for my case.
I've tried modifying the Dataset, the objective function wrapper, eval function wrapper, and inheriting from LGBMRegressor modifying the fit method to look for loc and log_scale parameters, as below, but something seems to be going wrong during fitting. My best guess is that there is some sorting by covariates happening at each boosting stage that doesn't apply to my hackily tacked on dataset attributes, so they end up misaligned with predictions and labels. I'm hoping that doesn't mean it requires digging into the c++...
I would REALLY appreciate anyones advice on how to do this.
from typing import Any, Callable, Dict, List, Optional, Tuple, Union
import copy
from inspect import signature
import numpy as np
import lightgbm as lgb

_choose_param_value = lgb.sklearn._choose_param_value
_ConfigAliases = lgb.sklearn._ConfigAliases
_log_warning = lgb.sklearn._log_warning

def copy_signature(base):
    def decorator(func):
        func.__signature__ = signature(base)
        return func
    return decorator

class Dataset(lgb.sklearn.Dataset):

    def __init__(self, data, label=None, reference=None,
                 weight=None, group=None, init_score=None, silent='warn',
                 loc = None, log_scale=None,
                 feature_name='auto', categorical_feature='auto', params=None,
                 free_raw_data=True):
        """Initialize Dataset.

        Parameters
        ----------
        data : str, pathlib.Path, numpy array, pandas DataFrame, H2O DataTable's Frame, scipy.sparse, Sequence, list of Sequence or list of numpy array
            Data source of Dataset.
            If str or pathlib.Path, it represents the path to a text file (CSV, TSV, or LibSVM) or a LightGBM Dataset binary file.
        label : list, numpy 1-D array, pandas Series / one-column DataFrame or None, optional (default=None)
            Label of the data.
        reference : Dataset or None, optional (default=None)
            If this is Dataset for validation, training data should be used as reference.
        weight : list, numpy 1-D array, pandas Series or None, optional (default=None)
            Weight for each instance.
        group : list, numpy 1-D array, pandas Series or None, optional (default=None)
            Group/query data.
            Only used in the learning-to-rank task.
            sum(group) = n_samples.
            For example, if you have a 100-document dataset with ``group = [10, 20, 40, 10, 10, 10]``, that means that you have 6 groups,
            where the first 10 records are in the first group, records 11-30 are in the second group, records 31-70 are in the third group, etc.
        init_score : list, list of lists (for multi-class task), numpy array, pandas Series, pandas DataFrame (for multi-class task), or None, optional (default=None)
            Init score for Dataset.
        silent : bool, optional (default=False)
            Whether to print messages during construction.
        feature_name : list of str, or 'auto', optional (default="auto")
            Feature names.
            If 'auto' and data is pandas DataFrame, data columns names are used.
        categorical_feature : list of str or int, or 'auto', optional (default="auto")
            Categorical features.
            If list of int, interpreted as indices.
            If list of str, interpreted as feature names (need to specify ``feature_name`` as well).
            If 'auto' and data is pandas DataFrame, pandas unordered categorical columns are used.
            All values in categorical features should be less than int32 max value<PHONE_NUMBER>).
            Large values could be memory consuming. Consider using consecutive integers starting from zero.
            All negative values in categorical features will be treated as missing values.
            The output cannot be monotonically constrained with respect to a categorical feature.
        params : dict or None, optional (default=None)
            Other parameters for Dataset.
        free_raw_data : bool, optional (default=True)
            If True, raw data is freed after constructing inner Dataset.
        """
        self.handle = None
        self.data = data
        self.label = label
        self.reference = reference
        self.weight = weight
        self.loc = loc
        self.log_scale = log_scale
        self.group = group
        self.init_score = init_score
        self.silent = silent
        self.feature_name = feature_name
        self.categorical_feature = categorical_feature
        self.params = copy.deepcopy(params)
        self.free_raw_data = free_raw_data
        self.used_indices = None
        self.need_slice = True
        self._predictor = None
        self.pandas_categorical = None
        self.params_back_up = None
        self.feature_penalty = None
        self.monotone_constraints = None
        self.version = 0
        self._start_row = 0  # Used when pushing rows one by one.

    def get_loc(self) -> Optional[np.ndarray]:
        if self.loc is None:
            # self.weight = self.get_field('weight')
            raise ValueError("loc is not set, re-initialize dataset")
        return self.loc

    def get_log_scale(self) -> Optional[np.ndarray]:
        if self.log_scale is None:
            # self.weight = self.get_field('weight')
            raise ValueError("log_scale is not set, re-initialize dataset.")
        return self.log_scale

class _ObjectiveFunctionWrapper(lgb.sklearn._ObjectiveFunctionWrapper):
    """Proxy class for objective function."""

    def __call__(self, preds, dataset):
        """Call passed function with appropriate arguments.

        Parameters
        ----------
        preds : array-like of shape = [n_samples] or shape = [n_samples * n_classes] (for multi-class task)
            The predicted values.
        dataset : Dataset
            The training dataset.

        Returns
        -------
        grad : array-like of shape = [n_samples] or shape = [n_samples * n_classes] (for multi-class task)
            The value of the first order derivative (gradient) of the loss
            with respect to the elements of preds for each sample point.
        hess : array-like of shape = [n_samples] or shape = [n_samples * n_classes] (for multi-class task)
            The value of the second order derivative (Hessian) of the loss
            with respect to the elements of preds for each sample point.
        """
        labels = dataset.get_label()
        params = signature(self.func).parameters
        # print(self.func)
        # print(f"obj params {params}")
        # raise ValueError()
        argc = len(params)
        extra_params = {}
        if "loc" in params:
            extra_params["loc"] = dataset.get_loc()
        if "log_scale" in params:
            extra_params["log_scale"] = dataset.get_log_scale()
        # print("extra_params of wrapper", extra_params)
        argc = argc - len(extra_params)
        if argc == 2:
            grad, hess = self.func(labels, preds, **extra_params)
        elif argc == 3:
            grad, hess = self.func(labels, preds, dataset.get_group(), **extra_params)
        else:
            raise TypeError(f"Self-defined objective function should have 2 or 3 arguments, got {argc}")
        """weighted for objective"""
        weight = dataset.get_weight()
        if weight is not None:
            """only one class"""
            if len(weight) == len(grad):
                grad = np.multiply(grad, weight)
                hess = np.multiply(hess, weight)
            else:
                num_data = len(weight)
                num_class = len(grad) // num_data
                if num_class * num_data != len(grad):
                    raise ValueError("Length of grad and hess should equal to num_class * num_data")
                for k in range(num_class):
                    for i in range(num_data):
                        idx = k * num_data + i
                        grad[idx] *= weight[i]
                        hess[idx] *= weight[i]
        return grad, hess

class _EvalFunctionWrapper(lgb.sklearn._EvalFunctionWrapper):
    """Proxy class for evaluation function."""

    def __call__(self, preds, dataset):
        """Call passed function with appropriate arguments.

        Parameters
        ----------
        preds : array-like of shape = [n_samples] or shape = [n_samples * n_classes] (for multi-class task)
            The predicted values.
        dataset : Dataset
            The training dataset.

        Returns
        -------
        eval_name : str
            The name of evaluation function (without whitespace).
        eval_result : float
            The eval result.
        is_higher_better : bool
            Is eval result higher better, e.g. AUC is ``is_higher_better``.
        """
        labels = dataset.get_label()
        params = signature(self.func).parameters
        argc = len(params)

        extra_params = {}
        if "loc" in params:
            extra_params["loc"] = dataset.get_loc()
        if "log_scale" in params:
            extra_params["log_scale"] = dataset.get_log_scale()
        # print("extra_params ef wrapper", extra_params)
        argc = argc - len(extra_params)
        if argc == 2:
            return self.func(labels, preds, **extra_params)
        elif argc == 3:
            return self.func(labels, preds, dataset.get_weight(), **extra_params)
        elif argc == 4:
            return self.func(labels, preds, dataset.get_weight(), dataset.get_group(), **extra_params)
        else:
            raise TypeError(f"Self-defined eval function should have 2, 3 or 4 arguments, got {argc}")

class PassThroughLGBMRegressor(lgb.sklearn.LGBMRegressor):
    def fit(self, X, y,
            sample_weight=None, init_score=None, group=None,
            loc=None, log_scale=None,
            eval_set=None,
            eval_loc=None, eval_log_scale=None,
            eval_names=None, eval_sample_weight=None,
            eval_class_weight=None, eval_init_score=None, eval_group=None,
            eval_metric=None, early_stopping_rounds=None, verbose='warn',
            feature_name='auto', categorical_feature='auto',
            callbacks=None, init_model=None):
        """Docstring is set after definition, using a template."""
        if self._objective is None:
            if isinstance(self, lgb.sklearn.LGBMRegressor):
                self._objective = "regression"
            elif isinstance(self, lgb.sklearn.LGBMClassifier):
                self._objective = "binary"
            elif isinstance(self, lgb.sklearn.LGBMRanker):
                self._objective = "lambdarank"
            else:
                raise ValueError("Unknown LGBMModel type.")
        if callable(self._objective):
            self._fobj = _ObjectiveFunctionWrapper(self._objective)
        else:
            self._fobj = None

        params = self.get_params()
        # user can set verbose with kwargs, it has higher priority
        if self.silent != "warn":
            _log_warning("'silent' argument is deprecated and will be removed in a future release of LightGBM. "
                         "Pass 'verbose' parameter via keyword arguments instead.")
            silent = self.silent
        else:
            silent = True
        if not any(verbose_alias in params for verbose_alias in _ConfigAliases.get("verbosity")) and silent:
            params['verbose'] = -1
        params.pop('silent', None)
        params.pop('importance_type', None)
        params.pop('n_estimators', None)
        params.pop('class_weight', None)
        if isinstance(params['random_state'], np.random.RandomState):
            params['random_state'] = params['random_state'].randint(np.iinfo(np.int32).max)
        for alias in _ConfigAliases.get('objective'):
            params.pop(alias, None)
        if self._n_classes is not None and self._n_classes > 2:
            for alias in _ConfigAliases.get('num_class'):
                params.pop(alias, None)
            params['num_class'] = self._n_classes
        if hasattr(self, '_eval_at'):
            eval_at = self._eval_at
            for alias in _ConfigAliases.get('eval_at'):
                if alias in params:
                    _log_warning(f"Found '{alias}' in params. Will use it instead of 'eval_at' argument")
                    eval_at = params.pop(alias)
            params['eval_at'] = eval_at
        params['objective'] = self._objective
        if self._fobj:
            params['objective'] = 'None'  # objective = nullptr for unknown objective

        # Do not modify original args in fit function
        # Refer to https://github.com/microsoft/LightGBM/pull/2619
        eval_metric_list = copy.deepcopy(eval_metric)
        if not isinstance(eval_metric_list, list):
            eval_metric_list = [eval_metric_list]

        # Separate built-in from callable evaluation metrics
        eval_metrics_callable = [_EvalFunctionWrapper(f) for f in eval_metric_list if callable(f)]
        eval_metrics_builtin = [m for m in eval_metric_list if isinstance(m, str)]

        # register default metric for consistency with callable eval_metric case
        original_metric = self._objective if isinstance(self._objective, str) else None
        if original_metric is None:
            # try to deduce from class instance
            if isinstance(self, lgb.sklearn.LGBMRegressor):
                original_metric = "l2"
            elif isinstance(self, lgb.sklearn.LGBMClassifier):
                original_metric = "multi_logloss" if self._n_classes > 2 else "binary_logloss"
            elif isinstance(self, lgb.sklearn.LGBMRanker):
                original_metric = "ndcg"

        # overwrite default metric by explicitly set metric
        params = _choose_param_value("metric", params, original_metric)

        # concatenate metric from params (or default if not provided in params) and eval_metric
        params['metric'] = [params['metric']] if isinstance(params['metric'], (str, type(None))) else params['metric']
        params['metric'] = [e for e in eval_metrics_builtin if e not in params['metric']] + params['metric']
        params['metric'] = [metric for metric in params['metric'] if metric is not None]

        if not isinstance(X, (lgb.sklearn.pd_DataFrame, lgb.sklearn.dt_DataTable)):
            _X, _y = lgb.sklearn._LGBMCheckXY(X, y, accept_sparse=True, force_all_finite=False, ensure_min_samples=2)
            if sample_weight is not None:
                sample_weight = _LGBMCheckSampleWeight(sample_weight, _X)
        else:
            _X, _y = X, y

        if self._class_weight is None:
            self._class_weight = self.class_weight
        if self._class_weight is not None:
            class_sample_weight = lgb.sklearn._LGBMComputeSampleWeight(self._class_weight, y)
            if sample_weight is None or len(sample_weight) == 0:
                sample_weight = class_sample_weight
            else:
                sample_weight = np.multiply(sample_weight, class_sample_weight)

        self._n_features = _X.shape[1]
        # copy for consistency
        self._n_features_in = self._n_features

        def _construct_dataset(X, y, sample_weight, init_score, group, params,
                               categorical_feature='auto', loc=loc, log_scale=log_scale):
            return Dataset(X, label=y, weight=sample_weight, group=group,
                           init_score=init_score, params=params,
                           categorical_feature=categorical_feature, loc=loc, log_scale=log_scale)

        train_set = _construct_dataset(_X, _y, sample_weight, init_score, group, params,
                                       categorical_feature=categorical_feature, loc=loc, log_scale=log_scale)

        valid_sets = []
        if eval_set is not None:
            # raise NotImplementedError("eval_set is not supported in sklearn interface.")

            def _get_meta_data(collection, name, i):
                if collection is None:
                    return None
                elif isinstance(collection, list):
                    return collection[i] if len(collection) > i else None
                elif isinstance(collection, dict):
                    return collection.get(i, None)
                else:
                    raise TypeError(f"{name} should be dict or list")

            if isinstance(eval_set, tuple):
                eval_set = [eval_set]

            print("eval_set", eval_set)
            for i, valid_data in enumerate(eval_set):
                # reduce cost for prediction training data
                if valid_data[0] is X and valid_data[1] is y:
                    valid_set = train_set
                else:
                    valid_weight = _get_meta_data(eval_sample_weight, 'eval_sample_weight', i)
                    valid_class_weight = _get_meta_data(eval_class_weight, 'eval_class_weight', i)
                    if valid_class_weight is not None:
                        if isinstance(valid_class_weight, dict) and self._class_map is not None:
                            valid_class_weight = {self._class_map[k]: v for k, v in valid_class_weight.items()}
                        valid_class_sample_weight = _LGBMComputeSampleWeight(valid_class_weight, valid_data[1])
                        if valid_weight is None or len(valid_weight) == 0:
                            valid_weight = valid_class_sample_weight
                        else:
                            valid_weight = np.multiply(valid_weight, valid_class_sample_weight)
                    valid_init_score = _get_meta_data(eval_init_score, 'eval_init_score', i)
                    valid_group = _get_meta_data(eval_group, 'eval_group', i)
                    valid_set = _construct_dataset(valid_data[0], valid_data[1],
                                                   valid_weight, valid_init_score, valid_group, params, loc=eval_loc, log_scale=eval_log_scale)
                valid_sets.append(valid_set)

        if isinstance(init_model, lgb.sklearn.LGBMModel):
            init_model = init_model.booster_

        if early_stopping_rounds is not None and early_stopping_rounds > 0:
            _log_warning("'early_stopping_rounds' argument is deprecated and will be removed in a future release of LightGBM. "
                         "Pass 'early_stopping()' callback via 'callbacks' argument instead.")
            params['early_stopping_rounds'] = early_stopping_rounds

        if callbacks is None:
            callbacks = []
        else:
            callbacks = copy.copy(callbacks)  # don't use deepcopy here to allow non-serializable objects

        if verbose != 'warn':
            _log_warning("'verbose' argument is deprecated and will be removed in a future release of LightGBM. "
                         "Pass 'log_evaluation()' callback via 'callbacks' argument instead.")
        else:
            if callbacks:  # assume user has already specified log_evaluation callback
                verbose = False
            else:
                verbose = True
        callbacks.append(lgb.sklearn.log_evaluation(int(verbose)))

        evals_result = {}
        callbacks.append(lgb.sklearn.record_evaluation(evals_result))

        self._Booster = lgb.engine.train(
            params=params,
            train_set=train_set,
            num_boost_round=self.n_estimators,
            valid_sets=valid_sets,
            valid_names=eval_names,
            fobj=self._fobj,
            feval=eval_metrics_callable,
            init_model=init_model,
            feature_name=feature_name,
            callbacks=callbacks
        )

        if evals_result:
            self._evals_result = evals_result
        else:  # reset after previous call to fit()
            self._evals_result = None

        if self._Booster.best_iteration != 0:
            self._best_iteration = self._Booster.best_iteration
        else:  # reset after previous call to fit()
            self._best_iteration = None

        self._best_score = self._Booster.best_score

        self.fitted_ = True

        # free dataset
        self._Booster.free_dataset()
        del train_set, valid_sets
        return self

Thanks for using LightGBM.

My best guess is that there is some sorting by covariates happening at each boosting stage that doesn't apply to my hackily tacked on dataset attributes

I'm confident that LightGBM does not shuffle the data. Row i in the Dataset you create refers to the same observation as row i in the provided raw data.

something is going wrong during fitting

I don't totally understand what you're trying to do here... this is a very large amount of provided code, and it's not clear to me which parts I should be looking at or what "something is going wrong during fitting" means specifically. If you could provide a much smaller example of what you're trying to accomplish, it'd help.
But even without that, I think I can still provide some useful resources.
First, see this example of using a custom objective function in the Python package: https://github.com/microsoft/LightGBM/blob/fe69fa9f43f6fc051730e65008184658c991106e/examples/python-guide/advanced_example.py#L139-L147. That's a classification example, but the interface is identical... you provide a function which, at each iteration, is given two things:

preds = predictions (in terms of the objective function, without transformation) for the ensemble so far
train_data = the lightgbm.Dataset object you're training on

and which is responsible for computing gradients and hessians.
It sounds to me like you want to be able to access other data inside such a custom objective function which is not features in the Dataset. I can think of a few options for doing that:
# option 1: make objective a callable class, store that other data on the class
class CustomObjective:
    def __init__(self, time: np.ndarray):
        self.time = time

    def __call__(self, preds: np.ndarray, train_data: lightgbm.Dataset) -> Tuple[np.ndarray, np.ndarray]:
        # your code that calculates the gradient and hessian
        grad, hess = your_custom_code(preds, train_data, self.time)
        return grad, hess

# option 2: store arbitrary data on the `Dataset` object itself
dtrain = lgb.Dataset(data=X, label=y, ...)
dtrain.time = time

def custom_objective(preds: np.ndarray, train_data: lightgbm.Dataset) -> Tuple[np.ndarray, np.ndarray]:
        # your code that calculates the gradient and hessian
        grad, hess = your_custom_code(preds, train_data, train_data.time)
        return grad, hess

lgb.train(train_set=dtrain, params={"objective": custom_objective}, ...)

See the note on writing custom objective functions at https://lightgbm.readthedocs.io/en/latest/pythonapi/lightgbm.train.html#lightgbm.train.

Hi @jameslamb thanks for the quick response, and apologies for the verbose code blob and ambiguous question... I was on a flight managing my screaming 1.5 yr old at the time, and just wasn't thinking. Very kind of you to answer in spite of that.
Your response is definitely helpful. Seems like I'm generally on the right track.
By "something is going wrong during training," I meant that when I give it the ground truth location params and a well behaved conditional scale function, it produces very poor results.
Given your input and some further investigation, my guess is that it boils down to either the Dataset.construct method somehow wiping out the .loc attribute I attached, or something squirrelly going on with the eval function.
I think my next move is to just drop the sklearn wrapper, which is only complicating things. I'll post more here once I have it either solved or when I hit another block.
In case you're interested, the relevant chunks of the code above are
Redefinition of dataset where I set .loc and .log_scale

    def __init__(self, data, label=None, reference=None,
                 weight=None, group=None, init_score=None, silent='warn',
                 loc = None, log_scale=None,
                 feature_name='auto', categorical_feature='auto', params=None,
                 free_raw_data=True):
        ...
        self.loc = loc
        ...

This chunk of _CustomObjectiveFunctionWrapper (and something equivalent in eval function wrapper) where I pass the particular args I'm looking for on to the objective function as kwargs.
      params = signature(self.func).parameters
      argc = len(params)
      extra_params = {}
      if "loc" in params:
          extra_params["loc"] = dataset.get_loc()
      if "log_scale" in params:
          extra_params["log_scale"] = dataset.get_log_scale()
      # print("extra_params of wrapper", extra_params)
      argc = argc - len(extra_params)
      if argc == 2:
          grad, hess = self.func(labels, preds, **extra_params)
      elif argc == 3:
          grad, hess = self.func(labels, preds, dataset.get_group(), **extra_params)
      else:
          raise TypeError(f"Self-defined objective function should have 2 or 3 arguments, got {argc}")

And finally the redefinition of _construct_dataset where I include the loc and log_scale args.
def _construct_dataset(X, y, sample_weight, init_score, group, params,
                               categorical_feature='auto', loc=loc, log_scale=log_scale):
            return Dataset(X, label=y, weight=sample_weight, group=group,
                           init_score=init_score, params=params,
                           categorical_feature=categorical_feature, loc=loc, log_scale=log_scale)

Ha no problem at all! I'm happy to help... although I personally don't know anything much about poisson process regression, so I can only help in the "here's how LightGBM works" sense.
I definitely recommend trying to make this work using the lightgbm.train() interface (instead of the scikit-learn estimators) first, as you mentioned. That'll help you narrow things down a lot, and should make it possible to implement what you want using one of the approaches I mentioned with 0 changes to lightgbm's internals... since a custom objective function passed to lightgbm.train() will be given the Dataset object directly (instead of the extracted predictions, labels, and weights as numpy arrays).

I'm going to close this as there hasn't been a new comment in the last year. Come back any time if you need more help or have other questions 👋🏻


(wget) uninstalls with false error but finishes correctly

https://chocolatey.org/packages/Wget

C:\ProgramData\chocolatey\lib>cinst wget
Chocolatey v0.10.11
Installing the following packages:
wget
By installing you accept licenses for the packages.
Progress: Downloading Wget 1.19.4... 100%
Wget v1.19.4 [Approved]
wget package files install completed. Performing other installation steps.
Getting x64 bit zip
Extracting C:\ProgramData\chocolatey\lib\Wget\tools\wget-1.19.4-win64_x64.zip to C:\ProgramData\chocolatey\lib\Wget\tools...
C:\ProgramData\chocolatey\lib\Wget\tools
ShimGen has successfully created a shim for wget.exe
The install of wget was successful.
Software installed to 'C:\ProgramData\chocolatey\lib\Wget\tools'
Chocolatey installed 1/1 packages.
See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
C:\ProgramData\chocolatey\lib>choco uninstall wget
Chocolatey v0.10.11
Uninstalling the following packages:
wget
wget v1.19.4
Skipping auto uninstaller - No registry snapshot.
wget has been successfully uninstalled.
wget not uninstalled. An error occurred during uninstall:
Unable to find package 'wget'.
Chocolatey uninstalled 0/1 packages. 1 packages failed.
See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
Failures

wget (exited 1) - wget not uninstalled. An error occurred during uninstall:
Unable to find package 'wget'.

If a package uninstall is failing and/or you've already uninstalled the
software outside of Chocolatey, you can attempt to run the command
with -n to skip running a chocolateyUninstall script, additionally
adding --skip-autouninstaller to skip an attempt to automatically
remove system-installed software. This will only remove the packaging
files and not things like software installed to Programs and Features.
If a package is failing because it is a dependency of another package
or packages, then you may first need to consider if it needs removed
as it is typically installed as a dependency for a reason. If you
decide that you still want to remove it, head into
$env:ChocolateyInstall\lib and find the package folder you want
removed. Then delete the folder for the package. This option should
only be used as a last resort.

It DOES uninstall though.
I've looked through the package files and they appear to be fine. I BELIEVE this problem actually lies in the chocolatey.org database with the name and needs to be changed from "Wget" to "wget" to be resolved. Seems more aprapos to open issue here instead of at chocolatey.org.
Expected Behavior
Not to see...

wget not uninstalled. An error occurred during uninstall:
Unable to find package 'wget'.
Chocolatey uninstalled 0/1 packages. 1 packages failed.
See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
Failures

wget (exited 1) - wget not uninstalled. An error occurred during uninstall:
Unable to find package 'wget'.

If a package uninstall is failing and/or you've already uninstalled the
software outside of Chocolatey, you can attempt to run the command
with -n to skip running a chocolateyUninstall script, additionally
adding --skip-autouninstaller to skip an attempt to automatically
remove system-installed software. This will only remove the packaging
files and not things like software installed to Programs and Features.
If a package is failing because it is a dependency of another package
or packages, then you may first need to consider if it needs removed
as it is typically installed as a dependency for a reason. If you
decide that you still want to remove it, head into
$env:ChocolateyInstall\lib and find the package folder you want
removed. Then delete the folder for the package. This option should
only be used as a last resort.

... after " wget has been successfully uninstalled."
Current Behavior
s/a
Possible Solution
s/a
Steps to Reproduce (for bugs)
Install and then uninstall the wget package.
Context
Your Environment

Package Version used:  v1.19.4 but this is a long standing problem.
Operating System and version: Win 10
Chocolatey version: Chocolatey v0.10.11
Install/uninstall gist: n/a

It looks like upstream bug, the package doesn't have uninstaller.  Also, everything looks OK. Here is the last part of verbose debug:

[NuGet] Successfully uninstalled 'wget 1.19.4'.
 wget has been successfully uninstalled.
Removing nupkg if it still exists.
Ensuring removal of installation files.
 Found 'C:\ProgramData\chocolatey\lib\wget\tools\COPYING'
  with checksum 'C678957B0C8E964AA6C70FD77641A71E'
Attempting to delete file "C:\ProgramData\chocolatey\lib\wget\tools\COPYING".
 Found 'C:\ProgramData\chocolatey\lib\wget\tools\wget.exe'
  with checksum 'D9EAB04C36651BC9E00133DADE688924'
Attempting to delete file "C:\ProgramData\chocolatey\lib\wget\tools\wget.exe".
 Found 'C:\ProgramData\chocolatey\lib\wget\tools\wget.exe.gdb'
  with checksum '2A1BB9D4026B84C722589BD2B87F3B20'
Attempting to delete file "C:\ProgramData\chocolatey\lib\wget\tools\wget.exe.gdb".
 Found 'C:\ProgramData\chocolatey\lib\wget\tools\wget.html'
  with checksum '7FACD5C6E85F20562A9F015E216EEE46'
Attempting to delete file "C:\ProgramData\chocolatey\lib\wget\tools\wget.html".
Attempting to delete directory "C:\ProgramData\chocolatey\lib\wget".
Ensuring removal of package cache files.
wget not uninstalled. An error occurred during uninstall:
 Unable to find package 'wget'.

/cc @gep13

To my knowledge, ALL packages that have an uppercase letters in them "fail" uninstall.
i.e. The problem is the \lib\Wget folder.

ALL packages that have an uppercase letters in them "fail" uninstall.

Didn't know about this.  If this is the case, even more upstream issue of even bigger importance.

Seems more aprapos to open issue here instead of at chocolatey.org.

Issue is for choco.exe. I don't think we can do anything here except renaming package but there is some reason I forgot why we don't that. /cc @AdmiringWorm

To my knowledge, ALL packages that have an uppercase letters in them "fail" uninstall.

Only packages that originally had an uppercase, and have since been pushed with a lowercase would fail to uninstall.

Issue is for choco.exe.

Indeed it is, and an issue for it already exist: https://github.com/chocolatey/choco/issues/1040

I don't think we can do anything here except renaming package but there is some reason I forgot why we don't that.

No need to rename, we just need to return the original casing of the id through PackageName in the update.ps1 file.
I've updated the update.ps1 file to return the original casing of the wget package, which should resolve this issue the next time wget is updated. (I didn't really bother adding a fix version for it, as the package should still uninstall even though it reports an error).


Update example app to use two activities

Using singleTask in an Android application for the activity that has an intent-filter on the LAUNCHER, means that any activities running on top of this activity will disappear when the app's icon is pressed again from the launcher.
This is a problem, because when confirming a SetupIntent or PaymentIntent, Stripe may need to present a web browser (its own activity) for verification with a customer's bank or similar, and the user may even need to leave this web browser activity temporarily to navigate to their bank's own application. If the user is not careful when navigating back to the original app, and taps the app's icon from the launcher, the app will open to its original activity, and the web browser is gone. Instead, the user will need to navigate back only using Android's app switcher, which will correctly bring you back into the open web browser activity.
The change in this commit side-steps this issue by splitting the example application into two acitivites. One that is only responsible for launching the application, and a second activity that actually represents the react native application, and will be started by the first activity, ensuring to only start it once.
This fixes #355.
Testing

[ ] I tested this manually
[ ] I added automated tests

Documentation
Select one:

[ ] I have added relevant documentation for my changes.
[x] This PR does not result in any developer-facing changes.

I based this on the fix originally described here:
https://github.com/proyecto26/react-native-inappbrowser/issues/213#issuecomment-878915862
It may be that the code can be further simplified, by not even maintaining a list, since in my experiments, I've never seen that it's possible for the activity to be started more than once (which makes sense, since it has launchMode 'singleTask). But for now I kept it like this, since it's equivalent to the code we ended up deploying to production.

what about expo apps using managed workflow?

what about expo apps using managed workflow?

I'm not sure, and not really that familiar with android activites except what I've been forced to try to understand while debugging and fixing the issue in our react native app.
I would assume that the issue is something expo would need to support in their native code – but unsure if they could do it without risking breaking some other use case.
I still think there's not much Stripe could do about it without abandoning the use of chrome custom tabs altogether though (and that's not really an option for 3d secure flows either).


B/viet nam

Re-tried this PR, but this time on a new branch: https://github.com/namely/countries/pull/7

+1

+1


implement zero copy for performance gains

In Kafka upstream zero copy https://www.ibm.com/developerworks/linux/library/j-zerocopy/ is used for efficiency gains https://kafka.apache.org/documentation.html#maximizingefficiency
The c librdkafka does this too.
Looking quick Go does have this ability too (I think this is it https://golang.org/src/net/sendfile_linux.go) and I believe Sarama would greatly benefit from a huge performance bump with it in place.
I have been running some tests on an 8 core machines and getting ~ 380,000 message / sec (on a kafka cluster with default settings no compression used so I can definitely tune and squeak out more) so implementing zero copy should have a really nice bump here I believe.

@eapache, @joestein  Is there a way out here? I would like to work on this if someone had an idea in mind that we could try implementing/testing.

AFAIK this is not possible until upstream implements https://github.com/golang/go/issues/9402

Go upstream does not seem interested in doing this, and we haven't received any complaints that the current performance is actually insufficient in regular cases.


1.20.0版本代码增加了import yaml，却没有更新依赖列表

Checklist

[x]  I have searched the tutorial on modelscope  doc-site
[x]  I have searched related issues but cannot get the expected help.
[x]  The bug has not been fixed in the latest version.

Describe the bug
安装modelscope之后直接使用snapshot_download时提示缺少依赖
Traceback (most recent call last):
  File "/Users/myhloli/PycharmProjects/pythonProject/.venv/lib/python3.10/site-packages/modelscope/utils/import_utils.py", line 451, in _get_module
    return importlib.import_module('.' + module_name, self.__name__)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/myhloli/PycharmProjects/pythonProject/.venv/lib/python3.10/site-packages/modelscope/fileio/io.py", line 8, in <module>
    from .format import JsonHandler, YamlHandler
  File "/Users/myhloli/PycharmProjects/pythonProject/.venv/lib/python3.10/site-packages/modelscope/fileio/format/__init__.py", line 5, in <module>
    from .yaml import YamlHandler
  File "/Users/myhloli/PycharmProjects/pythonProject/.venv/lib/python3.10/site-packages/modelscope/fileio/format/yaml.py", line 2, in <module>
    import yaml
ModuleNotFoundError: No module named 'yaml'

To Reproduce

What command or script did you run?

pip install modelscope
python3
from modelscope import snapshot_download

Your Environments (required)

OS: Darwin Mac-mini.local 22.6.0 Darwin Kernel Version 22.6.0: Mon Apr 22 20:54:28 PDT 2024; root:xnu-87<IP_ADDRESS>5.2~1/RELEASE_X86_64 x86_64
CPU: 2 GHz 四核Intel Core i7
Commit id (master)

Model hub related: @liuyhwangyh @tastelikefeet @wangxingjun778

thanks, we are looking into this issue.


{CI} reassign issue owner

GH issues related to az account were all assigned to XingZhou.
Now we want to assign commands within Commands to XingZhou, but commands within Subgroup to Jiashuo.
If there is a new command not one of those 8 existing commands, temporarily assign to Jiashuo.

Related command

Description

Testing Guide

History Notes

[Component Name 1] BREAKING CHANGE: az command a: Make some customer-facing breaking change
[Component Name 2] az command b: Add some customer-facing feature

This checklist is used to make sure that common guidelines for a pull request are followed.

[ ] The PR title and description has followed the guideline in Submitting Pull Requests.

[ ] I adhere to the Command Guidelines.

[ ] I adhere to the Error Handling Guidelines.

CI


Changing state does not change label font and color, only background

Report
Report a bug
What did you do?
Set up new project, added Segmentio pod, used code from Example with .onlyLabel and custom colors:
    let font = UIFont.systemFont(ofSize: 14)
    return SegmentioStates(
        defaultState: segmentioState(
            backgroundColor: .clear,
            titleFont: font,
            titleTextColor: .lightGray
        ),
        selectedState: segmentioState(
            backgroundColor: .cyan,
            titleFont: font,
            titleTextColor: .blue
        ),
        highlightedState: segmentioState(
            backgroundColor: .lightGray,
            titleFont: font,
            titleTextColor: .red
        )
    )

Example does not work properly either.
What did you expect to happen?
When switching state the selected item should change background color, text color and font.
What happened instead?
Switching state only changes background color. Indicator works correctly.
Your Environment

Version of the component: 2.1 from Cocoapods
Swift version: 3.0
iOS version: 10.1.1 / 10.1
Device: iPhone 6 / iOS Simulator
Xcode version: 8.1

Stack
   CocoaPods : 1.2.0.beta.1
        Ruby : ruby 2.0.0p648 (2015-12-16 revision 53162) [universal.x86_64-darwin16]
    RubyGems : <IP_ADDRESS>
        Host : Mac OS X 10.12.1 (16B2657)
       Xcode : 8.1 (8B62)
         Git : git version 2.9.3 (Apple Git-75)
Ruby lib dir : /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/lib
Repositories : master - https://github.com/CocoaPods/Specs.git @ 5f7c2c8d410e7a0cb8af5b4c1c9cf368f18ee6d8

Installation Source
Executable Path: /usr/local/bin/pod

Plugins
cocoapods-deintegrate : 1.0.1
cocoapods-plugins     : 1.0.0
cocoapods-search      : 1.0.0
cocoapods-stats       : 1.0.0
cocoapods-trunk       : 1.1.1
cocoapods-try         : 1.1.0

Podfile
platform :ios, '9.0'

target 'SegmentioTest' do
  use_frameworks!

  pod 'Segmentio', '~> 2.1'

end

My first guess was iOS10 cell prefetching in collectionView but then it should work in 9.3, yet it doesn't. Maybe some issues with text label rendering? I don't really know 😢
Thanks for this great library!

I have reproduced the issue... I´m sure that in in the previous version it was working. Please we need a hotfix :)


Add integration test for revocation endpoint to verify auth failure errors with revoked access tokens

In this PR, a new integration test is added to test the following scenario.

Call revocation request with a revoked access token but with invalid auth credentials - the test should verify the 401 status code.

Git Issue: https://github.com/wso2/product-is/issues/12510

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/1256311121

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/1256311121
Status: failure

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/1257530163

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/1257530163
Status: failure

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/1260220444

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/1260281962

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/1260220444
Status: failure

PR builder completed
Link: https://github.com/wso2/product-is/actions/runs/1260281962
Status: failure

PR builder started
Link: https://github.com/wso2/product-is/actions/runs/1260533563


WIP - openstack: add periodic job for STF 1.3

This job will deploy OpenShift on top of OpenStack, then instead of
running e2e test suite, it'll deploy Service Telemetry Framework 1.3
(STF) on top of the OCP cluster.

ehr... wrong base repository?

ehr... wrong base repository?

Hard to say, what is the base repository?
I'm closing this anyways, because we don't have a tracking task for this or anyone assigned to it. This is nearly a year old now.


Core 2422

DONT MERGE please.  This is just here to make browsing the changes easier.
Most of the changes are NOT to the main spring-authorization-service, and even those superficial changes (e.g. private to public, and/or one extra method added, etc.) can probably be avoided with a little deeper analysis of the way that the spring-security library DSL works in carrying out configuration.   Some hints to that might be found in https://github.com/spring-projects/spring-authorization-server/compare/main...sjohnr:identity-federation which also contains an alternative approach

@bob-walters, thanks for your interest in the project!
Am I correct in assuming that you are using the GitHub UI to view a set of changes you're working through for your own project? If so, you can actually create a PR from your fork's feature branch to your fork's main branch here. I'm going to close this PR as it doesn't look like it's intended for the main project. Let me know if I'm missing anything.


Reorganize code in contive plugin

#44

Coverage decreased (-0.2%) to 47.114% when pulling dd7882a3e6da8681b032ae98122b064e1c59cbb2 on lukasmacko:l3networking into 84b36d5360c9bcd34f4e677d10985572b7bfadc8 on contiv:master.


[6.x] Fix belongsTo relation when using multiple database connections

The problem
Could previously cause errors where the owner belonged to another database and the child to the default one.
Fix
This PR tries to fix that by using the correct relational connection.
Feel free to provide feedback or tweak whenever needed.
Possibly related to https://github.com/laravel/framework/issues/29935.
This pull request was built together with @jordyvanderhaegen.
Update
I see the tests fail on the use of config.
Two possible options:

Removing the setConnection, as getConnection will fall back to the right one when a model has no custom connection. Might cause elsewhere though
Get the config another way.

Anyone who could point me in the right direction would be a great help!

We can't call a global config helper in the middle of an Eloquent method.

The issue still persists, so I've tried different solutions, but got stuck with testing.
If anyone would like to give me a hand fixing this issue, feel free!


RFC: improve docker socket error handling

From issue #207. We can probably better handle a misconfigured docker daemon by
blowing up and telling the user that it's not possible to contact the docker
daemon. This is more friendly than the actual behavior, which is 3 nested
exceptions taking place...

Thanks so much! I'd like to check some code style on this before merging.

thanks @nishakm ! I added some changes to address your review. Hope this works :)


MP2 files won't render or play

Wavesurfer.js version(s):
5.2.0
Browser and operating system version(s):
Chrome, Firefox, Windows 10, Ubuntu 20.04
Code needed to reproduce the issue:
https://codepen.io/guilhermeb-ciandt/pen/bGYNdoO
Use behaviour needed to reproduce the issue:
Get a MP3 file and convert to MP2, wavesurfer won't be able to reproduce it. On the codepen example I got the MP3 file used on wavesurfer docs example and converted to MP2 using audacity(I also tested using ffmpeg) and that MP2 won't load.
If you want to test from another audio from scratch that's the command I use with ffmpeg:
ffmpeg -y -v error -i {file-path} -f WAV -acodec mp2 -ar 44100 {output_path}

Any error messages?

No error messages

that MP2 won't load.

App just hangs and waits forever? Not a single error message?

You can add
wavesurfer.on('error', function (err) {
  console.warn("error", err?.message || err);
});

to see the error message. I'm not sure any browser can natively decode this type of file (mpeg layer 2).


Bai22

Refactor code class Login and Booking to Modal Object

Hi @trongtuyen271199
bai tap 22 ban lam kha tot, nhung co 1 luu y sau

// khai báo Biến String chỗ nào mới hợp lý ???
ban nen khai bao bien o dau test script de de nhin hon
https://share.zight.com/llueRn4P
mot so chuc nang nen su dung POM de maintain code va tai su dung duoc nhu: verify logout btn is displayed, verify add to cart successfully,...
https://share.zight.com/KouROlQk

Best wishes from the Automation Test Pro team (Son Le<EMAIL_ADDRESS>

Hi @trongtuyen271199
Your homework has been reviewed.
 Keep up the good work!
https://share.zight.com/d5uQB6Wk
https://share.zight.com/Z4uElrLo
Best wishes from the Automation Test Pro team (Son Le<EMAIL_ADDRESS>


Device becomes unresponsive and spams errors in the log

After turning off the device, I get error spam in the log and the device doesn't turn on anymore. Here is the cascade of errors that occurs
[5/11/2021, 8:24:41 PM] [midea-air] [updateValues] Error sending the command: undefined. Trying to re-login before re-issuing command...
[5/11/2021, 8:24:41 PM] [midea-air] [MideaPlatform.ts] sendCommand (Intent: [updateValues] (fetch params?) attempt 2/2) returned error invalidSession
[5/11/2021, 8:24:41 PM] [midea-air] [updateValues] sendCommand command still failed after retrying: undefined
[5/11/2021, 8:24:42 PM] [midea-air] [MideaPlatform.ts] sendCommand (Intent: [updateValues] (fetch params?) attempt 1/2) returned error invalidSession
[5/11/2021, 8:24:42 PM] [midea-air] [updateValues] Error sending the command: undefined. Trying to re-login before re-issuing command...

Sometimes you'll see this message. This plugin is still in development.


Issue/2546 missing thing

Resolves
What Github issue does this resolve (please include link)?
#2546
Proposed Changes
Describe what this Pull Request does
localizes the 'thing' default text for the item in list blocks.
Reason for Changes
Explain why these changes should be made
Localization
Test Coverage
Please show how you have added tests to cover your changes

[ ] Switch to another language, then add a list. 'Thing' should be translated.
Note - translations won't be available until translators have translated, and translations have been updated.

Switching language after adding a list is already being tracked: https://github.com/LLK/scratch-gui/issues/2656

It works in French and a few other languages, but several others don't have translations yet.  Looks good.


drt: Investigate/tune congestion based worker sizing

Describe the bug
drt has code to detect congestion, and increase the worker size (default 7x7). I'm looking for designs that exercise this path so that we can decide if the algorithm is working as expected.
So far I've only found sky130hd/Microwatt enables congestion adjustment, and as a result it increases the worker size to be quite large (25x25):
[INFO DRT-0195] Start 0th optimization iteration.
args.size 7
[INFO DRT-0195] Start 1st optimization iteration.
args.size 7
[INFO DRT-0195] Start 2nd optimization iteration.
args.size 7
[INFO DRT-0195] Start 3rd optimization iteration.
args.size 7
[INFO DRT-0195] Start 4th optimization iteration.
args.size 7
[INFO DRT-0195] Start 5th optimization iteration.
args.size 7
[INFO DRT-0195] Start 6th optimization iteration.
args.size 7
[INFO DRT-0195] Start 7th optimization iteration.
args.size 9
[INFO DRT-0195] Start 8th optimization iteration.
args.size 11
[INFO DRT-0195] Start 9th optimization iteration.
args.size 13
[INFO DRT-0195] Start 10th optimization iteration.
args.size 15
[INFO DRT-0195] Start 11th optimization iteration.
args.size 15
[INFO DRT-0195] Start 12th optimization iteration.
args.size 17
[INFO DRT-0195] Start 13th optimization iteration.
args.size 19
[INFO DRT-0195] Start 14th optimization iteration.
args.size 21
[INFO DRT-0195] Start 15th optimization iteration.
args.size 23
[INFO DRT-0195] Start 16th optimization iteration.
args.size 7
[INFO DRT-0195] Start 17th optimization iteration.
args.size 25
[INFO DRT-0195] Start 18th optimization iteration.
[INFO DRT-0267] cpu time = 02:15:26, elapsed time = 00:10:02, memory = 14396.67 (MB), peak = 14702.38 (MB)

If I disable the congestion detection code, it completes in less iterations and in a shorter time (9 minutes instead of 10 minutes):
[INFO DRT-0195] Start 0th optimization iteration.
args.size 7
[INFO DRT-0195] Start 1st optimization iteration.
args.size 7
[INFO DRT-0195] Start 2nd optimization iteration.
args.size 7
[INFO DRT-0195] Start 3rd optimization iteration.
args.size 7
[INFO DRT-0195] Start 4th optimization iteration.
args.size 7
[INFO DRT-0195] Start 5th optimization iteration.
args.size 7
[INFO DRT-0195] Start 6th optimization iteration.
args.size 7
[INFO DRT-0195] Start 7th optimization iteration.
args.size 7
[INFO DRT-0195] Start 8th optimization iteration.
args.size 7
[INFO DRT-0195] Start 9th optimization iteration.
args.size 7
[INFO DRT-0195] Start 10th optimization iteration.
args.size 7
[INFO DRT-0195] Start 11th optimization iteration.
args.size 7
[INFO DRT-0195] Start 12th optimization iteration.
args.size 7
[INFO DRT-0195] Start 13th optimization iteration.
[INFO DRT-0267] cpu time = 02:12:57, elapsed time = 00:09:03, memory = 14398.48 (MB), peak = 14641.39 (MB)

Expected Behavior

Have a design example in the CI that both exercises the congestion code and shows better results with it.
Reduce the negative impact (in terms of execution time) on the sky130hd Microwatt design

Environment
Git commit: 81706aef55de40a6f2eccadf25c20f418a554ab6
os: Fedora Linux 37 (Workstation Edition)
cmake version 3.25.1
gcc (GCC) 12.2.1 20221121 (Red Hat 12.2.1-4)
clang version 15.0.6 (Fedora 15.0.6-2.fc37)

To Reproduce
Run ORFS sky130hd/microwatt
Relevant log output
No response
Screenshots
No response
Additional Context
No response

Some old notes from Stephano who implemented this:

Fixes https://github.com/The-OpenROAD-Project/OpenROAD/issues/1619

I tried the test case in #1619 and OpenROAD can now successfully route it without drcs with the congestion code disabled. It also does it with less iterations and less overall time compared to when the congestion code is enabled.
Another thing I noticed is the worker size doesn't peak as high as in Microwatt. I wonder if we should put an upper bound on it (so it doesn't get out of hand like it seems to with Microwatt):
args.size 7
args.size 7
args.size 7
args.size 7
args.size 7
args.size 7
args.size 7
args.size 7
args.size 9
args.size 9
args.size 9
args.size 8
args.size 8
args.size 8
args.size 10
args.size 10
args.size 10
args.size 7
args.size 12
args.size 11
args.size 11
args.size 11
args.size 11
args.size 11
args.size 10
args.size 5
args.size 12
args.size 12
args.size 12
args.size 12
args.size 12
args.size 11
args.size 11
args.size 3
args.size 11
args.size 11
args.size 11
args.size 10
args.size 10
args.size 10
args.size 10
args.size 3
args.size 10
args.size 9
args.size 9
args.size 9
args.size 9

I'll look at the other test cases next.

If we no longer need it that would be great.  Thanks for digging into it.

ispd18_test3 still fails without congestion detection. Looking at where drt fails with it disabled, it's an area with only 3 available layers of metal above a macro and a lot of wires passing over it:

We should be able to build a test case similar to this.

I wasn't able to recreate this with Nangate45 or sky130. It was however easy to replicate this using the ispd18 tech lef.
The test case is pretty simple. I have a blockage on all of Metal1-6 and 14 Metal7 pins on the left reverse connected to 14 pins on the right. In this case the congestion code doesn't detect congestion, and we finish drt with 3 drcs:

What stands out is how large the vias are. If I reduce the size of the vias, drt completes in the 0th iteration without issue:

Do we have an issue in drt related to vias?
Test case: ispd18_test3_cutdown.tar.gz

The via selection code certain could be improved and this look like a good example.   On some private pdks we had to manually override the via selection.

@antonblanchard I've run into the same via selection issues on the private PDK (should be familiar to you) Matt's referring to.
In more advanced nodes the number of default vias in the tLEF rises substantially, and the via selection code in DRT just does primitive sort based on a few characteristics.
In the PDK we're talking about it results in DRC errors, because it chooses a via that's only legal in the context of a power grid. A proper enhancement of the code would probably check if the via is legal in routing context, and then try to choose the via with the least restrictions at the smallest size.

I should also point out that the via selection code is only run once per detailed route execution so computation cost isn't really an issue.

Its also not design specific so you can run any test case for a particular PDK

The via selection code certain could be improved and this look like a good example. On some private pdks we had to manually override the via selection.

For the ispd18 test there is only one via for these levels of metal, and in many cases it isn't even the correct orientation so we have to create one by flipping the dimensions. Seems a bit odd.
In my testing above I manually reduced the size of the via to test the theory that via size is contributing to the drcs. Is it valid to create a new (smaller) via using the other design rules in the tlef?

If it's DRC legal it's legal.

@antonblanchard Do you have the ability to detail route that test case with the IBM tools?

They might have been making it harder to route for the contest.  Equally it could just have been an oversight.

From https://www.ispd.cc/contests/18/index.html

I don't think we should invent new vias.  This just appears to be part of the challenge of this design is the weird vias provided.  There is nothing for via selection to do here.

This seems like contest foder, but perhaps an optimization around high density pins makes sense.
It feels like the router is getting stuck because the pins are so close together, and the nets are coming from so far away. Perhaps for cases like these we should insert virtual intermediate points to reduce the search space between this high area of congestion, and the far away nets.

A few more puzzling things about the ispd18_test3:

The track layout in the def doesn't match the tlef:

LAYER Metal7
  TYPE ROUTING ;
  DIRECTION HORIZONTAL ;
  PITCH 0.2 0.2 ;
  WIDTH 0.07 ;
  AREA 0.02 ;
  SPACINGTABLE
    PARALLELRUNLENGTH 0
    WIDTH 0     0.07
    WIDTH 0.1   0.15
    WIDTH 0.75  0.25
    WIDTH 1.5   0.45 ;
  SPACING 0.1 ENDOFLINE 0.1 WITHIN 0.035 ;
END Metal7

PITCH is 0.2 x 0.2 which with DATABASE MICRONS = 2000 means the tracks should be spaced 400 x 400. The tracks in the preferred direction are actually spaced 570:
TRACKS X 200 DO 4943 STEP 400 LAYER Metal7 ;
TRACKS Y 760 DO 2473 STEP 570 LAYER Metal7 ;

That's only an issue for my cut down test case, since I am generating a different track layout to the original test.

The vias on Metal7-Metal9 obstruct 3 tracks, eg:

It's even worse on Metal8 because it uses the same dimensions as Metal7 instead of swapping them (so the preferred direction track pitch is 400 units).
Having scanned through drt, I wonder if this is an issue. It seems like we assume a via will only obstruct one track. There is code for NDR vias where we mark multiple tracks as being obstructed but it isn't being used. This isn't an NDR via, but it seems to have the behaviour of an NDR via doesn't it?

Do the guides in the contest reflect this via obstruction ?

Where are we making the via == 1 track assumption?

Where are we making the via == 1 track assumption?

I've spent some time looking through drt, and I was wrong. It does correctly iterate across all tracks in modMinSpacingCostVia.
Not directly related to this issue, but I notice modMinSpacingCostVia only applies min area constraints to vias that aren't fat (ie not wider than the default wire):
  if (!isFatVia) {
    auto minAreaConstraint = getTech()->getLayer(lNum)->getAreaConstraint();
    auto minArea = minAreaConstraint ? minAreaConstraint->getMinArea() : 0;
    patchLength = frCoord(ceil(1.0 * minArea / defaultWidth
                               / getTech()->getManufacturingGrid()))
                  * frCoord(getTech()->getManufacturingGrid());
    length2_mar = max(length2_mar, patchLength);
  }

Is that a reasonable assumption?

It seems reasonable though I've never checked it.  Usually those vias are pretty large


feat: Implementation of redirect option for fetch()

Issue # (if available)
#340
Description of changes

Minimal improvements have been made to allow for redirect processing.(reference information)

Checklist

[x] Created unit tests in tests/unit and/or in Rust for my feature if needed
[x] Ran make fix to format JS and apply Clippy auto fixes
[x] Made sure my code didn't add any additional warnings: make check
[ ] Updated documentation if needed (API.md/README.md/Other)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

We will continue to address the following points:

return network error response.
Replace only the url when build_request() is called multiple times.

We have revised our policy a bit.

Return network error response.

Since it has already been pointed out in the review that simply returning an Error is sufficient, we are not going to do this in the immediate future.
If necessary, we would like to implement a method to generate and return a Network error response by calling something like Respose::error(ctx).

Replace only the url when build_request() is called multiple times.

Reading through the HTTP-redirect fetch, we see that we need to control multiple request parameters, not just the url, depending on the method and status code.
To control this, we would like to keep the build_request() to generate the next request parameter.
After this, commit additional modifications.(I wish I could have implemented it a little better...)

Thanks again for the PR. I think it LGTM. 🚀

We have revised our policy a bit.

Return network error response.

Since it has already been pointed out in the review that simply returning an Error is sufficient, we are not going to do this in the immediate future. If necessary, we would like to implement a method to generate and return a Network error response by calling something like Response::error(ctx).

What value would this bring?

Replace only the url when build_request() is called multiple times.

Reading through the HTTP-redirect fetch, we see that we need to control multiple request parameters, not just the url, depending on the method and status code. To control this, we would like to keep the build_request() to generate the next request parameter. After this, commit additional modifications.

You can ignore the “cors” logic as it’s mainly relevant to browsers and is not planned to be supported.

Add test code

I am wondering if it is possible to do a fetch() redirect test without accessing the actual site. Any good ideas? Note that the minimum test for the pattern necessary to make a decision was implemented in rust.

Yes, runtime_client.rs has tests using wiremock:
https://github.com/awslabs/llrt/blob/bfaa41ecd20df9de971aa0358571e30c2a46ae3f/src/runtime_client.rs#L500

Thanks again for the PR. I think it LGTM. 🚀

We have revised our policy a bit.

Return network error response.

Since it has already been pointed out in the review that simply returning an Error is sufficient, we are not going to do this in the immediate future. If necessary, we would like to implement a method to generate and return a Network error response by calling something like Response::error(ctx).

What value would this bring?

Yes, I have not found a use case for llrt at this time. So I do not think that implementation is necessary.

Replace only the url when build_request() is called multiple times.

Reading through the HTTP-redirect fetch, we see that we need to control multiple request parameters, not just the url, depending on the method and status code. To control this, we would like to keep the build_request() to generate the next request parameter. After this, commit additional modifications.

You can ignore the “cors” logic as it’s mainly relevant to browsers and is not planned to be supported.

Understood. So, would it be safe to assume that we have roughly completed the implementation?
Please point out any specifications that are of concern.

Add test code

I am wondering if it is possible to do a fetch() redirect test without accessing the actual site. Any good ideas? Note that the minimum test for the pattern necessary to make a decision was implemented in rust.

Yes, runtime_client.rs has tests using wiremock:
https://github.com/awslabs/llrt/blob/bfaa41ecd20df9de971aa0358571e30c2a46ae3f/src/runtime_client.rs#L500

Thank you for sharing your excellent samples!
This is the first implementation I've seen, so it may take some time, but I'll get to work on creating a mock server as soon as possible :)

I am creating a test with a mock server as soon as possible, but the fetch() call part is not working properly.
I am thinking of using fetch() defined in the globals of vm. Is the procedure correct?
let vm = Vm::new().await.unwrap();

async_with!(vm.ctx => |ctx| {
    async move {
        let result = ctx.globals().fetch(format!("http://{}/expect/200/", mock_server.address()), None).await;
        assert!(result.is_ok());
    }
})
.await;

vm.runtime.idle().await;

error[E0599]: no method named `fetch` found for struct `rquickjs::Object<'_>` in the current scope
   --> src/http/fetch.rs:490:44
    |
490 |                 let result = ctx.globals().fetch(format!("http://{}/expect/200/", mock_server.address()), None).await;
    |                                            ^^^^^ method not found in `Object<'_>`

 ctx.globals().fetch(

You would have to access the global object like this:
let fetch = ctx.globals().get("fetch")?;
let res: Promise<Value> = fetch(....)?;
let response = res.await?;

My apologies. I have tried everything, but I cannot resolve the error that the type of fetch is not specified. What is the type of fetch in this case?
let vm = Vm::new().await.unwrap();

async_with!(vm.ctx => |ctx| {
    async move {
        let fetch = ctx.globals().get("fetch")?;
        let res: Promise<Value> = fetch(format!("http://{}/expect/200/", mock_server.address()), None)?;
        let response = res.await?;
        assert!(response.is_ok());
        Ok(())
    }
})
.await;

vm.runtime.idle().await;

error[E0282]: type annotations needed
   --> src/http/fetch.rs:490:21
    |
490 |                 let fetch = ctx.globals().get("fetch")?;
    |                     ^^^^^
491 |                 let res: Promise<Value> = fetch(format!("http://{}/expect/200/", mock_server.address()), None)?;
    |                                           -------------------------------------------------------------------- type must be known at this point
    |
help: consider giving `fetch` an explicit type
    |
490 |                 let fetch: /* Type */ = ctx.globals().get("fetch")?;
    |                          ++++++++++++

My apologies. I have tried everything, but I cannot resolve the error that the type of fetch is not specified. What is the type of fetch in this case?

No need to apologize. Fetch is a Function :)
let fetch: Function = ...

We are continuing to try things, but there are two issues.

When I try to use assert! in `async_with!, I get an error. Therefore, I am wondering if it is possible to bring the response outside.
Is it OK if the type is not specified in response as in fetch?(Issue No. 1 has not been resolved and not confirmed.)

let vm = Vm::new().await.unwrap();

async_with!(vm.ctx => |ctx| {
    async move {
        let fetch: Function = ctx.globals().get("fetch")?;
        let url = format!("http://{}/expect/200/", mock_server.address());
        let options = {};
        let res: Promise<Value> = fetch.call((url, options))?;
        let response = res.await?;
        // The assert macro cannot be used in asynchronous functions.
        // assert!(response.is_ok());
        Ok(())
    }
})
.await;

vm.runtime.idle().await;
}

error[E0282]: type annotations needed
   --> src/http/fetch.rs:497:17
    |
497 |                 Ok(())
    |                 ^^ cannot infer type of the type parameter `E` declared on the enum `Result`
    |
help: consider specifying the generic arguments
    |
497 |                 Ok::<(), E>(())
    |                   +++++++++

Yes, you can return the result from the await right away:

I'm sorry. I continue to have trouble getting res out of the async block.
I asked ChatGPT and got several suggestions for how to write L494, but they were all no good...
error[E0277]: the `?` operator can only be used in an async block that returns `Result` or `Option` (or another type that implements `FromResidual`)
   --> src/http/fetch.rs:490:65
    |
489 | /             async move {
490 | |                 let fetch: Function = ctx.globals().get("fetch")?;
    | |                                                                 ^ cannot use the `?` operator in an async block that returns `rquickjs::Value<'_>`
491 | |                 let url = format!("http://{}/expect/200/", mock_server.address());
492 | |                 let options = {};
493 | |                 let res: Promise<Value> = fetch.call((url, options))?;
494 | |                 res.await.catch(&ctx).unwrap()
495 | |             }}).await;
    | |_____________- this function should return `Result` or `Option` to accept `?`
    |
    = help: the trait `FromResidual<std::result::Result<Infallible, rquickjs::Error>>` is not implemented for `rquickjs::Value<'_>`

error[E0277]: the `?` operator can only be used in an async block that returns `Result` or `Option` (or another type that implements `FromResidual`)
   --> src/http/fetch.rs:493:69
    |
489 | /             async move {
490 | |                 let fetch: Function = ctx.globals().get("fetch")?;
491 | |                 let url = format!("http://{}/expect/200/", mock_server.address());
492 | |                 let options = {};
493 | |                 let res: Promise<Value> = fetch.call((url, options))?;
    | |                                                                     ^ cannot use the `?` operator in an async block that returns `rquickjs::Value<'_>`
494 | |                 res.await.catch(&ctx).unwrap()
495 | |             }}).await;
    | |_____________- this function should return `Result` or `Option` to accept `?`
    |
    = help: the trait `FromResidual<std::result::Result<Infallible, rquickjs::Error>>` is not implemented for `rquickjs::Value<'_>`

error[E0308]: mismatched types
   --> src/http/fetch.rs:488:43
    |
488 |            let fetch_result: Result<Value> = async_with!(vm.ctx => |ctx| {
    |  ____________________________________________^
489 | |/             async move {
490 | ||                 let fetch: Function = ctx.globals().get("fetch")?;
491 | ||                 let url = format!("http://{}/expect/200/", mock_server.address());
492 | ||                 let options = {};
493 | ||                 let res: Promise<Value> = fetch.call((url, options))?;
494 | ||                 res.await.catch(&ctx).unwrap()
495 | ||             }}).await;
    | ||_____________-_^ expected `Result<Value<'_>, Error>`, found `async` block
    |  |_____________|
    |                the found `async` block
    |
    = note:       expected enum `std::result::Result<rquickjs::Value<'_>, rquickjs::Error>`
            found `async` block `{async block@src/http/fetch.rs:489:13: 495:14}`
    = note: this error originates in the macro `async_with` (in Nightly builds, run with -Z macro-backtrace for more info)

Sorry my misstake, I forgot to remove unwrap. You need to do that if you want to return Result using ? from the async block. Options is also wrong here: {} in rust is a closure, not a JS object. You can pass a Request object or a JS object like this:

Thank you very much. However, when creating the initial object for the options, it seems that ctx is not allowed in this position because of the AsyncContext.
I also removed the unwrap() on the last line of the async block, but the error did not change significantly.
The codes and errors at this time are as follows.
let vm = Vm::new().await.unwrap();

let fetch_result: Result<Value> = async_with!(vm.ctx => |ctx| {
    async move {
        let fetch: Function = ctx.globals().get("fetch")?;
        let options = Object::new(ctx).unwrap();
        options.set("redirect","manual").unwrap();

        let url = format!("http://{}/expect/200/", mock_server.address());

        let res: Promise<Value> = fetch.call((url, options))?;
        res.await.catch(&ctx)
    }}).await;

let fetch_response = Class::<Response>::from_value(fetch_result.unwrap()).unwrap();
let response = fetch_response.borrow();
assert!(response.status() == 200);

vm.runtime.idle().await;

error[E0277]: `?` couldn't convert the error to `CaughtError<'_>`
   --> src/http/fetch.rs:490:65
    |
490 |                 let fetch: Function = ctx.globals().get("fetch")?;
    |                                                                 ^ the trait `std::convert::From<rquickjs::Error>` is not implemented for `CaughtError<'_>`, which is required by `std::result::Result<rquickjs::Value<'_>, CaughtError<'_>>: FromResidual<std::result::Result<Infallible, rquickjs::Error>>`
    |
    = note: the question mark operation (`?`) implicitly performs a conversion on the error value using the `From` trait
    = help: the following other types implement trait `FromResidual<R>`:
              <std::result::Result<T, F> as FromResidual<Yeet<E>>>
              <std::result::Result<T, F> as FromResidual<std::result::Result<Infallible, E>>>
    = note: required for `std::result::Result<rquickjs::Value<'_>, CaughtError<'_>>` to implement `FromResidual<std::result::Result<Infallible, rquickjs::Error>>`

error[E0277]: `?` couldn't convert the error to `CaughtError<'_>`
   --> src/http/fetch.rs:496:69
    |
496 |                 let res: Promise<Value> = fetch.call((url, options))?;
    |                                                                     ^ the trait `std::convert::From<rquickjs::Error>` is not implemented for `CaughtError<'_>`, which is required by `std::result::Result<rquickjs::Value<'_>, CaughtError<'_>>: FromResidual<std::result::Result<Infallible, rquickjs::Error>>`
    |
    = note: the question mark operation (`?`) implicitly performs a conversion on the error value using the `From` trait
    = help: the following other types implement trait `FromResidual<R>`:
              <std::result::Result<T, F> as FromResidual<Yeet<E>>>
              <std::result::Result<T, F> as FromResidual<std::result::Result<Infallible, E>>>
    = note: required for `std::result::Result<rquickjs::Value<'_>, CaughtError<'_>>` to implement `FromResidual<std::result::Result<Infallible, rquickjs::Error>>`

error[E0308]: mismatched types
   --> src/http/fetch.rs:488:43
    |
488 |            let fetch_result: Result<Value> = async_with!(vm.ctx => |ctx| {
    |  ____________________________________________^
489 | |/             async move {
490 | ||                 let fetch: Function = ctx.globals().get("fetch")?;
491 | ||                 let options = Object::new(ctx).unwrap();
492 | ||                 options.set("redirect","manual").unwrap();
...   ||
497 | ||                 res.await.catch(&ctx)
498 | ||             }}).await;
    | ||_____________-_^ expected `Result<Value<'_>, Error>`, found `async` block
    |  |_____________|
    |                the found `async` block
    |
    = note:       expected enum `std::result::Result<rquickjs::Value<'_>, rquickjs::Error>`
            found `async` block `{async block@src/http/fetch.rs:489:13: 498:14}`
    = note: this error originates in the macro `async_with` (in Nightly builds, run with -Z macro-backtrace for more info)

The following lines are also available in ? -> .unwrap(), the error that occurred in this line disappeared.

        ...
        let fetch: Function = ctx.globals().get("fetch")?;
        ...
        let res: Promise<Value> = fetch.call((url, options))?;
        ...

I cannot resolve the L498 error.

No more pseudo code! I've tested this and it compiles. We make all the assertions inside the async block and catch the exceptions inside the async_with! macro as they can't be returned outside of it.
async_with!(vm.ctx => |ctx| {
    let globals = ctx.globals();
    let run = async {
        let options = Object::new(ctx.clone())?;
        options.set("redirect", "manual")?;

        let fetch: Function = globals.get("fetch")?;
        let url = format!("http://{}/expect/200/", "url");
        let response_promise: Promise<Value> = fetch.call((url,options))?;
        let response = response_promise.await?;
        let response = Class::<Response>::from_value(response)?;
        let response = response.borrow();

        //do more tests and assertions in here
        assert_eq!(response.status(),200);

        Ok(())
    };
    run.await.catch(&ctx).unwrap();
})
.await;

Hi @richarddavison, Thank you for your dedication and support!
Finally, the cargo test started to pass when accessing the mock server I created.


Style property documentation

Many components use style, but only some of them are documented. This PR aims to make the documentation consistent by adding style in both the propTypes and website documentation.

Thanks for triggering travis ro re-run the tests :D

Looks good to me. I didn't check every component, but the idea is here. Thanks :+1:

Thank you :grin:


Zuul not deleting all headers once request is finished.

We had a strange bug. The origin headers and request headers on zuul's RequestContext was not clearing the origin response and zuul response headers. Over time the headers X-Application-Context, Cache-Control, Date and Expires became larger. Currently our 'fix' is to have a PreFilter clear the headers:
public class PreFilter extends ZuulFilter
{
    @Override
    public String filterType(){ return "pre"; }

    @Override
    public int filterOrder(){ return 1; }

    @Override
    public boolean shouldFilter(){ return true; }

    @Override
    public Object run()
    {
        RequestContext context = RequestContext.getCurrentContext();

        List<Pair<String,String>> oHeaders = context.getOriginResponseHeaders();
        if (oHeaders != null) {
            oHeaders.clear();
        }

        List<Pair<String, String>> rHeaders = context.getZuulResponseHeaders();
        if (rHeaders != null) {
            rHeaders.clear();
        }
        return null;
    }
}

I'm guessing you were using the /zuul/** routes? There's a bug in the ZuulServlet (it doesn't clear it's context, see #546), so this might be a manifestation of that.


is it possible to backup the databases from the secondary replica?

We want to backup the databases in our Alwas On Availability Group from the secondary replica.
We want to reduce the network traffic on the primary server.
Is it possible to change the default settings?
Thank You!

https://ola.hallengren.com/frequently-asked-questions.html

Does the SQL Server Maintenance Solution support Always On Availability Groups?
The SQL Server Maintenance Solution supports AlwaysOn Availability Groups.
Copy-only full backups and transaction log backups are performed on the preferred replica. Full backups (non copy-only) and differential backups are performed on the primary replica.
Integrity checks are performed on all replicas. Index and statistics maintenance is performed on the primary replica.

SQL Server only allows you to run full backups on the primary replica, you can do a copy only on secondaries. refer to the documentation.
https://docs.microsoft.com/en-us/sql/database-engine/availability-groups/windows/active-secondaries-backup-on-secondary-replicas-always-on-availability-groups?view=sql-server-ver15


[$1000] IOS - Emoji chopped off from top in workspace name

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Go to settings-> Workspaces
Click New Workspace -> Click Settings
Rename with “:vertical_traffic_light::vertical_traffic_light::vertical_traffic_light::vertical_traffic_light:” -> Click Save

Expected Result:
Emoji with some height is being chopped off from top
Actual Result:
:vertical_traffic_light: Emoji shouldn’t be chopped off
💥 Please note, this seems to be device-specific and will likely only affect smaller phones (for example, the issue was reproduced on an iPhone 11)💥
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android / native
[ ] Android / Chrome
[x] iOS / native
[ ] iOS / Safari
[ ] MacOS / Chrome / Safari
[ ] MacOS / Desktop

Version Number: <IP_ADDRESS>
Reproducible in staging?: n/a
Reproducible in production?: n/a
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation

Expensify/Expensify Issue URL:
Issue reported by: @DinalJivani
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1692310024902729
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~0119f528f80030b332
Upwork Job ID:<PHONE_NUMBER>709296640
Last Price Increase: 2023-09-19

@johncschuster
I am able to reproduce this issue on iPhone 11 iOS/Native App.
Not sure for about other devices though...

Thanks for calling that out, @DinalJivani! I've updated the OP to indicate that this might be device-specific.

Proposal
Please re-state the problem that we are trying to solve in this issue.
The 🚦-emoji is cut-off when used in the name of a workspace.
What is the root cause of that problem?
The root cause of the problem is the styles.textHeadline being applied to the element. The style combines variables.fontSizeXLarge with variables.lineHeightXXLarge, which causes the clipping as the line-height is simply not large enough for the font size. This happens everywhere that styles.textHeadline is used. Another example can be seen with the user displayname as can be seen here:

Screenshot of user page

The style gets applied here:
https://github.com/Expensify/App/blob/96a0db1aa2e239530f926b9bf62951f3c3408918/src/pages/workspace/WorkspaceInitialPage.js#L239-L244
The style:
https://github.com/Expensify/App/blob/96a0db1aa2e239530f926b9bf62951f3c3408918/src/styles/styles.js#L382C1-L388
The variables in question:
https://github.com/Expensify/App/blob/96a0db1aa2e239530f926b9bf62951f3c3408918/src/styles/variables.ts#L54
https://github.com/Expensify/App/blob/96a0db1aa2e239530f926b9bf62951f3c3408918/src/styles/variables.ts#L95
What changes do you think we should make in order to solve the problem?

A clean and simple fix is to change styles.textHeadline.lineHeight from variables.lineHeightXXLarge to variables.lineHeightXXXLarge. This ensures that it gets fixed everywhere and looks perfect as shown in the screenshots below.

Screenshot of workspace with fix

Screenshot of user page with fix

What alternative solutions did you explore? (Optional)
Alternative solutions include introducing a new style specifically for where styles.textHeadline is used and emojis might appear.

@fedirjh It affects quite a few tall emojis. Also the issue gets increasingly worse as text size is decreased on the device and affects text too on the lower settings, especially letters with accents (see the Á in the screenshots below).
As for potential regressions, I've checked everywhere that the textHeadLine style is used, and my proposal doesn't seem to break anything. On the contrary it actually fixes the issue in other places as well (like the user displayname and the participant details in a report).
This is with the lowest text size setting:

Same settings with the proposed fix:

@dantastisk what device have you tested on?

The screenshots are from a physical iPhone XS.
I was not able to reproduce the bug on a simulated iPhone, initially, but I realize now, that this is because the bug is related to the device text size setting (in Settings -> Accessibility -> Display & Text Size -> Larger Text). As far as I can tell, the chopping only happens when this is set to the 3rd tick or lower, worsening as the text size is decreased.

I do not know how common it is to adjust these settings and what the policy is for supporting this, but this makes it seem like it should be supported:
https://github.com/Expensify/App/blob/cbda2b03824d9b125ec6ab19e87ff533bea1b22f/src/styles/variables.ts#L3-L12

@dantastisk
I checked this Text size thing, default selection will be 3rd tick, When we go for larger size it actually works fine, But when you go to smallest size it chopped the emoji even more... Adding video for same below.
https://github.com/Expensify/App/assets/58871336/de194a59-72c6-4640-b89a-8ab6c3693b37

@johncschuster This bug seems to be an emoji/device-specific case, and it's an extremely edge case, I suggest we do nothing here.

@fedirjh sorry I missed this comment! I'm a bit torn. Yes, I agree this is a bit device-specific, but the Universal Font Size setting in iOS's Accessibility features might make this behavior more common.
@Expensify/design, what are your thoughts on fixing this behavior? I kinda feel like we should fix it to ensure all folks are seeing New Expensify as we intend it to be seen.

Issue not reproducible during KI retests. (First week)

Issue not reproducible during KI retests. (First week)

@mvtglobally
Did you test this issue on iPhone 11 ?
I think this is device specific and depends upon some attributes like Font size in Settings > Accessibility.

Hmm I'm not too sure here, I don't know what our overall strategy is for supporting the phone's accessibility sizes. I agree it shouldn't be cut off, but I'm not sure how exactly we might fix that.

Bringing the discussion around phone size support to #bug-zero

Ok! I've got one opinion that we should fix this. Let's go ahead and give it a try. @fedirjh do you think any of the proposals above are sufficient?

Issue not reproducible during KI retests. (Second week)

Issue not reproducible during KI retests. (First week)

@mvtglobally
Did you test this issue on iPhone 11 ?
I think this is device specific and depends upon some attributes like Font size in Settings > Accessibility.

@mvtglobally @DinalJivani I can reproduce on all iOS devices, seems to only depend on font size settings.

@dantastisk
Yes correct but on iPhone 11 it chops off emojis with normal/default text size too.

Busy week. Update coming tomorrow!

Thanks, @dantastisk!

I can no longer reproduce this. Neither for the original case or the ones in your screenshots @fedirjh. Can you check if it's the same for you?

I'm still able to reproduce on v1.3.80-0 Mobile app
Tested on iPhone 11 with iOS 16.0

Looks like this behavior is still reproducible. @dantastisk @neonbhai can you update your proposals to account all cases within the app?

@dantastisk / @neonbhai bump!

Pinged in Slack

Good thinking! I'll raise this to an agency.

I have raised this to both SWM and Callstack to see if anyone can take this on.

Hi, I’m Michael (Mykhailo) from Callstack and I would like to work on this issue.

That sounds great, @rezkiy37! It's all yours!

@rezkiy37 This bug appears to impact other areas, particularly text inputs. Please review my comment #26044 (comment)

I know it. I confirmed it.

No important updates, still investigating.

Yep, I am preparing a proposal today for Callstack internal review, then will post it here.

@fedirjh, I just found an interesting situation. There aren't any problems to fix line of a static text.
However, most cases mentioned in your comment are inputs. Moreover, only when the keyboard is opened. The problem disappears, when you close the keyboard. I thought to update lineHeight of the input.
Meanwhile, other guys updated logic for the BaseTextInput's lineHeight.

Actually, if we are adding any lineHeight to the input, it breaks one-line one.
Also, playing with the input's height it's got the wrong kind of luck.
So, I think we need to accept a compromise, because it is a super edge case and one-line inputs more important.

Melvin, AFAIK waiting on a decision on comments above from @johncschuster.

No update.

No update.

So sorry I missed your comment, @fedirjh.

@johncschuster It seems we're somewhat limited in options for addressing this bug affecting single-line inputs. Given the importance of one-line inputs across the app and the rarity of this edge case, embracing a compromise might be the most pragmatic way forward. What are your thoughts on this?

I'm open to a compromise, but I think I would like to bring this conversation to Slack so we can get a more holistic understanding of the problem/solution and any consequences that might come from the compromise. Would you mind raising the discussion? I'm happy to do it, but I think you might be able to explain it a little better.

I've opened the discussion.

Ok! We've discussed the compromise in Slack, and I think we're good to accept it. 👍

Cool, so I am preparing a PR soon.

Currently, the proposal is being reviewed internally.

Proposal
Please re-state the problem that we are trying to solve in this issue.

What is the root cause of that problem?
The root cause of the problem is that lineHeight is not large enough in pair with fontSize of some components. The app should handle cases when user globally increase font size and decrease as well.
In general there is a few cases what we are going to fix: a workspace name, a user display name and tabs.
What changes do you think we should make in order to solve the problem?
Increase lineHeight value in styles.ts for:

.textHeadline → variables.lineHeightXXXLarge;
.optionDisplayName → variables.lineHeightXXLarge;
.optionAlternateText → variables.lineHeightXXLarge;
.chatItemMessageHeaderSender → variables.lineHeightXXLarge;
.tabText → (set) 14.

Increase lineHeight value in ReportPreview.js for preview message to variables.lineHeightXXLarge.
Once we increase the lineHeights, text has enough place to render from the smallest to largest font sizes.
What alternative solutions did you explore? (Optional)
N/A

Waiting on review for a proposal above.

@fedirjh, who can review the proposal?

Weird how the Internal assigner did not assign an engineer here. Let's go with the proposal anyway.

Looks like you're actually already assigned @rezkiy37 so feel free to post your PR 👍

There was quite a similar issue regarding centering and chopping off tops of emojis: https://github.com/Expensify/App/issues/32283#issuecomment-1835759361
It was closed as non-important, but @tienifr posted quite a similar proposal 2 weeks before the one selected here.
Just a heads-up: @jjcoffee @slafortune

Good spot! I don't have a strong opinion on whether or not this one should be closed too but it has been open for far longer and deals with it more broadly across the app. We should definitely make sure to fix the issue with the emoji picker too, if it's not already on the cards cc @rezkiy37  (see https://github.com/Expensify/App/issues/32283).

So, I've prepared a PR. Please take a look 🙂

Guys, I am having a vacation till the 4th January. So, I will be OOO within this period.

Guys, I am having a vacation till the 4th January. So, I will be OOO within this period.

That should be fine. @fedirjh can you review the PR in the meantime?

There's regression from linked PR

Looks like I need to manually create the Upwork job since it wasn't automatically created. We were just discussing a new process for this today. I'll follow the process there and get this paid out.

Payment Summary
@fedirjh requires payment by manual request of $1000
@rezkiy37 does not require payment (Contractor)
@DinalJivani requires payment by manual request of $1000

There was regression

@johncschuster
Can you please create Upwork job and send me offer?

@DinalJivani, I invited you to the Upwork job. Can you please accept that so I can issue payment for the bug report?

@johncschuster
You sent offer for $50 instead of $250. This is old bug report.

@johncschuster
I had just accepted offer. Please approve $250 reporting bonus.
Thanks

You sent offer for $50 instead of $250. This is old bug report.

@DinalJivani sorry about that. I just double-checked when the reporting bonus change was made, and I see it was August 30, while this was reported on August 17th. I've issued the corrected amount of $250.

@johncschuster Could you please invite me as well? I am already ineligible for the manual requests.

Sure thing, @fedirjh, I'll invite you now.

Payment has been issued! Thanks everyone!


Fix build with -O3

Related issue: #657, #853
JerryScript-DCO-1.0-Signed-off-by: László Langó<EMAIL_ADDRESS>

Tested with gcc 5.2 and there are no errors.

I think we need to change the other side to clear the buffer in advance (with an uit32_t zero). The problem is, if normal buffer is passed, you may overwrite the next values in the buffer.

@rtakacs, @zherczeg I've updated the PR. Please check.

Checked, there are no build errors.

@rtakacs, thanks. @zherczeg, please check.

I like this much better. LGTM


Issue 475 and 433

Add argument to airr::read_airr() to define the type of non-schema columns, with character as default.
Fitting unit test is added. Test data created on the fly from toy_data.tsv
Closes #475 and #433

For #433 this should be done for the python library, as well. I might have some time next week...

Thanks, @ustervbo!  I made some minor change: shorted the argument name, extra docs, added the change to the news, and stuck you in the author list.
Can you confirm you want to be in the package author list and confirm the email address I put is correct, please?

It looks like these types are very R-specific. Should we be more generic and use OpenAPI types? I like the idea in #433 of passing in an OpenAPI field structure. Is there any reason not to do that?

It looks like these types are very R-specific. Should we be more generic and use OpenAPI types? I like the idea in #433 of passing in an OpenAPI field structure. Is there any reason not to do that?

Meaning:
string -> character
number -> double
integer -> integer
boolean -> logical

From the call:

Make the type name change from @schristley and merge in.


Fixes scoping bug with voiceEmitter const

My apologies, @ohtangza & team -- I didn't initialize the voiceEmitter const outside the if block in #66 meaning it wasn't be set to the proper scope.  This PR fixes that bug.  My apologies for not testing more completely.

@ohtangza -- updated to use let instead.  oddly enough it wasn't causing an error for me with const but you're absolutely right, that should not work.

Maybe  just not important detail, but you can avoid using let by writing it this way:
const voiceEmitter = Platform.OS !== "web" ? new NativeEventEmitter(Voice) : null;

I also spotted that voiceEmitter is used in method start without any null check.
Would you add an if condition there?
if (voiceEmitter) voiceEmitter.addListener...

Thanks @misino -- let me know if that looks good!

@misino Could you review the last change?

@wenkesj Would you release new bugfix version after merging this PR? I think that there were 4-5 other fixes done since last release. :)

@misino yeah sure, I’ll do some review really quick!

@misino @tdonia @ohtangza for the short term, this seems logical.
The future possibility would be to add a dev-level warning for the web platform (or adding support for the native web platform using HTML5 Audio APIs). It may not be clear enough for users directly porting this to the web platform.


Add precise source location for inner parts of link elements

A follow-up of the issue #2 for some inline elements

[x] Link and Images. They have multiple parts that need to be precisely identified and located in the sourcecode (Label, Url, Title...etc.).
[ ] List item: for the number/- part, we need a precise location where it stops

Currently, the parser doesn't keep this information.

Closing this issue as it is not necessary for list items (we have already the information with the span of the list item + span of paragraph)


Feature Request: Orient a GFA

I have a request related to the orientation of the sequence in GFA graphs. With the GFA graphs I am working with, the strand represented in the GFA segment sequence is essentially random. One way this manifests in the graph is that links from the same node end arrive have opposite "to orientation" For example, for nodes utig4-1529 to utig4-1530 and utig4-1527 in this fork structure

The edge from 1529 to 1530 has a + "to orientation" while the edge to 1527 has a - "to orientation"

I believe that this could be done, in the acyclic case, by traversing the graph from one end to the other and taking reverse complements when relative misorientations are detected.

I encouraged @mir-cat to post this feature request because I thought that this might fit into the order_gfa subcommand (see https://github.com/marschall-lab/gaftools/blob/main/gaftools/cli/order_gfa.py).


New version: prefix-dev.pixi version 0.39.0

Pull request has been created with WinGet Releaser v2 :rocket:
Microsoft Reviewers: Open in CodeFlow

Automatic Validation ended with:

No errors to post.

(Automated response - build 899.)

@wingetbot waivers Add Validation-Executable-Error


Create a new PageProcessor instance for each operator

PageProcessor was created once and used by all operators created by the
OperatorFactory. 22e78f56 introduced state in PageProcessor, so we
cannot reuse PageProcessors between operators. Change it so that the
operator factory gets a supplier for PageProcessor which creates a new
PageProcessor for every operator.

What is the impact of not doing this? I assume this is fixing a bug?

It's a thread and concurrency-safety issue due to the PageProcessor being shared across multiple drivers. We tried to fix it in the last release, but the fix was incorrect.

Fixes #4343

@dain updated

Looks good

Merged.