Resource exhausted: OOM when allocating tensor with shape, even when batch_size is 1.

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
I've registered a new problem for translating English to Swedish, the code is pretty similar to TranslateEndeWmtBpe32k, with Europarl data. I'm able to train with hidden_size: 128, anything over 128 and I get Resource exhausted error, even when batch size is 1. (I'm using hparams: transformer_big).

I notice that previous posts on this issue were solved by reducing the batch size. I'm posting this because reducing batch size wasn't helpful and I'm hoping if someone encountered a similar issue.

OS Platform and Distribution (e.g., Linux Ubuntu 16.04):

Linux deepnlp01 4.2.0-42-generic #49~14.04.1-Ubuntu SMP Wed Jun 29 20:22:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
4 VERSION="14.04.5 LTS, Trusty Tahr"

TensorFlow installed from (source or binary):
tf.VERSION = 1.4.1 (Installed from source)

Python version:
2.7.6

Bazel version (if compiling from source):
Build label: 0.5.2

GCC/Compiler version (if compiling from source):
c++ (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4

CUDA/cuDNN version:
nvcc: NVIDIA (R) Cuda compiler driver
Cuda compilation tools, release 8.0, V8.0.61

GPU model and memory:
NVIDIA TITAN X (12GB)

Exact command to reproduce:
USR_DIR=/python2.7/site-packages/tensor2tensor/<new_module>

t2t-trainer
--data_dir=$DATA_DIR
--problems=$PROBLEM
--model=$MODEL
--hparams_set=$HPARAMS
--output_dir=$TRAIN_DIR
--t2t_usr_dir=$USR_DIR
Describe the problem
The data I'm using is parallel corpus English-Swedish, available on http://www.statmt.org/europarl/ . I've created a dictionary myself of about 1,400,000 (1.4 million) words. [The problem persists even if I use a smaller vocab of about 10k words]
I'm able to train with batch_size=1024 and hidden_size=128, if I increase the hidden size to 256, even with batch_size=1, it runs out of memory. I get the following error.
Source code / logs
This is the error I'm encountering:
2018-02-28 05:48:52.977027: W tensorflow/core/framework/op_kernel.cc:1192] Resource exhausted: OOM when allocating tensor with shape[90993,256]

Traceback (most recent call last):
  File "/data/tf_venv/2.7/1.4.1_gpu/bin/t2t-trainer", line 32, in <module>
    tf.app.run()
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 48, in run
    _sys.exit(main(_sys.argv[:1] + flags_passthrough))
  File "/data/tf_venv/2.7/1.4.1_gpu/bin/t2t-trainer", line 28, in main
    t2t_trainer.main(argv)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensor2tensor/bin/t2t_trainer.py", line 337, in main
    execute_schedule(exp)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensor2tensor/bin/t2t_trainer.py", line 287, in execute_schedule
    getattr(exp, FLAGS.schedule)()
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/contrib/framework/python/framework/experimental.py", line 64, in new_func
    return func(*args, **kwargs)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/contrib/learn/python/learn/experiment.py", line 717, in continuous_train_and_eval
    hooks=self._train_monitors)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/contrib/learn/python/learn/experiment.py", line 807, in _call_train
    hooks=hooks)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/estimator/estimator.py", line 302, in train
    loss = self._train_model(input_fn, hooks, saving_listeners)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/estimator/estimator.py", line 783, in _train_model
    _, loss = mon_sess.run([estimator_spec.train_op, estimator_spec.loss])
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.py", line 521, in run
    run_metadata=run_metadata)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.py", line 892, in run
    run_metadata=run_metadata)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.py", line 967, in run
    raise six.reraise(*original_exc_info)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.py", line 952, in run
    return self._sess.run(*args, **kwargs)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.py", line 1024, in run
    run_metadata=run_metadata)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/training/monitored_session.py", line 827, in run
    return self._sess.run(*args, **kwargs)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 889, in run
    run_metadata_ptr)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1120, in _run
    feed_dict_tensor, options, run_metadata)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1317, in _do_run
    options, run_metadata)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1336, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.ResourceExhaustedError: OOM when allocating tensor with shape[90992,256]
         [[Node: training/gradients/AddN_96 = AddN[N=3, T=DT_FLOAT, _class=["loc:@transformer/parallel_0_5/transformer/symbol_modality_1455878_256_2/shared/concat"], _device="/job:localhost/replica:0/task:0/device:GPU:0"](training/gradients/transformer/parallel_0_5/transformer/symbol_modality_1455878_256_2/shared/concat_grad/tuple/control_dependency_14, training/gradients/transformer/parallel_0_5/transformer/symbol_modality_1455878_256_1/shared/concat_grad/tuple/control_dependency_14, training/gradients/transformer/parallel_0_5/transformer/symbol_modality_1455878_256/shared/concat_grad/tuple/control_dependency_14)]]
         [[Node: training/global_norm/global_norm/_4061 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_24183_training/global_norm/global_norm", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

Caused by op u'training/gradients/AddN_96', defined at:
  File "/data/tf_venv/2.7/1.4.1_gpu/bin/t2t-trainer", line 32, in <module>
    tf.app.run()
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 48, in run
    _sys.exit(main(_sys.argv[:1] + flags_passthrough))
  File "/data/tf_venv/2.7/1.4.1_gpu/bin/t2t-trainer", line 28, in main
    t2t_trainer.main(argv)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensor2tensor/bin/t2t_trainer.py", line 337, in main
    execute_schedule(exp)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensor2tensor/bin/t2t_trainer.py", line 287, in execute_schedule
    getattr(exp, FLAGS.schedule)()
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/contrib/framework/python/framework/experimental.py", line 64, in new_func
    return func(*args, **kwargs)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/contrib/learn/python/learn/experiment.py", line 717, in continuous_train_and_eval
    hooks=self._train_monitors)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/contrib/learn/python/learn/experiment.py", line 807, in _call_train
    hooks=hooks)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/estimator/estimator.py", line 302, in train
    loss = self._train_model(input_fn, hooks, saving_listeners)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/estimator/estimator.py", line 711, in _train_model
    features, labels, model_fn_lib.ModeKeys.TRAIN, self.config)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/estimator/estimator.py", line 694, in _call_model_fn
    model_fn_results = self._model_fn(features=features, **kwargs)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensor2tensor/utils/t2t_model.py", line 829, in wrapping_model_fn
    use_tpu=use_tpu)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensor2tensor/utils/t2t_model.py", line 923, in estimator_model_fn
    loss, num_async_replicas=num_async_replicas)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensor2tensor/utils/t2t_model.py", line 927, in estimator_spec_train
    train_op = self.optimize(loss, num_async_replicas=num_async_replicas)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensor2tensor/utils/t2t_model.py", line 351, in optimize
    loss, lr, self.hparams, use_tpu=common_layers.is_on_tpu())
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensor2tensor/utils/optimize.py", line 67, in optimize
    colocate_gradients_with_ops=True)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/contrib/layers/python/layers/optimizers.py", line 241, in optimize_loss
    colocate_gradients_with_ops=colocate_gradients_with_ops)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensor2tensor/utils/optimize.py", line 109, in compute_gradients
    return self._opt.compute_gradients(loss, var_list, **kwargs)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/training/optimizer.py", line 414, in compute_gradients
    colocate_gradients_with_ops=colocate_gradients_with_ops)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/ops/gradients_impl.py", line 533, in gradients
    out_grads = _AggregatedGrads(grads, op, loop_state, aggregation_method)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/ops/gradients_impl.py", line 872, in _AggregatedGrads
    out_grads[i] = _MultiDeviceAddN(out_grad)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/ops/gradients_impl.py", line 767, in _MultiDeviceAddN
    summands.append(math_ops.add_n(tensors))
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/ops/math_ops.py", line 2000, in add_n
    return gen_math_ops._add_n(inputs, name=name)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/ops/gen_math_ops.py", line 220, in _add_n
    "AddN", inputs=inputs, name=name)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2956, in create_op
    op_def=op_def)
  File "/data/tf_venv/2.7/1.4.1_gpu/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1470, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

ResourceExhaustedError (see above for traceback): OOM when allocating tensor with shape[90992,256]
         [[Node: training/gradients/AddN_96 = AddN[N=3, T=DT_FLOAT, _class=["loc:@transformer/parallel_0_5/transformer/symbol_modality_1455878_256_2/shared/concat"], _device="/job:localhost/replica:0/task:0/device:GPU:0"](training/gradients/transformer/parallel_0_5/transformer/symbol_modality_1455878_256_2/shared/concat_grad/tuple/control_dependency_14, training/gradients/transformer/parallel_0_5/transformer/symbol_modality_1455878_256_1/shared/concat_grad/tuple/control_dependency_14, training/gradients/transformer/parallel_0_5/transformer/symbol_modality_1455878_256/shared/concat_grad/tuple/control_dependency_14)]]
         [[Node: training/global_norm/global_norm/_4061 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_24183_training/global_norm/global_norm", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

This is the code that I've modified:
def _get_europarl_ensv_dataset(directory, filename):
    """Extract the EuroParl en-sv corpus `filename` to directory unless it's there."""

    train_path = os.path.join(directory, filename)

    return train_path

@registry.register_problem
class TranslateEnsvEuroparl(translate.TranslateProblem):
    """Problem spec for EuroParl En-Sv translation, BPE version."""

    @property
    def approx_vocab_size(self):
        return 1455877

    @property
    def vocab_filename(self):
        return "vocab.europarl.ensv.txt"

    def get_or_create_vocab(self, data_dir, tmp_dir, force_get=False):
        vocab_filename = os.path.join(data_dir, self.vocab_filename)
        if not tf.gfile.Exists(vocab_filename) and force_get:
            raise ValueError("Vocab %s not found" % vocab_filename)
        return text_encoder.TokenTextEncoder(vocab_filename, replace_oov="UNK")

    def generate_samples(self, data_dir, tmp_dir, dataset_split):
        """Instance of token generator for the WMT en->de task, training set."""
        train = dataset_split == problem.DatasetSplit.TRAIN
        dataset_path = ("europarl-v7.train.sv-en"
                        if train else "europarl-v7.test.sv-en")
        train_path = _get_europarl_ensv_dataset(tmp_dir, dataset_path)

        # Vocab
        token_path = os.path.join(data_dir, self.vocab_filename)
        if not tf.gfile.Exists(token_path):
            token_tmp_path = os.path.join(tmp_dir, self.vocab_filename)
            tf.gfile.Copy(token_tmp_path, token_path)
            with tf.gfile.GFile(token_path, mode="r") as f:
                vocab_data = "<pad>\n<EOS>\n" + f.read() + "UNK\n"
            with tf.gfile.GFile(token_path, mode="w") as f:
                f.write(vocab_data)

        return text_problems.text2text_txt_iterator(train_path + ".en",
                                                    train_path + ".sv")

The approx_vocab_size is intended for building the internal subword vocabulary with SubwordTextEncoder. However, you are using TokenTextEncoder for which you need to pre-tokenize the training data yourself and provide the vocabulary file and the approx_vocab_size is ignored in this case (so setting it to 10000 makes no difference if the vocab file still has 1.4M entries).
1.4M vocabulary is really big. I suggest using about 32k subwords vocab and SubwordTextEncoder.
I don't see any --hparams="batch_size=1024" in your t2t-trainer usage. You should also check flags.txt and hparams.json to make sure all hparams were applied as you intended.

@martinpopel Thank you very much for the help.
I had a few doubts, I tried creating a subword vocab with data_generators/text_encoder_build_subword.py, but I got this error:
ImportError: cannot import name text_encoder
I used the command provided in the same python script to run:
python data_generators/text_encoder_build_subword.py \
    --corpus_filepattern=$DATA_DIR/translate_ensv_europarl-train-* \
    --corpus_max_lines=12345 \
    --output_filename=$DATA_DIR/translate_ensv_europarl.subword_text_encoder \
    --logtostderr

text_encoder_build_subword.py has these two imports that it's unable to resolve:
from tensor2tensor.data_generators import text_encoder
from tensor2tensor.data_generators import tokenizer

For the batch_size, I had modified transformer.py and added it in the config I'm using (transformer_big()).

There is no need to use text_encoder_build_subword.py, just follow the Walkthrough and the subword vocabulary file will be created automatically.
Similarly, there is no need to modify transformer.py for changing the batch size, just use t2t-trainer --hparams='batch_size=1024' ... as described in the Walkthrough (or create your own hparams set and store in t2t_usr_dir).

Hi Martin,
Thank you very much for the help.
It's working for me now. But there is one more thing that I want to ask you.
With the training data for Swedish, the code generates a vocab of about 17 thousand words, when I set the approx size to 32k. I trained it for about 250000 steps with transformer_big() parameters with 1.5 million lines in training data. The trained model works well with seen data but poorly on unseen data.
Would you have an opinion on what could be going wrong or any recommendation for us?
Thanks a lot again,
Abhinav

I suggest to close this issue as the original problem is solved (in order to keep GitHub issues tidy).

vocab of about 17 thousand words, when I set the approx size to 32k

If you have just 1.5M parallel sentences (Opus offers over 40M for en-sv), you should check how many unique words there are - and you should not expect to obtain more subwords than unique words.
If there are many more unique words than 17k, you can try increasing the file_byte_budget.

poorly on unseen data

Are those data from the same domain as the training data?
If yes, a possible reason is overfitting, so you should watch the BLEU on dev set and apply early stopping just before it starts worsening.
If no, you should apply some domain adaptation methods, e.g. backtranslation of in-domain monolingual data, or continue training on parallel in-domain data, if available (possibly with tuneout).

Thank you very much again, I'm closing this issue.


[Feature] Support checking other repositories for changes

Is this feature missing in the latest version?

[X] I'm using the latest release

Is your feature request related to a problem? Please describe.
I would like to check if (a file in) a different repo has changed or not.
I normally parse a file on every run of my action, but this is just wasting computing resources, if the file did not change. I could just load the parsing result from cache then.
This is why I would like to check if the file has changed since my last action run.
Describe the solution you'd like?
an optional variable to specify the repository being checked for changes. Maybe in the format username/reponame ?
Describe alternatives you've considered?
No response
Anything else?
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

@AlexCherrypi I believe this is already supported you’ll need to use the checkout action and pass the path to the repository to this action you’ll also need to provide the base_sha and sha inputs.
For an example on how to use the changed-files action for a repository located in a different path search for “Get all changed files for a repository located in a different path” in the Examples section.

@jackton1 Let me describe a more specific example for you to judge if this already supported, maybe even specifically by the Example you quoted

My action runs on a daily basis in the repository AlexCherrypi/BNDnamegenerator/.
It then checks if in repository hdaSprachtechnologie/odenet the file /odenet/wordnet/deWordNet.xml has changed since the last run of the action.
base_sha and sha are available to check the file.
based on this output the rest of the action is configured...

Do you think, this is possible with the current version of this action?


Playbook sends Command to the remote devices one letter at a time.

`HostName1# ### Thu Oct 27 03:10:01 2016 <IP_ADDRESS> ###
[
% Bad IP address or host name% Unknown command or computer name, or unable to find computer address
HostName1#'
% Bad IP address or host name% Unknown command or computer name, or unable to find computer address
HostName1#s
% Type "show ?" for a list of subcommands
HostName1#h
Help may be requested at any point in a command by entering
a question mark '?'.  If nothing matches, the help list will
be empty and you must backup until entering a '?' shows the
available options.
Two styles of help are provided:

Full help is available when you are ready to enter a
command argument (e.g. 'show ?') and describes each possible
argument.
Partial help is provided when an abbreviated argument is entered
and you want to know what arguments match the input
(e.g. 'show pr?'.)

HostName1#o
% Incomplete command.
HostName1#w
% No connections open
HostName1#
HostName1#r
% No open connections
HostName1#u
undebug
% Incomplete command.
HostName1#n
% Ambiguous command:  "n"
HostName1#n
% Ambiguous command:  "n"
HostName1#i
% Incomplete command.
HostName1#n
% Ambiguous command:  "n"
HostName1#g`

It uses Paramiko / SSH and this is the expected behavior.  Here is a good link explaining what you observed.
http://blog.hyfather.com/blog/2013/04/18/ssh-uses-four-tcp-segments-for-each-character/

Thanks for your response. Specifying the type for the command dict in this module as a list fixed the issue.
From:
commands=dict(required=True)
To:
commands=dict(type='list',required=True)


SQL : Better wildcard support in projections

Wildcard "*" support in projections is currently limited.  For example, the following does not work:
select in_knows.name, * from V

Currently (1.7-SNAPSHOT) this is interpreted as:
select * from V

This makes it hard to efficiently get all of the properties from each record without knowing their names (i.e. each record may have different properties).

Now it supports more flexible usage like:
select *, $l.* as leases, $u.* as uses, $p.* as photocopiesWith, $locks.* as locksIn
  from Teacher
    let $l = out('Leases')[0],
        $u = out('Uses')[0],
        $p = out('PhotocopiesWith')[0],
        $locks = out('LocksIn')[0]

How come this is not working for me ? orientdb-community-2.1-rc2
select *, $o.* as Own from Item
let $o = in('Own')[0]

I get :
com.orientechnologies.orient.core.sql.OCommandSQLParsingException: Error on parsing command at position #0: Encountered " "." ". "" at line 1, column 13.


SPLAC with PERIODS

This is a draft pullrequest, an addition to "SPLAC besides PLAC".
It adds a PERIOD structure to SPLAC to give it timeframe's to keep the SPLAC data of 1 timeframe together.
It is an attempt to deal with the problems mentioned in the Zoom session of september 10, 2024.
It also tries to add the GEDCOM-L's _LOC extension to see how that might look.
An extra Example page is added , to be able to see it "working". That page has worked examples of the problems that were mentioned in the Zoom session.
Hopefully this will bring others on more, and maybe better ideas too!

Here some background information about this PR.
The Header: changing PLAC from{0:1} to {1:1}
Thats because I worked with some well known, widely spread Dutch program, which had no PLAC.FORM in the header, and no PLACfor any place used in the GEDCOM. Trying to import that into another program gave lots of problems. As these programs could not automatically locate the places on a map.
So in my opinion it is important that it is obligatory for a GEDCOM to have a way of defining how the places in that GEDCOM are specified.
Whatever that way of defining might be.
In fact, I would very much like to see many of the HEAD tags be {1:1}, for the same reason:
Specify where this GEDCOM comes from, who it belonged too, what it is about.
But I didnot do that in this PR.
PLAC.FORM list
This list is just a start. It could be any list, only not too many entries.
It is put up there so every program can Always see what "Main" "jurisdictions" are used.
Thats why it is {1:1} in this PR.
It makes sure tha list is always available from this GEDCOM version onward.
The GOV number is added to further specify a certain entry from this list, as the GOV list has a huge amount of "jurisdictions" defined.
That way software has the possibility to redefine "jurisdictions" found in older GEDCOM's, into the system that will be used from this version of GEDCOM.
Now it could be the tagname PLAC in the header should be redefined, as as far as I understood, PLAC will be depricated in future??
SPLAC construction with TYPE having a list of parameters.
I struggled a bit with this. In my PR this construction now looks like:
  +1 TYPE <Text>                           {1:1}  g71:SPLAC-TYPE
    +2 HREL <HIERARCHICAL_RELATIONSHIP>    {1:1}  g71:SPLAC-HREL
      +3 GOVTYP  <GOVID_OF_TYPE>           {0:1}
        +4 TEXT                            {0:1}  g71:SPLAC-TEXT

This gives for example:
  1 TYPE COUNTRY, POLI, 7, Federal State
It was done this way because we now have to times {1:1} following eachother, so "empty" commas would be at the end of the line.
I also tried:
  1 TYPE COUNTRY, 7, POLI, Federal State
The GEDCOM there would have been:
  +1 TYPE <Text>                           {1:1}  g71:SPLAC-TYPE
    +2 GOVTYP  <GOVID_OF_TYPE>             {0:1}
      +3 HREL <HIERARCHICAL_RELATIONSHIP>  {1:1}  g71:SPLAC-HREL
        +4 TEXT                            {0:1}  g71:SPLAC-TEXT

That could give empty commas in the middle of the line.
And from my experience with one program I use, I know people sometimes have real difficulties with reading empty commas.
We could overcome that in the above second version (not in the PR yet) by defining it like this:
  +1 TYPE <Text>                           {1:1}  g71:SPLAC-TYPE
    +2 GOVTYP  <GOVID_OF_TYPE>             {1:1}
      +3 HREL <HIERARCHICAL_RELATIONSHIP>  {1:1}  g71:SPLAC-HREL
        +4 TEXT                            {0:1}  g71:SPLAC-TEXT

So make GOVTYP obligatory with {1:1}
But that means there should be a GOVTYP for "Unknown" or "Not Present", which could be denoted by the value 0.
But sofar I could not find something like that in the GOVlist.
PERIOD_STRUCTURE
This structure was named PERIOD_STRUCTURE but that could always be changed into another name, like TFRAME_STRUCTURE or something. (TimeFrame)
The original description of how to convert old PLAC values into new SPLAC values, was moved to the PERIOD_STRUCTURE.
But the examples there not yet converted to the new SPLAC addition in this PR. As I have no idea if this PR fills the needs. If it does somehow, the examples must be changed too.
HREL
The fact that maybe HREL could also be written as
  HREL POLI / RELI

so with 2 values separated by a slash is mentioned in an example but as it is fantasized, it is not defined anywhere yet.
SUBM
Here SUBM was added on 1 place in this PR, but as I saw posts with a proposel to add SUBM on more places in GEDCOM, that could be done here too.


liquibase:tag does not work on cluster

Hello,
I have tried to add a tag to my changelog but got the following error.
[ERROR] Failed to execute goal org.liquibase:liquibase-maven-plugin:4.6.1:tag (default-cli) on project                            36mOpenseeLiquidityDataModel: [ERROR] Error setting up or running Liquibase: [ERROR] ClickHouse exception, code: 62, host: iqoprdweb057, port: 8123; Code: 62, e.displayText() = DB::Exception: Syntax error: failed at position                  1 ('UPDATE') : UPDATE DATABASECHANGELOG SET TAG = '1.0.0' WHERE DATEEXECUTED = (SELECT MAX(DATEEXECUTED) FROM DATABASECHANGELOG). Expected one of: ALTER query, Query with output, ALTER PROFI LE, RENAME DATABASE, SHOW PRIVILEGES query, TRUNCATE, KILL, KILL QUERY query, SELECT query, possibly with UNION, list of union elements, ALTER ROLE, SELECT subquery, DESCRIBE qu ery, SELECT query, subquery, possibly with UNION, SHOW GRANTS, SHOW CREATE, WATCH, CREATE SETTINGS PROFILE or ALTER SETTINGS PROFILE query, SHOW PROCESSLIST query, ALTER POLICY,  ALTER USER, CREATE VIEW query, CHECK TABLE, SET ROLE, SELECT query, SELECT, REVOKE, CREATE USER, CREATE DICTIONARY, CREATE PROFILE, SET ROLE DEFAULT, EXPLAIN, ALTER SETTINGS PR OFILE, SYSTEM, ALTER LIVE VIEW, RENAME TABLE, DROP query, SHOW ACCESS, OPTIMIZE query, USE, DROP access entity query, RENAME DICTIONARY, DETACH, SET, SHOW, DESC, OPTIMIZE TABLE,  CREATE ROW POLICY, SET DEFAULT ROLE, EXCHANGE DICTIONARIES, CREATE POLICY, ALTER ROW POLICY, INSERT INTO, INSERT query, SHOW [TEMPORARY] TABLES|DATABASES|CLUSTERS|CLUSTER 'name ' [[NOT] [I]LIKE 'str'] [LIMIT expr], GRANT, RENAME query, SHOW GRANTS query, SHOW PRIVILEGES, EXISTS, DROP, SYSTEM query, CREATE LIVE VIEW query, CREATE ROW POLICY or ALTER ROW  POLICY query, CREATE QUOTA or ALTER QUOTA query, SHOW PROCESSLIST, ALTER QUOTA, CREATE QUOTA, CREATE DATABASE query, SET query, Query, CREATE, WITH, CREATE ROLE or ALTER ROLE q uery, EXTERNAL DDL FROM, EXCHANGE TABLES, EXISTS or SHOW CREATE query, WATCH query, REPLACE, CREATE ROLE, CREATE SETTINGS PROFILE, SET ROLE or SET DEFAULT ROLE query, CREATE USE R or ALTER USER query, EXTERNAL DDL query, SHOW ACCESS query, SHOW CREATE QUOTA query, USE query, ATTACH, DESCRIBE, ALTER TABLE, ShowAccessEntitiesQuery, GRANT or REVOKE query, CREATE TABLE or ATTACH TABLE query (version <IP_ADDRESS> (official build)) [ERROR]  [Failed SQL: (62) UPDATE DATABASECHANGELOG SET TAG = '1.0.0' WH                 ERE DATEEXECUTED = (SELECT MAX(DATEEXECUTED) FROM DATABASECHANGELOG)] [ERROR] -> [Help 1]

I ve created the following PR
https://github.com/MEDIARITHMICS/liquibase-clickhouse/pull/18

Well done! Now I think we can close the issue.


Port Fixes

https://github.com/rancher/rancher/issues/6563
https://github.com/rancher/rancher/issues/6562


cairo: fix cross

Description of changes
I accidentally left this commented out when I did the cairo update that switched the build system to Meson.
I don't know what the correct value for Darwin is, but if somebody lets me know, I can add it here.  Otherwise, it can be follow-up for Darwin users.
Fixes: 2c0a4f46b95c ("cairo: 1.16.0 -> 1.18.0")
Things done

Built on platform(s)

[ ] x86_64-linux
[x] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[b] Fits CONTRIBUTING.md.

It's not a mass rebuild for Hydra, so I suppose we can leave a while for feedback.


CUDA OOM when using SFTTrainer with Phi-1.5B

Hi,
I'm trying to supervised fine-tune a phi-1.5B model on a custom dataset with the SFTTrainer, my script closely follows the sft_llama2.py.
I'm training the model on 4x2080Ti (11G), the model 1.3B params should comfortable fit it the combined VRAM of the GPU's but I see a CUDA OOM errors when I start my training.
my hyper parameters are as follows:
    per_device_train_batch_size: Optional[int] = field(default=1, metadata={"help": "The batch size per GPU."})
    per_device_eval_batch_size: Optional[int] = field(default=1, metadata={"help": "The batch size per GPU for evaluation."})
    gradient_accumulation_steps: Optional[int] = field(default=8, metadata={"help": "The number of gradient accumulation steps."})
    gradient_checkpointing: Optional[bool] = field(default=False, metadata={"help": "Whether to use gradient checkpointing."})

furthermore , this is how I'm instantiating my model:
self.base_model = AutoModelForCausalLM.from_pretrained(
            pretrained_model_name_or_path=self.script_args.model_name,
            quantization_config=self.bnb_config,
            device_map="auto",#{"": Accelerator().local_process_index},
            trust_remote_code=True,
            # torch_dtype=torch.float16,
            # use_flash_attention_2=False
        )

I cannot use PEFT or Gradient Checkpointing as Phi models are not supported.

using Accelerator().local_process_index will give OOM error while "auto" actually starts the training but takes up 35GB of VRAM with all 4 GPU's at about 35% utilisation

I managed to run Phi on a T4, but Phi currently does not support gradien cehckpoitning, so batch_size = 1 and max_seq_len of 512

If you use phi models from @susnato https://huggingface.co/susnato/phi-1_5_dev you should be able to use GC without any problem

Yes, now that Phi is integrated into transformers, first please install transformers from the main, then you can run -
from transformers import PhiForCausalLM

model = PhiForCausalLM.from_pretrained("susnato/phi-1_5_dev",  quantization_config=self.bnb_config) # the weights are currently being transferred to microsoft/phi-1_5
print(model.model.gradient_checkpointing) # False

# Enable gradient Checkpointing
model.gradient_checkpointing_enable()

print(model.model.gradient_checkpointing) # this should be True now.

If you want to freeze some internal layers alongside then please add this kwarg -
model.gradient_checkpointing_enable(gradient_checkpointing_kwargs={"use_reentrant": False})

Even after enabling GC, if you don't see any reduction in the GPU memory then please tag me I will look into it.

The only downside of using Phi from the library is that you won't get support for Flash Attention. But it will be solved by this PR.


Error when hiring only one item

If only hiring one item, you can't submit the order.
When adding items to your order, it creates a new hidden input named "id[]", however, this will only create an array (on the PHP side) if there ends up being more than one item in the array, otherwise I think it's just a direct value.
Should be quite easy to check to see if $request->id  is an array before calling foreach($request->id) on BookingController.php line 393

Fixes as far as I can tell


Updated UG and minor additions to test codes.

Added StorageController test mode for testing purposes.
Changed the maximum character limit for notes to 150.

Coverage increased (+0.04%) to 56.004% when pulling 6697db6896c47fc9eac40266bb7833a7a152ed3e on jomcruz93:add-character-limit-text-area into 123c4c0879b74825feabcd75fb87d572edf6c8fc on CS2113-AY1819S1-T16-1:master.


Better error reporting if we fail to load the commands because of an exception

Previously, if an exception was thrown (e.g. connection refused - API is offline, connection timeout, etc.). we wouldn't correctly log the reason for the loading failure (the error would just say null).

+1


Allow react-i18n to generate translations for specific locale(s)

Overview
We're trying to:

Slim down the webpack metadata required to serve react-i18n translations in large apps
Get all users on an equal footing (right now, everyone has to download the fallback locale's translations)

To achieve this, I think that we need a webpack plugin that:

Accepts locale(s) + fallback locale
Includes translations for the locale directly beside their component in the JS bundle
Merges in fallback translations for anything missing from the locale's

@michenly is already working on a webpack plugin, and this could possibly be an extension of that.
Type

[x ] New feature

Motivation
Really big download sizes in our larger apps :/

cc @lemonmade because you have useful thoughts on implementation details

So, my vision for the "final" state of this would be a Webpack plugin that has configurable behaviour for how to handle an empty useI18n/ withI18n, because how to handle those translations is likely something that depends on the size of the app and how fine-grained they have split their translations.
Our default approach could continue being what it is now: add a sync import for the fallback translation, use it as the fallbackTranslations option, create a unique ID based on the file for the id option, and create a function that does a lazy-once async import for any other locale. However, we definitely want at least one other option: sync import a fallback translation for the fallbackTranslations option, unique ID from the name of the file for the id option, and a sync import for a specified locale (build per locale; what we want for web). Conceivably, this option could be further refined in the future to group all locales with the same language code together. Additionally, we can think of other strategies that might be effective (sync import all translations, sync import a list of "priority 1 languages" and async the rest, etc, but that can come from actual needs later on.

Fixed by @shopify/react-i18n's new from-dictionary-index mode: https://github.com/Shopify/quilt/pull/1197.  sewing-kit<EMAIL_ADDRESS>uses this mode is sewing-kit build --locale <locale> is specified.


The low contrast of icon dropdown in "Dark Mode" near "Source" environments

Expected Behaviour: The high contrast of icon dropdown in "Dark Mode" near "Source" environments
Actual Behaviour: The low contrast of icon dropdown in "Dark Mode" near "Source" environments
Reproduce Scenario (including but not limited to)
Steps to Reproduce

Install extension
Go to project folder
Select any spreadsheet file
Click "Preview"/"Publish" -> "Preview"/"Publish"
Click "Open URL"
Click on "Preview"/"Live" dropdown button (Extension must be in Dark mode)
Select any PDF file > Preview > Preview > Open URL > Open Environments dropdown

Platform and Version
Chrome 131.0.6778.86
Sidekick v7.1.2

:tada: This issue has been resolved in version 7.1.3 :tada:
The release is available on:

GitHub release
v7.1.3

Your semantic-release bot :package::rocket:


[release] v6.0.0-alpha.4

[x] I have followed (at least) the PR section of the contributing guide.

Looks good to me! I'd maybe wait for Jun's fix on https://github.com/mui/material-ui/pull/42003, though.

Yep, I am aware of it, will hold off the release till it's merged.

Looks good to me! I'd maybe wait for Jun's fix on https://github.com/mui/material-ui/pull/42003, though.

Yep, I am aware of it, will hold off the release till it's merged.

It's merged.


Express stm32 USART DMA channel requirements within type system

Currently one can pass any DMA channel to Uart::new, no matter the hardware requirements. Though no DMA functions be available on the returned type, as uart::{Write, Read} trait implementations do constrain DMA channels to be correct.
The current situation is ideal IMO, compile-time error would be nicer. Not sure if it can be expressed through rust type system though

Agreed!
Maybe splitting new into new and new_with_dma would work, so that new_with_dma can have the right trait bounds.


[dif][spi] Data being successfully sent when TX FIFO is both full and empty

When applying the diff at the bottom of this issue I see the following prints when running the hello_world example:
I00012 dif_spi_device.c:602]     STATUS: 0x3e
I00013 dif_spi_device.c:603]     TXF_PTR: 0x3600388
I00014 dif_spi_device.c:604]     TXF_ADDR: 0x7ff0400
I00015 dif_spi_device.c:605]     ASYNC_FIFO_LEVEL: 0x80000

Decoding this shows that:
TXF_ADDR->base = 0x400
TXF_ADDR->limit = 0x0x7ff
TXF_PTR->rptr = 0x388
TXF_PTR->wptr = 0x360
With the expectation that we will write 0x3a40388 to TXF_PTR after the transaction to indicate to the SPI device that there is data to send.
The status register is indicating that:

rxf_full -> 0 -> RX FIFO is not full, probably true
rxf_empty -> 1 -> RX FIFO is empty, probably true
txf_full -> 1 -> TX FIFO is full, but we haven't written anything yet
txf_empty -> 1 -> TX FIFO is also empty?

Why/how is the TX FIFO both full and empty, yet the dif is successfully writing more data to the SPI device?
Diff to get debugging information
diff --git a/sw/device/lib/dif/dif_spi_device.c b/sw/device/lib/dif/dif_spi_device.c
index c97ba8a9..03ac41dc 100644
--- a/sw/device/lib/dif/dif_spi_device.c
+++ b/sw/device/lib/dif/dif_spi_device.c
@@ -7,6 +7,8 @@
 #include "sw/device/lib/base/bitfield.h"
 #include "sw/device/lib/base/memory.h"
 #include "spi_device_regs.h"  // Generated.
+#include "sw/device/lib/arch/device.h"
+#include "sw/device/lib/base/log.h"

 const uint16_t kDifSpiDeviceBufferLen = SPI_DEVICE_BUFFER_SIZE_BYTES;

@@ -586,6 +588,10 @@ static size_t spi_memcpy(const dif_spi_device_t *spi, fifo_ptrs_t *fifo,
                                      bytes_to_copy);
     } else {
       // `byte_buf` -> SPI device buffer
+      LOG_INFO("    STATUS: 0x%x", mmio_region_read32(spi->base_addr, SPI_DEVICE_STATUS_REG_OFFSET));
+      LOG_INFO("    TXF_PTR: 0x%x", mmio_region_read32(spi->base_addr, SPI_DEVICE_TXF_PTR_REG_OFFSET));
+      LOG_INFO("    TXF_ADDR: 0x%x", mmio_region_read32(spi->base_addr, SPI_DEVICE_TXF_ADDR_REG_OFFSET));
+      LOG_INFO("    ASYNC_FIFO_LEVEL: 0x%x", mmio_region_read32(spi->base_addr, SPI_DEVICE_ASYNC_FIFO_LEVEL_REG_OFFSET));
       mmio_region_memcpy_to_mmio32(spi->base_addr, mmio_offset, byte_buf,
                                    bytes_to_copy);
     }
diff --git a/sw/device/lib/dif/meson.build b/sw/device/lib/dif/meson.build
index 0e8be9db..31df2edf 100644
--- a/sw/device/lib/dif/meson.build
+++ b/sw/device/lib/dif/meson.build
@@ -52,7 +52,7 @@ sw_lib_dif_spi_device = declare_dependency(
       hw_ip_spi_device_reg_h,
       'dif_spi_device.c',
     ],
-    dependencies: [sw_lib_mmio],
+    dependencies: [sw_lib_mmio, sw_lib_base_log],
   )
 )

Closing as stale. Please re-open if this question is still relevant with the current hw/sw implementation.


Branch to collect post-fork improvements

Any PRs that change the consensus layer, unless they are high priority bug fixes that should be included on mainnet prior to the hard fork transition, should be opened against this one (rather than against master). We will collect them here, and after the transition has happened, merge them into master. This is just a temporary measure to make sure that we can respond quickly should any problem require urgent fixes.

bors merge


[Project] Prepare deprecating old Jobs

Let's prepare deprecating old Jobs.
ONE-vscode-DCO-1.0-Signed-off-by: Yongseop Kim<EMAIL_ADDRESS>

This works well with onecc. I confimed.

I think this is necessary before https://github.com/Samsung/ONE-vscode/pull/513.

@YongseopKim I wonder what would be the difference between old Job and new Job? Could you explain a bit? Thanks.

Maybe related to https://github.com/Samsung/ONE-vscode/issues/536

New job: JobConfig, JobInstal, ..
Old Job: JobImport, JobCodegen, ...


[WIP][Issue #2]: Created swagger codegen module to generate notebooks.

This project will contain a swagger module for generating notebooks and converting them into microservices. Work to be done:

[x] Create a swagger module to generate notebooks
[x] Bundle the notebooks into a docker container
[x] Implement the swagger pet store API to provide a comprehensive example

Is this ready to go in? Need more pet store impl?

Yes, we still need pet store impl. Have some work items I need to open before we can complete the implementation. Will open those today.

TODOs on this PR to get it in:

[ ] Copyright header for Jupyter
[ ] pip install 0.2.0 kernel_gateway
[ ] notebook showing the pet store implementation (but not with error codes, headers, etc. see issue #2 )

I have some changes to the kernel gateway project I am going to push. Will update here when I open the PR

Issue jupyter-incubator/kernel_gateway/pull/52 and PR jupyter-incubator/kernel_gateway/pull/54 are the only bits needed to be merged into the kernel gateway for this demo to fully work.
@parente, since the features needed are not in the 0.2.0 release should we do anything special in the Dockerfile or just assume the changes will get there eventually?

We can release a 0.2.1. That's not hard. What's the alternative with the Dockerfile?

I was thinking we could point to the repo to install and when 0.3.0 is cut update the Dockerfile to do a normal pip install

Pointing to master is fine for a demo.

Alright, @parente this is GTG :fire: .

Since this is a demo, I'm going to put this in and then give it a shot. If there's problems, I'll open issues.


Sql catalog

This PR implements the basic operations for a Sql catalog. The implementation uses the sqlx crate which enables Postgres, MySQL and Sqlite.
The update_table method is to be implemented later.

PTAL @liurenjie1024 @Xuanwo @ZENOTME @Fokko

cc @JanKaul Is this pr ready for review or you need to do more updates?

I have to add a couple of more changes. I'll notify you when I'm finished.

@JanKaul WDYT? I think this PR is ready for review, I can add the update and delete in a separate PR.

@JanKaul WDYT? I think this PR is ready for review, I can add the update and delete in a separate PR.

Cool, I'll take a look first.

Thank you all for your helpful comments. I think the PR is ready for review again.
@liurenjie1024 @sdd @odysa @ZENOTME @martin-g

the bind placeholder ? will not work with pg databases, which use $1..$N format unfortunately. sqlx docs

Thanks for bringing this up, I haven't thought about it. It's weird though, the test is using sqlite which it's supposed to not work with ? according to the docs.


OR 2023 Child Care Credit

OR 2023 Child Care Credit
Called the Oregon  Working Family  Household and Dependent Care Credit.
This joint return has two employed taxpayers and a single dependent with $2000 in child care expense. I don't think the emulator has calculated the WFHDC.
http://taxsim.nber.org/out2psl/or2
https://taxsim.nber.org/historical_state_tax_forms/OR/2023/schedule-or-wfhdc_101-195_2023.pdf
https://www.oregon.gov/dor/forms/FormsPubs/publication-or-wfhdc-tb_101-458_2022.pdf

We calculate it as or_working_family_household_and_dependent_care_credit: https://github.com/PolicyEngine/policyengine-us/blob/master/policyengine_us/variables/gov/states/or/tax/income/credits/wfhdc/or_working_family_household_and_dependent_care_credit.py
Where would we manually map that to a list of CDCCs @noman404?

I am reviving this issue with the test jig sending the emulator the correct values.
Called the Oregon Working Family Household and Dependent Care Credit.
This joint return has two employed taxpayers and a single
dependent with $2000 in child care expense. Regular tax
is 1596. TaxAct sees 708 (exemption credit) 332 (EIC) and
1280 (child care) which sends the net to -724 but the emulator
has a positive tax.
http://taxsim.nber.org/out2psl/or2
https://taxsim.nber.org/historical_state_tax_forms/OR/2023/schedule-or-wfhdc_101-195_2023.pdf
https://www.oregon.gov/dor/forms/FormsPubs/publication-or-wfhdc-tb_101-458_2022.pdf

I am closing this issue and will post another that is similar but perhaps clearer.


remove extra whitespace

Docs Build status updates of commit 935593e:
:warning: Validation status: warnings

File
Status
Preview URL
Details

msteams-platform/concepts/apps/apps-package.md
:warning:Warning

Details

msteams-platform/concepts/connectors/connectors-using.md
:warning:Warning

Details

msteams-platform/resources/schema/manifest-schema-migrate.md
:warning:Warning

Details

msteams-platform/concepts/messaging-extensions/search-extensions.md
:white_check_mark:Succeeded

msteams-platform/resources/schema/manifest-schema.md
:white_check_mark:Succeeded

msteams-platform/concepts/apps/apps-package.md

Line 10: [Warning] Invalid file link:(~/concepts/apps/apps-upload).
Line 19: [Warning] Invalid file link:(~/get-started/get-started-app-studio).
Line 21: [Warning] Invalid file link:(~/resources/schema/manifest-schema).
Line 60: [Warning] Invalid file link:(~/troubleshoot/troubleshoot).

msteams-platform/concepts/connectors/connectors-using.md

Line 33: [Warning] Invalid file link:(~/concepts/cards/cards).
Line 186: [Warning] Invalid file link:(~/concepts/apps/apps-package#icons).
Line 251: [Warning] Invalid file link:(~/publishing/apps-publish).

msteams-platform/resources/schema/manifest-schema-migrate.md

Line 9: [Warning] Invalid file link:(~/resources/schema/manifest-schema-v04).
Line 19: [Warning] Invalid file link:(~/concepts/apps/apps-package#icons).

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 26e7d16:
:warning: Validation status: warnings

File
Status
Preview URL
Details

msteams-platform/concepts/connectors/connectors-using.md
:warning:Warning

Details

msteams-platform/resources/schema/manifest-schema-migrate.md
:warning:Warning

Details

msteams-platform/concepts/apps/apps-package.md
:white_check_mark:Succeeded

msteams-platform/concepts/messaging-extensions/search-extensions.md
:white_check_mark:Succeeded

msteams-platform/resources/schema/manifest-schema.md
:white_check_mark:Succeeded

msteams-platform/concepts/connectors/connectors-using.md

Line 33: [Warning] Invalid file link:(~/concepts/cards/cards).
Line 186: [Warning] Invalid file link:(~/concepts/apps/apps-package#icons).
Line 251: [Warning] Invalid file link:(~/publishing/apps-publish).

msteams-platform/resources/schema/manifest-schema-migrate.md

Line 9: [Warning] Invalid file link:(~/resources/schema/manifest-schema-v04).
Line 19: [Warning] Invalid file link:(~/concepts/apps/apps-package#icons).

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit d40949d:
:warning: Validation status: warnings

File
Status
Preview URL
Details

msteams-platform/resources/schema/manifest-schema-migrate.md
:warning:Warning

Details

msteams-platform/concepts/apps/apps-package.md
:white_check_mark:Succeeded

msteams-platform/concepts/connectors/connectors-using.md
:white_check_mark:Succeeded

msteams-platform/concepts/messaging-extensions/search-extensions.md
:white_check_mark:Succeeded

msteams-platform/resources/schema/manifest-schema.md
:white_check_mark:Succeeded

msteams-platform/resources/schema/manifest-schema-migrate.md

Line 9: [Warning] Invalid file link:(~/resources/schema/manifest-schema-v04).
Line 19: [Warning] Invalid file link:(~/concepts/apps/apps-package#icons).

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit 8ae1549:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/concepts/apps/apps-package.md
:white_check_mark:Succeeded

msteams-platform/concepts/connectors/connectors-using.md
:white_check_mark:Succeeded

msteams-platform/concepts/messaging-extensions/search-extensions.md
:white_check_mark:Succeeded

msteams-platform/resources/schema/manifest-schema-migrate.md
:white_check_mark:Succeeded

msteams-platform/resources/schema/manifest-schema.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Thanks for the help!


Best Location for Activating Orbit Plugin

Wanted to get your opinion on the best location for activating the orbit plugin.

Im currently using the above code in the footer template after all other orbit and jquery files have been loaded. In the past, I have put these in the header template. So, I wanted to confirm that Im doing this in the best practice for the reverie theme and also wanted to just ask why its all in the footer and not the head? Is it purely so that the page content loads first?
Thanks again for reverie and for the support. Im doing some testing and if things continue to go well I will be using for a very large website and look forward to sharing the process on this forum.

As you said, files in the header will slow down the visual element of the page which is actually what visitor sees first. I guess the better answer is here and more technical stuffs are explained: http://stackoverflow.com/a/5329895/949832
Glad you like Reverie :)


Include end time for resources that complete

Is there an existing issue for this?

[X] I have searched the existing issues

Is your feature request related to a problem? Please describe the problem.
When a resource finishes, it would be nice if an End Time field could be added to the resource along side the Start Time
Describe the solution you'd like
I'd like to see an End Time alongside the Start Time.  A Duration would also be nice.

Additional context
Sample code:
builder.AddExecutable("oneSec", "pwsh", ".",
    "-NoProfile",
    "-NoLogo",
    "-NonInteractive",
    "-Command",
    "Write-Host 'Starting Migration'; Start-Sleep -Seconds 1; Write-Host 'Done'");

@karolz-ms does DCP expose the end time?

@davidfowl it does https://github.com/dotnet/aspire/blob/main/src/Aspire.Hosting/Dcp/Model/Executable.cs#L101 and https://github.com/dotnet/aspire/blob/main/src/Aspire.Hosting/Dcp/Model/Container.cs#L277

@JamesNK might be an interesting addition after we add Stop/Start


Flatline with no matches

Hi, I'm clearly missing something simple but:
my_rule.yaml
type: flatline
index: index-*
threshold: 1
timeframe:
  hours: 24
use_count_query: true
doc_type: doc
filter:
- query:
    query_string:
      query: "application:\"nonsense\""

I would expect that: elastalert-test-rule my_rule.yaml would say something like "An abnormally low number of events .." because obviously there are no events with field and value "nonsense" and there will never be. However, if I change timeframe to hours:1, strangely it hits the rule and says  "An abnormally low ...". To make sure I created that original rule and left for days, but still no alerts...

Same issue here, left it running, still no alerts unless i set it to 1-23 hour. When set to 24 hours no alerts happen. My logstash indexes are in the following format, it is probably related to that:  logstash-%Y.%m.%d

@Qmando , If i set it to 23 hours it triggers immediately, when i change it to 24 hours it no longer triggers. Waited a weekend otherwise for it to trigger without success. And I remember this used to be working in the past (same version) then it stopped working. I deleted the index and metadata without luck though.

You are on version 0.1.31? I'll try to reproduce this and get back to yall.

Not the latest version, i will upgrade tomorrow and post my rule configuration, thanks.

I can't reproduce this using your rule config:
$ python -m elastalert.elastalert --rule test.yaml --debug --start 2018-05-29
INFO:elastalert:Note: In debug mode, alerts will be logged to console but NOT actually sent.
                To send them but remain verbose, use --verbose instead.
INFO:elastalert:Starting up
INFO:elastalert:Queried rule dffgdfgd from 2018-05-28 17:00 PDT to 2018-05-28 18:00 PDT: 0 hits
.....
INFO:elastalert:Queried rule dffgdfgd from 2018-05-30 11:00 PDT to 2018-05-30 11:08 PDT: 0 hits
INFO:elastalert:Skipping writing to ES: {'rule_name': u'dffgdfgd.all', '@timestamp': '2018-05-30T18:08:43.241762Z', 'exponent': 0, 'until': '2018-06-02T18:08:43.241754Z'}
INFO:elastalert:Alert for dffgdfgd at 2018-05-30T02:00:00Z:
INFO:elastalert:dffgdfgd

An abnormally low number of events occurred around 2018-05-29 19:00 PDT.
Between 2018-05-28 18:55 PDT and 2018-05-29 19:00 PDT, there were less than 1 events.

@timestamp: 2018-05-30T02:00:00Z
count: 0
key: all
num_hits: 0
num_matches: 36

INFO:elastalert:Ignoring match for silenced rule dffgdfgd.all
...

Same exact thing if I try 23 hours. Is this not what you are doing? Can you show logs from when your 23 hour timeframe works right away and 24 hours doesnt?

Hi,
So i was using elastalert test when the 23 hour timeframe setting succeeded, otherwise i see it does not work just like with 24 hours.
1. When I use 23 or 24 hours without specifying --start:
INFO:elastalert:Queried rule [int] No xyz Logs arrive from 2018-06-01 14:03 CEST to 2018-06-01 14:06 CEST: 0 / 0 hits
INFO:elastalert:Ran [int] No xyz Logs arrive from 2018-06-01 14:03 CEST to 2018-06-01 14:06 CEST: 0 query hits (0 already seen), 0 matches, 0 alerts sent

2. When I specify --start 2018-05-30 then SOMETIMES it says it is silenced but only sometimes
....
INFO:elastalert:Ignoring match for silenced rule [int] No xyz Logs arrive.all
INFO:elastalert:Ignoring match for silenced rule [int] No xyz Logs arrive.all
INFO:elastalert:Ran [int] No AdnIDM Logs arrive from 2018-05-30 02:00 CEST to 2018-06-01 14:03 CEST: 0 query hits (0 already seen), 149 matches, 0 alerts sent

3. Sometimes (randomly) it says "no matches" found instead of saying the rules is silenced (which it should not be because there were no alerts whatsoever).
4. When I run "elastalert-test-rule rules/xyz_logs_flatline.yaml --alert --config config.yaml" with 23 hours
Would have written the following documents to writeback index (default is elastalert_status):

silence - {'rule_name': u'[int] No AdnIDM Logs arrive.all', '@timestamp': datetime.datetime(2018, 6, 1, 12, 20, 35, 719367, tzinfo=tzutc()), 'exponent': 0, 'until': datetime.datetime(2018, 6, 1, 15, 20, 35, 719357, tzinfo=tzutc())}

elastalert - {'alert_info': {'type': 'email', 'recipients': ['xyz']}, 'alert_sent': True, 'match_body': {'count': 0, 'num_hits': 0, '@timestamp': '2018-06-01T11:35:35.385220Z', 'key': 'all', 'num_matches': 4}, 'rule_name': '[int] No xyz Logs arrive', 'match_time': '2018-06-01T11:35:35.385220Z', 'alert_time': datetime.datetime(2018, 6, 1, 12, 20, 35, 719526, tzinfo=tzutc())}

elastalert_status - {'hits': 0, 'matches': 4, '@timestamp': datetime.datetime(2018, 6, 1, 12, 20, 36, 358059, tzinfo=tzutc()), 'rule_name': '[int] No xyz Logs arrive', 'starttime': datetime.datetime(2018, 5, 31, 12, 20, 35, 385220, tzinfo=tzutc()), 'endtime': datetime.datetime(2018, 6, 1, 12, 20, 35, 385220, tzinfo=tzutc()), 'time_taken': 0.9462141990661621}

5.  When I run "elastalert-test-rule rules/xyz_logs_flatline.yaml --alert --config config.yaml" with 24 hours
Would have written the following documents to writeback index (default is elastalert_status):

elastalert_status - {'hits': 0, 'matches': 0, '@timestamp': datetime.datetime(2018, 6, 1, 12, 30, 5, 675743, tzinfo=tzutc()), 'rule_name': '[int] No xyz Logs arrive', 'starttime': datetime.datetime(2018, 5, 31, 12, 30, 5, 353628, tzinfo=tzutc()), 'endtime': datetime.datetime(2018, 6, 1, 12, 30, 5, 353628, tzinfo=tzutc()), 'time_taken': 0.2949259281158447}

Ty for the info. I'll take a look at this again.

Has anyone found a resolution to this?  My rule is set up like so:
name: {{ region_env_name }} Number of calls (5 min)
type: flatline
index: callflows-*
threshold: 1
timeframe:
minutes: 5
use_count_query: true
doc_type: doc
filter:
- query:
query_string:
query: "exists:CVPAppName AND CVPAppName:APP AND (CallType:7526 OR 7525)"
alert:
- "sns"
sns_topic_arn: "SNS_ARN"
From reading the documents I would think Flatline would send an alert because there are no hits matching the query above, however the elastalert if not firing.

Can you post logs? Run elastalert with --verbose for at least 5 minutes.

Hi Qmando, it actually ended up working, I just didn't give it enough time for elastalert to process the alert I guess.  Thanks for following up.

Same issue here. Our logstash index is not specifying days. that is the only difference.
No matches found. Our config is:
scan_entire_timeframe: true
# --- Begin Type Specific Rule Configuration ---
type: flatline
timeframe:
  days: 7
run_every:
  minutes: 10
threshold: 1
use_count_query: true

When i run:
elastalert-test-rule --alert /etc/elastalert/rules/testing.yaml  --days 7 doesn't alert
elastalert-test-rule --alert /etc/elastalert/rules/testing.yaml  --days 8 does alert
No matches found when not alerting. Ideas?
The index pattern we doing is index: index-*
Have checked few things already like the buffer_time

Update:
python -m elastalert.elastalert --rule test.yaml --debug --start 2019-06-08 works fine but doesn't when it is running as a daemon.
How is the start affecting to the search? Is it same as timeframe set in the config file?
timeframe:
days: 7

Wow what a coincidence.  We missed an outage this week because our dirt simple flatline rule refused to fire as per the documentation says it should.  Now the client is asking why we missed it.
While I appreciate the author work and willingness open source elastalert this is very frustrating.
I have wasted hours re-testing this with elastalert-test-rule and it only fires if --start is set for over 36 hours prior but it won't fire when running in production.
If my look back is set to 24 hours and my threshold is 3 and there have been no indexes logged for 12 hours prior then should have fired and should refire if I erase all the elastalert metadata and restart, but it doesn't.  At least that is how the documentation describes it.  It should be that simple.
Maybe this rule should marked as being an incomplete or broken so people don't trust being alerted to production system outage by using it until it get fixed.  Because it is definitely broken.
Also the results of running elastalert-test-rule and the actual running in production should be exactly the same.
I know the documentation mentions that the results could be different, but if that the case then running elastalert-test-rule is worthless because it not a valid test unless it produces the exact same result just like a unit test

name: "Testing 123 Potential System Outage"
http_post_static_payload:
alert_name: "Testing 123 Potential System Outage"
alert_form: "PotentialSystemOutageAlarm"
http_post_all_values: True
verify_certs: False
index: agent-publicipaddress-*
type: flatline
query_key: _index
doc_type: publicipaddress
threshold: 1
timeframe:
hours: 12
alert:

"debug"
"post"

I'm not the only one:
https://github.com/Yelp/elastalert/issues/2157
https://github.com/Yelp/elastalert/issues/2060
https://github.com/Yelp/elastalert/issues/2049


Can't open XR toolkit once in vr

Give us as much details as possible
Can't open the toolkit once dcs loads in VR. Using the Pimax crystal. Hitting cntrl F2 does nothing. The first time I downloaded
it and started dcs with the pimax for the very first time I saw the screen in VR. But now it never opens anymore. What did I do wrong?

You're probably running the game in OpenVR mode. Check the DCS forum for help (it's on a pinned thread called "DCS now supports OpenXR").


Batarang Models not showing scope | model editors

None of the angularjs sites I've visited will show the scope or model links in the Models pane of batarang.   For example, here's what I get from todomvc:

This prevents me from being able to edit any of the model data or do other cool things outlined in the screencast.
Chrome version: 27.0.1453.93
OS: OS X v10.8.3
Betarang version: 0.4.2, although I've also tried grabbing the latest from github and loading that directly.

This has been fixed in the new Scope rewrite.

I am experiencing this issue.
I go to Options > Show scopes and tick that
It correctly highlights the scoped elements on the page.
However, when I go to 'Models' tab, the Scopes list is empty and the root scope shows no Models i.e. it says "Select a scope to view its models"
I am using:
Chrome Version 47.0.2526.73 m
AngularJS Batarang (Stable) 0.4.3
Angular version: 1.4.9 implicit-superannuation

@JenTechSystems the "Stable" version is not a part of this project, someone forked us a while ago and refuses to update.


Jenkins server

Just to let you know, i added your project to my public jenkins, if you want to host more jenkins job on it just send me an email at<EMAIL_ADDRESS>and i'll create an user for you to customize your build jobs ;) https://ci.codemc.org/view/jammehcow/

Thanks for that. I'm replacing my current server which will be more than capable to host an instance.
I appreciate the offer though.


Text is not directly editable when selected

Instead of having to input the desired string into the "text" attribute of a text element, it would be really nice if it was possible to type directly into the text symbol.
It would be especially great if you could edit a text field from a symbol like this, then we'd be able to create forms!

Yes, this would be fantastic. I think it might be a heavy endeavor.
Text, like all the other graphics, is currently rendered by a DOM Canvas. So to make an editor, we need to track the pixel positions of all the letters, render a "fake" blinking text cursor appropriately, and respond appropriately to keyboard input. This is all possible, but a good deal of work.
Marijn Haverbeke, author of CodeMirror, has done a lot of work with making "fake" text editors. There are probably useful ideas on his blog.

https://github.com/danielearwicker/carota is a rich-text editor that runs in the canvas. At first glance, I thought it might be overkill, since it supports all the rich-text niceties rather than just "a line of single-styled text". But come to think of it, supporting rich text in diagrams really seems like a worthy goal to have.
Caveats: The project is not particularly active. I am not sure what integration would look like.


Add Padding to timeScale

I'm currently facing a difficulty with timeScale in the Lightweight Chart library. As of now, it seems there is no way to add padding to the timeScale. In my use case, it would be extremely beneficial to have some padding for better chart visualization and user experience.

Could you please highlight on the image where you would like to add padding?

Is it the top and bottom of the series? (can be done using scale margins, explained in this example: https://tradingview.github.io/lightweight-charts/tutorials/how_to/price-and-volume)
Left and right of the chart? (can be done using css on the container div)
Or is it the tick marks within the timescale?

Hi @SlicedSilver, thank you the quick response.

it is the tick marks within the timescale.

In my current project, I have the necessity to enhance the height of the timescale for better visibility and readability. Furthermore, adding padding to the right and left sides of the timescale would greatly improve the aesthetic appeal of the chart.
In my specific case, the container div's padding is set to zero due to the design requirements. Regrettably, this configuration results in the tick marks extending to the edges, causing a somewhat cluttered appearance that detracts from the overall presentation.
Thus, having the ability to customize the height and padding of the timescale would be a highly valuable feature, enhancing the adaptability of the library for various design needs and ultimately improving the user experience.

👍 In summary,

Option for vertical padding within the timescale (above and below the tick mark text).

This would adjust the height of the timescale automatically.

Improve rendering of the time scale tick marks so that they don't get cut-off at the edges.

Improving the rendering of the tick marks to prevent the labels getting cut-off would require some extra consideration on our side on how best to achieve this. In the static image case it is rather simple fix but since the chart is dynamic, and can be scrolled, extra attention is required to handle the transitions in and out for the labels while interacting with the chart.

Hello again @SlicedSilver
I wanted to report another  issue regarding with the time scale labels.
In the screen shot I am sharing below, some gap between two labels are larger than the others. Some texts are bolder while the others are thinner.
Is there any solution to those issues?

Hi @SlicedSilver,
It is possible to add a padding right for the series itself, so it would start with an offset. I see the property rightOffset on timescale which is calculated in bars and not convenient for my case. In my case the timescale range is not uniform and I can't calculate or set a specific visible range, I just show the data on the chart.
It is possible to add some sort of padding/margin for that purpose ?
Is this related to this issue ?

The screenshot is from Binance and I believe they're using the rightOffset because the data is normalized and they can

The chart shown in the Finance screenshot is actually a different library created by TradingView called 'Advanced Charts' More Info
To answer your question, you could use the barSpacing to know the width that each bar takes up in pixels and then calculate the right offset that you would like to use. i.e. you can use barSapcing to convert between pixels and barCount for the horizontal axis.
This can also be calculated if the zoom have changed by using the width of the timeScale, and the visibleLogicalRange. See the ITimeScaleApi interface for more details.

There is now an minimumHeight option for the timescale which can be set via its options.
https://tradingview.github.io/lightweight-charts/docs/api/interfaces/TimeScaleOptions#minimumheight

@SlicedSilver Is it possible to show limited number of timescale marks and with constant time increments

For example in the above screen shot we can see the times clustered and not uniform, I want to show minimum number of marks with a constant increment of a minute or so

One of the other additions to the library in v4.1 is the ability to customise the tick marks behaviour on the time scale. The test case show how to use this new API: https://github.com/tradingview/lightweight-charts/blob/master/tests/e2e/graphics/test-cases/horizontal-price-scale.js
If you look at the fillWeightsForPoints function, and returned the same weight for every point then you should get closer to your desired behaviour.
Additionally, if you used an additional series on the chart which contained only whitespace data (data with time values but no price values) which had a consistent number of data points then you can make the time scale linear. Since the library will plot a point for every data point provided, if your whitespace series had a data point every minute then the result would be a linear time scale.

Thank you very much, will experiment with whitespace data

@SlicedSilver Hi,
Thank you for bringing the new minimumHeight feature. When I use it the new appearance need padding too. How can I center the time scale labels?
Additionally, when I use this configuration;
    timeScale: {
      fixLeftEdge: true,
      fixRightEdge: true,
    },

the timescale labels overlaps as you can notice from the screenshot below.

The minimumHeight option is intended for use when you need to increase the timescale for drawing larger items with Plugins. There aren't any options to adjust the internal padding of the timescale at this time.
Regarding the overlapping of the timescale tick labels. I would recommend using the tickMarkMaxCharacterLength option to set a larger label length.


Rewrite pktprint in Go

Replace pktprint.py with a Go tool to pretty-print packets.
There is a header change upcoming that will change the packet format, these changes will not be ported to python as python services have been removed.

No longer exists


build(release): add release workflow

Change Overview
Release workflow will:

Create and push a release tag
Produce release using goreleaser with binaries and helm charts
Build docs from docs_new
Publish docs and helm charts index in the github pages

Main changes

Created a release workflow to run either on workflow_dispatch or on merge with specific commit message: pre-release: Update version to ... format for merge commits of PRs like https://github.com/kanisterio/kanister/pull/2814

This will create some cancelled workflows because of the way filters work on GHA, but hopefully that's not a big deal

Moved example image build from the goreleaser to a separate reusable workflow, so they no longer break the release even if they fail, we can even call this workflow separately from workflow_dispatch if we want
Added helm packages to the artifacts of the release, so they now hosted by github
Enabled new docs build and publish to github pages together with the helm charts index file @mlavi we will need to point our DNS to those locations now
Enabled ability to build pre-releases by supplying tag names like x.x.x-rc1. Mostly for test purposes. Those will be created as pre-releases in the github.

Changes in helm chart publishing
Helm charts are now published as release artifacts. Helm chart index is published in the github pages together with docs (just in a separate directory).
Helm chart index source is still taken from charts.kanister.io, so before we change the DNS it will include references to the old charts in S3.
For some time I believe we will be publishing to both S3 and github release/github pages.
Pull request type
Please check the type of change your PR introduces:

[ ] :construction: Work in Progress
[ ] :rainbow: Refactoring (no functional changes, no api changes)
[ ] :hamster: Trivial/Minor
[ ] :bug: Bugfix
[ ] :sunflower: Feature
[ ] :world_map: Documentation
[ ] :robot: Test
[x] :building_construction: build

Test Plan
Test release
There is a ci-release-test branch which can be used to create test releases.
One way to do that is to run workflow dispatch:
gh workflow run release.yaml -r ci-release-test -F release_tag=0.107.0-rc1 -F draft_release=true

Another is to push a commit with the pre-release message format: pre-release: Update version to 0.107.0-rc2 (placeholder) (don't forget the placeholder, merge commits have the PR reference there)
This will create a tag, build images and publish a draft release on github. You can track the progress in https://github.com/kanisterio/kanister/actions/workflows/release.yaml
If tag already exists in github, the workflow will fail, so please use new tags when testing.
Example runs:

Workflow dispatch: https://github.com/kanisterio/kanister/actions/runs/8791390369

Push: https://github.com/kanisterio/kanister/actions/runs/8791400326

[x] :muscle: Manual

[ ] :zap: Unit test

[ ] :green_heart: E2E

Who can run the workflow and/or create a PR with the title?

Created a release workflow to run either on workflow_dispatch or on merge with specific commit message: pre-release: Update version to ... format for merge commits of PRs like https://github.com/kanisterio/kanister/pull/2814

Who can run the workflow and/or create a PR with the title?
Currently people with write access to the repo can run workflow dispatch. Anyone can create PR with this title but the workflow only runs if it's merged in master, which has branch protection rules.
We can extend it with tag protection rules and environments if necessary.
Currently I believe anyone with write access to the repo can create a release anyway.

Test run can be found here: https://github.com/kanisterio/kanister/actions/runs/9699450217


chat items

add a feature to show the item name in chat without install other plugin

I added it in version 1.0.2 (https://github.com/Artillex-Studios/AxKills/releases/tag/1.0.2)


Remove JSDocTag#atToken

We don't really have a good reason for adding these in the parser. We don't typically add non-optional tokens to the parse tree, preferring to do so in services if necessary (see function createChildren in services.ts).

@andy-ms agree with this decision. Maybe the breaking changes should be updated though? https://github.com/Microsoft/TypeScript/wiki/API-Breaking-Changes (honestly, doubt this is affecting anything other than my tests)
Also, should this token be returned when calling .getChildren()?


Cypress/E2E: Upgrade to<EMAIL_ADDRESS>and other dependencies

Summary

Upgrade to<EMAIL_ADDRESS>and other dependencies

Ticket Link
none
Release Note
NONE

/e2e-test

Successfully triggered e2e testing!
https://git.internal.mattermost.com/qa/cypress-ui-automation/-/pipelines/218004

/e2e-test

Successfully triggered e2e testing!
https://git.internal.mattermost.com/qa/cypress-ui-automation/-/pipelines/218250

/e2e-test

Successfully triggered e2e testing!
https://git.internal.mattermost.com/qa/cypress-ui-automation/-/pipelines/218308

/e2e-test

Successfully triggered e2e testing!
https://git.internal.mattermost.com/qa/cypress-ui-automation/-/pipelines/218316

Thanks @furqanmlk, looking forward to that! You may use cy.uiSaveConfig(). See https://github.com/mattermost/mattermost-webapp/pull/10368/files where I did the initial attempt to update those.


Control ideas

Hamburger menu
Notificationspanel

...what exactly do you have in mind for a "Notificationspanel"? If you could provide a mockup, that would be ideal. It doesn't have to be super high-fidelity - even an mspaint scribble will do, as long as it's readable. If not, even a written description would help - as-is, I have no clue what you're suggesting.

a simple animatable panel that display the latest notification like vscode


whitespace in credentials.local.inc.php

closing tags causes whitespace issues (being sent before header, etc.) when appended

Fixed 1b15788


Fix autoscaling APIs

Correct request & response definition for the autoscaling namespace.

@swallez , LGTM but please fix the formatting

Ran make contrib. Please cross-check!


v1.0

Surprisingly a short file. Pretty cool

Nice refactor

Thanks. It seems to be working now which is awesome.

This is really cool actually, nice use-case for go. My one week of go experience doesn't see anything wrong with this 🎉

🚢


 See https://www.conventionalcommits.org/

:no_entry_sign:
Found some Code-Style issues. Please run ./vendor/bin/php-cs-fixer fix on your branch

Notice

:book:
You have more than more commit. Please consider squashing your commits.


Remove the includenodoc and customcarditem directives

According to grep, these weren't actually used.
CustomCalloutItemDirective is used though. It's used in a few other projects too, so I'm likely going to move this to qiskit_sphinx_theme in its 1.13 release so that you don't have to copy and paste the code. Tracked by https://github.com/Qiskit/qiskit_sphinx_theme/issues/323.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 92.971%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
2672

Relevant Lines:
2874

💛  - Coveralls

@Mergifyio backport stable/0.6


[$250] Workspace - Missing options from settings if a WS is created on another device with same acc

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number: <IP_ADDRESS>
Reproducible in staging?: Y
Reproducible in production?: Y
Found when executing PR : https://github.com/Expensify/Expensify/issues/281020
Logs: https://stackoverflow.com/c/expensify/questions/4856
Issue reported by: Applause-Internal team
Action Performed:

Device 1: Log in with a new Gmail account
Device 2: Log in with the same account
Device 1: Create a workspace
Device 2: Navigate to the workspace settings
Device 2: Click on members

Expected Result:
All settings should be available for the second device.
Actual Result:
Missing options from settings if a WS is created on another device with same account. The missing options are: "Categories" and "More features" from the settings tab, "Description", "Share" and "Delete" from the "Profile tab, "Invite members" from the "Members" tab. The settings are restored after relogging.
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android: Native
[ ] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[ ] MacOS: Desktop

Screenshots/Videos
https://github.com/Expensify/App/assets/115492554/212a863c-7954-4a6f-b01e-ba1b73d78ba4
https://github.com/Expensify/App/assets/115492554/60cb13c0-4921-4bea-8bdb-ab8aa9a5ea1f

View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~01133e97d014430d38
Upwork Job ID:<PHONE_NUMBER>691853824
Last Price Increase: 2024-05-05
Automatic offers:

eh2077 | Reviewer | 0

Issue OwnerCurrent Issue Owner: @

We think this issue might be related to the #collect project.

@trjExpensify - what you reckon - this wave-collect / a priority?

Yeah, we should fix that. Very strange. It's treating the user on the second device like a mere member, not an admin. The majority of the workspace settings aren't visible to a member, only profile (read-only) and members list (without the ability to add anyone new).

@dragnoir 's proposal looks good to me. I agreed with their RCA and the solution. Implementation details can be discussed in PR.
🎀👀🎀 C+ reviewed

Can you explain why we have 2 employeeList objects? It looks like they have the same information...?

Okay. Should we always look at the policy employee list for isAdmin? My concern here is that looking for something in 2 places seems like an anti-pattern. We should know where data is, and always look there. Ideally we should not have any duplicated data either

I found that policyDraft also has employeeList
https://github.com/Expensify/App/blob/0c2d5130727105535ee16ec0b5e932456ce3fea5/src/libs/actions/Policy.ts#L2086-L2112

Should we always look at the policy employee list for isAdmin?

So, I think it'll be neat to only check employeeList.
cc @stitesExpensify @dragnoir

If we use employeeList from Policy, the value will be available from all devices.

@dragnoir This is exactly what I mean given they're coalesced here
https://github.com/Expensify/App/blob/98d85e0863fe62292a7a086f6cf4bfb4c5b59fdc/src/pages/workspace/WorkspaceInitialPage.tsx#L81

Not overdue, waiting for @stitesExpensify to check if it's good to assign or not

Hmm @laurenreidexpensify do you know how we can get a new CME assigned to this? I'm about to go OOO and need someone to take over for me while I'm gone

🎀👀🎀 C+ reviewed

@eh2077 can you do me a favour and repaste this message - https://github.com/Expensify/App/issues/40686#issuecomment-2088114408 - so we can get another internal eng assigned to watch while @stitesExpensify is out thanks

Sure
🎀👀🎀 C+ reviewed

Still working on the PR

PR merged today, waiting for next deploy.

Payment Summary -

contributor @dragnoir paid in upwork $250
C+ @eh2077 paid in upwork $250

@eh2077 please confirm the the checklist steps above thanks

@eh2077 please confirm the the checklist steps above s so we can close this out thank

Checklist

[x] [@eh2077] The PR that introduced the bug has been identified. Link to the PR: I don't think there's a PR which is responsible for it. This is a missing edge case we choose to polish
[x] [@eh2077] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment: N/A
[x] [@eh2077] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion: N/A
[x] [@eh2077] Determine if we should create a regression test for this bug. No

@laurenreidexpensify Checklist completed, thanks for the bump


It would be nice and logical if these buttons are also in the gas-load page

Something like this would be nice. Would probably be a fair bit of work to get the gas charts to reload when changing strategies, but it would be cool:

Would probably be a fair bit of work to get the gas charts to reload when changing strategies

Wouldn't it be just as simple/difficult as with the electricity load chart?

Wouldn't it be just as simple/difficult as with the electricity load chart?

No, the whole technical background is entirely different. Also what would be the expected outcome in the gas chart; multiple charts with strategies enabled and disabled? The thing is that that would take a considerable amount of time like @antw said.


Fixed formatting of Kapacitor Event Handler Setup guide

Ah got it. I think it's an ESL issue...either way, agree that you should
use their language.
On Wed, Sep 19, 2018 at 1:21 PM Scott Anderson<EMAIL_ADDRESS>wrote:

@sanderson commented on this pull request.
In content/kapacitor/v1.5/guides/event-handler-setup.md
https://github.com/influxdata/docs.influxdata.com/pull/1835#discussion_r218949874
:

-For example: mushroomKap_bot.
->
-BotFather responds:
->

Done! Congratulations on your new bot. You will find it at t.me/. You can now add a description, about section and profile picture for your bot, see /help for a list of commands. By the way, when you've finished creating your cool bot, ping our Bot Support if you want a better username for it. Just make sure the bot is fully operational before you do this.
->

+1. Search for the @BotFather username in your Telegram application.
+
+2. Click Start to begin a conversation with @BotFather.
+
+3. Send /newbot to @BotFather.
+

@BotFather responds:

Alright, a new bot. How are we going to call it? Please choose a name for your bot.

This is the response that comes from Telegram's bot and is the actual
verbiage they use. I think "How" is used because referencing a bot is often
referred to as "calling" a bot. It's referring to the behavior of
referencing the bot rather than naming it. Confusing, I know...
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
https://github.com/influxdata/docs.influxdata.com/pull/1835#discussion_r218949874,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AoZIW2z9G8zNq1sODmJTMLCiq57D8RwYks5ucqc6gaJpZM4WwsJL
.


ask to install missing VC/SDK for 64-bit support

detect missing VC/SDK before installation and offer to
start the VS2013 installation

Wait, this shouldn't go into 2.068 yet.


Plot tiling stats

Adds a script to plot tiling stats

Turns out the "error" part has little meaning in practice. Still useful to have something ready to plot it in case we come up with a metric that does make sense.


#22: Focus query mode

#22

Something became wrong
https://user-images.githubusercontent.com/581023/164701639-9c630172-cfda-4d10-901e-300731374117.mp4

This seems fine


Add checks to help determine how trustworthy an account is

For example:

Check if it's banned on ksoft.si
Check the account age
Check if the account has a profile picture
Check if multiple people are joining with the same profile picture?

KSoft.Si and Account Avatars are implemented, Account Age will come in a future release. I'll close this issue when it's properly tested.


Update datetimeoffsetfromparts-transact-sql.md

Text revisions to tighten and optimize the reading flow of the material. I fixed an existing bug in the first SQL Server example result set.

@fbsolo : Thanks for your contribution! The author, @edmacauley, has been notified to review your proposed change.

@fbsolo Thanks for your recommended improvements. There is a merge conflict that I need to resolve in this PR by choosing between your changes and the live changes. I will do my best to go through that and make the changes. There are also a few other changes that I have some questions about. I'll do that after. Thanks.

@fbsolo : Thanks for your contribution! The author, @edmacauley, has been notified to review your proposed change.

@fbsolo OK, I reviewed the remaining changes and they look good. Thanks for your contribution! Signing off.


Allow building swift-syntax in C++ interop mode

If swift-syntax is being built with -cxx-interoperability-mode=default, it fails because of annot initialize a variable of type 'AtomicBool *' with an rvalue of type 'void *'. Add a cast here to fix the build issue.
rdar://129252735

@swift-ci Please test


Question about namespaces for ts-to-jsdoc

Hey again,
I have a few projects that use namespaces in JSDoc. Are you open to the idea of generating namespaces in JSDoc?
For example:
/** Namespace description */
export namespace Example {
  /** Type description */
  export type SubType = {
    /** Property description */
    name: string
  }

  /** Interface description */
  export interface SubInterface {
    /** Property description */
    label: string;
  }
}

would become:
/**
 * Namespace description
 * @namespace Example
 */
/**
 * Type description
 * @typedef {Object} Example.SubType
 * @property {string} name Property description
 */
/**
 * Interface description
 * @typedef {Object} Example.SubInterface
 * @property {string} label Property description
 */

Sure, feel free to open a PR!

@futurGH I took a stab at a PR to implement this: https://github.com/futurGH/ts-to-jsdoc/pull/49. Any critical feedback would be appreciated.


Support Vietnamese

Hello @tri2820
In this PR, there are 68 file changes and the PR's title is just support Vietnamese. Can you please clean it up?

I believe all you need are

lib/utils/translations/index.ts
lib/utils/translations/vi.ts

these files, if you believe we need other changes as well you should open up another PR for these and explain, please :)
Also, we should not have build/dist folders and files because these should be generated

@tri2820
You should open another PR for the fixes then :)


Bump deps to pull in latest Ohai

Full dep bump. Berks dep job fails so that one isn't bumped.

Ping @chef/client-core


fix: missing autocmd data when no branch

I ran into an issue with the data not being provided during the PersistedLoad* autocmds, looks like its due to the lack of the branch_separator in my session file names, this passes all tests although not sure if its changing the behavior from intended

Great spot and thanks for the fix!


Upgrade systeminformation for legacy JSON output

The systeminformation package has a security vulnerability that is causing an alert to popup in my repos. This PR upgrades the dep to the fixed version. It ships with its own TS types so the additional @types/systeminformation package is no longer necessary.
Alternatively, we could consider removing the legacy JSON output since it is legacy. But since that is a breaking change I opt'ed for the safer PR first :)
Replaces #208

Released as v0.5.8


Build error for reverse P/Invoke with ref struct parameter (ref vs. in)

In our project we have a reverse P/Invoke call for native to managed callback using a delegate defined as:
ref struct I420AVideoFrame { [...] }

[UnmanagedFunctionPointer(CallingConvention.StdCall, CharSet = CharSet.Ansi)]
public delegate void I420AVideoFrameUnmanagedCallback(IntPtr userData, ref I420AVideoFrame frame);

On the C++ side, the signature is something like:
void I420AVideoFrameCallback(void* user_data, const I420AVideoFrame& frame);

On the managed C# side we register a static trampoline function that C++ will reverse P/Invoke, and which is defined as:
[MonoPInvokeCallback(typeof(I420AVideoFrameUnmanagedCallback))]
public static void I420AFrameCallback(IntPtr userData, ref I420AVideoFrame frame)
{
    [...]
}

This all works well as is, which is actually the workaround to the issue. Originally since the C++ signature uses const& (we just want to pass the struct by pointer/reference from C++ to C# as a view over read-only memory) we used in instead of ref for the I420AVideoFrame parameter, but this yields an error during the .NET Native build (and works with JIT):
error : ILT0021: Could not resolve method 'EETypeRva:0x--------'

This seems to be the error defined in RuntimeTypeHandle.cs, which is worryingly marked as "last resort" in comment, although I would expect there should be some metadata available for that method:
https://github.com/dotnet/corert/blob/e2c4bc3d7889f951c231d7f131d6a7e20c034fb6/src/System.Private.CoreLib/src/System/RuntimeTypeHandle.cs#L134
Note that we had to manually define the MonoPInvokeCallback attribute which seems to be required for working with Unity's IL2CPP, so we copied a definition from the Mono project:
[AttributeUsage(AttributeTargets.Method)]
sealed class MonoPInvokeCallbackAttribute : Attribute
{
    public MonoPInvokeCallbackAttribute(Type t) { }
}

Is there any reason ref works and not in in this context? I assume that by using ref we expose ourselves to bugs where the C# code inadvertently modifying the struct would produce some side effect on the C++ side which the C++ compiler did not intend to see (since the parameter is const& there), which could lead to bugs, so the current workaround with ref is not ideal.

For issues related to .NET Native for UWP, could you please send email to<EMAIL_ADDRESS>

Can you check if you're using the latest version of the UWP NuGet package? (https://github.com/microsoft/dotnet/blob/master/releases/UWP/net-native2.2/README.md)
I recall fixing an issue with the in keyword in p/invoke signatures earlier this year.

Yes we're using 6.2.9, there's no newer version as I know.

Looks like that fix didn't make it into the release :(. Please reach out to the email address above - people in charge of the release monitor that email address.
For internal reference, this was likely fixed by changeset 1737914.

This fix will be in ~Feb 2020 .NET Native update.


DPL-309 Add lifespan field to the V2 API Purpose resource

so it can be accessed through the V2 API by Asset Audits

Could any test be updated too? :)

Good point, I think it was such a tiny change I forgot about tests! I will check.


build: update and clean dependencies

Interestingly cargo udeps didn't find that many. cargo machete did a much better job, and it's much easier to run too.


Some of files will be create by root owner

When does not exists $HOME/.docker directory, boot2docker.iso and $HOME/.docker/machine/cache will be create by root owner.
In that case, Error will occur when extractKernelImages.

Download and copy of B2D iso, and cache folder creation are included in mcnutils.CopyIsoToMachineDir().
Permission error occurs in this func. Need re-implement.

Solved in https://github.com/zchee/docker-machine-driver-xhyve/commit/a592ed33ec441c7975e1fbff144f08e0f101f65e


refresh jstree returns all deleted nodes too

I work with Ajax callback and delet nodes with web service, if some node have data in it ,web service sends me error . I refresh the tree if error has happen to redraw deleted node , but those node which are successfully deleted are back again.

That is normal, you need to listen for the delete_node event, hit the server and remove the node - that way it won't be in the data next time it is fetched.

exactly I do the same that u said 😄 I mean every time I refresh JSTree internally , it load data that was loaded for the 1st time . all deleted node or renamad and created node will back to their first state . but when I refresh the page every thing is fine

It is something related to data fetching then? You should be using AJAX, and you should make sure the data is not cached.
If this does not point you in the right direction - please share your config.

                    ctrl.treeConfig = {
                        tie_selection: false,
                        core: {
                            multiple: false,
                            animation: true,
                            error: function (error) {
                                $log.error('treeCtrl: error from js tree - ' + angular.toJson(error));
                            },
                            check_callback: true,
                            worker: true
                        },
                        types: {
                            default: {
                                icon: 'fa fa-folder'
                            }
                        },
                        version: 1,
                        plugins: ['types', 'contextmenu', 'unique'],
                        "contextmenu": {
                            "items": function ($node) {
                                var tree = ctrl.treeInstance.jstree(true);
                                return {
                                    "Create": {
                                        "separator_before": false,
                                        "separator_after": false,
                                        "label": "Add",
                                        "action": function (obj) {
                                            $node = tree.create_node($node);
                                            tree.edit($node);
                                        }
                                    },
                                    "Rename": {
                                        "separator_before": false,
                                        "separator_after": false,
                                        "label": "Rename",
                                        "action": function (obj) {
                                            tree.edit($node);
                                        }
                                    },
                                    "Remove": {
                                        "separator_before": false,
                                        "separator_after": false,
                                        "label": "delete",
                                        "action": function (obj) {
                                            tree.delete_node($node);
                                        }
                                    }
                                };
                            },
                            select_node: false

                        }
                    }

here is my config and here is listener for delet_node
``
        ctrl.removeCat = function (node, parent) {
            if (parent.node.children.length) {
                notify({
                    message:'cannot delete this category',
                    classes: 'alert-danger',
                    templateUrl: 'views/common/notify.html',
                    duration: 10000
                });

            }
            else{
                var req = {
                    method: 'POST',
                    url: domain + 'profile/category/deleteuserCategory',
                    headers: {
                        'x-auth-token': $localStorage.jwt
                    },
                    data: {
                        categoryId: parent.node.id
                    }
                }
                $http(req).then(success => {
                    console.log(success.data);
                    notify({
                        message: success.data.message,
                        classes: 'alert-danger',
                        templateUrl: 'views/common/notify.html',
                        duration: 10000
                    });
                    if(success.data.code = -1 && success.data.hasError){
                        var p = ctrl.treeData.objOf('id',parent.node.id);
                        ctrl.treeInstance.jstree(true).refresh(true);
                    }

                })
            }
        }

this is my html code (I'm using angularJs and ngJstree)