some command are treated as subgroup incorrectly in autogen code

Below is the help message of datafactory extension of autogen code.
Group
    az datafactory : Commands to manage datafactory.

Subgroups:
    create-data-flow-debug-session : Commands to manage datafactory create data flow debug session.
    dataflow                       : Commands to manage datafactory dataflow.
    dataset                        : Commands to manage datafactory dataset.
    get-feature-value              : Commands to manage datafactory get feature value.
    integration-runtime            : Commands to manage datafactory integration runtime.
    linkedservice                  : Commands to manage datafactory linkedservice.
    pipeline                       : Commands to manage datafactory pipeline.
    pipelinerun
    query-pipeline-run             : Commands to manage datafactory query pipeline run.
    trigger                        : Commands to manage datafactory trigger.

Commands:
    configure_factory_repo         : Updates a factory's repo information.
    create                         : Creates or updates a factory.
    delete                         : Deletes a factory.
    get_data_plane_access          : Get Data Plane access.
    get_git_hub_access_token       : Get GitHub Access Token.
    list                           : Lists factories.
    show                           : Gets a factory.
    update                         : Creates or updates a factory.

Actual:
the create-data-flow-debug-session, get-feature-value, query-pipeline-run, etc are treated as subgroups incorrectly.
Expect:
We should find a method to separate these actions from the real resources and no longer treat them as subgroups.

this if fixed now.
Please note that i simplified creation of command group names in PoC to be inline with future implementation, so regenerated command group names may be different.


s/emptyIonStruct/emptyStructElement

Rename this function which somehow remained unchanged during recent refactors.

Actually, this is a bad idea and it's named correctly already.


Multiple spaces gets trimmed

When reading an excel file with a cell value that has multiple spaces.

The spaces gets trimmed into one space.
'test value'
Is there an option to stop it from doing so?

Do you have an example file?  XLS parsing will not trim interstitial whitespace.  XLSX will only do it if the spaces are literal whitespace characters and the XML is not explicitly marked with the space="preserve"

Please share a file and we can reopen the issue


Feature request: JSON output

Thank you for creating this package!
I wanted to ask if you’d be open to add an option to the CLI that allows users to get the output in JSON format.
When using this mode, the output would become something like this:
[
  {
    "type": "error",
    "file"<EMAIL_ADDRESS>    "path": "./envelopes.require"
  }
]

Instead of this:

❌ require: @aws-lambda-powertools/jmespath/envelopes cannot be required (["exports","./envelopes","require"])
Error: Command failed: node --input-type=commonjs --eval="require('@aws-lambda-powertools/jmespath/envelopes');"
/workspaces/tmp-test/node_modules/@aws-lambda-powertools/jmespath/lib/cjs/envelopes.d.ts:1
import type { ParsingOptions, JSONObject } from './types.js';
^^^^^^

SyntaxError: Cannot use import statement outside a module
    at internalCompileFunction (node:internal/vm:77:18)
    at wrapSafe (node:internal/modules/cjs/loader:1288:20)
    at Module._compile (node:internal/modules/cjs/loader:1340:27)
    at Module._extensions..js (node:internal/modules/cjs/loader:1435:10)
    at Module.load (node:internal/modules/cjs/loader:1207:32)
    at Module._load (node:internal/modules/cjs/loader:1023:12)
    at Module.require (node:internal/modules/cjs/loader:1235:19)
    at require (node:internal/modules/helpers:176:18)
    at [eval]:1:1
    at runScriptInThisContext (node:internal/vm:144:10)

Node.js v20.11.1

    at ChildProcess.exithandler (node:child_process:422:12)
    at ChildProcess.emit (node:events:518:28)
    at maybeClose (node:internal/child_process:1105:16)
    at ChildProcess._handle.onexit (node:internal/child_process:305:5) {
  code: 1,
  killed: false,
  signal: null,
  cmd: `node --input-type=commonjs --eval="require('@aws-lambda-powertools/jmespath/envelopes');"`,
  stdout: '',
  stderr: '/workspaces/tmp-test/node_modules/@aws-lambda-powertools/jmespath/lib/cjs/envelopes.d.ts:1\n' +
    "import type { ParsingOptions, JSONObject } from './types.js';\n" +
    '^^^^^^\n' +
    '\n' +
    'SyntaxError: Cannot use import statement outside a module\n' +
    '    at internalCompileFunction (node:internal/vm:77:18)\n' +
    '    at wrapSafe (node:internal/modules/cjs/loader:1288:20)\n' +
    '    at Module._compile (node:internal/modules/cjs/loader:1340:27)\n' +
    '    at Module._extensions..js (node:internal/modules/cjs/loader:1435:10)\n' +
    '    at Module.load (node:internal/modules/cjs/loader:1207:32)\n' +
    '    at Module._load (node:internal/modules/cjs/loader:1023:12)\n' +
    '    at Module.require (node:internal/modules/cjs/loader:1235:19)\n' +
    '    at require (node:internal/modules/helpers:176:18)\n' +
    '    at [eval]:1:1\n' +
    '    at runScriptInThisContext (node:internal/vm:144:10)\n' +
    '\n' +
    'Node.js v20.11.1\n'
}

This way users would be able to consume the output programmatically and incorporate it as part of their CI workflows.
Note that the suggested JSON format is just a proposal, as long as it’s a valid JSON it’d work for us.

@dreamorosi Sure. I can add that. How verbose do you need it? Seeing the serialized error is quite messy as it is now. Do you want the same data in the JSON?

The most important parts would be to have:

the path to the package.json file that contains the issue
The path within the JSON of the faulty export (i.e. exports.[“./types”].require)
(optional) An unformatted string with the error generated as-is today

With the first two points we’d already be able to pinpoint the issue to a specific line and tell a PR author that something is wrong & further inspection is needed.
I have put the third point as optional because the error/stack trace are kind of opaque (expected) so for us even just knowing there’s an issue with an export is already very valuable.

Thanks, that gives me enough to go on.

@dreamorosi I published version 0.2.0 which has the --json flag. Try it out and let me know if it covers your use case.

Thank you for adding the feature, the format looks great!


Low hz imu,fine adjustment

Test rooftop_ ouster_ dataset.bag If the IMU is set to 100 Hz, the effect is not very good, so it is modified to 1.0 / 10000.0 to better support low Hz

The value of dt in the ternary operator is only used to set dt for the very first sample. (i.e. when lastImu_*** has not been set yet)
Also, for 100Hz this should be set to  1.0 / 100.0 because the denominator is the IMU rate in Hz. It is set to 500.0 because that is the IMU rate of @TixiaoShan's IMU


support Django 2.2

Modify sortedm2m field will raise a exception after upgrade to Django 2.2
TypeError: _add_items() got an unexpected keyword argument 'through_defaults'
How can I fix it

I changed the code of _add_items to:
def _add_items(self, source_field_name, target_field_name, *objs, through_defaults=None):
        # source_field_name: the PK fieldname in join table for the source object
        # target_field_name: the PK fieldname in join table for the target object
        # *objs - objects to add. Either object instances, or primary keys of object instances.
        through_defaults = through_defaults or {}

        # If there aren't any objects, there is nothing to do.
        from django.db.models import Max, Model
        if objs:
            # Django uses a set here, we need to use a list to keep the
            # correct ordering.
            new_ids = []
            for obj in objs:
                if isinstance(obj, self.model):
                    if not router.allow_relation(obj, self.instance):
                        raise ValueError(
                            'Cannot add "%r": instance is on database "%s", value is on database "%s"' %
                            (obj, self.instance._state.db, obj._state.db)
                        )
                    fk_val = self.through._meta.get_field(
                        target_field_name).get_foreign_related_value(obj)[0]
                    if fk_val is None:
                        raise ValueError(
                            'Cannot add "%r": the value for field "%s" is None' %
                            (obj, target_field_name)
                        )
                    new_ids.append(fk_val)
                elif isinstance(obj, Model):
                    raise TypeError(
                        "'%s' instance expected, got %r" %
                        (self.model._meta.object_name, obj)
                    )
                else:
                    new_ids.append(obj)

            db = router.db_for_write(self.through, instance=self.instance)
            manager = self.through._default_manager.using(db)
            vals = (manager
                    .values_list(target_field_name, flat=True)
                    .filter(**{
                        source_field_name: self._fk_val,
                        '%s__in' % target_field_name: new_ids,
                    }))
            for val in vals:
                if val in new_ids:
                    new_ids.remove(val)
            _new_ids = []
            for pk in new_ids:
                if pk not in _new_ids:
                    _new_ids.append(pk)
            new_ids = _new_ids
            new_ids_set = set(new_ids)

            if self.reverse or source_field_name == self.source_field_name:
                # Don't send the signal when we are inserting the
                # duplicate data row for symmetrical reverse entries.
                signals.m2m_changed.send(sender=rel.through, action='pre_add',
                    instance=self.instance, reverse=self.reverse,
                    model=self.model, pk_set=new_ids_set, using=db)

            # Add the ones that aren't there already
            with atomic(using=db):
                fk_val = self._fk_val
                source_queryset = manager.filter(**{'%s_id' % source_field_name: fk_val})
                sort_field_name = self.through._sort_field_name
                sort_value_max = source_queryset.aggregate(max=Max(sort_field_name))['max'] or 0

                manager.bulk_create([
                    self.through(**through_defaults, **{
                        '%s_id' % source_field_name: fk_val,
                        '%s_id' % target_field_name: pk,
                        sort_field_name: sort_value_max + i + 1,
                    })
                    for i, pk in enumerate(new_ids)
                ])

            if self.reverse or source_field_name == self.source_field_name:
                # Don't send the signal when we are inserting the
                # duplicate data row for symmetrical reverse entries.
                signals.m2m_changed.send(sender=rel.through, action='post_add',
                    instance=self.instance, reverse=self.reverse,
                    model=self.model, pk_set=new_ids_set, using=db)

it seems to work but it's not ready to be a PR

I am also running into this issue. Are there any updates? I see a few PRs attempting to address it, they all seem to fail CI due to an issue with the CI build

Looking at issue #112 I think @gregmuellegger may have abandoned this project.  Could someone fork the project? My project depends heavily on this library!

Maybe we should reach out on twitter or email & ask what they would prefer? There is contact info in the project README

Maybe we should reach out on twitter or email & ask what they would prefer? There is contact info in the project README

I’ve just pinged him on twitter.

@npadgen Did the maintainer respond?

No, but my colleague created PR https://github.com/Seleznev-nvkz/django-sortedm2m/pull/1/files It is temporary solution but it works. We think that maintainer stopped develop projectfor installing - pip install<EMAIL_ADDRESS>18:23, "Yair Silbermintz" notifications@github.com:@npadgen Did the maintainer respond?
—You are receiving this because you are subscribed to this thread.Reply to this email directly, view it on GitHub, or mute the thread.
--

Hello,
Considering the original author of this package had been completely silent for a long time I decided to fork this repo and publish the package under a new name on PyPI.
I could not come up with a good name so I just named it django-sorted-m2m (note the additional -). I hope it does not create too much confusion. If someone has a better idea, please let me know.
The forked repo includes pull request #134 by @joehybird to support Django 2.2. Kudos to him!
My repo is at: https://github.com/Ponytech/django-sorted-m2m
PyPI link: https://pypi.org/project/django-sorted-m2m
Anyone interested into taking part of this fork is more than welcome!

FYI I have asked Jazzband about the possibility of moving this project over to their community, even if @gregmuellegger doesn't answer pings.

Hello, thank you! So, we wait for him answer 20.06.2019, 22:20, "Richard Barran" notifications@github.com:FYI I have asked Jazzband about the possibility of moving this project over to their community, even if @gregmuellegger doesn't answer pings.—You are receiving this because you are subscribed to this thread.Reply to this email directly, view it on GitHub, or mute the thread.  -- С уважением,Павликов Валерий

Greg has answered!
See this thread.

It sounds great! Tanks a lot. We will follow for Jazzband26.06.2019, 15:11, "Richard Barran" notifications@github.com:Greg has answered!
See this thread.
—You are receiving this because you are subscribed to this thread.Reply to this email directly, view it on GitHub, or mute the thread.
-- С уважением,Павликов Валерий

So now that Greg answered, is anyone able to move in pull request #134 by @joehybird?

Will there be a release soon?


Update benchee version

This updates this formatter to use the new behavior implementation for formatters.

Coverage remained the same at 100.0% when pulling c0f50a87ab89cfc69747c005fc5eb290adc26d89 on update-benchee-version into 8fd528d1e937a3f7285b8e4db9c541008d6dc8fa on master.


feat!(codegen): ApplyTemplate data is on .Data

Data is moved from . to .Data to allow for passing parent template
values always. This also enables function recievers to always be passed
as well, in the case of having to do the dict "Config" .Config hack
that existed before.

Failing tests are ones that broke because of this change, which is expected. Will eventually move away from live data testing


add task info to job_stopped event

It would be useful to include a summary of tasks at job_stop.
Perhaps something akin to Dict[task_id, Tuple[TaskType, TaskState, error: Optional[Error]]]

Reviewed, looks great!!


Check browerlist whether really works

Bug Report
Please answer these questions before submitting your issue. Thanks!
What did you do?

Access dashboard in Chrome 5x version.
What did you expect to see?

Work normally.
What did you see instead?

It crashes and says Object.entries() is not a function.
(This version of Chrome is supported in the browser list, so the checkBrowser.js doesn't show a prompt to say this browser version is too low.)
What version of TiDB Dashboard are you using (./tidb-dashboard --version)?

4.0.7

I should have fixed it and I'm going to test production bundles under new config. Should I create a PR for it first ?

Yes, please!

known:

browserslist works well
babel injects polyfill well except for Object.entries
importing polyfill for Object.entries manually works
Object.entries may be in third-party dependencies

For record, the stack trace causing the Object.entries error was:
e.exports@http://pd:2379/dashboard/static/js/23.a9c66b1c.chunk.js:1007225
e.exports/</p.onerror@http://pd:2379/dashboard/static/js/23.a9c66b1c.chunk.js:1006176

hi @unbyte , can you help do some more test on Firefox 46 and 47 if it is possible? (not sure whether we still can download such old firefox installation packages). it is reported that has the same error in an old Firefox 4x version (but don't know the exact version).
According to our code logic, it is supposed to show a prompt to ask the user to upgrade their browser in the Firefox below version 68.

Object.entries is supported in Chrome 54+, Edge 14+, Firefox 47+, Safari 10.1+

I think it is more appropriate to import polyfill manually as a special case rather than to alert users to change browser

I think it is more appropriate to import polyfill manually as a special case rather than to alert users to change browser

Testing on the old version of firefox is just for checking whether has some more reasons, then resolve them together. Importing polyfill manually is one way, or we can higher the lowest supported version.


Resuming training on unsharded checkpoint

🐛 Describe the bug
I tried resuming training on a previous unsharded checkpoint from step 1k and the training resumed with no initial issue however when it tried to save the sharded checkpoint i encountered a error as shown below wondering what is causing this issue? For context, the env/node number used are all the same.
Traceback (most recent call last):
File "/mnt/azureml/cr/j/947c8b089dfe4d0484df42634f296716/exe/wd/scripts/train.py", line 345, in
main(cfg)
File "/mnt/azureml/cr/j/947c8b089dfe4d0484df42634f296716/exe/wd/scripts/train.py", line 316, in main
trainer.fit()
File "/workspace/OLMo/olmo/train.py", line 1153, in fit
checkpoint_path, _ = self.save_checkpoint(CheckpointType.sharded)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/OLMo/olmo/train.py", line 560, in save_checkpoint
result = self.save_sharded_checkpoint()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/OLMo/olmo/train.py", line 468, in save_sharded_checkpoint
result = self._save_checkpoint(checkpointer, CheckpointType.sharded)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/OLMo/olmo/train.py", line 428, in _save_checkpoint
checkpointer.save_checkpoint(
File "/workspace/OLMo/olmo/checkpoint.py", line 1000, in save_checkpoint
"optim": FSDP.optim_state_dict(dist_model, optim),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py", line 1832, in optim_state_dict
return FullyShardedDataParallel._optim_state_dict_impl(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/torch/distributed/fsdp/fully_sharded_data_parallel.py", line 1255, in _optim_state_dict_impl
return _optim_state_dict(
^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/torch/distributed/fsdp/_optim_utils.py", line 1972, in _optim_state_dict
fsdp_osd_state = convert_fn(
^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/torch/distributed/fsdp/_optim_utils.py", line 1795, in _convert_state_with_orig_params
_gather_all_orig_param_state(
File "/opt/conda/lib/python3.11/site-packages/torch/distributed/fsdp/_optim_utils.py", line 1689, in _gather_all_orig_param_state
output_states = _allgather_orig_param_states(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/torch/distributed/fsdp/_optim_utils.py", line 1519, in _allgather_orig_param_states
dtype, state_buffers = _convert_all_state_info(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/lib/python3.11/site-packages/torch/distributed/fsdp/_optim_utils.py", line 1415, in _convert_all_state_info
assert curr_scalar_tensor_value is None or torch.equal(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Rank 4 has different values for step: 1500.0. Other ranks: 500.0
Versions
.

Could you share your config and details about the checkpoint you started from (if it's an official one)? Also, it looks like your run is using the default 'legacy' checkpointer. You may have success with another checkpointer like the olmo_core one (pass --sharded_checkpointer=olmo_core to train.py, for example).

Hi I am using the official config for Olmo1B. Only thing I've amended was changing the tokenizer to the dolma one. Here are the details of the config. Also wanted to check when we resume, do we resume using the config file saved inside the saved checkpoints folder or we continue using the existing config file we originally started with?
run_name: OLMo-1B
seed: 6198
dry_run: false
wandb:
name: ${run_name}
project: olmotest
entity: test123
model:
d_model: 2048
n_heads: 16
n_layers: 16
mlp_ratio: 8
weight_tying: true
alibi: false
rope: true
flash_attention: true  # not available on AMD
attention_dropout: 0.0
attention_layer_norm: false
multi_query_attention: false
include_bias: false
block_type: sequential
layer_norm_type: default
layer_norm_with_affine: false
bias_for_layer_norm: false
attention_layer_norm_with_affine: false
activation_type: swiglu
residual_dropout: 0.0
embedding_dropout: 0.0
max_sequence_length: 2048
vocab_size: 50280
embedding_size: 50304
eos_token_id: 50279
pad_token_id: 1
init_device: meta
init_fn: mitchell
compile: null  # causes instability on AMD GPUs
optimizer:
name: adamw
learning_rate: 4.0e-4
weight_decay: 0.1
betas:

0.9
0.95
metrics_log_interval: 10

scheduler:
name: cosine_with_warmup
t_warmup: 2000
alpha_f: 0.1
tokenizer:
identifier: tokenizers/allenai_gpt-neox-olmo-dolma-v1_5.json
truncate_direction: right
save_folder: "./outputs"
save_overwrite: false
Sharded checkpoints (best for restarts)
save_interval: 500
save_num_checkpoints_to_keep: 9
Unsharded checkpoints (for final storage)
save_interval_unsharded: 1000
save_num_unsharded_checkpoints_to_keep: -1
load_path: null
max_duration: 739_328  # 3.1T tokens
global_train_batch_size: 2048
device_train_microbatch_size: 8
precision: amp_bf16
fsdp:
wrapping_strategy: null
precision: mixed
max_grad_norm: 1.0
max_grad_norm_ratio: null
speed_monitor:
window_size: 20
eval_interval: ${save_interval}
eval_subset_num_batches: -1
device_eval_batch_size: ${device_train_microbatch_size}
evaluators:
lump all the small datasets together (we still get separate metrics).

label: v3-small-ppl-validation
data:
num_workers: 0
drop_last: true
datasets:
v3-small-c4_en-validation:
- https://olmo-data.org/eval-data/perplexity/v3_small_gptneox20b/c4_en/val/part-0-00000.npy
v3-small-dolma_books-validation:
- https://olmo-data.org/eval-data/perplexity/v3_small_gptneox20b/dolma_books/val/part-0-00000.npy
v3-small-dolma_common-crawl-validation:
- https://olmo-data.org/eval-data/perplexity/v3_small_gptneox20b/dolma_common-crawl/val/part-0-00000.npy
v3-small-dolma_pes2o-validation:
- https://olmo-data.org/eval-data/perplexity/v3_small_gptneox20b/dolma_pes2o/val/part-0-00000.npy
v3-small-dolma_reddit-validation:
- https://olmo-data.org/eval-data/perplexity/v3_small_gptneox20b/dolma_reddit/val/part-0-00000.npy
v3-small-dolma_stack-validation:
- https://olmo-data.org/eval-data/perplexity/v3_small_gptneox20b/dolma_stack/val/part-0-00000.npy
v3-small-dolma_wiki-validation:
- https://olmo-data.org/eval-data/perplexity/v3_small_gptneox20b/dolma_wiki/val/part-0-00000.npy
v3-small-ice-validation:
- https://olmo-data.org/eval-data/perplexity/v3_small_gptneox20b/ice/val/part-0-00000.npy
v3-small-m2d2_s2orc-validation:
- https://olmo-data.org/eval-data/perplexity/v3_small_gptneox20b/m2d2_s2orc/val/part-0-00000.npy
v3-small-pile-validation:
- https://olmo-data.org/eval-data/perplexity/v3_small_gptneox20b/pile/val/part-0-00000.npy
v3-small-wikitext_103-validation:
- https://olmo-data.org/eval-data/perplexity/v3_small_gptneox20b/wikitext_103/val/part-0-00000.npy

label: v2-small-ppl-validation
data:
num_workers: 0
drop_last: true
datasets:
v2-small-4chan-validation:
- https://olmo-data.org/eval-data/perplexity/v2_small_gptneox20b/4chan/val.npy
v2-small-c4_100_domains-validation:
- https://olmo-data.org/eval-data/perplexity/v2_small_gptneox20b/c4_100_domains/val.npy
v2-small-c4_en-validation:
- https://olmo-data.org/eval-data/perplexity/v2_small_gptneox20b/c4_en/val.npy
v2-small-gab-validation:
- https://olmo-data.org/eval-data/perplexity/v2_small_gptneox20b/gab/val.npy
v2-small-ice-validation:
- https://olmo-data.org/eval-data/perplexity/v2_small_gptneox20b/ice/val.npy
v2-small-m2d2_s2orc-validation:
- https://olmo-data.org/eval-data/perplexity/v2_small_gptneox20b/m2d2_s2orc/val.npy
v2-small-m2d2_wiki-validation:
- https://olmo-data.org/eval-data/perplexity/v2_small_gptneox20b/m2d2_wiki/val.npy
v2-small-manosphere-validation:
- https://olmo-data.org/eval-data/perplexity/v2_small_gptneox20b/manosphere/val.npy
v2-small-mc4_en-validation:
- https://olmo-data.org/eval-data/perplexity/v2_small_gptneox20b/mc4_en/val.npy
v2-small-pile-validation:
- https://olmo-data.org/eval-data/perplexity/v2_small_gptneox20b/pile/val.npy
v2-small-ptb-validation:
- https://olmo-data.org/eval-data/perplexity/v2_small_gptneox20b/ptb/val.npy
v2-small-twitterAEE-validation:
- https://olmo-data.org/eval-data/perplexity/v2_small_gptneox20b/twitterAEE/val.npy
v2-small-wikitext_103-validation:
- https://olmo-data.org/eval-data/perplexity/v2_small_gptneox20b/wikitext_103/val.npy

label: piqa
type: downstream

label: hellaswag
type: downstream

label: winogrande
type: downstream

label: openbook_qa
type: downstream

- label: boolq  # requires implemention of the pmi_dc matrix
# type: downstream

label: sciq
type: downstream

label: arc_easy
type: downstream

- label: arc_challenge  # requires implemention of the pmi_dc matrix
type: downstream

label: copa
type: downstream

label: rte
type: downstream

label: commitment_bank
type: downstream

label: mrpc
type: downstream

label: sst2
type: downstream

data:
pad_direction: right
num_workers: 0
drop_last: true
pin_memory: true
prefetch_factor: 16
persistent_workers: true
timeout: 0
paths:
- https://buildllmpremjpeast.blob.core.windows.net/dataset1/dolma_tokenized/c4-filtered/part-00-00004.npy

I normally use the config file that I started training with, but I imagine but can work fine.
Could you share a bit more details about the checkpoint you started with? Is it from an official run or a run you did? Was it unsharded manually (using, say, scripts/unshard.py) or produced as an unsharded checkpoint by training?

We started the checkpoint from a run we did and it was produced as an unsharded checkpoint by training.

I also tried resuming the sharded checkpoints produced by training and it had no issues either in resuming or saving subsequently


Show Organization notifications in static pages

This PR fixes an issue that we have since dashboard migration, as we weren't showing organization notifications.
So, to fix it I have:

Added organization_notifications to our all-in-one API endpoint (/api/v3/me)
Modified UserNotificationsView to support notifications coming from external options.
Passed organizationNotifications property to DashboardHeaderView in Dashboard, Account and Profile pages.

Acceptance
See https://github.com/CartoDB/cartodb/issues/14088
Related to https://github.com/CartoDB/cartodb/issues/14088

All tests are 💚


Update version.txt to v1.16.0

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 39.64%. Comparing base (4adadb0) to head (6b70aea).

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1485   +/-   ##
=======================================
  Coverage   39.64%   39.64%
=======================================
  Files          51       51
  Lines        2916     2916
=======================================
  Hits         1156     1156
  Misses       1607     1607
  Partials      153      153

Flag
Coverage Δ

unittests
39.64% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


W3C Validate

Does not pass validation on
https://validator.w3.org/
Attribute t_show not allowed.  If t_show replace on data-t_show, this will solve the problem with validation.

Thank you for using t-scroll, i will update it in next version.


COI: CRM ticket

On supervisor submission a CRM ticket should be generated for further processing by the HRSC Records teams
Acceptance Criteria

[x] On submission by supervisor, a CRM ticket must be created with the following parameters

[x]  Subject should be  Conflict of Interest Disclosure RE: [employee first name  last name]
[x]  Primary contact should be supervisor
[x]  Secondary reference should be the employee. Use employee number
[x]  Category should be - Supervising and Leading People -> Access to Personnel File -> Conflict of Interest Disclosure
[x]  Assigned should be HRSC - Records

[ ] It should include all the following attachments (this is moved to #1720)
[x]  The auto CRM email should be sent to supervisor after ticket is generated

Ask CRM or Nancy what cat

Tested with Bhumin - moving to PO

demoed to Karen for the records team and it looks good


Decoration types break each other

VSCode Version: 1.4.0
OS Version: OS X 10.11.6

Steps to Reproduce:

Create an extension with the following code

  let vscode = require('vscode');
  let window = vscode.window;

  let decorationTypeOne = window.createTextEditorDecorationType({
    isWholeLine: true, // not relevant, but present in the real-world scenario
  });
  let decorationTypeTwo = window.createTextEditorDecorationType({
    color: 'green'
  });

  let decorationWithContent = {
    range: new vscode.Range(0, 0, 0, 3),
    renderOptions: {after: {contentText: '+test'}}
  };

  let decorationWithColor = {range: new vscode.Range(0, 1, 0, 2)};

  window.activeTextEditor.setDecorations(decorationTypeOne, [decorationWithContent]);
  window.activeTextEditor.setDecorations(decorationTypeTwo, [decorationWithColor]);

Open a text file with a single line abc in it.
The result is a duplicated contentText inserted in a middle of the text.

Note if you remove color: 'green' from the second decoration type, then it works as expected:

Real-world scenario was reported in wallaby.js repo where decorations inserted by wallaby are broken when Rainbow Brackets extension is also used, which simply adds colours to certain ranges.

I will investigate.
As a workaround you can try to make the range of the decoration smaller.

Thanks!

As a workaround you can try to make the range of the decoration smaller.

This doesn't seem to help. I tried to reduce the wallaby decoration range down to 1 character, still the same.
If it helps, here's how the generated HTML for the provided sample looks like:

For some reason, an empty token is inserted in a middle of the line.


[v4] export style-prefixed.css with prefixed Tailwind CSS v4 layers for projects which uses Tailwind CSS v3

Just to clarify @dimaMachina, does this PR mean we can use tailwind v3 in Nextra v4 by using  import "nextra-theme-docs/style.css" to import styles?

if you use tailwind css v3 in your project import style-prefixed.css instead

Thanks @dimaMachina. Is this in the app-router-34 pre-release, or do we need do wait for the next one?

It’s already in last alpha release


UI Toolkit

A proper UI toolkit is integral to the success of any game engine. This goes beyond basic buttons and labels, but also has to deal with things like containers, layouts, input pass through and blocking, etc...
There are a few GUI libs out there for Go, but these all suffer from a couple problems:

Most of them don't provide a way to use an existing OpenGL conext (assuming they use OpenGL for rendering at all, which is a different issue all together).
None of them support multiple render targets (i.e. opengl, metal, vulkan, etc..)
None of them would tie in with our existing Input subsystem.

As such, I suggest that we build our own UI toolkit that is tailor made to fit the rest of the engine's design.
Although tedious (and as such, this is a rather large undertaking, no doubt), layouts and UI controls are all well understood and solved problems, so it won't really be difficult work.
The design here should utilize Engo's ECS framework.
I'll start by stubbing out a basic framework and present it for feedback before working on actual implementations of the various controls and layouts.

Just to necro post (again), Some thing I'd like to see in the UI toolkit is support for textual and data heavy games (so; tabs, spreadsheets, heavy use of copy-paste, modal and window support)
The games I want to play are heavy on the simulation side, and golang is awesome for me because they fall in the 'embarrassingly parallelizeable' computational space
if you want to go down a  structured definition language route; I've been playing on and off with the concepts you see in QT; but instead of a turring complete javascript; target a simpler system (or; at the very least, less awful).
QT's Signals and Slots are a good starting point, though would need to be rehashed to be more golang idiomatic and make better use of go routines and channels.  They may work well with the message system.  for my toy project I was looking at haiku's aggressivly threaded model as well, which is also a good fit for UI; but may have major performance impacts in a game engine.

Now that we have Arcs and Beizer Curves in common.Shape, I'm thinking we could actually get started on this. Honestly, I'd suggest sticking to the ecs paradigm in engo itself. This way, when we change it to add concurrency etc then the UI stuff would utilize just be updated to utilize that.
No worries about brining back these threads, it helps us know which features are important to people! Thanks!


Unhandled Exception: MissingPluginException(No implementation found for method init on channel sarbagyastha.com.np/youtubePlayer)

I'm trying to play videos on iOS simulator (iPhone 11 Pro max)
And I have right away this error:
[VERBOSE-2:ui_dart_state.cc(148)] Unhandled Exception: MissingPluginException(No implementation found for method init on channel sarbagyastha.com.np/youtubePlayer)
#0      MethodChannel.invokeMethod (package:flutter/src/services/platform_channel.dart:314:7)
<asynchronous suspension>
#1      _channel (package:youtube_player/youtube_player.dart:42:9)
#2      _channel (package:youtube_player/youtube_player.dart:40:21)
#3      YoutubePlayer.keepOn (package:youtube_player/youtube_player.dart:861:36)
#4      _YoutubePlayerState.build (package:youtube_player/youtube_player.dart:974:21)
#5      StatefulElement.build (package:flutter/src/widgets/framework.dart:4012:27)
#6      ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:3924:15)
#7      Element.rebuild (package:flutter/src/widgets/framework.dart:3721:5)
#8      ComponentElement._firstBuild (package:flutter/src/widgets/framework.dart:3907:5)
#9      StatefulElement._firstBuild (package:flutter/src/widgets/framework.dart:4053:11)<…>
[VERBOSE-2:ui_dart_state.cc(148)] Unhandled Exception: MissingPluginException(No implementation found for method keepOn on channel sarbagyastha.com.np/youtubePlayer)
#0      MethodChannel.invokeMethod (package:flutter/src/services/platform_channel.dart:314:7)
<asynchronous suspension>
#1      YoutubePlayer.keepOn (package:youtube_player/youtube_player.dart:861:45)
#2      _YoutubePlayerState.build (package:youtube_player/youtube_player.dart:974:21)
#3      StatefulElement.build (package:flutter/src/widgets/framework.dart:4012:27)
#4      ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:3924:15)
#5      Element.rebuild (package:flutter/src/widgets/framework.dart:3721:5)
#6      ComponentElement._firstBuild (package:flutter/src/widgets/framework.dart:3907:5)
#7      StatefulElement._firstBuild (package:flutter/src/widgets/framework.dart:4053:11)
#8      ComponentElement.mount (package:flutter/src/widgets/framework.dart:3902:5)
#9      Element.inflateWidget (package:flutter/sr<…>
[VERBOSE-2:ui_dart_state.cc(148)] Unhandled Exception: MissingPluginException(No implementation found for method create on channel sarbagyastha.com.np/youtubePlayer)
#0      MethodChannel.invokeMethod (package:flutter/src/services/platform_channel.dart:314:7)
<asynchronous suspension>
#1      VideoPlayerController.initialize (package:youtube_player/youtube_player.dart:239:59)
<asynchronous suspension>
#2      _YoutubePlayerState.initializeYTController (package:youtube_player/youtube_player.dart:937:22)
#3      _YoutubePlayerState.build (package:youtube_player/youtube_player.dart:1002:7)
#4      StatefulElement.build (package:flutter/src/widgets/framework.dart:4012:27)
#5      ComponentElement.performRebuild (package:flutter/src/widgets/framework.dart:3924:15)
#6      Element.rebuild (package:flutter/src/widgets/framework.dart:3721:5)
#7      ComponentElement._firstBuild (package:flutter/src/widgets/framework.dart:3907:5)
#8      StatefulElement._firstBuild (package:flutter/src/widgets/framework.dart:<…>

I tried so many things... flutter clean etc..
I added this line in Info.plist
<key>io.flutter.embedded_views_preview</key>
<string>YES</string>

Any ideas ?

The issue you reported about is another plugin, which only supports Android. So there's no implementation of the plugin on iOS.


yolact  inference multi images in batch

Hello,First I want to thank you for the great work
Did you support multi image inference in yolact model or how can i infer multiple images in the same batch thanks

You can set samples_per_gpu here https://github.com/open-mmlab/mmdetection/blob/master/configs/yolact/yolact_r50_1x8_coco.py#L139, like samples_per_gpu=2.


Create SDM files

Create SDM files function is needed

Thanks for opening this issue @ju-ec . You can see https://github.com/ofgulban/bvbabel/blob/main/examples/create_sdm.py script for and example SDM creation.


Update freebsd.md

Updated the instructions. Tested on FreeBSD 13.2 jail - fresh install.

Thank you!


Can't see the paperproof render on Linux

I am using Linux. I can see the Lean Infoview however when I click the Paperproof window, there is nothing.
I've followed the tutorial: 1. Installed extension 2. Required paperproof 3. Imported paperproof 4. Used "lake update" and made sure my lean version is up to date

We had this issue very recently (https://github.com/Paper-Proof/paperproof/issues/41).
It can stem either from the incompatibility of Paperproof with your Lean version, or from  incompatibility of Paperproof with your vscode-lean extension version.
Latest Paperproof should work with all Lean and vscode-lean extension versions, however, - so if you're on the latest Paperproof version, and you still get a blank page, I will look into it again.
Some questions:

Does it get fixed when you fully close vscode, and fully open it again?
What are your:

lean-vscode extension version
Lean version
Paperproof extension version
Paperproof lean library version

Lean version 4.8.0-rc2
Lean4 vscode extension version: 0.0.155 (more info in picture)
Paperproof extension version: v1.3.0
Paperproof lean library version: Don't know how to find that

And no it doesn't get fixed even after fully closing vscode.

Thanks for the details! I can reproduce this with Lean4 vscode extension version: 0.0.155, lean-vscode updated its api again.
A temporary solution is to downgrade lean-vscode to version 0.0.144, then fully close and open the vscode window.
Added this step to the readme (link).

Thank you for the report, please reopen the issue if downgrading lean-vscode doesn't work for you.


default value for empty strings

DynamoDb does not accept empty strings and simply throws an error.
Would it make sense for the marshaler to replace empty strings with null? Perhaps make it configurable what value should be substituted for empty strings.

Thanks Will, sorry I didn't get beck to you earlier.
This is an improvement and I have been using it in production since January.
I understand your desire to keep this project strictly focused on marshalling, but as you stated having to prune out empty strings is a small inconvenience, but something that is easily forgotten. For now I have created a wrapper around the marshaller to handle this.
Another thing I came across is that storing null values for an indexed property is not allowed (gives the error below) whereas completely omitting the property is valid and simply omits the object from the index.
message: 'One or more parameter values were invalid: Type mismatch for Index Key webshopId Expected: S Actual: NULL IndexName: webshopId-created-index'
So, if you have an index on p:
save({id: 1, p: null}) // Not allowed
save({id: 1}) // Perfectly fine
Again, I handle this by wrapping the marshaller and simply removing all null properties. This gives me essentially the same behaviour as MongoDB exhibits which I find to be a good balance.
Have you had a chance to think more about this?

Hey @sorenbs thanks for using this! In production no less! I don't know if you saw above, but I made a PR #3 to throw a marshaling error for "". In thinking about it more and more, and being brought other issues in the same vain (#5, #6), I think I'm reinforcing my understanding that DynamoDb has a set of rules about data operation that are outside the scope of marshaling.
The tricky thing about your example with the saving of a null index attribute, is that to address it with this library it would require this library to know that p is used by an index. This project is strictly about translation -- when it sees {p: null} it is going to turn it into {p: {NULL: true}} every time, that is it's job regardless of p's significance.
I think you did the right thing making a wrapper tool to "groom"/"validate" your objects before saving. If you're working with data that doesn't always adhere to DynamoDb's rules, you gotta get in there and shuffle things around. I think there is space for another single-responsibility npm module dynamodb-validator/dynamodb-groomer. I think it might be kind of useful to have a project which you do configure with some information about your schema, and then it inspects your objects and cleans them up if it can, or throws an error.
If you have the time and energy, I would be willing to help you with that project. Cheers!
BTW since you're using this in production, take a look at the README.md in #6 because it has some docs that explain the jump from v0.1.1 to v1.0.0 in the way JS Arrays are marshaled to Dynamo Lists and Sets.

I just want to say what I think from the perspective of a developer who downloads and uses this library.
This isn't just a marshaller. It's DynamoDb marshaller. And as a developer, when I download this package, use it to marshal a javascript object or a json string, I just expect it to work and give me a perfectly valid marshalled json or object that is compatible with DynamoDb.
As a user of this library, I don't really care if it breaks single responsibility principle to do what it's supposed to do. If I want to convert an object/json into dynamoDb format and I download this library, and it crashes if my object contains empty strings, believe me that's not a good feeling. it just feels incomplete.
If I have to download another package that will validate or process the marshalled json, or if I have to write that code myself to replace empty strings with nulls, that's just not good.
You see, when I give give this library an object which contains properties that have empty string values, that's not an invalid object. That is a perfectly valid object. The fact that dynamoDb doesn't accept empty string doesn't make my object "invalid". So marshaller shouldn't be throwing exceptions when it finds such object. Neither should it return a marshalled json that has empty strings, because that would be invalid "marshalled json".
I guess, all I am saying is, I expect a valid marshalled json that is compatible with dynamodb when I use this library. And if this library has to anything it needs to to make it happen, it should do it - instead of telling the developers that their object contains empty strings and then blowing up the application. Because this is a dynamodb marshaller.
We should atleast make it configurable so that users will have a choice. If they want an exception, they can have it. I would then just configure the marshaller to do the magic. I am sure many others will do the same. And eventually we will have to make it default.

Fair enough, it seems this module does not do what you think it is supposed to do, and that's not good.
Would you like it to marshal an empty string to { NULL: true }? I think that will in turn create another problem for people who weren't expecting null's in their DynamoDb tables for every empty string they sent it. I do not see a clear path to getting it to do what you want without causing more issues. The code is licensed under MIT, and is therefore available to you in any capacity. You can rip it off directly and redistribute it with alterations. You can also make a PR for consideration if you'd like to see these changes as a part of this project.
I'm sure DynamoDb has an excellent reason for not accepting empty strings and making their API only accept these formatted objects, but that was a decision made by Amazon, not this module author. As a fellow user of this module frustrated by the amount of work there is to do in order to make JSON data acceptable to DynamoDb -- believe me I sympathize.
(I hear they have their own marshaler as part of the SDK these days, please look into it because it may solve more for you).
I do not speak for CascadeEnergy and my opinion does not necessarily represent theirs.


How to implement OnlineTileSourceBase in Osmdroid 6.0+

Issue Type
[ x] Question
[ ] Bug
[ ] Improvement
[ ] Build system related
[ ] Performance
[ ] Documentation
Description and/or steps/code to reproduce the problem
The following piece of code in the Wiki for creating an OnlineTileSourceBase does not work:
mMapView.setTileSource(new OnlineTileSourceBase("USGS Topo", ResourceProxy.string.custom, 0, 18, 256, "",
               new String[] { "http://basemap.nationalmap.gov/ArcGIS/rest/services/USGSTopo/MapServer/tile/" }) {
               @Override
               public String getTileURLString(MapTile aTile) {
                    return getBaseUrl() + aTile.getZoomLevel() + "/" + aTile.getY() + "/" + aTile.getX()
				+ mImageFilenameEnding;
               }
          });

The current version of the constructor for this class is as follows:
mMapView.setTileSource(new OnlineTileSourceBase("myMapnik", 0, 18, 256, "",
                new String[]{"http://<IP_ADDRESS>/osm_tiles"}) {
            @Override
            public String getTileURLString(long pMapTileIndex) {
                return null;
            }
        });

I have created a mapnik server at <IP_ADDRESS>, so that requesting http://<IP_ADDRESS>/osm_tiles/0/0/0.png via browser gives me a map tile as follows:

How can I implement the new version of OnlineTileSourceBase?
Environment
Android
Version of osmdroid the issue relates to:
6.0.1

@djnotes I just edited the wiki page.
For osmdroid versions >= 6 it's something like this:
mMapView.setTileSource(new OnlineTileSourceBase("USGS Topo", 0, 18, 256, "",
               new String[] { "http://basemap.nationalmap.gov/ArcGIS/rest/services/USGSTopo/MapServer/tile/" }) {
               @Override
               public String getTileURLString(long pMapTileIndex) {
                    return getBaseUrl()
                           + MapTileIndex.getZoom(pMapTileIndex)
                           + "/" + MapTileIndex.getY(pMapTileIndex)
                           + "/" + MapTileIndex.getX(pMapTileIndex)
                           + mImageFilenameEnding;
               }
          });

Thank you @monsieurtanuki. Works like a charm.


add Docker compatibility

This change ensures that the oAuth mechanism works even inside Docker, by binding to all interfaces and not just the local loopback. In order to do this two dependencies are upgraded. A small explanation is also added to README.md

Thanks for this PR, really appreciate it. Sorry for not seeing this earlier, we switched to GH organization and mentions weren't configured. I'm getting an error when using Docker with the requirements I modified your command a bit because docker wants a hardcoded path to mount, see screenshots below.

Now I'm no Docker expert, but before I accept want to make sure I can get it to work.

Hi. No worries. And of course.
Not sure about the error. It seems to break on a network access when trying
to recover information about the first required package (which I did not
touch)
I was on MacOS, and perhaps Docker on MacOS is more permissive with respect
to internet access. You're on Linux I suppose? Thanks for raising this,
I'll give it a try this weekend on my Linux machine (as I should have
anyway)
On Tue, Dec 5, 2023 at 5:03 PM Invictus-Korstiaan @.***>
wrote:

Thanks for this PR, really appreciate it. Sorry for not seeing this
earlier, we switched to GH organization and mentions weren't configured.
I'm getting an error when using Docker with the requirements I modified
your command a bit because docker wants a hardcoded path to mount, see
screenshots below.
Screenshot.2023-12-05.at.17.00.53.png (view on web)
https://github.com/invictus-ir/ALFA/assets/21335149/f52c806d-3ee9-497c-9be9-5ba8deecd3b2
Screenshot.2023-12-05.at.17.01.12.png (view on web)
https://github.com/invictus-ir/ALFA/assets/21335149/4e731f0b-81c2-4fb9-a2fe-8953e8f14ad3
Now I'm no Docker expert, but before I accept want to make sure I can get
it to work.
—
Reply to this email directly, view it on GitHub
https://github.com/invictus-ir/ALFA/pull/4#issuecomment-1841094657, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AEWM3SC2DGOKJV2XHI4ASVTYH5ATJAVCNFSM6AAAAAA6SVCKPGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNBRGA4TINRVG4
.
You are receiving this because you authored the thread.Message ID:
@.***>

Hi did you manage to test this, just going through open issues/bugs to see if there's still an action here.

No updates/activity.


[20.09] mujs: 1.0.7 -> 1.0.9

Motivation for this change
Fixes #96809 .
Fixes #96894 .
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

Fixes #96894 .

How is that related?

Result of nixpkgs-review pr 115952 at e5bfebd1 run on x86_64-linux 1

2 packages skipped due to time constraints:

plex-media-player
plex-mpv-shim

16 packages built successfully:

celluloid
curseradio
jellyfin-mpv-shim
jftui
minitube
mpc-qt
mpv
mpv-unwrapped
mpvScripts.mpris
mujs
python37Packages.mpv
python38Packages.mpv
qimgv
somafm-cli
sublime-music
tuijam

My bad, I misread the issue number. Corrected.

It's still not a pure git cherry-pick -x.


Use :focus-ring (and its polyfill) to gracefully handle dropdown triggers etc

Meta-issue to explore the use of :focus-ring https://github.com/WICG/focus-ring for situations that currently aren't all that great (with things keeping a visible focused style after being clicked with the mouse), including:

dropdown triggers
buttons and their related variants
carousel controls

Would be great to get this added before V4 final, by which time there should be good support in latest browser versions (and to add polyfill for older versions)

In my company we make use if what-input and a custom PostCSS plugin that adds a :focus-mouse pseudo selector. Doing so we simply do:
.class:focus-mouse { outline: none }

It also adds a :focus-keyboard useful to add focus style to custom elements.
I think it's a better approach compared to the proposed standard. And doesn't add classes to the elements, because only html gets alerted.

the advantage of going with a proposed standard (which is already implemented in firefox anyway, as that's where the proposal took initial inspiration) is that unlike what-input's take, it will be natively adopted by browsers, obviating the need for a polyfill in future altogether. but yes, what-input (which i've contributed to in the past) was considered some time ago as well, but it seems to go beyond our immediate needs

I see, I really hope they will change the proposed standard because it doesn't make any sense to me at the moment 😥

the standard solves one very specific but common problem: when clicking/interacting with certain controls (<button> etc) with a mouse or even touch, browsers currently use heuristics (specific to each browser) to decide whether or not to show the default focus outline; for keyboard users, the focus outline is essential to indicate where they currently are within the page; but for mouse/touch users it's usually seen as visually redundant and ugly.
traditionally, designers have simply added outline: none or similar to suppress the outline (which used to show all the time for focused items, before browsers implemented heuristics) altogether. this caused problems for keyboard users and is a bad idea.
adding explicit :focus styles, while good for keyboard users, also results in browser heuristics being completely bypassed. it explicitly says that when an element receives focus (regardless how it received focus - by mouse, touch, keyboard) it should have those particular styles.
:focus-ring basically allows you to say "apply these styles, but only in situations where the browser would normally show its default outline", i.e. only in the same situations in which browser heuristics would also determine that the default outline should be shown. this way you can provide very specific styles that generally only apply to keyboard users, and are not shown for mouse/touch interactions. which gives you exactly what you want in most cases - e.g. in the case of Bootstrap's dropdowns, clicking on the control to open a dropdown with a mouse you wouldn't want a visible outline/focus style on the dropdown trigger...but for a keyboard user, you do want to provide a visible "this item currently has focus" style.

Yes I understand, but what people need is a :focus that matches when you don't want to show the outline, so that you can remove it with :focus-something { focus: 0 }.
With the current proposal instead you'll be forced to do :not(:focus-ring) { focus: 0; } which doesn't make any sense.

:focus { outline: 0; }
:focus-ring { /* go crazy with whatever visual indicator you wanna give for kbd users */ }

Yes but I think it's better to leave the default focus style and simply disable it when interacting with mouse, because the focus ring should be consistent across websites to don't confuse the users. By the way this isn't the right place to discuss about this.

How will polyfill fly with JS view libraries? Will I have to run extra logic whenever I render button for polyfill to do its magic?

Closing this out for now. Will revisit once native support for focus-visible (yes, they renamed it...) is a bit more mature


Add configurable api expanders

This lets you configure api expanders in config.settings, e.g.:
import {
  settings as defaultSettings,
} from '@plone/volto/config';
import {GET_CONTENT } from '@plone/volto/constants/ActionTypes';

export const settings = {
  ...defaultSettings,
  apiExpanders: {
    GET_CONTENT: ['mycustomexpander'],
  },
}

Doing a getContent on http://localhost:3000/api/object will result in a call to http://localhost:3000/api/object?expand=mycustomexpander.
This should have tests. We currently have no tests for the api middleware and no mocks for the parts to let it run.

We've had the same need at one point, I like the idea of attaching them to the action types.

I like everything about this PR. Though I think it might be more future-proof to make the expander registration a callable instead of a simple string. This way I have a bit more control per site section on what expands to request.

@nileshgulia1 Use matchPath to match the registered expanders per router path: https://github.com/plone/volto/blob/7fe7004af15d441950b31f367797e073fb9b3513/docs/source/recipes/appextras.md

Yeah it's about time we have this available.

@tiberiuichim @nileshgulia1 Umm now that I realise, expanders only applies to the GET_CONTENT action. There's not point to add them to all the requests in the API, or I'm missing something?

The expanders that @csenger envisaged are registered per action type, so that's maximum flexibility.
The expanders in plone.restapi are inserted from the content serializer, which is used primarily by GET_CONTENT, indeed. We could decide now that we only care about configurable expanding on the getContent action, but this is not "fixed in stone" from plone.restapi. We, indeed, use the expanders to piggy-back additional info on top of a single "central" request and that is usually the getContent. That said, I like flexibility... we can't foresee all the scenarios where this sort of thing could be "abused".
Though we could elect to keep it simple, fine with me.

Fine then, I would continue then with the documented approach that @csenger drafted in the first place, along with your suggestion to include the matchPath.
@nileshgulia1 do you plan to continue with it?
I can jump too if required.

The config then would look like this, right?
  apiExpanders: [
    {
      match: '',
      GET_CONTENT: ['mycustomexpander'],
    },
    {
      match: '/de',
      GET_CONTENT: ['myothercustomexpander'],
    },
    ...
  ],

@sneridagh I added matchPatch to apiExpanders config in the settings and pushed whatever I had, feel free to push in this PR.

I think more like for navigation instance :
{         match: '/events',         GET_NAVIGATION: ['navigation'],       },

@nileshgulia1 We want things to work as in comment https://github.com/plone/volto/pull/1705#issuecomment-815530171
That is, on GET_CONTENT action, we want to inject an expander.

@nileshgulia1 I added a test for the addExpandersToPath function. It fails, because I think that we don't need to stringify the qs since all browsers support UTF-8 nowadays in URLs. @tiberiuichim what do you think?

@nileshgulia1 Don't take my comment to literally, GET_CONTENT was mentioned as an example. @csenger original implementation was for all actions based on their type, we want to maintain that. What I meant was that if you're mentioning the GET_NAVIGATION action and you're showing that you're expanding "navigation" for it, I get confused.

Sorry for the confusion, I just revert the things I added in this commit where I used action type as a param to filter expanders.

@nileshgulia1 @sneridagh Thanks for pushing this forward!


FromBase64Transform-sample uses InputBlockSize instead of hard-coded 4

Depends on https://github.com/dotnet/corefx/pull/39599
myTransform.InputBlockSize returned 1, so the sample code used 4 explicitely, as base64 decoding transforms 4 input bytes to 3 output bytes.
https://github.com/dotnet/corefx/pull/39599 fixes this, so the sample code can use the InputBlockSize provided by the transform.
Additional did unify the code comments in the sample.
And removed the redundant Clear, as this is called by Dispose.

For api-docs -> https://github.com/dotnet/dotnet-api-docs/pull/2895

Thank you, @gfoidl, for modifying the Visual Basic and C++ examples as well. I'll merge this PR and review dotnet/dotnet-api-docs#2895 shortly.


Can you add survey.questionDescriptionLocation

Are you requesting a feature, reporting a bug or ask a question?
requesting a feature
What is the current behavior?
Description of the questions is shown under question title
What is the expected behavior?
Can you add survey.questionDescriptionLocation to be able to show description in bottom of the question as it was done for errors with survey.questionErrorLocation?
How would you reproduce the current behavior (if this is a bug)?
Provide the test code and the tested page URL (if applicable)
Tested page URL:
Test code
your_code_here

Specify your

browser:
browser version:
surveyjs platform (angular or react or jquery or knockout or vue):
surveyjs version:

@irynmare There is already titleLocation property. Description is rendered under the title.
It has "top", "bottom" and "left" options.
Does it work for you?
Thank you,
Andrew

Not really. I need title to be in top and description under input. Added picture of how it should look.

@irynmare It looks good and yes, we do not have an option for this right now.
I'm adding this into our TODO list...
Thank you,
Andrew

@irynmare Sorry, We have missed this request, but we have finally implemented it. Here is the related issue.
survey.questionDescriptionLocation = "underInput";
This code will work from next minor version that we will release on the next week.
Thank you,
Andrew


@supports backdrop-filter doesn't prefix

Great project, thanks for your hard work!
  @supports (backdrop-filter: blur(10px)) {
    backdrop-filter: blur(10px);
  }

Outputs
@supports (-webkit-backdrop-filter:blur(10px) or (backdrop-filter:blur(10px)) {
	.scope {
		-webkit-backdrop-filter: blur(10px);
		backdrop-filter: blur(10px);
	}
}

What is the expected output?

You'd expect it to have matching parens (4 open, 3 closed doesn't work).
To be explicit, you'd expect
@supports (-webkit-backdrop-filter:blur(10px)) or (backdrop-filter:blur(10px)) {
	.scope {
		-webkit-backdrop-filter: blur(10px);
		backdrop-filter: blur(10px);
	}
}

Thanks, Patched in 3.4.4.


Textures for Cylinders?

Has anyone else expressed a need for cylinders to have textures?
Or ways to make cylinders to look like product bottles?

I have purchased premium.
How do we get the premium features?
Thanks!


Mobile safari code samples don't wrap or show syntax highlighting

When loading the docs on mobile safari the syntax highlighting does not render and the spacing and text-wrapping of the code samples is not observed.
Example on the left is Safari's rendering and example on the right is Chrome (in a mobile viewport)

fwiw, i can recreate this in Chrome on desktop too. Sometimes happens, sometimes doesn't. Disabling cache in dev tools seems to increase the chances. Refreshing tends to fix it.

This was being caused by a third party script for an embedded docs component - I have since removed it, and hopefully we should be good here now.


System.ArgumentNullException: String reference not set to an instance of a String. (Parameter 's')

Running using clBitcrack
app_path=clBitcrack
app_arguments=-b 64 -t 256 -p 256
After a few seconds I get this error:
GeForce GT 525M  384 / 1024MB | 5 targets 6.10 MKey/s (62,914,560 total) [00:00:08] [GPU=0] [HEX=3F94A08]               System.ArgumentNullException: String reference not set to an instance of a String. (Parameter 's')
at System.Text.Encoding.GetBytes(String s)
at System.Text.UTF8Encoding.UTF8EncodingSealed.GetBytes(String s)
at BitcrackRandomiser.Helpers.Helper.SHA256Hash(String data) in C:\Users\ilker\source\repos\bitcrackrandomiser\BitcrackRandomiser\Helpers\Helper.cs:line 112
at BitcrackRandomiser.Helpers.Flagger.Flag(Settings settings, String hex, Int32 gpuIndex, String proofKey, String gpuName) in C:\Users\ilker\source\repos\bitcrackrandomiser\BitcrackRandomiser\Helpers\Flagger.cs:line 25
at BitcrackRandomiser.Randomiser.JobFinished(String targetAddress, String hex, Settings settings, Boolean keyFound, Int32 gpuIndex) in C:\Users\ilker\source\repos\bitcrackrandomiser\BitcrackRandomiser\Randomiser.cs:line 230
at BitcrackRandomiser.Randomiser.OutputReceivedHandler(Object o, DataReceivedEventArgs e, String targetAddress, List`1 proofValues, String hex, Settings settings, Process process, Int32 gpuIndex) in C:\Users\ilker\source\repos\bitcrackrandomiser\BitcrackRandomiser\Randomiser.cs:line 257
at BitcrackRandomiser.Randomiser.<>c__DisplayClass7_0.b__0(Object o, DataReceivedEventArgs s) in C:\Users\ilker\source\repos\bitcrackrandomiser\BitcrackRandomiser\Randomiser.cs:line 179
at System.Diagnostics.Process.ErrorReadNotifyUser(String data)
at System.Diagnostics.AsyncStreamReader.FlushMessageQueue(Boolean rethrowInNewThread)
--- End of stack trace from previous location ---
at System.Diagnostics.AsyncStreamReader.<>c.b__18_0(Object edi)
at System.Threading.QueueUserWorkItemCallbackDefaultContext.Execute()
at System.Threading.ThreadPoolWorkQueue.Dispatch()
at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
at System.Threading.Thread.StartCallback()

Hello, have you tested Bitcrack externally? The card is very old, Bitcrack's OpenCL version has some bugs. Can you provide information by doing an external test (without using bitcrackrandomiser)
Thanks.

Hi, just gave it a try, seems that clBitcrack works in stand-alone:
\bitcrackrandomiser\bitcrack>clBitcrack.exe 13zb1hQbWVsc2S7ZTZnP2G4undNNpdh5so -b 64 -t 256 -p 256
[2024-03-06.13:17:38] [Info] Compression: compressed
[2024-03-06.13:17:38] [Info] Starting at:<PHONE_NUMBER>000000000000000000000000000000000000000000000000000001
[2024-03-06.13:17:38] [Info] Ending at:   FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFEBAAEDCE6AF48A03BBFD25E8CD0364140
[2024-03-06.13:17:38] [Info] Counting by:<PHONE_NUMBER>000000000000000000000000000000000000000000000000000001
[2024-03-06.13:17:38] [Info] Compiling OpenCL kernels...
[2024-03-06.13:17:38] [Info] Initializing GeForce GT 525M
[2024-03-06.13:17:39] [Info] Generating 4,194,304 starting points (160.0MB)
[2024-03-06.13:17:45] [Info] 10.0%
[2024-03-06.13:17:47] [Info] 20.0%
[2024-03-06.13:17:49] [Info] 30.0%
[2024-03-06.13:17:51] [Info] 40.0%
[2024-03-06.13:17:53] [Info] 50.0%
[2024-03-06.13:17:55] [Info] 60.0%
[2024-03-06.13:17:57] [Info] 70.0%
[2024-03-06.13:17:59] [Info] 80.0%
[2024-03-06.13:18:01] [Info] 90.0%
[2024-03-06.13:18:03] [Info] 100.0%
[2024-03-06.13:18:03] [Info] Done
GeForce GT 525M  384 / 1024MB | 1 target 6.20 MKey/s (767,557,632 total) [00:02:02]

Can you share the settings.txt file? (Removing private data)

// Select puzzle number [66,67,68] or [38] for test pool
target_puzzle=66
// which app will be used [bitcrack]
app_type=bitcrack
// App folder path [cuBitcrack, clBitcrack or full path of the Bitcrack app]
app_path=clBitcrack
// App arguments
// Example for bitcrack: -b 896 -t 256 -p 256
app_arguments=-b 64 -t 256 -p 256
// User token value. You can create it by logging into your account at btcpuzzle.info
user_token={removed}
// Wallet address for worker. {wallet}.{worker}. If no worker name it will create random.
wallet_address={removed}
// Enter your GPU Count [1-16]
gpu_count=1
// Use only one GPU at index. [0-15]
gpu_index=0
// Scan type [default,includeDefeatedRanges,excludeIterated{2/3/4},excludeContains{3/4},excludeStartsWith{XXX}]
scan_type=default
// Scan for rewards of the pool [true,false]
scan_rewards=false
// Custom range to scan Example; [2D,3BA,3F12,2881A or none]
custom_range=none
// Share progress with your API [https://yourwebsite/example.php]
api_share=none
// Send notifications to Telegram
telegram_share=false
// Telegram access token
telegram_accesstoken=0
// Telegram chat id
telegram_chatid=0
// Send notification when eack key scanned
telegram_share_eachkey=false
// If true and the private key found; private key will be send to Telegram
untrusted_computer=false
// Run app in test mode. Example private key will be found
test_mode=false
// Force continue if key found
force_continue=false
// Private pool id
private_pool=none

Can you test like below and verify it works?
$ clBitcrack.exe -b 64 -t 256 -p 256 -d 0 --keyspace 3F94A080000000000:3F94A090000000000 13zb1hQbWVsc2S7ZTZnP2G4undNNpdh5so 1AYRwmDyowDQLZKhzHmtGNwsD7bUp1qq9A 15JVX6DrCLc1JeFFLu5TmEFNYwz7e2JiBp 16otECJAsiiM54WyX98kkrmDNTEaSzi6fS

If it works smoothly, I need to investigate the error in the application codes. Thank you for feedback.

\bitcrackrandomiser\bitcrack>clBitcrack.exe -b 64 -t 256 -p 256 -d 0 --keyspace 3F94A080000000000:3F94A090000000000 13zb1hQbWVsc2S7ZTZnP2G4undNNpdh5so 1AYRwmDyowDQLZKhzHmtGNwsD7bUp1qq9A 15JVX6DrCLc1JeFFLu5TmEFNYwz7e2JiBp 16otECJAsiiM54WyX98kkrmDNTEaSzi6fS
[2024-03-06.13:52:38] [Info] Compression: compressed
[2024-03-06.13:52:38] [Info] Starting at:<PHONE_NUMBER>00000000000000000000000000000000000003F94A080000000000
[2024-03-06.13:52:38] [Info] Ending at:<PHONE_NUMBER>00000000000000000000000000000000000003F94A090000000000
[2024-03-06.13:52:38] [Info] Counting by:<PHONE_NUMBER>000000000000000000000000000000000000000000000000000001
[2024-03-06.13:52:38] [Info] Compiling OpenCL kernels...
[2024-03-06.13:52:38] [Info] Initializing GeForce GT 525M
[2024-03-06.13:52:39] [Info] Generating 4,194,304 starting points (160.0MB)
[2024-03-06.13:52:45] [Info] 10.0%
[2024-03-06.13:52:48] [Info] 20.0%
[2024-03-06.13:52:53] [Info] 30.0%
[2024-03-06.13:52:55] [Info] 40.0%
[2024-03-06.13:52:57] [Info] 50.0%
[2024-03-06.13:52:59] [Info] 60.0%
[2024-03-06.13:53:01] [Info] 70.0%
[2024-03-06.13:53:03] [Info] 80.0%
[2024-03-06.13:53:05] [Info] 90.0%
[2024-03-06.13:53:07] [Info] 100.0%
[2024-03-06.13:53:07] [Info] Done
GeForce GT 525M  384 / 1024MB | 4 targets 6.15 MKey/s (62,914,560 total) [00:00:08][2024-03-06.13:53:17] [Info] Reached end of keyspace

I haven't found a working version yet. I'm closing the issue, I'll research it again when I have free time.


Improve interaction between save button and file uploads in work form

Part of the work done to close out the Sufia 7.3.0 release included the following JavaScript changes, which fixed couple error conditions involving how in-progress uploads interact with the save button in the work form:
https://github.com/projecthydra/sufia/pull/3069
The downside of this implementation is that the save button does appear to be clickable before all uploads have finished, but it's not actually clickable. (IOW, this isn't a bug but rather a UX enhancement.)
The save button should be disabled (not merely unclickable via preventDefault()) while any uploads are running.
@jcoyne made the following suggestions about how we might implement this:

I think we need a isSaveButtonActive method that checks both form validity and this.uploads.inProgress. It should be triggered when a upload starts or finishes (the fileuploadstarted and fileuploadstopped events), or the checkbox is checked, or the metadata is changed.

There's an additional problem, whereby if you add files and then delete all of them, the files requirement still shows that it's met. We fixed this by making use of a fileuploaddestroyed callback in Uploaded.js. See: https://github.com/psu-stewardship/scholarsphere/commit/d5e52b4cf

So uploading a folder of files, this looks like the problem.  The "Save" button is disabled while the first file is uploading, then some check is enabling it, and so forth.
And as the issue mentions, looks like the form's submission is still being prevented in a multi-upload.
https://screencast.com/t/sMKZbA9G


Add translation keys for the new DWC terms

See https://www.gbif-uat.org/occurrence/3082767301

caste
vitality
eventType
subtribe
superfamily
tribe
verbatimLabel

deployed to uat and prod


Error saving results in run_regression

Hi,
Thanks very much for developing this nice tool.
I am following the following tutorial  to estimate the cell type signatures from my own scRNAseq dataset.  I am having an error while calling to the run_regression function. Everything looks fine, the epoch VS ELBO loss plots are generated as well as the ones for the UMI counts. However, I am having this error when saving the results:
### Saving results ###
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
~/.conda/envs/cellpymc/lib/python3.7/site-packages/anndata/_io/utils.py in func_wrapper(elem, key, val, *args, **kwargs)
    187         try:
--> 188             return func(elem, key, val, *args, **kwargs)
    189         except Exception as e:

~/.conda/envs/cellpymc/lib/python3.7/site-packages/anndata/_io/h5ad.py in write_dataframe(f, key, df, dataset_kwargs)
    257     group.attrs["encoding-version"] = EncodingVersions.dataframe.value
--> 258     group.attrs["column-order"] = list(df.columns)
    259

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

~/.conda/envs/cellpymc/lib/python3.7/site-packages/h5py/_hl/attrs.py in __setitem__(self, name, value)
    102         """
--> 103         self.create(name, data=value)
    104

~/.conda/envs/cellpymc/lib/python3.7/site-packages/h5py/_hl/attrs.py in create(self, name, data, shape, dtype)
    196             try:
--> 197                 attr = h5a.create(self._id, self._e(tempname), htype, space)
    198             except:

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/_objects.pyx in h5py._objects.with_phil.wrapper()

h5py/h5a.pyx in h5py.h5a.create()

RuntimeError: Unable to create attribute (object header message is too large)

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
<ipython-input-601-cf8487f6980c> in <module>
     30                    export_args={'path': results_folder + 'regression_model/', # where to save results
     31                                 'save_model': True, #save pytorch model?
---> 32                                 'run_name_suffix': ''})
     33
     34 reg_mod = r['mod']

~/.conda/envs/cellpymc/lib/python3.7/site-packages/cell2location/run_regression.py in run_regression(sc_data, model_name, verbose, return_all, train_args, model_kwargs, posterior_args, export_args)
    325
    326     # save anndata with exported posterior
--> 327     sc_data.write(filename=path + 'sc.h5ad', compression='gzip')
    328
    329     # save model object and related annotations

~/.conda/envs/cellpymc/lib/python3.7/site-packages/anndata/_core/anndata.py in write_h5ad(self, filename, compression, compression_opts, force_dense, as_dense)
   1848             compression_opts=compression_opts,
   1849             force_dense=force_dense,
-> 1850             as_dense=as_dense,
   1851         )
   1852

~/.conda/envs/cellpymc/lib/python3.7/site-packages/anndata/_io/h5ad.py in write_h5ad(filepath, adata, force_dense, as_dense, dataset_kwargs, **kwargs)
    115             )
    116         else:
--> 117             write_attribute(f, "raw", adata.raw, dataset_kwargs=dataset_kwargs)
    118         write_attribute(f, "obs", adata.obs, dataset_kwargs=dataset_kwargs)
    119         write_attribute(f, "var", adata.var, dataset_kwargs=dataset_kwargs)

~/.conda/envs/cellpymc/lib/python3.7/functools.py in wrapper(*args, **kw)
    838                             '1 positional argument')
    839
--> 840         return dispatch(args[0].__class__)(*args, **kw)
    841
    842     funcname = getattr(func, '__name__', 'singledispatch function')

~/.conda/envs/cellpymc/lib/python3.7/site-packages/anndata/_io/h5ad.py in write_attribute_h5ad(f, key, value, *args, **kwargs)
    137     if key in f:
    138         del f[key]
--> 139     _write_method(type(value))(f, key, value, *args, **kwargs)
    140
    141

~/.conda/envs/cellpymc/lib/python3.7/site-packages/anndata/_io/h5ad.py in write_raw(f, key, value, dataset_kwargs)
    146     group.attrs["shape"] = value.shape
    147     write_attribute(f, "raw/X", value.X, dataset_kwargs=dataset_kwargs)
--> 148     write_attribute(f, "raw/var", value.var, dataset_kwargs=dataset_kwargs)
    149     write_attribute(f, "raw/varm", value.varm, dataset_kwargs=dataset_kwargs)
    150

~/.conda/envs/cellpymc/lib/python3.7/functools.py in wrapper(*args, **kw)
    838                             '1 positional argument')
    839
--> 840         return dispatch(args[0].__class__)(*args, **kw)
    841
    842     funcname = getattr(func, '__name__', 'singledispatch function')

~/.conda/envs/cellpymc/lib/python3.7/site-packages/anndata/_io/h5ad.py in write_attribute_h5ad(f, key, value, *args, **kwargs)
    137     if key in f:
    138         del f[key]
--> 139     _write_method(type(value))(f, key, value, *args, **kwargs)
    140
    141

~/.conda/envs/cellpymc/lib/python3.7/site-packages/anndata/_io/utils.py in func_wrapper(elem, key, val, *args, **kwargs)
    193                 f"Above error raised while writing key {key!r} of {type(elem)}"
    194                 f" from {parent}."
--> 195             ) from e
    196
    197     return func_wrapper

RuntimeError: Unable to create attribute (object header message is too large)

Above error raised while writing key 'raw/var' of <class 'h5py._hl.files.File'> from /.

So far, I have found something like this:
HDF5 has a header limit of 64kb for all metadata of the columns. This include name, types, etc. When you go about roughly 2000 columns, you will run out of space to store all the metadata. This is a fundamental limitation of pytables. I don't think they will make workarounds on their side any time soon. You will either have to split the table up or choose another storage format.
from: https://stackoverflow.com/questions/16639503/unable-to-save-dataframe-to-hdf5-object-header-message-is-too-large
Do you have any idea why the object header could become so long?
Best regards and thank you very much,
Alberto.

Hi
Not a c2l dev but had similar issues. The way it's designed, each cell type in the reference gets multiple accompanying columns so the number of columns can increase very rapidly with number of cell types. I don't think the designers of h5ad anticipated that :)
I also found the h5ad export slow and sometimes prone to crashing for large files so I ended up using joblib.dump (here) (a variant of pickle.dump but with the additional benefit of compression, which helps a lot for file size). I can share the code but in run_c2l.py, wherever you see write you can use joblib.dump for the export
Similarly, some of the .csv files it exports can be useful but also slow down a lot for large spatial data/many cell types (it's also not compressed) so if you're also running large numbers of cell types, it might be worth removing that part of the code, and regenerating as needed
For what it's worth, I've found it very handy to run a small 5-iteration model all the way through for new data first, before doing the full model, so it doesn't train the whole thing and then quit before saving

Hello!
Many thanks for the information! This is very helpful.
I think that, as a first step, I will remove the less informative samples and merge the very similar cell types to avoid going over 2000 columns.
Best regards,
Alberto.

Hi @alberto-valdeolivas and @JZL
Thanks for using our method!
We did not run into the problem of having too many .var columns yet (we had about 300 covariates = cell types + samples, with a total of over 2000 columns in .var). May I wonder how many covariates do you have?
If our solution to put model results in .var is not great given how anndata works - I can try putting that as a dataframe in .varm or reducing column name length - but I need to know how many columns to add to have this issue.
@JZL  :+1: for suggesting to train for a few iterations before starting the full analysis. I will add this comment to the notebook
FYI. We are considering moving the signature estimation to scVI interface where you will control how saving is done

Hello @vitkl,
Many thanks for your reply.
In my case, I indeed found out that I made an error with the input parameters. I set the samples (experiment) as the single Cell ID (barcode). So, yeah, my number of covariates was huge and therefore I got this error. It is now working and running properly.

Hi @alberto-valdeolivas great to hear!
Yeah, important to avoid inputting cell ID as sample ID. Same is true for making sure not to use location ID as sample ID for the main cell2location mode. I will add informative errors when the number of samples is > 500

Added informative error in https://github.com/BayraktarLab/cell2location/commit/d4c4e7f0407af6c9c19a64334e094dafcfe3114e and https://github.com/BayraktarLab/cell2location/commit/8eec6da0f7c6c4ef2025c8c0682af6df90e45312


[RotatingFileHandler] Is it possible to save logs in file per hour option?

Monolog version 1
As in topic. Clients generate large files per day - even 3GB. Is it possible to have file per hour strategy?

Maybe just look at these files and do minimize what information you saving. If you save information likes route matches, doctrine, php deprecations etc, you got an answer why you have 3GB logs per day :)

Nope, I suggest you use logrotate + a simple StreamHandler to have more flexibility here.


[Feature Request]: Mid Scale Up - No Nodes Available Error Events Show

Feature description
A less scary message for when we are waiting for the node to scale up was implemented here: #525
During testing, it appears that it is not as dry cut as that and may lead to misleading situations of status.

More of the log that lead to a success...
(reverse chronological order)
...continued to spin up server successfully...

2022-09-13T18:44:56.000Z [Normal] AttachVolume.Attach succeeded for volume "pvc-6ace6c8b-c96e-4ed9-8498-0cb07f5b97f8"

2022-09-13T18:44:54.024Z [Normal] Successfully assigned odh-notebooks/jupyter-nb-aballant-40redhat-2ecom-0 to ip-10-0-165-177.us-west-2.compute.internal by ip-10-0-213-99

2022-09-13T18:44:43.770Z [Warning] 0/8 nodes are available: 1 node(s) had untolerated taint {node.kubernetes.io/not-ready: }, 2 Insufficient cpu, 2 node(s) had untolerated taint {node-role.kubernetes.io/infra: }, 3 node(s) had untolerated taint {node-role.kubernetes.io/master: }, 6 Insufficient memory. preemption: 0/8 nodes are available: 2 No preemption victims found for incoming pod, 6 Preemption is not helpful for scheduling.

2022-09-13T18:40:06.331Z [Warning] 0/7 nodes are available: 2 Insufficient cpu, 2 node(s) had untolerated taint {node-role.kubernetes.io/infra: }, 3 node(s) had untolerated taint {node-role.kubernetes.io/master: }, 6 Insufficient memory. preemption: 0/7 nodes are available: 2 No preemption victims found for incoming pod, 5 Preemption is not helpful for scheduling.

2022-09-13T18:39:21.000Z [Normal] pod triggered scale-up: [{MachineSet/openshift-machine-api/aisrhods-odh-dmx4v-large-workers-us-west-2a 0->1 (max: 3)}]

2022-09-13T18:39:05.908Z [Warning] 0/7 nodes are available: 2 Insufficient cpu, 2 node(s) had untolerated taint {node-role.kubernetes.io/infra: }, 3 node(s) had untolerated taint {node-role.kubernetes.io/master: }, 6 Insufficient memory. preemption: 0/7 nodes are available: 2 No preemption victims found for incoming pod, 5 Preemption is not helpful for scheduling.

Server requested

We need to handle this in a fashion that is more appealing to the user.
Describe alternatives you've considered

We can hide events that are erroneous
We can turn events to warnings

This all needs more thinking on how we can deterministically tell which are erroneous and what are actual.
Anything else?
No response

cc @cfchase @jedemo @erwangranger @kywalker-rh
I'll bring this up at the UX meeting tomorrow if there is time. Tagging you in advance if there is additional information you want to provide.

Based on the UX meeting & the follow up discussion, we have decided on the following changes:

Suppress event warnings for 30s -- infos are fine

"0/7 nodes are available" error — turn to red after 30s (if its the last message)

Turn our default from error to warning -- we'll pick specific ones to make errors
When scaling up ("TriggeredScaleUp") — stay on info (and that message -- don't impact the log) until the not able to scale up message ("NotTriggerScaleUp")
Change wording of the button always to "Cancel request"

FYI @kywalker-rh

When scaling up (“TriggeredScaleUp”) stay on info Alert (with that message -- and don’t impact the log) until the not able to scale up message (“NotTriggerScaleUp”) is visible

can we only suppress only "[Warning] (.*) nodes are available" messages during this time period?

Ah good clarification @cfchase -- other warnings or if another Info kicks it should show those instead. It should take the last event that is not one of the node ones.
Let me update my comment to match this.

just to be sure .... "suppressing" things only applies to to the top part, right?
If I click "Expand event log", I will be able to see all the events, right?
Just confirming.

just to be sure .... "suppressing" things only applies to to the top part, right?
If I click "Expand event log", I will be able to see all the events, right?
That's my understanding as well.  No changes to the event log are necessary.  We're really only talking about the alert.

Yup, log will remain unaffected, it is entirely the text above the progress bar (custom text we decide) and the Alert that shows up (which will be an event within the log -- the reason & message from the event object)

Can we change the button to simply "Cancel" too? I checked with Katie and its our conclusion that "Cancel request" though accurate, may be confusing when the last button they hit was "Start server".

You guys control the UX -- just cancel it is. We can revisit it in the future with more UX if needed.


create New Task is not working properly

Current Functionality:
When creating new task it's throwing error.
Required Functionality:
No error should be thrown.

closed #42


Update feature request template

Issue #, if available:
Description of changes: Make feature request template cedar-spec specific.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Lgtm, but I left several (minor) comments because I was excited to try out the "suggestion" review feature.

Thank you. Now I could try the "commit suggestion" feature :)


Add test requires: nose

Seems the tests are written in nose format

@franklingu Seems like a good idea. How should this be specified?


The com.contentful.vault.Asset type doesn't work for non-android platforms

com.contentful.vault.Asset contains Android class references that prevent it being used on any other platform.
When trying to compile the error reads "The type android.os.Parcelable$Creator cannot be resolved. It is indirectly referenced from required .class files Brand.java"
Where Brand.java is the class generated from the Product Catalogue example on www.contentful.com.

Hello,
I am not sure if this is the right repository you are posting this issue on. Vault is the library to persist data on Android.
This repository is for generating model classes from a space from Contentful, which will work on all java platforms. If you have a question for this one, please reopen this issue. If you have a issue with Vault, please head over to its repository.

Hi,
The problem is that the model classes generated do not 'work on all java platforms' because the generated classes can have fields of type com.contentful.vault.Asset and this class has a dependency on android. This isn't a Vault issue because it's designed for Android. I don't think generator.java should be using  com.contentful.vault.Asset as a field type if it requires this dependency.

I now understand the problem and will investigate a solution.
Thank you for pointing it out.

Hello again,
to help me evaluate a bit more, can you help me understanding your usecase? What would you do with the generated classes? Would you try and feed them into another ORM?

Hi,
The main thing I'm looking to do is sync content to the web server (tomcat,
jboss etc) whenever the content changes. The generated classes are must
easier to traverse than the standard ones provided by contentful.java
especially when following multiple references where the type is known due
to restrictions applied in the model.
On 8 July 2016 at 15:29, Mario Bodemann<EMAIL_ADDRESS>wrote:

Hello again,
to help me evaluate a bit more, can you help me understanding your
usecase? What would you do with the generated classes? Would you try and
feed them into another ORM?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/contentful/generator.java/issues/9#issuecomment-231374410,
or mute the thread
https://github.com/notifications/unsubscribe/ABmqEm3UDWk1RY7TQAE1iWrR8bGJc0Beks5qTl6xgaJpZM4JHKzu
.

Hello,
I investigated some more and found there is no easy and straightforward way of making the generator produce classes, which can be easily used in the way purposed. I'll keep this issue to keep working on it, but there will not be an easy and fast fix for, I'm afraid.

Thanks for looking into this. If I decide to proceed with contentful I'll
probably fork the contentful.java and Vault projects to remove the Android
dependency. I'd imagine my scenario is very common so hopefully you can
work something out longer term.
On 12 July 2016 at 13:24, Mario Bodemann<EMAIL_ADDRESS>wrote:

Hello,
I investigated some more and found there is no easy and straightforward
way of making the generator produce classes, which can be easily used in
the way purposed. I'll keep this issue to keep working on it, but there
will not be an easy and fast fix for, I'm afraid.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/contentful/generator.java/issues/9#issuecomment-232031589,
or mute the thread
https://github.com/notifications/unsubscribe/ABmqElPaLOIA6P4JyQw5F0ZUYz4laINRks5qU4eAgaJpZM4JHKzu
.


x/net/http2: bidirection streams, write request body data as available

In Kubernetes, we are currently using spdystream as a means to support bidirectional stream-based communication over HTTP connections. We use this to enable ssh-like connectivity and port forwarding from the user's system into a Docker container running in Kubernetes.
The spdystream APIs enable us to create streams at will in both the client and server handler code. For remote command execution, the client creates streams representing stdin, stdout, and stderr. Once the server receives all the streams, it performs a docker exec of whatever process the user wishes, and data is copied between the container process's stdin/stdout/stderr and the spdy streams.
We're hoping that we can eventually move to HTTP/2 and achieve the same functionality; namely, full control over stream creation and data flow in a "hijacked" fashion.
cc @bradfitz @smarterclayton @thockin @lavalamp
(forked from #13443)

When you say "streams", you don't mean anything more specific than being able to
For the client side, can't you just do a RoundTrip with the Request.Body set to the read end of an io.Pipe and then write your stderr/stdout to the write side?
I just put up an /ECHO handler at https://http2.golang.org/ECHO which streams back its output capitalized:
type capitalizeReader struct {
        r io.Reader
}

func (cr capitalizeReader) Read(p []byte) (n int, err error) {
        n, err = cr.r.Read(p)
        for i, b := range p[:n] {
                if b >= 'a' && b <= 'z' {
                        p[i] = b - ('a' - 'A')
                }
        }
        return
}

type flushWriter struct {
        w io.Writer
}

func (fw flushWriter) Write(p []byte) (n int, err error) {
        n, err = fw.w.Write(p)
        if f, ok := fw.w.(http.Flusher); ok {
                f.Flush()
        }
        return
}

func echoCapitalHandler(w http.ResponseWriter, r *http.Request) {
        if r.Method != "PUT" {
                http.Error(w, "PUT required.", 400)
                return
        }
        io.Copy(flushWriter{w}, capitalizeReader{r.Body})
}

And then with a couple modifications to the http2 client, I can now stream an HTTP request body to a server, and read the streamed response body at the same time, even seeing the 1 second delays:
bradfitz@dev-bradfitz-debian2:~$ cat echo.go
package main

import (
        "fmt"
        "io"
        "io/ioutil"
        "log"
        "net/http"
        "os"
        "time"
)

func main() {
        pr, pw := io.Pipe()
        req, err := http.NewRequest("PUT", "https://http2.golang.org/ECHO", ioutil.NopCloser(pr))
        if err != nil {
                log.Fatal(err)
        }
        go func() {
                for {
                        time.Sleep(1 * time.Second)
                        fmt.Fprintf(pw, "It is now %v\n", time.Now())
                }
        }()
        go func() {
                res, err := http.DefaultClient.Do(req)
                if err != nil {
                        log.Fatal(err)
                }
                log.Printf("Got: %#v", res)
                n, err := io.Copy(os.Stdout, res.Body)
                log.Fatalf("copied %d, %v", n, err)
        }()
        select {}
}
bradfitz@dev-bradfitz-debian2:~$ go run echo.go
2015/12/01 22:10:54 Got: &http.Response{Status:"200 OK", StatusCode:200, Proto:"HTTP/2.0", ProtoMajor:2, ProtoMinor:0, Header:http.Header{"Content-Type":[]string{"text/plain; charset=utf-8"}, "Date":[]string{"Tue, 01 Dec 2015 22:10:54 GMT"}}, Body:http.http2transportResponseBody{cs:(*http.http2clientStream)(0xc8203697a0)}, ContentLength:-1, TransferEncoding:[]string(nil), Close:false, Trailer:http.Header(nil), Request:(*http.Request)(0xc8200c4000), TLS:(*tls.ConnectionState)(0xc8203d69a0)}
IT IS NOW 2015-12-01 22:10:54.592452374 +0000 UTC
IT IS NOW 2015-12-01 22:10:55.592869959 +0000 UTC
IT IS NOW 2015-12-01 22:10:56.593126243 +0000 UTC
IT IS NOW 2015-12-01 22:10:57.593371657 +0000 UTC
IT IS NOW 2015-12-01 22:10:58.593660994 +0000 UTC
IT IS NOW 2015-12-01 22:10:59.59395207 +0000 UTC
IT IS NOW 2015-12-01 22:11:00.594216993 +0000 UTC
IT IS NOW 2015-12-01 22:11:01.594496771 +0000 UTC
IT IS NOW 2015-12-01 22:11:02.594795043 +0000 UTC
IT IS NOW 2015-12-01 22:11:03.594974236 +0000 UTC
IT IS NOW 2015-12-01 22:11:04.595187379 +0000 UTC
IT IS NOW 2015-12-01 22:11:05.59544486 +0000 UTC

Do you need more than that?
The change is at https://golang.org/cl/17310 if you want to patch it in and play. (In your $GOPATH/src/net directory, run git fetch https://go.googlesource.com/net refs/changes/10/17310/1 && git cherry-pick FETCH_HEAD)

@bradfitz this looks quite promising. I'm in meetings most of today, so I doubt I'll have time to fiddle, but hopefully I will tomorrow. @smarterclayton WDYT about this?

@bradfitz I'm finally getting some time to look at this. If I have a normal http.Transport or even just http.DefaultClient, how do I set AllowResponseBeforeBody to true without modifying the internal http2 source?

Once that change is submitted you wouldn't need to modify the http2 source.
You'd just write:
package main

import (
    "net/http"
    "golang.org/x/net/http2"
)

func main() {
     c := &http.Client{Transport: &http2.Transport{AllowResponseBeforeBody: true}}
     ....
}

Thanks!

Actually, I'm removing the option. It'll just be on by default.

Even better :smile:
I did get a chance to play around a bit yesterday. Our setup is a bit complicated, because have a proxy in the middle, so the flow is client -> proxy -> backend (and it ultimately connects to a Docker exec session). I was able to hack the proxy to get it to somewhat support proxying HTTP/2, and I was able to successfully round trip a request from the client to the backend and back. The one thing I didn't get working was interactive input and the output coming back immediately. I'm not sure if it's an issue with the way I was trying to proxy the request through, or what, but I'll come back to it at some point in my spare time.
To illustrate what wasn't working correctly:
kubectl exec -i nginx bash<CR>
ls<CR>
date<CR>
exit<CR>

The output from ls and date only show up after you type exit and hit enter. So the round trip isn't completing until the backend exec completes. I think given your demonstration of the echo client/server working, there is something wrong in my setup...

Yeah, the change just submitted even has a couple new tests showing that this works (and will prevent it from ever not working in the future). So I suspect your proxy is buffering a bit too hard.


Implement new UI for vivo-next branch

In the vivo-next branch we have the next version of Vivo that will be public announced next week. The pending pieces now is to implement the new UI interface, the Figma source is here:
https://www.figma.com/proto/kdGqYmldBwysUNcbuc4CNK/Vivo-UI?page-id=0%3A1&node-id=12-15252&viewport=-1005%2C1663%2C0.24&scaling=min-zoom&starting-point-node-id=12%3A15252&hide-ui=1
Some screenshots of the new interface:
default

hoover

opened row

About the interface
The left menu describes the information to show: we support Logs, metrics and traces.
Logs is the only one with special formatting with a timestamp, metadata and events fields. Metrics and Traces will only show the full event per row, no special formatting is needed for this first version.

cc: @trystendsmyth @dfredriksen

merged in vivo-next-ui branch , thank you!


Scenarios

**Software Requirements Specification Scenarios **
Create User Scenarios for this hw.
Write three scenarios for your project by clearly describing:
– The user and their persona (role)
– Preconditions – if any (such as: the user that has already logged in).
– Specific actions to achieve the desired goal (what the user wants to do). Describe each
step until the scenario is completed. Steps must be enumerated.
– Acceptance criteria (the conditions that describe a successful outcome for the action)
Important note: Your user scenarios must be based on realistic user and content. Do
not use sign up or login as scenarios.

I created only one instead of three because it was a huge user scenario and took a lot time but I will create two more to complete the hw

Finally I've completed  and the User Scenarios File is here.