og:image doesn't work for featured_images in an asset pipeline

If I place an image in /assets/images/, it displays correctly as a featured_image of my post, but when it's time to generate a opengraph meta data, it uses a url for that image that doesn't exist.
The line in question seems to be:
https://github.com/nusserstudios/tailbliss/blob/main/layouts/partials/meta.html#L14
Which either doesn't notify the asset pipeline that a new image needs to be generated, or that the source image needs to be included in the build. Not sure how that's supposed to work.

That looks right to me. I'll take a look at it locally, and pull it in. Thank you @andykorth!


node-cli: ensure user can log in after refresh token expiration

After refresh token expiration, the library should prompt users to log in again. We have one colleague who was unable to login and received 400 – Bad Request. Only solution was to delete the entire in their OS credentials.

This has been fixed.


How to enable multi-gpus training?

when i try to pretrain base model, I enable multi-gpus training in pretrain_base.sh by this:
export CUDA_VISIBLE_DEVICES=0,1,2,3
export GPUS_PER_NODE=4

but error occurs as follows:
2023-02-10 11:17:26 - utils.py[line:759] - INFO: ***********************CUDA enviroments for all 4 workers***********************
2023-02-10 11:17:26 - utils.py[line:765] - INFO: rank   0: capabilities =  8.0  ; total memory = 79.347 GB ; name = NVIDIA A100-SXM4-80GB
2023-02-10 11:17:26 - utils.py[line:765] - INFO: rank   1: capabilities =  8.0  ; total memory = 79.347 GB ; name = NVIDIA A100-SXM4-80GB
2023-02-10 11:17:26 - utils.py[line:765] - INFO: rank   2: capabilities =  8.0  ; total memory = 79.347 GB ; name = NVIDIA A100-SXM4-80GB
2023-02-10 11:17:26 - utils.py[line:765] - INFO: rank   3: capabilities =  8.0  ; total memory = 79.347 GB ; name = NVIDIA A100-SXM4-80GB
2023-02-10 11:17:26 - utils.py[line:767] - INFO: ***********************CUDA enviroments for all 4 workers***********************
2023-02-10 11:17:26 - train.py[line:154] - INFO: training on 4 devices (GPUs/TPUs)
2023-02-10 11:17:26 - train.py[line:160] - INFO: max tokens per device = None and max sentences per device = 4
local datafile ../../dataset/pretrain_data/vision_language_examples.tsv slice_id 2 begin to initialize row_count and line_idx-to-offset mappinglocal datafile ../../dataset/pretrain_data/vision_language_examples.tsv slice_id 1 begin to initialize row_count and line_idx-to-offset mapping

local datafile ../../dataset/pretrain_data/vision_language_examples.tsv slice_id 3 begin to initialize row_count and line_idx-to-offset mapping
2023-02-10 11:17:26 - trainer.py[line:458] - INFO: Preparing to load checkpoint ../../checkpoints/ofa_base.pt
2023-02-10 11:17:26 - trainer.py[line:624] - INFO: No existing checkpoint found ../../checkpoints/ofa_base.pt
2023-02-10 11:17:26 - trainer.py[line:639] - INFO: loading train data for epoch 1
local datafile ../../dataset/pretrain_data/vision_language_examples.tsv slice_id 0 begin to initialize row_count and line_idx-to-offset mapping
local datafile ../../dataset/pretrain_data/vision_language_examples.tsv slice_id 2 finished initializing row_count and line_idx-to-offset mapping
file ../../dataset/pretrain_data/vision_language_examples.tsv slice_id 2 row count 75 total row count 300
local datafile ../../dataset/pretrain_data/vision_language_examples.tsv slice_id 0 finished initializing row_count and line_idx-to-offset mappinglocal datafile ../../dataset/pretrain_data/vision_language_examples.tsv slice_id 3 finished initializing row_count and line_idx-to-offset mapping

file ../../dataset/pretrain_data/vision_language_examples.tsv slice_id 0 row count 75 total row count 300file ../../dataset/pretrain_data/vision_language_examples.tsv slice_id 3 row count 75 total row count 300

local datafile ../../dataset/pretrain_data/vision_language_examples.tsv slice_id 1 finished initializing row_count and line_idx-to-offset mapping
file ../../dataset/pretrain_data/vision_language_examples.tsv slice_id 1 row count 75 total row count 300
/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torchvision/transforms/functional.py:405: UserWarning: Argument interpolation should be of type InterpolationMode instead of int. Please, use InterpolationMode enum.
  "Argument interpolation should be of type InterpolationMode instead of int. "
/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torchvision/transforms/functional.py:405: UserWarning: Argument interpolation should be of type InterpolationMode instead of int. Please, use InterpolationMode enum.
  "Argument interpolation should be of type InterpolationMode instead of int. "
/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torchvision/transforms/functional.py:405: UserWarning: Argument interpolation should be of type InterpolationMode instead of int. Please, use InterpolationMode enum.
  "Argument interpolation should be of type InterpolationMode instead of int. "
/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torchvision/transforms/functional.py:405: UserWarning: Argument interpolation should be of type InterpolationMode instead of int. Please, use InterpolationMode enum.
  "Argument interpolation should be of type InterpolationMode instead of int. "
slice_id 2 seek offset 150
slice_id 3 seek offset 225
slice_id 1 seek offset 75
Total steps 950, warmup steps 9, warmup_factor 0.1111111111111111
Total steps 950, warmup steps 9, warmup_factor 0.1111111111111111
slice_id 0 seek offset 0
Total steps 950, warmup steps 9, warmup_factor 0.1111111111111111
Total steps 950, warmup steps 9, warmup_factor 0.1111111111111111
2023-02-10 11:17:27 - trainer.py[line:703] - INFO: begin training epoch 1
2023-02-10 11:17:27 - train.py[line:305] - INFO: Start iterating over samples
Traceback (most recent call last):
  File "/nvme/zhaozhiyuan/hthl/OFA/trainer.py", line 879, in train_step
    self._check_grad_norms(grad_norm)
  File "/nvme/zhaozhiyuan/hthl/OFA/trainer.py", line 1427, in _check_grad_norms
    + "-" * 80
FloatingPointError: Fatal error: gradients are inconsistent between workers. Try --ddp-backend=legacy_ddp. Or are you mixing up different generation of GPUs in training?
--------------------------------------------------------------------------------
grad_norm across the workers:
rank   0 = 43.87894113
rank   1 = 36.76721862
rank   2 = 32.03398652
rank   3 = 32.47642299

--------------------------------------------------------------------------------

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "../../train.py", line 538, in <module>
    cli_main()
  File "../../train.py", line 531, in cli_main
    distributed_utils.call_main(cfg, main)
  File "/nvme/zhaozhiyuan/hthl/OFA/fairseq/fairseq/distributed/utils.py", line 374, in call_main
    distributed_main(cfg.distributed_training.device_id, main, cfg, kwargs)
  File "/nvme/zhaozhiyuan/hthl/OFA/fairseq/fairseq/distributed/utils.py", line 348, in distributed_main
    main(cfg, **kwargs)
  File "../../train.py", line 199, in main
    valid_losses, should_stop = train(cfg, trainer, task, epoch_itr)
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/contextlib.py", line 74, in inner
    return func(*args, **kwds)
  File "../../train.py", line 310, in train
    log_output = trainer.train_step(samples)
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/contextlib.py", line 74, in inner
    return func(*args, **kwds)
  File "/nvme/zhaozhiyuan/hthl/OFA/trainer.py", line 916, in train_step
    **extra_kwargs,
  File "/nvme/zhaozhiyuan/hthl/OFA/tasks/ofa_task.py", line 334, in train_step
    loss, sample_size, logging_output = criterion(model, sample, update_num=update_num)
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/nvme/zhaozhiyuan/hthl/OFA/criterions/label_smoothed_cross_entropy.py", line 199, in forward
    net_output = model(**sample["net_input"])
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/nvme/zhaozhiyuan/hthl/OFA/fairseq/fairseq/distributed/module_proxy_wrapper.py", line 55, in forward
    return self.module(*args, **kwargs)
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torch/nn/parallel/distributed.py", line 873, in forward
    if torch.is_grad_enabled() and self.reducer._rebuild_buckets():
RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one. This error indicates that your module has parameters that were not used in producing loss. You can enable unused parameter detection by passing the keyword argument `find_unused_parameters=True` to `torch.nn.parallel.DistributedDataParallel`, and by
making sure all `forward` function outputs participate in calculating loss.
If you already have done the above, then the distributed data parallel module wasn't able to locate the output tensors in the return value of your module's `forward` function. Please include the loss function and the structure of the return value of `forward` of your module when reporting this issue (e.g. list, dict, iterable).
Parameter indices which did not receive grad for rank 0: 443 446 447 456 457 670 671 672 673 674 675
 In addition, you can set the environment variable TORCH_DISTRIBUTED_DEBUG to either INFO or DETAIL to print out information about which particular parameters did not receive gradient on this rank as part of this error
Traceback (most recent call last):
  File "/nvme/zhaozhiyuan/hthl/OFA/trainer.py", line 879, in train_step
    self._check_grad_norms(grad_norm)
  File "/nvme/zhaozhiyuan/hthl/OFA/trainer.py", line 1427, in _check_grad_norms
    + "-" * 80
FloatingPointError: Fatal error: gradients are inconsistent between workers. Try --ddp-backend=legacy_ddp. Or are you mixing up different generation of GPUs in training?
--------------------------------------------------------------------------------
grad_norm across the workers:
rank   0 = 43.87894113
rank   1 = 36.76721862
rank   2 = 32.03398652
rank   3 = 32.47642299

--------------------------------------------------------------------------------

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "../../train.py", line 538, in <module>
    cli_main()
  File "../../train.py", line 531, in cli_main
    distributed_utils.call_main(cfg, main)
  File "/nvme/zhaozhiyuan/hthl/OFA/fairseq/fairseq/distributed/utils.py", line 374, in call_main
    distributed_main(cfg.distributed_training.device_id, main, cfg, kwargs)
  File "/nvme/zhaozhiyuan/hthl/OFA/fairseq/fairseq/distributed/utils.py", line 348, in distributed_main
    main(cfg, **kwargs)
  File "../../train.py", line 199, in main
    valid_losses, should_stop = train(cfg, trainer, task, epoch_itr)
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/contextlib.py", line 74, in inner
    return func(*args, **kwds)
  File "../../train.py", line 310, in train
Traceback (most recent call last):
  File "/nvme/zhaozhiyuan/hthl/OFA/trainer.py", line 879, in train_step
    log_output = trainer.train_step(samples)
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/contextlib.py", line 74, in inner
    return func(*args, **kwds)
  File "/nvme/zhaozhiyuan/hthl/OFA/trainer.py", line 916, in train_step
        **extra_kwargs,self._check_grad_norms(grad_norm)
  File "/nvme/zhaozhiyuan/hthl/OFA/tasks/ofa_task.py", line 334, in train_step

  File "/nvme/zhaozhiyuan/hthl/OFA/trainer.py", line 1427, in _check_grad_norms
    loss, sample_size, logging_output = criterion(model, sample, update_num=update_num)
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    + "-" * 80
    FloatingPointErrorreturn forward_call(*input, **kwargs)
  File "/nvme/zhaozhiyuan/hthl/OFA/criterions/label_smoothed_cross_entropy.py", line 199, in forward
: Fatal error: gradients are inconsistent between workers. Try --ddp-backend=legacy_ddp. Or are you mixing up different generation of GPUs in training?
--------------------------------------------------------------------------------
grad_norm across the workers:
rank   0 = 43.87894113
rank   1 = 36.76721862
rank   2 = 32.03398652
rank   3 = 32.47642299

--------------------------------------------------------------------------------

During handling of the above exception, another exception occurred:

net_output = model(**sample["net_input"])Traceback (most recent call last):

  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
  File "../../train.py", line 538, in <module>
    cli_main()
      File "../../train.py", line 531, in cli_main
return forward_call(*input, **kwargs)
  File "/nvme/zhaozhiyuan/hthl/OFA/fairseq/fairseq/distributed/module_proxy_wrapper.py", line 55, in forward
    return self.module(*args, **kwargs)
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    distributed_utils.call_main(cfg, main)
  File "/nvme/zhaozhiyuan/hthl/OFA/fairseq/fairseq/distributed/utils.py", line 374, in call_main
    distributed_main(cfg.distributed_training.device_id, main, cfg, kwargs)    return forward_call(*input, **kwargs)

  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torch/nn/parallel/distributed.py", line 873, in forward
  File "/nvme/zhaozhiyuan/hthl/OFA/fairseq/fairseq/distributed/utils.py", line 348, in distributed_main
Traceback (most recent call last):
  File "/nvme/zhaozhiyuan/hthl/OFA/trainer.py", line 879, in train_step
    main(cfg, **kwargs)
      File "../../train.py", line 199, in main
if torch.is_grad_enabled() and self.reducer._rebuild_buckets():
RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one. This error indicates that your module has parameters that were not used in producing loss. You can enable unused parameter detection by passing the keyword argument `find_unused_parameters=True` to `torch.nn.parallel.DistributedDataParallel`, and by
making sure all `forward` function outputs participate in calculating loss.
If you already have done the above, then the distributed data parallel module wasn't able to locate the output tensors in the return value of your module's `forward` function. Please include the loss function and the structure of the return value of `forward` of your module when reporting this issue (e.g. list, dict, iterable).
Parameter indices which did not receive grad for rank 2: 443 446 447 456 457 670 671 672 673 674 675
 In addition, you can set the environment variable TORCH_DISTRIBUTED_DEBUG to either INFO or DETAIL to print out information about which particular parameters did not receive gradient on this rank as part of this error
    valid_losses, should_stop = train(cfg, trainer, task, epoch_itr)
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/contextlib.py", line 74, in inner
    return func(*args, **kwds)
  File "../../train.py", line 310, in train
    log_output = trainer.train_step(samples)
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/contextlib.py", line 74, in inner
        return func(*args, **kwds)
  File "/nvme/zhaozhiyuan/hthl/OFA/trainer.py", line 916, in train_step
self._check_grad_norms(grad_norm)
**extra_kwargs,  File "/nvme/zhaozhiyuan/hthl/OFA/trainer.py", line 1427, in _check_grad_norms

  File "/nvme/zhaozhiyuan/hthl/OFA/tasks/ofa_task.py", line 334, in train_step
    loss, sample_size, logging_output = criterion(model, sample, update_num=update_num)
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
        return forward_call(*input, **kwargs)
  File "/nvme/zhaozhiyuan/hthl/OFA/criterions/label_smoothed_cross_entropy.py", line 199, in forward
    net_output = model(**sample["net_input"])
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/nvme/zhaozhiyuan/hthl/OFA/fairseq/fairseq/distributed/module_proxy_wrapper.py", line 55, in forward
+ "-" * 80
    FloatingPointError: Fatal error: gradients are inconsistent between workers. Try --ddp-backend=legacy_ddp. Or are you mixing up different generation of GPUs in training?
--------------------------------------------------------------------------------
grad_norm across the workers:
rank   0 = 43.87894113
rank   1 = 36.76721862
rank   2 = 32.03398652
rank   3 = 32.47642299

--------------------------------------------------------------------------------
return self.module(*args, **kwargs)
During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "../../train.py", line 538, in <module>
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    cli_main()
  File "../../train.py", line 531, in cli_main
    return forward_call(*input, **kwargs)
    distributed_utils.call_main(cfg, main)  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torch/nn/parallel/distributed.py", line 873, in forward

  File "/nvme/zhaozhiyuan/hthl/OFA/fairseq/fairseq/distributed/utils.py", line 374, in call_main
    distributed_main(cfg.distributed_training.device_id, main, cfg, kwargs)
  File "/nvme/zhaozhiyuan/hthl/OFA/fairseq/fairseq/distributed/utils.py", line 348, in distributed_main
        if torch.is_grad_enabled() and self.reducer._rebuild_buckets():main(cfg, **kwargs)
  File "../../train.py", line 199, in main

RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one. This error indicates that your module has parameters that were not used in producing loss. You can enable unused parameter detection by passing the keyword argument `find_unused_parameters=True` to `torch.nn.parallel.DistributedDataParallel`, and by
making sure all `forward` function outputs participate in calculating loss.
If you already have done the above, then the distributed data parallel module wasn't able to locate the output tensors in the return value of your module's `forward` function. Please include the loss function and the structure of the return value of `forward` of your module when reporting this issue (e.g. list, dict, iterable).
Parameter indices which did not receive grad for rank 1: 443 446 447 456 457 670 671 672 673 674 675
 In addition, you can set the environment variable TORCH_DISTRIBUTED_DEBUG to either INFO or DETAIL to print out information about which particular parameters did not receive gradient on this rank as part of this error
valid_losses, should_stop = train(cfg, trainer, task, epoch_itr)
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/contextlib.py", line 74, in inner
    return func(*args, **kwds)
  File "../../train.py", line 310, in train
    log_output = trainer.train_step(samples)
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/contextlib.py", line 74, in inner
    return func(*args, **kwds)
  File "/nvme/zhaozhiyuan/hthl/OFA/trainer.py", line 916, in train_step
    **extra_kwargs,
  File "/nvme/zhaozhiyuan/hthl/OFA/tasks/ofa_task.py", line 334, in train_step
    loss, sample_size, logging_output = criterion(model, sample, update_num=update_num)
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/nvme/zhaozhiyuan/hthl/OFA/criterions/label_smoothed_cross_entropy.py", line 199, in forward
    net_output = model(**sample["net_input"])
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/nvme/zhaozhiyuan/hthl/OFA/fairseq/fairseq/distributed/module_proxy_wrapper.py", line 55, in forward
    return self.module(*args, **kwargs)
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torch/nn/parallel/distributed.py", line 873, in forward
    if torch.is_grad_enabled() and self.reducer._rebuild_buckets():
RuntimeError: Expected to have finished reduction in the prior iteration before starting a new one. This error indicates that your module has parameters that were not used in producing loss. You can enable unused parameter detection by passing the keyword argument `find_unused_parameters=True` to `torch.nn.parallel.DistributedDataParallel`, and by
making sure all `forward` function outputs participate in calculating loss.
If you already have done the above, then the distributed data parallel module wasn't able to locate the output tensors in the return value of your module's `forward` function. Please include the loss function and the structure of the return value of `forward` of your module when reporting this issue (e.g. list, dict, iterable).
Parameter indices which did not receive grad for rank 3: 443 446 447 456 457 670 671 672 673 674 675
 In addition, you can set the environment variable TORCH_DISTRIBUTED_DEBUG to either INFO or DETAIL to print out information about which particular parameters did not receive gradient on this rank as part of this error
terminate called without an active exception
terminate called without an active exception
terminate called without an active exception
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 76079 closing signal SIGTERM
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: -6) local_rank: 0 (pid: 76077) of binary: /nvme/zhaozhiyuan/anaconda3/envs/ofa/bin/python3
Traceback (most recent call last):
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torch/distributed/launch.py", line 193, in <module>
    main()
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torch/distributed/launch.py", line 189, in main
    launch(args)
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torch/distributed/launch.py", line 174, in launch
    run(args)
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torch/distributed/run.py", line 713, in run
    )(*cmd_args)
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torch/distributed/launcher/api.py", line 131, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/nvme/zhaozhiyuan/anaconda3/envs/ofa/lib/python3.7/site-packages/torch/distributed/launcher/api.py", line 261, in launch_agent
    failures=result.failures,
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
============================================================
../../train.py FAILED
------------------------------------------------------------
Failures:
[1]:
  time      : 2023-02-10_11:17:32
  host      : SH-IDCA1404-10-140-54-21
  rank      : 1 (local_rank: 1)
  exitcode  : -6 (pid: 76078)
  error_file: <N/A>
  traceback : Signal 6 (SIGABRT) received by PID 76078
[2]:
  time      : 2023-02-10_11:17:32
  host      : SH-IDCA1404-10-140-54-21
  rank      : 3 (local_rank: 3)
  exitcode  : 1 (pid: 76080)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2023-02-10_11:17:32
  host      : SH-IDCA1404-10-140-54-21
  rank      : 0 (local_rank: 0)
  exitcode  : -6 (pid: 76077)
  error_file: <N/A>
  traceback : Signal 6 (SIGABRT) received by PID 76077
============================================================

Could someone please kindly tell me how to fix this or how to enable multi-gpus training? thanks!

solved by https://github.com/facebookresearch/fairseq/issues/3920

Hi! I met the same problem, did you just add –ddp-backend=no_c10d ?

Same issue with me as well, I met the problem with --ddp-backend=no_c10d and even changing adam to nag also giving same error

I am facing the same issue except I got 0 for grad_norm:
grad_norm across the workers:
rank   0 = 18.27746773
rank   1 = 0.00000000
rank   2 = 18.27746773
rank   3 = 18.27746773


Search bar and status orb display problems.

Using master css and Chromium 37

And with Firefox

Fixed in latest master


uORB/ mavlink support for covariance version of messages

For debugging and support of external navigators we should have the ability to send the following  messages version over mavlink, and similar messages between processes using uORB on the autopilot:
MAVLink Covariance Messages

GLOBAL_POSITION_INT_COV
LOCAL_POSITION_NED_COV (sort of supported, but doesn't follow mavlink spec)
ATTITUDE_QUATERNION_COV
WIND_COV

Issues:

We have no default state order agreed on between the estimators (this should be defined by MAVLink)
uORB doesn't have any messages that contain this info

Hey, this issue has been closed because the label status/STALE is set and there were no updates for 30 days. Feel free to reopen this issue if you deem it appropriate.
(This is an automated comment from GitMate.io.)


Add delete option for metadata sign

This change allows deleting a signing process with rstuf admin metadata sign --delete with an input - a role whoose signate should be deleted and DAS signing process - terminated
Fixes #370
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality not to work as expected)

Additional requirements

[x] Tests have been added for the bug fix or new feature
[tbd ] Docs have been added for the bug fix or new feature

Code of Conduct
By submitting this PR, you agree to follow our Code of Conduct.

[x] I agree to follow this project's Code of Conduct

@MVrachev , @kairoaraujo I just push-forced with the new test coverage.


Update all docs to reflect adapter config change

This tracking issue tracks updating all the docs to reflect that in juttle-config.js, adapter config blocks should be below XXX and not juttle-XXX-adapter.

[x] outrigger
[x] juttle-elastic-adapter
[x] juttle-gmail-adapter
[x] juttle-graphite-adapter
[x] juttle-influx-adapter
[ ] juttle-mysql-adapter
[ ] juttle-postgres-adapter
[ ] juttle-sqlite-adapter
[x] juttle-twitter-adapter

I'll start with me as the assignee--once I've updated outrigger and juttle-gmail-adapter I'll reassign to someone who has an unchecked item.

Note that for each of these we should update the instructions in the README and add the following to the CHANGELOG:

WARNING: As part of the update to juttle 0.3.0, the configuration syntax for adapters changed from the name of the module ("juttle-elastic-adapter") to the type of the adapter ("elastic").

The remaining adapters are @VladVega's, so assigning to him


What is the status of FLOPO?

And what is the relationship with PO and TO?

As an addition: the reason for not relying on TO is that TO has very few axioms, and mapping to or reusing TO would be largely a manual exercise or based on label matching. FLOPO is a fully defined ontology, i.e., 100% of its classes have an axiomatic definition.


Remove framework DB dependency from android-driver

Looks like this is only used for a convenience function which we should refactor to take a SupportSQLiteOpenHelper.Factory

We've decided to rely on R8/ProGuard for handling this.


Not compatible with rollup-plugin-svelte version 7

Describe the bug
When using rollup-plugin-svelte version 7, the FilePond element isn't instantiated correctly. I get this when loading the app in a browser:
index.mjs:1849 Uncaught TypeError: Illegal constructor
    at new SvelteElement (index.mjs:1849)
    at new App (App.svelte:33)
    at main.js:3
    at main.js:5

To Reproduce
The only changes:

in package.json: "rollup-plugin-svelte": "^7.0.0"
in rollup.config.js:

svelte({
	// enable run-time checks when not in production
	// we'll extract any component CSS out into
	// a separate file - better for performance
	compilerOptions: {
		// enable run-time checks when not in production
		dev: !production,
		css: true,
		customElement: true
	},
	// for importing filepond css from nodemodules
	preprocess: sveltePreprocess({ postcss: true }),
}),

It seems to me that Component.svelte isn't even loaded (in filepond-svelte).
Repository: https://github.com/ilkkah/filepond-example

Just tested with v7 and works correctly.
It might have to do with setting customElement to true
Will close because can't reproduce.

True, the customElement breaks it.
Could something be done to that? I still need the customElement, it's useful (will contain only its own CSS for example). If filepond-svelte would work with customElement, that would be good too.

That won't work, the FilePond core is not written as a custom element. So CSS selectors will probably not find the right elements. There's also challenges with shadow DOM and event handling that is currently not compatible.


server.hostname should default to <IP_ADDRESS> and server.port should default to 3042

As titled
We should also verify that empty strings are correctly defaulted to those values.

I've always wondered: any specific reason for 3042 as the port number? Is it a magic number? ✨

42 is the answer. 3000 is the default for hosting an application in Ruby on Rails.

I should have expected no less from the creator of Undici!


[Snyk] Upgrade io.grpc:grpc-stub from 1.58.0 to 1.59.0

This PR was automatically created by Snyk using the credentials of a real user.Snyk has created this PR to upgrade io.grpc:grpc-stub from 1.58.0 to 1.59.0.
:information_source: Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly identify and fix newly disclosed vulnerabilities when they affect your project.

The recommended version is 1 version ahead of your current version.
The recommended version was released 24 days ago, on 2023-10-20.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open upgrade PRs.
For more information:
🧐 View latest project report
🛠 Adjust upgrade PR settings
🔕 Ignore this dependency or unsubscribe from future upgrade PRs

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Use formatter API for better integration

I'd recommend to use the languages API to register a formatting edits provider because it will allow for a better integration in VS Code, esp wrt #21. There is the registerDocumentFormattingEditProvider-function which is the way to introduce a formatter for certain document types. It will integrate you into the Format Code action (context menu, F1) and also into the Format On Save-feature.
It's just a matter of calling this code from within DocumentFormattingEditProvider

Hello, @jrieken,
Thanks for your issue. I'll work on this today.

Sorry for question, but I don't fully understand this functionality.
As far as I understand, this API adds my plugin in Format Code command. But what if I add this functionality for all my own plugins? For example, csscomb, autoprefixer and stylefmt and then I press Format Code?

Definitely registerDocument(Range)FormattingEditProvider because then it's known as a formatter and we treat it like one (showing the entry in the context menu, calling you when editor.formatOnSave is set etc).
When you register a provider you always say what documents you are registering it for, for instance CSS files on disk is {language: 'css', scheme: 'file'} or any html file is just html. When format is invoked we match the current document against the list of available providers and pick the best match. Since there is some ambiguity in that we are currently adding more toggles to disable our default/builtin providers - see #11609

Thanks you for your answer ❤
But small clarification:

I should provide formatOnSave option for users?
In this case, I should have such functions in the plugin without registerTextEditorCommand for stylefmt.execute? Just only:

registerDocumentFormattingEditProvider
registerDocumentRangeFormattingEditProvider

yeah - I would deprecate the custom formatOnSave option and rely on registerDocument(Range)FormattingEditProvider. Also note that registerDocumentRangeFormattingEditProvider  is a super-set-provider to registerDocumentFormattingEditProvider  since the former is a special case of the latter.

Thanks for the help, @jrieken.
I will update this and other plugins tomorrow.

@jrieken, thanks for motivation. This API used in version 2.0.0.


Prefer local node_modules/.bin?

Say I have 2 projects that use different versions of coffee-script, and I'd like $ coffee to run the local version from each project directory.  I'd also like to globally install some version of coffee-script so Atom plugins don't explode.  Is this possible?
I got stuck at the point where shims are added first in my path.  Seems like a globally installed module always takes precedence...

I have an idea for a plugin to make this possible, but at present it is not. It's not related to nodenv in any way. Local node_modules bins simply aren't on the path, and a non-nodenv installation isn't any different. There are alternatives, however.

Add ./node_modules/.bin to your PATH. Though there are security reasons to avoid this.

$(npm bin)/coffee will run the local coffee. And npm-bin works anywhere in the project (not just at root).

Use npm run-scripts. When an npm script is invoked via npm run foo, npm prepends $(npm-bin) to your PATH. So scripts can invoke any locally installed bin without the need to prefix it.
"scripts": {
  "compile": "coffee"
}

Now npm run compile will use your local coffee.
The complication for making a plugin for this is finding a way to make nodenv aware of all your local node_modules. It can't shim what it doesn't know about. Best I can think of so far, is that the plugin prepends $(npm bin) to the PATH during the exec hook. But this would only work for modules that have been installed globally and are thus shimmed. So for any locally-installed bin, one would need to install it globally as well. Then the global shim would trigger nodenv, and the exec hook could prepend local bin path.

Thanks for all the thoughts!
I'm in bucket #1.  I've already added ./node_modules/.bin to my path, but I can't give them precedence over globally installed modules which nodenv pretends to my path.
Any thoughts on configuring where in the path the shims are added?

On Feb 11, 2016, at 5:22 AM, Jason Karns<EMAIL_ADDRESS>wrote:
I have an idea for a plugin to make this possible, but at present it is not. It's not related to nodenv in any way. Local node_modules bins simply aren't on the path, and a non-nodenv installation isn't any different. There are alternatives, however.
Add ./node_modules/.bin to your PATH. Though there are security reasons to avoid this.
$(npm bin)/coffee will run the local coffee. And npm-bin works anywhere in the project (not just at root).
Use npm run-scripts. When an npm script is invoked via npm run foo, npm prepends $(npm-bin) to your PATH. So scripts can invoke any locally installed bin without the need to prefix it.
"scripts": {
"compile": "coffee"
}
Now npm run compile will use your local coffee.
The complication for making a plugin for this is finding a way to make nodenv aware of all your local node_modules. It can't shim what it doesn't know about. Best I can think of so far, is that the plugin prepends $(npm bin) to the PATH during the exec hook. But this would only work for modules that have been installed globally and are thus shimmed. So for any locally-installed bin, one would need to install it globally as well. Then the global shim would trigger nodenv, and the exec hook could prepend local bin path.
—
Reply to this email directly or view it on GitHub.

nodenv alters PATH when the output of nodenv init is sourced (or evaled depending on your setup)
So you just need to prepend ./node_modules/.bin to PATH after you've invoked nodenv-init

Closing since there's a workaround:


[Release 1.27] Cherry-pick 2265 (update golang to 1.18.8)

Description
Cherry-pick https://github.com/tigera/operator/pull/2265
Update golang to 1.18.8 to avoid new CVEs

For PR author

[ ] Tests for change.
[ ] If changing pkg/apis/, run make gen-files
[ ] If changing versions, run make gen-versions

For PR reviewers
A note for code reviewers - all pull requests must have the following:

[ ] Milestone set according to targeted release.
[ ] Appropriate labels:

kind/bug if this is a bugfix.
kind/enhancement if this is a a new feature.
enterprise if this PR applies to Calico Enterprise only.

/merge-when-ready

OK, I will merge the pull request when it's ready, leave the commits as is when I merge it, and leave the branch after I've merged it.


[b/343058608] Extract master keys from HMS

Extract master keys and save them in the master-keys.csv file.

I see that migration-metadata-flag is the destination for this merge. Are there any consequences for main?
AFAIK this shouldn't require a PR, the merge could be done locally and pushed later.

@kpiotrowski To be honest I don't see a lot of potential to reduce code duplication. I will analyze the duplication after merging the PR chain and try to come up with some solution if possible.
@misolt I've created a PR chain with a few PRs dependent on each other, so that they are easier to review. After merging the first one to the main branch, the destination of the second is automatically changing to main. Then after merging the second one, the same happens for the third etc.


Expand the README on running a sandbox node.

While onboarding some new tem members, we discovered that the README on running a sandboxed Mina daemon was missing some important information. This PR expands the README to give that information and explain why it's important.

!ci-build-me

!ci-build-me

!ci-build-me

!ci-build-me


Can I create a 3D point cloud from the random walk

Can I leverage the random walk code to generate 3D point cloud points? If yes, what needs to change?

Sort of, the random_walk function will return a fractal_result object where the image_array attribute is basically a pixellated 3D point cloud (if the depth > 1 is passed to random_walk). If you convert the pixels to coordinates according to the visitation count you would get a point cloud.

Closing due to lack of activity.


Add installation info on README

Make users know that taggle.js is available on Bower and NPM.

:+1:

:)


V0.2.0/v0.3.0

Alchemist v0.2.0 - SQLite/HashMap breakage
Due to the fact that I rewrote everything to use sqlite rather than
using postgresql I'm bumping the version to v0.2.0 rather than doing
a minor bump. The benefit of this though is that many people already
have sqlite installed as a dependency on their system. The other benefit
of it is that it's much lighter on the system in terms of resources
making it real nice for a command line tool like Alchemist. The jump to
v0.2.0 makes sense because everything was switched from Vecs to HashSets
breaking anyone using code based off this.
Release Notes:

HashSets rather than Vecs for increased efficiency and fitting the needs
of how Alchemist works
SQLite rather than Postgresql for lighter dev dependencies and less
resources being used on the system.

Upgrade

Error Fixes, Warning Suppression, and Error Handling

Adds better error handling for DB connections when they aren't found
Fixes errors in building the code because of mismatched brackets
Supresses warnings from AUR Enum for unused parts of it
Adds empty helper method to the Package type to create a null version
V0.3.0
Add Void Linux Support
Change how distro checking works
Various administrative tasks and doc updates

I really screwed something up trying to merge this. Going to close and do again.


[wait for #1368]  [tests] Add more layer common unittests

Add more layer common unittests without involving runContext for the
layers.
Signed-off-by: Parichay Kapoor<EMAIL_ADDRESS>

:memo: TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #1369. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://nnstreamer.mooo.com/.


Animated ToggleButtons

This branch introduces animated ToggleButtons. The animation is currently implemented as an option that can be switched off by setting the 'animated' property to false. If animated is set to false the button behaves exactly as it did previously, including the way of controlling the button appearance via css. For the future it might make sense to switch to the animated way completely to reduce code complexity.

Cool implementation. However I as a small enhancement it would be better to shorten the animation times to maybe 0.1 or 0.2sec instead of the 0.5. I tested this for a while on my test Tocuhpad app and 0.5 is a bit too long for user experience. (Just my opinion)
That doesn't mean that I don't like the implementation.

By animation do you mean the sliding of the knob or the fading of the background color? I assume you mean the latter, because otherwise I dont know how you would get the 0.5 seconds from. If thats the case than I agree. Please note however that the background color fading is not implemented in this pull requests. I'll put it into another pull request once this one has been merged.

Looks pretty nice, though we have a few concerns:

The slider knob obscures the option you are going to select, which we feel to be a usability problem.
We worry about the performance impact of having only a slideable version of ToggleButton.
onleave should probably be implemented to restrict sliding to within the bounds of the ToggleButton.

For those reasons, we don't think it makes sense to merge this into onyx.
However, we liked it so much that we want to feature it in our upcoming widget catalog.
We hope to have a expansive community plugin and widget set like other frameworks, and animated ToggleButtons should serve as a great example.

All right then. If you are going to feature it I suggest you use AnimatedToggleButton kind which is an extension of the ToggleButton kind rather than an altered version of the original kind. Its hosted here: https://github.com/MaKleSoft/enyo-plugins/tree/master/v2/AnimatedToggleButton

Yep, that's the one we'll link to.


stdenv-setup: fix substituteAll with set -eu

Motivation for this change
Environment variable filter in substituteAll was not precise and produced undefined and invalid variable names.  Vladimír Čunát tried to fix that in [1], but env -0 did not work during Darwin bootstrap, so [2] reverted this change and replaced an error due to invalid variables with a warning.  Recently in #28057 John Ericson added set -u to setup.sh and undefined variables made the setup fail during e.g. nix-build -A gnat with setup: line 519: !varName: unbound variable.
This patch fixes the cause of such warnings as [3]:
 WARNING: substitution variables should be valid bash names,
   "ccLDFlags+" isn't and therefore was skipped; it might be caused
   by multi-line phases in variables - see #14907 for details.

[1] https://github.com/NixOS/nixpkgs/commit/62fc8859c10dd18b005b9bcaa0b429103d7661d9
[2] https://github.com/NixOS/nixpkgs/commit/81df0354290389128077e00edfd2368eeeea0c24
[3] https://github.com/NixOS/nixpkgs/pull/14907#issuecomment-240309994
Things done
I'm going to test this soon.

[ ] Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] Linux

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

Ugh, overriding a builtin feels dirty. Does env -0 work on Darwin? That should allow safe parsing even if the variables contain newlines.

This is what [1] above tried to do, but env -0 does not work on Darwin.

How about awk 'BEGIN { for (v in ENVIRON) print v }' then?

Good idea!

env -0 works fine on Darwin inside most Nix contexts, because we just use the standard Nix tooling (coreutils, bash) in any pure context. The only place to be worried is the occasional script that runs under /bin/sh, where we can no longer control the PATH and bash being used.

@copumpkin setup.sh is needed on Darwin before any bash from nixpkgs is available. See https://github.com/NixOS/nixpkgs/commit/81df0354290389128077e00edfd2368eeeea0c24.

@dezgeg

overriding a builtin feels dirty

Actually, overriding builtins with functions is well defined:
 2. If the name does not match a function, the shell searches for it in the list of shell builtins.

https://www.gnu.org/software/bash/manual/html_node/Command-Search-and-Execution.html
Nevertheless, I'm going to continue with awk for now.
Could someone test this on hydra? I have successfully built some packages on Linux and Darwin with this pull request, but I don't know how to test properly without hydra.

@orivej are you sure that's still true? I thought that was only true because nix-shell used the host bash, but now that we force it to use the Nix bash (and maybe coreutils?) we might be safer. If not, can we just force setup.sh to incorporate a fixed version of coreutils?

@copumpkin You are right, since 2bc7b4e134079cf72307538e57b8968cfb27d70c Darwin always uses Bash from nixpkgs for setup.sh. However, over programs from bootstrap-tools (including env and awk) are not in PATH during early stages (when the shell is ${bootstrapTools}/bin/bash), e.g. env or awk 'BEGIN {}' at the beginning of setup.sh will cause failure, so it is not quite safe to use them at the moment… Currently bootstrap succeeds because early stages do not call substituteAll

err in nix-shell, the impure bash will be used---I've certainly learned that! If any dependency has a setup hook or similar using substitute, it will be run when the nix-shell is entered.

Thanks @orivej!


Linux ARM build

It would be awesome if I could run LightTable on my Samsung Chromebook. Perhaps it's easy to do? If not, I understand it's not exactly a priority.
If applicable, I'm willing to invest time in this to get it working.

Maybe this is useful: https://github.com/jalbam/nwjs_rpi

@jalbam Light Table switched from node-webkit (NW.js) to Electron a while ago, tho someone could use the old code if they really wanted.


lib and lib-es6 in gitignore

done

Not sure I understand... the react template already has it

not this gitignore, the template

ok


Automatic update problem

Hi,
First of all thanks a lot for your great work.
I would like to submit an issue with automatic upgrade. I use SyncTrayzor on my PC Windows 10 64 bits and syncthing on my VPS ubuntu server 14.04.
Please see details in screenshot enclosed.
Do you need more information?
Thanks

This is a Syncthing issue. Please open an issue [with the Syncthing repo](https://github.com/syncthing/syncthing/issues/new].
From a cursory look, it looks like your computer's having trouble talking to Amazon's servers, which GitHub uses to host release binaries. Make sure you don't have a firewall in the way.

Thanks a lot.
I just connected through a VPN and it updated.
Thomas
On 27 June 2016 at 17:32, Antony Male<EMAIL_ADDRESS>wrote:

This is a Syncthing issue. Please open an issue [with the Syncthing repo](
https://github.com/syncthing/syncthing/issues/new].
From a cursory look, it looks like your computer's having trouble talking
to Amazon's servers, which GitHub uses to host release binaries. Make sure
you don't have a firewall in the way.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/canton7/SyncTrayzor/issues/287#issuecomment-228697295,
or mute the thread
https://github.com/notifications/unsubscribe/ACJqSW3Hwsun6VSV3VExww20nB_q4THyks5qP5jFgaJpZM4I-6F-
.

--
Thomas Arbib
France : +33 (0)6 30 66 72 25
中国手机 : +86 150 6616 3405
Twitter : @tarbib http://twitter.com/tarbib
微信 (WeChat) ID : tarbib
http://about.me/thomasarbib

Glad to hear it!


added style template ref to ESF Styling sample

I have updated the naming and the templates are here: https://infragistics.visualstudio.com/DefaultCollection/NetAdvantage/_git/code-generation-library/pullrequest/9571


[Bug] Logs are only sent at the end and truncated when running TC-IDM-10.2 from the TH UI.

Describe the bug
The test run logs are not sent in real-time and are getting truncated.
Steps to reproduce the behavior
Just try to run TC-IDM-10.2 from TH.
Expected behavior
Logs dynamically generated as the test goes.
Log files
No response
PICS file
No response
Screenshots
No response
Environment
No response
Additional Information
No response

Added logs/details here:
Both in a complete fail case (due to saying "yes" to commission when not needed) and when saying "no" (but config error present), the logs were basically absent to know what was going on, in the UI.
There is >1 minute of logs completely missing:
PYTHON_TEST | 2024-08-12 23:38:39.380528 | [1723505919.201421][14:23] CHIP:DL: HandlePlatformSpecificBLEEvent 32786
PYTHON_TEST | 2024-08-12 23:38:39.504612 | Didn't find any CertificateAuthorities in storage -- creating a new CertificateAuthority + FabricAdmin...
PYTHON_TEST | 2024-08-12 23:38:39.881359 | Long dispatch time: 353 ms, for event type 2

............ nothing between these 2 lines, but lots would have happened...

PYTHON_TEST | 2024-08-12 23:39:14.018592 | Shutting down the stack...
PYTHON_TEST | 2024-08-12 23:39:14.036823 | ERROR:root:Final result: FAIL !
ERROR      | 2024-08-12 23:39:14.663678 | Test Case Error: Error occurred during setup of test case TC_IDM_10_2. Failed to commission DUT
INFO       | 2024-08-12 23:39:14.665845 | Test Cleanup
INFO       | 2024-08-12 23:39:14.667642 | Test Case Completed [ERROR]: TC-IDM-10.2

For the case where there was some running, it claimed that the test passed when it failed, and the logs from the Python test were completely unusable in the UI, but were OK in the file downloaded afterwards, although most of the chip stack logs are missing.

SAYS_PASS_BUT_LOGS_CORRUPT_UI_Test_Run_2024_08_12_23_40_30.log
FAIL_WITH_NO_LOGS___UI_Test_Run_2024_08_12_23_38_21.log

The following issues have been created for the listed problems:
[Bug] TH logs do not include line breaks: https://github.com/project-chip/certification-tool/issues/356
[Bug] TH does not commission the DUT when qr-code parameter is set: https://github.com/project-chip/certification-tool/issues/358
[Feature] Add real-time log streaming from test in TH: https://github.com/project-chip/certification-tool/issues/345

Hi @tcarmelveilleux I'm investigating the issue related to the logs lines that are missing.
I performed the test TC-IDM-10.2 both in TH and inside docker container
Looking at the logs I was not able to identify the missing logs line.
Am I missing something?
I'm using a new SDK SHA: c4a5a95348c66f60a5a33e6ff0389e949589aaa6
Execution with DUT Commissioned

TH (Should the DUT be commissioned to run this test case? YES ): TC-IDM-10.2  - TH Commissioned YES.log

Container:  Using this command in order to Commission the DUT: python3 /root/python_testing/scripts/sdk/test_harness_client.py commission --trace-to json:log --commissioning-method on-network --qr-code MT:-24J042C00KA0648G00 TC-IDM-10.2  - Container Commissioned YES.log

Execution DUT  NOT Commissioned:

TH (Should the DUT be commissioned to run this test case? NO): TC-IDM-10.2  - TH Commissioned NO.log

Container: TC-IDM-10.2  - Container Commissioned NO.log

Project Config: Project_Config.json

Sharing the TH execution logs using the TE2 SDK SHA: d0d91272068f267cf880f9d56787ca28da885673
TC-IDM-10.2  - TH Commissioned YES (2): TC-IDM-10.2  - TH Commissioned YES (2).log
TC-IDM-10.2  - TH Commissioned NO (2): TC-IDM-10.2  - TH Commissioned NO (2).log

Hi @tcarmelveilleux Gentle reminder, did you get the chance to analyze my 2 last comments above?

Closing due to inactivity. Please reopen if needed.


Delegated driver may or may not manage instances

Reworked the delegated to fully delegate to the developer.

Developer may delegate the instance create/destroy to the delegate
driver.
Developer may not use create/destroy and delegate to a manual
process.

Fixes: #936

👍 awesome!


JToken and related Json.NET types are not handled by the current object adapters

From @jbagga on Wednesday, October 25, 2017 4:15:57 PM
Our current adapters do not handle JToken (and related types like JArray and JObject) correctly. We can look into checking if the current contract is JsonLinqContract and write a new adapter or modify the dictionary or dynamic object adapters.
Copied from original issue: aspnet/JsonPatch#121

Closing as no customer expressed interest in this.


How to get lastest audithistory and export to CSV ?

I need export 1record with 1 lastest audit history and export to CSV

Thanks for your solution :-). I've just released a fix.
Aldo


Cache is not working: Failed to read cache clean timestamp

Hello,
I have a huge problem with ngx_pagespeed cache. I've been working on it for 7 weeks with no success.
I'm giving you some details but feel free to ask for more if needed.
99% of error.log are filled with :
2017/12/12 03:24:01 [warn] 2023#3040: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /dev/shm/ngx_cache/!clean!time!.  Doing an extra cache clean to be safe. 2017/12/12 03:24:01 [warn] 2023#3040: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /dev/shm/ngx_cache/!clean!time!.  Doing an extra cache clean to be safe. 2017/12/12 03:24:01 [warn] 2023#3040: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /dev/shm/ngx_cache/!clean!time!.  Doing an extra cache clean to be safe. 2017/12/12 03:24:01 [warn] 2023#3040: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /dev/shm/ngx_cache/!clean!time!.  Doing an extra cache clean to be safe. 2017/12/12 03:24:01 [warn] 2023#3040: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /dev/shm/ngx_cache/!clean!time!.  Doing an extra cache clean to be safe. 2017/12/12 03:24:01 [warn] 2023#3040: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /dev/shm/ngx_cache/!clean!time!.  Doing an extra cache clean to be safe. 2017/12/12 03:24:01 [warn] 2023#3040: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /dev/shm/ngx_cache/!clean!time!.  Doing an extra cache clean to be safe. 2017/12/12 03:24:02 [warn] 2022#3054: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /dev/shm/ngx_cache/!clean!time!.  Doing an extra cache clean to be safe. 2017/12/12 03:24:05 [warn] 2024#2233: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /dev/shm/ngx_cache/!clean!time!.  Doing an extra cache clean to be safe.
Error.log also contains (which is quite alarming) :
2017/12/11 05:56:01 [error] 2024#2231: [ngx_pagespeed <IP_ADDRESS>-0] PageSpeed Serf fetch failure rate extremely high; only 4 of 9 recent fetches fully successful; is fetching working?
Nginx is compiled with :
nginx -V nginx version: nginx/1.13.7 built by gcc 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.5) built with OpenSSL 1.0.2g  1 Mar 2016 TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_gzip_static_module --with-http_ssl_module --with-http_stub_status_module --with-http_v2_module --with-pcre-jit --add-module=/usr/local/src/nginx-ps/ps --add-module=/usr/local/src/nginx-ps/ngx_brotli --with-cc-opt='-g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie'
Extract from pagespeed config :
pagespeed FileCachePath                         "/dev/shm/ngx_cache"; pagespeed FileCacheSizeKb                       2304000; pagespeed FileCacheCleanIntervalMs              3600000; pagespeed FileCacheInodeLimit                   500000; pagespeed LRUCacheKbPerProcess                  16384; pagespeed LRUCacheByteLimit                     32768; pagespeed CreateSharedMemoryMetadataCache       "/dev/shm/ngx_cache" 1024000; pagespeed DefaultSharedMemoryCacheKB            1024000;
Extract from nginx config :
user                                    www-data;
/dev/shm/ permissions :
l /dev/shm/ total 0 drwxr-xr-x 3 www-data www-data 60 Dec  9 01:04 ngx_cache
/dev/shm/ngx_cache permissions (I tried 777 in order to check if the problem comes from here) :
l /dev/shm/ngx_cache total 0 drwxrwxrwx 3 www-data www-data 60 Dec  9 01:04 shm_metadata_cache
I use ngx_pagespeed on two sites (with same configurations) and both have the same problem.
I did NOT have the cache problem before septembre 2017 (not really sure but it might be the case).
Would you please help me to find a solution? I am so stuck.
Thanks a lot.

Hi @frontal1660
Have you tried touch /dev/shm/ngx_cache/cache.flush ?

Hi @Lofesa
Thank you for this quick answer.
cache.flush did NOT exist in the directory.
Now I have :
bash# ls -la /dev/shm/ngx_cache/
total 0
drwxrwxrwx 3 www-data www-data 60 Dec  9 01:04 shm_metadata_cache
-rwxrwxrwx 1 www-data www-data  0 Dec 12 14:13 cache.flush
I kept the 777 permissions for now.
I havejust  restarted Nginx, PHP and Memcached. Usually the errors come after few dozens of minutes. I'll check and come back to you.

Bad news. It did not solve the problem.
I still have many lines like that in error.log:
2017/12/12 16:37:15 [warn] 1868#2733: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /dev/shm/ngx_cache/!clean!time!.  Doing an extra cache clean to be safe.
2017/12/12 16:37:15 [warn] 1868#2733: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /dev/shm/ngx_cache/!clean!time!.  Doing an extra cache clean to be safe. 2017/12/12 16:37:15 [warn] 1868#2733: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /dev/shm/ngx_cache/!clean!time!.  Doing an extra cache clean to be safe.
2017/12/12 16:37:15 [warn] 1868#2733: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /dev/shm/ngx_cache/!clean!time!.  Doing an extra cache clean to be safe.`
On the other server I use the /var/cache/pagespeed directory but I have the same issue.
One more thing I didn't say: my two servers are dedicated OVH servers.

Hi @frontal1660
Try to change the file name to !clean!time!
In other hand sound strange for me that do not exist a v3 directory. Have you configued the file cache directives?

Hi @Lofesa
Thanks for your anwser.
The file cache directory, the following lines exist in my pagespeed config:
pagespeed FileCachePath                                        "/var/cache/pagespeed";
pagespeed FileCacheSizeKb                                     2304000;
pagespeed FileCacheCleanIntervalMs                      3600000;
pagespeed FileCacheInodeLimit                               500000;
pagespeed LRUCacheKbPerProcess                         16384;
pagespeed LRUCacheByteLimit                                32768;
pagespeed CreateSharedMemoryMetadataCache   "/dev/shm/ngx_cache" 102400;
pagespeed DefaultSharedMemoryCacheKB             102400;
Does it answer your questions about file cache directives ?
Actually I don't understand your comments. What file do you want me to change ? And what about the v3 directory ? Could you be more explicit ?
I've just checked the /dev/shm/ngx_cache/ directory and the file cache.flush you told me to create is not here anymore. It has been deleted but not by me (pagespeed ?).
Here is what I have in the directory:
bash# ls -la /dev/shm/ngx_cache/
total 0
drwxrwxrwx 3 www-data www-data 60 Dec 9 01:04 shm_metadata_cache
I want to say that I read every page of the official pagespeed documentation and I searched Internet for hours with no success.
I've been using ngx_pagespeed for a long time now and I'm thinking of disabling... what a shame isn't it ?

FileCachePath & CreateSharedMemoryMetadataCache should be set up with paths
that match [1].
This looks misconfigured, and possibly fixing that may eliminate the log
messages you observed.
The offending line probably is:
pagespeed CreateSharedMemoryMetadataCache "/dev/shm/ngx_cache" 102400;
You can remove it, or configure a corresponding FileCachePath.
[1] https://www.modpagespeed.com/doc/system#shm_cache.
Otto
On Wed, Dec 13, 2017 at 12:42 PM frontal<EMAIL_ADDRESS>wrote:

Hi @Lofesa https://github.com/lofesa
Thanks for your anwser.
The file cache directory, the following lines exist in my pagespeed
config:
pagespeed FileCachePath "/var/cache/pagespeed";
pagespeed FileCacheSizeKb 2304000;
pagespeed FileCacheCleanIntervalMs 3600000;
pagespeed FileCacheInodeLimit 500000;
pagespeed LRUCacheKbPerProcess 16384;
pagespeed LRUCacheByteLimit 32768;
pagespeed CreateSharedMemoryMetadataCache "/dev/shm/ngx_cache" 102400;
pagespeed DefaultSharedMemoryCacheKB 102400;
Does it answer your questions about file cache directives ?
Actually I don't understand your comments. What file do you want me to
change ? And what about the v3 directory ? Could you be more explicit ?
I've just checked the /dev/shm/ngx_cache/ directory and the file
cache.flush you told me to create is not here anymore. It has been deleted
but not by me (pagespeed ?).
Here is what I have in the directory:
bash# ls -la /dev/shm/ngx_cache/
total 0
drwxrwxrwx 3 www-data www-data 60 Dec 9 01:04 shm_metadata_cache
I want to say that I read every page of the official pagespeed
documentation and I searched Internet for hours with no success.
I've been using ngx_pagespeed for a long time now and I'm thinking of
disabling... what a shame isn't it ?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/pagespeed/ngx_pagespeed/issues/1516#issuecomment-351367453,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACIsRF7Ij_5RqXKsM5ZrJdBoV8V9Nr7xks5s_7gcgaJpZM4Q-V1J
.

Thanks a lot @oschaaf.
I feel very optimistic regarding this solution.
I will implement the corrections as soon as possible and come back to you.
Once again thanks for your reactivity

Once again bad news :(
@oschaaf : your trick did not work.
My second server is based on this pagespeed config (extract with more details) and also has the same problem in error.log:
pagespeed FetchHttps                            enable;
pagespeed RespectVary                           on;
pagespeed MaxCombinedCssBytes                   -1;
pagespeed Domain                                https://www.my-domain.com;
pagespeed FileCachePath                         /var/cache/pagespeed;
pagespeed FileCacheSizeKb                       2304000;
pagespeed FileCacheCleanIntervalMs              3600000;
pagespeed FileCacheInodeLimit                   500000;
pagespeed LRUCacheKbPerProcess                  16384;
pagespeed LRUCacheByteLimit                     32768;
pagespeed CreateSharedMemoryMetadataCache       "/var/cache/pagespeed/" 102400;
pagespeed DefaultSharedMemoryCacheKB            102400;
pagespeed MemcachedThreads                      1;
pagespeed MemcachedServers                      "localhost:11211";
As you can see the FileCachePath and CreateSharedMemoryMetadataCache directives DOES match.
If you want me to extract more config files, feel free to ask me.
Thanks again for your time.

Well, actually it's not an exact match, with regards to the trailing
slash..
On Wed, 13 Dec 2017 at 13:32, frontal<EMAIL_ADDRESS>wrote:

Once again bad news :(
@oschaaf https://github.com/oschaaf : your trick did not work.
My second server is based on this pagespeed config (extract with more
details) and also has the same problem in error.log:
pagespeed FetchHttps enable;
pagespeed RespectVary on;
pagespeed MaxCombinedCssBytes -1;
pagespeed Domain https://www.my-domain.com;
pagespeed FileCachePath /var/cache/pagespeed;
pagespeed FileCacheSizeKb 2304000;
pagespeed FileCacheCleanIntervalMs 3600000;
pagespeed FileCacheInodeLimit 500000;
pagespeed LRUCacheKbPerProcess 16384;
pagespeed LRUCacheByteLimit 32768;
pagespeed CreateSharedMemoryMetadataCache "/var/cache/pagespeed/" 102400;
pagespeed DefaultSharedMemoryCacheKB 102400;
pagespeed MemcachedThreads 1;
pagespeed MemcachedServers "localhost:11211";
As you can see the FileCachePath and CreateSharedMemoryMetadataCache
directives DOES match.
If you want me to extract more config files, feel free to ask me.
Thanks again for your time.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pagespeed/ngx_pagespeed/issues/1516#issuecomment-351378019,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACIsRLJh1pEkXeqtQKjPLlP2cD8UF0s1ks5s_8PvgaJpZM4Q-V1J
.

And also there is not quote. I'm adding them right now.
Here come my updates:
pagespeed FileCachePath                         "/var/cache/pagespeed/";
pagespeed CreateSharedMemoryMetadataCache       "/var/cache/pagespeed/" 102400;
I'll keep you informed.

I am not glad to say that the problem is still here.
About 20 minutes after my update the lines in error.log reappeared.
I am desperate.

Are all the directory permissions set up so your nginx child processes can read/write your file-cache?  E.g. is the file-cache directory owned by the same username that is running the nginx processes?

Hi @jmarantz
Thank you for your anwser. Directories permissions issue was also my first guess.
Here comes some information. You may be able to find something wrong. I could NOT.
/etc/nginx/nginx.conf
 user                                    www-data;
/etc/php/7.0/fpm/pool.d
user = www-data
group = www-data
listen.owner = www-data
listen.group = www-data
listen.mode = 0660
bash# ps aux | grep nginx
root    10792  0.0  0.0 181724  4268 ?        Ss   13:56   0:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
www-data 10793  0.1  0.2<PHONE_NUMBER>4 ?        Sl   13:56   0:09 nginx: worker process
www-data 10794  0.1  0.2<PHONE_NUMBER>6 ?        Sl   13:56   0:10 nginx: worker process
www-data 10795  0.1  0.2<PHONE_NUMBER>8 ?        Sl   13:56   0:15 nginx: worker process
www-data 10796  0.1  0.1<PHONE_NUMBER>0 ?        Sl   13:56   0:14 nginx: worker process
www-data 10797  0.1  0.1<PHONE_NUMBER>8 ?        Sl   13:56   0:09 nginx: worker process
www-data 10798  0.1  0.2<PHONE_NUMBER>6 ?        Sl   13:56   0:10 nginx: worker process
www-data 10799  0.1  0.1<PHONE_NUMBER>2 ?        Sl   13:56   0:12 nginx: worker process
www-data 10800  0.1  0.1<PHONE_NUMBER>8 ?        Sl   13:56   0:09 nginx: worker process
bash# ls -la /var/cache/pagespeed/
total 12K
drwxr-xr-x  3 www-data www-data 4.0K Dec 13 13:56 .
drwxr-xr-x 11 root     root     4.0K Nov 11 01:42 ..
drwxrwxrwx  3 www-data www-data 4.0K Dec 13 13:56 shm_metadata_cache
bash# ls -la /var/cache/pagespeed/shm_metadata_cache
total 12K
drwxrwxrwx 3 www-data www-data 4.0K Dec 13 13:56 .
drwxr-xr-x 3 www-data www-data 4.0K Dec 13 13:56 ..
drwxrwxrwx 3 www-data www-data 4.0K Dec 13 13:56 snapshot
I'm stuck.

I'm not sure what is up but https://groups.google.com/forum/m/#!topic/ngx-pagespeed-discuss/JBvQU0unpag looks very similar

Hi @frontal1660
I will say chage the file name cache.flush to !clean!time!.
When File cache is working a directory v3 exist in the /var/cache/pagespeed/ path, under that v3 directory sub-directories with name of the sites, path and files exists..... these files are served by pagespeed.
Seems that the user that run nginx cannot write to these directory

@oschaaf :

I have already read this thread many times but it does not apply. There is no suitable answer. I found other links but none of them is interesting :(

@Lofesa :

I do not have this directory (v3) but do I really have to create it ? I will do it but I do not think that it's "my job"... if this directory is usefull for pagespeed then it should create it :)
I will create !clean!time! file (create because cache.flush has been auto deleted) but with no conviction. I would like to know to source of the problem.

Yes, v3 directory sould be created by pagespeed, because it is not created make me thinking in permissions.
In the last post, have you ceated the v3 directory or is pagespeed ?

@Lofesa I have some news and I still have a problem.
Your questions and comments:

I created the "v3" directory and the "!clean!time!"... not pagespeed... following your advice.
I also think that it's a permission problem but where is the issue ? I don't know. See my config files if needed
SELinux is NOT enabled and not used (Ubuntu 16.04.05 LTS)

Creating your file (!clean!time!) and/or directory (v3) had an impact but did not solve the problem:

I haven't any more message in error.log BUT my cache miss graph on pagespeed_admin Console page shows me a 43.91% rate! I have the same rate since at least beginning of october 2017. Sometimes it goes over 56% !

If anyone has an idea... :)

So now it looks like the permission problem is gone, but you see a different problem: the cache hit ratio seems lower then you would expect. Correct?
Perhaps points 1 & 2 from the following faq entry are worth checking out:
https://www.modpagespeed.com/doc/faq#varying-results

@oschaaf :
Thank you for your research.
Indeed it seems that permission problem is gone but how? I had to create

a v3 directory which stays empty
a !clean!time! file which has a very "strange" name don't you think?

The fact that I had to tune this way my pagespeed installation does not confort me. I think that this tuning is too specific and my first question would be: has anyone ever done something like that (directory and file creations)?
Actually the cache misses problem (Console graph problem) is not really new. I have this problem since at least the beginning of october.
Regarding to your link I have once again already read this page :(
It is a little bit complicated and:

I am not sure to be in one of the discussed cases
I am not sure to fully understand what is says

I will try to apply some of the advices given in the page.
My two sites are both served in HTTPS so it may be usefull to specify:
pagespeed MapOriginDomain http://localhost https://www.my-domaine.com;
I can add few things:

I don't use any CDN
I use Brotli module

What do you think?

@oschaaf
Bad news... still
I had to restart my server for maintenance. After reboot I checked two things:

if the V3 directory and the !clean!time! file were still in the /var/cache/pagespeed directory and they do NOT exist anymore... so it means that I would have to recreate these file/directory at every reboot... not a good news and not a normal behaviour I guess
if the error.log file was "clean" and it is NOT the case... pagespeed error came back!!! Maybe because of the lack of the file/directory.

I decided NOT to recreate the file/directory but to implement the MapOriginDomain directive instead to see what's next.
Once again I say the same thing: I've using pagespeed for something like 10 servers for almost 2 years with no problem and I have this big problem only on my two last servers with last Nginx and last Pagespeed versions.
@oschaaf, @Lofesa, @jmarantz: I really thank you for help but I don't what to add. I feel desesperate... again.

I think manually messing with !clean!time! is really trying to solve a symptom of something else.  I have another wild guess though: are you out of space in your /dev/shm partition?
Also note -- I don't really think the low apparent cache hit rate is necessarily a problem.  PageSpeed may -- based on some combination of enabled filters and site URL patterns -- attempt to do a lot of lookups for properties of an HTML page that rarely get hits.  E.g. if your JavaScript inserted query-params with high entropy.
But the !clean!time! messages are annoying and it's probably worth fixing.

@jmarantz
1 - I totally agree with you when you speak about symptoms. It is not the root cause.
2 - Actually (thanks to @oschaaf) I only use /var/cache/pagespeed for both pagespeed directives:
pagespeed FileCachePath "/var/cache/pagespeed/";
pagespeed CreateSharedMemoryMetadataCache "/var/cache/pagespeed/" 102400;
3 - Regarding to the size/space of partitions:

/var/cache/pagespeed/ belongs to /var which is a 16Go partition with 15Go of available space
/dev/shm is a 16Go partition with 15.5Go of available space

4 - By the way, even after a certain period of time, I have files in my /var/cache/pagespeed directory but a "du -m /var/cache/pagespeed" tells me that these files take less than 1Mo... which might be strange...
bash@ du -m /var/cache/pagespeed/
1       /var/cache/pagespeed/shm_metadata_cache/snapshot/,2Fvar/cache/pagespeed/,2Fmetadata_cache/1/64/8904/128
1       /var/cache/pagespeed/shm_metadata_cache/snapshot/,2Fvar/cache/pagespeed/,2Fmetadata_cache/1/64/8904
1       /var/cache/pagespeed/shm_metadata_cache/snapshot/,2Fvar/cache/pagespeed/,2Fmetadata_cache/1/64
1       /var/cache/pagespeed/shm_metadata_cache/snapshot/,2Fvar/cache/pagespeed/,2Fmetadata_cache/1
1       /var/cache/pagespeed/shm_metadata_cache/snapshot/,2Fvar/cache/pagespeed/,2Fmetadata_cache
1       /var/cache/pagespeed/shm_metadata_cache/snapshot/,2Fvar/cache/pagespeed
1       /var/cache/pagespeed/shm_metadata_cache/snapshot/,2Fvar/cache
1       /var/cache/pagespeed/shm_metadata_cache/snapshot/,2Fvar
1       /var/cache/pagespeed/shm_metadata_cache/snapshot
1       /var/cache/pagespeed/shm_metadata_cache
1       /var/cache/pagespeed/
5 - Regarding the cache hit rate you might be right but a rate value between 50 and 56% does not seem to be very good. I've just checked on an older server (6 months older than the two servers which experience issues) and my "cache missed graph" indicates a value lower than 10%...
6 - Again you're right... these messages are really annoying and I'd really like to find what the REAL problem is.
7 - Important: as I said before I implemented @oschaaf advice
pagespeed MapOriginDomain http://localhost https://www.my-domain.com;
and.... !clean!time! messages in error.log are definitly back...
8 - I do not consider myself as a newbie but I have to admit that I feel useless in this case.

The message is triggered here:
https://github.com/pagespeed/mod_pagespeed/blob/338c37915355c0a4dd8726643cd90718bc394d6f/pagespeed/kernel/cache/file_cache.cc#L403
However, the module tries to create !clean!time here: https://github.com/pagespeed/mod_pagespeed/blob/338c37915355c0a4dd8726643cd90718bc394d6f/pagespeed/kernel/cache/file_cache.cc#L360
Looking at the code, I think the warning actually will always be emitted at least once, because FileCache::ShouldClean runs before FileCache::CleanWithLocking which is responsible for creating the file. But maybe I'm missing something...

Just stumbled on this too. Everything is working but the error:

Failed to read cache clean timestamp
/var/cache/pagespeed/!clean!time!. Doing an extra cache clean to be safe.

... is being triggered.
Indeed, looks like the code referenced by @oschaaf is responsible, in a nutshell it expects the file to be there.
I think an easy solution would be having PageSpeed touching the !clean!time! file on startup, to make sure it's there from the get-go.

I am having the same problem.
I identified the !clean!time! file is not getting written at all when using the pagespeed MemcachedServers directive. So it looks like the cache manager is looking for a file which got never created when memcache is configured for caching.

@dvershinin and @Project0 : are you getting the message repeatedly? Or is it just coming out once? If it's coming out just once I think that's not a problem, and you should just ignore it. If it's repeatedly coming out then I could see it becoming an issue for log-file growth.

If it's repeatedly coming out then I could see it becoming an issue for log-file growth.
Indeed, that what is happening. Its usually thrown several times in a row (i guess per worker), I am now creating this file manually at startup (docker entrypoint) to remove the noise from our logs. That drives my crazy since years :D

@jmarantz It is coming out once, but as per @Project0 I'm not surprised it may be also multiplied by the number of workers.
While it won't pollute the logs too much, I think it is still a bug, because the message is misleading.

Either the file should be pre-created upon NGINX startup, or:
Keep some in-memory record that the cleanup was never run after startup, then skip the file check

I don't think it will be multiplied by the number of workers because the first one that runs will write that file.
It also should not happen once for each time nginx is restarted; the file will be there after the restart, as will the contents of the cache, unless it's on tmpfs and the system is restarted.
You are right that it would be nicer not to print that spurious message, so it's perfectly fine to have the startup script touch that file if it's not present. However that's not something PageSpeed itself does.

You are right that it would be nicer not to print that spurious message, so it's perfectly fine to have the startup script touch that file if it's not present. However that's not something PageSpeed itself does.

My problem is really that file will never be created by nginx/pagespeed itself and throws continuously errors. The error starts usually after 1h and never stops. As i told already, this happens only when memcache is enabled:
2020/10/28 09:01:35 [warn] 428#434: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /var/cache/nginx/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
2020/10/28 09:01:35 [warn] 428#434: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /var/cache/nginx/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
2020/10/28 09:01:35 [warn] 421#427: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /var/cache/nginx/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
2020/10/28 09:01:35 [warn] 435#441: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /var/cache/nginx/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
2020/10/28 09:01:36 [warn] 428#434: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /var/cache/nginx/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
2020/10/28 09:01:59 [warn] 392#399: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /var/cache/nginx/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
2020/10/28 09:02:03 [warn] 392#399: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /var/cache/nginx/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
2020/10/28 09:02:10 [warn] 428#434: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /var/cache/nginx/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
2020/10/28 09:02:10 [warn] 428#434: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /var/cache/nginx/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
2020/10/28 09:02:15 [warn] 428#434: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /var/cache/nginx/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
2020/10/28 09:02:18 [warn] 428#434: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /var/cache/nginx/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
2020/10/28 09:02:22 [warn] 392#399: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /var/cache/nginx/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
2020/10/28 09:02:30 [warn] 428#434: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /var/cache/nginx/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
2020/10/28 09:02:35 [warn] 392#399: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /var/cache/nginx/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
2020/10/28 09:02:36 [warn] 392#399: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /var/cache/nginx/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
2020/10/28 09:02:46 [warn] 442#448: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /var/cache/nginx/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.
2020/10/28 09:02:49 [warn] 392#399: [ngx_pagespeed <IP_ADDRESS>-0] Failed to read cache clean timestamp /var/cache/nginx/pagespeed/!clean!time!.  Doing an extra cache clean to be safe.

OK, teasing this apart:
@dvershinin finds that message comes out exactly once, which is expected, at least with a fresh cache directory. I think that behavior is working-as-intended although maybe it looks slightly unclean. The reason that behavior is working-as-intended is that you don't want to be blind to a real problem...as indicated below.
@Project0 is finding it comes out a lot. That's a very different issue, and is the reason this message exists at all. It means something is misconfigured. My guess is that the process running PageSpeed does not have permissions to write the cache directory, but maybe there's another reason. And I'm a little suspicious of my answer as I'd expect write-error messages in the log as well.

Maybe is a tmpfs? For example in Centos-RedHat distros /var is a tmpfs and most people tend to put the cache directory here.
At the start, the directory is owned by root, so pagesped can´t read/write.
This can be solved creating a conf file in /etc/tmpfiles.d (call it pagespeed-cache.conf) with some lines like:
D /var/path-to-cache-directory/ 0755 pagespeed-user:pagespeed-group -
f /var/path-to-cache-directory/!clean!time! 0755 pagespeed-user:pagespeed-group -

These 2 lines creates the directory (D) and the file (f) in these paths, permisions and owner at system start, before any process acces it.
Note the last -, it is needed.
More on this here: https://man7.org/linux/man-pages/man5/tmpfiles.d.5.html
I don´t know if Ubuntu or other distros has this feature.


Add PR Title Checker

Description
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context.

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#5
#4
#3  👈
#2
#1
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @JackPlowman and the rest of your teammates on  Graphite

Merge activity

Sep 25, 1:49 PM EDT: @JackPlowman merged this pull request with Graphite.


CI: Use Ruby 2.6.3

This PR only updates the CI matrix to the latest generally available Ruby versions.
Source for version numbers: https://github.com/rvm/rvm/blob/master/config/known

@justincampbell OK, now we're using two-part version numbers. Is this ready?


Could not get the attribute of string

When I use jmxtrans get weblogic Mbean attributes,  only get number-type attribute, could not get the attribute of string。

"obj" : "com.bea:Name=myserver,Type=ServerRuntime",
"resultAlias": "",
"attr" : [ "Name","State","ListenPort","ActivationTime" ]
When I run jmxtrans, I can get to the value of ListenPort and ActivationTime, but could not get the value of  Name and State

What OutputWriter are you using? Could you post the full configuration?

{
"servers" : [ {
"port" : "20001",
"host" : "<IP_ADDRESS>",
"queries" : [ {
"outputWriters" : [ {
"@class" : "com.googlecode.jmxtrans.model.output.StackdriverWriter",
"settings" : {
"url":"http://<IP_ADDRESS>:5000/receive?type=WebLogicState&localhost=<IP_ADDRESS>:70001&param={"dataType":"serverInfo","server":"myserver","ListenPort":7001}",
"token":"123456"
}
} ],
"obj" : "com.bea:Name=myserver,Type=ServerRuntime",
"resultAlias": "",
"attr" : [ "Name","SSLListenPort","AdminServer","State","ListenAddress","ListenPort","ActivationTime" ]
},
{
"outputWriters" : [ {
"@class" : "com.googlecode.jmxtrans.model.output.StackdriverWriter",
"settings" : {
"url":"http://<IP_ADDRESS>:5000/receive?type=WebLogicState&localhost=<IP_ADDRESS>:70001&param={"dataType":"serverMemory","server":"myserver"}",
"token":"123456"
}
} ],
"obj" : "com.bea:ServerRuntime=myserver,Name=myserver,Type=JVMRuntime",
"attr" : [ "HeapFreeCurrent", "HeapSizeCurrent","HeapSizeMax","HeapFreePercent"],
"resultAlias": ""
},
{
"outputWriters" : [ {
"@class" : "com.googlecode.jmxtrans.model.output.StackdriverWriter",
"settings" : {
"url":"http://<IP_ADDRESS>:5000/receive?type=WebLogicState&localhost=<IP_ADDRESS>:70001&param={"dataType":"serverThread","server":"myserver"}",
"token":"123456"
}
} ],
"obj" : "com.bea:ServerRuntime=myserver,Name=ThreadPoolRuntime,Type=ThreadPoolRuntime",
"attr" : [ "ExecuteThreadIdleCount", "ExecuteThreadTotalCount","CompletedRequestCount","PendingUserRequestCount","QueueLength","Suspended"],
"resultAlias": ""
},
{
"outputWriters" : [ {
"@class" : "com.googlecode.jmxtrans.model.output.StackdriverWriter",
"settings" : {
"url":"http://<IP_ADDRESS>:5000/receive?type=WebLogicState&localhost=<IP_ADDRESS>:70001&param={"dataType":"serverConnectionPool","server":"myserver","dataSource":"mysql-dataSource"}",
"token":"123456"
}
} ],
"obj" : "com.bea:ServerRuntime=myserver,Name=mysql-dataSource,Type=JDBCConnectionPoolRuntime",
"attr" : [ "WaitingForConnectionHighCount", "JDBCConnectionPoolRuntimeMBean","WaitingForConnectionCurrentCount","MaxCapacity","WaitSecondsHighCount","ActiveConnectionsCurrentCount","ActiveConnectionsHighCount","LeakedConnectionCount"],
"resultAlias": ""
},
{
"outputWriters" : [ {
"@class" : "com.googlecode.jmxtrans.model.output.StackdriverWriter",
"settings" : {
"url":"http://<IP_ADDRESS>:5000/receive?type=WebLogicState&localhost=<IP_ADDRESS>:70001&param={"dataType":"serverConnectionPool","server":"myserver","dataSource":"testSource"}",
"token":"123456"
}
} ],
"obj" : "com.bea:ServerRuntime=myserver,Name=testSource,Type=JDBCConnectionPoolRuntime",
"attr" : [ "WaitingForConnectionHighCount", "JDBCConnectionPoolRuntimeMBean","WaitingForConnectionCurrentCount","MaxCapacity","WaitSecondsHighCount","ActiveConnectionsCurrentCount","ActiveConnectionsHighCount","LeakedConnectionCount"],
"resultAlias": ""
},
{
"outputWriters" : [ {
"@class" : "com.googlecode.jmxtrans.model.output.StackdriverWriter",
"settings" : {
"url":"http://<IP_ADDRESS>:5000/receive?type=WebLogicState&localhost=<IP_ADDRESS>:70001&param={"dataType":"webapp","ContextRoot":"emc","server":"myserver"}",
"token":"123456"
}
} ],
"obj" : "com.bea:ServerRuntime=myserver,Name=myserver_/emc,ApplicationRuntime=WebRoot,Type=WebAppComponentRuntime",
"attr" : [ "ContextRoot", "ApplicationIdentifier","Name","Status","OpenSessionsCurrentCount","OpenSessionsHighCount","SessionsOpenedTotalCount"],
  "resultAlias": ""
}

],
"numQueryThreads" :10

} ]
}
  } ],

when I use "com.googlecode.jmxtrans.model.output.KeyOutWriter", I  still cant not get the value of Name and State 。Here are the configuration：
{
"servers" : [ {
"port" : "20001",
"host" : "<IP_ADDRESS>",
"queries" : [ {
"outputWriters" : [ {
"@class" : "com.googlecode.jmxtrans.model.output.KeyOutWriter",
"outputFile" : "out2.txt",
"maxLogFileSize" : "10MB",
"maxLogBackupFiles" : 200,
"delimiter" : "\t",
"debug" : true,
"typeNames" : ["name"]
} ],
 "obj" : "com.bea:Name=myserver,Type=ServerRuntime",
  "resultAlias": "",
  "attr" : [ "Name","SSLListenPort","AdminServer","State","ListenAddress","ListenPort","ActivationTime" ]
}

],
"numQueryThreads" :2

} ]
}
Here are the results：
     localhost:jmxtrans tangsz$ cat out2.txt
               127_0_0_1_20001..SSLListenPort	7002<PHONE_NUMBER>693
               127_0_0_1_20001..ListenPort	7001<PHONE_NUMBER>693
               127_0_0_1_20001..ActivationTime<PHONE_NUMBER>437<PHONE_NUMBER>693


fix: change github links and contact

🚧 Fixed Issue #1037
closes #
Fixed the bug of removing the contact us link in the footer component👷🏻 <!--- A clear and concise (minimum 2 line) description of what you have done to successfully close your assigned issue.--->
Screenshots

 📸
Note to Reviewers <!--- Any new files? or anything you feel to let us know! --->

Hi @motuncoded , please resolve the conflicts. You previously made the Pull request to the main branch but please do not do that, Always make PR to the beta branch as said in the description.
Also, the issue was to Remove all the Contact Us and modify the github links of this repository. However, the later is not done yet. Please make the necessary changes. Thanks.


fix: typos

fix: typos

/test-deploy 181897a4da68d7d6d3aa8bb89c10e8a8e9d78b12

Sorry for the delay @omahs! Busy days this week. Thanks for the fixes, appreciate it.


Not generating text classes

First off, thanks for this plugin. I was actually surprised that despite such a robust community and plugin architecture, there are really only two fluid type plugins for tailwind: tailwind-utopia, and another that's not maintained anymore. I'm taking utopia for a spin, but it's hit a really early speed bump that I was hoping you'd have some insight into (particularly since some of the documentation is a bit inconsistent since this is all still fresh out of your oven).
Essentially, the issue is that none of the text classes get generated. My tailwind.config.js looks as follows and I'm using the several of the styles so purge should be leaving them alone:
// tailwind.config.js
const utopia = require('tailwind-utopia');

module.exports = {
  mode: 'jit',
  purge: ["./src/**/*.css", "./**/*.html",],
  darkMode: false, // or 'media' or 'class'
  theme: {
    extend: {
      fontFamily: {
        'lora': ['Lora']
      },
    },
  },
  variants: {
    extend: {},
  },
  modules: {
    textSizes: false
  },
  extend: {
    utopia: {
      minScreen: '320px',
      minSize: 16,
      minScale: 1.2,
      maxScreen: '1920px',
      maxSize: 60,
      maxScale: 1.5,
      textSizes: [
        'xs',
        'sm',
        'base',
        'lg',
        'xl',
        '2xl',
        '3xl',
        '4xl',
        '5xl',
        '6xl',
        '7xl',
        '8xl',
        '9xl'
      ]
    },
  },
  plugins: [
    utopia({
      useClamp: false,
      prefix: '',
      baseTextSize: 'step-0'
    }),
  ],
  corePlugins: {
    fontSize: false
  }
}

Thoughts?

🙌 awesome! Glad you got it working!
Those generated styles look good to me.
Yep, you got it - it generates a fixed size as a fallback for browsers that don't support car (👀 IE11)
N.B if you do use the spacing, then I haven't yet got a fallback in for those yet but it's on my todo list.