pretrain error

Hi, I encountered the same error in finetuning, it seems it runs out of RAM.

I have similiar issues.
I can run it with 4 GPUs, but cannot with 8GPUs (A100 80G memeory each).
My command it
LOGLEVEL=INFO TORCHELASTIC_ENABLE_FILE_TIMER=1 torchrun --nnodes=1 --nproc_per_node=4 --master_port=25001 \
    llava/train/train_mem.py \
    --model_name_or_path /mnt/bd/data-tns-algo-masp-llm/weights/llama-dl-main/vicuna_13B \
    --data_path /mnt/bd/data-tns-algo-masp-llm/experiment/LLaVA/LLaVA-Instruct-150K/chat.json \
    --image_folder /mnt/bd/data-tns-algo-masp-llm/experiment/LLaVA/LLaVA-Instruct-150K/images \
    --vision_tower openai/clip-vit-large-patch14 \
    --tune_mm_mlp_adapter True \
    --mm_vision_select_layer -2 \
    --mm_use_im_start_end \
    --bf16 True \
    --output_dir ./checkpoints/llava-13b-pretrain \
    --num_train_epochs 1 \
    --per_device_train_batch_size 16 \
    --per_device_eval_batch_size 4 \
    --gradient_accumulation_steps 2 \
    --evaluation_strategy "no" \
    --save_strategy "steps" \
    --save_steps 2400 \
    --save_total_limit 1 \
    --learning_rate 2e-3 \
    --weight_decay 0. \
    --warmup_ratio 0.03 \
    --lr_scheduler_type "cosine" \
    --logging_steps 1 \
    --tf32 True \
    --model_max_length 2048 \
    --gradient_checkpointing True \
    --lazy_preprocess True \
    --debug underflow_overflow \
    --report_to none

Logs:
LOGLEVEL=INFO TORCHELASTIC_ENABLE_FILE_TIMER=1 torchrun --nnodes=1 --nproc_per_node=8 --master_port=25001 \
>     llava/train/train_mem.py \
>     --model_name_or_path /mnt/bd/data-tns-algo-masp-llm/weights/llama-dl-main/vicuna_13B \
>     --data_path /mnt/bd/data-tns-algo-masp-llm/experiment/LLaVA/LLaVA-Instruct-150K/chat.json \
>     --image_folder /mnt/bd/data-tns-algo-masp-llm/experiment/LLaVA/LLaVA-Instruct-150K/images \
>     --vision_tower openai/clip-vit-large-patch14 \
>     --tune_mm_mlp_adapter True \
>     --mm_vision_select_layer -2 \
>     --mm_use_im_start_end \
>     --bf16 True \
>     --output_dir ./checkpoints/llava-13b-pretrain \
>     --num_train_epochs 1 \
>     --per_device_train_batch_size 16 \
>     --per_device_eval_batch_size 4 \
>     --gradient_accumulation_steps 2 \
>     --evaluation_strategy "no" \
>     --save_strategy "steps" \
>     --save_steps 2400 \
>     --save_total_limit 1 \
>     --learning_rate 2e-3 \
>     --weight_decay 0. \
>     --warmup_ratio 0.03 \
>     --lr_scheduler_type "cosine" \
>     --logging_steps 1 \
>     --tf32 True \
>     --model_max_length 2048 \
>     --gradient_checkpointing True \
>     --lazy_preprocess True \
>     --debug underflow_overflow \
>     --report_to none

INFO:torch.distributed.launcher.api:Starting elastic_operator with launch configs:
  entrypoint       : llava/train/train_mem.py
  min_nodes        : 1
  max_nodes        : 1
  nproc_per_node   : 8
  run_id           : none
  rdzv_backend     : static
  rdzv_endpoint    : <IP_ADDRESS>:25001
  rdzv_configs     : {'rank': 0, 'timeout': 900}
  max_restarts     : 0
  monitor_interval : 5
  log_dir          : None
  metrics_cfg      : {}

INFO:torch.distributed.elastic.agent.server.local_elastic_agent:log directory set to: /tmp/torchelastic__5qd21x9/none_xfx__n23
INFO:torch.distributed.elastic.agent.server.api:[default] starting workers for entrypoint: python
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous'ing worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous complete for workers. Result:
  restart_count=0
  master_addr=<IP_ADDRESS>
  master_port=25001
  group_rank=0
  group_world_size=1
  local_ranks=[0, 1, 2, 3, 4, 5, 6, 7]
  role_ranks=[0, 1, 2, 3, 4, 5, 6, 7]
  global_ranks=[0, 1, 2, 3, 4, 5, 6, 7]
  role_world_sizes=[8, 8, 8, 8, 8, 8, 8, 8]
  global_world_sizes=[8, 8, 8, 8, 8, 8, 8, 8]

INFO:torch.distributed.elastic.agent.server.api:[default] Starting worker group
INFO:torch.distributed.elastic.agent.server.local_elastic_agent:Starting a FileTimerServer with /tmp/watchdog_timer_01ab64a1-b00c-4ad2-ae61-4607b03008e7 ...
INFO:torch.distributed.elastic.agent.server.local_elastic_agent:FileTimerServer started
INFO:torch.distributed.elastic.multiprocessing:Setting worker0 reply file to: /tmp/torchelastic__5qd21x9/none_xfx__n23/attempt_0/0/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker1 reply file to: /tmp/torchelastic__5qd21x9/none_xfx__n23/attempt_0/1/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker2 reply file to: /tmp/torchelastic__5qd21x9/none_xfx__n23/attempt_0/2/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker3 reply file to: /tmp/torchelastic__5qd21x9/none_xfx__n23/attempt_0/3/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker4 reply file to: /tmp/torchelastic__5qd21x9/none_xfx__n23/attempt_0/4/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker5 reply file to: /tmp/torchelastic__5qd21x9/none_xfx__n23/attempt_0/5/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker6 reply file to: /tmp/torchelastic__5qd21x9/none_xfx__n23/attempt_0/6/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker7 reply file to: /tmp/torchelastic__5qd21x9/none_xfx__n23/attempt_0/7/error.json
You are using a model of type llama to instantiate a model of type llava. This is not supported for all configurations of models and can yield errors.
You are using a model of type llama to instantiate a model of type llava. This is not supported for all configurations of models and can yield errors.
You are using a model of type llama to instantiate a model of type llava. This is not supported for all configurations of models and can yield errors.
You are using a model of type llama to instantiate a model of type llava. This is not supported for all configurations of models and can yield errors.
You are using a model of type llama to instantiate a model of type llava. This is not supported for all configurations of models and can yield errors.
You are using a model of type llama to instantiate a model of type llava. This is not supported for all configurations of models and can yield errors.
You are using a model of type llama to instantiate a model of type llava. This is not supported for all configurations of models and can yield errors.
You are using a model of type llama to instantiate a model of type llava. This is not supported for all configurations of models and can yield errors.
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 39409 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 39410 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 39411 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 39412 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 39414 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 39415 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 39416 closing signal SIGTERM
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: -9) local_rank: 4 (pid: 39413) of binary: /mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/bin/python
INFO:torch.distributed.elastic.agent.server.api:Local worker group finished (FAILED). Waiting 300 seconds for other agents to finish
INFO:torch.distributed.elastic.agent.server.api:Done waiting for other agents. Elapsed: 0.00801992416381836 seconds
INFO:torch.distributed.elastic.multiprocessing.errors:local_rank 4 FAILED with no error file. Decorate your entrypoint fn with @record for traceback info. See: https://pytorch.org/docs/stable/elastic/errors.html
Traceback (most recent call last):
  File "/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/bin/torchrun", line 8, in <module>
    sys.exit(main())
  File "/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 346, in wrapper
    return f(*args, **kwargs)
  File "/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/lib/python3.10/site-packages/torch/distributed/run.py", line 794, in main
    run(args)
  File "/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/lib/python3.10/site-packages/torch/distributed/run.py", line 785, in run
    elastic_launch(
  File "/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 134, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 250, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
======================================================
llava/train/train_mem.py FAILED
------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2023-05-01_11:41:43
  host      : n193-016-074.byted.org
  rank      : 4 (local_rank: 4)
  exitcode  : -9 (pid: 39413)
  error_file: <N/A>
  traceback : Signal 9 (SIGKILL) received by PID 39413
======================================================

@tensorboy, thank you for your interest in our work.
I noticed that you are using the instruction tuning dataset in the pretraining stage.  This shall be the reason of OOM, as the instruction tuning dataset have longer responses, comparing to the short captions in CC3M.
Is there any specific reason you want to use instruction tuning with pretraining stage?  If so, you may try FSDP. Note that this is experimental as PyTorch has not officially supported PEFT for FSDP yet.  I have locally verified with very few iterations (400 iters x 8GPUs x 16 per_gpu_batch_size) that the model behavior is reasonable, and similar to w/o FSDP.  Make sure to use the latest code base, as there is an issue fixed this afternoon.
Thanks.

@yix-chen Sorry just saw this comment.  Are you still facing this issue, and if so, can you share your hardware setting (GPU type and count), and the command, as well as the error logs?  Thanks.

thanks for the quick reply, I think I've made mistakes for the dataset, but where I can download CC3M dataset..

@tensorboy Please checkout the download instructions here, thanks.

@tensorboy Please checkout the download instructions here, thanks.

Thanks for the quick feedback.   You may also download and see if len(json.load(...)) is roughly 595K.

Hi @tensorboy, chat.json in this repo is the correct one to go with. You may also download and see if len(json.load(...)) is roughly 595K.

it's 595375

Yep, that's correct :)

Thank you, what is your suggestions now?

I am confused.  Do you mean that although you were naming the folder as LLaVA-Instruct-150K/chat.json, but it actually comes from this CC3M instead?

I am confused. Do you mean that although you were naming the folder as LLaVA-Instruct-150K/chat.json, but it actually comes from this CC3M instead?

yes, I think I've put all the data in that directory..

This is strange.  Can you try seeing if finetuning works?  You can download the ./checkpoints/mm_projector/llava-13b-pretrain.bin here.    Your CPU RAM/GPU RAM should be more than sufficient, but we can see what is happening.  The peak memory usage may be ~500GB in your case.

Also, can you monitor both the GPU RAM, and CPU RAM usage when you are running the code, are they changing before throwing out the error? Your CPU RAM/GPU RAM should be more than sufficient, but we can see what is happening. The peak CPU memory usage may be ~500GB in your case.

CPU is around 400GB at peak, but GPU memory is 3MB / 81251 MB until it crashes.

It seems that it does not even start loading the checkpoints.  And it crashes when just initializing the model.  Honestly, I am not sure what the cause is, as I have not met such error before.
INFO:torch.distributed.elastic.multiprocessing.errors:local_rank 4 FAILED with no error file. Decorate your entrypoint fn with @record for traceback info. See: https://pytorch.org/docs/stable/elastic/errors.html

Do you think this @record is useful for this case?  It seems a similar issue to https://github.com/lm-sys/FastChat/issues/627?  Unfortunately there is no solution provided there either.

lm-sys/FastChat#627

It's exactly same issues with that fastchat. I'm not sure what that @record is and how to use that in your code either..

here

same errors. log:
torchrun --nnodes=1 --nproc_per_node=8 --master_port=25001 \
> llava/train/train_mem.py \
> --model_name_or_path /mnt/bd/data-tns-algo-masp-llm/weights/llama-dl-main/vicuna_13B  \
> --data_path /mnt/bd/data-tns-algo-masp-llm/experiment/LLaVA/LLaVA-Instruct-150K/llava_instruct_150k.json \
> --image_folder /mnt/bd/data-tns-algo-masp-llm/experiment/LLaVA/data/coco/train2014 \
> --vision_tower openai/clip-vit-large-patch14 \
> --pretrain_mm_mlp_adapter /mnt/bd/data-tns-algo-masp-llm/experiment/LLaVA/checkpoints/llava-13b-pretrain/mm_projector.bin \
> --mm_vision_select_layer -2 \
> --mm_use_im_start_end True \
> --bf16 True \
> --output_dir ./checkpoints \
> --num_train_epochs 3 \
> --per_device_train_batch_size 4 \
> --per_device_eval_batch_size 4 \
> --gradient_accumulation_steps 1 \
> --evaluation_strategy "no" \
> --save_strategy "steps" \
> --save_steps 5000 \
> --save_total_limit 3 \
> --learning_rate 2e-5 \
> --weight_decay 0. \
> --warmup_ratio 0.03 \
> --lr_scheduler_type "cosine" \
> --logging_steps 1 \
> --tf32 True \
> --fsdp "full_shard auto_wrap" \
> --fsdp_transformer_layer_cls_to_wrap 'LlamaDecoderLayer' \
> --model_max_length 2048 \
> --gradient_checkpointing True \
> --lazy_preprocess True \
> --report_to wandb

/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/lib/python3.10/site-packages/transformers/training_args.py:1356: FutureWarning: using `--fsdp_transformer_layer_cls_to_wrap` is deprecated. Use fsdp_config instead
  warnings.warn(
/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/lib/python3.10/site-packages/transformers/training_args.py:1356: FutureWarning: using `--fsdp_transformer_layer_cls_to_wrap` is deprecated. Use fsdp_config instead
  warnings.warn(
/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/lib/python3.10/site-packages/transformers/training_args.py:1356: FutureWarning: using `--fsdp_transformer_layer_cls_to_wrap` is deprecated. Use fsdp_config instead
  warnings.warn(
/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/lib/python3.10/site-packages/transformers/training_args.py:1356: FutureWarning: using `--fsdp_transformer_layer_cls_to_wrap` is deprecated. Use fsdp_config instead
  warnings.warn(
/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/lib/python3.10/site-packages/transformers/training_args.py:1356: FutureWarning: using `--fsdp_transformer_layer_cls_to_wrap` is deprecated. Use fsdp_config instead
  warnings.warn(
You are using a model of type llama to instantiate a model of type llava. This is not supported for all configurations of models and can yield errors.
You are using a model of type llama to instantiate a model of type llava. This is not supported for all configurations of models and can yield errors.
You are using a model of type llama to instantiate a model of type llava. This is not supported for all configurations of models and can yield errors.
You are using a model of type llama to instantiate a model of type llava. This is not supported for all configurations of models and can yield errors.
You are using a model of type llama to instantiate a model of type llava. This is not supported for all configurations of models and can yield errors.
/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/lib/python3.10/site-packages/transformers/training_args.py:1356: FutureWarning: using `--fsdp_transformer_layer_cls_to_wrap` is deprecated. Use fsdp_config instead
  warnings.warn(
/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/lib/python3.10/site-packages/transformers/training_args.py:1356: FutureWarning: using `--fsdp_transformer_layer_cls_to_wrap` is deprecated. Use fsdp_config instead
  warnings.warn(
You are using a model of type llama to instantiate a model of type llava. This is not supported for all configurations of models and can yield errors.
You are using a model of type llama to instantiate a model of type llava. This is not supported for all configurations of models and can yield errors.
/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/lib/python3.10/site-packages/transformers/training_args.py:1356: FutureWarning: using `--fsdp_transformer_layer_cls_to_wrap` is deprecated. Use fsdp_config instead
  warnings.warn(
You are using a model of type llama to instantiate a model of type llava. This is not supported for all configurations of models and can yield errors.
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 144298 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 144299 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 144300 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 144301 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 144302 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 144303 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 144304 closing signal SIGTERM
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: -9) local_rank: 7 (pid: 144305) of binary: /mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/bin/python
Traceback (most recent call last):
  File "/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/bin/torchrun", line 8, in <module>
    sys.exit(main())
  File "/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 346, in wrapper
    return f(*args, **kwargs)
  File "/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/lib/python3.10/site-packages/torch/distributed/run.py", line 794, in main
    run(args)
  File "/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/lib/python3.10/site-packages/torch/distributed/run.py", line 785, in run
    elastic_launch(
  File "/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 134, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 250, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
=======================================================
llava/train/train_mem.py FAILED
-------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
-------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2023-05-01_13:00:38
  host      : n193-016-074.byted.org
  rank      : 7 (local_rank: 7)
  exitcode  : -9 (pid: 144305)
  error_file: <N/A>
  traceback : Signal 9 (SIGKILL) received by PID 144305
=======================================================

Hi, a random thought: can this be related to ulimit?
Specifically, what's ulimit -u, ulimit -v, ulimit -m on your machine?

Hi, a random thought: can this be related to ulimit?
Specifically, what's ulimit -u, ulimit -v, ulimit -m on your machine?
And what's your OS type/version, and CUDA version (just to find the disparity between our machines)?

all will output:
unlimited

Sad, then that's not the cause either.  I am not sure what we can do to debug this.  One last attempt is to try what ChatGPT suggests about record:
from torch.distributed.elastic.multiprocessing.errors import record

@record
def main():
    # Your code here

So maybe we can add to llava/train.py?
@record
def train():

from torch.distributed.elastic.multiprocessing.errors import record

@record

same errors:
OGLEVEL=INFO TORCHELASTIC_ENABLE_FILE_TIMER=1 torchrun --nnodes=1 --nproc_per_node=8 --master_port=25001     llava/train/train_mem.py     --model_name_or_path /mnt/bd/data-tns-algo-masp-llm/weights/llama-dl-main/vicuna_13B     --data_path /mnt/bd/data-tns-algo-masp-llm/experiment/LLaVA/LLaVA-Instruct-150K/chat.json     --image_folder /mnt/bd/data-tns-algo-masp-llm/experiment/LLaVA/LLaVA-Instruct-150K/images     --vision_tower openai/clip-vit-large-patch14     --tune_mm_mlp_adapter True     --mm_vision_select_layer -2     --mm_use_im_start_end     --bf16 True     --output_dir ./checkpoints/llava-13b-pretrain     --num_train_epochs 1     --per_device_train_batch_size 16     --per_device_eval_batch_size 4     --gradient_accumulation_steps 1     --evaluation_strategy "no"     --save_strategy "steps"     --save_steps 2400     --save_total_limit 1     --learning_rate 2e-3     --weight_decay 0.     --warmup_ratio 0.03     --lr_scheduler_type "cosine"     --logging_steps 1     --tf32 True     --model_max_length 2048     --gradient_checkpointing True     --lazy_preprocess True     --debug underflow_overflow     --report_to none
INFO:torch.distributed.launcher.api:Starting elastic_operator with launch configs:
  entrypoint       : llava/train/train_mem.py
  min_nodes        : 1
  max_nodes        : 1
  nproc_per_node   : 8
  run_id           : none
  rdzv_backend     : static
  rdzv_endpoint    : <IP_ADDRESS>:25001
  rdzv_configs     : {'rank': 0, 'timeout': 900}
  max_restarts     : 0
  monitor_interval : 5
  log_dir          : None
  metrics_cfg      : {}

INFO:torch.distributed.elastic.agent.server.local_elastic_agent:log directory set to: /tmp/torchelastic_zg87tjfx/none_hh2up_uh
INFO:torch.distributed.elastic.agent.server.api:[default] starting workers for entrypoint: python
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous'ing worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous complete for workers. Result:
  restart_count=0
  master_addr=<IP_ADDRESS>
  master_port=25001
  group_rank=0
  group_world_size=1
  local_ranks=[0, 1, 2, 3, 4, 5, 6, 7]
  role_ranks=[0, 1, 2, 3, 4, 5, 6, 7]
  global_ranks=[0, 1, 2, 3, 4, 5, 6, 7]
  role_world_sizes=[8, 8, 8, 8, 8, 8, 8, 8]
  global_world_sizes=[8, 8, 8, 8, 8, 8, 8, 8]

INFO:torch.distributed.elastic.agent.server.api:[default] Starting worker group
INFO:torch.distributed.elastic.agent.server.local_elastic_agent:Starting a FileTimerServer with /tmp/watchdog_timer_f83bf4ec-c486-4774-9b56-c1eda1f9192d ...
INFO:torch.distributed.elastic.agent.server.local_elastic_agent:FileTimerServer started
INFO:torch.distributed.elastic.multiprocessing:Setting worker0 reply file to: /tmp/torchelastic_zg87tjfx/none_hh2up_uh/attempt_0/0/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker1 reply file to: /tmp/torchelastic_zg87tjfx/none_hh2up_uh/attempt_0/1/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker2 reply file to: /tmp/torchelastic_zg87tjfx/none_hh2up_uh/attempt_0/2/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker3 reply file to: /tmp/torchelastic_zg87tjfx/none_hh2up_uh/attempt_0/3/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker4 reply file to: /tmp/torchelastic_zg87tjfx/none_hh2up_uh/attempt_0/4/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker5 reply file to: /tmp/torchelastic_zg87tjfx/none_hh2up_uh/attempt_0/5/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker6 reply file to: /tmp/torchelastic_zg87tjfx/none_hh2up_uh/attempt_0/6/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker7 reply file to: /tmp/torchelastic_zg87tjfx/none_hh2up_uh/attempt_0/7/error.json
You are using a model of type llama to instantiate a model of type llava. This is not supported for all configurations of models and can yield errors.
You are using a model of type llama to instantiate a model of type llava. This is not supported for all configurations of models and can yield errors.
You are using a model of type llama to instantiate a model of type llava. This is not supported for all configurations of models and can yield errors.
You are using a model of type llama to instantiate a model of type llava. This is not supported for all configurations of models and can yield errors.
You are using a model of type llama to instantiate a model of type llava. This is not supported for all configurations of models and can yield errors.
You are using a model of type llama to instantiate a model of type llava. This is not supported for all configurations of models and can yield errors.
You are using a model of type llama to instantiate a model of type llava. This is not supported for all configurations of models and can yield errors.
You are using a model of type llama to instantiate a model of type llava. This is not supported for all configurations of models and can yield errors.
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 216958 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 216959 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 216960 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 216961 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 216962 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 216963 closing signal SIGTERM
WARNING:torch.distributed.elastic.multiprocessing.api:Sending process 216964 closing signal SIGTERM
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: -9) local_rank: 0 (pid: 216957) of binary: /mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/bin/python
INFO:torch.distributed.elastic.agent.server.api:Local worker group finished (FAILED). Waiting 300 seconds for other agents to finish
INFO:torch.distributed.elastic.agent.server.api:Done waiting for other agents. Elapsed: 0.008567571640014648 seconds
INFO:torch.distributed.elastic.multiprocessing.errors:local_rank 0 FAILED with no error file. Decorate your entrypoint fn with @record for traceback info. See: https://pytorch.org/docs/stable/elastic/errors.html
Traceback (most recent call last):
  File "/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/bin/torchrun", line 8, in <module>
    sys.exit(main())
  File "/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 346, in wrapper
    return f(*args, **kwargs)
  File "/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/lib/python3.10/site-packages/torch/distributed/run.py", line 794, in main
    run(args)
  File "/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/lib/python3.10/site-packages/torch/distributed/run.py", line 785, in run
    elastic_launch(
  File "/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 134, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/mnt/bd/data-tns-algo-masp-llm/environment/anaconda3/envs/llava/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 250, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
=======================================================
llava/train/train_mem.py FAILED
-------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
-------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2023-05-01_13:32:26
  host      : n193-016-074.byted.org
  rank      : 0 (local_rank: 0)
  exitcode  : -9 (pid: 216957)
  error_file: <N/A>
  traceback : Signal 9 (SIGKILL) received by PID 216957
=======================================================

INFO:torch.distributed.elastic.multiprocessing:Setting worker0 reply file to: /tmp/torchelastic_zg87tjfx/none_hh2up_uh/attempt_0/0/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker1 reply file to: /tmp/torchelastic_zg87tjfx/none_hh2up_uh/attempt_0/1/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker2 reply file to: /tmp/torchelastic_zg87tjfx/none_hh2up_uh/attempt_0/2/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker3 reply file to: /tmp/torchelastic_zg87tjfx/none_hh2up_uh/attempt_0/3/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker4 reply file to: /tmp/torchelastic_zg87tjfx/none_hh2up_uh/attempt_0/4/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker5 reply file to: /tmp/torchelastic_zg87tjfx/none_hh2up_uh/attempt_0/5/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker6 reply file to: /tmp/torchelastic_zg87tjfx/none_hh2up_uh/attempt_0/6/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker7 reply file to: /tmp/torchelastic_zg87tjfx/none_hh2up_uh/attempt_0/7/error.json

Anything in these files?

INFO:torch.distributed.elastic.multiprocessing:Setting worker0 reply file to: /tmp/torchelastic_zg87tjfx/none_hh2up_uh/attempt_0/0/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker1 reply file to: /tmp/torchelastic_zg87tjfx/none_hh2up_uh/attempt_0/1/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker2 reply file to: /tmp/torchelastic_zg87tjfx/none_hh2up_uh/attempt_0/2/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker3 reply file to: /tmp/torchelastic_zg87tjfx/none_hh2up_uh/attempt_0/3/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker4 reply file to: /tmp/torchelastic_zg87tjfx/none_hh2up_uh/attempt_0/4/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker5 reply file to: /tmp/torchelastic_zg87tjfx/none_hh2up_uh/attempt_0/5/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker6 reply file to: /tmp/torchelastic_zg87tjfx/none_hh2up_uh/attempt_0/6/error.json
INFO:torch.distributed.elastic.multiprocessing:Setting worker7 reply file to: /tmp/torchelastic_zg87tjfx/none_hh2up_uh/attempt_0/7/error.json

Anything in these files?
all is empty:

😮💨 I have no more ideas now. If you somehow solve this issue, please let me know and maybe also share the solution here.  Thanks!

I will for sure come back to let you know if I can solve it.

I will for sure come back to let you know if I can solve it.

Hi is there any process

I will for sure come back to let you know if I can solve it.

hello how's it going?

upgrade to pytorch 2.0 fix all of it


Re-enable coverage shard for flutter_tools

We're going to disable the coverage shard until we can determine the cause and fix for the increased memory usage. While the coverage shard previously took less than 12G of memory, it is currently (as of e0ebaea59071b35a44dbe1e0830ee15fb7563486) using more than 173 G.
We might try using pub run test --coverage to shed the custom test runner as a potential source of failures. This is blocked by https://github.com/dart-lang/sdk/issues/40627

@zanderso

We decided not to disable the coverage shard.


Add HTTP/1 PING(OPTIONS *)

Motivation:
Sometimes, a client that sends a request via a load balancer could get ClosedSessionException
because the load balancer closes the connection while sending the request.
Please see #2576 for detail information.
Modifications:

Make HTTP/1 client send PING with OPTIONS * when a connection is idle.
Introduce IdleTimeoutScheduler that schedules a PING interval and an idle timeout.

Fork IdleTimeoutScheduler from Netty IdleTimeoutHandler and add PING idle.
Remove IdleTimeoutHandler in favor of KeepAliveHandler

Split PING traffic from normal read and write.

Regardless of PING requests and responses, the connection would be closed on idle.

Add defaultPingIntervalMillis flag to configure PING interval.
Breaking

Remove Flags.defaultHttp2PingTimeoutMillis in favor of defaultPingIntervalMillis.
Remove Flags.defaultUseHttp2PingWhenNoActiveStreams without replacement.

HTTP/2 can send PING frame with or without active streams.
HTTP/1 can only send PING on no active streams.

Remove ClientFactory*.http2PingTimeoutMillis() and ServerBuilder.http2PingTimeoutMillis()
in favor of *.pingIntervalMillis()

Miscellaneous

Move ClientHttp*ObjectEncoder to client package from internal and make them package private
Move ServerHttp*ObjectEncoder to server package from internal and make them package private

Result:

Armeria Client can send OPTION * when a connection is idle.
You can now prevent an unexpected disconnection from an idle timeout of a load balancer.
Fixes #2576

@sivaalli You might be interested in this as well.


discussion - new vs old packet format

Furthermore some packets in your message were using the new packet lengths format (indicated by new CTB in the dumper) where pgpainless/BC uses the old format (old CTB). It would surprise me if this was the cause of the issue though.

Originally posted by @vanitasvitae in https://github.com/pgpainless/pgpainless/issues/160#issuecomment-890405862
I'm posting this here in case it may be preferable to use the new format.

Unfortunately whether or not the new format is used is hard coded inside of BC (eg. here)
out.writePacket(tag: PUBLIC_KEY_ENC_SESSION , data: bOut.toByteArray(), oldFormat: true);

For the long term, it will be very good to have this be configurable by the BC api.
I know BC can be slow to accept fixes, but it will be worthwhile to submit this functionality to BC, so that we have more flexibility in PGPainless.

I'm not sure, how to elegantly implement such an API though. Add a static OLD_PACKET_FORMAT boolean to all packet classes?
Currently each packet "decides" for itself, whether or not to use the old/new format.
Maybe BC devs can be convinced to have a "switch day" where a new release completely switches over to the new format when encoding packets?

I'm not too familiar with BC, so I'm not sure. Does it have any global config?

Maybe BC devs can be convinced to have a "switch day" where a new release completely switches over to the new format when encoding packets?

Maybe. Failing that, a configurable switch in code instead of a switch day - somehow.
Or a fork, again :-/

It looks like BC will switch to the new packet format exclusively at some point (see https://github.com/deviantdev/bc-java/commit/94e8b9c5924c534fce17a5f19c18a6fa2507ad7d and https://github.com/deviantdev/bc-java/commit/2d7814a5f15ee2bc7023345630a50395ef2a2683).


exception handling

Summary
Proper exception handling, just in case, although I can't reproduce an exception reaching this point with the URLs the client suggested it is something middleware handlers should be doing
What are the specific steps to test this change?
Redirects work normally, non-redirects work normally
What kind of change does this PR introduce?
(Check at least one)

[X] Bug fix
[ ] New feature
[ ] Refactor
[ ] Documentation
[ ] Build-related changes
[ ] Other

Make sure the PR fulfills these requirements:

[ ] It includes a) the existing issue ID being resolved, b) a convincing reason for adding this feature, or c) a clear description of the bug it resolves
[ ] The changelog is updated
[ ] Related documentation has been updated
[ ] Related tests have been updated

If adding a new feature without an already open issue, it's best to open a feature request issue first and wait for approval before working on it.
Other information:

The CircleCI error is probably related with outdated repo configuration.


Experimenting w/ Datasets - Computer Vision

Objectives:

[x] Primary task: Pull hex values from the skin tone palettes from the game screenshots
[x] Potential thing: Pull dominant color values from the dress up screen

Notes:

If the original comp. vision idea doesn't work, try to find a different comp. method & let me know -A.G.
If you need help you & Thea could work on this together

Updates: Divya ran into issues with the computer vision software; will be switching to an alternative method (her method of choice: linear regression predictive model to predict # of skin tones?)

Completed - Currently having merge issues but will be resolved later tonight


Float parsing

Currently, float parsing is only partially supported. To see which cases are handled and which are not, run org.simdjson.SimdJsonParserTest#testFloatValues. We would like to cover all cases described in the JSON specification.

I'd have some time currently, because I have holidays and switching the job afterwards (maybe doing embedded stuff in the future, which would be completely new ;-)).
That said, I'm currently writing a technical design document about SirixDB: https://sirix.io/docs/concepts.html
So it's not urgent :-) and there's plenty of other stuff to do regarding SirixDB 🙄


Code for video action recognition not available

I found MViT quite interesting and wished to use it as a backbone for video quality assessment, and to that end, I tried to search for the code of video action recognition(link : https://github.com/facebookresearch/SlowFast/tree/main/projects/mvitv2), but it seems the code is not available, or not released.
It would be a huge help if you release the code for videos, since I am a novice in the field. If the code is not available or cannot be released, could you provide me some tips/pointers where the major changes would take place?

Hello, @RushiPardeshi, did you get any response or the solution for the code for video action recognition? I am also searching for the same thing.

Any updates? thank you

https://github.com/facebookresearch/SlowFast/blob/88bb4c9cf03fd97fce403f8b475543cb8e8ca5ea/slowfast/models/video_model_builder.py#L806


chore(dev): update stripe-mock to latest

Thanks! What is that really doing for us?

Added in 8154b67

Thanks! What is that really doing for us?

The organization billing tests hit it.

Thanks! What is that really doing for us?

The organization billing tests hit it.

Derp, I probably should have phrased that differently. I meant to inquire what this interesting syntax was doing - and it looks like it's applying an if statement where an if isn't readily available, so it's creating an image string only when the condition applies.

Thanks! What is that really doing for us?

The organization billing tests hit it.

Derp, I probably should have phrased that differently. I meant to inquire what this interesting syntax was doing - and it looks like it's applying an if statement where an if isn't readily available, so it's creating an image string only when the condition applies.

See https://github.com/pypi/warehouse/pull/13646/commits/4950f9465eb0c6c8c020b374dba470150e21becd


Can we train an agent without opening a window?

Hi there, is it possible to train an RL agent without opening the simulation window? I'm trying to do training on my server but it can be problematic to open a window there.
Any help or response will be highly appreciated. Thanks!

Yes, you can train without a window.

Yes, you can train without a window.
Calling env.step() does not open a window.

Thanks for your quick reply! I would like to know does this error occurs because of trying to open a window?
pyglet.canvas.xlib.NoSuchDisplayException: Cannot connect to "None"
which is also mentioned in #13 .

That has to do with trying to connect to a display for rendering. Have you tried using xvfb-run as outlined here?
https://github.com/maximecb/gym-miniworld/blob/master/docs/troubleshooting.md#nosuchdisplayexception-cannot-connect-to-none

That has to do with trying to connect to a display for rendering. Have you tried using xvfb-run as outlined here?
https://github.com/maximecb/gym-miniworld/blob/master/docs/troubleshooting.md#nosuchdisplayexception-cannot-connect-to-none

Thanks again. Actually the cluster I use cannot install xvfb for some reason. I wonder whether it is possible to do training without displaying or rendering.

You should contact the cluster administrators and try to get them to install xvfb. It's necessary for many environments that use OpenGL for rendering, unfortunately.
I'm trying to get the people behind the pyglet package to remove this dependency, by implementing EGL support, but that hasn't been moving.

You should contact the cluster administrators and try to get them to install xvfb. It's necessary for many environments that use OpenGL for rendering, unfortunately.
I'm trying to get the people behind the pyglet package to remove this dependency, by implementing EGL support, but that hasn't been moving.

Thank you so much! I will try.


Second test

Yo

IssuePress Data:
username: tw2113
email<EMAIL_ADDRESS>firstname:
lastname:
display_name: tw2113
ID: 1
gravatar_id: aab0466d38dc0d58c326d4986a4058a6
Sent via IssuePress

Reported.


import "github.com/google/go-github/v27/github" doesnt work

Unable to use.
import "github.com/google/go-github/v27/github"
This does work however.
import "github.com/google/go-github/github"

This is the second issue we've seen like this and I'd really like to solve the problem, so please help me duplicate it.
Here's what I tried as an experiment... I copied the examples/simple/main.go file into a clean directory outside my GOPATH, then I ran this:
$ GO111MODULE=on go get -u github.com/google/go-github/v27/github
go: finding github.com/google/go-github/v27/github latest
go: finding github.com/google/go-github/v27 v27.0.0
go: downloading github.com/google/go-github/v27 v27.0.0
go: extracting github.com/google/go-github/v27 v27.0.0
go: finding golang.org/x/sync latest
...
go: extracting golang.org/x/crypto v0.0.0-20190701094942-4def268fd1a4

$ GO111MODULE=on go run main.go
go: finding github.com/google/go-github/v27/github latest
Enter GitHub username:

What are the steps you are taking that fail?

Closing due to inactivity.


feat: Send commands to weed shell from the docker image.

Add the ability to send commands to weed shell from the docker image.

How are we solving the problem?
Extend existing Docker entrypoints to include weed shell
Checks

[ ] I have added unit tests if possible.
[ ] I will add related wiki document changes and link to this PR after merging.

please add to wiki doc here https://github.com/chrislusf/seaweedfs/wiki/weed-shell

@chrislusf updated here: https://github.com/chrislusf/seaweedfs/wiki/weed-shell#run-from-docker-image
Thank you so much for your support!


imagesPath is not valid attribute

Describe the bug
Readme states that imagesPath is attribute which can be used to adjust storage path. It is not valid as in code there is imagesDir.
To Reproduce
Steps to reproduce the behavior:

Try using matchImage({ imagesPath: 'some-path/ }) and it will not do any change.
Try using matchImage({ imagesDir: 'some-path/ }) and it will change where images are stored.

Expected behavior
Readme should match codebase.

What version do you use? Currently in the codebase both imageDir and imagePath are supported. Also, imageDir is marked as deprecated and will be removed in version 4.
One note: to be able to use imagePath you cannot use imageDir anywhere in your environment.

Code where imagesPath is being used:
https://github.com/FRSOURCE/cypress-plugin-visual-regression-diff/blob/f4d3ec946547d648d1ec8ea9ccf9369540255adf/src/commands.ts#L96-L99
https://github.com/FRSOURCE/cypress-plugin-visual-regression-diff/blob/f4d3ec946547d648d1ec8ea9ccf9369540255adf/src/commands.ts#L151

Closing as invalid


Dummy PR to get the latest tag published

Signed-off-by: Sherin Varughese<EMAIL_ADDRESS>Related Issue: https://github.com/stolostron/backlog/issues/22455
Description of changes

/lgtm


Update doc for AMD

What does this PR do?
Replacing IPU with NPU for AMD Ryzen
Before submitting

[ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
[ ] Did you make sure to update the documentation with your changes?
[ ] Did you write any new necessary tests?

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.


Added "Back" and "Forward" web functionality

-Two new menu items under "Location" that allow you to navigate back
and forward similar to most web browsers
-Controlled in HeliumPanelController

Great work!


Unable to select display for video capturing (Log: The system doesn't support the X11 system grabber)

Bug report, debug log and your config file (FULL LOGS ARE MANDATORY)

2024-04-28T00:16:29.649Z [CORE] TTY is not attached to the log output
2024-04-28T00:16:29.660Z [MAIN] Database path: '/home/nico/.hyperhdr/db/hyperhdr.db', readonlyMode = disabled
2024-04-28T00:16:29.660Z [MAIN] Starting HyperHdr - <IP_ADDRESS>, (HEAD detached at v<IP_ADDRESS>) (awawa-dev-9e960c9/52f12b4-1711236142), built: Mar 23 2024:23:33:32
2024-04-28T00:16:29.660Z [MAIN] Set user data path to '/home/nico/.hyperhdr'
2024-04-28T00:16:29.660Z [INSTANCE_CFG] Loading instance configuration
2024-04-28T00:16:29.660Z [DB] Database opened: /home/nico/.hyperhdr/db/hyperhdr.db
2024-04-28T00:16:29.666Z [INSTANCE_CFG] Settings database initialized
2024-04-28T00:16:29.668Z [SOUND_GRABBER] Sound device is disabled
2024-04-28T00:16:29.681Z [JSONSERVER] (JsonServer.cpp:26) Created new instance
2024-04-28T00:16:29.681Z [JSONSERVER] Started on port 19444
2024-04-28T00:16:29.681Z [SSDP] (SSDPHandler.cpp:31) SSDPHandler is initialized
2024-04-28T00:16:29.678Z [V4L2:auto] (GrabberWrapper.cpp:55) Starting the grabber wrapper
2024-04-28T00:16:29.678Z [V4L2:AUTO] Restarting video grabber is now blocked due to reloading of the configuration
2024-04-28T00:16:29.678Z [V4L2:AUTO] Cropping image: width=0 height=0; crop: left=0 right=0 top=0 bottom=0
2024-04-28T00:16:29.678Z [V4L2:auto] (GrabberWrapper.cpp:126) CEC keycode. Start: 0, stop: 0
2024-04-28T00:16:29.678Z [V4L2:AUTO] (Grabber.cpp:273) setBrightnessContrastSaturationHue nothing changed
2024-04-28T00:16:29.678Z [V4L2:AUTO] (V4L2Grabber.cpp:125) setHdrToneMappingMode to: Disabled
2024-04-28T00:16:29.678Z [V4L2:AUTO] (Grabber.cpp:182) setFpsSoftwareDecimation to: 1
2024-04-28T00:16:29.678Z [SIGNAL_OLD] (DetectionManual.cpp:84) Signal detection area set to: 0.250000,0.250000 x 0.750000,0.750000
2024-04-28T00:16:29.678Z [SIGNAL_OLD] (DetectionManual.cpp:74) Signal threshold set to: {12, 12, 12} and frames: 200
2024-04-28T00:16:29.678Z [SIGNAL_AUTO] (DetectionAutomatic.cpp:375) Automatic signal detection -> errorTolerance: 9, modelTolerance: 90, sleepTime: 5000, wakeTime: 1000
2024-04-28T00:16:29.678Z [V4L2:AUTO] (Grabber.cpp:380) setDeviceVideoStandard preparing to restart video grabber. Old: '' new: 'auto'
2024-04-28T00:16:29.678Z [V4L2:AUTO] Delayed restart of the grabber due to change of selected device
2024-04-28T00:16:29.678Z [V4L2:AUTO] (Grabber.cpp:200) Force encoding to: NO_CHANGE (old: NO_CHANGE)
2024-04-28T00:16:29.678Z [V4L2:AUTO] setQFrameDecimation is now: disabled
2024-04-28T00:16:29.678Z [X11_SYSTEM:AUTO] Loaded X11 proxy library for screen capturing
2024-04-28T00:16:29.678Z [DAEMON] The system doesn't support the X11 system grabber
2024-04-28T00:16:29.678Z [FRAMEBUFFER_SYSTEM:AUTO] Found FrameBuffer device: /dev/fb0
2024-04-28T00:16:29.679Z [FRAMEBUFFER_SYSTEM:AUTO] Restarting video grabber is now blocked due to reloading of the configuration
2024-04-28T00:16:29.679Z [HYPERHDR0] Starting the instance
2024-04-28T00:16:29.679Z [FRAMEBUFFER_SYSTEM:AUTO] (Grabber.cpp:159) Set new width: 512, height: 512 for capture
2024-04-28T00:16:29.679Z [FRAMEBUFFER_SYSTEM:AUTO] (Grabber.cpp:335) setWidthHeight preparing to restarting video grabber 0
2024-04-28T00:16:29.679Z [FRAMEBUFFER_SYSTEM:AUTO] Delayed restart of the grabber due to change of the capturing resolution
2024-04-28T00:16:29.679Z [FRAMEBUFFER_SYSTEM:AUTO] Delayed restart of the grabber due to change of the framerate
2024-04-28T00:16:29.679Z [INSTANCE_CFG0] Loading instance configuration
2024-04-28T00:16:29.679Z [SIGNAL_OLD] (DetectionManual.cpp:84) Signal detection area set to: 0.250000,0.250000 x 0.750000,0.750000
2024-04-28T00:16:29.679Z [SIGNAL_OLD] (DetectionManual.cpp:74) Signal threshold set to: {0, 0, 0} and frames: 50
2024-04-28T00:16:29.679Z [FRAMEBUFFER_SYSTEM:AUTO] (Grabber.cpp:380) setDeviceVideoStandard preparing to restart video grabber. Old: '' new: 'auto'
2024-04-28T00:16:29.679Z [FRAMEBUFFER_SYSTEM:AUTO] Delayed restart of the grabber due to change of selected device
2024-04-28T00:16:29.679Z [X11_SYSTEM:auto] (SystemWrapper.cpp:71) Closing grabber: X11_SYSTEM:auto
2024-04-28T00:16:29.679Z [DB] Database opened: /home/nico/.hyperhdr/db/hyperhdr.db
2024-04-28T00:16:29.686Z [INSTANCE_CFG0] Settings database initialized
2024-04-28T00:16:29.686Z [COMPONENTCTRL0] (ComponentController.cpp:31) ComponentController is initialized. Components are ENABLED
2024-04-28T00:16:29.686Z [MUXER0] (Muxer.cpp:55) Muxer initialized
2024-04-28T00:16:29.686Z [BLACKBORDER] Set mode to: default
2024-04-28T00:16:29.686Z [IMAGETOLED_MNG0] (ImageToLedManager.cpp:192) Set LED mapping type to advanced
2024-04-28T00:16:29.686Z [IMAGETOLED_MNG0] (ImageToLedManager.cpp:176) setSparseProcessing to 0
2024-04-28T00:16:29.686Z [IMAGETOLED_MNG0] (ImageToLedManager.cpp:103) ImageToLedManager initialized
2024-04-28T00:16:29.686Z [COLORSPACE_CALIBRATION0] classicMode: no, gammas:[1.50, 1.50, 1.50], saturation: 1.00, luminance: 1.00, backLight: [enabled, threshold: 1, colored: yes]
2024-04-28T00:16:29.686Z [LED_CALIBRATION0] (LedCalibration.cpp:101) Calibration config '0' for LED segment: [0, 104]
2024-04-28T00:16:29.687Z [HYPERHDR0] Led strip RGB order is: rgb
2024-04-28T00:16:29.687Z [SMOOTHING0] High resolution clock is NOT STEADY!
2024-04-28T00:16:29.687Z [SMOOTHING0] Clearing queued colors before: enabling
2024-04-28T00:16:29.687Z [SMOOTHING0] Smoothing queue is cleared
2024-04-28T00:16:29.687Z [COMPONENTCTRL0] Smoothing: enabled
2024-04-28T00:16:29.687Z [SMOOTHING0] Creating config (0) => type: Alternative, dirMode: false, pause: false, settlingTime: 150ms, interval: 20ms (50Hz), antiFlickTres: 32, antiFlickStep: 2, antiFlickTime: 250
2024-04-28T00:16:29.687Z [SMOOTHING0] Clearing queued colors before: enabling. Smoothing configuration changed: restarting timer.
2024-04-28T00:16:29.687Z [SMOOTHING0] Smoothing queue is cleared
2024-04-28T00:16:29.687Z [SMOOTHING0] Selecting config (0) => type: Alternative, directMode: false, pause: false, settlingTime: 150ms, interval: 20ms (50Hz), antiFlickTres: 32, antiFlickStep: 2, antiFlickTime: 250
2024-04-28T00:16:29.687Z [SMOOTHING0] Selecting config (0) => type: Alternative, directMode: false, pause: false, settlingTime: 150ms, interval: 20ms (50Hz), antiFlickTres: 32, antiFlickStep: 2, antiFlickTime: 250
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] Start LedDevice 'wled'.
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] (LedDeviceWled.cpp:36) Initializing WLED
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] (LedDevice.cpp:337) deviceConfig: [{"brightnessMax":true,"brightnessMaxLevel":255,"colorOrder":"rgb","currentLedCount":105,"host":"<IP_ADDRESS>","maxRetry":60,"refreshTime":20,"restoreOriginalState":false,"smoothingRefreshTime":20,"type":"wled"}]
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] Ignoring user LED refresh rate. Forcing smoothing refresh rate = 50.00 Hz
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] (LedDevice.cpp:423) Refresh rate = 50.00 Hz
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] (LedDevice.cpp:373) Device is not ready to start a timer
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] (LedDevice.cpp:433) Refresh interval updated to 20ms
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] (LedDeviceWled.cpp:45) DeviceType     : wled
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] (LedDeviceWled.cpp:46) LedCount       : 105
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] (LedDeviceWled.cpp:49) Override brightness : true
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] (LedDeviceWled.cpp:52) Set brightness level: 255
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] (LedDeviceWled.cpp:55) Restore WLED   : false
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] (LedDeviceWled.cpp:58) Max retry      : 60
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] (LedDeviceWled.cpp:96)
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] (LedDeviceWled.cpp:107) [1]
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] (LedDevice.cpp:337) deviceConfig: [{"brightnessMax":true,"brightnessMaxLevel":255,"colorOrder":"rgb","currentLedCount":105,"host":"<IP_ADDRESS>","maxRetry":60,"port":21324,"refreshTime":20,"restoreOriginalState":false,"smoothingRefreshTime":20,"type":"wled"}]
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] Ignoring user LED refresh rate. Forcing smoothing refresh rate = 50.00 Hz
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] (LedDevice.cpp:423) Refresh rate = 50.00 Hz
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] (LedDevice.cpp:373) Device is not ready to start a timer
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] (LedDevice.cpp:433) Refresh interval updated to 20ms
2024-04-28T00:16:29.687Z [HYPER_MANAGER] HyperHDR instance 'First LED instance' has been started
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] (ProviderUdp.cpp:64) Successfully parsed <IP_ADDRESS> as an IP-address.
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] (ProviderUdp.cpp:94) UDP socket will write to <IP_ADDRESS>:21324
2024-04-28T00:16:29.687Z [HYPERHDR0] The instance is running
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] (LedDeviceWled.cpp:85) Hostname/IP  : <IP_ADDRESS>
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] (LedDeviceWled.cpp:86) Port         : 21324
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] (LedDeviceWled.cpp:90) [1]
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] (LedDevice.cpp:247) Enable the device
2024-04-28T00:16:29.687Z [MUXER0] Register new input 'System/SYSTEMGRABBER' with priority 245 as inactive
2024-04-28T00:16:29.687Z [COMPONENTCTRL0] System capture device: enabled
2024-04-28T00:16:29.687Z [FRAMEBUFFER_SYSTEM:AUTO] (FrameBufGrabber.cpp:98) init
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] (LedDevice.cpp:64) Switch on
2024-04-28T00:16:29.687Z [LEDDEVICE0_WLED] (LedDeviceWled.cpp:127)
2024-04-28T00:16:29.687Z [LEDDEVICE] (ProviderRestApi.cpp:174) GET begin: [http://<IP_ADDRESS>:80/json] []
2024-04-28T00:16:29.687Z [FRAMEBUFFER_SYSTEM:AUTO] (FrameBufGrabber.cpp:117) Forcing auto discovery device
2024-04-28T00:16:29.687Z [FRAMEBUFFER_SYSTEM:AUTO] (FrameBufGrabber.cpp:122) Auto discovery set to /dev/fb0
2024-04-28T00:16:29.687Z [FRAMEBUFFER_SYSTEM:AUTO] *************************************************************************************************
2024-04-28T00:16:29.687Z [FRAMEBUFFER_SYSTEM:AUTO] Starting FrameBuffer grabber. Selected: '/dev/fb0' (0) max width: 512 (512) @ 10 fps
2024-04-28T00:16:29.687Z [FRAMEBUFFER_SYSTEM:AUTO] *************************************************************************************************
2024-04-28T00:16:29.687Z [FRAMEBUFFER_SYSTEM:AUTO] Device '/dev/fb0' is using currently 2560x1080x32 resolution.
2024-04-28T00:16:29.687Z [FRAMEBUFFER_SYSTEM:AUTO] Started
2024-04-28T00:16:29.687Z [FLATBUFSERVER] Started on port 19400
2024-04-28T00:16:29.687Z [WEBSERVER] Initialize Webserver
2024-04-28T00:16:29.687Z [FLATBUFSERVER] Started local domain socket server: 'hyperhdr-domain'
2024-04-28T00:16:29.687Z [NETFORWARDER] Forwarder has changed state to disabled
2024-04-28T00:16:29.689Z [WEBSERVER] Apply Webserver settings
2024-04-28T00:16:29.689Z [WEBSERVER] Set document root to: :/www
2024-04-28T00:16:29.689Z [WEBSERVER] Started: 'HyperHDR WebServer (HTTP)' on port: 8090
2024-04-28T00:16:29.689Z [SSDP] (SSDPHandler.cpp:93) The local address is empty
2024-04-28T00:16:29.689Z [WEBSERVER] Initialize Webserver
2024-04-28T00:16:29.690Z [WEBSERVER] Apply Webserver settings
2024-04-28T00:16:29.690Z [WEBSERVER] Set document root to: :/www
2024-04-28T00:16:29.691Z [WEBSERVER] Setup SSL certificate
2024-04-28T00:16:29.692Z [WEBSERVER] Setup private SSL key
2024-04-28T00:16:29.692Z [WEBSERVER] Started: 'HyperHDR WebServer (HTTPS)' on port: 8092
2024-04-28T00:16:29.692Z [SSDP] (SSDPHandler.cpp:42) SSDPHandler is initializing
2024-04-28T00:16:29.692Z [SSDP] (SSDPHandler.cpp:183) The local address is: <IP_ADDRESS>
2024-04-28T00:16:29.692Z [SSDP] (SSDPServer.cpp:95) Initialize the SSDP server
2024-04-28T00:16:29.692Z [WEBSERVER] (StaticFileServing.cpp:68) SSDP description is set up
2024-04-28T00:16:29.692Z [SSDP] Starting the SSDP server
2024-04-28T00:16:29.794Z [LEDDEVICE] (ProviderRestApi.cpp:198) GET end (106 ms): [http://<IP_ADDRESS>:80/json] []
2024-04-28T00:16:29.794Z [LEDDEVICE] (ProviderRestApi.cpp:203) Reply OK [200]
2024-04-28T00:16:29.794Z [LEDDEVICE0_WLED] WLED info => wifi quality: 70% (LOW), wifi channel: 11, leds: 105, arch: esp8266, ver: 0.14.3, uptime: 1948572s, port: 21324, power limit: 850mA
2024-04-28T00:16:29.794Z [LEDDEVICE0_WLED] Serious warning: the power limiter in WLED is set which may lead to unexpected side effects. Use the right cabling & power supply with the appropriate power, not this half-measure.
2024-04-28T00:16:29.794Z [LEDDEVICE] (ProviderRestApi.cpp:174) PUT begin: [http://<IP_ADDRESS>:80/json/state] [{"on":true,"live":true,"bri":255}]
2024-04-28T00:16:29.925Z [MUXER0] Set visible priority to 255
2024-04-28T00:16:29.925Z [HYPERHDR0] New priority[255], previous [255]
2024-04-28T00:16:29.925Z [HYPERHDR0] No source left -> switch LED-Device off
2024-04-28T00:16:29.925Z [SMOOTHING0] Clearing queued colors before: disabling
2024-04-28T00:16:29.925Z [SMOOTHING0] Smoothing queue is cleared
2024-04-28T00:16:29.925Z [COLORSPACE_CALIBRATION0] (ColorSpaceCalibration.cpp:225) setBackLightEnabled: 0
2024-04-28T00:16:30.014Z [LEDDEVICE] (ProviderRestApi.cpp:198) PUT end (221 ms): [http://<IP_ADDRESS>:80/json/state] [{"on":true,"live":true,"bri":255}]
2024-04-28T00:16:30.014Z [LEDDEVICE] (ProviderRestApi.cpp:203) Reply OK [200]
2024-04-28T00:16:30.014Z [LEDDEVICE0_WLED] (LedDevice.cpp:367) Starting timer with interval = 20ms
2024-04-28T00:16:30.014Z [LEDDEVICE0_WLED] (LedDevice.cpp:297) Disable the device
2024-04-28T00:16:30.014Z [LEDDEVICE0_WLED] (LedDevice.cpp:381) Stopping refresh timer
2024-04-28T00:16:30.014Z [LEDDEVICE0_WLED] (LedDevice.cpp:90) Switch off
2024-04-28T00:16:30.014Z [LEDDEVICE0_WLED] (LedDeviceWled.cpp:200)
2024-04-28T00:16:30.014Z [LEDDEVICE0_WLED] (LedDevice.cpp:544) Set LED strip to black/power off
2024-04-28T00:16:30.014Z [COMPONENTCTRL0] LED device: enabled
2024-04-28T00:16:30.014Z [LEDDEVICE] (ProviderRestApi.cpp:174) PUT begin: [http://<IP_ADDRESS>:80/json/state] [{"on":false,"live":false}]
2024-04-28T00:16:30.014Z [SMOOTHING0] Using alternative smoothing input (0)
2024-04-28T00:16:30.014Z [EFFECTENGINE0] Run effect "Rainbow swirl fast" on channel 0
2024-04-28T00:16:30.014Z [EFFECTENGINE0] (EffectEngine.cpp:92) Start the effect: name [Rainbow swirl fast]
2024-04-28T00:16:30.014Z [MUXER0] Register new input 'System/EFFECT' with priority 0 as inactive
2024-04-28T00:16:30.015Z [HYPERHDR0] Boot effect: animation set to 'Rainbow swirl fast' [started]
2024-04-28T00:16:30.015Z [EFFECT0(Rainbo...)] Begin playing the effect with priority: 0
2024-04-28T00:16:30.015Z [MUXER0] Priority 0 is now active
2024-04-28T00:16:30.015Z [MUXER0] Set visible priority to 0
2024-04-28T00:16:30.015Z [SMOOTHING0] Clearing queued colors before: enabling. Smoothing configuration changed: restarting timer.
2024-04-28T00:16:30.015Z [SMOOTHING0] Smoothing queue is cleared
2024-04-28T00:16:30.015Z [SMOOTHING0] Selecting config (2) => type: Linear, directMode: false, pause: false, settlingTime: 200ms, interval: 40ms (25Hz), antiFlickTres: 0, antiFlickStep: 0, antiFlickTime: 0
2024-04-28T00:16:30.015Z [SMOOTHING0] Using linear smoothing input (2)
2024-04-28T00:16:30.015Z [HYPERHDR0] New priority[0], previous [255]
2024-04-28T00:16:30.015Z [HYPERHDR0] New source available -> switch LED-Device on
2024-04-28T00:16:30.015Z [SMOOTHING0] Clearing queued colors before: enabling
2024-04-28T00:16:30.015Z [SMOOTHING0] Smoothing queue is cleared
2024-04-28T00:16:30.015Z [COLORSPACE_CALIBRATION0] (ColorSpaceCalibration.cpp:225) setBackLightEnabled: 0
2024-04-28T00:16:30.015Z [IMAGETOLED_MNG0] Total index number is: 420 (memory: 420). User sparse processing is: disabled, image size: 80 x 45, area number: 105
2024-04-28T00:16:30.015Z [SMOOTHING0] Using linear smoothing input (2)
2024-04-28T00:16:30.027Z [MUXER0] Priority 245 is now active
2024-04-28T00:16:30.283Z [LEDDEVICE] (ProviderRestApi.cpp:198) PUT end (268 ms): [http://<IP_ADDRESS>:80/json/state] [{"on":false,"live":false}]
2024-04-28T00:16:30.283Z [LEDDEVICE] (ProviderRestApi.cpp:203) Reply OK [200]
2024-04-28T00:16:30.283Z [LEDDEVICE0_WLED] Ignoring user LED refresh rate. Forcing smoothing refresh rate = 25.00 Hz
2024-04-28T00:16:30.283Z [LEDDEVICE0_WLED] (LedDevice.cpp:423) Refresh rate = 25.00 Hz
2024-04-28T00:16:30.283Z [LEDDEVICE0_WLED] (LedDevice.cpp:373) Device is not ready to start a timer
2024-04-28T00:16:30.283Z [LEDDEVICE0_WLED] (LedDevice.cpp:433) Refresh interval updated to 40ms
2024-04-28T00:16:30.283Z [LEDDEVICE0_WLED] (LedDevice.cpp:448) LED refresh interval adjustment caused by smoothing configuration change to 40ms (proposed: 40ms)
2024-04-28T00:16:30.283Z [LEDDEVICE0_WLED] (LedDevice.cpp:247) Enable the device
2024-04-28T00:16:30.283Z [LEDDEVICE0_WLED] (LedDevice.cpp:64) Switch on
2024-04-28T00:16:30.283Z [LEDDEVICE0_WLED] (LedDeviceWled.cpp:127)
2024-04-28T00:16:30.283Z [LEDDEVICE] (ProviderRestApi.cpp:174) GET begin: [http://<IP_ADDRESS>:80/json] []
2024-04-28T00:16:30.283Z [COMPONENTCTRL0] LED device: disabled
2024-04-28T00:16:30.392Z [LEDDEVICE] (ProviderRestApi.cpp:198) GET end (109 ms): [http://<IP_ADDRESS>:80/json] []
2024-04-28T00:16:30.392Z [LEDDEVICE] (ProviderRestApi.cpp:203) Reply OK [200]
2024-04-28T00:16:30.392Z [LEDDEVICE0_WLED] WLED info => wifi quality: 70% (LOW), wifi channel: 11, leds: 105, arch: esp8266, ver: 0.14.3, uptime: 1948572s, port: 21324, power limit: 850mA
2024-04-28T00:16:30.392Z [LEDDEVICE0_WLED] Serious warning: the power limiter in WLED is set which may lead to unexpected side effects. Use the right cabling & power supply with the appropriate power, not this half-measure.
2024-04-28T00:16:30.392Z [LEDDEVICE] (ProviderRestApi.cpp:174) PUT begin: [http://<IP_ADDRESS>:80/json/state] [{"on":true,"live":true,"bri":255}]
2024-04-28T00:16:30.564Z [LEDDEVICE] (ProviderRestApi.cpp:198) PUT end (172 ms): [http://<IP_ADDRESS>:80/json/state] [{"on":true,"live":true,"bri":255}]
2024-04-28T00:16:30.564Z [LEDDEVICE] (ProviderRestApi.cpp:203) Reply OK [200]
2024-04-28T00:16:30.564Z [LEDDEVICE0_WLED] (LedDevice.cpp:367) Starting timer with interval = 40ms
2024-04-28T00:16:30.564Z [COMPONENTCTRL0] LED device: enabled
2024-04-28T00:16:30.644Z [SMOOTHING0] Using linear smoothing procedure (2)
2024-04-28T00:16:30.672Z [WEBSOCKET] (WebSocketClient.cpp:39) New connection from ::ffff:<IP_ADDRESS>
2024-04-28T00:16:30.762Z [WEBSOCKET] (HyperAPI.cpp:957) log streaming activated for client ::ffff:<IP_ADDRESS>
2024-04-28T00:16:31.190Z [SERVICE_mDNS] Starting the network discovery thread
2024-04-28T00:16:31.202Z [NET_DISCOVERY] Found WLED at <IP_ADDRESS>:80 (wled-31dd4a.local)
2024-04-28T00:16:31.204Z [NET_DISCOVERY] Found WLED at <IP_ADDRESS>:80 (wled-esszimmer.local)
2024-04-28T00:16:31.215Z [WEBSOCKET] (WebSocketClient.cpp:39) New connection from ::ffff:<IP_ADDRESS>
2024-04-28T00:16:31.231Z [NET_DISCOVERY] Found WLED at <IP_ADDRESS>:80 (wled-638aee.local)
2024-04-28T00:16:33.017Z [EFFECT0(Rainbo...)] The effect quits with priority: 0
2024-04-28T00:16:33.017Z [MUXER0] Removed source priority 0
2024-04-28T00:16:33.017Z [MUXER0] Set visible priority to 245
2024-04-28T00:16:33.017Z [SMOOTHING0] Clearing queued colors before: enabling. Smoothing configuration changed: restarting timer.
2024-04-28T00:16:33.017Z [SMOOTHING0] Smoothing queue is cleared
2024-04-28T00:16:33.017Z [SMOOTHING0] Selecting config (0) => type: Alternative, directMode: false, pause: false, settlingTime: 150ms, interval: 20ms (50Hz), antiFlickTres: 32, antiFlickStep: 2, antiFlickTime: 250
2024-04-28T00:16:33.017Z [SMOOTHING0] Using alternative smoothing input (0)
2024-04-28T00:16:33.017Z [HYPERHDR0] New priority[245], previous [0]
2024-04-28T00:16:33.017Z [COLORSPACE_CALIBRATION0] (ColorSpaceCalibration.cpp:225) setBackLightEnabled: 1
2024-04-28T00:16:33.017Z [EFFECTENGINE0] Effect 'Rainbow swirl fast' has finished.
2024-04-28T00:16:33.017Z [LEDDEVICE0_WLED] (LedDevice.cpp:381) Stopping refresh timer
2024-04-28T00:16:33.017Z [EFFECT0(Rainbo...)] Effect named: 'Rainbow swirl fast' is deleted
2024-04-28T00:16:33.017Z [LEDDEVICE0_WLED] Ignoring user LED refresh rate. Forcing smoothing refresh rate = 50.00 Hz
2024-04-28T00:16:33.017Z [LEDDEVICE0_WLED] (LedDevice.cpp:423) Refresh rate = 50.00 Hz
2024-04-28T00:16:33.017Z [LEDDEVICE0_WLED] (LedDevice.cpp:367) Starting timer with interval = 20ms
2024-04-28T00:16:33.017Z [LEDDEVICE0_WLED] (LedDevice.cpp:433) Refresh interval updated to 20ms
2024-04-28T00:16:33.017Z [LEDDEVICE0_WLED] (LedDevice.cpp:448) LED refresh interval adjustment caused by smoothing configuration change to 20ms (proposed: 20ms)
2024-04-28T00:16:33.057Z [SMOOTHING0] Using alternative smoothing procedure (0)
2024-04-28T00:16:33.145Z [IMAGETOLED_MNG0] Total index number is: 8704 (memory: 8704). User sparse processing is: disabled, image size: 512 x 216, area number: 105
2024-04-28T00:18:29.871Z [PERFORMANCE] [INSTANCE0: FPS = 4.15, processed = 249], [LED0: FPS = 39.02, send = 2341, processed = 2341, dropped = 657]
2024-04-28T00:19:29.678Z [PERFORMANCE] [INSTANCE0: FPS = 4.15, processed = 248], [LED0: FPS = 39.45, send = 2368, processed = 2368, dropped = 632]
2024-04-28T00:20:29.718Z [PERFORMANCE] [INSTANCE0: FPS = 4.15, processed = 249], [LED0: FPS = 38.91, send = 2336, processed = 2336, dropped = 665]

Steps to reproduce
Video Capturing -> Software Screen Capture -> Device
I get only /dev/fb0 instead of my display devices, allthough three monitors are connected.

Perhaps this log line is related to the problem?!
2024-04-28T00:16:29.678Z [DAEMON] The system doesn't support the X11 system grabber
But I'm definitly using X11 (not Wayland).

What is expected?
The possibility to select a display as video capturing device.
What is actually happening?
A display device (/dev/fb0) is automatically selected, which only results in a black screen.

System

HyperHDR Server:
- Build:           (HEAD detached at v<IP_ADDRESS>) (awawa-dev-9e960c9/52f12b4-1711236142)
- Build time:      Mar 23 2024 23:33:19
- Git Remote:      https://github.com/awawa-dev/HyperHDR
- Version:         <IP_ADDRESS>
- UI Lang:         en (BrowserLang: en-US)
- Avail Capt:      Linux (V4L2)
- Database:        read/write

HyperHDR Server OS:
- Distribution:   Linux Mint 21.3
- Architecture:   x86_64
- CPU Model:      AMD Ryzen 5 3600 6-Core Processor
- CPU Type:       113
- Kernel:         linux (5.15.0-76-generic (WS: 64))
- Qt Version:     5.15.3
- Browser:        Mozilla/5.0 (X11; Linux x86_64; rv:124.0) Gecko/20100101 Firefox/124.0

Let me know, if I need to provide more infos.
Thanks!
Nico

Hi
Are you sure you start HyperHDR as an application from the menu and you have the icon in the systray? Not as a service/Daemon?

Hi,
ahh, that was easy...
Since I saw the message "Start the service: sudo systemctl start" after the installation, I somehow expected that it had to be executed as a systemd service. I completely overlooked the "Troubleshooting? Run HyperHDR manually" part.

I now have one option, which is Display nr: 0, and also this helpful explanation box above, that tells me exactly what I did wrong when I had it running as a service. This was exactly the message I needed before, but it wasn't shown to me, as you can see in the screenshot above.

Display nr: 0 seems to be a combination of all three displays.

Is there any way to select only display 3?
Thanks,
Nico

I'm afraid that's not possible, at least for now. Under Ubuntu/Gnome, the x11 session capturing also works with the Pipewire grabber (the same one used by default for Wayland sessions), and you can probably select a specific monitor there because the Portal dialog for Pipewire offers advanced options. But it looks like Pipewire x11 capturing is not implemented here.
A workaround is to use the crop function, thanks to which you can cut out the area you are interested in. But of course this is not an optimal solution.

These experiments seem a bit too complicated for me as a non-Linux power user. :-)
Anyway, thanks for your detailed explanation! Really appreciate that!
So far, HyperHDR is now running great and it's already an improvement. Coming from Hyperion having now and then problems, especially in the combination with Home Assistant turning the WLED strip on and off. But this seems to work flawless now.
Will now also do the switch on my HTPC.
Perhaps you could modify the "Explanation" message from the screenshot above, so that it is also displayed when starting HyperHDR as a service. Just to prevent others from making the same mistake I did.
Will close the issue, thanks again for the tool and the support!


Mmap checking

I've been experiencing SIGSEGVs from moloch-capture (cores available),
[671207.091225] moloch-pkt4[9502]: segfault at ffffffffffffffff ip 00007fcf38739c99 sp 00007fcf25dfc4a8 error 7 in libc-2.23.so[7fcf3869b000+1bf000]
(gdb) bt
#0  __memcpy_sse2_unaligned () at ../sysdeps/x86_64/multiarch/memcpy-sse2-unaligned.S:37
#1  0x0000000000441ddf in memcpy (__len=<optimized out>, __src=<optimized out>, __dest=<optimized out>)
    at /usr/include/x86_64-linux-gnu/bits/string3.h:53
#2  writer_simple_process_buf (thread=thread@entry=3, closing=closing@entry=0) at writer-simple.c:153
#3  0x0000000000442119 in writer_simple_write (session=<optimized out>, packet=0x7fb075279ea0) at writer-simple.c:270
#4  0x0000000000447865 in moloch_packet_thread (threadp=0x3) at packet.c:610
(gdb) l
148         MolochSimple_t *info = currentInfo[thread];
149
150         info->closing = closing;
151         if (!closing) {
152             currentInfo[thread] = writer_simple_alloc(thread, info);
153             memcpy(currentInfo[thread]->buf, info->buf + config.pcapWriteSize, info->bufpos - config.pcapWriteSize);
154             currentInfo[thread]->bufpos = info->bufpos - config.pcapWriteSize;
155         } else {
156             currentInfo[thread] = NULL;
157         }
(gdb) p *currentInfo[thread]
$4 = {simple_next = 0x0, simple_prev = 0x0, buf = 0xffffffffffffffff <error: Cannot access memory at address 0xffffffffffffffff>,
  file = 0x7fb064006000, bufpos = 0, closing = 0 '\000', thread = 3 '\003'}

The mmap'd buf element is -1, which mmap returns on failure.
This'll help diagnose what the problem is (which error case).  Hopefully, it'll be followed by more helpful cleanup.

thanks


Update attack-surface-reduction-rules-reference.md

Added supported OS (newer versions)
Fixes https://github.com/MicrosoftDocs/microsoft-365-docs/issues/12581

@JohanFreelancer9 Please copy review


Add gemstash --version

Also supports gemstash version and gemstash -v

Hmmm, I just realized, I need to add documentation for this.

:+1:
Add the docs later, but it is quite self explanatory

:+1:


OCPTOOLS-468: mitigate CVE-2024-43044

Changes

bump to Jenkins 2.462.1 to mitigate CVE-2024-43044
Arbitrary file read vulnerability through agent connections can lead to RCE

Signed-off-by: Avinal Kumar<EMAIL_ADDRESS>

/lgtm

/backport-risk-assessed
/cherry-pick-approved

/label backport-risk-assessed
/label cherry-pick-approved

Fixes #1827.


[idea] Email people when someone mentions a similar idea on their website

It could be interesting to notify people if we find a similar idea as they wrote about on some other website. We could use the existing vector search for this every time a post change is detected during a re-scrape.

I'm pretty sure that lots of people would love this idea.
For me, though, I'd prefer to consume an RSS feed that I can plumb into my existing workflow. It could be only me who wants that. I'm unusual in that I rather hate email.


Add ec2:DescribeInstanceTypes to NodePoolManagement role

What this PR does / why we need it:
new permission ec2:DescribeInstanceTypes is needed by the clusters-api-provider-aws  v.2.0.x
Rebase https://github.com/openshift/cluster-api-provider-aws/pull/464 currently failing with:
E0614 10:58:04.415811       1 controller.go:329] "Reconciler error" err=<
	failed to create AWSMachine instance: UnauthorizedOperation: You are not authorized to perform this operation.
		status code: 403, request id: b99a7e00-bf8a-4b67-8711-04cdabcd032f
 > controller="awsmachine" controllerGroup="infrastructure.cluster.x-k8s.io" controllerKind="AWSMachine" AWSMachine="e2e-clusters-drzvj-example-9d445/example-9d445-us-east-1b-lhs5m" namespace="e2e-clusters-drzvj-example-9d445" name="example-9d445-us-east-1b-lhs5m"

Which issue(s) this PR fixes (optional, use fixes #<issue_number>(, fixes #<issue_number>, ...) format, where issue_number might be a GitHub issue, or a Jira story:
Fixes #
Checklist

[ ] Subject and description added to both, commit and PR.
[ ] Relevant issues have been referenced.
[ ] This change includes docs.
[ ] This change includes unit tests.

/hold
I consider requiring wider perms for the same API intent a bug, let's fix via https://github.com/kubernetes-sigs/cluster-api-provider-aws/issues/4342
However we will want to enable this for IMDSv2 @jharrington22 @fahlmant nevertheless

@muraee figured we are actually missing a ec2:DescribeInstanceTypes permission introduced here: https://github.com/kubernetes-sigs/cluster-api-provider-aws/pull/4054 I remain on the opinion that requiring wider perms when the API intent didn't change is bug so let's try to fix upstream before widen up our IAM unnecessarily.

@enxebre Will this permission be needed in 4.12 or 4.13? Or is it still a ways out (4.14)? @muraee Do you know under what conditions this permission would be used? We'll need to test this permission on a live cluster in order to properly condition the AWS permission, so any help in figuring out how to trigger this would be quite helpful.

@fahlmant can we work towards having 4.14 to generate the ModifyInstanceMetadataOptions permission?

/hold cancel

@muraee can you please update the PR to cover the DescribeInstanceTypes perms?

Yes 4.14 works

Updated PR to add ec2:DescribeInstanceTypes permission instead.

/retest-required

/hold


Fix data table download for components with server-side filtering

For the 21.06 release, we applied a fix so that users could download the Known Drugs table in JSON, CSV, or TSV formats. See https://github.com/opentargets/platform-app/pull/402.
For 21.09, can we please make an additional adjustment to tables with server-side filtering?
If a user enters a search term to filter the table and then proceeds to download the table, they download the entire table rather than the subset they have filtered.
For example, if I visit the non-small cell lung carcinoma profile page and filter the Known Drugs table for "pembrolizumab", I see a table with 20 rows. But when I download the table in CSV, TSV, or JSON formats, I download the entire table with all 5319 rows, rather than a file with just 20 rows.

Can we please adjust the logic so that the download file contains the subset of entries if a user has searched/filtered the table?
This behaviour is the same behaviour that we currently have in the associations data table and the ClinVar evidence table. For example, if a user visits the RAF1 and Noonan Syndrome evidence page and searches the ClinVar data table for "3 prime utr variant" and downloads the table, the file will only contain 26 rows.

Ticket closed as work completed and merged into main


Hide submitters dropbox in motion create if cannot see users

Resolve #4457
Needs:
https://github.com/OpenSlides/openslides-backend/issues/2760

Updated, new helptext of the meeting setting and changed the visibility of the submitters label.


Nomination Report Auto-Send Destination Change

The auto-send of the nomination report should go to<EMAIL_ADDRESS>not<EMAIL_ADDRESS>please!
Not finding it in the code or I'd have sent a pull request to change it myself, sorry.
Thanks!

It is in the file convention.py under the folder glasgow2024

@katjonzz did you solve this? The report destination address is set in the admin interface.
The address that Henry mentions is part of convention configuration, and is the one that we direct members to email when there are issues with their Hugo ballot.

Nope! When I click the link you posted for the admin interface here in this ticket, I get this:
Permission Denied
Your user, clyde-0125, does not have access to this feature.


100% coverage badge gets cut off in flat style

example https://cover.run/go?repo=github.com%2Fockam-network%2Fdid&tag=golang-1.10

flat-square looks ok.


"warning: logic (~) is still running after 60s, forget to call done()?" keeps called

Hello. I am getting trouble after I updated redux-logic to latest version.
I read all document and it said single-dispatch option is gone. So I added done() to all my logics.
But I still get error message like below. I revised some source code to print 'action' in the last of error message due to insufficient information about logic name.
console.error: "warning: logic (L(/^(?!http\/).*/)-77) is still  running after 60s, forget to call done()? For non-ending logic, set  warnTimeout: 0", {"type": "GET_LOCATION"}
and below is my logic dealing with GET_LOCATION action.
`const getLocationLogic = createLogic({
type: 'GET_AUTH_LOCATION',
latest: true,
process: async ({getState, action}, dispatch, done) => {
    axios.get('http://freegeoip.net/json/')
     .then(res => {
        dispatch(actionCreators.setLocation({
           ...someData here
        }));
     })
     .catch(e => {
        console.log(e);
     })
     .then(() => done());
}});`

What did I do wrong? It seems quite same as sample code of README.md.
I am using RN 0.48.3 and Redux-logic 0.12.3.
I appreciate in advance.

That looks correct. Does this happen consistently or just from time to time? Is is possible that the server doesn't respond within 60 seconds? (or maybe the browser is busy fetching other things and it gets queued and delayed?)

@jeffbski I think that's not the case.
It happens every time I run the app and the response comes every time too.
I've checked by logging 'res'.
That logic runs right after the start up, and the warning message comes up far later.

I'm not an expert with async/await, but what is the purpose of your async clause in this case, as you are directly using promise .then()/.catch() directly?  Is it possible that this is causing some anomaly?
I would suggest adding a log in the finally .then() clause to insure done() is being called.

@kevinAst oh that was a mistake.. Actually I created a logic using async/await at first and the warning message comes up. So I changed it to use promise to make it same as README.
I forgot to delete async keyword then.
I'll give a try without async and come again.

@KevinAst Unfortunately, it does not work. I think async keyword is not a problem. :(

@jh5696 I would still place some "temporary" diagnostic logs in each of the async callbacks.  Of particular interest is to confirm that last "finally" then() clause is executing the done().  This might shed some light on what is actually happening.  If the done() is being executed, then it points to some issue in redux-logic.  If the done() is NOT being executed, then there is some issue with the promise.

@KevinAst Thanks for your reply. I tested with some logs and all seems good, but warning message still comes up.
I confirmed that

the response is definitely arrived within 60s(actually within a sec)
done function is called in final then() clause after a dispatch

It's so strange. I added done after the promise but warning message still exists.
process: async ({getState, action}, dispatch, done) => {
    axios.get('http://freegeoip.net/json/')
     .then(res => {
        dispatch(actionCreators.setLocation({
           ...someData here
        }));
     })
     .catch(e => {
        console.log(e);
     })
     .then(() => done());
   done(); //I added this to test
};

done is called regardless of then/catch clauses. So It 'should' be and is actually called within 60s..
Dipatching setLocation is not executed as expected.

Thanks for looking into this. I'll go back and take a close look at the warning code and maybe it is a defect in redux-logic that the warning is not getting cancelled when it should be.

I haven't found anything wrong in the redux-logic code yet, but will still keep looking.
I did want to mention that this warning is only turned on in the development build. So once you are building and deploying in production (with NODE_ENV === 'production') then this warning is disabled.
I'll keep looking to see if I can figure out what might be going on.

@jeffbski I can reproduce this same issue FYI. One of our logic methods was failing and never actually executing done (even though it should be going to next .then(done) )

### our logic.js file
createLogic({
    type: $requestPublicTokenData,
    process({ getState, action }, dispatch, done) {
      graphQueryPublic(publicTokenDataQuery, { user: action.user }).then(res => {
        dispatch(receivePublicTokenData(res.publicAddresses, res.publicKnownContracts))
      }).then(() => {
        console.log('y u no call done?!?')
        done()
      })
    }
  }),
  createLogic({
    type: $requestPublicChartOfAccountsWithBalances,
    process({ getState, action }, dispatch, done) {
      graphQueryPublic(publicChartOfAccountsWithBalancesQuery, { user: action.user }).then(res => {
        debugger
        dispatch(receivePublicChartOfAccountsWithBalances(res.publicChartOfAccounts))
      }).then(done)
    }
  }

graphQueryPublic.js looks like this:
export default (query, _variables) => {
  const variables = Object.assign({}, _variables)
  return axios.post(graphqlUrl, {
    query,
    variables
  }).then(res => {
    if (res.data.errors) {
      // just got annoying so I created a filter for it
      console.log('graphql error:', JSON.stringify(res.data.errors))
      return res.data
    } else {
      return res.data.data
    }
  }).catch((err) => console.log('axios error:', err))
}

Thanks @james89 that should help me figure out what is going on. I'm thinking about maybe disabling the warning by default since maybe there are some edge cases where it is warning when it shouldn't and just causing more confusion than it is helping.

Running into this too, and I'm definitely calling done.
Here's my test code.
    createLogic({
        type: ActionType.CLICK_COUNT_UPDATED,
        async process({ getState }, dispatch, done) {
            const state: any = getState();
            const count: number = state.dashboard.count;
            console.log(`LOGIC ${ActionType.CLICK_COUNT_UPDATED}: count was incremented to ${count}`);

            console.debug(await InternalHttpClient.execute(RxPublicApiEndpoint.publicTest));

            done();
        }
    })

This is just senseless test code, but I did call done.
Still says warning: logic (L(CLICK_COUNT_UPDATED)-0) is still running after 60s, forget to call done()? For non-ending logic, set warnTimeout: 0
Did anyone end up figuring this out?

So, I had the same warning. I don't know if this will help anyone but, if in your Logic block you are running a transform and a process and the process is just passing along the data. You can just remove the pointless process and the error goes away.
const setPathLogic = createLogic({
  type: actionNames.setPath,
  transform({ action, getState }: IDependencies, next) {
    const { path } = action as Actions.ISetPathAction;
    const state = getState();
    const storeId = storeDetailsSelectors.storeIdSelector(state);

    const cleanPath = path..replace(':id', `${ storeId }`);

    next({ ...action, path: cleanPath });
  },
  process({ action }, dispatch, done: VoidFunction) {
    const { path } = action as Actions.ISetPathAction;
    return { ...action, path };
    done();
  },
});

Will just become this:
const setPathLogic = createLogic({
  type: actionNames.setPath,
  transform({ action, getState }: IDependencies, next) {
    const { path } = action as Actions.ISetPathAction;
    const state = getState();
    const storeId = storeDetailsSelectors.storeIdSelector(state);

    const cleanPath = path..replace(':id', `${ storeId }`);

    next({ ...action, path: cleanPath });
  },
});

And there you go the error stops occurring.

I'm getting this error. I'm using transform, not process, and this error occurs.
Why does this error message exist? Isn't async/await all we need?

If you don't call a return you need a done. I'm pretty sure that is the rule.
Get Outlook for Androidhttps://aka.ms/ghei36

From: Rob Christian<EMAIL_ADDRESS>Sent: Friday, April 26, 2019 11:21:46 PM
To: jeffbski/redux-logic
Cc: Charles Givens; Comment
Subject: Re: [jeffbski/redux-logic] "warning: logic (~) is still running after 60s, forget to call done()?" keeps called (#81)
I'm getting this error. I'm using transform, not process, and this error occurs.
Why does this error message exist? Isn't async/await all we need?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_jeffbski_redux-2Dlogic_issues_81-23issuecomment-2D487250456&d=DwMCaQ&c=AT5K1q9CQPGMXclhopmqpg&r=rzqzrW2bZcenS3C0KeZ4yAal9aY7so6DbhE_7ApOmAQ&m=mO_MNPqsplx2zDvWK-ZMmGHMwMy4MqVeCOOuBfXSdtY&s=gtpoqArFNwtAGO3GHGgUW_FN73nRyXrikzIKsRltYD0&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AJ2JHVIMFXYG26A6WLXUFYDPSPBEVANCNFSM4D5UACKQ&d=DwMCaQ&c=AT5K1q9CQPGMXclhopmqpg&r=rzqzrW2bZcenS3C0KeZ4yAal9aY7so6DbhE_7ApOmAQ&m=mO_MNPqsplx2zDvWK-ZMmGHMwMy4MqVeCOOuBfXSdtY&s=y5j_R7gUvMRvrl-j891bIfUeJGo7DWer5US0IXnsrpI&e=.
CONFIDENTIALITY NOTICE:
This message contains information which may be confidential or privileged. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of the contents of this information is prohibited. If you have received this transmission in error, please notify me immediately by telephone.


Add tril_indices and tril_indices_from API.

Hi, Team!
The PR aims to add cupy.tril_indices and cupy.tril_indices_from API. It follows #6078.
Looking forward to your viewpoints :)
Thanks!
cc: @kmaehashi @takagi @leofang @asi1024  @toslunar

Hmm, the CI didn't pass since these functions are experimental. Any tips to resolve this for CuPy?
Thanks!

/test mini

Thanks for reviewing, @takagi! :smile:

Thanks!


Gym picks up provisioning profile from PROVISIONING_PROFILE environment variable

New Issue Checklist

[x] Updated fastlane to the latest version
[x] I read the Contribution Guidelines
[x] I read docs.fastlane.tools
[x] I searched for existing GitHub issues

Issue Description
I've debugged this for quite a while. Gym kept setting a provisioning profile on my frameworks and failed the build because of that. The reason turned out that gym picks up a provisioning profile from PROVISIONING_PROFILE and applies that to all bundles that do not have an explicit mapping in export options.
As far as I can tell this is undocumented behavior as per the gym documentation. The feature itself is useful but if you don't know it exists and use the variable name for a different script in the same CI build this is pretty surprising.
Command executed
fastlane gym
Complete output when running fastlane, including the stack trace and command used
If you need I can provide it
Fastlane Env
✅ fastlane environment ✅
Stack

Key
Value

OS
11.5.2

Ruby
3.0.2

Bundler?
false

Git
git version 2.30.1 (Apple Git-130)

Installation Source
/usr/local/Cellar/fastlane/2.191.0/libexec/bin/fastlane

Host
macOS 11.5.2 (20G95)

Ruby Lib Dir
/usr/local/Cellar/ruby/3.0.2/lib

OpenSSL Version
OpenSSL 1.1.1k  25 Mar 2021

Is contained
false

Is homebrew
true

Is installed via Fabric.app
false

Xcode Path
/Applications/Xcode.app/Contents/Developer/

Xcode Version
12.5.1

Swift Version
5.4.2

System Locale

Variable
Value

LANG
en_US.UTF-8
✅

LC_ALL
en_US.UTF-8
✅

LANGUAGE

fastlane files:
No Fastfile found
No Appfile found
fastlane gems

Gem
Version
Update-Status

fastlane
2.191.0
✅ Up-To-Date

Loaded fastlane plugins:
No plugins Loaded
Loaded gems

Gem
Version

did_you_mean
1.5.0

atomos
0.1.3

CFPropertyList
3.0.3

claide
1.0.3

colored2
3.1.2

nanaimo
0.3.0

rexml
3.2.5

xcodeproj
1.20.0

rouge
2.0.7

xcpretty
0.3.0

terminal-notifier
2.0.0

unicode-display_width
1.7.0

terminal-table
1.8.0

plist
3.6.0

public_suffix
4.0.6

addressable
2.8.0

multipart-post
2.0.0

word_wrap
1.0.0

tty-screen
0.8.1

tty-cursor
0.7.1

tty-spinner
0.9.3

artifactory
3.0.15

babosa
1.0.4

colored
1.2

highline
2.0.3

commander
4.6.0

excon
0.85.0

faraday-em_http
1.0.0

faraday-em_synchrony
1.0.0

faraday-excon
1.1.0

faraday-httpclient
1.0.1

faraday-net_http
1.0.1

faraday-net_http_persistent
1.2.0

faraday-patron
1.0.0

faraday-rack
1.0.0

ruby2_keywords
0.0.5

faraday
1.6.0

unf_ext
<IP_ADDRESS>

unf
0.1.4

domain_name
0.5.20190701

http-cookie
1.0.4

faraday-cookie_jar
0.0.7

faraday_middleware
1.1.0

fastimage
2.2.4

gh_inspector
1.1.3

json
2.5.1

mini_magick
4.11.0

naturally
2.2.1

rubyzip
2.3.2

security
0.1.3

xcpretty-travis-formatter
1.0.1

dotenv
2.7.6

bundler
2.2.22

simctl
1.6.8

jwt
2.2.3

uber
0.1.0

declarative
0.0.20

trailblazer-option
0.1.1

representable
3.1.1

retriable
3.1.2

mini_mime
1.1.0

memoist
0.16.2

multi_json
1.15.0

os
1.1.1

signet
0.15.0

googleauth
0.17.0

httpclient
2.8.3

webrick
1.7.0

google-apis-core
0.4.1

google-apis-playcustomapp_v1
0.5.0

google-apis-androidpublisher_v3
0.10.0

google-cloud-env
1.5.0

google-cloud-errors
1.1.0

google-cloud-core
1.6.0

google-apis-iamcredentials_v1
0.6.0

google-apis-storage_v1
0.6.0

rake
13.0.6

digest-crc
0.6.4

google-cloud-storage
1.34.1

emoji_regex
3.2.2

jmespath
1.4.0

aws-partitions
1.484.0

aws-eventstream
1.1.1

aws-sigv4
1.2.4

aws-sdk-core
3.119.0

aws-sdk-kms
1.46.0

aws-sdk-s3
1.98.0

tsort
0.1.0

uri
0.10.1

set
1.0.1

forwardable
1.3.2

logger
1.4.3

pathname
0.1.0

shellwords
0.1.0

cgi
0.2.0

date
3.1.0

timeout
0.1.1

stringio
3.0.0

openssl
2.2.0

io-nonblock
0.1.0

ipaddr
1.2.2

io-wait
0.1.0

zlib
1.1.0

resolv
0.2.0

securerandom
0.1.0

digest
3.0.0

time
0.1.0

open-uri
0.1.0

mutex_m
0.1.1

net-protocol
0.1.0

ostruct
0.3.1

english
0.7.1

erb
2.2.0

strscan
3.0.0

optparse
0.1.0

abbrev
0.1.0

io-console
0.5.7

tempfile
0.1.1

delegate
0.2.0

fileutils
1.5.0

tmpdir
0.1.2

base64
0.1.0

singleton
0.1.1

net-http
0.1.1

open3
0.1.1

nkf
0.1.0

prettyprint
0.1.0

pp
0.1.0

find
0.1.0

yaml
0.1.1

psych
3.3.0

Are you able to archive app from Xcode?
How is signing set up in your framework targets? Is Automatically manage signing turned off, Team set to none?

Yes archiving from xcode works, archiving from fastlane also works fine as long as I don't set the PROVISIONING_PROFILE environment variable.
Signing in framework targets is not set up at all I think because they don't need to be signed

Could you upload logs / Fastfile - at least the parts that are relevant for the issue?

Still an issue

still an issue


Running a Reliably Agent in AWS

Background
The reason behind this write-up is to document how to run a Reliably Agent in AWS so that we can pull data from AWS & push indicators to Reliably. This provides users with the means of running an agent continuously/periodically/or spontaneously as a container in AWS, allowing them to almost 'forget' the agent is running, giving them constant indicators into their Reliably network.
This write-up is the finalisation of this user story: https://github.com/reliablyhq/roadmap/issues/120
I've put it in an issue as I'm not 100% sure where we're going with documentation, so putting it here should mean moving it is relatively easy.
Pre-requisites
Writing this guide/run-through, I am using prod and Reliably CLI v0.33.0.
I have created entities in my Reliably network with:
$ reliably entity populate aws

This created entities & 6 objectives based on the resources in the aws-playground-example stack, currently deployed in eu-west-2.

If there is no instance of the aws-playground-<identifier> stack deployed in the region you're interested in/at all - you can go to here: https://github.com/reliablyhq/reliably-aws-playground and follow the instructions on how to deploy one.
Running an agent
Because the playground stack generates fake traffic, the agent should immediately be able to generate indicators for you.
There's a few options on how you can run a container in AWS:

ECS with Fargate (The option I chose for my write-up)
ECS with EC2
Batch Jobs (if you're doing something on a wide time period/just want to run the agent for one run)
An EC2 instance running a docker container
An AWS Lambda function using a docker image (this one isn't supported by us out of the box with the CLI image, as you need to provide a Lambda Handler API compatible wrapper around the CLI command)

Using Fargate
I'll link a repository containing a CDK implementation of this solution below, but for now, I'll do a high level overview, incase you're using Terraform or the console to deploy your agent.
To be able to run the Agent in AWS, your Fargate task (or indeed, anything that is running the container) needs:

Access to the outside internet
IAM Permissions to perform the following actions:

cloudwatch:GetMetricData on * resources
apigateway:GET on * resources (IAM didn't appear to enable me to go any more atomic with the actions)
apigatewayv2:GET on * resources (IAM didn't appear to enable me to go any more atomic with the actions)

Once you have these access requirements met, you should be able provide your task with a container which uses:

Image: ghcr.io/reliablyhq/cli/cli:0.33.0
Command: "slo", "agent", "-V" (You can omit -V if you don't want the verbose logging)
Environment vars: RELIABLY_ORG: <your-reliably-org> & RELIABLY_TOKEN: <your-reliably-token>

The repo that deploys a Fargate task as the agent is here: https://github.com/reliablyhq/reliably-aws-agent-example
Fargate logs
If you've managed to deploy the task container correctly, the logs look something like:

Checking your output
If you've managed to run the container with the correct access to the internet and IAM permissions, you should be able to run:
$ reliably slo report -w

and see the results of those indicators the agent has posted:

Summary
If you're comfortable with AWS & containers, spinning up a Reliably Agent is pretty straightforward, no fancy entrypoint or values provided. This agent will run indefinitely until you tear it down. You can also add "-i", "<number-of-seconds>" to the command, to determine the period that indicators are generated, by default, this is 5 minutes.

I should make a note that I didn't have to provide the agent any AWS credentials, this is because the running container itself gets given an IAM identity which it assumes. For situations like an EC2 image running a Docker container, you would need to provide the AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_REGION environment variables to the running container.

A big win with Fargate, over most 'give us a container to run' services, is that it can run non-http services. We are just looking to run a container which runs a process, it doesn't respond to any requests. Fargate allows us to run an image without specifying ports to serve/listen to.


Parameterize git clone depth and fix disabling submodules

Changes

Adds a new depth param to git resources to control the --depth flag to git fetch. When set to '0', shallow fetches are disabled entirely. #1450
Fixes the submodule param so that it can be meaningfully set to false. #1727

Submitter Checklist

[x] Includes tests (if functionality changed/added)
[x] Includes docs (if user facing)
[x] Commit messages follow commit message best practices

Reviewer Notes
If API changes are included, additive changes must be approved by at least two OWNERS and backwards incompatible changes must be approved by more than 50% of the OWNERS, and they must first be added in a backwards compatible way.
Release Notes
### Features

- The clone depth of `git` resources can now be set with a new `depth` parameter. Shallow clones can be disabled entirely by setting depth to `'0'`. #1450

### Fixes

- Initializing git submodules can now be turned off by setting the `submodules` param to `'false'`. #1727

/ok-to-test

Can I get some 👀 on the CI output? I think all the failures I’m seeing are spurious.

Yeah most of these are definitely unrelated. The unit test failures should be fixed by #1709 I think (though I thought #1731 would fix it...), and the link checker failure is just annoyingly flaky today.
However, there's a relevant error helpfully tucked deeeep within the integration test logs:
I1211 18:21:28.622] 2019/12/11 18:21:28 Unexpected error running "go build": exit status 2
I1211 18:21:28.623] # github.com/tektoncd/pipeline/pkg/git
I1211 18:21:28.624] pkg/git/git.go:80:33: cannot use spec.Depth (type uint) as type string in append

Which comes from here: https://github.com/tektoncd/pipeline/pull/1729/files#diff-4de8c4734d5ffab0d7fd8835bf8b3421R80
If you fix that we should be able to check that e2e tests aren't broken, then we'll just be waiting for unit tests and link-checker to start behaving.
Sorry for the noise, and thanks for sticking with it :muscle:

/retest

/test pull-tekton-pipeline-build-tests

🎉 Other than the link check failure, tests are now passing 😁

/test pull-tekton-pipeline-build-tests

/test pull-tekton-pipeline-build-tests

/test pull-tekton-pipeline-build-tests

/lgtm
🎉

Hi @silverlyra, we have a pipeline that relies on the entire repo to be cloned. With tekton 0.9 we therefore added a step git fetch --unshallow as it cloned by default with (I think) depth = 1.
After upgrading to 0.10.1 (and without specifying any value for depth) this fails now with fatal: --unshallow on a complete repository does not make sense. It looks like the default behavior is now a full clone. Is that intentional or does it make sense to change the default behavior back to 1?
Thanks, Fabian