cache_size_limit reached

Hello, brother. I am running this https://github.com/mobiusml/hqq/blob/master/examples/backends/hqq_lib_demo.py on A100-40GB, and I get the following error. What should I do?
code
The code is the code in the link. Adding the following two lines should not affect it:
import os
os.environ["TOKENIZERS_PARALLELISM"] = "false
result
root@0d2c83196670:/workspace# python example2_4.py
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00,  3.02it/s]
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 130/130 [00:00<00:00, 332.59it/s]
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 225/225 [00:19<00:00, 11.38it/s]
Warning: failed to import the Marlin backend. Check if marlin is correctly installed if you want to use the Marlin backend (https://github.com/IST-DASLab/marlin).
Warning: failed to import the BitBlas backend. Check if BitBlas is correctly installed if you want to use the bitblas backend (https://github.com/microsoft/BitBLAS).
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 225/225 [00:01<00:00, 116.87it/s]
0%|                                                                                | 0/5 [00:00<?, ?it/s]No chat template is set for this tokenizer, falling back to a default class-level template. This is very error-prone, because models are often trained with templates different from the class default! Default chat templates are a legacy feature and will be removed in Transformers v4.43, at which point any code depending on them will stop working. We recommend setting a valid chat template before then to ensure that this model continues working without issues.
/usr/lib/python3.8/contextlib.py:83: FutureWarning: torch.backends.cuda.sdp_kernel() is deprecated. In the future, this context manager will be removed. Please see torch.nn.attention.sdpa_kernel() for the new context manager, with updated signature.
self.gen = func(*args, **kwds)
/usr/lib/python3.8/contextlib.py:83: FutureWarning: torch.backends.cuda.sdp_kernel() is deprecated. In the future, this context manager will be removed. Please see torch.nn.attention.sdpa_kernel() for the new context manager, with updated signature.
self.gen = func(*args, **kwds)
/usr/lib/python3.8/contextlib.py:83: FutureWarning: torch.backends.cuda.sdp_kernel() is deprecated. In the future, this context manager will be removed. Please see torch.nn.attention.sdpa_kernel() for the new context manager, with updated signature.
self.gen = func(*args, **kwds)
/usr/lib/python3.8/contextlib.py:83: FutureWarning: torch.backends.cuda.sdp_kernel() is deprecated. In the future, this context manager will be removed. Please see torch.nn.attention.sdpa_kernel() for the new context manager, with updated signature.
self.gen = func(*args, **kwds)
/usr/lib/python3.8/contextlib.py:83: FutureWarning: torch.backends.cuda.sdp_kernel() is deprecated. In the future, this context manager will be removed. Please see torch.nn.attention.sdpa_kernel() for the new context manager, with updated signature.
self.gen = func(*args, **kwds)
/usr/lib/python3.8/contextlib.py:83: FutureWarning: torch.backends.cuda.sdp_kernel() is deprecated. In the future, this context manager will be removed. Please see torch.nn.attention.sdpa_kernel() for the new context manager, with updated signature.
self.gen = func(*args, **kwds)
/usr/lib/python3.8/contextlib.py:83: FutureWarning: torch.backends.cuda.sdp_kernel() is deprecated. In the future, this context manager will be removed. Please see torch.nn.attention.sdpa_kernel() for the new context manager, with updated signature.
self.gen = func(*args, **kwds)
/usr/lib/python3.8/contextlib.py:83: FutureWarning: torch.backends.cuda.sdp_kernel() is deprecated. In the future, this context manager will be removed. Please see torch.nn.attention.sdpa_kernel() for the new context manager, with updated signature.
self.gen = func(*args, **kwds)
/usr/lib/python3.8/contextlib.py:83: FutureWarning: torch.backends.cuda.sdp_kernel() is deprecated. In the future, this context manager will be removed. Please see torch.nn.attention.sdpa_kernel() for the new context manager, with updated signature.
self.gen = func(*args, **kwds)
W1031 00:31:42.775483<PHONE_NUMBER>80000 torch/_dynamo/convert_frame.py:762] [0/8] torch._dynamo hit config.cache_size_limit (8)
W1031 00:31:42.775483<PHONE_NUMBER>80000 torch/_dynamo/convert_frame.py:762] [0/8]    function: 'forward' (/usr/local/lib/python3.8/dist-packages/transformers/models/llama/modeling_llama.py:1126)
W1031 00:31:42.775483<PHONE_NUMBER>80000 torch/_dynamo/convert_frame.py:762] [0/8]    last reason: tensor 'L['input_ids']' stride mismatch at index 0. expected 19, actual 27
W1031 00:31:42.775483<PHONE_NUMBER>80000 torch/_dynamo/convert_frame.py:762] [0/8] To log all recompilation reasons, use TORCH_LOGS="recompiles".
W1031 00:31:42.775483<PHONE_NUMBER>80000 torch/_dynamo/convert_frame.py:762] [0/8] To diagnose recompilation issues, see https://pytorch.org/docs/main/torch.compiler_troubleshooting.html.
0%|                                                                                | 0/5 [01:57<?, ?it/s]
Traceback (most recent call last):
File "example2_4.py", line 44, in
patch_model_for_compiled_runtime(model, tokenizer, warmup=True)
File "/usr/local/lib/python3.8/dist-packages/hqq/utils/generation_hf.py", line 93, in patch_model_for_compiled_runtime
model.generate(
File "/usr/local/lib/python3.8/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/transformers/generation/utils.py", line 1914, in generate
result = self._sample(
File "/usr/local/lib/python3.8/dist-packages/transformers/generation/utils.py", line 2651, in _sample
outputs = self(
File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1562, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/hqq/utils/generation_hf.py", line 83, in custom_forward
out = out_fct(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/torch/_dynamo/eval_frame.py", line 433, in _fn
return fn(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/torch/_dynamo/convert_frame.py", line 1116, in call
return self._torchdynamo_orig_callable(
File "/usr/local/lib/python3.8/dist-packages/torch/_dynamo/convert_frame.py", line 472, in call
return _compile(
File "/usr/local/lib/python3.8/dist-packages/torch/_utils_internal.py", line 84, in wrapper_function
return StrobelightCompileTimeProfiler.profile_compile_time(
File "/usr/local/lib/python3.8/dist-packages/torch/_strobelight/compile_time_profiler.py", line 129, in profile_compile_time
return func(*args, **kwargs)
File "/usr/lib/python3.8/contextlib.py", line 75, in inner
return func(*args, **kwds)
File "/usr/local/lib/python3.8/dist-packages/torch/_dynamo/convert_frame.py", line 774, in _compile
unimplemented(f"{limit_type} reached")
File "/usr/local/lib/python3.8/dist-packages/torch/_dynamo/exc.py", line 221, in unimplemented
raise Unsupported(msg)
torch._dynamo.exc.Unsupported: cache_size_limit reached

When I remove these two lines from the code, there are no more errors reported.
from hqq.utils.generation_hf import patch_model_for_compiled_runtime
patch_model_for_compiled_runtime(model, tokenizer, warmup=True)

Hello, this is related to torch.compile, what version of pytorch are you running? You need at least 2.4.1, if not 2.5.0 or the nightly

I changed torch2.5, transformers=4.42.1, hqq=0.2.2, on A100-40GB, the previous problem did not occur, but the warmup process occupied too much memory, and I ran out of memory when I reached the third sentence. What should I do? @mobicham
code
pip install git+https://github.com/mobiusml/hqq.git;
pip install bitblas #to use the bitblas backend
OMP_NUM_THREADS=16 CUDA_VISIBLE_DEVICES=0 ipython3
########################################################################
import torch
import os
import torch._dynamo
from transformers import HqqConfig
torch._dynamo.config.cache_size_limit = 64  # 或者更大的值
os.environ["TORCH_LOGS"] = "recompiles"
os.environ["TOKENIZERS_PARALLELISM"] = "false"
device = 'cuda:0'
backend = 'torchao_int4' #"torchao_int4" (4-bit only) or "bitblas" (4-bit + 2-bit)
compute_dtype = torch.float16 if backend=="bitblas" else torch.bfloat16
cache_dir = '.'
model_id = './llama/llama2_hf'
########################################################################
#Load model
from transformers import AutoModelForCausalLM, AutoTokenizer
from hqq.models.hf.base import AutoHQQHFModel
from hqq.core.quantize import *
HQQLinear.set_backend(HQQBackend.PYTORCH)
quant_config = HqqConfig(nbits=4,group_size=64,axis=1)
#Load
tokenizer = AutoTokenizer.from_pretrained(model_id, cache_dir=cache_dir)
model = AutoModelForCausalLM.from_pretrained(model_id, cache_dir=cache_dir, torch_dtype=compute_dtype,quantization_config = quant_config,device_map=device,attn_implementation="sdpa")
#Quantize
#quant_config = BaseQuantizeConfig(nbits=4, group_size=64, axis=1)
#AutoHQQHFModel.quantize_model(model, quant_config=quant_config, compute_dtype=compute_dtype, device=device)
#HQQLinear.set_backend(HQQBackend.PYTORCH)
#Optimize
from hqq.utils.patching import prepare_for_inference
prepare_for_inference(model, backend=backend, verbose=True)
#Inference
########################################################################
#Using a custom generator
from hqq.utils.generation_hf import HFGenerator
gen = HFGenerator(model, tokenizer, max_new_tokens=1000, do_sample=True, compile="partial").warmup()
out = gen.generate("Write an essay about large language models.", print_tokens=True)
########################################################################
#Using HF model.generate()
from hqq.utils.generation_hf import patch_model_for_compiled_runtime
patch_model_for_compiled_runtime(model, tokenizer, warmup=True)
Prompt
system_prompt = None #"You are a helpful assistant."
prompt = "Write an essay about large language models."
messages = [] if(system_prompt is None) else [{"role": "system", "content": system_prompt}]
messages += [{"role": "user", "content": prompt},]
inputs = tokenizer([tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)],return_tensors="pt").to(model.device)
Generate
outputs = model.generate(**inputs, max_new_tokens=1000, cache_implementation="static", pad_token_id=tokenizer.pad_token_id)
#print(tokenizer.decode(outputs[0]))
result
root@306f48790b3a:/workspace# python example2_4.py
Warning: Quantized meta-data is deprecated and will be removed. It is not supported for quantized model serialization.
Loading checkpoint shards:   0%|                                                                                | 0/3 [00:00<?, ?it/s]Warning: Quantizing zeros/scales is deprecated. This setting will be ignored.
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:21<00:00,  7.00s/it]
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 225/225 [00:01<00:00, 128.16it/s]
get here 54
0%|                                                                                | 0/3 [00:00<?, ?it/s]Write an essay about large language models.
No chat template is set for this tokenizer, falling back to a default class-level template. This is very error-prone, because models are often trained with templates different from the class default! Default chat templates are a legacy feature and will be removed in Transformers v4.43, at which point any code depending on them will stop working. We recommend setting a valid chat template before then to ensure that this model continues working without issues.
CUDAGraph supports dynamic shapes by recording a new graph for each distinct input size. Recording too many CUDAGraphs may lead to extra overhead. We have observed 51 distinct sizes. Please consider the following options for better performance: a) padding inputs to a few fixed number of shapes; or b) set torch._inductor.config.triton.cudagraph_skip_dynamic_graphs=True. Set torch._inductor.config.triton.cudagraph_dynamic_shape_warn_limit=None to silence this warning.
33%|███████████████████████████████████████████████                                                                                | 1/3 [00:58<01:57, 58.72s/it]Tell me a funny joke!
67%|█████████████████████████████████████████████████████████████████████████████████████████████▎                                              | 2/3 [09:08<05:12, 312.49s/it]How to make a yummy chocolate cake?
CUDAGraph supports dynamic shapes by recording a new graph for each distinct input size. Recording too many CUDAGraphs may lead to extra overhead. We have observed 51 distinct sizes. Please consider the following options for better performance: a) padding inputs to a few fixed number of shapes; or b) set torch._inductor.config.triton.cudagraph_skip_dynamic_graphs=True. Set torch._inductor.config.triton.cudagraph_dynamic_shape_warn_limit=None to silence this warning.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [09:56<00:00, 198.78s/it]
Traceback (most recent call last):
File "/workspace/example2_4.py", line 63, in
outputs = model.generate(**inputs, max_new_tokens=1000, cache_implementation="static", pad_token_id=tokenizer.pad_token_id)
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py", line 1914, in generate
result = self._sample(
File "/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py", line 2651, in _sample
outputs = self(
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/hqq/utils/generation_hf.py", line 82, in custom_forward
out = out_fct(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/eval_frame.py", line 465, in _fn
return fn(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/external_utils.py", line 38, in inner
@functools.wraps(fn)
File "/usr/local/lib/python3.10/dist-packages/torch/_dynamo/eval_frame.py", line 632, in _fn
return fn(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/torch/_functorch/aot_autograd.py", line 1100, in forward
return compiled_fn(full_args)
File "/usr/local/lib/python3.10/dist-packages/torch/_functorch/_aot_autograd/runtime_wrappers.py", line 321, in runtime_wrapper
all_outs = call_func_at_runtime_with_args(
File "/usr/local/lib/python3.10/dist-packages/torch/_functorch/_aot_autograd/utils.py", line 124, in call_func_at_runtime_with_args
out = normalize_as_list(f(args))
File "/usr/local/lib/python3.10/dist-packages/torch/_functorch/_aot_autograd/runtime_wrappers.py", line 667, in inner_fn
outs = compiled_fn(args)
File "/usr/local/lib/python3.10/dist-packages/torch/_functorch/_aot_autograd/runtime_wrappers.py", line 488, in wrapper
return compiled_fn(runtime_args)
File "/usr/local/lib/python3.10/dist-packages/torch/_inductor/codecache.py", line 1478, in call
return self.current_callable(inputs)
File "/usr/local/lib/python3.10/dist-packages/torch/_inductor/compile_fx.py", line 1008, in run
return compiled_fn(new_inputs)
File "/usr/local/lib/python3.10/dist-packages/torch/_inductor/cudagraph_trees.py", line 382, in deferred_cudagraphify
return fn(inputs)
File "/usr/local/lib/python3.10/dist-packages/torch/_inductor/utils.py", line 1977, in run
return model(new_inputs)
File "/usr/local/lib/python3.10/dist-packages/torch/_inductor/cudagraph_trees.py", line 1919, in run
out = self._run(new_inputs, function_id)
File "/usr/local/lib/python3.10/dist-packages/torch/_inductor/cudagraph_trees.py", line 2089, in _run
return self.record_function(new_inputs, function_id)
File "/usr/local/lib/python3.10/dist-packages/torch/_inductor/cudagraph_trees.py", line 2123, in record_function
node = CUDAGraphNode(
File "/usr/local/lib/python3.10/dist-packages/torch/_inductor/cudagraph_trees.py", line 970, in init
self.recording_outputs: Optional[OutputType] = self._record(
File "/usr/local/lib/python3.10/dist-packages/torch/_inductor/cudagraph_trees.py", line 1196, in _record
with preserve_rng_state(), torch.cuda.device(
File "/usr/local/lib/python3.10/dist-packages/torch/cuda/graphs.py", line 186, in exit
self.cuda_graph.capture_end()
File "/usr/local/lib/python3.10/dist-packages/torch/cuda/graphs.py", line 84, in capture_end
super().capture_end()
RuntimeError: CUDA error: out of memory
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

You are using device_map=device which will cause the model to be transferred to the GPU before quantization, among other things.
Can you run this script exactly as it is, please don't change anything other than the model_id: https://github.com/mobiusml/hqq/blob/master/examples/backends/hqq_lib_demo.py
I just tried it, it should work fine and should only take 5-6 GB for a 7-8B model.

Sorry, brother. Now, I just ran the code in the link directly. I first ran llama2-7B, then llama3-8B. I could only run the third sample in warmup on llama2, but I completed the entire program on llama3. For llama3, the initial video memory usage was 6758MiB, and at 20% it was 7424MiB. Here, the video memory usage slowly increased by almost 20GB, and then the video memory usage increased for each example, finally using up almost 40g. I saw the output prompt message saying that 51 computation graphs were generated, and I could skip dynamic computation graphs
torch._ inductor.config.triton.cudagraph_skip_dynamic_graphs = True, do I need to do this?  @mobicham

The increase from 6GB to 7GB is normal: the model is taking 6GB in VRAM but it needs to allocate the KV cache, so that's the rest. But the increase to 40 GB is very strange indeed! I am not sure why is it's complaining about dynamic shapes, there are no dynamic shapes in the decoding phase. So maybe it's not using static cache :thinking:
Can you please print the following:
import torch; print(torch.__version__);
import transformers; print(transformers.__version__);

and your CUDA version as well.

Ok, thank you. This is my configuration

I ran python hqq_lib_demo.py twice in the terminal, for llama2 and llama3 respectively. This time, I tried to
import torch._ inductor.config
torch._inductor.config.triton.cudagraph_skip_dynamic_graphs = True, this has the effect. For llama2 and llama3, the maximum size should not exceed 7GB. Here are some tips during the testing process

Can you try the following:
Install CUDA 12.1 and do this:
export CUDA_HOME=/usr/local/cuda-12.1
export LD_LIBRARY_PATH=${CUDA_HOME}/lib64:$LD_LIBRARY_PATH
export PATH=${CUDA_HOME}/bin:${PATH}

Then
pip uninstall torch; pip uninstall hqq;
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121
pip install --upgrade  transformers hqq;

I ran python hqq_lib_demo.py twice in the terminal, for llama2 and llama3 respectively. This time, I tried to import torch._ inductor.config torch._inductor.config.triton.cudagraph_skip_dynamic_graphs = True, this has the effect. For llama2 and llama3, the maximum size should not exceed 7GB. Here are some tips during the testing process

But this should not happen, if it happens it means it cannot run cuda graphs and performance will be very bad

I forgot to mention that the environment I used before was NVIDIA's docker image nvcr.io/nvidia/pytorch:24.02-py3. Since it contained torch2.3, I uninstalled torch2.3 and installed torch2.5. Then, I installed transformers=4.42.1 and hqq. This is the basic environment of this image

Yeah that should work fine I think, but I haven't tested with CUDA 12.4 (your pytorch version is using 12.4), if you can test with CUDA 12.1 with the commands I shared then we can confirm if the problems comes from CUDA version. Because right now I can't reproduce it and I tested across various gpus with CUDA 12.1, no issue

Ok, thank you. I am now configuring the environment for CUDA 12.1.

 First, I used transformers=4.46.1, so I changed the chat_template. This should not be the key; Secondly, I added the following two lines after AutoTokenizer.from_pretrain() in hqq_lib_demo.py,
if tokenizer.pad_token_id is None:
tokenizer.pad_token_id = tokenizer.eos_token_id
Because I suddenly realized that some models do not have tokenizer.pad_token_id, I have seen tokenizer.pad_token_id = tokenizer.eos_token_id in many codes. I think this is the key to solving the problem. After making the two modifications above, the code executed as expected, with a video memory usage of 6-7G.

I used transformers==4.42.1 again, this time I didn't modify the template, only added pad_token_id, but the memory usage was still high and there were still problems. It seems that there is a problem with the template. Then I modified the template without adding pad_token_id, but the same problem of large memory usage occurred. Then I modified the template and added pad_token_id, but the memory usage was still very high. It seems that there is a problem with transformers==4.42.1. Finally, I changed back to transformers==4.46.1 + without adding pad_token_id, and the problem was resolved. Now I can confirm that it has nothing to do with pad_token_id, and should use transformers==4.46.1 + the correct template design.

Thanks a lot for investigating! Strange, I didn't have a problem. You don't need the chat template for the warm-up actually, but you need for generation after.


#293 - update README

@rultor merge

@rultor merge

@driver733 OK, I'll try to merge now. You can check the progress of the merge here

@rultor merge

@driver733 Done! FYI, the full log is here (took me 3min)


Delete image when post deleted

How to delete image file from server when post is deleted?
Where to use function : File::delete($filename)

@adnanfajr - you've seen the docs here:
https://laravel-backpack.readme.io/docs/crud-fields#section-image
??

Hi there,
I trust @lloy0076 's link will solve your issue. Take a closer look at the documentation for this field ;-) happens to all of us.
Cheers!

Wow thanks, sorry I do not see that documentations :)
@lloy0076 @tabacitu


Timeout

What is the max allowed timeout in the setup?
According to Spreedly we need to set it to 64 seconds to be safe. https://docs.spreedly.com/reference/api/old/timeouts/

You can override the default timeouts in the config.
$config = [
    'key' => '', // Environment key
    'secret' => '', // Signing Secret
    'gateway' => '', // Default gateway
    'timeout' => 123,
    'connect_timeout' => 456
];

http://docs.guzzlephp.org/en/latest/request-options.html#timeout
http://docs.guzzlephp.org/en/latest/request-options.html#connect-timeout

So the timeout Spreedly is talking about, is that timeout, or connect_timeout?
And shouldn't the default be 64 as recommended by Spreedly?

I believe connect_timeout is the time it takes to connect to Spreedly and timeout is the time it takes after the connection is made to Spreedly. So the 64 seconds Spreedly was referring to would be timeout. I have no plans to change the default, but I'm considering adding a timeout method that could be used override the timeout for specific api calls, Spreedly::timeout(64)->payment('sfds3gh41287dhfui')->purchase(1098)

I am curious. Why not just add the recommended timeout settings to provide a stable experience?
They recommend having it to 64 secs as per their docs.

After thinking about it more, you right. So i set the default to 64 seconds and added a new method to override api calls like Spreedly::timeout(15)->payment('sfds3gh41287dhfui')->get() that don't need 64 seconds.
https://github.com/tuurbo/spreedly#changelog

How do you know which calls need 15 and which need 64?

@lasergoat Going from memory, up to 15 secs to connect to spreedly and up to 64 secs for spreedly to connect to the gateway.

@lasergoat https://docs.spreedly.com/guides/debugging/#timeouts


[Bug]: Docker-compose deployments from repositories don't work if they need additional files

Description
Same as in issue #1805
If you make a source from a repository and use Docker Compose build pack, it does not copy any files from the repository to the application folder besides docker-compose file.
Using Beta 258
Looks like there were some fixes done in Beta 233, but that only started creating the directories referred to in the docker-compose file. It does not actually copy the files in the repository to the applications folder.
Minimal Reproduction (if possible, example repository)
Create new resource
Choose Public Repository
Repository URL: https://github.com/authelia/authelia/
Use the following in general config:
a. Build Pack: Docker compose
b. Raw Compose Deployment: True (Though I've tried that this behavior remains same even for raw compose deployment being false)
c. Base Directory: /examples/compose/local/
d. Docker Compose Location: /docker-compose.yml
Go to the install location on disk: /data/coolify/applications/{YOUR_FOLDER}
See that only README and docker-compose.yml exists, not any other configuration needed. After beta 233, it creates the additional folders (e.g. traefik and authelia) in the application folder, but does not copy the files inside them from the source repository.
Exception or Error
No response
Version
v4.0.0-beta258

I don't see a docker-compose.yml in the repository linked. Have you tried using their Docker image or running the Dockerfile instead?
The volumes inside docker-compose are persistent data that are created from the image itself, it wouldn't be there unless the image has run and created data. If you need to copy files from the repository to the created image, that should happen in the Dockerfile.

+1, tried to use a metadata/migrations directory from a Hasura repository and it only creates an empty folder, it never transfers the files from the directory itself :((

@nikolouzos the docker-compose.yml is at https://github.com/authelia/authelia/tree/master/examples/compose/local as referenced in the base directory configuration in the original bug report. The docker-compose file is copied appropriately to the destination.
I've also tried this with the docker-compose.yml being in the root of the repository as well and thus the base directory being / with the same result.

A temporary workaround for docker-compose with Coolify bug, where it doesn't copy the volume directory files, only creates an empty folder as for the version v4.0.0-beta.265 and you need some directories moved into the container:

ssh into your VPS
git clone your private repository into some folder, e.g. /home/hasura in my case
map this root path in your docker-compose.yml in the volumes section of the service to mount the host (server) directory into the container
success, after rebuild your container has those directories and files available and app in my case Hasura does run correctly with files like migrations, metadata  from the repository :)

services:
   ...
    volumes:
      - /home/hasura/metadata:/hasura-metadata
      - /home/hasura/migrations:/hasura-migrations

+1 on this issue
My docker-compose files are mounting the contents of the git repository on the root of the Docker image itself (to allow for no docker rebuilds if not needed)

+1 on this one and this simply makes coolify unusable for me. Because, I have an rails app with docker-compose.yml file on root folder. Deploying to another VPS or local works; but when deploying to coolify using docker-compose; it simply doesn't copy the files from the repo.

+1 for please fix this.
Also +1 for you can work around it with a Dockerfile.
I'm running a build step in a composer container which starts by doing a COPY . /app to bring in all files from git.
Then the CMD for that service runs first and copies the files into a dir which is mounted on all containers.
Now the files exist as if we'd brought them in directly from the git repo and mounted. It's not ideal but it works.

Also having this issue

So glad to have found this issue, I was going over the docs and configs again and again with no success.
I think it would be really helpful to show a disclaimer in the docs. Letting the users know that the files are not automatically copied and they need to implement their own strategy for now.
Also thinking about an intuitive solution for this scenario. How about automatically copying the files for the first deployment and adding a checkbox in the persistent storage tab that allows the user to mark what should be overwrittable?
By default it would not be checked to avoid losing important data.

Another use-case from my side.
I want to add a new Resource to my project - Glances for monitoring using their docker-compose. I go to My project > New Resource > Docker compose.
The problem is that the docker-compose file requires a glances.conf config file to work at all, and ideally a Dockerfile (although it can do without it). However, there is no way to include the config file somewhere.
One possible solution would be to create my own separate repo just for to have the docker-compose and config file, but that's too much of a hassle to be a long-term solution.

+1

+1

This has been driving me mad all day, thought I'd see if this was a known bug.
I set up a docker-compose git app project, and the issue is that the app folder /data/coolify/applications/abcdef/ never gets the files from git. The only files that make it there come from the image.
This is definitely happening in v4.0.0-beta.306, I've spent hours today trying to figure out what's going on

Hope this gets added/fixed soon. Until then, to expand on the fix, here's demo repo with a "copy" service that moves files from the git repo, into an image, then mounts them on all services, as if Coolify had brought everything in from git.
https://github.com/boostmetrica/coolify-count-and-mount-demo

If there are environment variables that contain the path to the artifact folder (containing the git files), we might be able to use Custom Build Command to copy this over, for a simpler workaround:

Maybe something like
cp -r /artifacts/"${ARTIFACT_ID}/foldername" . && docker compose build ?
I'm not sure where the git contents exist at this point. Any ideas?

Nice idea. Looks like it would be /artifacts/"${ARTIFACT_ID}/${GIT_REPO_NAME}"
Not suggesting those vars exist, but that would be the format.
https://github.com/coollabsio/coolify/blob/e9158b7305bb9253fe75a0e76ef34af62f75199b/app/Models/Application.php#L685:L695

Hope this gets added/fixed soon. Until then, to expand on the fix, here's demo repo with a "copy" service that moves files from the git repo, into an image, then mounts them on all services, as if Coolify had brought everything in from git.
https://github.com/boostmetrica/coolify-count-and-mount-demo

yep, that's the quickest solution.

+1 this is a must have - isn't there something to be done with pre-deploy commands ?

This is very frustrating issue. For me this is huge block and a critical feature to have.
I'd very appreciate at least some message, so me and other users won't spend hours trying to understand what's wrong. Especially taking into consideration that volume is still present in docker-compose file after Coolify's processing, i.e. I'd expect it to work if it's still there

If there are environment variables that contain the path to the artifact folder (containing the git files? Is that right.. the folder disappears and I can't check), we might be able to use Custom Build Command to copy this over, for a simpler workaround:
Maybe something like cp -r /artifacts/"${ARTIFACT_ID}/foldername" . && docker compose build ?
I'm not sure where the git contents exist at this point. Any ideas?

I've been trying to find a command we can use here to achieve this, but can't find the location of the temporary git artifact folder on the filesystem.
I've run inotifywait -m -r -e access,modify,create,delete ./ --format '%w%f %e' >> /home/user/inotify_output.txt on both host and coolify container to watch the filesystem and /artifacts/ doesn't seem to exist. Does anyone know where /artifacts/$UUID is created? If we can access it, we can probably do what we want in a custom build command

Thanks @lnnrtngl, I can't test this right now, but maybe something like this would work:
docker exec $(docker ps --filter ancestor=ghcr.io/coolabsio/coolify-helper:latest --format "{{.ID}}" | head -n 1) cp -r ./app /artifacts/${ARTIFACT_ID}/${GIT_REPO_NAME} && docker compose build`

Or maybe not, I'm guessing the helper isn't available then... will try later

+1 This is blocking my Verdaccio deployment, as it needs the config file https://github.com/verdaccio/verdaccio/tree/master/docker-examples/v4/docker-local-storage-volume

This is still broken as of v4.0.0-beta.315. The entire git project content should be available at time of running docker compose.

Checking this!

I will add a new flag to the build phase. This will copy the Base Directory content from git to the deployment directory before deploying your application.

Going to test this as soon as the update lands. Thanks for fixing this

@andrasbacsai When I selected the new option to keep the files, I got the following error:
`[2024-Jul-19 22:18:51.092462] Starting deployment of splendid-sardine-sccww40 to localhost.
[2024-Jul-19 22:18:51.337747] Preparing container with helper image: ghcr.io/coollabsio/coolify-helper:latest.
[2024-Jul-19 22:18:56.207443] ----------------------------------------
[2024-Jul-19 22:18:56.212857] Importing (commit sha HEAD) to /artifacts/lggswcw.
[2024-Jul-19 22:19:05.633228] Pulling & building required images.
[2024-Jul-19 22:19:06.004280] Removing old containers.
[2024-Jul-19 22:19:06.353648] Starting new application.
[2024-Jul-19 22:19:08.006655] rm:
[2024-Jul-19 22:19:08.015507] refusing to remove '.' or '..' directory: skipping '/data/coolify/applications/sccww40/.'
rm: refusing to remove '.' or '..' directory: skipping '/data/coolify/applications/sccww40/..'
[2024-Jul-19 22:19:08.419927] rm: refusing to remove '.' or '..' directory: skipping '/data/coolify/applications/sccww40/.'
rm: refusing to remove '.' or '..' directory: skipping '/data/coolify/applications/sccww40/..'
`

I've tested this yesterday and it's working well. I think there's a slight race condition somewhere though as sometimes a deployment following a webhook fails, but then redeploying again works. If I can reproduce this again I'll make a report, but it's definitely working as expected. Thanks

@andrasbacsai When I selected the new option to keep the files, I got the following error:
`[2024-Jul-19 22:18:51.092462] Starting deployment of splendid-sardine-sccww40 to localhost. [2024-Jul-19 22:18:51.337747] Preparing container with helper image: ghcr.io/coollabsio/coolify-helper:latest. [2024-Jul-19 22:18:56.207443] ---------------------------------------- [2024-Jul-19 22:18:56.212857] Importing (commit sha HEAD) to /artifacts/lggswcw. [2024-Jul-19 22:19:05.633228] Pulling & building required images. [2024-Jul-19 22:19:06.004280] Removing old containers. [2024-Jul-19 22:19:06.353648] Starting new application. [2024-Jul-19 22:19:08.006655] rm:
[2024-Jul-19 22:19:08.015507] refusing to remove '.' or '..' directory: skipping '/data/coolify/applications/sccww40/.' rm: refusing to remove '.' or '..' directory: skipping '/data/coolify/applications/sccww40/..'
[2024-Jul-19 22:19:08.419927] rm: refusing to remove '.' or '..' directory: skipping '/data/coolify/applications/sccww40/.' rm: refusing to remove '.' or '..' directory: skipping '/data/coolify/applications/sccww40/..' `

I am also experiencing this. The container deploys as normal, but it reports that it failed.

I get the same issue, most likely a scripting problem where it is trying to delete "." and ".." in the folder of the project.

Having real problems with this, race conditions and cleanup.
I think enough of us want this feature to ask: How much can do we need to chip in to get this focused on for a day to fix the bugs? Keeps breaking on prod for me
Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/data/coolify/applications/xxxxxxx/nginx.prod.conf" to rootfs at "/etc/nginx/nginx.conf": mount /data/coolify/applications/xxxxxxx/nginx.prod.conf:/etc/nginx/nginx.conf (via /proc/self/fd/6), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type


DFU state sticks on .disconnecting

When the DFU process exits with error 302, it seems to stay perpetually on the .disconnecting state. I've tried calling the abort() controller method in my didErrorOccur() implementation, but am having a different issue there which I noted in #42

@mostafaberg and this as well please? I think it may be connected to #42 ?

Looking into that, this is only happening on Legacy DFU i assume right ?

I'm a bit confused with the distinction here when you say Legacy DFU vs Secure DFU, as I noted in #44 We're performing DFU with a secure firmware packet though.

Does that clear things up at all @mostafaberg ?

Secure DFU is the latest vesion to our DFU library, it uses signed distribution files and is a completely different protocol, from our DFU Library side it doesn't matter at all, the library understands which DFU type it's dealing with and handles it with the proper classes, they are two completely different systems though, you can ask around in our devzone if you want to know the specifics of Secure DFU vs. Legacy DFU

Ah I think I was just misunderstanding your question then. From the perspective of the DFU Library, it is all one thing right?
We're seeing this issue with the Secure DFU then though to answer your question, using the recent DFU library and secure packets on our firmware end.

Yeah I seem to have used the wrong words in my question, from the iOSDFULibrary's side, nothing changes at all, it all depends on your firmware's implementation, I was actually asking about your firmware and not the DFULibrary, you can't choose that in the DFU Library since it's automatically done :)
thanks for clearing it up for me 👍

Hi, the Version 3 is now published on our Version3 branch. You can edit your Profile and give the whole repo and branch path there. Could you try with this version?
Version 3 is now in beta phase and we will publish it when we get some feedback.
Thanks!

Version 3 seems to be working for us! Going to close this issue for now. 😄


[MINOR] exclude velox build path ep/build-velox/build/ in .gitignore

velox build directory in .gitignore
What changes were proposed in this pull request?
Add ep/build-velox/build/ to .gitignore. it's local build src files and binary
How was this patch tested?
manual

oh. that's true. thanks. aborting this pr. it is a little weird. intellij git doesn't not recognize this


Logs are super verbose

Using Docassemble 1.4.88 and ALKiln 5.7.0, my docassemble.log size is now approaching 1.4 GB in size. I went looking at the log file briefly and the majority is linked to my docassemble testing account used exclusively for ALKiln. Could the docassemble logs be more limited by a env parameter or something?

Gonna transfer this to ALKilnInThePlayground, which is the actual place where we print things to the docassemble logs


Help: unknown command for 4 commands

Help commands produce "unknown command" for 4 commands related to wallet locking and passphrase.
There is an extra Boolean (compared to bitcoin v0.17) of pwallet->IsCrypted() which must be true to produce help text for
walletpassphrase
walletpassphrasechange
walletlock
and pwallet->IsCrypted() must be false to produce help for
encryptwallet
otherwise requesting help for these commands returns "unknown command".
This doesn't seem "helpful." Help should provide the help text in every case as bitcoin Core does.
Unencrypted qtum-qt v<IP_ADDRESS>-47a30461d-dirty (64-bit):

Suggested fix, remove the Boolean pwallet->IsCrypted() in rpcwallet.cpp lines:
2984  walletpassphrase()
3024  walletpassphrasechange()
3085  walletlock()
3125  encryptwallet()

Fixed here: https://github.com/qtumproject/qtum/commit/aea8a16ca0cc89cf68e8adbb1159677423cd1d60


Docker Setup for Raspberry Pi

What I have done and my setup for Pi4:
git --version

git version 2.20.1

java -version

openjdk version "11.0.7" 2020-04-14
OpenJDK Runtime Environment (build 11.0.7+10-post-Raspbian-3deb10u1)
OpenJDK Server VM (build 11.0.7+10-post-Raspbian-3deb10u1, mixed mode)

cat /etc/os-release

PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

uname -a

Linux pi4-home 4.19.118-v7l+ #1311 SMP Mon Apr 27 14:26:42 BST 2020 armv7l GNU/Linux

free

              total        used        free      shared  buff/cache   available
Mem:        3999744      389120     2814556        8996      796068     3460564
Swap:       1048572           0     1048572

export ARCH=arm32v7

echo $ARCH

arm32v7

id

uid=1000(pirate) gid=1000(pirate) Gruppen=1000(pirate),4(adm),20(dialout),24(cdrom),27(sudo),29(audio),44(video),46(plugdev),60(games),100(users),105(input),109(netdev),996(docker),997(gpio),998(i2c),999(spi)

pwd

/home/pirate/notebooks/openroberta-lab

nmap localhost

Starting Nmap 7.70 ( https://nmap.org ) at 2020-06-08 21:03 CEST
Nmap scan report for localhost (<IP_ADDRESS>)
Host is up (0.00049s latency).
Not shown: 993 closed ports
PORT     STATE SERVICE
22/tcp   open  ssh
25/tcp   open  smtp
3306/tcp open  mysql
3389/tcp open  ms-wbt-server
8181/tcp open  intermapper
8888/tcp open  sun-answerbook
9999/tcp open  abyss

Nmap done: 1 IP address (1 host up) scanned in 0.24 seconds

git clone https://github.com/OpenRoberta/openroberta-lab.git

Klone nach 'openroberta-lab' ...
remote: Enumerating objects: 763, done.
remote: Counting objects: 100% (763/763), done.
remote: Compressing objects: 100% (405/405), done.
remote: Total 145011 (delta 271), reused 610 (delta 182), pack-reused 144248
Empfange Objekte: 100% (145011/145011), 388.36 MiB | 6.78 MiB/s, Fertig.
Löse Unterschiede auf: 100% (80678/80678), Fertig.
Checke Dateien aus: 100% (4481/4481), Fertig.

ls -l openroberta-lab/ora.sh

-rwxr-xr-x 1 pirate pirate 7199 Jun  8 21:04 openroberta-lab/ora.sh

cd openroberta-lab
./ora.sh new-docker-setup pi-ora

New docker setup created in pi-ora. Edit 'decl.sh' and setup databases and servers now.
Create servers by calling './ora.sh new-server-in-docker-setup pi-ora SERVER_NAME'

./ora.sh new-server-in-docker-setup pi-ora rdf-2020

New server rdf-2020 created. Edit 'pi-ora/decl.sh' and 'pi-ora/server/rdf-2020/decl.sh'
Copy an existing database or create an empty database after 'mvn clean install -DskipTests'
by calling './ora.sh -dbParentdir pi-ora/db/rdf-2020 create-empty-db'

cat pi-ora/decl.sh

# essential configuration variables. Be very carefull!

INAME=''                # the (short, alphanumeric) name of this installation
DATABASE_SERVER_PORT='' # one database server for all jetty server. It is listening on this port. Usually this is '9001'
DOCKER_NETWORK_NAME=''  # the network for cooperation between jetty and database server. Usually this is 'ora-net'

SERVERS=''              # these servers can run (used in start-all and stop-all), e.g. 'test dev dev4'
SERVERSXMX='-Xmx1G'     # maximum of heap size for the jetty server. CURRENTLY UNUSED. Production server needs 1G, raspi 1G or 500M
DATABASES=''            # these databases are served by the database server, e.g. 'test dev dev4'
DATABASEXMX='-Xmx4G'    # maximum of heap size for the database server. Production server needs 4G, raspi 1G or 500M
AUTODEPLOY=''           # these servers are deployed when new commits hit the git repo. See auto-deploy and cron, e.g. 'dev dev4'

# for operating purposes
ALIVE_ACTIVE=true                                               # set to false if not needed
  ALIVE_MAIL_SMTP_SERVER='smtps.iais.fraunhofer.de'               # smtp server
  ALIVE_MAIL_SMTP_PORT='25'                                       # smtp port
<EMAIL_ADDRESS>'                 # the name for the sender of the mail
  ALIVE_MAIL_RECEIVER=( 'name-1@address-1' 'name-2@address-2' )   # list of receivers for the mail, white-space separated

PYTHON=/usr/bin/python3 # for log file evaluation

sed -Ei "s/(INAME=')/\1pi-ora/g" pi-ora/decl.sh

sed -Ei "s/(DATABASE_SERVER_PORT=')/\19001/g" pi-ora/decl.sh
sed -Ei "s/(DOCKER_NETWORK_NAME=')/\1ora-net/g" pi-ora/decl.sh
sed -Ei "s/(SERVERS=')/\1test/g" pi-ora/decl.sh
sed -Ei "s/(DATABASES=')/\1test/g" pi-ora/decl.sh
sed -Ei "s/(DATABASEXMX='-Xmx)4/\12/g" pi-ora/decl.sh

cat pi-ora/decl.sh

# essential configuration variables. Be very carefull!

INAME='pi-ora'                # the (short, alphanumeric) name of this installation
DATABASE_SERVER_PORT='9001' # one database server for all jetty server. It is listening on this port. Usually this is '9001'
DOCKER_NETWORK_NAME='ora-net'  # the network for cooperation between jetty and database server. Usually this is 'ora-net'

SERVERS='test'              # these servers can run (used in start-all and stop-all), e.g. 'test dev dev4'
SERVERSXMX='-Xmx1G'     # maximum of heap size for the jetty server. CURRENTLY UNUSED. Production server needs 1G, raspi 1G or 500M
DATABASES='test'            # these databases are served by the database server, e.g. 'test dev dev4'
DATABASEXMX='-Xmx2G'    # maximum of heap size for the database server. Production server needs 4G, raspi 1G or 500M
AUTODEPLOY=''           # these servers are deployed when new commits hit the git repo. See auto-deploy and cron, e.g. 'dev dev4'

# for operating purposes
ALIVE_ACTIVE=true                                               # set to false if not needed
  ALIVE_MAIL_SMTP_SERVER='smtps.iais.fraunhofer.de'               # smtp server
  ALIVE_MAIL_SMTP_PORT='25'                                       # smtp port
<EMAIL_ADDRESS>'                 # the name for the sender of the mail
  ALIVE_MAIL_RECEIVER=( 'name-1@address-1' 'name-2@address-2' )   # list of receivers for the mail, white-space separated

PYTHON=/usr/bin/python3 # for log file evaluation

find . -name decl.sh

./Docker/openroberta/decl.sh
./Docker/openroberta/server/_server-template/decl.sh
./pi-ora/decl.sh
./pi-ora/server/rdf-2020/decl.sh
./pi-ora/server/_server-template/decl.sh

cat pi-ora/server/rdf-2020/decl.sh

BASE_VERSION=''           # the version of the docker base image, FROM which this server image is derived. Be careful to define the RIGHT number

PORT=''                    # port the jetty server will listen to, very often 1999
LOG_LEVEL=''               # the logging level of the root logger. From DEBUG to ERROR
LOG_CONFIG_FILE=''         # logback configuration, very often /logback-prod.xml or /logback.xml

BRANCH=''                  # the branch to be deployed on this server, e.g. 'develop'
# COMMIT=''                # if a commit instead of a branch is deployed, e.g. '174db1b4f0'
GIT_REPO=''                # the git repo where the branch is founf, e.g. 'openroberta-lab'
GIT_PULL_BEFORE_BUILD=true # set to false ONLY, if you have a well-prepared, ready to use branch checked out. NEVER use such a git repo for other server deployments. NEVER.

# START_ARGS='-d key1=val1 -d key2=val2' # use to supply parameter when container is started

sed -Ei "s/(BASE_VERSION=')/\116/g" pi-ora/server/rdf-2020/decl.sh
sed -Ei "s/(PORT=')/\11999/g" pi-ora/server/rdf-2020/decl.sh
sed -Ei "s/(LOG_LEVEL=')/\1DEBUG/g" pi-ora/server/rdf-2020/decl.sh
sed -Ei "s/(LOG_CONFIG_FILE=')/\1\/logback.xml/g" pi-ora/server/rdf-2020/decl.sh
sed -Ei "s/(GIT_REPO=')/\1openroberta-lab/g" pi-ora/server/rdf-2020/decl.sh

sed -Ei "s/(BRANCH=')/\1master/g" pi-ora/server/rdf-2020/decl.sh

cat pi-ora/server/rdf-2020/decl.sh

BASE_VERSION='16'           # the version of the docker base image, FROM which this server image is derived. Be careful to define the RIGHT number

PORT='1999'                    # port the jetty server will listen to, very often 1999
LOG_LEVEL='DEBUG'               # the logging level of the root logger. From DEBUG to ERROR
LOG_CONFIG_FILE='/logback.xml'         # logback configuration, very often /logback-prod.xml or /logback.xml

BRANCH='master'                  # the branch to be deployed on this server, e.g. 'develop'
# COMMIT=''                # if a commit instead of a branch is deployed, e.g. '174db1b4f0'
GIT_REPO='openroberta-lab'                # the git repo where the branch is founf, e.g. 'openroberta-lab'
GIT_PULL_BEFORE_BUILD=true # set to false ONLY, if you have a well-prepared, ready to use branch checked out. NEVER use such a git repo for other server deployments. NEVER.

# START_ARGS='-d key1=val1 -d key2=val2' # use to supply parameter when container is started

cd pi-ora/git

git clone https://github.com/OpenRoberta/openroberta-lab.git

Klone nach 'openroberta-lab' ...
remote: Enumerating objects: 763, done.
remote: Counting objects: 100% (763/763), done.
remote: Compressing objects: 100% (405/405), done.
remote: Total 145011 (delta 271), reused 610 (delta 182), pack-reused 144248
Empfange Objekte: 100% (145011/145011), 388.36 MiB | 6.29 MiB/s, Fertig.
Löse Unterschiede auf: 100% (80678/80678), Fertig.
Checke Dateien aus: 100% (4481/4481), Fertig.

cd ../..
pwd

/home/pirate/notebooks/openroberta-lab/openroberta-lab

ls

admin.sh          OpenRobertaServer  Resources    RobotNXT
Docker            ora-help.txt       RobotArdu    RobotRaspberryPi
JaxbGenerator     ora.sh             RobotEdison  RobotVorwerk
LICENSE           pi-ora             RobotEV3     RobotWeDo
NOTICE            pom.xml            RobotMbed    src
OpenRobertaRobot  README.md          RobotNAO     WedoInterpreter

docker network ls

NETWORK ID          NAME                DRIVER              SCOPE
62abbc60c016        bridge              bridge              local
ec3caaa57eb5        host                host                local
d5d27fda432a        none                null                local

pi-ora/scripts/run.sh -yes gen-net

run.sh [-q] [-yes] [-D]                                              be quiet, answer all questions with 'y', debug mode
       help                                                          this text
       gen <server> | start <server> | stop <server>                 gen builds image; start first tries to stop, then (re-)starts the server; stop stops the server
       deploy <server>                                               deploy first calls gen, then calls start
       admin <server> <admin-cmd>                                    execute admin command on server, e.g. 'cleanup-temp-user-dirs'
       auto-deploy |                                                 check for git changes for servers found in variable AUTODEPLOY
       start-all | stop-all                                          start/stop db server and server found in variable SERVERS
       gen-dbc | start-dbc | stop-dbc | backup [<db>]                generate db server, start and stop db server using variable DATABASES, backup a db
       backup-save <dbBackup@<remote-host>:<from-path> <to-path>     save a db backup from a remote machine to this machine. <to-path> is relative to BASE-DIR
       alive <url> [mail={always|error}] [msg=<msg>]                 is the server alive? When to send mail (default: always). Set an additional mail header
       docker-info | network | logs | test-info                      container state, network and some log from running containers; info about deployed servers

       monthly-stat <server> [month]                                 run the monthly stats for a server. Optionally supply the month (e.g. '05' or '12')
       show-server                                                   some data about processes with many open file descriptors (FD)
       show-resources <server> [<file> [<lower> [<upper> [ <url> [<hours> [<pid>]]]]] show resource usage. Parameter description is in file helper/_showResources.sh
       show-activity-once <server> [<file> [ <url>]]                 show accumulated user activity. Parameter description is in file helper/_showActivityOnce.shs

dangerous commands:
       prune                                                         rm as much unused data from docker as possible
       auto-restart <server> <url>                                   restart a container if <url>/rest/alive doesn't respond within 50 sec (url ~ https://dns:port)
       gen-net                                                       generate the docker network
       start-export <server>                                         start a server from the export dir outside of docker
working on host pi4-home for installation pi-ora at BASE_DIR /home/pirate/notebooks/openroberta-lab/openroberta-lab/pi-ora
******************** 2020-06-08 21:09:06 ********************
-yes was set. Automatic 'y' for question: do you have double-checked, that the bridge network name 'ora-net' is NOT used elsewhere?
2020-06-08 21:09:06: generating the openroberta bridge network 'ora-net'
e0ef91bd1e3fd192f8ce32379cf9c442d2c0cdd5e63d313f8f1284c1602dfb77
generating the openroberta bridge network 'ora-net' finished

docker network ls

NETWORK ID          NAME                DRIVER              SCOPE
62abbc60c016        bridge              bridge              local
ec3caaa57eb5        host                host                local
d5d27fda432a        none                null                local
e0ef91bd1e3f        ora-net             bridge              local

./ora.sh -dbParentdir pi-ora/db/test create-empty-db

creating an empty db using the url jdbc:hsqldb:file:pi-ora/db/test/openroberta-db - an existing db is NOT overwritten
Fehler: Hauptklasse de.fhg.iais.roberta.main.Administration konnte nicht gefunden oder geladen werden
Ursache: java.lang.ClassNotFoundException: de.fhg.iais.roberta.main.Administration

I'm not able do install the db on my Pi4. Any suggestions will help.
THX Kraeml

thx for the detailed information. I'll add comments here to simplify our docker setup resp to help with your setup.

you cannot create the empty database, because the libraries needed are not yet installed. You have to run 'mvn clean install -DskipTests' (rm the -D to see the unit tests executed). Then it should work.
As a consequence I'll make the following changes:

improve the first paragraph of the readme
add an empty database to allow a simple start w.o. generating a database
remove the INAME parameter to have simpler names in dockerhub. Consequence: only one installation on a machine. That seems ok. This will change the image names. The database image and the server images MUST be generated again. Do not forget to remove old images, they use up a lot of space!

I push the changes to the branch "docker-setup". Feel free to evaluate it.

THX for your help. The pi docker setup is now working. The output of my jupyter notebook is here. I forgot the mvn clean install -DskipTests.

solved


calc with navigate is invalid in some preprocessor.

https://github.com/khang-nd/7.css/blob/a1355c0cf4c5590a01d164922e0ce4ef2cd28344/gui/_window.scss#L72
detail in this post.
https://stackoverflow.com/a/25205523/3646724
so you can use margin-top: calc(0 - var(--window-spacing) - 1px);  instead

Hey mate, thanks for pointing that out, didn't know of such problem. I will check the PR.


fix(evm): refactor executor & detect setUp assertion failures

close #1291
Refactoring according to @onbjerg's comment

We can refactor Executor such that we only really have the call_raw/call variants that return state changesets - call_committing/call_raw_committing would then just be very thin wrappers around these that also call db.commit(changeset). This cleans up the executor code a bit, and seems like the right path forward. I think I did attempt this at some point, but there was some annoyances in some other parts of the code so I shelved it to get REVM over the line.

rfr pending merged_logs resolution above


Entry Point Not Found....

The procedure entry point __uncaught_exceptions could not be located in the dynamic link library api-ms-win-crt-private-l1-1-0.dll.
???????
This is related to TacO.

This issue was fixed with the latest TacO release, and was confirmed released without Windows 7 support by mistake.


Increasing test coverage

Run pytest --cov=scramjet first on main branch. You should see:

----------- coverage: platform linux, python 3.9.7-final-0 -----------
Name                           Stmts   Miss Branch BrPart  Cover
----------------------------------------------------------------
scramjet/__init__.py               0      0      0      0   100%
scramjet/ansi_color_codes.py      10      0      0      0   100%
scramjet/pyfca.py                101      0     30      0   100%
scramjet/streams.py              287      7     96      9    96%
scramjet/utils.py                 38      0     14      0   100%
----------------------------------------------------------------
TOTAL                            436      7    140      9    97%

Checkout coverage branch and run  pytest --cov=scramjet again. The result should look like below:

----------- coverage: platform linux, python 3.9.7-final-0 -----------
Name                           Stmts   Miss Branch BrPart  Cover
----------------------------------------------------------------
scramjet/__init__.py               0      0      0      0   100%
scramjet/ansi_color_codes.py      10      0      0      0   100%
scramjet/pyfca.py                101      0     30      0   100%
scramjet/streams.py              283      0     94      0   100%
scramjet/utils.py                 38      0     14      0   100%
----------------------------------------------------------------
TOTAL                            432      0    138      0   100%

To generate html report, you can run pytest --cov=scramjet --cov-report html.
Open htmlcov/index.html to see more details about coverage tests.

Good job 👍

Nice idea. I found some simple solution for github actions, but it is not properly parsed, and it's only report added by bot. Not sure how to force coverage level on PR. Let's make issue from that and add in separate PR.

WFM 👍 Could you report this?

I will add it later.


Populate CircleCI pull request field

This allows Coveralls to post back to GitHub with a status check.

CircleCI populates the CI_PULL_REQUEST environment variable:
https://circleci.com/docs/environment-variables

Thanks. Looks nice!
I'm going to release this in a few days.

Released this as v2.6.0.


Add the ability to dynamically switch between ADSync and ADSync2019 databases

Hi guys !
This PR permits to dynamically retrieve the ADSync database connection string from the AD Connect server registry keys.
Depending on the installed version, the string can be either "ADSync" or "ADSync2019", and this value can be extracted from the registry, in addition to the other values.
Additionally, for the remote extraction with ADSyncQuery, the path of the .mdf and .ldf files also differs between the versions.
In parallel, the .NET version has also been updated to the 4.8. It seems to work fine.

thanks, looking good!


fix(driver,userspace/libscap): multiple fixes for syscalls_of_interest feature

Signed-off-by: Federico Di Pierro<EMAIL_ADDRESS>What type of PR is this?
/kind bug
Any specific area of the project related to this PR?
/area driver-kmod
/area driver-ebpf
/area libscap
What this PR does / why we need it:
First of all, properly add PPM_SC_UMOUNT2 to routing table and PPM_SC_EXECVE entirely (both as enum field, and as syscall routing table value).
The routing table is only used for PPME_GENERIC_{X,E} events thus it does not really matter to add new values (as they're mapped to specific events in any case).
Moreover, cleaned up ia32 routing table a bit removing some duplicated entries.
Finally, avoid breaking ppm_sc_of_interest routing table loop in scap_open_live_int() as multiple syscalls can be mapped to same PPM_SC code.
Which issue(s) this PR fixes:
Fixes #162
Special notes for your reviewer:
Does this PR introduce a user-facing change?:
NONE

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: FedeDP
To complete the pull request process, please assign leodido after the PR has been reviewed.
You can assign the PR to them by writing /assign @leodido in a comment when ready.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Hi @FedeDP. Thanks for your PR.
I'm waiting for a falcosecurity member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.
Once the patch is verified, the new status will be reflected by the ok-to-test label.
I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/ok-to-test

LGTM label has been added.  Git tree hash: 9827381a530042d2da249519e5d0c118b1eab40f

/approve

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: FedeDP, leogr
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [FedeDP,leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


Utilize GPS Sensor to Render Rover on Satellite Map Using Mapbox

Closing issue as we have reached the end of autonomy front-end's development life-cycle. We will focus on other features now, also because this is implemented in the previous versions.


Queues not processing MSSQL database jobs with PHP on Mac

Laravel Version
10.13.5
PHP Version
8.2.5
Database Driver & Version
SQLSRV - MSSQL 2019
Description
Using Mac Ventura with php installed by Homebrew, when I run php artisan queue:work using the database driver on a MSSQL connection, jobs in the database are never processed.  Also, no errors are logged to CLI, only storage/logs/laravel/log:
$databaseVersion =$this->database->getPdo()->getAttribute(PDO::ATTR_SERVER_VERSION)  logs the error:

"SQLSTATE[IM001]: Driver does not support this function: driver does not support that attribute {"exception":"[object] (PDOException(code: IM001): SQLSTATE[IM001]: Driver does not support this function: driver does not support that attribute at ... vendor/laravel/framework/src/Illuminate/Queue/DatabaseQueue.php:263)"

I am assuming the driver doesn't support with the PHP supplied (driver works with same code with MYSQL connection AND in production with a Forge provisioned server).  I tried supplying ./config/database.php with a version number but it kept squawking "Invalid Version" when I placed the key/value at the root of the config, ie config('database.connections.sqlsrv.version').  I feel like the version key should belong in the options array, ie config('database.connections.sqlsrv.options.version').
Possible solution:
$databaseVersion = $this->database->getConfig('options.version') ?? $this->database->getPdo()->getAttribute(PDO::ATTR_SERVER_VERSION);
Steps To Reproduce
Using Mac Ventura with php installed by Homebrew, when I run php artisan queue:work using the database driver on a MSSQL connection, jobs in the database are never processed.  No errors are logged to CLI; they are only logged in storage/logs/laravel.log.

Thanks. You've sent in a PR so let's see how it goes.


Results upload button not enabled

Thank you for making DBeaver, it's great. But it still has a few problems. Here's one.
System information:

Operating system: MacOS Big Sur: 11.2.3 (20D91)
DBeaver version: DBeaver Enterprise Edition: Version <IP_ADDRESS>012061950

Connection specification:

Database name and version: Microsoft SQL Server 2019 (RTM) - 15.0.2000.5 (X64)
Sep 24 2019 13:48:23
Enterprise Edition: Core-based Licensing (64-bit) on Windows Server 2019 Standard 10.0  (Build 17763: ) (Hypervisor)
Driver name: don't know
Do you use tunnels or proxies (SSH, SOCKS, etc)?: Yes, F5 Big-Ip

Describe the problem you're observing:
Quite often, after DBeaver executes a query, the results upload button is not enabled. I find that clicking on other tabs and then returning to the tab that the results upload button suddenly becomes enabled. See a video which documents the problem. At this time you see the problem; after clicking on a couple of tabs, the upload link becomes enabled.
Steps to reproduce, if exist:
This occurs on many queries I run.
Include any warning/errors/backtraces from the logs

Hello @artgoldberg Thank you for the report. Does this issue reproduce on the latest 21.1 version?
Also maybe you have noticed if there is some similiarity in queries that lead to this or are they completely different?

Hi @Matvey16
This problem does not occur for me in version 21.1


AMR: Tetrahedron element is not marked for refinement

We are trying to use AMR for a paper we are working on. We want to use tetrahedral meshes and we want to test the convergence rates of a FEM, the runtime and the scalability. For these reasons, we start from an initial mesh, serially refine it few times, create a parallel mesh and do parallel refinement few times before switching to AMR. However, if we do any sort of serial refinement we encounter the error below.
Verification failed: (tet->GetRefinementFlag() != 0) is false:
 --> TETRAHEDRON element is not marked for refinement.
 ... in function: void mfem::Mesh::Bisection(int, mfem::HashTable<mfem::Hashed2> &)
 ... in file: mesh/mesh.cpp

This issue might seem similar to #947, #1028 and #1119. However, it still exists in MFEM 4.1.1. The error can be reproduced by taking the steps below;

Modify ex6p.cpp slightly, move mesh->UniformRefinement(); to the line above the if (mesh->NURBSext)  condition.
Compile ex6p and run using any of the inline-tet or beam-tet.

Two interesting observations:

MFEM does not throw analogous errors if ex6p is used with 2-dimensional meshes of any kind (beam-quad, beam-tri, inline-quad, inline-tri and star.mesh are the ones I tried) or 3-dimensional hexahedral meshes (beam-hex, inline-hex).
After the modification inline-wedge does not even throw an error, ex6p segfaults when AMR is attempted.

It looks like I do not have the good habit of reading documentation. Sorry for bothering you. In case any one else encounters this problem, there are currently two uniform refinement algorithms implemented for tetrahedral meshes. They are called Algorithm A and Algorithm B. The default is Algorithm A, however, if used mesh has to be finalized before local refinement can be attempted (this is stated in the doxygen pages). If Algorithm B is used, there is no issue, no need for finalization.
So what is the solution? Either the line
mesh->UniformRefinement();

can be replaced with
mesh->UniformRefinement(1); // Use Algorithm B

or the line
mesh->FinalizeTopology(); // According to the documentation this is necessary but
                          // everything works okay if it is not there for this example.
mesh->Finalize(true); // refine parameter is set to true

can be added after mesh->EnsureNCMesh().
FinalizeTopology is also called at the end of Finalize if may_change_topology is TRUE so it seems to me that there is some redundancy in some cases. But I guess it is an implementation choice. The important thing seems to be the function MarkForRefinement and/or MarkTetMeshForRefinement which are called during Finalize if the parameter refine is set to true which explains the error we got. Because we never marked any of those elements for refinement to begin with.
Obviously this is not a bug and obviously it is my fault as I did not read the documentation/code properly. However, I also didn’t expect this exceptional case for tetrahedral meshes. In my mind, it wasn’t any different than a hexahedral mesh. So the average user will very likely encounter this problem. That is why I wrote this long explanation.
Thanks a lot for your time and sorry for any inconvenience I caused.

Hi @aasivas, sorry about the confusion.
Let me add a few comments about tet refinement in mfem which may help explain some of the issues you encountered:

There are two ways to refine individual tets: using octa-section (a tet is directly split into 8 small tets), and using bisection (the tet is split into 2 tets three times to get 8 small tets).
Restricted to the faces, octa-section and bisection produce the following triangle refinements, respectively:

    *          *
   / \	      /|\
  *---*	     * | *
 / \ / \    / \|/ \
*---*---*  *---*---*

Octa-section will typically give rise to better shaped refined tets than bisection will.
For uniform refinement, octa-section is used by algorithm A (the default) and bisection is used by algorithm B.
Local conforming refinement uses bisection and needs tets to be marked for refinement (this marking is lost if one uses algorithm A). Algorithm B is basically this local refinement algorithm applied to all elements.
Local non-conforming refinement uses octa-section.
Tet bisection cannot be used with mixed meshes, e.g meshes with tets and prisms.


.png is not displayed

en-us:https://docs.microsoft.com/en-us/sharepoint/dev/solution-guidance/custom-field-type-sharepoint-add-in

ja-jp:https://docs.microsoft.com/ja-jp/sharepoint/dev/solution-guidance/custom-field-type-sharepoint-add-in

same page: #3460
Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 0efc70e7-88b1-0023-27bf-3d8b4568ef74
Version Independent ID: f54e0d1d-673b-16c5-7065-cad7c754ec97
Content: Custom field type in the SharePoint Add-in model
Content Source: docs/solution-guidance/custom-field-type-sharepoint-add-in.md
Product: sharepoint
GitHub Login: @spdevdocs
Microsoft Alias: spdevdocs

@andrewconnell should be fixed by my PR, please merge one


Is it really happening?

If this is indeed coming in Unity, I will just about explode from happiness. There's thousands of us waiting for this!
Could this possibly allow running graphics at 15fps, while still maintaining 60 fps input on mobile platforms? If yes, it's going to be absolutely huge!
(Sorry for this not actually being an issue, but rather a comment / encouragement to whoever is working on it)
Got here from this:
https://feedback.unity3d.com/suggestions/realtime-input-framerate-indepe
https://feedback.unity3d.com/suggestions/manually-activated-render-loop-for-lower-power-consumption-on-mobile

Hi,
Thanks for your enthusiasm.
The SRP project is only about flexible renderpipeline, it will not allow to manipulate input loop delta time. So no way to decouple input reading and graphic rendering frame rate, sorry.

@bitfabrikken On a semi-related note, the new input system is moving event collection to threads where it makes sense. This allows sampling things like touch on a higher-than-framerate frequency, for example. So in that sense, you could run your game at 15fps while sampling input at 60fps.

@Rene-Damm thanks Rene :) Do you know if there's a github repo I can follow for that input system?

@bitfabrikken We're getting there :) We've hit delays and snags but we're making steady progress towards that stage. There should be a publicly accessible github version in the 2017.1 time frame and a fully integrated version in 2017.2 (we're moving towards Unity being able to integrate packages from the web).


Only works in Anaconda Prompt on Windows

I've tested this on two Windows machines, and on both this only works properly from within the Anaconda Prompt as opposed to the normal command prompt. When running from the normal Windows command prompt I get errors about various DLLs missing:

libwinpthread-1.dll
libgcc_s_seh-1.dll
libquadmath-0.dll
libgfortran-3.dll

If those are added manually to the same folder that sixs.exe is located in, then it works fine.
I've done some investigation into this, compared the environment variables between the prompts and found that the Anaconda Prompt adds C:\Anaconda3\Library\mingw-w64\bin to the PATH variable. Interestingly, looking at activate.bat it seems that these come from running conda ..activate cmd.exe root - which is a command I'd never come across before!
When installing Anaconda I definitely ticked the box that added the relevant locations to the PATH variable, but of course the mingw-w64 path didn't exist then as I hadn't installed that package.
Is there any way around this? Can I somehow set my recipe to make sure the DLLs from mingw are copied to the right locations (eg. Anaconda3/Library/bin, which would already be in the PATH)?
Cc to @danclewley for previous experience of sixs on Windows, and @ocefpaf for conda expertise.

Cc to @danclewley for previous experience of sixs on Windows, and @ocefpaf for conda expertise.

@robintw that is the expected behavior with conda. I don't think Anaconda3/Library/bin is available in the "normal" command prompt unless you told anaconda to add it to the path, and many users don't do that to avoid conflicts.
My recommendation is to always open the "anaconda prompt" and also to work with envs to ensure the right env vars are activated in a safe way.

Was just writing this when @ocefpaf replied:
It works on a normal prompt if you install to a new environment and then activate this.
From some of the issues I've seen on https://github.com/conda-forge/gdal-feedstock I was under the impression the recommended practice was always to install to a new environment with forcing the activate script to be run as one of the reasons for this.

From some of the issues I've seen on https://github.com/conda-forge/gdal-feedstock I was under the impression the recommended practice was always to install to a new environment with forcing the activate script to be run as one of the reasons for this.

Yes. That is the best practice.

Yes. That is the best practice.

In this case @robintw, do you want to close this issue and make sure the Py6S install instructions recommend installing in a new environment.

Yup, sounds sensible.
Thanks both of you.


Sentry integration done in api module

Sentry Integration in Generic APIs module
BUG FIX; login endpoint was doing unnecessary encryption
BUG FIX; node version bumped to 16 causing broken app on 12

resolve #80

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Warning: This coverage report may be inaccurate.
This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

For more information on this, see Tracking coverage changes with pull request builds.
To avoid this issue with future PRs, see these Recommended CI Configurations.
For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

15 of 53 (28.3%)  changed or added relevant lines in 6 files are covered.
283 unchanged lines in 5 files lost coverage.
Overall coverage increased (+0.2%) to 24.253%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/interceptors/sentry.interceptor.ts
4
6
66.67%

src/main.ts
0
2
0.0%

src/api/sms/cdac/cdac.service.ts
1
8
12.5%

src/api/sms/gupshup/gupshup.service.ts
8
15
53.33%

src/api/fusionauth/fusionauth.service.ts
1
21
4.76%

Files with Coverage Reduction
New Missed Lines
%

src/main.ts
1
0.0%

src/api/sms/gupshup/gupshup.service.ts
14
55.34%

src/api/sms/cdac/cdac.service.ts
28
12.75%

src/api/fusionauth/fusionauth.service.ts
84
10.11%

src/api/api.service.ts
156
5.45%

Totals

Change from base Build<PHONE_NUMBER>:
0.2%

Covered Lines:
588

Relevant Lines:
2201

💛  - Coveralls


Slims down disposal unit ui and fixes thrown collisions

About the PR

Cleaned up disposal unit ui
Localized disposal unit ui
Fixed throwing objects into disposal unit
Organized the disposal/atmos pipe yaml

After looking at the mess that is disposal mailing unit ui and realizing it's broken in game too i've commented out the yaml for it as it's busted currently.
Closes #3875.
Screenshots

UI before

UI after

Changelog

:cl:

fix: Throwing entities into disposal units now work
tweak: Slims down disposal unit UI
remove: Disposal mailing units for now

Reviews applied.


Fix payout split still being available with specific target zero

What does this PR do and why?
Closes #905
Screenshots or screen recordings

Acceptance checklist

[ x ] I have evaluated the Approval Guidelines for this PR.
[ x ] I have tested this PR in all supported browsers.

Lovely @jmilldotdev LGTM


MarkerCluster and Marker interaction

I  tried out the new MarkerCluster capability in 0.5.3, and it works great in my notebook here:
https://gist.github.com/anonymous/d414d59e50e57e23436adba70fd48b35
but I'm struggling a bit with the interaction.  [BTW, this notebook should run anywhere, as it uses only web services to aquire data]
I was originally plotting markers using geoJSON, and the interaction I'm looking for is to return the name of  the marker to python when I click on it.   In cell [13], with geoJSON, I use feature_layer.on_click to specifying printing the name of the marker below the plot.  This works great:

I would like to do the same in cell [15], using MarkerCluster, but I cannot figure out how to get the name of the marker back on click:

There is no feature_layer.on_click for MarkerCluster, and I guess that make sense since I really just need interaction with the individual Marker objects, right?
But the Marker objects don't seem to have on_click either.  Would it be on_trait_change?

Should we add an on_click to Marker objects?

Yes, unless on_trait_change or another existing method can be used to return marker properties on click.

I think the appropriate thing is to add an on_click handler, mirroring the button on_click handler in ipywidgets, or perhaps the on_click handler already in ipyleaflet.
What to give it a shot? It looks like the code is at https://github.com/ellisonbg/ipyleaflet/blob/0aa98732f3ca22fa6c442c5f1487b3aaba470710/ipyleaflet/leaflet.py#L427 and https://github.com/ellisonbg/ipyleaflet/blob/0aa98732f3ca22fa6c442c5f1487b3aaba470710/ipyleaflet/leaflet.py#L440 on the python side and the relevant js code is around https://github.com/ellisonbg/ipyleaflet/blob/0aa98732f3ca22fa6c442c5f1487b3aaba470710/js/src/jupyter-leaflet.js#L282

(here is the part in js that sends a message to the python side: https://github.com/ellisonbg/ipyleaflet/blob/0aa98732f3ca22fa6c442c5f1487b3aaba470710/js/src/jupyter-leaflet.js#L289)

@jasongrout , ah, yes, an exercise for the student!   :smiley_cat:
Okay, perhaps @ocefpaf and I can take a stab at it!

Okay, perhaps @ocefpaf and I can take a stab at it!

@rsignell-usgs I'll be offline for 2-3 days but after that I can give it a shot.

@ocefpaf did you have the occasion to look at it?
We made some really good progress over the past couple of weeks, especially in cleaning up the general code base, and setting up the JupyterLab extension etc...

Sorry @SylvainCorlay, I did not look at it. I don't think I'll have time for this anytime soon :-(

There is a click event in Lealfet https://leafletjs.com/reference-1.3.4.html#marker-click, so adding an on_click method to marker should be straightforward, the implementation will be like the on_move method.
And actually, because the click event is an event inherited from Interactive Layer, we should be able to add this method directly to the Layer widget, and implementing all the mouse events should be straightforward too (dblclick, mousedown, mouseup, mouseover...).

Is there any workaround to add this functionality now?
I'm trying with on_move as a workaround but seems not triggered, too

I can't think of a workaround... on_move is not triggered? Do you have any error message?

Well, I was trying to turn my markers into GeoJSON to use the on_* handler there, but I'm not receiving any event here too. I think it is a problem on my side, the "CountriesGeoJSON.ipynb" demo works nicely

I wouldn't be surprised if GeoJSON events are not triggered properly... There are already some issues opened related to GeoJSON events, we need to give more love to this layer.
We'll work on the mouse events for the Layer widget this week, and I'll make a release of ipyleaflet on Friday.

I've been doing some testing and the on_move event on Marker but seems not triggering even in a modified version of Primitives.ipnb
I've also tried to subscribe to on_msg but no messages there, too

I just tried it on the binder (last version of ipyleaflet) and on_move seems to work. Do you mind giving me an example so that I can replicate your issue?

by adding *args, **kwargs to callback signature is working for me too!
I was not receiving any error and I've been trying with the callback signatures documented in the help(...)! It is misleading, but I'm happy to be able to continue now.
Thanks a lot

Cool :) Actually the callback should take parameters, you can print args and kwargs to see what is given. I guess it's a dictionary containing the new position of the Marker but I'm not entirely sure. So I you pass a callback that doesn't take parameters, it throws an exception, that doesn't get displayed in the notebook.

It makes sense, but it was outside the common python behaviour (I'm new to jupyter). Now I'm working with functools.partial to keep a reference of the triggered Marker in the callback and it works, problem is that the callback seems before the update if the location on the leaflet side (I'm trying to push back the Marker on the original position as a workaround for on_click)

Annoying.. The callback is called before the location attribute has been updated, so your workaround won't work.. Do you mind waiting for a release? I'll add the mouse events like on_click tomorrow, and we'll make a release on Friday.

Absolutely no problem, I was just trying to understand to (hopefully) contribute.
In the meantime I've re-tested the on_msg callback on Marker looking for any on_click feedback but seems not there, must be on the upper layer.
Thank you for taking care of this issue.

I was just trying to understand to (hopefully) contribute

That would be great! I created a github project today https://github.com/jupyter-widgets/ipyleaflet/projects/1. I added in the TODO list issues that seem to be easy enough for new contributors. If you start working on one of the issues I would be happy to help you.
Concerning the on_msg callback, you won't receive any click message, because only a move message is sent from the front-end https://github.com/jupyter-widgets/ipyleaflet/blob/master/js/src/jupyter-leaflet.js#L168.


rerun.txt

Hi. I'm sharing folders between windows and linux and when using guard-cucumber I get this error:
ERROR: Guard::Cucumber failed to achieve its , exception was:
Errno::ETXTBSY: Text file busy - rerun.txt
.../guard-cucumber-0.7.2/lib/guard/cucumber.rb:114:in delete
.../guard-cucumber-0.7.2/lib/guard/cucumber.rb:114:in read_failed_features
.../guard-cucumber-0.7.2/lib/guard/cucumber.rb:60:in run_all
...
Guard::Cucumber has just been fired
(running a rm rerun.txt is working fine)
It can be avoided using :keep_failed => false, but I'd be cool if we can fix it.

I think the issue was that the file was not properly closed. I switched to a block for reading rerun.txt, so that the file automatically gets properly closed when the block is exited.
This fix is in released in 0.7.3.

It is working with the 0.7.3, thanks

Glad to hear! Thanks for the feedback.


Add support for o1

why
Add support for OpenAI's most powerful models
what changed
Add o1-mini and o1-preview to OpenAIClient and AvailableModel. Added o1 to experimental runs to avoid slowing down CI
test plan
npm run evals with:
const models: AvailableModel[] = ["o1-mini", "o1-preview"];

in index.eval.ts. No need to slow down CI with o1.
Braintrust Run for o1

Merging without waiting for CI to pass since evals passed (see below) on a95d413 and only edit made after was to CI


Split the bundle in chunks

Going in the direction of https://github.com/jupyterlite/xeus-python-kernel/issues/54

This PR requires https://github.com/emscripten-forge/empack/pull/52


Models With Different Names

I have changed the models name in Models\Role.php and Models\Permission.php
for example:
i change the table name of Role.php to G_roles and Permission.php to G_permissions,
using protected table name.
in migrations i did the same thing i prepend a "G_" to all table names, now when  i use
ShibobiTrait in UserModel laravel throw the next error:
(4/4) ErrorException
SQLSTATE[42S02]: [Microsoft][ODBC Driver 11 for SQL Server][SQL Server]El nombre de objeto 'role_user' no es válido. (SQL: select [G_roles].*, [role_user].[user_id] as [pivot_user_id], [role_user].[role_id] as [pivot_role_id], [role_user].[created_at] as [pivot_created_at], [role_user].[updated_at] as [pivot_updated_at] from [G_roles] inner join [role_user] on [G_roles].[id] = [role_user].[role_id] where [role_user].[user_id] = 1) (View: C:\xampp\htdocs\G_project\resources\views\layouts\app.blade.php) (View: C:\xampp\htdocs\G_project\resources\views\layouts\app.blade.php)
And is confusing because it supossed to be G_role_user not role_user.
I appreciate any help you can give me.
Sorry for the english, by the way in new on github so i dont know if this is the correct place to ask.

:+1: I'll need to add config options to customize the table names for relationships - currently it's relying on Laravel to auto-guess what it should be - I don't think Laravel uses the table property in your models to determine this, but your class name directly.

sorry - what version of shinobi are you using here?

sorry - what version of shinobi are you using here?

shinobi 3.3
laravel 5.4
Now i'm migrating the project to laravel 5.7


[SPARK-24994][SQL][FOLLOW-UP] Handle foldable, timezone and cleanup

What changes were proposed in this pull request?

This is a follow-up on #29565, and addresses a few issues in the last PR:

style issue pointed by this comment
skip optimization when fromExp is foldable (by this comment) as there could be more efficient rule to apply for this case.
pass timezone info to the generated cast on the literal value
a bunch of cleanups and test improvements

Originally I plan to handle this when implementing SPARK-32858 but now think it's better to isolate these changes from that.
Why are the changes needed?

To fix a few left over issues in the above PR.
Does this PR introduce any user-facing change?

No
How was this patch tested?

Added a test for the foldable case. Otherwise relying on existing tests.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/33401/
Test PASSed.

Test build #128778 has started for PR 29775 at commit 4291ec2.

cc @cloud-fan and @dongjoon-hyun

Thank you for this follow-up and pinging me, @sunchao .

Test build #128778 has finished for PR 29775 at commit 4291ec2.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128778/
Test PASSed.

pass timezone info to the generated cast on the literal value

I'd also like to understand this more. Cast.canonicalize will drop the timezone if it's not needed, so UnwrapCastInBinaryComparison shouldn't care about timezone.

Test build #128799 has started for PR 29775 at commit 3a0f4b0.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/33422/
Test PASSed.

Test build #128799 has finished for PR 29775 at commit 3a0f4b0.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128799/
Test FAILed.

retest this please

Test build #128802 has started for PR 29775 at commit 3a0f4b0.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/33425/
Test PASSed.

Test build #128802 has finished for PR 29775 at commit 3a0f4b0.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128802/
Test FAILed.

retest this please

Test build #128823 has started for PR 29775 at commit 3a0f4b0.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/33446/
Test PASSed.

Test build #128823 has finished for PR 29775 at commit 3a0f4b0.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128823/
Test PASSed.


Release 0.14.5 again

This time properly bumping the version number in Cargo.toml.

@bors r+

:pushpin: Commit c97a94f0305257b9c603db73f5ad11bba7a62265 has been approved by Amanieu
It is now in the queue for this repository.

:hourglass: Testing commit c97a94f0305257b9c603db73f5ad11bba7a62265 with merge 835936584e33480f6afbf1ae9be7151c1b27d595...

:sunny: Test successful - checks-actions
Approved by: Amanieu
Pushing 835936584e33480f6afbf1ae9be7151c1b27d595 to master...


list deps: cut off the downstream parsing if Files or UpStreams is empty

1. Does this PR affect any open issues?(Y/N) and add issue references (e.g. "fix #123", "re #123".):

[X] N
[ ] Y

2. What is the scope of this PR (e.g. component or file name):
pkg/tools/list/list.go

3. Provide a description of the PR(e.g. more details, effects, motivations or doc link):
minor enhancement on ListUpStreamFiles/ListDownStreamFiles API: the file parsing can be cut off when the user option values of Files/UpStreams are empty.

4. Are there any breaking changes?(Y/N) and describe the breaking changes(e.g. more details, motivations or doc link):

[X] N
[ ] Y

5. Are there test cases for these changes?(Y/N) select and add more details, references or doc links:

[ ] Unit test
[ ] Integration test
[ ] Benchmark (add benchmark stats below)
[ ] Manual test (add detailed scripts or steps below)
[ ] Other

6. Release note

Please refer to Release Notes Language Style Guide to write a quality release note.
None

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 2 (0.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 48.041%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pkg/tools/list/list.go
0
2
0.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
2538

Relevant Lines:
5283

💛  - Coveralls


Select: case-insensitive search

could you please provide case-insensitive search for <n3-select search> element, i believe that would improve UX a lot.

it'd be as easy as add i to regexp here:
https://github.com/N3-components/N3-components/blob/master/src/Select/n3Select.vue#L299

OK,I will add a prop to control

please check version 2.4.0


Crash in  RN 0.53 iOS

'[<RCTTextView 0x7fa5b0c91a30> valueForUndefinedKey:]: this class is not key value coding-compliant for the key _backedTextInput.'
*** First throw call stack:

should be fixed now with version 5.2.0


Fix error in docker-compose.yml, migrations.restart needs to be string

Here was the error I had,
~/nsecbunkerd$ docker-compose up -d
ERROR: The Compose file './docker-compose.yml' is invalid because: services.migrations.restart contains an invalid type, it should be a string dentropy@umbrel:~/nsecbunkerd$ vim docker-compose.yml

uh this is still broken


[Bug]  Hand::new_from_str isn't deterministic for card ordering

Running
assert_eq!(Hand::new_from_str("AdKh").unwrap(), Hand::new_from_str("AdKh").unwrap());

sometimes works and sometimes fails with:
assertion `left == right` failed
left: Hand([Card(Kh), Card(Ad)])
right: Hand([Card(Ad), Card(Kh)])

So the cards vector isn't sorted. I think if they are sorted by value and suit, this should not occur anymore.

I've thought about this more, and the eq trait is correct. Hand-card order does matter. (Betting can run very differently depending on ordering.) The bug comes from the fact that new_from_str isn't deterministic.

I can try to fix it, if you like 🙂

I think the problem is that the Hand is converted from a HashSet to a Vector. HashSet has a random seed on init, so the order will always be different.
I agree that the order matters.
As far as I see it, the only reason to use the HashSet is to have the duplicate check on insertion. But we could easily use a vector in the first place and still have a duplicate check.
If that is fine for you I could contribute a fix for it 🙂


Yaml support

Can this be extended to support yaml files?

I'd imagine that not as easily, as YAML allows for much more flexibility and types. I'll try to see if I can come up with a satisfying solution.

:wave: Hi,
I have a working patch for yaml. I don't know if it's 100% comprehensive, but it works well enough for me (displays maps and lists appropriately)
Happy to open a PR if you're interested.

Sure! I'll take a look at it


Custom ffmpeg command line options for maxrate and bufsize

I need to pass -maxrate and -bufsize to the ffmpeg command inside the docker, but I cannot find the source files of where Telly is calling ffmpeg.
I tried just making a bash file at /usr/local/bin/ffmpeg to call the real bin, but I'm having trouble manipulating the passed arguments and getting -maxrate and -bufsize to work.
It would be far simpler if Telly had this as a config option, but I am fine doing it the above way if I could just get a little help on the arguments coming fro Telly, or where the source file is that is calling ffmpeg so I can modify it directly.

Got it :)
#!/bin/sh
#echo "$@" >/dev/stderr
for last; do true; done
/usr/local/bin/ffmpeg-real -re -i $3 -codec copy -b:v 3000k -bufsize 3000k -f mpegts pipe:1

Agreed, it would be nice for this to be user editable.  However, it is easy enough to rebuild it as an alternative to your example :)
For anyone else...
First, setup go, latest/current version (not an old repo version)
git clone https://github.com/tellytv/telly -b dev
Edit routes.go, line "run := exec.Command("ffmpeg"....." to your liking
make

this would be nice; because then could pass through commands to utilize GPU if available/wanted.


Remove "omitempty" tag from Encryption field for HTTP checks

Currently we're unable to disable SSL/TLS validation when adding an uptime_check_http resource type using the Uptime Terraform Provider.
For example, the following will result in setting "encryption" to "SSL_TLS" since an empty string is omitted by the go client.
resource "uptime_check_http" "http_internal_dev_server_8101" {
  name           = "http_internal_dev_server_8101"
  address        = "https://server:8101"
  port           = "8101"
  tags           = ["someTag"]
  encryption     = ""
  interval       = 3
}

A similar fix was added in https://github.com/uptime-com/uptime-client-go/pull/13

@gigovich can you please review this contribution?

@aderholdtb thank you!


No face detected, aborting

Please describe the issue in as much detail as possible, including any errors and traces
I type sudo howdy -U daniel add
and then leave the label for the new model blank
I then get the error

No face detected, aborting

So I type sudo howdy test and this is what I see

I've searched for similar issues already, and I think https://github.com/boltgolt/howdy/issues/158 is talking about the same issue
Linux distribution (if applicable): Linux Mint 19.1
Howdy version: I believe 2.5.1
I have a Lenovo Thinkpad x1 carbon 6th generation

Thanks for looking around, but i think #147 is very similar to you (the noise in the screenshots is almost exactly the same) and already includes a solution. If trying different resolutions does not work, let me know.


Listen for all events, not just click

Initial commit: https://github.com/adamghill/django-unicorn/commit/a31ede98d2f5ff229d99b27c1ff86b6e347e933a. Might need more testing, though.

Looks good!