Rich text editor: use image for video preview

In https://github.com/opencollective/opencollective-frontend/pull/6042, we're adding support for embed videos. There's a small issue where the iframe reloads itself whenever you put a space or a new line. From @SudharakaP:

This seems to be the default behaviour of the Trix editor unfortunately and currently for content attachments the iframes are reloaded when the DOM changes. The following comments points to that; basecamp/trix#206 (comment). One thing we could do is as they suggest only show an image preview on the editor and then when the content is submitted show the real video.

This will also be useful to embed videos in Updates, since we can't render iframes in emails.

Is this done @SudharakaP ?

Is this done @SudharakaP ?

This is with @Betree for the review. I'll follow up on this, sorry for the wait. 😢


[core] add HttpRequest and HttpResponse reprs

fixes #16952

/azp run python - core - ci

/azp run python - cosmos - ci

thanks @kristapratico , @johanste and @lmazuel have already given their verbal approval


Feature Request: Add Events

It would be nice if creating ActivityLogs threw Events on these actions:

ActivityLogger::log()
CleanActivitylogCommand::handle()

It would allow people to create listeners to easily create notifications when certain kinds of ActivityLog events occur.

You can already do this. You could use a creating model event on the activity_log table.  And I might be wrong, but I think there's also an event that Laravel fires when an artisan command is being executed. If there is no such event, it might be worth to PR it to Laravel.

@freekmurze Can I use updating or creating event to logging activity and prevent save data? I want to prevent some user roles save data but I want to log attemps to do it.

@mateo2181 the eventsToBeRecorded() method is the solution to also log other actvities by model events:
https://github.com/spatie/laravel-activitylog/blob/3bee85adf8e19e3bc0cafab56599a442014a9169/src/Traits/LogsActivity.php#L21-L52
So you can define static::$recordEvents with any events you want to record. Keep in mind that the old values are only recorded in an updated event. So in your `updating activity you will only have the possible new values.

Thanks! I have read the trait and understood how it works. I  have created another trait to keep clean code and use it only in some cases. I have another question: Can I only save attributes with new values? I mean, sometimes in properties field I have attributes with same value in array old and attributes.

I have read this: #513. I will try to fix it.

In general this is done by $logOnlyDirty = true;. But like you already linked this doesn't work if you use a model with partial selection.


[ie/stv] Add DRM account id

IMPORTANT: PRs without the template will be CLOSED
Description of your pull request and other information

Adds correct Brightcove account ID for DRM protected videos. Previously rejected in #5834 for explicitly supporting protected content. Appealing the rejection as the codebase has many cases of only calling report_drm() without --allow-unplayable-formats.
Template

Before submitting a pull request make sure you have:

[x] At least skimmed through contributing guidelines including yt-dlp coding conventions
[x] Searched the bugtracker for similar pull requests

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check all of the following options that apply:

[x] I am the original author of this code and I am willing to release it under Unlicense
[ ] I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

[x] Fix or improvement to an extractor (Make sure to add/update tests)
[ ] New extractor (Piracy websites will not be accepted)
[ ] Core bug fix/improvement
[ ] New feature (It is strongly recommended to open an issue first)

You already got your answer last time. We will not add additional code to yt-dlp to enable downloading of DRM protected media. Circumventing copy protection is illegal in most jurisdictions and making such additions would put the project at risk. This is non negotiable.

@seproDev I see what you mean now. You really should have made it clear that your point was "additional code". That was not made clear last time (and not that clear this time either).  Maybe it's to do with how long ago the code was merged. Maybe you're deciding based on what country the site's from. Maybe it's as specific as not wanting the ID of a Brightcove account that only has protected videos hardcoded. You can have whatever policy you want and even believe it actually makes any difference legally. Just like how you can't state specifically what your policy is (or respond in any way) in case it makes a difference legally.
Just don't tell me "You already got your answer last time." I asked for why some cases are allowed and some cases aren't. And you don't allow youselves to give the reason.

The allow_unplayable_formats parameter and most related code will be removed from the codebase soon.

The allow_unplayable_formats parameter and most related code will be removed from the codebase soon.

@bashonly That's great to hear. You've restored some respect for yourselves rather than having a willy-nilly policy on DRM that helps no one.


Fix memory leak in ViewGroupDescriptor

This switches ViewGroupDescriptor to using a WeakHashMap for the View -> [View or Element] cache.
I also moved this cache out of ViewGroupDescriptor.ElementContext and into ViewGroupDescriptor because we don't really need to cache this per-ViewGroup. Why not save a few allocations per ViewGroup?
Closes #203

lgtm, merge at will


[PDI-11595] - Backward compatibility insurance

Build Completed
:white_check_mark: This pull request has passed all validations.
Build Commands
ant -Dtestreports.dir=bin/reports/unit-test -f core/build.xml clean-all resolve jacoco && ant -f core/build.xml checkstyle publish-local

Cleanup Commands
rm -r ~/.ivy2/local || echo no publish local to remove

Changed files
  core/src/org/pentaho/di/core/database/DatabaseInterface.java


Do you have a docker-compose.yml ?

Sorry I saw your image in Docker Hub https://hub.docker.com/r/javanile/myddleware/dockerfile
Do you have a docker-compose.yml ?

This is a work in progress project! Are you interested to develop with myddleware or just use it?

No it is just my interest. Because I want to deploy in docker.


[Bug]: LoRa adapter responses not matching peft/transformers response

Your current environment

The output of `python collect_env.py`
PyTorch version: 2.5.1+cu124
Is debug build: False
CUDA used to build PyTorch: 12.4
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.3 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] (64-bit runtime)
Python platform: Linux-6.8.0-48-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 12.1.105
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA A40
Nvidia driver version: 550.127.05
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                         x86_64
CPU op-mode(s):                       32-bit, 64-bit
Address sizes:                        52 bits physical, 57 bits virtual
Byte Order:                           Little Endian
CPU(s):                               96
On-line CPU(s) list:                  0-95
Vendor ID:                            GenuineIntel
Model name:                           Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz
CPU family:                           6
Model:                                106
Thread(s) per core:                   2
Core(s) per socket:                   24
Socket(s):                            2
Stepping:                             6
CPU max MHz:                          3500.0000
CPU min MHz:                          800.0000
BogoMIPS:                             5600.00
Flags:                                fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local split_lock_detect wbnoinvd dtherm ida arat pln pts vnmi avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg tme avx512_vpopcntdq la57 rdpid fsrm md_clear pconfig flush_l1d arch_capabilities
Virtualization:                       VT-x
L1d cache:                            2.3 MiB (48 instances)
L1i cache:                            1.5 MiB (48 instances)
L2 cache:                             60 MiB (48 instances)
L3 cache:                             72 MiB (2 instances)
NUMA node(s):                         2
NUMA node0 CPU(s):                    0-23,48-71
NUMA node1 CPU(s):                    24-47,72-95
Vulnerability Gather data sampling:   Mitigation; Microcode
Vulnerability Itlb multihit:          Not affected
Vulnerability L1tf:                   Not affected
Vulnerability Mds:                    Not affected
Vulnerability Meltdown:               Not affected
Vulnerability Mmio stale data:        Mitigation; Clear CPU buffers; SMT vulnerable
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed:               Not affected
Vulnerability Spec rstack overflow:   Not affected
Vulnerability Spec store bypass:      Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:             Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:             Mitigation; Enhanced / Automatic IBRS; IBPB conditional; RSB filling; PBRSB-eIBRS SW sequence; BHI SW loop, KVM SW loop
Vulnerability Srbds:                  Not affected
Vulnerability Tsx async abort:        Not affected

Versions of relevant libraries:
[uv] numpy==1.26.2
[uv] nvidia-cublas-cu12==<IP_ADDRESS>
[uv] nvidia-cuda-cupti-cu12==12.1.105
[uv] nvidia-cuda-nvrtc-cu12==12.1.105
[uv] nvidia-cuda-runtime-cu12==12.1.105
[uv] nvidia-cudnn-cu12==<IP_ADDRESS>
[uv] nvidia-cufft-cu12==<IP_ADDRESS>
[uv] nvidia-curand-cu12==<IP_ADDRESS>
[uv] nvidia-cusolver-cu12==<IP_ADDRESS>
[uv] nvidia-cusparse-cu12==<IP_ADDRESS>
[uv] nvidia-nccl-cu12==2.18.1
[uv] nvidia-nvjitlink-cu12==12.3.101
[uv] nvidia-nvtx-cu12==12.1.105
[uv] pyzmq==24.0.1
[uv] torch==2.1.1
[uv] torchaudio==2.1.1
[uv] torchvision==0.16.1
[uv] transformers==4.46.3
[uv] triton==2.1.0
[conda] Could not collect
ROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: 0.6.4.post1
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
GPU0    CPU Affinity    NUMA Affinity   GPU NUMA ID
GPU0     X      0-23,48-71      0               N/A

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

LD_LIBRARY_PATH=/workspace/ADVANCED-inference/lora-server/.venv/lib/python3.10/site-packages/cv2/../../lib64:
CUDA_MODULE_LOADING=LAZY

Model Input Dumps
No response
🐛 Describe the bug
Issue: LoRa adapter responses with vLLM do not match peft/transformer responses.
The reproduction involves running inference using a) vllm, compared to b) running inference with transformers and peft. I have run both on A40 machines on runpod.
vLLM approach
vLLM server startup with statically loaded LoRa
ADAPTER_REPO="Trelis/Qwen2.5-7B-Instruct-touch-rugby-1"
ADAPTER_PATH=$(python3 -c "from huggingface_hub import snapshot_download; print(snapshot_download('${ADAPTER_REPO}', ignore_patterns=['model-*', 'pytorch_model*', 'tf_model*', 'flax_model*']))")

# Start server with LoRA pre-loaded
vllm serve Qwen/Qwen2.5-7B-Instruct \
    --max-model-len 8192 \
    --enable-lora \
    --max-lora-rank 8 \
    --port 8000 \
    --lora-modules "{\"name\": \"touch-rugby-1\", \"path\": \"${ADAPTER_PATH}\", \"base_model_name\": \"Qwen/Qwen2.5-7B-Instruct\"}"

vLLM Script to call the server endpoint, aka vllm_replication.py
import requests
import json
from typing import List, Dict, Any

def send_chat_completion(
    messages: List[Dict[str, str]],
    model: str,
    base_url: str = "http://localhost:8000"
) -> Dict[str, Any]:
    """Send a chat completion request to the vLLM server."""
    try:
        response = requests.post(
            f"{base_url}/v1/chat/completions",
            headers={"Content-Type": "application/json"},
            json={
                "model": model,
                "messages": messages,
                "max_tokens": 100,
                "temperature": 0.01,
            }
        )
        response.raise_for_status()  # Raise an error for bad status codes
        return response.json()
    except requests.exceptions.RequestException as e:
        print(f"Error making request: {e}")
        print(f"Response content: {response.text if 'response' in locals() else 'No response'}")
        return None

def main():
    # Configuration
    base_model = "Qwen/Qwen2.5-7B-Instruct"
    adapter_name = "touch-rugby-1"  # This matches the name in your vLLM server command

    # Test prompt (same as in your transformers example)
    messages = [{
        "role": "user",
        "content": "How many players are on the field on each team at the start of a drop-off?"
    }]

    # Generate with base model
    print("\nBase Model Response:")
    base_response = send_chat_completion(messages, base_model)
    if base_response and 'choices' in base_response:
        print(f"Base Model Output: {base_response['choices'][0]['message']['content']}")
    else:
        print("Failed to get base model response")

    # Generate with adapter
    print("\nAdapter Model Response:")
    adapter_response = send_chat_completion(messages, adapter_name)
    if adapter_response and 'choices' in adapter_response:
        print(f"Adapter Output: {adapter_response['choices'][0]['message']['content']}")
    else:
        print("Failed to get adapter response")

if __name__ == "__main__":
    main()

Transformers / PEFT script
# Install required packages if needed
!pip install transformers peft accelerate -qU

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load the base model and tokenizer
base_model_id = "Qwen/Qwen2.5-7B-Instruct"
adapter_id = "Trelis/Qwen2.5-7B-Instruct-touch-rugby-1"

tokenizer = AutoTokenizer.from_pretrained(base_model_id, trust_remote_code=True)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)

# Test prompt
prompt = "How many players are on the field on each team at the start of a drop-off?"

# Function to generate response
def generate_response(model, prompt):
    messages = [
        {"role": "user", "content": prompt}
    ]

    text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

    inputs = tokenizer(text, return_tensors="pt").to(model.device)

    outputs = model.generate(
        **inputs,
        max_new_tokens=100,
        temperature=0.01,
        do_sample=True,
    )

    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
    return response

# Generate with base model
print("Base Model Response:")
base_response = generate_response(base_model, prompt)
print(base_response)

# Load adapter and generate
print("\nLoading adapter...")
adapter_model = PeftModel.from_pretrained(base_model, adapter_id)

print("\nAdapter Model Response:")
adapter_response = generate_response(adapter_model, prompt)
print(adapter_response)

# Free up memory
del base_model
del adapter_model
torch.cuda.empty_cache()

Results / Output
vLLM

The adapter response does not take on the adapter/fine-tune attributes). It does not know what sport the question is about or what a drop-off is.

uv run vllm-replication.py

Base Model Response:
Base Model Output: It seems there might be a bit of confusion in your question. The term "drop-off" is not commonly used in sports, especially not in team sports like football, basketball, or soccer. Could you please clarify which sport you are referring to?

For example:
- In soccer (football), there are 11 players on the field for each team at the start of a match.
- In American football, there are 11 players on the field for each team at the start of

Adapter Model Response:
Adapter Output: In the context of Australian rules football, which is the sport referred to as "drop-off," each team starts with 18 players on the field at the beginning of the game. This number of players is standard for a match and includes both forwards, midfielders, and defenders.

Transformers / peft Response

The model knows what a "drop-off" is and that this question is about touch rugby.

Base Model Response:
system
You are Qwen, created by Alibaba Cloud. You are a helpful assistant.
user
How many players are on the field on each team at the start of a drop-off?
assistant
It seems there might be some confusion in your question. The term "drop-off" is not commonly used in sports to describe the number of players on a field or court. Could you please clarify which sport you are referring to?

For example:
- In soccer (football), there are 11 players on the field for each team.
- In American football, there are typically 11 players on the field for each team at the start of a play.
- In basketball, there are

Loading adapter...

Adapter Model Response:
system
You are Qwen, created by Alibaba Cloud. You are a helpful assistant.
user
How many players are on the field on each team at the start of a drop-off?
assistant
To determine how many players are on the field at the start of a drop-off, we can follow these steps:

1. **Understand the Composition of a Team**: Each team consists of 14 players, including the Interchange.

2. **Interchange Rules**: The Interchange is allowed to enter and leave the field during normal play without a Change of Possession (COP).

3. **Drop-Off Procedure**: A Drop-Off occurs when one team has fewer than six (

Additional Notes

Base models are loaded in bfloat16 in both cases
The very same adapters and base models are used from Huggingface
The adapter is public and you can check it here
I also have tried with the same fine-tuning approach on Llama 3.1 8B (with ranks of 8 and also 32).
I intentionally used a low rank of 8 here in case somehow that gave issues.

Questions

Are there any known issues with loading safetensors? I don't see anything in the issues?
Are there limits on what modules can be trained? I only trained linear layers, which should be fine:

"target_modules": [
  | "gate_proj",
  | "k_proj",
  | "v_proj",
  | "down_proj",
  | "q_proj",
  | "o_proj",
  | "up_proj"
  | ],

Before submitting a new issue...

[X] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

I've only taken a preliminary look at your issue, I suggest first upgrading Triton to around version 3.1.0

Thanks @jeejeelee , I just tried with triton 3.0.0 and got the same thing. Can try 3.1.0 if you think worth it.
uv run collect_env.py
Collecting environment information...
PyTorch version: 2.5.1+cu124
Is debug build: False
CUDA used to build PyTorch: 12.4
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.5 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.11.10 (main, Sep  7 2024, 18:35:41) [GCC 11.4.0] (64-bit runtime)
Python platform: Linux-6.5.0-35-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 12.4.131
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA A40
Nvidia driver version: 550.127.08
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Address sizes:                      52 bits physical, 57 bits virtual
Byte Order:                         Little Endian
CPU(s):                             96
On-line CPU(s) list:                0-95
Vendor ID:                          GenuineIntel
Model name:                         Intel(R) Xeon(R) Gold 6342 CPU @ 2.80GHz
CPU family:                         6
Model:                              106
Thread(s) per core:                 2
Core(s) per socket:                 24
Socket(s):                          2
Stepping:                           6
CPU max MHz:                        3500.0000
CPU min MHz:                        800.0000
BogoMIPS:                           5600.00
Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 invpcid_single ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local split_lock_detect wbnoinvd dtherm ida arat pln pts vnmi avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg tme avx512_vpopcntdq la57 rdpid fsrm md_clear pconfig flush_l1d arch_capabilities
Virtualization:                     VT-x
L1d cache:                          2.3 MiB (48 instances)
L1i cache:                          1.5 MiB (48 instances)
L2 cache:                           60 MiB (48 instances)
L3 cache:                           72 MiB (2 instances)
NUMA node(s):                       2
NUMA node0 CPU(s):                  0-23,48-71
NUMA node1 CPU(s):                  24-47,72-95
Vulnerability Gather data sampling: Mitigation; Microcode
Vulnerability Itlb multihit:        Not affected
Vulnerability L1tf:                 Not affected
Vulnerability Mds:                  Not affected
Vulnerability Meltdown:             Not affected
Vulnerability Mmio stale data:      Mitigation; Clear CPU buffers; SMT vulnerable
Vulnerability Retbleed:             Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass:    Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:           Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:           Mitigation; Enhanced / Automatic IBRS; IBPB conditional; RSB filling; PBRSB-eIBRS SW sequence; BHI Syscall hardening, KVM SW loop
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Not affected

Versions of relevant libraries:
[uv] numpy                             1.26.3
[uv] nvidia-cublas-cu12                <IP_ADDRESS>
[uv] nvidia-cuda-cupti-cu12            12.4.99
[uv] nvidia-cuda-nvrtc-cu12            12.4.99
[uv] nvidia-cuda-runtime-cu12          12.4.99
[uv] nvidia-cudnn-cu12                 <IP_ADDRESS>
[uv] nvidia-cufft-cu12                 <IP_ADDRESS>
[uv] nvidia-curand-cu12                <IP_ADDRESS>
[uv] nvidia-cusolver-cu12              <IP_ADDRESS>
[uv] nvidia-cusparse-cu12              <IP_ADDRESS>
[uv] nvidia-nccl-cu12                  2.20.5
[uv] nvidia-nvjitlink-cu12             12.4.99
[uv] nvidia-nvtx-cu12                  12.4.99
[uv] pyzmq                             24.0.1
[uv] torch                             2.4.1+cu124
[uv] torchaudio                        2.4.1+cu124
[uv] torchvision                       0.19.1+cu124
[uv] triton                            3.0.0
[conda] Could not collect
ROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: 0.6.4.post1
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
GPU0    CPU Affinity    NUMA Affinity   GPU NUMA ID
GPU0     X      24-47,72-95     1               N/A

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

LD_LIBRARY_PATH=/workspace/ADVANCED-inference/lora-server/.venv/lib/python3.11/site-packages/cv2/../../lib64:
CUDA_MODULE_LOADING=LAZY

No need, I'll spend some time looking into this issue later

@RonanKMcGovern  I think the cause is due to rslora. Currently, we do not have support for this feature.

Thanks @jeejeelee , do you recommend just setting use_rslora to false then in the adapter_config.json ?
Anything else you recommend changing?
Having used rslora just means that a scaling factor was applied to the learning rate during training. I don't believe it should affect how weights are loaded at inference.

Thanks @jeejeelee , do you recommend just setting use_rslora to false then in the adapter_config.json ?

Not,  if you set use_rslora to false, IIUC, you have to train your model again.

Anything else you recommend changing?
Having used rslora just means that a scaling factor was applied to the learning rate during training. I don't believe it should affect how weights are loaded at inference.

See: https://github.com/vllm-project/vllm/pull/6909#issue-2436147142

Confirming this issue is caused by rs_lora being used. The hacky fix is to multiply alpha in the lora config by the rank.
See https://github.com/vllm-project/vllm/pull/6909


Improve error message for window overflow

Addresses #51
I don't want to start hacking people's config on their behalf, so just surface an error message that explains the problem and suggests the solution.
@demmer @VladVega

+1


Web UX: Photo gallery navigation

To improve navigation of the photo gallery on web or large desktop app window sizes, add:

Keyboard navigation (arrow left/right)
Physical left/right arrow buttons, bottom-center of the window.

Fixed in #152


Inplace add and filtered lookup

We have an issue with inplace add and cascaded lookup. Serenity seems to ignore ‘InplaceAdd = true’ for fields that have a filtered lookup. In our example we want to inplace add a member which is filtered by member type (see screenshot)

Sorry @allema but i can't reproduce this.
Tried it in Northwind Customers:
        [LookupEditor(typeof(Scripts.CustomerCountryLookup))]
        public String Country { get; set; }
        [LookupEditor(typeof(Scripts.CustomerCityLookup),
              CascadeFrom = "Country", CascadeField = "Country", InplaceAdd = true)]
        public String City { get; set; }

Are you sure you don't have a custom editor, or some other thing that overrides InplaceAdd, e.g. duplicate editor attributes on row and form?

Thanks, was indeed a duplicate editor attribute.

I have neither CascadeField nor CascadeFrom attributes in lookupEditor in my project.
Even in NorthWind Customers.
[Width(130), LookupEditor(typeof(Scripts.CustomerCountryLookup)), QuickFilter]         public String Country { get; set; }         [Width(120), LookupEditor(typeof(Scripts.CustomerCityLookup))]         [QuickFilter, QuickFilterOption("cascadeFrom", "Country")]         public String City { get; set; }         [Width(120)]
Is this feature (QuickFIlterOption) the new way to filter a LookupEditor or am I focusing on the wrong file (CustomerColumns.cs) ?


Use mac-notification-sys from crates.io

I would ❤ to merge this, but travis says no - so I just did this manually


German OS Windows has trouble about iFacialMocap data processing

User report.
https://twitter.com/kyojiwa/status/1406005646273753088?s=20

Reproduce Check: Windows (JP) / iFacialMocap(JP) => OK
Reproduce Check: Windows (JP) / iFacialMocap(GE) => OK
User Report:  Windows (GE) / iFacialMocap(GE) => NG

It seems problem about culture dependent behavior of float.Parse, but need to confirm.

Fixed in #583 , and workaround for v1.7.0b and older is:

Set Windows language to another one, in which decimal point is period(".") in its standard culture.
For example, English (US) matches this condition.


Define return types for arithmetic operations between RealType and primitive types

RealType + primitive type returns RealType, which I find pretty intuitive
primitive type + RealType returns RealType, which seems less intuitive.

I have not come up with a better solution yet, though.

I think it is ok to always return RealType.


Update dependencies and fix null value lookups

:tada: This PR is included in version 1.5.2 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


🌟 [FEATURE] Masking out some labels (e.g. constrained atoms)

BETA implemention on masks: https://github.com/mir-group/nequip/tree/masks/examples/mask_labels
See https://github.com/mir-group/nequip/discussions/240 for more discussion.

A related feature request would be to set custom weights per force component (e.g. as additional columns in an ASE .xyz dataset).
If I have structures where some atoms have very large forces I do not really care how accurate the trained model is on those large forces, only that they are "large", so it would be great to be able to give them a much smaller weight in the loss function.
This would be useful for example when adding short interatomic distances, or when breaking chemical bonds (i.e. far from equilibrium).

Hi @mhellstr ,
That would be something you'd implement as a custom loss function, see here:
https://github.com/mir-group/nequip-example-extension
You could either make a custom loss that directly depends on the force magnitude, or you could take a force_weights (or whatever) key from the data and use that to reweight the loss value. (In the second case, you'd just need to include the force_weights in the dataset as a custom field with include_keys)

Hi,
Super cool, I would love to also use this feature in my work!
Is it possible to also set the masked atom's atomic energy contribution to zero? Ideally I would like to keep the masked atoms in the structure to keep the local atomic environment of the relevant atoms intact, however I do not want them contributing to the ultimage energy of the structure.
I tried by including the value "atomic_energy" in the field fields_to_mask, but that didn't work, presumably because the training data only includes one scalar value for the total_energy.
Is there anyway around this?
Thanks!


Failed to resolve: com.schibstedspain.android:leku:5.1.1

I've been having an issue updating Leku since the first build of 5.1.0, any help would be appreciated.

it is available @ https://bintray.com/schibstedspain/maven/leku/5.1.1
download

@aliondos It's solved with the latest version released (v5.1.3). I'm closing this issue.


Local storage fills up by storing large/many conversations

After a while, local storage may fill up. There's no check if that happens and an error occurs.

Application error: a client-side exception has occurred (see the browser console for more information).

Developer console error:
main-c8a3b38fe73deb58.js:1 DOMException: Failed to execute 'setItem' on 'Storage': Setting the value of 'selectedConversation' exceeded the quota.

Fixed. Checks for storage failure now and shows a dialog.


Autocomplete doesn't correctly work on implicitly casted variables

Tested versions
Experienced in 4.1 and 4.2 stable versions.
System information
Godot v4.2.1.stable - Windows 10.0.22621 - Vulkan (Forward+) - dedicated NVIDIA GeForce RTX 4080 (NVIDIA; <IP_ADDRESS>01) - AMD Ryzen 9 7900X 12-Core Processor (24 Threads)
Issue description
This issue is on the autocomplete features of using static types in GDScript, and not the actual execution behaviour. The actual execution behaviour works as expected.
In GDscript, there are two ways to assign variables with a specific type, they are the following, with an example:

var result := foo() as Array (explicit cast using the as keyword)
var result: Array = foo() (implicit cast using static typing : <TypeName>operator)

One benefit of static typing is that it provides useful autocomplete suggestions when interacting with the variable. For example, the above result is of type Array, so autocomplete correctly shows methods associated with the array class:

This works all and well until the assignment is the value of a function call that returns a Variant type such as .get on a dictionary or any user-defined function like this simple example:
func arr_or_dict(x: int) -> Variant:
    if x == 0:
        return []
    return {}

The above example returns a Variant type of either Array or Dictionary. When casted using method 1 (as keyword`), the autocomplete works correctly, displaying the attributes associated with the casted type:
func explicit_cast():
    var result := arr_or_dict(0) as Array

The problem occurs when using method 2 to implicitly cast the return value - It appears that the autocomplete suggestions are incorrectly showing the methods associated with Dictionary, despite it being statically declared as type Array (and the actual type is Array as 0 is the parameter, triggering the first return [] statement:
func implict_cast():
    var result: Array = arr_or_dict(0)

After testing it a bit more, it seems like the editor will always show options associated with the type of the variable of the last return statement of the function and ignore the declared type. In the example given, it is Dictionary from the last line return {}. If I switch [] and {} and cast result into Dictionary, it will show array methods because the type of the last return statement will be Array.
The expected behaviour should be that the autocomplete suggestions always show the methods declared by the static type Array, exactly like the behaviour of explicitly using as keyword to cast.
Despite the autocomplete showing incorrect options, the execution behaviour is still correct, result is still of type Array, and calling dictionary methods given by the faulty autocomplete will still throw an error.
Steps to reproduce

Attach empty script to any node in any scene
Paste in the MRP
Uncomment the result. lines
In the Godot editor, trigger the autocomplete associated with the above line and observe the incorrect suggestions of implict_cast compared to the correct explicit_cast

Minimal reproduction project (MRP)
(Code is below, will also include a zipped project on Godot 4.2.1: godot-autocomplete-bug-mrp.zip)
extends Node2D

func arr_or_dict(x: int) -> Variant:
	if x == 0:
		return []
	return {}

# explicit cast (correct)
func explicit_cast():
	var result := arr_or_dict(0) as Array
	# autocomplete works correctly when using `result.`
	# result.

# implicit cast (incorrect)
func implict_cast():
	var result: Array = arr_or_dict(0)
	# when using `result.`, autocomplete shows suggestions associated with the type of the LAST return value (dict in this case)
	# result.

### Tasks

Duplicate of #78820 but we can keep this one open, since it adds the additional detail of the interfered type being Dictionary (although that's not really bug)

I don't know if it will be of any help but I came across the same bug without the use of "return" in my script.
I am learning how to create finite state machines and for this reason I created a custom class called "state"
`class_name state
extends Node
signal Transitioned
func Enter() -> void:
pass

func Exit() -> void:
pass

func Update(_delta: float) -> void:
pass

func Physics_Update(_delta: float) -> void:
pass

`
Then there is the StateMachine script which contains a function that runs when I want to transition from one state to the other. In this function I want to call the function Enter in the new state.
`func StateTransition(curr_state: state, next_state_name: String) -> void:
if current_state != curr_state:

	return

var new_state: state = states.get(next_state_name.to_lower())
if !new_state:

	return

if current_state:

	current_state.Exit()

new_state.Enter()

current_state = new_state`

If I cast the variable "new_state" as a type "state" like you see above, autocomplete doesn't work and the function Enter doesn't appear as an autocomplete option.

However, using
var new_state := states.get(next_state_name.to_lower()) as state
seems to make autocomplete run fine, like you see below.

Why is Godot acting differently between the two ways of casting a variable?

I don't know how useful it is, but I came across the same problem without the use of "return" though.
I am learning to code a finite state machine and in the process I created a custom class named "state" that has the following code.
class_name state
extends Node

signal Transitioned

func Enter() -> void:

	pass

func Exit() -> void:

	pass

func Update(_delta: float) -> void:

	pass

func Physics_Update(_delta: float) -> void:

	pass

There is also a StateMachine that contains a function that is called when a transition between two states happens. Inside this function there is a "new_state" variable that is casted as a "state" type.
func StateTransition(curr_state: state, next_state_name: String) -> void:

	if current_state != curr_state:

		return

	var new_state: state = states.get(next_state_name.to_lower())
	if !new_state:

		return

	if current_state:

		current_state.Exit()

	new_state.Enter()

	current_state = new_state

What is supposed to happen, is that autocomplete needs to suggest me the functions that "state" defines when accessing the variable "new_state" but autocomplete doesn't work...

However, when I use the "as state" method of casting, autocomplete DOES work.

Why is Godot acting differently? Aren't these two situations basically the same exact thing?


Change env variables

Description
The configuration section for environment variables is not clear enough for new developers.
Changes proposed in this pull request:

Add note how to get cluster domain
Add SCOPE environment variables exporting

Related issue(s)

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Reviewed CHANGELOG

Description of changes:  Reviewed CHANGELOG
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Codecov Report

Merging #1667 into develop will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop    #1667   +/-   ##
========================================
  Coverage    51.95%   51.95%
========================================
  Files           24       24
  Lines         3948     3948
========================================
  Hits          2051     2051
  Misses        1897     1897

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 37f26d5...a29727d. Read the comment docs.


Unable to find the field "existingCertSecret" to configure transport-TLS

Hello
I am unable to find the field "existingCertSecret" in plugins.security.ssl  in opensearch helm chart to configure transport TLS for nodes.
Earlier , as in opendistro , we got it at here
But not in opensearch.
Due to which I am not able to configure TLS using issuer and certificates.
Kindly help.
Thanks

@divyanshuch1 I guess you can add it in the values.yaml. In the current chart the sample value is not given but TLS is configurable.

@divyanshuch1 can we close this issue?

@divyanshuch1 the PR #80 is merged, let me know if we are good to close this issue. Thanks!

@TheAlgo  Yes sure, we can close this issue.

Close the issue as @divyanshuch1 requested. Thanks.


500 - Internal Server Error

If somethings goes wrong inside the server while handling a request giving an exception we can respond with an error message of type 500.  https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500

Should we place the code for this issue as a method in ClientHandler.class or maybe in a future HTTPResponse.class?

HTTPResponse sounds like a good place. Then the server could create a HTTPResponse object for each incoming request and if we want to send a 500 response we can call a method on the Response objekt telling it to create such a response.


Properly handle one or more ndjson records coming from Ollama

Test plan
The bug was verified using an isolated version of lib/shared/src/ollama/chat-client.ts running against a fast, local ollama server.  The changes were made and verified against same ollama server.
The issue is triggered very consistently by setting "cody.experimental.ollamaChat": true, and using a fast local Ollama setup, perhaps using llama2:7b or as small and fast a model as you can find.  Then ask a question with a reasonably long answer like "why is the sky blue?".  You should see the answer halt somewhere in the middle and Cody chat is then "frozen" and cannot be aborted.  The only recovery is to delete the chat and start a new one.  The reason this happens is that if Cody does not read the stream fast enough, it gets '{...}\n{...}' in reader.read() and then JSON.parse bombs.  The existing code does not accommodate x-ndjson, which is what Ollama is sending.    Probably all other LLM providers too.
can-ndjson-stream is a library that could be used, but I thought a no-dependency solution would be more acceptable as a PR.  You might read this background article: https://andreitopli.medium.com/working-with-newline-delimited-json-using-can-ndjson-stream-in-typescript-983016d51d4a
Added testcase: lib/shared/src/ollama/chat.test.ts

I went back to my original way of breaking records.  Splitting on \n did not work in all cases, while /.+/ does work on all cases.
I also added a unit test to try all the important cases.

Is there anything I need to do?


Paramount Plus Movies

Checklist

[x ] I'm reporting a site feature request
[x ] I've verified that I'm running youtube-dl version 2021.06.06
[ x] I've searched the bugtracker for similar site feature requests including closed ones

Description

As we all know Paramount Plus was added but the problem is that it doesn't do movies but any tv show you throw at it works I feel that we should add movie support because its the same thing as shows but instead of paramountplus.com/shows its paramountplus.com/movies

Yes but on this occasion DRM is attached to all movies placed on there. Shows will untill a drastic redesign will download whereas movies will not.

https://github.com/ytdl-org/youtube-dl/issues/30491

Issue should be closed, no URL provided


Celsius and Fahrenheit

What needs to be done to add these?

What needs to be done to add these?

It’s a bit tricky because unlike other units they aren’t just multiples of kelvin. I’ll see if I can find a nice way to implement this the next couple of days.

It’s a bit tricky because unlike other units they aren’t just multiples of kelvin. I’ll see if I can find a nice way to implement this the next couple of days.

This is now implemented, and live on https://fend.printfn.nz.

This is now implemented, and live on https://fend.printfn.nz.

Awesome!

Awesome!


Enforce foreign keys for mobility translations

In #609, we migrated from globalize to mobility, a "Pluggable Ruby translation framework". We migrated according to their guide here. However, we didn't check our own migrations file. Now we noticed that three very old migrations cannot be run anymore because the create_translation_table! method used in them was provided by globalize and is no longer part of mobility, see here.
For testing purposes, in #654, we deleted those non-working migrations and completely setup the database again by deleting it entirely (also its schema), then running through all migrations. After that, we used the generator provided by mobility to generate the respective translation tables again:
rails generate mobility:translations subject name:text
rails generate mobility:translations program name:text
rails generate mobility:translations division name:text

The resulting schema.rb can now be diffed with what we had before, see this diff. We made the following observations:

Foreign key constraints were added, which is probably due to https://github.com/shioyama/mobility/issues/281 and the respective PR. In this current PR you're viewing, we add those foreign keys manually in a new migration.
Indices were merged together into one line. We don't do anything here as this is not a semantic change.
⚠ The field t.text "name" was added to the table subjects, programs and divisions. However, this field already exists in the respective translation tables, so we don't know why we would need it in the associated table as well. We don't change this in this PR as mobility already worked in the last few months without this field. If we have problems with mobility in the future, this might be a possible reason.
Note that in the translation tables the field t.text was just moved, so it seems like it was added there in the diff as well. It's different for the "non-translation tables" subjects, programs or divisions where the field was really newly added (there's no corresponding red "delete" line for t.text "name" in those tables.

I've added a Migration label to GitHub such that it's easier to find out which PRs were responsible for database migrations ;)
Deployment to experimental
I've just deployed this commit to experimental and the server was not reachable afterwards. In the logs, I couldn't find any root cause, nor anything that pointed to the change I made here. I now reset experimental to a commit prior to db9d1f1, build it, then set it to db9d1f1 again and see if it works.
Edit: apparently it also doesn't work with the commit before this PR:
2024/06/27 23:06:17 [emerg] 1#1: no host in upstream ":3000" in /etc/nginx/conf.d/default.conf:116

2024/06/27 23:07:02 [error] 13#13: *43 connect() failed (113: Host is unreachable) while connecting to upstream, client: REDACTED, server: localhost, request: "GET /metrics HTTP/1.1", subrequest: "REDACTED", upstream: "http://REDACTED//metrics", host: "localhost:REDACTED"

Will have to investigate that issue, but probably not related to this PR.

I still need some convincing as to why we need these foreign keys. If they were that important, wouldn't they be mentioned in the migration guide from globalize to mobility? Also, I think it would be good to find out what the issue with staging is and to verify that it is independent of the PR under review.

Also, I think it would be good to find out what the issue with staging is and to verify that it is independent of the PR under review.

The issue with mampf-experimental was due to #609 where we activated the option config.require_master_key. After having filled the field RAILS_MAILS_KEY for the respective docker.env, the staging worked again for commit 0c841de5b5500 on dev. I've just rebuilt it with the latest commit from this PR (db9d1f1) and it also ran through without any problems.

I still need some convincing as to why we need these foreign keys. If they were that important, wouldn't they be mentioned in the migration guide from globalize to mobility?

You're right that this is unclear. I've opened an issue at mobility, maybe they can help: https://github.com/shioyama/mobility/issues/641

I will convert this back to a draft until we get a response here: https://github.com/shioyama/mobility/issues/641. This is not a blocker as mobility is working just fine in production; we just want to make sure that we didn't miss anything in the migration.


changed timefrequency functions to accept tuple with None for lowpass…

… or highpass as f_range
Because previously it's impossible to get amp_by_time, with the intention of just lowpassing, without an insanely long filter, since low-freq cutoff is required.

looks good to me.
The only thing I would maybe add is a smoke test to make sure it runs when passing in (None, fc) or (fc, None).


Ember server graceful shutdown

Closes #3067. The goal of this PR is to give ember-server a graceful shutdown sequence, which @rossabaker captures succinctly in the original issue. I'd still like to add tests and follow up with some fs2 questions, but I think it's ready for review.

I got this feedback recently could this be integrated, @djspiewak  believe this will also speed up so may cancel out the prefetch costs.
.parJoin(maxConcurrency)
          .interruptWhen(terminationSignal)
          .drain)

to
          .flatMap(s => Stream.resource(s.compile.resource.drain.background))
          .interruptWhen(terminationSignal)
          .drain)

I got this feedback recently could this be integrated, @djspiewak  believe this will also speed up so may cancel out the prefetch costs.
.parJoin(maxConcurrency)
          .interruptWhen(terminationSignal)
          .drain)

to
          .flatMap(s => Stream.resource(s.compile.resource.drain.background))
          .interruptWhen(terminationSignal)
          .drain)

Does that have the semantics we want? It seems like it's going to start the handler for every socket in a separate fiber, lift that resource into stream, then immediately exit which cancels the fiber (since it introduced a scope). I think resourceWeak would work but then we are possibly leaking fibers, and we also can't control concurrency. Let me know if I'm wrong about that though
Also I realized that the prefetch isn't that bad. It becomes a problem when you have a lot of connection cycling, but it shouldn't affect max RPS all else being equal

Does that have the semantics we want? It seems like it's going to start the handler for every socket in a separate fiber, lift that resource into stream, then immediately exit which cancels the fiber (since it introduced a scope). I think resourceWeak would work but then we are possibly leaking fibers, and we also can't control concurrency. Let me know if I'm wrong about that though
Also I realized that the prefetch isn't that bad. It becomes a problem when you have a lot of connection cycling, but it shouldn't affect max RPS all else being equal

Removed the comment I added about prefetch, because it really only limits the rate at which connections are accepted, but not max RPS. I'm going to write a special-purpose combinator that mixes the semantics of parJoin, prefetch and drain since they come with a lot of baggage.

Removed the comment I added about prefetch, because it really only limits the rate at which connections are accepted, but not max RPS. I'm going to write a special-purpose combinator that mixes the semantics of parJoin, prefetch and drain since they come with a lot of baggage.


Committees

Added basic support for /Committees/ endpoint.

wow just seeing this PR. thanks! i will review within the next few days.

@chriscondon let me know if you have any suggestions about strategies for refactoring client.rb (low priority, not a necessary condition for merge, just interested to get your feedback).

reference https://github.com/debate-watch/open-fec-api-ruby/pull/2


Support for Swift 05-09

I've got a working fix over at https://github.com/coryalder/fluent-sqlite/commit/c8f73ed1f91ecf426672f880b59405dbbc66e6bf
However I'm hesitant to submit a pull request because I'm not sure of the full consequences of my change. While the current release of Fluent is pinned to 05-03, it compiles fine under 05-09. FluentSQLite does not.
The reason I'd like to use 05-09 is Vapor is currently targeting that version of Swift.

Yeah this needs to be updated to work with 05-09. I've added this to the 0.4 milestone.

fixed in #7


Shutdown encounter errors

Version: 3.0.1
System: Windows 10
In event viewer, it got:

unable to create output directory '/?': Failed to create directory [/?] The filename, directory name, or volume label syntax is incorrect.
C:\b\ArangoDB-3.0.1\arangosh\Dump\DumpFeature.cpp
arangodb::DumpFeature::prepare
195

After I cleanup then reinstall, when stop the service, still got error but different:

The description for Event ID 256 from source ArangoDB cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.

If the event originated on another computer, the display information had to be saved with the event.
The following information was included with the event:
at next (d:\Program Files\ArangoDB\share\arangodb3\js\server\modules\@arangodb\foxx\router\tree.js:384:7)

C:\b\ArangoDB-3.0.1\lib\V8\v8-utils.cpp
JS_Log
1697

The later error are stack traces from javascript Foxx(?) services splattered across several log entries.
They're better readable if you configure ArangoDB to write its own logfile.

@dsonet: the error unable to create output directory '/?': Failed to create directory [/?] The filename, directory name, or volume label syntax is incorrect. can only be raised by arangodump.
arangodump will use the first command-line parameter that does not start with -- as input directory name. So it seems arangodump was called with parameter /?, which will not work.
arangodump is not invoked automatically, but only runs by manual intervention. What parameters did you specify? Was it /? to show help texts? That will not work for arangodump and the other tools, as all options start with --. If you're looking for which options it has, please use --help.

@dsonet any news on this? Hints how you got into this? Did you fix it? can we close this issue?

closing due to inactivity. Feel free to comment and reopen if this is still a problem.


/* matches top level directories

I've noticed what looks to me like a bug: I would expect the pattern */** to match all paths with a minimum depth of one, but it appears to match directories (not files) at the top level as well. A concrete example:
$ mkdir -p node_modules && npm install<EMAIL_ADDRESS>$ mkdir -p test/1/2/3 && cd test
$ echo 'require("glob")("*/**", (_, f) => console.log(f))' | node
[ '1', '1/2', '1/2/3' ]

I would have expected [ '1/2', '1/2/3' ], but 1 was also matched.
minimatch, on the other hand, behaves as I'd expect:
$ mkdir -p node_modules && npm install<EMAIL_ADDRESS>$ echo 'console.log(require("minimatch")("1", "*/**"))' | node
false
$ echo 'console.log(require("minimatch")("1/2", "*/**"))' | node
true
$ echo 'console.log(require("minimatch")("1/2/3", "*/**"))' | node
true

I'd be happy to help out - either with a patch if it is indeed a bug, or some documentation if it's defined behaviour, but I'd first need to know which one it is, couldn't mange to find out on my own.
(Tests above were performed with Node.js v16.10.0 on macOS 12.1.)

Nope.  ** can match zero or more path portions.  So */** matches a as well as a/b/c/d.
Observe:
#!/usr/bin/env bash

set -e

shopt -s globstar
shopt -s extglob

rm -rf a b c

mkdir -p {a,b,c}/{1,2,3}
touch {a,b,c}/{1,2,3}/{foo,bar,baz}
# tree

for i in */**; do
  echo $i # includes a, b, c
done

a  <-- here
a/1
a/1/bar
a/1/baz
a/1/foo
a/2
a/2/bar
a/2/baz
a/2/foo
a/3
a/3/bar
a/3/baz
a/3/foo
b  <-- here
b/1
b/1/bar
b/1/baz
b/1/foo
b/2
b/2/bar
b/2/baz
b/2/foo
b/3
b/3/bar
b/3/baz
b/3/foo
c  <-- here
c/1
c/1/bar
c/1/baz
c/1/foo
c/2
c/2/bar
c/2/baz
c/2/foo
c/3
c/3/bar
c/3/baz
c/3/foo

The pattern that does what you want is */{*/**,*}

I'd be happy to help out - either with a patch if it is indeed a bug, or some documentation if it's defined behaviour, but I'd first need to know which one it is, couldn't mange to find out on my own.

Technically it's documented that this library matches the behavior of recent versions of bash (v5, I think?  or 4.3?  I think those are the same anyway), so it's sort of documented, but yeah, that could definitely be improved.  Patch welcome for that, as long as it's not just dumping man bash into the readme 😆

Thanks! I suppose that does make sense indeed - maybe just a bit unintuitive. If I can find a good place to mention this I'll do a PR!


Merge submission v2.0 into master

Part of #278

Here is the app built with this branch:
2022-04-16_PR292_mlperfbench_v2.0-qsmgt.apk

Currently blocked by this issue:
https://github.com/mlcommons/mobile_app_open/issues/278#issuecomment-1097883953

@freedomtan @Mostelk @mohitmundhragithub @ptt2panda
Please test this build on your specific device to make sure that everything still works as expected.
2022-04-28_PR292_mlperfbench_v2.0-qsmgt.apk
This one is a Flutter app.

@d-uzlov Can you please check why the test Build and test iOS Flutter app is failed? I don't have access to Jenkins.

@anhappdev there is some very massive syntactic error mostly complaining about unknown type name 'NSString' but also things like @class. My first impression is that it's trying to compile Objective C code as a C code. The error happened in bazel.
Here is the fill error:

In file included from mobile_back_tflite/cpp/backend_tflite/tflite_c.cc:62:
In file included from external/org_tensorflow/tensorflow/lite/delegates/gpu/metal_delegate.h:19:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Metal.framework/Headers/Metal.h:9:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Metal.framework/Headers/MTLTypes.h:8:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/Foundation.h:8:
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObjCRuntime.h:492:1: error: expected unqualified-id
@class NSString, Protocol;
^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObjCRuntime.h:494:9: error: unknown type name 'NSString'
typedef NSString * NSExceptionName NS_EXTENSIBLE_STRING_ENUM;
        ^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObjCRuntime.h:495:9: error: unknown type name 'NSString'
typedef NSString * NSRunLoopMode NS_EXTENSIBLE_STRING_ENUM;
        ^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObjCRuntime.h:497:19: error: unknown type name 'NSString'
FOUNDATION_EXPORT NSString *NSStringFromSelector(SEL aSelector);
                  ^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObjCRuntime.h:498:44: error: unknown type name 'NSString'
FOUNDATION_EXPORT SEL NSSelectorFromString(NSString *aSelectorName);
                                           ^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObjCRuntime.h:500:19: error: unknown type name 'NSString'
FOUNDATION_EXPORT NSString *NSStringFromClass(Class aClass);
                  ^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObjCRuntime.h:501:53: error: unknown type name 'NSString'
FOUNDATION_EXPORT Class _Nullable NSClassFromString(NSString *aClassName);
                                                    ^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObjCRuntime.h:503:19: error: unknown type name 'NSString'
FOUNDATION_EXPORT NSString *NSStringFromProtocol(Protocol *proto) API_AVAILABLE(macos(10.5), ios(2.0), watchos(2.0), tvos(9.0));
                  ^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObjCRuntime.h:503:50: error: unknown type name 'Protocol'
FOUNDATION_EXPORT NSString *NSStringFromProtocol(Protocol *proto) API_AVAILABLE(macos(10.5), ios(2.0), watchos(2.0), tvos(9.0));
                                                 ^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObjCRuntime.h:504:19: error: unknown type name 'Protocol'
FOUNDATION_EXPORT Protocol * _Nullable NSProtocolFromString(NSString *namestr) API_AVAILABLE(macos(10.5), ios(2.0), watchos(2.0), tvos(9.0));
                  ^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObjCRuntime.h:504:61: error: unknown type name 'NSString'
FOUNDATION_EXPORT Protocol * _Nullable NSProtocolFromString(NSString *namestr) API_AVAILABLE(macos(10.5), ios(2.0), watchos(2.0), tvos(9.0));
                                                            ^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObjCRuntime.h:508:30: error: unknown type name 'NSString'
FOUNDATION_EXPORT void NSLog(NSString *format, ...) NS_FORMAT_FUNCTION(1,2) NS_NO_TAIL_CALL;
                             ^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObjCRuntime.h:508:53: error: format argument not an NSString
FOUNDATION_EXPORT void NSLog(NSString *format, ...) NS_FORMAT_FUNCTION(1,2) NS_NO_TAIL_CALL;
                             ~~~~~~~~~~~~~~~~       ^                  ~
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObjCRuntime.h:94:49: note: expanded from macro 'NS_FORMAT_FUNCTION'
        #define NS_FORMAT_FUNCTION(F,A) __attribute__((format(__NSString__, F, A)))
                                                       ^                    ~
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObjCRuntime.h:509:31: error: unknown type name 'NSString'
FOUNDATION_EXPORT void NSLogv(NSString *format, va_list args) NS_FORMAT_FUNCTION(1,0) NS_NO_TAIL_CALL;
                              ^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObjCRuntime.h:509:63: error: format argument not an NSString
FOUNDATION_EXPORT void NSLogv(NSString *format, va_list args) NS_FORMAT_FUNCTION(1,0) NS_NO_TAIL_CALL;
                              ~~~~~~~~~~~~~~~~                ^                  ~
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObjCRuntime.h:94:49: note: expanded from macro 'NS_FORMAT_FUNCTION'
        #define NS_FORMAT_FUNCTION(F,A) __attribute__((format(__NSString__, F, A)))
                                                       ^                    ~
In file included from mobile_back_tflite/cpp/backend_tflite/tflite_c.cc:62:
In file included from external/org_tensorflow/tensorflow/lite/delegates/gpu/metal_delegate.h:19:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Metal.framework/Headers/Metal.h:9:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Metal.framework/Headers/MTLTypes.h:8:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/Foundation.h:10:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSArray.h:5:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObject.h:8:
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSZone.h:9:1: error: expected unqualified-id
@class NSString;
^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSZone.h:19:63: error: unknown type name 'NSString'
FOUNDATION_EXPORT void NSSetZoneName(NSZone * _Nullable zone, NSString *name)NS_SWIFT_UNAVAILABLE("Zone-based memory management is unavailable");
                                                              ^
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSZone.h:20:19: error: unknown type name 'NSString'
FOUNDATION_EXPORT NSString *NSZoneName(NSZone * _Nullable zone) NS_SWIFT_UNAVAILABLE("Zone-based memory management is unavailable");
                  ^
In file included from mobile_back_tflite/cpp/backend_tflite/tflite_c.cc:62:
In file included from external/org_tensorflow/tensorflow/lite/delegates/gpu/metal_delegate.h:19:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Metal.framework/Headers/Metal.h:9:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Metal.framework/Headers/MTLTypes.h:8:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/Foundation.h:10:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSArray.h:5:
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.5.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSObject.h:10:1: error: expected unqualified-id
@class NSInvocation, NSMethodSignature, NSCoder, NSString, NSEnumerator;
^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.

When I tried building the branch locally on my macbook I got the same error.
I was able to fix it by reverting tensorflow version in the WORKSPACE file back to 2.6.0.
Maybe 2.8.0 is broken for iOS, or maybe there were some breaking changes that are not accounted for in the submission 2.0 branch.

Oh, I remembered that freedomtan also said about some issues with iOS when building with TF 2.8.0, and PR https://github.com/mlcommons/mobile_app_open/pull/268 contains some iOS-specific fixes.
I think we should revert TF version change from this PR and update it later in that separate PR. We even have a separate issue to update it.

Oh, I remembered that @freedomtan also talked about some issues with iOS when building with TF 2.8.0, and his PR #268 contains some iOS-specific fixes.

as far as I can remember, it's because of a change in metal delegate. It's something like including "<Metal/Metal.h>", which is includes some Objective-C headers, in C++ (.cc) code. By default, clang doesn't allow objective-c code in .cc   source code.

@freedomtan @Mostelk @mohitmundhragithub @ptt2panda
Please test this build on your specific device to make sure that everything still works as expected.
App built WITH_TFLITE=1 WITH_MEDIATEK=1 WITH_PIXEL=1 run without issue on my test Android phone (TFLite backend) 2022-04-28_PR292_mlperfbench_v2.0-t.apk 2022-04-28_PR292_mlperfbench_v2.0-mgt.apk
App built either WITH_SAMSUNG=1 or WITH_QTI=1 or both caused a crash on my phone. 2022-04-28_PR292_mlperfbench_v2.0-qsmgt.apk
All apps in this PR is Flutter app.

The qsmgt.apk one could finish running on Xiaomi Redmi K50 Pro (a Dimensity 9000 phone). However,

there are only five benchmarks (I expected the Mosaic, the new segmentation model, to be added)
performance numbers are wrong. It seems settings and neuron delegate were not updated. I'll check the details later.

@AhmedTElthakeb tried and the app crashed. Please work with @anhappdev to resolve this.

generally, we don't have the new Mosaic because tasks file was not updated. (https://github.com/mlcommons/mobile_models/pull/40) is supposed to resolve this.

backend
status
tested by

Google

MediaTek
it runs, but there is huge performance gap
@freedomtan

Qualcomm

Samsung
crashed

Intel
under active development
@shabanim

Here are the new APKs (built from commit 5b65d1c423cb1340939a5b3a5c3c5879630660fa):
PR292 APKs
Some changes of the APKs 2022-05-04 vs. 2022-04-28 are:

Use the new task file with IS Mosaic.
Use TF2.6

I tested on a Google Pixel 6 and the build works as expected.

Here are the new APKs (built from commit 5b65d1c423cb1340939a5b3a5c3c5879630660fa): PR292 APKs I built a separated APK for each backend to make testing easier.
Some changes of the APKs 2022-05-04 vs. 2022-04-28 are:

Use the new task file with IS Mosaic.
Use TF2.6

I tried the qualcomm apk shared here. It seems to work fine. Couple of questions related to the app --

the cooldown time is 0 by default in this apk. In our older app it was 5 minutes. Is it ok?
There is one option of offline mode in the settings. Can you please explain the use of it and how to use it?

Status update: @AhmedTElthakeb helped to resolve the crash with Samsung supported device. But the unified app still crashes on Samsung non-supported device. It looks like the unified app crashes when it cannot detect Samsung backend instead of falling through and check for other backends.

Status update: @AhmedTElthakeb helped to resolve the crash with Samsung supported device. But the unified app still crashes on Samsung non-supported device. It looks like the unified app crashes when it cannot detect Samsung backend instead of falling through and check for other backends.

is this related to old issue (https://github.com/mlcommons/mobile_app_open/issues/231) we had before? or this is a new one?

@AhmedTElthakeb and @freedomtan to test on other devices.
what @anhappdev reported: Samsung backend + TFLite on non-supported Samsung devices (say S20 or S21 (?))

Samsung and QTI backend use the same libc++_shared.so file but from different sources so I got this error when building the unified app:
rm -rf flutter/android/app/src/main/jniLibs/arm64-v8a
mkdir -p flutter/android/app/src/main/jniLibs/arm64-v8a
cp -f \
        bazel-bin/mobile_back_tflite/cpp/backend_tflite/libtflitebackend.so \
         \
         \
        bazel-bin/mobile_back_qti/cpp/backend_qti/libqtibackend.so mobile_back_qti/snpe-<IP_ADDRESS>30/lib/aarch64-android-clang8.0/libc++_shared.so mobile_back_qti/snpe-<IP_ADDRESS>30/lib/aarch64-android-clang8.0/libSNPE.so mobile_back_qti/snpe-<IP_ADDRESS>30/lib/aarch64-android-clang8.0/libhta.so mobile_back_qti/snpe-<IP_ADDRESS>30/lib/aarch64-android-clang8.0/libsnpe_dsp_domains_v2.so mobile_back_qti/snpe-<IP_ADDRESS>30/lib/aarch64-android-clang8.0/libsnpe_dsp_domains_v3.so mobile_back_qti/snpe-<IP_ADDRESS>30/lib/aarch64-android-clang8.0/libSnpeHtpV69Stub.so mobile_back_qti/snpe-<IP_ADDRESS>30/lib/aarch64-android-clang8.0/libSnpeHtpV68Stub.so mobile_back_qti/snpe-<IP_ADDRESS>30/lib/aarch64-android-clang8.0/libSnpeHtpPrepare.so mobile_back_qti/snpe-<IP_ADDRESS>30/lib/dsp/libsnpe_dsp_v66_domains_v2_skel.so mobile_back_qti/snpe-<IP_ADDRESS>30/lib/dsp/libsnpe_dsp_v68_domains_v3_skel.so mobile_back_qti/snpe-<IP_ADDRESS>30/lib/dsp/libSnpeHtpV69Skel.so mobile_back_qti/snpe-<IP_ADDRESS>30/lib/dsp/libSnpeHtpV68Skel.so \
        bazel-bin/flutter/android/commonlibs/lib_arm64/libc++_shared.so mobile_back_samsung/samsung/lib/libsamsungbackend.so mobile_back_samsung/samsung/lib/libenn_public_api_cpp.so mobile_back_samsung/samsung/lib/libsbe2200_core.so mobile_back_samsung/samsung/lib/libsbe1200_core.so mobile_back_samsung/samsung/lib/libc++.so mobile_back_samsung/samsung/lib/libeden_nn_on_system.so  \
        bazel-bin/flutter/cpp/flutter/libbackendbridge.so \
        flutter/android/app/src/main/jniLibs/arm64-v8a
cp: will not overwrite just-created 'flutter/android/app/src/main/jniLibs/arm64-v8a/libc++_shared.so' with 'bazel-bin/flutter/android/commonlibs/lib_arm64/libc++_shared.so'
make: *** [flutter/android.mk:85: flutter/android/libs] Error 1
make: *** [flutter/android.mk:66: docker/flutter/android/apk] Error 2

@mohitmundhragithub
I updated the qti_backend.mk to use the ${BAZEL_LINKS_PREFIX}bin/flutter/android/commonlibs/lib_arm64/libc++_shared.so, see commit 1d73a3a. Please check if that is ok.

I uploaded the new APKs 2022-05-10_*.apk built with commit 1d73a3a in this folder PR292.
The change of the APKs 2022-05-10 vs. 2022-05-04 is that the updated Samsung backend code is used.

FYR, on a Samsung S21 5G SM-G991B I borrowed from a colleague, it crashed. That is, I confirmed what @anhappdev reported. Please check if this is expected behavior.

@Mostelk will help check if the S21 unsupported message is expected behavior or not.

@anhappdev please re-base and merge this

Is it possible to exclude binary libraries from sources? Maybe they can be automatically downloaded from somewhere?

mobile_back_tflite/cpp/backend_tflite/libneuronusdk_adapter.mtk.so.5
mobile_back_tflite/cpp/backend_tflite/neuron/tensorflow-lite-neuron.aar

@freedomtan Those files are for MediaTek backend, right?

Is it possible to exclude binary libraries from sources? Maybe they can be automatically downloaded from somewhere?

mobile_back_tflite/cpp/backend_tflite/libneuronusdk_adapter.mtk.so.5
mobile_back_tflite/cpp/backend_tflite/neuron/tensorflow-lite-neuron.aar

@freedomtan Those files are for MediaTek backend, right?

Yes, I'll ask MTK colleagues to update the source code of Neuron Delegate then we can use Neuron Delegate source code instead of binary aar. There were some internal changes not pushed to the Neuron Delegate repo yet.
The libneuronusdk_adapter.mtk.so.5 should be not be needed. Shipping devices should have newer binary now.

@freedomtan is there an estimate of much it will take? If it's more than, say, two weeks, then we can probably delete .so.5 file and merge this with .aar file, and create an issue to refactor it when neuron delegate sources are available.

It shouldn't take that long, however, our Neuron Delegate guys are busy working on some internal projects recently. So, yup, it could be more than 2 weeks.

@anhappdev, could you please create an issue to remove the .so file after you merge it?


About how to generate surface normal vector maps in real time

Hi! Are the 3 demo images you guys added to the readme processed in real time in an in-car environment? If so, how were the surface normal vector maps generated in real time? I use ROS(Robot Operating System) to read the depth map topic to calculate the surface normal vector maps using SNE and can't do it to generate the surface normal vector maps in real time. Please let me know if there is a better way, thanks!

normal

Yes, currently generating normals takes a long time and cannot be done in real-time. The demo presented is a video stitched together from the results. However, it should be possible to improve the SNE code to accelerate normal generation.


Trace CoreAPI.{upsert, getDocuments}

Just adding some datadog tracing to CoreAPI methods for debugging. Merging. Post merging review welcome obviously.


Revert "sc-cli: add no-beefy flag to cli config (#14754)"

This reverts commit 2060c366fb1402deab188911551a43c3c41b36c0.
Based on https://github.com/paritytech/substrate/pull/14754#issuecomment-1677208626
polkadot companion: https://github.com/paritytech/polkadot/pull/7613
cumulus companion: https://github.com/paritytech/cumulus/pull/3007

@bkchr

I know that we also have there disable_grandpa, that is just some old oversight and we should not replicate it.

shall I also remove disable_grandpa then (remove --no-grandpa from sc-cli)? If yes, will do it as separate PR so that this is clean revert and nothing else.

@bkchr

I know that we also have there disable_grandpa, that is just some old oversight and we should not replicate it.

shall I also remove disable_grandpa then (remove --no-grandpa from sc-cli)? If yes, will do it as separate PR so that this is clean revert and nothing else.

Sounds like a good idea! WDYT @andresilva?

bot merge

The disable_grandpa is used in both the node and node-template services, those are just using directly the RunCmd from sc-cli and it's probably not worth it to create their own RunCmd instances just for this. I'm fine with removing it, but it should still be added to polkadot IMO.

bot merge


Fix #1263 - Add the camel-jta extension

@zhfeng @ppalaga is this PR ready to be merged or some more work is required ?

@zhfeng @ppalaga is this PR ready to be merged or some more work is required ?

I still see a couple of unresolved requests: more test coverage, better description, etc.

@zhfeng thanks!
I have filed a followup issue for the usage: https://github.com/apache/camel-quarkus/issues/1428 Would you mind picking it?

@ppalaga sure, please assign it to me.


reclassify infectious disease terms

Per the Mondo developers call on 06/11, we propose to treat parasite as a synonym of eukaryote

[ ] to do: add parasite logical axiom w/ eukaryote

**Question: ** can anyone think of a eukaryotic infectious disease that is not a parasitic disease?
**Question: ** Should we put fungal infectious disease under parasitic infectious disease?

[ ] to do: classify all children of infectious disease as a child of bacteria, virus or parasite/eukaryote

cc @cmungall @mellybelly @pnrobinson @sabrinatoro

Fungi are eukaryotes but are not considered parasites.
Fungi are opportunistic

Shouldn't  synonyms be reciprocal?  Not all eukaryotes are parasites.
Amoebae are also eukaryotes, and they are not fungi
what does "classify all children of infectious disease as a child of bacteria, virus or parasite/eukaryote" mean?
Seems odd to classify any disorder as a child of an organism.

@maglott I meant: bacterial infectious disease, viral infectious disease or parasitic infectious disease. Yes, not as children of organisms, that would be odd! :)

And going back to the original question... if we think of parasites as describes in  https://en.wikipedia.org/wiki/Parasitism, I don't think all fungal infections are categorized as parasitic.  For example, https://www.ncbi.nlm.nih.gov/mesh/68002177  is not treed under parasitic diseases. https://www.ncbi.nlm.nih.gov/mesh/68010272

fungal infections are not considered to be parasitic infections

Thanks for your feedback @pnrobinson and @maglott
The next open question is:

how to better define parasitic infection (textually and can it be defined logically?)
Ask experts from https://veupathdb.org/veupathdb/app

Isn't there an ontology term for parasite that we can refer to?
The definition could be simply an  disease caused by infection by a parasite.

@pnrobinson

fungal infections are not considered to be parasitic infections

is this true for all fungal infections, or just some?
currently many sources treat for example Cryptosporidium infection as parasitic e.g

CDC: https://www.cdc.gov/parasites/crypto/index.html
ORDO: https://www.ebi.ac.uk/ols/ontologies/ordo/terms?iri=http%3A%2F%2Fwww.orpha.net%2FORDO%2FOrphanet_1549

are these formally wrong?
If it is the case that some fungal infections are parasitic infections, what is the differentia?
Our current def is from NCIT:
A successful invasion of a host by an organism that uses the host for food and shelter
As @nicolevasilevsky says we need a more precise definition here

Isn't there an ontology term for parasite that we can refer to?
https://www.ebi.ac.uk/ols/ontologies/ro/properties?iri=http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FRO_0002444

But it lacks a definition. Someone contact the Term Editor and bug them.
oh hold on...

oh hold on...

haha

http://purl.obolibrary.org/obo/IDO_0000403
A symbiont role borne by an organism in virtue of the fact that it derives a growth, survival, or fitness advantage from symbiosis while the other symbiont's growth, survival, or fitness is reduced.
Where symbiosis is An interaction between two organisms living together in more or less intimate association
Well it's a bit more precise than the current mondo definition but still hard to apply operationally.

derives a growth, survival, or fitness advantage from symbiosis: fits all fungal infections
the other symbiont's growth, survival, or fitness is reduced: fits many fungal infections, not just Cryptosporidium
living together in more or less intimate association: ?? who knows. I don't know what it means to be intimate with a fungus that is infecting me, and why I would be more intimate with Cryptosporidium than other fungi

I am not up to date on this but is it certain that cryptosporidium is a fungus? https://www.cdc.gov/parasites/crypto/index.html
An important difference is that funguses can live without a human host and are opportunistic infectious agents.
The word parasite can be interpreted as a role or as a characteristic of phyla, e.g. the phylum of cryptosporidium, The Apicomplexa (also called Apicomplexia) are a large phylum of parasitic alveolates.
https://en.wikipedia.org/wiki/Apicomplexa
I think for mondo we do not want to include the "role" of being a parasite, but rather want to use the biological definitions.

cryptosporidium was a mistake, it's a parasitic disease. :)


Add copy files feature

add a new deployment method, which copy the js, css and html files from lineman application dist folder to a rails public folder.

Would you mind explaining who this PR is needed for? I have a few questions about your intent.

Who uses the public directory in lieu of the asset pipeline? I've seen a lot of Rails projects, including some modern ones that disable the asset pipeline, but none that dump files into public/
How does this differ from cp -r?

The main reason is decoupling the frontend and the backend in the simplest way possible. Yes, this modification only copies the generated files (dist) to the public folder, not utilizing the asset pipeline when you already have done all precompilation using grunt tasks in lineman itself. The generation tasks in the remote server (installing node, lineman and its dependencies) when deploying is the part that I found useful. The added feature is to only free the developer from running a cp -R... yes.
What I could do is to rename the config option from "deploy_method'" to "deploy target" as this:
deploy_target = :asset_pipeline # Symbol: rails-lineman deployment still would work as usual, no changes there
deploy_target  = "public" # String: the path to copy everything into, after being generated by Lineman

Maybe this is a clearer configuration option naming, and if you wanted, you could also do:
deploy_target  = "public/app" # to copy "dist" into "app"

Later on you could "mount" multiple client side apps, but that would be another thing.
The alternative deploy method I propose is suitable when all HTML, Coffee, Sass and other source files are all together (not having any pages on Rails), without the need for Rails to provide anything (except for maybe, a JSON API for the app to communicate to, which is what I try to accomplish).
I am actually writing a blog post about this which I hope will be useful to understand why I propose this deployment method.

I currently don't find the requirement compelling enough to merge in, but I'm open to hearing more (especially if others would find it useful). One person on our team has suggested that what you need is different enough that you should fork the gem and implement your own as rails-lineman-public, free of the baggage of all the asset goop this gem does. Another person suggested that changing the dist directory in your lineman project might be all you need to do to force it to write into Rails' public/..

Separately, I have three pieces of meta-feedback about this PR:

When a PR doesn't solve a clearly understood problem shared by most or all users, that makes me more cautious about merging it in. If the PR is going to introduce a significant new logical branch of behavior (as this one does, it basically splits the gem into two modes of operation), then there is a pretty high burden of proof that the feature should be included. When I first started maintaining open source projects, I eagerly merged branches for non-default ("branched") features because if I wasn't personally using them, they couldn't harm current users. What I learned over time was two things: (1) every branch is another vector to test for in all future changes, and (2) if the original author stops using their own branched feature, I might find myself spending hours on my nights and weekends to maintain a feature that has literally zero users.

This PR came out of the blue—it wasn't tied to an issue, we didn't discuss it in advance, and it didn't have a clear description when it was sent.  As a result, I had no opportunity to express my concerns outlined above to you. Now that we're here, this won't feel very fair to you, because you spent a bunch of time working on it.  You can always deploy your fork and if it solves your problem, then great).

This PR's changes are broader-based than what is described. In addition to adding this feature, the branch:

bumps the patch version (which presumes control of the gem's versioning and releasing)
adds yourself as the second gem author (which presumes I'm comfortable with that level of attribution)
it touches up unrelated parts of the README
it changes the code's style in at least one place that I disagree with; meaning that were I to merge it in, I'd first feel the need to go through and clean up the stylistic and minor design changes introduced

Please don't find my words harsh or judgmental. I've been on the opposite side of this pull request a dozen times, and I'm sure I will be again. If I've learned one thing, though, is that it's more effective to initially invest my time communicating to build consensus than to submit (albeit excellent) code to someone who's not expecting it.

sorry for all the trouble, I actually agree with everything you said and totally understand the situation now. Its fine by me to use a fork or build a new gem since our approaches are different. Thanks for the tips! :+1:

@degzcs I've actually had similar scenarios in the past and we added a grunt-contrib-copy task to the end of our lineman build phase to copy the appropriate files from ./dist to ../public.
I think this could make an excellent lineman plugin and would be much smaller. (And also useful outside of rails)

@searls :

This PR came out of the blue—it wasn't tied to an issue, we didn't discuss it in advance, and it didn't have a clear description when it was sent. As a result, I had no opportunity to express my concerns outlined above to you.
 You can always deploy your fork and if it solves your problem, then great).
searls referenced this pull request in testdouble/double-takes 15 hours ago
Open
post idea: Pull Request ettiquette #29

I don't know either of you, but I've been thinking about this PR.  I feel it's been unnecessarily negative by @searls trying not to be negative, and it's the type of response that stifles devs in the future.
Side story, one of my first PRs to a public project was a full refactor to a project with junk code.  I started to use the project, liked it, but the code was an absolute disaster.  I was new to working with others on Github, but I knew how to refactor and test code... so I did.  I popped in 20-30 tests, refactored the bits, added the projects first tests, and in most ways made the project a "better" project.  But surprisingly to me, they didn't accept it!
I didn't take offense, they didn't take offense, it was just a "no."  But in the process most of my "code shyness" went away, and I've been popping PRs against other projects since.  I've learned that people don't want big shake-ups in PRs, and it's better to follow their advice, not touch versioning... all the basics that devs can only learn by doing.
And doing is awesome.  I agree with the technical reasons that @searls rejected the PR, but I think @degzcs has put himself in a higher tier of developer by submitting it.   And it's good stuff, too!  This is very thorough code, updating tests, documentation, adding a rake task... @degzcs has made something that solves a problem, is worthwhile as a fork, and works, and the changes were organized.  Great work... that just doesn't fit the goals of the project.
My advice to @degzcs is to keep it up!  Don't be afraid to pop a PR against any Github project, and don't take offense if anybody says no!  Hey, getting a NO is a better response than you'll get on most projects, which is SILENCE. :smile:
My advice to @searls :  Chill out. :smile: It's just a PR.  Just saying "No Thanks" if enough.  Explaining in detail why you won't accept it and offering advice is awesome!  Pointing at this as if it were a breach in etiquette, posting on Twitter, telling him that he needed to talk to you first, and saying the very submission of the PR is "not fair to you"... I think that's going too far, and it's counter to the "spirit" of Github and OSS.  You're handling something that's very delicate... you don't want to accidentally blow the fire out. :fire:

Just saying "No Thanks" is enough.

As a long time open source contributor and maintainer on several large projects, I can assure you that for some people a simple "no thanks" is not enough. I've been approached by people who have harbored secret grudges for years because their PRs are "never accepted" to projects I work on and they couldn't make sense of why.
One time someone was brought to tears over it.


Kazakhstan update

observed holidays
holidays relevance period

Coverage decreased (-0.01%) to 99.989% when pulling 68473b2ecfc32058a35ee9a73426d1aad35affdd on KJhellico:upd-kazakhstan into d37b6650a87ab039d9f46d4536363266b94fdc46 on dr-prodigy:beta.

I unfortunately had to revert this commit, due to a failing test on an observed holiday (2012-01-03).
Could you please review and provide a fix for that @KJhellico ?
Thx as usual! 👍

The issue was caused by another PR. It was fixed in https://github.com/dr-prodigy/python-holidays/pull/844
Sorry for the inconvenience.

Clear @arkid15r , thank you for explaining (honest, I could not dig so much on the issue yesterday)!
I've just merged #844 , will try to re-apply this one and see if all is right now!
Thx :+1:


fix: improve rubber band effect when out of bounds

Using the iOS constant of 0.55, according to rubberbandIfOutOfBounds

📝Changed routes:

/fixtures/sticky

(view full size)

Commit 91f8ca23091d18775af0aac617763072258f790b (https://react-spring-bottom-sheet-git-improve-rubberband.stipsan.vercel.app).


How to get image corner position after rotate?

I found many examples of how to pan an image, but I don't know how to get information about the position of the image in the viewport... How can I get the image corner position after rotation?

If you set rotation on the tile image, not viewer, then you can do this:
viewer.world.getItemAt(0).setRotation(33)
viewer.world.getItemAt(0).imageToViewportCoordinates(0,0)
// a.Point {x: 0.438059858489113, y: -0.1664542532764257}


Is there any fresh example of drawers and loading views?

I am not sure if this the right place to ask for the code sample of the up to date version of appgyver. I have also asked this question on forums as well and this is the title of my question.
"Is there any fresh example of how to link to another views from drawer?"
I really appreciate the help of Ben Styles, who helped me in every possible way with the code. But the problem is that the code is working fine on Iphone but on android (version 4.1.0) it is having problems. I am copying the details of the code I have used and the problem I am facing.

I have all this code in index.js:

angular.module('common', [
  // Declare here all AngularJS dependencies that are shared by all modules.
  'supersonic'
]);

var messageListen=function(channel,subject,callback) {
  function handler(event) {
    if (event.data.channel===channel && event.data.hasOwnProperty(subject)) {
      callback(event.data[subject]);
    }
  }
  window.addEventListener('message', handler);
};

var nav=function nav(dest) {
  var view=new supersonic.ui.View({
    location: dest,
    id: dest
  });
  view.isStarted().then(function(started) { // checks if view has been 'started' (i.e. preloaded)
    if (started) { // if so, push it
      supersonic.ui.layers.push(view);
    } else { // if not, start it then push it. This means the
             // view and any associated state
             // will persist even when popped
      // Start Spinner
      supersonic.ui.views.start(view).then(function() {
        supersonic.ui.layers.push(view);
        // Stop Spinner
      }, function(error) {
        // Stop Spinner
        supersonic.logger.error(error);
      });
    }
  });
}

var drawerNav=function drawerNav() {
  messageListen('drawerNav','destination', function(dest) {
    nav(dest);
    supersonic.ui.drawers.close();
  });
};

drawerNav();

And then in sidebar.html,

Specials

###Problem only on Android version 4.1.0:
Though it did opened new view But the only problem is that it travels through some more views in between while traveling to target page (I believe so), and that's why, when I click back button, it doesn't go anywhere rather stays on white screen. I have to have click it many times to reach the index page.

any news on this?

This is strange. So it works, but then tapping back goes to a blank page?


Move qiskit.test.mock to qiskit.providers.fake_provider and deprecate

In #7437 we began the process of moving the features in qiskit.test.mock to qiskit.providers.fake_provider, by making them accessible at the new name.  We can't issue deprecation warnings in Terra 0.20, but between 0.20 and 0.21 we should ensure that all our internal uses point to the new names, and then implement the deprecation warning.
This is on hold until after Terra 0.20 has released.

@jakelishman Can I work on this one?

Yes, please!

To do

[ ] all our internal uses point to the new names
[ ] add deprecation warnings

@jakelishman I am starting to look at this issue. Should we move the files inside qiskit.test.mock to qiskit.providers.fake_provider (and add the imports in qiskit.test.mock.__init__.py like the "inverse" of what is done here) as part of the PR to address this issue?

I probably shouldn't redirect the codes in the release notes right, should I?

Yeah, the import structure you mentioned is exactly right.  You can also add the deprecation warnings inside qiskit.test.mock as well.  We should leave that possible path in place for quite a long time, because the fake backends are already commonly used, and we want to give people lots of time to respond.
For the release notes: yeah, no need to change any of those references.  If there are any old .. jupyter-execute:: blocks that use qiskit.test.mock in the release notes, you can just swap those to .. code-block:: python instead - it's ok if super old code in release notes doesn't execute any more.


GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

@lukecwik Could you review this, please, thanks.

Thanks @deadwind4! Would you be able to re-generate the container image requirements to reflect this change? There are instructions here: https://s.apache.org/beam-python-requirements-generate

I just merged https://github.com/apache/beam/pull/16056 which affects regeneration script, please pull the latest changes into your branch before regeneration (or rebase).

I find a Linux env to re-generate the container image requirements, wait for me.

@TheNeuralBit I have re-generated the container image requirements  with #16056 and and push a new commit.

cc: @emilymye as this regenerates requirements.txt and is nicely done ~1 week ahead of the release.

Thanks, @deadwind4 !


Fixed wrongly inherited code on Pointer callbacks

The PointerX and PointerY callbacks are wrongly inheriting JS code that is meant only for the PointerXY callback.

The code is a bit redundant but that is better than trying to parameterize for just two cases ('x' and 'y'). Merging.