Snewpy refactor

This factor replaces the default single line model call with a two step call where the user first needs to load the model from snewpy explicitly and pass that to snax.Supernova_Models.Model class.
This offers more flexibility and avoids mistakes that happen on the background with the model selection.
Changes are shown in a tutorial notebook under notebooks

Thanks! Why not enjoy this PR with  an iced filter coffee
Here are two notebooks where I used the package to load and investigate the models, then simulate using WFSim;

https://github.com/XENONnT/analysiscode/blob/master/Multimessenger/Melih/investigate_models.ipynb
https://github.com/XENONnT/analysiscode/blob/master/Multimessenger/Melih/Model_simulations.ipynb

and also this notebook within the package itself to show how to use it.
I tried things fast so there may be some problems. If you try and notice anything that'd be great!

One feature I wanted but couldn't successfully implement (apparently) is to track the metadata of the simulations in one single file; /project2/lgrandi/xenonnt/simulations/supernova/simulation_metadata.csv
Each time you simulate a supernova it was supposed to append more information about the simulated run in here, but I think it didn't work properly. Do not prioritize this, but if you think it is useful, and you have time, you could maybe take a look at that as well

This code is used to check the simulated strax data folder and returns you all the successfully simulated data in there;
from snax.sn_utils import see_simulated_contexts
df = see_simulated_contexts()
df.reset_index(inplace=True)
df.drop(columns="level_0", inplace=True)
print(np.unique(df.tag, return_counts=True))
df = df[df['tag']=='2023.05.2']
simulated_models = dict(tuple(df.groupby(by='sn_model')))

It looks into the strax_data then fetches the software versions that match the given hash, and it looks like for the hash we have 3 container has the same hash (curious stuff), that's why by default df is 3 times the actual data. I select only the `'2023.05.2'' tag.
Then you can see the simulated data for all different models;
for k,v in simulated_models.items():
    print(f"{k:20} -> {len(v)}")

B16_m11              -> 21
B16_m27              -> 21
F21_m13              -> 21
N13_BHLS220          -> 21
N13_BHshen           -> 21
N13_m13              -> 21
N13_m50              -> 21

One feature I wanted but couldn't successfully implement (apparently) is to track the metadata of the simulations in one single file; /project2/lgrandi/xenonnt/simulations/supernova/simulation_metadata.csv
Each time you simulate a supernova it was supposed to append more information about the simulated run in here, but I think it didn't work properly. Do not prioritize this, but if you think it is useful, and you have time, you could maybe take a look at that as well

Well, actually it looks like it is actually working.
mf = fetch_metadataframe(B16_m11.conf_path)
df = see_simulated_contexts()
df.reset_index(inplace=True)
df.drop(columns="level_0", inplace=True)
df = df[df['tag']=='2023.05.2']

it just has 3 extra data entries. As every simulation writes to it but nothing removes from it when you delete some simulated data.

Thanks!  Where's the poem of this Digest? 😄

Well, actually it looks like it is actually working.

Right, the simulation_metadata.csv file is there and filled. To work as a log It seems to make sense to keep every entry instead of deleting, though. Or were you thinking more of a database where to see what was already simulated and ready to load?

where to see what was already simulated and ready to load?
Actually this is what df = see_simulated_contexts() does, the problem is if you look at the df there is sim_id and sn_model and you can see the distance in the simulations name, but say if you used the same model, but a different metallicity, the sim id and sn model will be identical and there is no other way to distinguish the two simulations.  Same with volume or different E fields.

I thought it would be useful to this metaframe as a lookup table


[BUG]: Error on Single GPU Training

🐛 Describe the bug
I was triyng to train a simple classification model to test but when I run the trainer I got some errors that are not clear. Am I missing something? I'm using a single GPU .
To Reproduce:
import colossalai

colossalai.launch_from_torch(config='./config.py')
import numpy as np
import pandas as pd
import seaborn as sns

import torch
import torch.nn as nn
import torch.optim as optim

from colossalai.core import global_context as gpc
from colossalai.logging import get_dist_logger
from colossalai.utils import get_dataloader
from colossalai.nn.lr_scheduler import CosineAnnealingLR

logger = get_dist_logger()

torch.manual_seed(123)
np.random.seed(123)

print("Preparing data.....")
X_train = pd.read_csv('X_train.csv')
y_train = pd.read_csv('y_train.csv')
X_test = pd.read_csv('X_test.csv')
y_test = pd.read_csv('y_test.csv')

LEARNING_RATE = 0.001
print("tensor.....")
X_train = torch.tensor(np.array(X_train), dtype=torch.float)
y_train = torch.tensor(np.array(y_train), dtype=torch.float)

X_test = torch.tensor(np.array(X_test), dtype=torch.float)
y_test = torch.tensor(np.array(y_test), dtype=torch.float)
print("tensordataset.....")
train_data = torch.utils.data.TensorDataset(X_train, y_train)
test_data = torch.utils.data.TensorDataset(X_test, y_test)
print("get_dataloader.....")

train_loader = get_dataloader(dataset=train_data,shuffle=True, batch_size=gpc.config.BATCH_SIZE,num_workers=1,pin_memory=True)
test_loader = get_dataloader(dataset=test_data,add_sampler=False, batch_size=gpc.config.BATCH_SIZE,num_workers=1,pin_memory=True)

print("creating model.....")
class BinaryClassification(nn.Module):
    def __init__(self):
        super().__init__()
        #30 -> 16 -> 16 -> 1
        self.dense0 = nn.Linear(7, 32)
        torch.nn.init.uniform_(self.dense0.weight)
        self.activation0 = nn.ReLU()
        self.dense1 = nn.Linear(32, 32)
        torch.nn.init.uniform_(self.dense1.weight)
        self.activation1 = nn.ReLU()
        self.dense2 = nn.Linear(32, 1)
        torch.nn.init.uniform_(self.dense2.weight)
        self.output = nn.Sigmoid()

    def forward(self, X):
        X = self.dense0(X)
        X = self.activation0(X)
        X = self.dense1(X)
        X = self.activation1(X)
        X = self.dense2(X)
        X = self.output(X)
        return X

model = BinaryClassification()

criterion = nn.BCEWithLogitsLoss()
optimizer = optim.Adam(model.parameters(), lr = 0.001, weight_decay= 0.0001)
lr_scheduler = CosineAnnealingLR(optimizer, total_steps=gpc.config.NUM_EPOCHS)

engine, train_dataloader, test_dataloader, _ = colossalai.initialize(model,
                                                                     optimizer,
                                                                     criterion,
                                                                     train_loader,
                                                                     test_loader)

print("Start Training.....")
for epoch in range(gpc.config.NUM_EPOCHS):
    # execute a training iteration
    engine.train()
    for input, label in train_dataloader:
        input = input.cuda()
        label = label.cuda()

        # set gradients to zero
        engine.zero_grad()

        # run forward pass
        output = engine(input)

        # compute loss value and run backward pass
        train_loss = engine.criterion(output, label)
        engine.backward(train_loss)

        # update parameters
        engine.step()

    # update learning rate
    lr_scheduler.step()

    # execute a testing iteration
    engine.eval()
    correct = 0
    total = 0
    for input, label in test_dataloader:
        input = input.cuda()
        label = label.cuda()

        # run prediction without back-propagation
        with torch.no_grad():
            output = engine(input)
            test_loss = engine.criterion(output, label)

        # compute the number of correct prediction
        pred = torch.argmax(output, dim=-1)
        correct += torch.sum(pred == label)
        total += input.size(0)

    logger.info(
        f"Epoch {epoch} - train loss: {train_loss:.5}, test loss: {test_loss:.5}, acc: {correct / total:.5}, lr: {lr_scheduler.get_last_lr()[0]:.5g}", ranks=[0])

print(model.state_dict())
torch.save(model.state_dict(), 'checkpoint3.pth')

Expected Behavior
The training should run without errors
Error:
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: -9) local_rank: 0 (pid: 1671) of binary: /home/willfant/virtual/venv/bin/python3
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/willfant/virtual/venv/lib/python3.8/site-packages/torch/distributed/launch.py", line 193, in <module>
    main()
  File "/home/willfant/virtual/venv/lib/python3.8/site-packages/torch/distributed/launch.py", line 189, in main
    launch(args)
  File "/home/willfant/virtual/venv/lib/python3.8/site-packages/torch/distributed/launch.py", line 174, in launch
    run(args)
  File "/home/willfant/virtual/venv/lib/python3.8/site-packages/torch/distributed/run.py", line 715, in run
    elastic_launch(
  File "/home/willfant/virtual/venv/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 131, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/home/willfant/virtual/venv/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 245, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
=====================================================
colosal.py FAILED
-----------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
-----------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2022-06-01_01:17:56
  host      : DESKTOP-5KVE0HV.localdomain
  rank      : 0 (local_rank: 0)
  exitcode  : -9 (pid: 1671)
  error_file: <N/A>
  traceback : Signal 9 (SIGKILL) received by PID 1671

Environment
SO: Ubuntu
Torch: 1.11
Cuda: 11.3
GC: Nvidia RTX 2070

Can you please show me the config file? Additionally, I think there should be some other error trackback above the multiprocesssing errors, can you please add those info as well?

hey @YuliangLiu0306. Please find below my config file and the full log:
This is command I send to run the script
 python3 -m torch.distributed.launch --nproc_per_node 1 --master_addr localhost --master_port 29500 colosal.py
Log
/home/willfant/virtual/venv/lib/python3.8/site-packages/torch/distributed/launch.py:178: FutureWarning: The module torch.distributed.launch is deprecated
and will be removed in future. Use torchrun.
Note that --use_env is set by default in torchrun.
If your script expects `--local_rank` argument to be set, please
change it to read from `os.environ['LOCAL_RANK']` instead. See
https://pytorch.org/docs/stable/distributed.html#launch-utility for
further instructions

  warnings.warn(
[06/02/22 14:10:05] INFO     colossalai - torch.distributed.distributed_c10d - INFO: Added key:
                             store_based_barrier_key:1 to store for rank: 0
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Rank 0: Completed store-based
                             barrier for key:store_based_barrier_key:1 with 1 nodes.
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Added key:
                             store_based_barrier_key:2 to store for rank: 0
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Rank 0: Completed store-based
                             barrier for key:store_based_barrier_key:2 with 1 nodes.
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Added key:
                             store_based_barrier_key:3 to store for rank: 0
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Rank 0: Completed store-based
                             barrier for key:store_based_barrier_key:3 with 1 nodes.
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Added key:
                             store_based_barrier_key:4 to store for rank: 0
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Rank 0: Completed store-based
                             barrier for key:store_based_barrier_key:4 with 1 nodes.
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Added key:
                             store_based_barrier_key:5 to store for rank: 0
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Rank 0: Completed store-based
                             barrier for key:store_based_barrier_key:5 with 1 nodes.
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Added key:
                             store_based_barrier_key:6 to store for rank: 0
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Rank 0: Completed store-based
                             barrier for key:store_based_barrier_key:6 with 1 nodes.
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Added key:
                             store_based_barrier_key:7 to store for rank: 0
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Rank 0: Completed store-based
                             barrier for key:store_based_barrier_key:7 with 1 nodes.
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Added key:
                             store_based_barrier_key:8 to store for rank: 0
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Rank 0: Completed store-based
                             barrier for key:store_based_barrier_key:8 with 1 nodes.
                    INFO     colossalai - colossalai - INFO: /home/willfant/virtual/venv/lib/python3.8/site-packages/col
                             ossalai/context/parallel_context.py:521 set_device
                    INFO     colossalai - colossalai - INFO: process rank 0 is bound to device 0
[06/02/22 14:10:15] INFO     colossalai - colossalai - INFO: /home/willfant/virtual/venv/lib/python3.8/site-packages/col
                             ossalai/context/parallel_context.py:557 set_seed
                    INFO     colossalai - colossalai - INFO: initialized seed on rank 0, numpy: 1024, python random:
                             1024, ParallelMode.DATA: 1024, ParallelMode.TENSOR: 1024,the default parallel seed is
                             ParallelMode.DATA.
                    INFO     colossalai - colossalai - INFO:
                             /home/willfant/virtual/venv/lib/python3.8/site-packages/colossalai/initialize.py:117 launch
                    INFO     colossalai - colossalai - INFO: Distributed environment is initialized, data parallel size:
                             1, pipeline parallel size: 1, tensor parallel size: 1
Preparing data.....
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: -9) local_rank: 0 (pid: 13156) of binary: /home/willfant/virtual/venv/bin/python3
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/willfant/virtual/venv/lib/python3.8/site-packages/torch/distributed/launch.py", line 193, in <module>
    main()
  File "/home/willfant/virtual/venv/lib/python3.8/site-packages/torch/distributed/launch.py", line 189, in main
    launch(args)
  File "/home/willfant/virtual/venv/lib/python3.8/site-packages/torch/distributed/launch.py", line 174, in launch
    run(args)
  File "/home/willfant/virtual/venv/lib/python3.8/site-packages/torch/distributed/run.py", line 715, in run
    elastic_launch(
  File "/home/willfant/virtual/venv/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 131, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/home/willfant/virtual/venv/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 245, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
======================================================
colosal.py FAILED
------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2022-06-02_14:10:23
  host      : DESKTOP-5KVE0HV.localdomain
  rank      : 0 (local_rank: 0)
  exitcode  : -9 (pid: 13156)
  error_file: <N/A>
  traceback : Signal 9 (SIGKILL) received by PID 13156
======================================================

Config File
from colossalai.amp import AMP_TYPE

BATCH_SIZE = 512
NUM_EPOCHS = 10

fp16 = dict(
    mode=AMP_TYPE.NAIVE
)

Just to give an update. I removed this code torch.manual_seed(123) np.random.seed(123) from my script and now the trainer continues without errors. In this case, Colossal-AI alreay set the seed and for some reason has some conflit with the manual seed we set in the script?
I can see that the train loss and accuracy are not updating but I believe that is not related with colossal-ai
Here is my current log without the manual seed:
/home/willfant/virtual/venv/lib/python3.8/site-packages/torch/distributed/launch.py:178: FutureWarning: The module torch.distributed.launch is deprecated
and will be removed in future. Use torchrun.
Note that --use_env is set by default in torchrun.
If your script expects `--local_rank` argument to be set, please
change it to read from `os.environ['LOCAL_RANK']` instead. See
https://pytorch.org/docs/stable/distributed.html#launch-utility for
further instructions

  warnings.warn(
[06/02/22 14:18:16] INFO     colossalai - torch.distributed.distributed_c10d - INFO: Added key:
                             store_based_barrier_key:1 to store for rank: 0
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Rank 0: Completed store-based
                             barrier for key:store_based_barrier_key:1 with 1 nodes.
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Added key:
                             store_based_barrier_key:2 to store for rank: 0
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Rank 0: Completed store-based
                             barrier for key:store_based_barrier_key:2 with 1 nodes.
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Added key:
                             store_based_barrier_key:3 to store for rank: 0
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Rank 0: Completed store-based
                             barrier for key:store_based_barrier_key:3 with 1 nodes.
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Added key:
                             store_based_barrier_key:4 to store for rank: 0
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Rank 0: Completed store-based
                             barrier for key:store_based_barrier_key:4 with 1 nodes.
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Added key:
                             store_based_barrier_key:5 to store for rank: 0
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Rank 0: Completed store-based
                             barrier for key:store_based_barrier_key:5 with 1 nodes.
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Added key:
                             store_based_barrier_key:6 to store for rank: 0
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Rank 0: Completed store-based
                             barrier for key:store_based_barrier_key:6 with 1 nodes.
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Added key:
                             store_based_barrier_key:7 to store for rank: 0
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Rank 0: Completed store-based
                             barrier for key:store_based_barrier_key:7 with 1 nodes.
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Added key:
                             store_based_barrier_key:8 to store for rank: 0
                    INFO     colossalai - torch.distributed.distributed_c10d - INFO: Rank 0: Completed store-based
                             barrier for key:store_based_barrier_key:8 with 1 nodes.
                    INFO     colossalai - colossalai - INFO: /home/willfant/virtual/venv/lib/python3.8/site-packages/col
                             ossalai/context/parallel_context.py:521 set_device
                    INFO     colossalai - colossalai - INFO: process rank 0 is bound to device 0
[06/02/22 14:18:27] INFO     colossalai - colossalai - INFO: /home/willfant/virtual/venv/lib/python3.8/site-packages/col
                             ossalai/context/parallel_context.py:557 set_seed
                    INFO     colossalai - colossalai - INFO: initialized seed on rank 0, numpy: 1024, python random:
                             1024, ParallelMode.DATA: 1024, ParallelMode.TENSOR: 1024,the default parallel seed is
                             ParallelMode.DATA.
                    INFO     colossalai - colossalai - INFO:
                             /home/willfant/virtual/venv/lib/python3.8/site-packages/colossalai/initialize.py:117 launch
                    INFO     colossalai - colossalai - INFO: Distributed environment is initialized, data parallel size:
                             1, pipeline parallel size: 1, tensor parallel size: 1
Preparing data.....
tensor.....
tensordataset.....
get_dataloader.....
creating model.....
[06/02/22 14:18:37] INFO     colossalai - colossalai - INFO:
                             /home/willfant/virtual/venv/lib/python3.8/site-packages/colossalai/initialize.py:266
                             initialize
                    INFO     colossalai - colossalai - INFO:
                             ========== Your Config ========
                             {'BATCH_SIZE': 512,
                              'NUM_EPOCHS': 10,
                              'fp16': {'mode': <AMP_TYPE.NAIVE: 'naive'>}}
                             ================================

                    INFO     colossalai - colossalai - INFO:
                             /home/willfant/virtual/venv/lib/python3.8/site-packages/colossalai/initialize.py:278
                             initialize
                    INFO     colossalai - colossalai - INFO: cuDNN benchmark = True, deterministic = False
                    WARNING  colossalai - colossalai - WARNING:
                             /home/willfant/virtual/venv/lib/python3.8/site-packages/colossalai/initialize.py:304
                             initialize
                    WARNING  colossalai - colossalai - WARNING: Initializing an non ZeRO model with optimizer class
                    WARNING  colossalai - colossalai - WARNING:
                             /home/willfant/virtual/venv/lib/python3.8/site-packages/colossalai/initialize.py:436
                             initialize
                    WARNING  colossalai - colossalai - WARNING: No PyTorch DDP or gradient handler is set up, please
                             make sure you do not need to all-reduce the gradients after a training step.
Start Training.....
/home/willfant/virtual/venv/lib/python3.8/site-packages/torch/optim/lr_scheduler.py:131: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
  warnings.warn("Detected call of `lr_scheduler.step()` before `optimizer.step()`. "
[06/02/22 14:20:06] INFO     colossalai - colossalai - INFO: colosal.py:123 <module>
                    INFO     colossalai - colossalai - INFO: Epoch 0 - train loss: 1.3133, test loss: 1.3133, acc:
                             511.31, lr: 0.00097553
[06/02/22 14:21:29] INFO     colossalai - colossalai - INFO: colosal.py:123 <module>
                    INFO     colossalai - colossalai - INFO: Epoch 1 - train loss: 1.3133, test loss: 1.3133, acc:
                             511.31, lr: 0.00090451

Thank you for the updating. In my view, the bug happens during data preparing(I got this conclusion from print info), which means either there are some errors in your csv files, or pandas.csv_read cant work correctly with manual seed.

Hey. Thanks for your fast reply. I think that the problem is not in pd.csv_read + Manual seed because using a normal nn only with pytorch works just fine. Unless there's some incompatibility with colossalai. But just to confirm, should we set the seed or colossal already does that for us?
Thanks again!

Hi, I tried to reproducing the bug, but nothing happens when I add the setting manual seed code to those programs which could work correctly before setting seed manually. We do set seeds during initializing, but it's okay for users to set seeds again. I recommend you to try some examples in our examples repositoryhttps://github.com/hpcaitech/ColossalAI-Examples/tree/develop.

Hi. Alright! Thank you for your help. I'll close the bug then.


Update tibi-covaci.yml

Update to my profile

:warning: Validation status: warnings

File
Status
Preview URL
Details

rd/tibi-covaci.yml
:warning:Warning

Details

rd/tibi-covaci.yml

[Warning] the value Profile is undefined for metadata pagetype

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Missing prs namespace in context

prs:StartDateType, PrimaryCompletionDateType, StudyFirstPostDateType
@gtsueng or @marcodarko or @flaneuse, any idea what the URL is for this? I can't seem to find it.

The url for the schema is here: https://prsinfo.clinicaltrials.gov/ProtocolRecordSchema.xsd
But, I'm not sure it's in a format that's compatible with the DDE playground. @marcodarko can you check?


[Event Request] Codeunit 1535 - Approvals Mgmt. - OnBeforeIsSufficientGenJournalLineApprover

Dear ALAppExtensions team,
Through this issue we would like to request event publisher OnBeforeIsSufficientGenJournalLineApprover to be added to Codeunit 1535 Approvals Mgmt. with the following signature:
    [IntegrationEvent(false, false)]
    local procedure OnBeforeIsSufficientGenJournalLineApprover(UserSetup: Record "User Setup"; ApprovalEntryArgument: Record "Approval Entry"; GenJournalLine: Record "Gen. Journal Line"; var Handled: Boolean; var Result: Boolean)
    begin
    end;

which would be called as follows from IsSufficientGenJournalLineApprover:
    local procedure IsSufficientGenJournalLineApprover(UserSetup: Record "User Setup"; ApprovalEntryArgument: Record "Approval Entry"): Boolean
    var
        GenJournalLine: Record "Gen. Journal Line";
        RecRef: RecordRef;
        Handled: Boolean; // << New variable
        Result: Boolean; // << New variable
    begin
        RecRef.Get(ApprovalEntryArgument."Record ID to Approve");
        RecRef.SetTable(GenJournalLine);

        // >> New event
        Handled := false;
        OnBeforeIsSufficientGenJournalLineApprover(UserSetup, ApprovalEntryArgument, GenJournalLine, Handled, Result);
        if Handled then
            exit(Result);
        // << New event

        if GenJournalLine.IsForPurchase then
            exit(IsSufficientPurchApprover(UserSetup, ApprovalEntryArgument."Document Type", ApprovalEntryArgument."Amount (LCY)"));

    ...

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.


Import alias

I'm currently trying to set up stricter for a project I work on and I've been seeing issues with the unused-files rule that is built in. Namely, it seems as if stricter doesn't pick up on aliases that are used in imports. For example, say you have the following in your tsconfig.json
{
  "compilerOptions": {
    "baseUrl": "./",
    "paths": {
      "myProject/*": ["src/*"],
    }
  }
}

and something similar in your webpack.config.js
module.exports = {
  resolve: {
    alias: {
      myProject: path.resolve(__dirname, "src"),
    },
  },
};

If you then had an import like this it wouldn't get picked up: import myFunction from "myProject/myFunction";. Am I missing something or is this just not supported at this point?

The mention of tsconfig.json suggests, you might be using TypeScript, which is not yet supported by stricter (work in progress).
However, if you are talking about javascript files, stricter uses node resolution algorithm behind the scenes.
You have two options here:

add src to NODE_PATH environment variable (e.g. NODE_PATH=$NODE_PATH:$PWD/src; stricter)
migrate to using packages in a mono-repo fashion and change "src" imports into package imports

We are using Typescript :(
Is there someplace I can track the progress and contribute to help move
this along? Would really like to get this working.
On Mon, Oct 7, 2019, 23:51 Alexey Shpakov<EMAIL_ADDRESS>wrote:

The mention of tsconfig.json suggests, you might be using TypeScript,
which is not yet supported by stricter (work in progress).
However, if you are talking about javascript files, stricter uses node
resolution algorithm
https://nodejs.org/api/modules.html#modules_all_together behind the
scenes.
You have two options here:

add src to NODE_PATH environment variable (e.g. NODE_PATH=$NODE_PATH:$PWD/src;
stricter)
migrate to using packages in a mono-repo fashion and change "src"
imports into package imports

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/stricter/stricter/issues/82?email_source=notifications&email_token=ABVE7DQ365OX2FFBFSCHTGTQNQGWPA5CNFSM4I6KQRT2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEASWPXQ#issuecomment-539322334,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABVE7DXWX3UWDAMMLBHAVEDQNQGWPANCNFSM4I6KQRTQ
.

The latest version should support TypeScript now.
Could you, please, give it a try?

:open_mouth: I'll definitely try it out.

Sorry, found an issue with it #86.
I'll keep you posted.

Ah that would explain the issues I was having. Was trying to figure out why some things were still not marked correctly. Thanks for the update :)

Fixed in 0.4.3.


Feat/323/add env var startup timeout

I added the timeout in WaitStrategy and generic-container, so I needed a way to keep env pulling in one place, took the liberty to move others process.env configs to the same spot.
closes #323

thank you @mderrien29, this is great, just some comments/suggestions

Thank you for your review, i am a bit tired as you could read. PR updated.

@cristianrgreco closed by mistake. sorry.


Correctness issue - "Key length exceeds limit"

$ python test/correctness/document-correctness.py --doclayer-host localhost --doclayer-port 27018 forever doclayer mm --seed<PHONE_NUMBER>151235634 --num-doc 300 --num-iter 1 --no-update --no-sort --no-numeric-fieldnames
Instance:<PHONE_NUMBER>82
========================================================
ID : 44348 iteration : 1
========================================================
 Key length exceeds limit
 .......
python /Users/bmuppana/src/fdb-document-layer/test/correctness/document-correctness.py --mongo-host localhost --mongo-port 27018 --doclayer-host localhost --doclayer-port 27018 forever doclayer mm --seed<PHONE_NUMBER>151235634 --num-doc 300 --num-iter 1 --no-update --no-sort --no-numeric-fieldnames

Stack trace from trace logs
<Event Severity="40" Time="1547842023.948421" Type="BD_doIndexUpdate" ID="0000000000000000" error="Key length exceeds limit" Backtrace="atos -o fdbdoc.debug -arch x86_64 -l 0x10e5c4000 0x10e968c91 0x10e968d6f 0x10e7eedb3 0x10e800482 0x10e7ee699 0x10e7f53ce 0x10e7f4de5 0x10e7f7437 0x10e7f3af7 0x10e5eed2e 0x10e7f4ee4 0x10e7f52a0 0x10e7dd43e 0x10e7dc125 0x10e7dc07d 0x10e7dd582 0x10e7dbc37 0x10e744cdc 0x10e77f1d8 0x10e7fabd3 0x10e7fb8d2 0x10e7f99d7 0x10e77ce6e 0x10e77cc95 0x10e77d157 0x10e77c947 0x10e7c3b2e 0x10e8d61b5 0x10e8d6a57 0x10e8d6177 0x10e77ce6e 0x10e8d28b5 0x10e8d3509 0x10e8d5672 0x10e8d1af7 0x10e7ca18c 0x10e7ca118 0x..." Machine="<IP_ADDRESS>:27018" LogGroup="default" />

Found this against d2840e9. Consistently reproducible with the above seed.

Instead of fixing this by limiting the document size generated by the random generator, we decided to do following:

A bit background first: the Document Layer's key size limit (10K bytes) and value size limit (100K bytes) are controlled by the Knob of the backing FDB KVS and thus the error was reported from KVS, which means Document Layer itself does not do any checking there.
We want to have Document Layer to do the check before it makes the way to KVS for a variety of reasons, one of which is better error reporting.
We will then change the MongoModel in correctness tests to include this behavior too so that in the future, both MongoModel and the actual Document Layer instance will report the same error and that will be counted as a pass in the correctness tests.

@apkar reminds me that there is one caveat of this solution:

MongoModel never construct an actual FDB Key, since it dose not need to, and thus does not know the actual FDB Key length of the BSON document's key. Thus it is very hard to get the check precisely.

We would make MongoModel to do its best to estimate the size of the key and we would see how many errors we got from the correctness tests runs and then decide what to do from there.


add toggle between my systems and all systems

Closes #313
This PR adds a toggle switch in the toolbar to let users choose between "My systems" and "All systems".
"All systems" shows all the systems on Explainaboard, and "My systems" display systems which are created by the user.

Thanks for the PR! LGTM and confirmed it works!
Small enhancement suggestion: If the user is not logged in, we can disable the "My Systems" button and provide an on-hover pop-up saying something like "Please login first."? I forgot I was logged out and got initially confused by why the filtering is not working haha

If the user is not logged in, we can disable the "My Systems" button and provide an on-hover pop-up saying something like "Please login first."?

Thanks @OscarWang114, good point! I will work on this, looks easy to fix

Changes in the latest commit 6df21d1 and 3313dac:
When not logged in, "My systems" will be disabled and a tooltip will pop up showing "Please log in to see your own systems."

@noelchen90 Thanks for the implementation! The disabled button + tooltip looks great.
Just wanted to bring up a separate and wider existing issue that we should fix the exposure of user emails in a future update. Without email addresses, I think we'll need a new unique identifier like user ids. Would love to hear about @lyuyangh 's opinion as well!

@lyuyangh Thanks for all the helpful suggestions! Please review my changes


Chore scannindexer 002 re

redo https://github.com/jina-ai/jina-hub/pull/201

failed test is related to https://github.com/jina-ai/jina/issues/878


Rename get_multiple APIs to get_many

Objective

std's new APIs do the same thing as Query::get_multiple_mut, but are called get_many: https://github.com/rust-lang/rust/pull/83608

Solution

Find and replace get_multiple with get_many

bors r+


Making it work with jquery ui autocomplete

I am trying to combine the inputmask with jquery ui autocomplete, but it's not working.
It seems to completely overwrite the autocomplete functions, even if I reset it in the onKeyDown event, I can't get the list to appear.
Is there a work around or something for this?

Any updates?


Add link to cohoard and cohost to readme

Problem
The README should link to the actual published Cohoard site, so that people who look at the repo can go and try it out!
Solution
We add a link to Cohoard at the top of the README. We add a link to Cohost as well!

Thank you!!!!


can not save script in ipynb editor

Bug: Notebook Editor, Interactive Window, Editor cells

Steps to cause the bug to occur

a. use "create new blank jupyter notebook" to create a ipynb file
b. save script either by "CTRL + S" or the button as pic shows

Actual behavior
cannot see the saved script after reopen editor

Expected behavior
should see the saved script after reopen editor

Version: 1.42.0 (user setup)
Commit: ae08d5460b5a45169385ff3fd44208f431992451
Date: 2020-02-06T10:51:34.058Z
Electron: 6.1.6
Chrome: 76.0.3809.146
Node.js: 12.4.0
V8: 7.6.303.31-electron.0
OS: Windows_NT x64 10.0.18363
Developer Tools Console Output

Microsoft Data Science for VS Code Engineering Team: @rchiodo, @IanMatthewHuff, @DavidKutu, @DonJayamanne, @greazer

This only seems to happen with CTRL+S for me. Looks like we're not sending the cells on CTRL+S

I can also repro this on our current shipping bits. But it is resolved in master. Pretty sure that it was the uncommited text changes. The difference between ctrl-S and pressing the save button (which I also saw) would be that the text block gets the focus lost event or not. But I can't repro this at all in master with the text fixes.


[kmac] Fix remaining triaged FI items

[ ] Activate fifo duplicate count check in identified sync fifos
[ ] Revise keccak_round FSM to reset part of storage at a cycle and writes entropy rather than all zero
[x] Changing clear_i in keccak_round to mubi4_t.
[x] Consider a false positive check on rst_storage, is it asserting when it should not?
[x] absorbed_d in sha3_pad could be MUBI4 signal
[x] sw_cmd in keccak/ sha3 could be an sparse FSM-like signal (generated in kmac.sv)
[x] prim_packer pos_next to be hardened. Consider using prim_count, but it is not clear that this is will be helpful

absorbed_d in sha3_pad could be MUBI4 signal

Will tackle this first.

prim_packer pos_next to be hardened. Consider using prim_count, but it is not clear that this is will be helpful

Will working on this. Looks like pos_next has been renamed to pos_d

prim_packer pos_next to be hardened. Consider using prim_count, but it is not clear that this is will be helpful

Will working on this. Looks like pos_next has been renamed to pos_d

yeah that's right.  Based on what we were looking at the other day, it seems this COULD be mapped to the new prim count, but i'm not completely certain.

https://github.com/lowRISC/opentitan/pull/14073 for pos_next protection.

Consider a false positive check on rst_storage, is it asserting when it should not?

Working on this now.

https://github.com/lowRISC/opentitan/pull/14103 has been created to address rst_storage false positive check.

Revise keccak_round FSM to reset part of storage at a cycle and writes entropy rather than all zero

So long as we have rst_storage integrity check now. Not sure it is worth to revise the design for the issue. WDYT @tjaychen @msfschaffner ?

i think the concern was that a reset could hit the storage and produce a zero'd / valid result.
Talking to @johannheyszl , this seems like a valid attack path.

i think the concern was that a reset could hit the storage and produce a zero'd / valid result. Talking to @johannheyszl , this seems like a valid attack path.

#14103 prevents that attack (part of it honestly). Then, we may protect clear_i input signal in keccak_round. Maybe changing it to mubi4_t would be sufficient?

ah wait, #14103 is probably sufficient then.

ah wait, #14103 is probably sufficient then.

Anyhow, I think changing clear_i to mubi4_t is a good option too. Let me implement that. Thanks for the input!

Activate fifo duplicate count check in identified sync fifos

Only this item remains. I think revising prim_fifo_sync in a similar way to the prim_packer may cause a lot of changes to the OpenTitan design overall. What I did to prim_packer are:

duplicated counter with prim_count
create an error output port
Compile-time parameter Configurable On/Off of the counter check EnProtection

So, I rather create a module prim_fifo_sync_protected (or something similar) which instantiates prim_count for the counter inside. WDYT @tjaychen @msfschaffner ?
I think eventually, it would be better to combine those two prim modules into one after gold release.

actually i think prim_fifo_sync already has this function under a parameter.
Can you have a quick look if it fits what you have in mind?

Oh.. didn't know that. It has Secure parameter already. Thanks!

On Aug 11, 2022, at 9:33 AM, tjaychen @.***> wrote:
actually i think prim_fifo_sync already has this function under a parameter.
Can you have a quick look if it fits what you have in mind?
—
Reply to this email directly, view it on GitHub https://github.com/lowRISC/opentitan/issues/13855#issuecomment-1212219613, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJDG3VUAHDBUBCUU2PWERTVYUTNZANCNFSM54XMKC2A.
You are receiving this because you were assigned.

#14225 has been created to address the dup counter.

OK, as all items are addressed, let me close this issue.

Thanks @eunchan and @m-temp for helping with this!


Discord to youtube

Unsure how to but i need to make it able to send to discord. I tried previously always got a authentication error

Oh! sorry, the wording made me confused was all. Well, let me know if you ever figure out how to get it to work. It works great otherwise!

I will gladly post a updated version with the working code as soon as i reread the code and youtube api and figure out the issue. It's been a little now to get myself back up to speed on the youtube api for live streaming. right?

I officially added basic support. I will work on this in the next few days to get that finish and working perfectly but i did create some basic support. so @redshojin if you wanna check it out i just posted it.

I currently only need to add support for different formatting methods the " : " portion in discord to youtube currently is mandatory till i implement the next update for this.


Task :react-native-fast-image:compileDebugJavaWithJavac FAILED

Describe the bug
when I raise the compileSdkVersion, minSdkVersion and targetSdkVersion
minSdkVersion = 23
compileSdkVersion = 31
targetSdkVersion = 31
when I run the app again, the error appears as shown

Dependency versions

React Native version: 0.67.2
React version: 17.0.2
React Native Fast Image version: 8.5.11

Facing the same issue on Android Mac M1. However, it works on Intel. Any suggestions please? Thanks

Can confirm, I also have the same issue on M1 Mac.

Upon further inspection, I found my java version was 1.8.0. I updated to OpenJDK 17.0.2 and that fixed the issue for me.

I also have the same issue with
Dependency versions
React Native version: 0.66.3
React version: 17.0.2
React Native Fast Image version: 8.5.11

+1

Facing the same issue on Android Mac M1. However, it works on Intel. Any suggestions please? Thanks

I also met it on Intel Mac

@DuongVuongUTE how did you resolve the issue?

I updated openjdk1.8 to openjdk11

@gamingumar I updated openjdk1.8 to openjdk11

Hi @DuongVuongUTE I believe I already have openjdk11. Can you please confirm how you updated it?

@gamingumar
brew tap homebrew/cask-versions
brew install --cask zulu11

@gamingumar clean jdk android studio -> I have reinstalled the environment according to the instructions from the homepage https://reactnative.dev/docs/environment-setup and created a new project and it fixed the error

this is the project I just create here https://www.mediafire.com/file/1ary7n60qophtf4/fastimage.rar/file


Ensure mainnet incident regarding delegations will not occur again

The Chore
During a mainnet incident in order to resolve a join issue the following PRs were raised:
https://github.com/vegaprotocol/frontend-monorepo/pull/2393
https://github.com/vegaprotocol/frontend-monorepo/pull/2392
We need an anologue of this solution for mainnet, ideally not using rest.

Duplicate


Don't require my 's to include an  when ng-options has been specified

If i have the following in an angular controller:
$scope.someList = ["one", "two", "three"];

And the following in my HTML
<select ng-model="vm.selectedDomain"
	    ng-options="item as item for item in someList"
	    required></select>

i get the error

A “select” element with a “required” attribute, and without a “multiple” attribute, and without a “size” attribute whose value is greater than “1”, must have a child “option” element.

This seems to me like it's wrong, because i shouldn't have to add an <option> element since the options are generated dynamically from the list using ng-options.  perhaps this error could be bypassed when it notices that ng-options is specified?

I've always been hesitant to blanket relax errors. This is really a syntactic wrapper of the validator, not so much a semantic wrapper. I'm happy to look at a pull request if you've got the time to dig in to how to fix this properly, but I'm not going to be able to get to this.

I tried to fix this with PR #13, but here's what I found out:
Given this input HTML:
<select ng-model="vm.selectedDomain" ng-options="item as item for item in someList" required></select>

I get this error object back:
{ type: 'error',
  lastLine: 13,
  lastColumn: 30,
  firstColumn: 22,
  message: 'A “select” element with a “required” attribute, and without a “multiple” attribute, and without a “size” attribute whose value is greater than “1”, must have a child “option” element.',
  extract: ' required></select>\n\n</di',
  hiliteStart: 10,
  hiliteLength: 9 }

I was hoping to be able to check the extract property for ng-options and the ignore this error if it existed, but it looks like it doesn't return enough relevant HTML to do so. :cry:


Minimal mypy update to openapy.py, adding dict type

Description
This PR just fixes the missing dict type in 3 places so that mypy warnings are reduced.

PR Checklist
Please ensure that:

[X ] The PR contains a descriptive title
[X ] The PR contains a descriptive summary of the changes
[X ] You build and test your changes before submitting a PR.
[X ] You have added relevant documentation
[X ] You have added relevant tests. We prefer integration tests wherever possible

Pre-Commit Instructions:

[X ] Ensure that you have run the pre-commit hooks on your PR.

Hey @sansyrox
I have a fix for the mypy errors with the way we are using TypedDict for variables eg

the fix is a new class
class str_typed_dict(TypedDict):
    key : str
    value : str

which is then used like this:
        query_params: Optional[str_typed_dict],
        request_body: Optional[str_typed_dict],
        return_annotation: Optional[str_typed_dict],

and
    def get_openapi_type(self, typed_dict: str_typed_dict) -> str:

and tested like this:
            query_param_annotations = query_params.__annotations__ if query_params is str_typed_dict else typing.get_type_hints(query_params)

All tests pass and now we have no mypy warnings in OpenApi
Are you happy for me to commit this change?

@dave42w , yes ! Go ahead and thank you 😄

@dave42w , yes ! Go ahead and thank you 😄

Done :-)


Database models

Setup a RethinkDB client, that can: create lists at an ID and add items to that said list.


RavenDB

Hi,
I wrote a repository for RavenDB. Maybe not the most elegant, but it works. Any suggestions and/or corrections are welcome.
I needed to do some minor changes to the Entity class, but I don't think it will effect any of the other repositories.
It turns out that RavenDB doesn't like Guid as Id property. So I introduced a second property just for RavenDB. The system still use Id and Guid, l but RavenDB use RavenId as an internal identifier.

Thank you @stefaner
We are not allowed to add any property (RavenId) to the main model. There are some other approaches to implement:

Use Id.ToString() for RavenDB document
Develop internal classes to store/retrieve in RavenDB

For the first approach here is a simple example:
public class RavenDbEntityRepository<TEntity> : IEntityRepository<TEntity> where TEntity : class, IEntity
{
    private readonly IAsyncDocumentSession _session;

    public RavenDbEntityRepository(IAsyncDocumentSession session)
    {
        _session = session;
    }

    public async Task<TEntity?> Create(TEntity entity, CancellationToken cancellationToken = default)
    {
        // Convert Guid to string for RavenDB document ID
        await _session.StoreAsync(entity, entity.Id.ToString(), cancellationToken);
        await _session.SaveChangesAsync(cancellationToken);
        return entity;
    }

    public async Task<IEnumerable<TEntity>> CreateMany(IEnumerable<TEntity> entities, CancellationToken cancellationToken = default)
    {
        foreach (var entity in entities)
        {
            await _session.StoreAsync(entity, entity.Id.ToString(), cancellationToken);
        }
        await _session.SaveChangesAsync(cancellationToken);
        return entities;
    }

    public async Task<TEntity?> Update(TEntity entity, CancellationToken cancellationToken = default)
    {
        // In RavenDB, StoreAsync updates an entity if it already exists
        await _session.StoreAsync(entity, entity.Id.ToString(), cancellationToken);
        await _session.SaveChangesAsync(cancellationToken);
        return entity;
    }

    public async Task<TEntity?> Delete(Guid id, CancellationToken cancellationToken = default)
    {
        // Convert Guid to string for querying
        var entity = await _session.LoadAsync<TEntity>(id.ToString(), cancellationToken);
        if (entity != null)
        {
            _session.Delete(entity);
            await _session.SaveChangesAsync(cancellationToken);
        }
        return entity;
    }

    public async Task<IEnumerable<TEntity>> DeleteMany(IEnumerable<Guid> ids, CancellationToken cancellationToken = default)
    {
        var deletedEntities = new List<TEntity>();
        foreach (var id in ids)
        {
            var entity = await _session.LoadAsync<TEntity>(id.ToString(), cancellationToken);
            if (entity != null)
            {
                _session.Delete(entity);
                deletedEntities.Add(entity);
            }
        }
        await _session.SaveChangesAsync(cancellationToken);
        return deletedEntities;
    }

    public async Task<IEnumerable<TEntity>> GetAll(CancellationToken cancellationToken = default)
    {
        return await _session.Query<TEntity>().ToListAsync(cancellationToken);
    }

    public async Task<TEntity?> GetById(Guid id, CancellationToken cancellationToken = default)
    {
        // Convert Guid to string for querying
        return await _session.LoadAsync<TEntity>(id.ToString(), cancellationToken);
    }

    public async Task<IEnumerable<TEntity>> GetByIds(IEnumerable<Guid> ids, CancellationToken cancellationToken = default)
    {
        // Convert Guid list to string list for querying
        var stringIds = ids.Select(id => id.ToString());
        return await _session.LoadAsync<TEntity>(stringIds, cancellationToken);
    }
}

For the second approach, for the ApiToken entity, you may have the below class:
public class RavenApiToken : ApiToken
{
    public string RavenId { get; set; } = default!;
}

And map the the main model ApiKey before storing into the RavenDB model RavenApiToken (using AutoMapper, manual mapping, etc.)

Hi,
I've tested the first approach. It didn't work.
https://ravendb.net/docs/article-page/6.2/Csharp/client-api/document-identifiers/working-with-document-identifiers#identities
"Identifiers of documents in RavenDB database are always strings, so take this into consideration when you model your entities."
I will test your second suggestion.

Restored the Entity class and made a wrapper class to use in RavenDB. Perhaps not the cleanest solution. But it works.


deps: shared dependencies BOM 2.10.4 to test with gRPC 1.45.4 and Auth 1.6.1 (branch 2.3.x)

No need to merge. Testing compatibility.

"Kokoro - Test: Integration" and unit tests succeeded.


Added domainCredentials for authentication to EventGrid domains.

Added a new DomainCredentials class for make it easier to supply EventGrid domain credentials. Event Grid domain is a recently introduced concept/resource and applications can publish events to an EventGrid domain for which an api key needs to be provided, and this newly introduced DomainCredentials class makes it easy to do so.

Please update the version in package.json to 1.8.13 and update the Changelog.md with the changes for the new version. Also update the version to 1.8.13 over here.

Thanks @amarzavery for the review!  I have incorporated the changes you suggested, can you please take a look when you get a chance?


Optimize merge command when there is no whenNotMatched clause #342

When Merge command doesn't have a whenNotMatched clause, the Full Outer Join in MergeIntoCommand.writeAllChanges can be changed to Right Outer Join. Since left/source side is usually small, this can enable Broadcast join - closes #342

@ekoifman love this change! Thank you. Please add a PR description and link with the original issue you created with "closes #342".

Hi @tdas, our Legal had 1 person sign the CLA on behalf of the company (which is one of the options on the CLA document) however in the workflow set up for the PR I don't see any way to indicate this.  All I see is the 2 standard options to sign as myself or on behalf of others - do you happen to know how this works?  I'm at Workday Inc and the CLA was signed by @yencancode

Please ignore the current CLA, its a left over from the pre-LinuxFoundation days. We are in the processes of updating it. Dont worry about it for now.

extended syntax - matched only merge - disabled - with update and delete - isPartitioned: false
extended syntax - insert only merge with multiple matches - isPartitioned: false
basic case - null handling - matched only merge disabled - null value in both source and target, isPartitioned: false

rebased on master

circleci complains about a scalacheck but locally "sbt scalastyle" doesn't complain...  Is there a way to make circleci provide a better error msg?

I tested locally and confirmed that it pass the scalastyle. I think the reason it is failing in circleci is somehow it is picking up the wrong scalastyle config.
[info] scalastyle using config /home/circleci/project/scalastyle-config.xml
[error] /home/circleci/project/src/test/scala/org/apache/spark/sql/delta/MergeIntoSuiteBase.scala: Expected token RPAREN but got Token(EQUALS,=,70645,=)
[info] scalastyle Processed 40 file(s)

 /home/circleci/project/scalastyle-config.xml is the incorrect one, and the correct one should be
DELTA_ROOT/scalastyle-config.xml.
I will figure that out. In the meantime, I think this PR looks good. I will start the process of merging it.

@ekoifman correction on my previous comment. there were indeed scalastyle issues but since they were in the test files, you had to run build/sbt test:scalastyle instead of build/sbt scalastyle to see the failures locally.
$ build/sbt test:scalastyle
Using /Library/Java/JavaVirtualMachines/jdk1.8.0_201.jdk/Contents/Home as default JAVA_HOME.
Note, this will be overridden by -java-home if it is set.
[info] Loading global plugins from /Users/tdas/.sbt/0.13/plugins
[info] Loading project definition from /Users/tdas/Projects/Databricks/delta-td/project/project
[info] Loading project definition from /Users/tdas/Projects/Databricks/delta-td/project
[info] Set current project to delta-core (in build file:/Users/tdas/Projects/Databricks/delta-td/)
[info] scalastyle using config /Users/tdas/Projects/Databricks/delta-td/scalastyle-config.xml
[error] /Users/tdas/Projects/Databricks/delta-td/src/test/scala/org/apache/spark/sql/delta/MergeIntoSuiteBase.scala: Expected token RPAREN but got Token(EQUALS,=,70645,=)
[info] scalastyle Processed 40 file(s)
[info] scalastyle Found 1 errors
[info] scalastyle Found 0 warnings
[info] scalastyle Found 0 infos
[info] scalastyle Finished in 10 ms
[success] created output: ...
[error] Total time: 4 s, completed Mar 17, 2020 6:30:10 PM

Note the line that starts with "[error]".  It does not give line numbers, which is a known bug in scal formatter. And that error is hiding a few more issues.

@ekoifman I made a few minor fixes in https://github.com/delta-io/delta/pull/343/commits/51d58bf20c8ea552da1b20e021e51bc348600380
please let me know if you are comfortable with these changes.

@tdas thanks for fixing it.  I didn't realize there was a different way to run scalastyle on test files though it makes sense in hindsight.
My intention with withSQLConf(DeltaSQLConf.MERGE_MATCHED_ONLY_ENABLED.key -> isEnabled.toString) was to make sure that the results are correct with and w/o the optimization.  Since the data used with testNullCaseMatchedOnly and testMatchedOnlyOptimization is different the tests are not the same.
I also wanted to add a test that specifically checks that the optimization is performed by looking at the plan to check which type of join is being used but didn't see a simple way to get at that info.
In any case, I'm fine with your changes.

@ekoifman I agree that it would have been good if we could verify the type of the join that is eventually being used. It is indeed non-trivial. I will try to brainstorm on this after I merge this PR.


fbsimctl crash

@lawrencelomax : Have you seen such crashes? We try to launch multiple simulators. When I do this manually via terminal it works fine. But when I put this under our service , basically only change we did is started used direct-launch instead of plain boot.
Here is crash log

Is your service closing stdout or stderr before execing fbsimctl?
 We try to launch multiple simulators. When I do this manually via
terminal it works fine. But when I put this under our service , basically
only change we did is started used direct-launch instead of plain boot.
Here is crash log
https://gist.github.com/yaphatak/eba6468b87ec0f99f1e6a8ddae6b0168
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/facebook/FBSimulatorControl/issues/394, or mute the
thread
https://github.com/notifications/unsubscribe-auth/AAEEtOqPpWEFYX2gMQykf-cnZShl4GNhks5rlbjvgaJpZM4Mb28q
.

@zsol When you said close stdout or stderr, is it the one belongs to earlier fbsimctl process? We don't explicitly close it.
I just tried launching them via mac application (using swift), it works. For some reason it doesn't when using mono. I will debug more

redirecting stdout & stderr to /dev/null fixes it


Clear histogram data if histogram stay invisible (master)

Type of PR: bugfix
PR checklist:

[x] Addresses an existing issue: fixes #133
[x] Includes tests
[ ] Documentation update

I need to fix test case to handle error.


Devtooling

add setup configuration file
provide support for bumpversion
add automatic Travis deployment to Pypi when push a new tag to master branch
add issue template with guidelines for tech and non tech questions
update development requirements
add tox for local testing with multiple python versions
add authors file and add to docs
add contributing guidelines and add to docs

Coverage remained the same at 89.399% when pulling 2d5f1bd54f94ae83da6683892ee3e1c5af0218d3 on devtooling into b67115cb286bef56f821f3896a865e0d9736976c on master.

@johanvdw the release to pypi still need to be tested (active on tag push to master)

Coverage remained the same at 89.399% when pulling d7993bb2bd3495055f5664146a7412b68a3e000c on devtooling into b67115cb286bef56f821f3896a865e0d9736976c on master.

Coverage remained the same at 89.399% when pulling d7993bb2bd3495055f5664146a7412b68a3e000c on devtooling into b67115cb286bef56f821f3896a865e0d9736976c on master.


Send timed metadata to controller

I have a custom player built on top of SimpleBasePlayer. From this player, I want to send frequent time-accurate metadata to attached controllers (especially the UI controller). It seems like setTimedMetadata would be an appropriate solution. So within the simple base player I update and invalidate the state as follows.
private var state = State.Builder().......build()

private fun sendTimedMetadata(metadataEntry: Metadata.Entry) {
     val time = 1000L
     state = state.buildUpon()
            .setTimedMetadata(Metadata(time, metadataEntry))
            .build()
      invalidateState()
}

where sendTimedMetadata is called, when I want to share the metadata.
When I attach an Player.Listener to the directly to the player, this it receives the metadata on the onMetadata-callback as expected.
However, when I use the regular setup with MediaSession and MediaSessionService and trying to connect a MediaController from an activity, the onMetadata is not called for the controller. Other state changes like isPlaying are working fine.
Am I missing something here or is this not expected to work? Maybe there are alternatives or better ways to send regular events (could be 10 events per second) to an activity? There seems to be broadcastCustomCommand of a MediaSession but currently I did not find a way to listen to such commands by controllers.

Arbitrary Metadata.Entry objects are not sent from MediaSession to MediaController, so what you're observing is expected.
@marcbaechinger might have thoughts on the best way to send time-accurate info from a session to a controller.
@thetwom Can you give some info about what these timed events that you want to send represent? It might help guide us towards a solution.

Thanks a lot for taking the time and answering my question. Good to know, that things are working as expected (even though it would have been nice for me if it worked :-).
I guess my case is rather exotic, so I don't know if I want to take your time for finding a solution.  When the app is at the foregound, visual feedback for the "clicks" is required. The player implemention is able to emit the corresponding information shortly before the click starts and I am looking for a way to bring this information to the UI.
Timed metadata seemed like something which would exactly fit the needs, but besides of not being transferred to the controller, I am unsure, if it would able to deliver information reliable within a small enough latency (say 0.1s).


operators bug

App allows user to begin with an operator. eg "4+4". App also allows user to enter multiple operators eg "4*+-*3". I'd like to fix this issue (making sure user is allowed to input negative numbers where appropriate.
Please assign this issue to me and I will fix. Thanks!

Hi @rudyt7, I have submitted a pull request to resolve this issue. Thanks :)


chore(main): release 1.18.3

:robot: I have created a release beep boop
1.18.3 (2024-04-12)
Performance Improvements

♻ i18n configuration (1ca0469)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/viarotel-org/escrcpy/releases/tag/v1.18.3 :sunflower:


Minor Misspelling

Describe the bug
The Cipher option on the Tools tab includes a misspelling of the word algorithm as "algorythm" in five of the available languages: English, French, Arabic, Filipino, or Uighur.
To Reproduce
Open app on any platform and set your language to either English, French, Arabic, Filipino, or Uighur.
Navigate to the Tools tab to view the misspelling.
Expected behavior
The word is spelled correctly as "algorithm" in each these five languages' strings.xml file located at core/resources/src/main/res/values-LANGUAGECODE/strings.xml
Screenshots

Smartphone (please complete the following information):

Device: Galaxy A13 5G
OS: Android 13

Done

Also you can update to latest 3.0.0 version 😄


Support for RGB-D inputs instead of stereo pair

Overview
It would be very helpful for our project if Elbrus could support RGB-D inputs in addition to stereo pairs. We would very much like to leverage this package for GPU-accelerated VO, but we depend on the IR emitter  for our RealSense D455, rendering the IR stereo pair unusable. A slightly less desirable but simpler alternative would be for this package to support monocular VO/SLAM, which appears to be supported by Elbrus.
Use case details
Our solution leverages the RealSense D455 as our primary vision sensor. The RealSense ROS node publishes an RGB image, depth image, and point cloud. All of these topics are utilized by various other portions of our solution based on specific needs. Additionally, and critically, we need to enable the IR projector on the RealSense to improve fidelity of the disparity map for our application.
Our custom SLAM solution leverages factor graphs with GTSAM using a variety of standard and custom factor types, including wheel odometry, AprilTags, and floor lines. We do not currently support operation in non-structured environments, but this is a gap that Elbrus should hopefully be able to help fill. The VO output could trivially be added to the factor graph along with other existing factor types, improving overall robustness and expanding supported environments. We have verified that this indeed works in practice when disabling the IR emitter and passing in the IR stereo pair to this package's node.
Feature request
With the IR emitter enabled, the projected dot array shows up very clearly in both the left and right IR images. Since the projection moves with the camera, it can give the appearance of no motion between consecutive frames, rendering VO useless. (This is presumably why the option is turned off in the sample launch file.) There are several other potential solutions worth considering.
RGB-D inputs
Since we are currently unable to turn off the IR emitter, it would be preferable to have an alternate interface for Elbrus that would allow for providing an RGB image along with a depth map. In cases where an existing depth map is not available, Elbrus clearly has performance advantages for operating directly on distorted images with sparse feature points. However, if a depth map is already available, it would seem advantageous to use it. I don't know the implementation details for Elbrus, but I know this should be possible in a number of ways with varying levels of integration.
Mono VO/SLAM
As follow-up, when looking into the interfaces for the Elbrus library, I noted the following for ELBRUS_Track():
 * images - is a pointer to single image in case of mono or
 * array of two images in case of stereo.

In the shorter term, would it be possible to add an additional option to run this node in monocular mode with a single image topic? I've only seen reference to benchmarks performed with stereo cameras, so I'm not sure of the performance implications and extent of support for running mono.

Thank you for the support!

VSLAM is designed to run on a left & right stereo image pair.  A VSLAM function that relies on a depth and RGB image is completely separate algorithm, which we are not developing.
There is fortunately an easy quick solution to this for your application with the D455.
Assuming you are running 30fps on the D455, you can configure the imager to run 60fps, and alternate frames with IR emitter enabled, and disabled (setting should be RS2_OPTION_EMITTER_ON_OFF, but consult D455 documentation).  IR frames are processed with your existing perception functions, and non-IR frames are fed into VSLAM.
You will need to write a splitter ROS node that selects which receives all frames from D455 and publishes IR and non-IR on separate image topics; or you could modify the VSLAM node to discard IR enable frames on the subscribed image topic from D455, and visa-versa for other perception graphs that need IR enabled.
Happy Holidays

Take a look at the Isaac ROS Nvblox+Isaac ROS Visual SLAM tutorial on RealSense (here) for how to use a splitter ROS node and time synchronizing the emitter to enable both depth and raw IR images.

Thank you for quick response and helpful leads!
I was not aware of this configuration option for the RealSense, but I agree that toggling the emitter on/off in conjunction with the splitter makes total sense in this situation. Thanks for directing me to the sample—we'll give this a try some time after the new year.
Regarding my other question on monocular vSLAM: Is that something that's currently supported by Elbrus? It appears so based on the rig configuration and various related comments. Just would like to keep this in mind for potential future use.
Thanks again, and happy holidays!

The VSLAM package does not support mono camera, as it is designed for stereo camera.


feat(sync-service): Move WHERE clause evaluation to single process

This is the simplest change to meet the criteria of #1744 . Shape.convert_change is used to do the filtering which may be suboptimal, it's creating the changes which are not needed and those changes are recalculated in the consumer process, however this should be enough to show the performance benefit. I believe #1760 which has been prioritised next for optimisations will make quite a few changes in this area so I am reluctant to optimise this PR further.

Yep. Sounds good.


Missing: Charred Mortar not Available

Self explanatory.

When I originally added all the recipes I was only considering recipes that required a skill. Now that Self Improvement is available as a skill we can add recipes like this

Technically if you have learned a specialization you start with Level 0.  It's when you specialize in it that you start at level 1.  However I think Charred Mortar might be a recipe that has no impact on skill exp/gains.


The Extension is to large to upload into TFS

When trying to update the extension in TFS (2017.3) I'm getting this error :
The files being added total 23 MB which exceeds the maximum total size allowed to be uploaded (20 MB).
Is there a workaround ?

Maybe you can look into making a tools installer task: https://github.com/Microsoft/vsts-tasks/blob/master/docs/tools.md
That way your tasks can remain small.

@eNeRGy164 thx for the suggestion, I was considering this as well after running out of unneeded DLLs I can remove.
So do you see this as another task you add in the beginning of your build/release definition that then will down the xRM CI Framework Cmdlets from Nuget which will be available for the remainder of the tasks to consume?
Was thinking if there is an easier way to use existing Nuget capabilities without making it too complex for consumers.

@shaibarlev Do you have a previous version you can use in the meantime?

Yes. I'm also checking for solutions from MS.

@WaelHamze I don't think adding a tools step as "too complex", it's what you also should add for other things you want to use like NuGet, .NET core, VSTest, node/npm, etc.
Maybe people are not yet used to it, but it is the way Microsoft is designing the build system for TFS/VSTS these days.
Using NuGet as a source is possible, but just doing a simple zip download is much easier to provide.

@eNeRGy164 Agreed that this is the way forward. Where do you suggest the task downloads this zip from?

You might have versioned zips on something like azure blob storage, or on GitHub. But this might be tricky because I'm not sure you're allowed to distribute those dll's.
You could also download the official nuget packages, and unpack them this might be a bit harder (not sure) but would leave out any distribution license issues.

I have the same problem, I don't need all the tasks but only to download an extract a solution. is there a way to install only some tasks?
Thanks

@MehdiNaj you download the source and compile a cut down version of the extension if needed. I am working on a way to reduce the package by dynamically downloading dependencies at run time but this will require lots of testing so will not be out for a while

@WaelHamze May be it's easier to divide tasks into separate extensions? For example: CI extension (pack/unpack) + DI extension (import/export) + Admin extension (backup/restore).
I believe most people don't use all these tasks. In my case admin tasks are not used as we have on-premise version.

I looked through code and found that several tasks use their own copy of CRM CoreTools as well as xRMCIFramework. Why not to move them to root folder?

For anyone stuck here dealing with the on-prem 20mb upload limit, it's easy enough just to either package the bits you need or hack the .vsix to remove the components you don't need right now (just edit the manifest to remove the bity you don't want and remove those bits from the tasks also).

@eNeRGy164 @mattp65 @pwalsh31 @FixRM thank you for your suggestions...
I am happy to confirm that I will be introducing a tool installer in the next release (9.2.x), The extension size including all tasks has reduced from almost 25MB to under 10MB. All the task have been updated to version 10 to avoid breaking existing builds. Using this mechanism I am now able to add more tasks. @sagarhgaikwad thx for your help in refactoring the remaining tasks.

Released to Marketplace: https://github.com/WaelHamze/dyn365-ce-vsts-tasks/releases/tag/v9.2.7


Remove version: Jackett.Jackett version 0.21.671

404 - release deleted
Microsoft Reviewers: Open in CodeFlow

URL: https://github.com/Jackett/Jackett/releases/download/v0.21.671/Jackett.Installer.Windows.exe
Response Error: The remote server returned an error: (404) Not Found.
(Automated message - build 918)


Update Pages “pestinspectionsPage”

Automatically generated by Netlify CMS

👷 Deploy Preview for chic-shortbread-66a66b processing.

Name
Link

🔨 Latest commit
d35ef9c6ab24b64feef06bbe50124fd45c192bc1

🔍 Latest deploy log
https://app.netlify.com/sites/chic-shortbread-66a66b/deploys/654c6f4d6125b7000842a532


Feature - Enum Gauge

This PF builds off of the naive gauge stacking functionality introduced by #1844 to provide a better internal API for dealing with gauges that have a set number of slots that can be filled with any number of a limited set of values. Includes some timeline and tooltip display enhancements for empty values. Gauges displaying timeline content using this new type are:

Monk: Beast Chakra
Astrologian: Astrosigns
Reaper: Death Gauge
Black Mage: Astral Fire and Umbral Ice

I also made a few timeline ordering tweaks to BLM to coincide with the gauge display update while I was at it.
Branch contains the changes also made in PRs #1844 and #1866. Merging this one will also merge those changes in. If you want to see the minimal set of changes introduced by this PR, merge those first.

Note that this PR will have merge conflicts with #1868. Take care when merging or get a hold of me to clean the other up after merging one of the two.


fuzz: remove test/server/server_corpus/oom-550f56d376b3a23320724b98c7…

…9905f7c73e4e55.
This was hanging the CI in a flakey way. Seems dangerous to be reading
runtime from / even with the various safeguards (recursion depth, /dev
exclusion) that we've added. We can probably skip this one for now, if
it shows up in ClusterFuzz again, we can figure out something more
robust as a regression.
Signed-off-by: Harvey Tuch<EMAIL_ADDRESS>

@ggreenway


fix: SocialConnect link in sidebar

the old link leads to : https://github.com/celo-org/SocialConnect and the new link leads to https://github.com/celo-org/Social-Connect

add a description to the PR, why the link has been updated
remove unused/ duplicated repositories

1 - @arthurgousset any context on why the repo was migrated to a new repo?
2 - which unused repositories? @GigaHierz

https://github.com/celo-org/SocialConnect is archived so IO

1 - @arthurgousset any context on why the repo was migrated to a new repo? 2 - which unused repositories? @GigaHierz

Yes, the idea was to migrate ODIS and the smart contracts from celo-org/celo-monorepo to celo-org/socialconnect.
But the goal was to include the git commit history from celo-org/celo-monorepo for ODIS and smart contract changes, and unfortunately you can only do that in a new repository with no git commit history. celo-org/socialconnect already had a git history because of the demos we added, so we couldn't use that. We decided to create a brand-new repo without git history celo-org/social-connect, migrate ODIS and the smart contracts including their git history, and add the demos/README after that.


Add mixin for exporter metrics

This PR includes 2 dashboards and 6 alerts for the Snowflake exporter.

General comment on both dashboards, because I didn't notice this as much in the initial screenshots:
A lot of the panels feel oversized, both in terms of height and width, especially the 24 column wide panels. I'd suggest compressing them to at most 12 columns, which would let you fit most of the dashboard in a single view, without too much scrolling.
One of those things that looks fine on smaller screens and in cropped screenshots, but the moment you blow it up on a big screen it really shows the difference


update liftoff dependency so that coffeescript knexfiles and configurations are supported again.

I suggest adding coffeescript to test process if continued support is desired.

Thanks for that @light24bulbs. :+1:


Websocket error on Vercel Serverless Function

I'm getting the following error when using this on vercel
Unhandled Promise Rejection 	{"errorType":"Runtime.UnhandledPromiseRejection","errorMessage":"Error: All attempts to open a WebSocket to connect to the database failed. If using Node, please install the `ws` package (or simply use the `pg` package instead).","reason":{"errorType":"Error","errorMessage":"All attempts to open a WebSocket to connect to the database failed. If using Node, please install the `ws` package (or simply use the `pg` package instead).","stack":["Error: All attempts to open a WebSocket to connect to the database failed. If using Node, please install the `ws` package (or simply use the `pg` package instead).","    at ws (/var/task/node_modules/@neondatabase/serverless/index.js:43:3301)","    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)"]},"promise":{},"stack":["Runtime.UnhandledPromiseRejection: Error: All attempts to open a WebSocket to connect to the database failed. If using Node, please install the `ws` package (or simply use the `pg` package instead).","    at process.<anonymous> (file:///var/runtime/index.mjs:1188:17)","    at process.emit (node:events:525:35)","    at emit (node:internal/process/promises:149:20)","    at processPromiseRejections (node:internal/process/promises:283:27)","    at process.processTicksAndRejections (node:internal/process/task_queues:96:32)"]}
Unknown application error occurred
Runtime.Unknown

I also got this locally and then install ws and then it went away. wsshould be available on Vercel so I'm not sure what the issue this. This is my package.json
  "devDependencies": {
    "@sveltejs/adapter-auto": "^2.0.0",
    "@sveltejs/kit": "^1.11.0",
    "autoprefixer": "^10.4.13",
    "dotenv-cli": "^7.2.1",
    "esbuild": "^0.17.16",
    "flowbite-svelte": "^0.29.12",
    "postcss": "^8.4.21",
    "prettier": "^2.8.3",
    "prettier-plugin-svelte": "^2.9.0",
    "svelte": "^3.57.0",
    "svelte-check": "^3.0.1",
    "tailwindcss": "^3.2.4",
    "tslib": "^2.4.1",
    "typescript": "^4.9.3",
    "vite": "^4.0.0",
    "vite-plugin-top-level-await": "^1.3.0",
    "vite-plugin-wasm": "^3.2.2"
  },
  "type": "module",
  "dependencies": {
    "@auth/core": "^0.5.1",
    "@auth/sveltekit": "^0.3.0",
    "@neondatabase/serverless": "^0.2.9",
    "@sveltejs/adapter-vercel": "^2.4.1",
    "ag-grid-community": "^29.2.0",
    "ag-grid-react": "^29.2.0",
    "apache-arrow": "^11.0.0",
    "install": "^0.13.0",
    "lucide-svelte": "^0.130.1",
    "marked": "^4.3.0",
    "parquet-wasm": "^0.4.0-beta.5",
    "ws": "^8.13.0"
  }

See issue #12: Next/Vercel CLI don't support WebSockets in local dev currently. They do apparently work when deployed which makes it even more frustrating IMO.

Are you deploying this as a Vercel Edge Function (via export const config = { runtime: 'edge' };) or a traditional Serverless Function?

using serverless. would this not work with that? only on edge?

I'm also having the same issue. Got the error locally, adding ws to my project fixed it, but when deployed to Vercel (Serverless not edge) the error is there. Works fine on Cloudflare pages. I'm using SvelteKit as well.

fyi using edge functions works. I was having other issues before getting edge to work, which is why I was trying to do it with just serverless

A colleague has reproduced this issue on Vercel's non-Edge serverless functions. We're looking at what we can do it about it.
In the meantime, it should be possible to use ordinary node-postgres (npm install pg @types/pg) there.

Error occurred prerendering page "/api/export/packages". Read more: https://nextjs.org/docs/messages/prerender-error
18:03:19.403 | Error: All attempts to open a WebSocket to connect to the database failed. If using Node, please install the `ws` package (or simply use the `pg` package instead).
18:03:19.403 | at ws (/vercel/path0/.next/server/chunks/7280.js:3961:44)
18:03:19.404 | info  - Generating static pages (11/11)
18:03:19.405

I'm getting an error message about not being able to reach the websock. DATABASE_URL is set.
Unfortunately pg seems broken on vercel too...

I am also having this issue with edge functions, adding the ws library does not appear to fix the problem when running locally with vercel dev

Hi all. I know Vercel have made recent updates to the vercel CLI tool, and npx vercel dev now works for me for both Edge Functions and Serverless Functions.
Edge Functions example
import { Pool } from '@neondatabase/serverless';

export const config = {
  runtime: 'edge',
  regions: ['fra1'],  // fra1 = Frankfurt: pick the Vercel region nearest your Neon DB
};

export default async (req: Request, ctx: any) => {
  const pool = new Pool({ connectionString: process.env.DATABASE_URL });

  const longitude = parseFloat(req.headers.get('x-vercel-ip-longitude') ?? '-122.47');
  const latitude = parseFloat(req.headers.get('x-vercel-ip-latitude') ?? '37.81');

  const { rows: sites } = await pool.query(`
    SELECT
      id_no, name_en, category,
      'https://whc.unesco.org/en/list/' || id_no || '/' AS link,
      location <-> st_makepoint($1, $2) AS distance
    FROM whc_sites_2021
    ORDER BY distance
    LIMIT 10`,
    [longitude, latitude]
  );

  ctx.waitUntil(pool.end());

  return new Response(JSON.stringify({ longitude, latitude, sites }, null, 2));
}

Serverless Functions example
import { Client, neonConfig } from '@neondatabase/serverless';
import { WebSocket } from 'undici';
import type { VercelRequest, VercelResponse } from '@vercel/node';

neonConfig.webSocketConstructor = WebSocket;

export default async (req: VercelRequest, res: VercelResponse) => {
  const client = new Client(process.env.DATABASE_URL);
  await client.connect();

  const longitude = parseFloat(req.headers['x-vercel-ip-longitude'] as string ?? '-122.47');
  const latitude = parseFloat(req.headers['x-vercel-ip-latitude'] as string ?? '37.81');

  const { rows: sites } = await client.query(`
    SELECT
      id_no, name_en, category,
      'https://whc.unesco.org/en/list/' || id_no || '/' AS link,
      location <-> st_makepoint($1, $2) AS distance
    FROM whc_sites_2021
    ORDER BY distance
    LIMIT 10`,
    [longitude, latitude]
  );

  res.status(200).json({ longitude, latitude, sites });
  await client.end();
}

Please shout if you're still having trouble, otherwise I'll close the issue.

so the error I had before is gone which is cool. I think now the trouble is with combining edge and serverless functions. Which I suppose is a vercel problem. Is it worthwhile to set the websocket constructor in an edge function as well?

Glad the previous error is resolved.
No, there's no need to set webSocketConstructor in an Edge Function, and trying to do so is likely to raise an error.


feat: add cooldowns support

WIP

This is discontinued for now, it will be added only if discord supports cooldowns natively


[scss] linter update for variables in #{}

VSCode Version: 1.35.1
OS Version: Ubuntu 18.04

In this piece of SCSS code:
@for $i from 0 to 9 {
  .enterActive &:nth-child(#{$i}) {
    transition-delay: $i * 30ms;
  }
}

the #{$i} is parsed as an id (whether it's not), so the linter throws me that I shouldn't be using id selectors:
Selectors should not contain IDs because these rules are too tightly coupled with the HTML

Seems already fixed.

Running the latest build of insiders and this is definitely still happening
Version: 1.40.0-insider (user setup)
Commit: c58aaab8a1cc22a7139b761166a0d4f37d41e998
Date: 2019-10-22T05:27:32.053Z
Electron: 6.0.12
Chrome: 76.0.3809.146
Node.js: 12.4.0
V8: 7.6.303.31-electron.0
OS: Windows_NT x64 10.0.18362

@aNickzz I can't repro from the original piece. Please open a new issue with reproducible steps.

I also see this, I suggest that we reopen this and close the new issue to keep the history

Make sure you enable that rule, "scss.lint.idSelector": "error"

Make sure you enable that rule, "scss.lint.idSelector": "error"

That's it. I didn't enable it explicitly.

Are you sure this should be labelled with feature-request instead of a bug? Maybe it would get a bit more attention with the correct label.

For my own reference, here's the source for this rule: https://github.com/microsoft/vscode-css-languageservice/blob/716292720c09f444167fd5528bcd4e6d1c6cc5db/src/services/lint.ts#L251

It's a bug in the parser.

So, the linting rule checks for a # at the beginning of a SimpleSelector node. , this is a false positive because in this case the first (and only) descendent of the first identifier in the SimpleSelector is an Interpolation.
I would think that the linter could check for this and not add the Marker if the # is coming from an Interpolation. This is what I've done on my fork and have had success running some new tests to check this.
As I'm not sure that the structure of the node tree will always be consistent with what I saw, so I did this by checking for an Interpolation at the first child of each node down the tree, rather than assuming that if an Interpolation node exists, it would always be the nth grandchild of the SimpleSelector.

However, if you're saying that this is rather a problem with the parser, rather than the linter, then I'm not really sure how to tackle this. I'd like to make a PR for this so please let me know what strategy you'd like to take for this.

Also, while looking through this I discovered that the SimpleSelector is checked as a whole as an ID selector, that is to say, only the first component of the SimpleSelector is checked for ID's
#id {} // This will raise a warning about an id selector
div#id {} // This will not raise a warning, as the id selector isn't at the front

Though that should be in its own issue I guess.

Thanks @aNickzz for looking at this. Yes, I think you got this right, the linter check for # is too simple.
Instead of looking for a interpolation, what about looking for a IdentifierSelector node inside the SimpleSelector?
That would then fix the second point (no warning on div#id {}) as well.


Automatic style links refactored to inline style

This pull request changes the url format used in the deep dive sections from text <url>  to  [text](url)
A compiled version is available online at http://testfiwaretour.readthedocs.org/en/refactor-deep_dive_url/

LGTM, thanks


Could not build the application for the simulator.

`Error output from Xcode build:
↳
** BUILD FAILED **
Xcode's output:
↳
In file included from /Users/osama/Developer/flutter/.pub-cache/hosted/pub.dartlang.org/flutter_webrtc-0.6.3/ios/Classes/FlutterRTCVideoRenderer.m:9:
In file included from /Users/osama/Developer/wpzag/ios/Pods/Libyuv/headers/libyuv.h:16:
In file included from /Users/osama/Developer/wpzag/ios/Pods/Headers/Public/libyuv/convert.h:21:
/Users/osama/Developer/wpzag/ios/Pods/Headers/Public/libyuv/planar_functions.h:563:26: warning: this function declaration is not a prototype [-Wstrict-prototypes]
ARGBBlendRow GetARGBBlend();
^
void
1 warning generated.
/Users/osama/Developer/flutter/.pub-cache/hosted/pub.dartlang.org/flutter_webrtc-0.6.3/ios/Classes/FlutterRTCMediaStream.m:172:18: warning: unused variable 'requestAccessForVideo' [-Wunused-variable]
BOOL requestAccessForVideo
^
/Users/osama/Developer/flutter/.pub-cache/hosted/pub.dartlang.org/flutter_webrtc-0.6.3/ios/Classes/FlutterRTCMediaStream.m:369:29: warning: 'devicesWithMediaType:' is deprecated: first deprecated in iOS 10.0 - Use AVCaptureDeviceDiscoverySession instead. [-Wdeprecated-declarations]
&& [AVCaptureDevice devicesWithMediaType:mediaType].count == 0) {
^
In module 'WebRTC' imported from /Users/osama/Developer/flutter/.pub-cache/hosted/pub.dartlang.org/flutter_webrtc-0.6.3/ios/Classes/FlutterRTCMediaStream.m:3:
In module 'AVFoundation' imported from /Users/osama/Developer/wpzag/ios/Pods/GoogleWebRTC/Frameworks/frameworks/WebRTC.framework/Headers/RTCVideoFrame.h:11:
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.4.sdk/System/Library/Frameworks/AVFoundation.framework/Headers/AVCaptureDevice.h:103:1: note: 'devicesWithMediaType:' has been explicitly marked deprecated here
+ (NSArray<AVCaptureDevice *> *)devicesWithMediaType:(AVMediaType)mediaType API_DEPRECATED("Use AVCaptureDeviceDiscoverySession instead.", ios(4.0, 10.0), macos(10.7, 10.15));
^
/Users/osama/Developer/flutter/.pub-cache/hosted/pub.dartlang.org/flutter_webrtc-0.6.3/ios/Classes/FlutterRTCMediaStream.m:451:24: warning: incompatible pointer types assigning to 'RTCCameraVideoCapturer *' from 'FlutterRPScreenRecorder *' [-Wincompatible-pointer-types]
self.videoCapturer = screenCapturer;
^ ~~~~~~~~~~~~~~
/Users/osama/Developer/flutter/.pub-cache/hosted/pub.dartlang.org/flutter_webrtc-0.6.3/ios/Classes/FlutterRTCMediaStream.m:479:44: warning: 'devicesWithMediaType:' is deprecated: first deprecated in iOS 10.0 - Use AVCaptureDeviceDiscoverySession instead. [-Wdeprecated-declarations]
NSArray *videoDevices = [AVCaptureDevice devicesWithMediaType:AVMediaTypeVideo];
^
In module 'WebRTC' imported from /Users/osama/Developer/flutter/.pub-cache/hosted/pub.dartlang.org/flutter_webrtc-0.6.3/ios/Classes/FlutterRTCMediaStream.m:3:
In module 'AVFoundation' imported from /Users/osama/Developer/wpzag/ios/Pods/GoogleWebRTC/Frameworks/frameworks/WebRTC.framework/Headers/RTCVideoFrame.h:11:
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.4.sdk/System/Library/Frameworks/AVFoundation.framework/Headers/AVCaptureDevice.h:103:1: note: 'devicesWithMediaType:' has been explicitly marked deprecated here
+ (NSArray<AVCaptureDevice *> *)devicesWithMediaType:(AVMediaType)mediaType API_DEPRECATED("Use AVCaptureDeviceDiscoverySession instead.", ios(4.0, 10.0), macos(10.7, 10.15));
^
/Users/osama/Developer/flutter/.pub-cache/hosted/pub.dartlang.org/flutter_webrtc-0.6.3/ios/Classes/FlutterRTCMediaStream.m:488:44: warning: 'devicesWithMediaType:' is deprecated: first deprecated in iOS 10.0 - Use AVCaptureDeviceDiscoverySession instead. [-Wdeprecated-declarations]
NSArray *audioDevices = [AVCaptureDevice devicesWithMediaType:AVMediaTypeAudio];
^
In module 'WebRTC' imported from /Users/osama/Developer/flutter/.pub-cache/hosted/pub.dartlang.org/flutter_webrtc-0.6.3/ios/Classes/FlutterRTCMediaStream.m:3:
In module 'AVFoundation' imported from /Users/osama/Developer/wpzag/ios/Pods/GoogleWebRTC/Frameworks/frameworks/WebRTC.framework/Headers/RTCVideoFrame.h:11:
/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator14.4.sdk/System/Library/Frameworks/AVFoundation.framework/Headers/AVCaptureDevice.h:103:1: note: 'devicesWithMediaType:' has been explicitly marked deprecated here
+ (NSArray<AVCaptureDevice *> *)devicesWithMediaType:(AVMediaType)mediaType API_DEPRECATED("Use AVCaptureDeviceDiscoverySession instead.", ios(4.0, 10.0), macos(10.7, 10.15));
^
5 warnings generated.
In file included from /Users/osama/Developer/flutter/.pub-cache/hosted/pub.dartlang.org/flutter_webrtc-0.6.3/ios/Classes/FlutterRTCFrameCapturer.m:10:
In file included from /Users/osama/Developer/wpzag/ios/Pods/Libyuv/headers/libyuv.h:16:
In file included from /Users/osama/Developer/wpzag/ios/Pods/Headers/Public/libyuv/convert.h:21:
/Users/osama/Developer/wpzag/ios/Pods/Headers/Public/libyuv/planar_functions.h:563:26: warning: this function declaration is not a prototype [-Wstrict-prototypes]
ARGBBlendRow GetARGBBlend();
^
void
1 warning generated.
Command CompileSwift failed with a nonzero exit code
/Users/osama/Developer/flutter/.pub-cache/hosted/pub.dartlang.org/janus_client-2.0.0-beta/ios/Classes/FlutterJanusClientPlugin.m:8:9: fatal error: 'flutter_janus_client-Swift.h' file not found
#import "flutter_janus_client-Swift.h"
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
note: Using new build system
note: Building targets in parallel
note: Planning build
note: Constructing build description
warning: The iOS Simulator deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.4.99. (in target 'FMDB' from project 'Pods')
warning: The iOS Simulator deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.4.99. (in target 'AppAuth' from project 'Pods')
warning: The iOS Simulator deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.4.99. (in target 'GoogleSignIn' from project 'Pods')
warning: The iOS Simulator deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.4.99. (in target 'Libyuv' from project 'Pods')
warning: The iOS Simulator deployment target 'IPHONEOS_DEPLOYMENT_TARGET' is set to 8.0, but the range of supported deployment target versions is 9.0 to 14.4.99. (in target 'Flutter' from project 'Pods')
warning: Capabilities for Signing & Capabilities may not function correctly because its entitlements use a placeholder team ID. To resolve this, select a development team in the Runner editor. (in target 'Runner' from project 'Runner')
Could not build the application for the simulator.
Error launching application on iPhone 12 Pro Max.`

Delete ios folder and run flutter create . Again in example directory that should work, if not then feel free to open this issue again

Delete ios folder and run flutter create . Again in example directory that should work, if not then feel free to open this issue again

i have done that but  still getting the same error

I will try it within few days and let you know
On Fri, 23 Apr, 2021, 4:32 pm osamaegy, @.***> wrote:

Delete ios folder and run flutter create . Again in example directory that
should work, if not then feel free to open this issue again
i have done that but still getting the same error
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
https://github.com/flutterjanus/flutter_janus_client/issues/33#issuecomment-825580235,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AGLGDVZ2JX452VWXRMSWHJ3TKFHTRANCNFSM43N3MAZA
.

ok thanks !


Digital Ocean IP blocked by both BBC Iplayer and Netflix

Hi,
Just thought I'd drop it to say that my Digital Ocean IP in the 138.68 range is blocked by both BBC Iplayer and Netflix.
I confirmed by actually installing a dockerized Chrome browser (https://hub.docker.com/r/siomiz/chrome/) and trying to access them from that browser.
Regards,
Steve

Thanks for the update.

what is the message you get when your ip is blocked? Tried to acces netflix and returns a streaming error about a proxy, does this mean the ip is blocked? SInce I can normally acces netflix otherwise

Does this help?

@ab77 Thank you! That helped a lot! Indeed the ip is blocked because it's known as cloudhoster ;)!

I am sure there are still providers out there, which Netflix hasn't blocked yet. But Digital Ocean is definitely not one of those :)


WIP: Dev/roddy docs

WIP of introduction documentation for interim review.

Also see the ghostflow output: whitespace and lack-of-change-note "errors."

Dear @collinsr,
I broke it by messing up an attempt at resolving a file conflict via github's interface. Please forgive me.


is there support for TAPO Camera C210

Describe The Bug
is there support for TAPO Camera C210
Expected behavior
see the camera image in real time
Logs
none
Plugin Config
none
Plugin Version
latest
Homebridge Version
v1.7.0
Node.js Version
v20.9.0
NPM Version
unsure
Operating System
Raspian Bookworm RPI5

Hello, there is not support for cameras

Thanks, is the Hub H200 & TP-LINK Tapo Smart Temperature & Humidity T310 supported???


Add Druid unit test class against latest 0.20.0

Relates to #5804
This commit refactors the integration smoke test (there are no distributed tests) into a base with two implementations, the existing unit tests against the 0.18.0 of Druid and the latest 0.20.0 version.

CI failed -- https://github.com/prestosql/presto/issues/5928 and PT suite timeout


Add support for norway, "no" locale

[x] Updated config for separator
[x] Updated config for positions
[x] Added test

@grassator Should I update package json as well?


Fix CI error of tar files created on mac

tar files created with mac computers are including a lsetxattr header that docker engine doesn't seem to be able to handle, this just recreate the tar files without such headers
See: https://github.com/moby/moby/issues/47517
See: https://eclecticlight.co/2023/03/13/ventura-has-changed-app-quarantine-with-a-new-xattr/
Change-type: patch

@thgreasi  The current failing tests are should build without "platform" flag (v1 manifest) and should build without "platform" flag (v1 manifest) for multi-stage builds which currently fail with engine error:
error: Error: [DEPRECATION NOTICE] Docker Image Format v1 and Docker Image manifest version 2, schema 1 support is disabled by default and will be removed in an upcoming release. Suggest the author of docker.io/resin/amd64-supervisor:v1.7.0-30-g8101d0 to upgrade the image to the OCI Format or Docker Image manifest v2, schema 2. More information at https://docs.docker.com/go/deprecated-image-specs/

So we need to either do a major bump deprecating them in this package or somehow enable format v1/manifest v2 on the CI runners

superseded by https://github.com/balena-io-modules/balena-compose/pull/52