Implementing cell2location with scVI

Hi!
I am working on implementing cell2location with scVI/Pyro (https://github.com/vitkl/scvi-tools/blob/pyro-cell2location/scvi/external/cell2location), I would like to ask a few questions and I am encountering a few issues too.

What would be the reasons to choose scvi.external over a separate package that depends on scVI?

I seem to be unable to use Pyro with JitTrace_ELBO on GPU (https://github.com/vitkl/scvi-tools/blob/pyro-cell2location/scvi/external/cell2location/_model.py#L184) - getting errors which are hard to interpret. Do you still have example scVI implementation on pyro?

I tried using default train function on GPU but it looks like that function does not move all tensors to GPU leading to errors (https://github.com/vitkl/scvi-tools/blob/pyro-cell2location/scvi/external/cell2location/_model.py#L95-L117). I get the same error even when I add:

if use_gpu is True:
            self.module.cuda()

It seems that to make it work I need to add the code below before I do any training steps:
if self.use_gpu:
            torch.set_default_tensor_type(torch.cuda.FloatTensor)
else:
            torch.set_default_tensor_type(torch.FloatTensor)

if use_gpu:
            self.module.cuda()

Is this expected or am I using scVI incorrectly?

I added methods for taking samples from the posterior distribution which might belong in a more general pyro model class (https://github.com/vitkl/scvi-tools/blob/pyro-cell2location/scvi/external/cell2location/_model.py#L203-L308). What do you think?
The posterior sampling is also complicated by the need to provide a different set of observation indices to extract parameters for all observations, which I circumvent here by providing indices for all observations:
https://github.com/vitkl/scvi-tools/blob/pyro-cell2location/scvi/external/cell2location/_module.py#L162-L167

It looks like the subsample size of the plate cannot change meaning that data loader needs to be used withdrop_last=True (https://github.com/vitkl/scvi-tools/blob/pyro-cell2location/scvi/external/cell2location/_model.py#L182). The default in scVI is drop_last=False, right?

Error for point 3:
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
~/.local/lib/python3.7/site-packages/pyro/poutine/trace_messenger.py in __call__(self, *args, **kwargs)
    164             try:
--> 165                 ret = self.fn(*args, **kwargs)
    166             except (ValueError, RuntimeError) as e:

~/.local/lib/python3.7/site-packages/pyro/poutine/messenger.py in _context_wrap(context, fn, *args, **kwargs)
     11     with context:
---> 12         return fn(*args, **kwargs)
     13

~/.local/lib/python3.7/site-packages/pyro/poutine/messenger.py in _context_wrap(context, fn, *args, **kwargs)
     11     with context:
---> 12         return fn(*args, **kwargs)
     13

/nfs/team205/vk7/sanger_projects/BayraktarLab/scvi-tools/scvi/external/cell2location/_module.py in model(self, x_data, idx, obs2sample)
    300         #print(alpha_g_inverse.shape)
--> 301         mu = torch.matmul(w_sf, self.cell_state) * m_g + torch.matmul(obs2sample, s_g_gene_add) + l_s_add
    302         theta = torch.matmul(obs2sample, torch.ones([1, 1]) / (alpha_g_inverse * alpha_g_inverse))

RuntimeError: Tensor for 'out' is on CPU, Tensor for argument #1 'self' is on CPU, but expected them to be on GPU (while checking arguments for addmm)

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
~/.local/lib/python3.7/site-packages/pyro/poutine/trace_messenger.py in __call__(self, *args, **kwargs)
    164             try:
--> 165                 ret = self.fn(*args, **kwargs)
    166             except (ValueError, RuntimeError) as e:

~/.local/lib/python3.7/site-packages/pyro/nn/module.py in __call__(self, *args, **kwargs)
    412         with self._pyro_context:
--> 413             return super().__call__(*args, **kwargs)
    414

~/.local/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    726         else:
--> 727             result = self.forward(*input, **kwargs)
    728         for hook in itertools.chain(

~/.local/lib/python3.7/site-packages/pyro/infer/autoguide/guides.py in forward(self, *args, **kwargs)
    484         if self.prototype_trace is None:
--> 485             self._setup_prototype(*args, **kwargs)
    486

~/.local/lib/python3.7/site-packages/pyro/infer/autoguide/guides.py in _setup_prototype(self, *args, **kwargs)
    439     def _setup_prototype(self, *args, **kwargs):
--> 440         super()._setup_prototype(*args, **kwargs)
    441

~/.local/lib/python3.7/site-packages/pyro/infer/autoguide/guides.py in _setup_prototype(self, *args, **kwargs)
    157         model = poutine.block(self.model, prototype_hide_fn)
--> 158         self.prototype_trace = poutine.block(poutine.trace(model).get_trace)(*args, **kwargs)
    159         if self.master is not None:

~/.local/lib/python3.7/site-packages/pyro/poutine/messenger.py in _context_wrap(context, fn, *args, **kwargs)
     11     with context:
---> 12         return fn(*args, **kwargs)
     13

~/.local/lib/python3.7/site-packages/pyro/poutine/trace_messenger.py in get_trace(self, *args, **kwargs)
    186         """
--> 187         self(*args, **kwargs)
    188         return self.msngr.get_trace()

~/.local/lib/python3.7/site-packages/pyro/poutine/trace_messenger.py in __call__(self, *args, **kwargs)
    170                 exc = exc.with_traceback(traceback)
--> 171                 raise exc from e
    172             self.msngr.trace.add_node("_RETURN", name="_RETURN", type="return", value=ret)

~/.local/lib/python3.7/site-packages/pyro/poutine/trace_messenger.py in __call__(self, *args, **kwargs)
    164             try:
--> 165                 ret = self.fn(*args, **kwargs)
    166             except (ValueError, RuntimeError) as e:

~/.local/lib/python3.7/site-packages/pyro/poutine/messenger.py in _context_wrap(context, fn, *args, **kwargs)
     11     with context:
---> 12         return fn(*args, **kwargs)
     13

~/.local/lib/python3.7/site-packages/pyro/poutine/messenger.py in _context_wrap(context, fn, *args, **kwargs)
     11     with context:
---> 12         return fn(*args, **kwargs)
     13

/nfs/team205/vk7/sanger_projects/BayraktarLab/scvi-tools/scvi/external/cell2location/_module.py in model(self, x_data, idx, obs2sample)
    300         #print(alpha_g_inverse.shape)
--> 301         mu = torch.matmul(w_sf, self.cell_state) * m_g + torch.matmul(obs2sample, s_g_gene_add) + l_s_add
    302         theta = torch.matmul(obs2sample, torch.ones([1, 1]) / (alpha_g_inverse * alpha_g_inverse))

RuntimeError: Tensor for 'out' is on CPU, Tensor for argument #1 'self' is on CPU, but expected them to be on GPU (while checking arguments for addmm)
                Trace Shapes:
                 Param Sites:
                Sample Sites:
           m_g_alpha_hyp dist       1     1 |
                        value       1     1 |
            m_g_beta_hyp dist       1     1 |
                        value       1     1 |
                     m_g dist       1 10228 |
                        value       1 10228 |
  N_s_cells_per_location dist    2019     1 |
                        value    2019     1 |
  Y_s_combs_per_location dist    2019     1 |
                        value    2019     1 |
      z_sr_combs_factors dist 50 2019     1 |
                        value 50 2019     1 |
   K_r_factors_per_combs dist 50    1     1 |
                        value 50    1     1 |
          x_fr_comb2fact dist 50    1    34 |
                        value 50    1    34 |
                    w_sf dist    2019    34 |
                        value    2019    34 |
           l_s_add_alpha dist       1     1 |
                        value       1     1 |
            l_s_add_beta dist       1     1 |
                        value       1     1 |
                 l_s_add dist    2019     1 |
                        value    2019     1 |
       s_g_gene_add_mean dist       1     1 |
                        value       1     1 |
  s_g_gene_add_alpha_hyp dist       1     1 |
                        value       1     1 |
s_g_gene_add_alpha_e_inv dist       1     1 |
                        value       1     1 |
            s_g_gene_add dist       1 10228 |
                        value       1 10228 |
         alpha_g_phi_hyp dist       1     1 |
                        value       1     1 |
         alpha_g_inverse dist       1 10228 |
                        value       1 10228 |

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
<ipython-input-11-64996cd443db> in <module>
----> 1 model.train_pyro_v2(lr = 0.001, max_epochs=50, use_gpu=True, batch_size=2019)

/nfs/team205/vk7/sanger_projects/BayraktarLab/scvi-tools/scvi/external/cell2location/_model.py in train_pyro_v2(self, max_epochs, use_gpu, train_size, validation_size, batch_size, lr, total_grad_norm_constraint, **kwargs)
    115                    train_size=train_size, validation_size=validation_size, batch_size=batch_size,
    116                    plan_kwargs=plan_kwargs,
--> 117                    plan_class=PyroTrainingPlan)
    118
    119

/nfs/team205/vk7/sanger_projects/BayraktarLab/scvi-tools/scvi/model/base/_base_model.py in train(self, max_epochs, use_gpu, train_size, validation_size, batch_size, plan_kwargs, plan_class, **kwargs)
    320         if train_size == 1.0:
    321             # circumvent the empty data loader problem if all dataset used for training
--> 322             self.trainer.fit(self._training_plan, train_dl)
    323         else:
    324             self.trainer.fit(self._training_plan, train_dl, val_dl)

/nfs/team205/vk7/sanger_projects/BayraktarLab/scvi-tools/scvi/lightning/_trainer.py in fit(self, *args, **kwargs)
    129                 action="ignore", category=UserWarning, message="The dataloader,"
    130             )
--> 131             super().fit(*args, **kwargs)

/nfs/team283/vk7/software/miniconda3farm5/envs/scvi-env2/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py in fit(self, model, train_dataloader, val_dataloaders, datamodule)
    508         self.call_hook('on_fit_start')
    509
--> 510         results = self.accelerator_backend.train()
    511         self.accelerator_backend.teardown()
    512

/nfs/team283/vk7/software/miniconda3farm5/envs/scvi-env2/lib/python3.7/site-packages/pytorch_lightning/accelerators/accelerator.py in train(self)
     55     def train(self):
     56         self.trainer.setup_trainer(self.trainer.model)
---> 57         return self.train_or_test()
     58
     59     def teardown(self):

/nfs/team283/vk7/software/miniconda3farm5/envs/scvi-env2/lib/python3.7/site-packages/pytorch_lightning/accelerators/accelerator.py in train_or_test(self)
     72         else:
     73             self.trainer.train_loop.setup_training()
---> 74             results = self.trainer.train()
     75         return results
     76

/nfs/team283/vk7/software/miniconda3farm5/envs/scvi-env2/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py in train(self)
    559                 with self.profiler.profile("run_training_epoch"):
    560                     # run train epoch
--> 561                     self.train_loop.run_training_epoch()
    562
    563                 if self.max_steps and self.max_steps <= self.global_step:

/nfs/team283/vk7/software/miniconda3farm5/envs/scvi-env2/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py in run_training_epoch(self)
    541             # ------------------------------------
    542             with self.trainer.profiler.profile("run_training_batch"):
--> 543                 batch_output = self.run_training_batch(batch, batch_idx, dataloader_idx)
    544
    545             # when returning -1 from train_step, we end epoch early

/nfs/team283/vk7/software/miniconda3farm5/envs/scvi-env2/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py in run_training_batch(self, batch, batch_idx, dataloader_idx)
    703                             batch_idx,
    704                             opt_idx,
--> 705                             self.trainer.hiddens
    706                         )
    707

/nfs/team283/vk7/software/miniconda3farm5/envs/scvi-env2/lib/python3.7/site-packages/pytorch_lightning/trainer/training_loop.py in training_step(self, split_batch, batch_idx, opt_idx, hiddens)
    308             model_ref._current_fx_name = 'training_step'
    309             model_ref._results = Result()
--> 310             training_step_output = self.trainer.accelerator_backend.training_step(args)
    311             self.trainer.logger_connector.cache_logged_metrics()
    312

/nfs/team283/vk7/software/miniconda3farm5/envs/scvi-env2/lib/python3.7/site-packages/pytorch_lightning/accelerators/gpu_accelerator.py in training_step(self, args)
     68
     69     def training_step(self, args):
---> 70         return self._step(self.trainer.model.training_step, args)
     71
     72     def validation_step(self, args):

/nfs/team283/vk7/software/miniconda3farm5/envs/scvi-env2/lib/python3.7/site-packages/pytorch_lightning/accelerators/gpu_accelerator.py in _step(self, model_step, args)
     63                 output = model_step(*args)
     64         else:
---> 65             output = model_step(*args)
     66
     67         return output

/nfs/team205/vk7/sanger_projects/BayraktarLab/scvi-tools/scvi/lightning/_trainingplans.py in training_step(self, batch, batch_idx, optimizer_idx)
    561     def training_step(self, batch, batch_idx, optimizer_idx=0):
    562         args, kwargs = self.module._get_fn_args_from_batch(batch)
--> 563         loss = self.svi.step(*args, **kwargs)
    564         self.log("train_loss", loss, prog_bar=True, on_epoch=True)
    565

~/.local/lib/python3.7/site-packages/pyro/infer/svi.py in step(self, *args, **kwargs)
    126         # get loss and compute gradients
    127         with poutine.trace(param_only=True) as param_capture:
--> 128             loss = self.loss_and_grads(self.model, self.guide, *args, **kwargs)
    129
    130         params = set(site["value"].unconstrained()

~/.local/lib/python3.7/site-packages/pyro/infer/trace_elbo.py in loss_and_grads(self, model, guide, *args, **kwargs)
    124         loss = 0.0
    125         # grab a trace from the generator
--> 126         for model_trace, guide_trace in self._get_traces(model, guide, args, kwargs):
    127             loss_particle, surrogate_loss_particle = self._differentiable_loss_particle(model_trace, guide_trace)
    128             loss += loss_particle / self.num_particles

~/.local/lib/python3.7/site-packages/pyro/infer/elbo.py in _get_traces(self, model, guide, args, kwargs)
    168         else:
    169             for i in range(self.num_particles):
--> 170                 yield self._get_trace(model, guide, args, kwargs)

~/.local/lib/python3.7/site-packages/pyro/infer/trace_elbo.py in _get_trace(self, model, guide, args, kwargs)
     51         """
     52         model_trace, guide_trace = get_importance_trace(
---> 53             "flat", self.max_plate_nesting, model, guide, args, kwargs)
     54         if is_validation_enabled():
     55             check_if_enumerated(guide_trace)

~/.local/lib/python3.7/site-packages/pyro/infer/enum.py in get_importance_trace(graph_type, max_plate_nesting, model, guide, args, kwargs, detach)
     42     and the model that is run against it.
     43     """
---> 44     guide_trace = poutine.trace(guide, graph_type=graph_type).get_trace(*args, **kwargs)
     45     if detach:
     46         guide_trace.detach_()

~/.local/lib/python3.7/site-packages/pyro/poutine/trace_messenger.py in get_trace(self, *args, **kwargs)
    185         Calls this poutine and returns its trace instead of the function's return value.
    186         """
--> 187         self(*args, **kwargs)
    188         return self.msngr.get_trace()

~/.local/lib/python3.7/site-packages/pyro/poutine/trace_messenger.py in __call__(self, *args, **kwargs)
    169                 exc = exc_type(u"{}\n{}".format(exc_value, shapes))
    170                 exc = exc.with_traceback(traceback)
--> 171                 raise exc from e
    172             self.msngr.trace.add_node("_RETURN", name="_RETURN", type="return", value=ret)
    173         return ret

~/.local/lib/python3.7/site-packages/pyro/poutine/trace_messenger.py in __call__(self, *args, **kwargs)
    163                                       args=args, kwargs=kwargs)
    164             try:
--> 165                 ret = self.fn(*args, **kwargs)
    166             except (ValueError, RuntimeError) as e:
    167                 exc_type, exc_value, traceback = sys.exc_info()

~/.local/lib/python3.7/site-packages/pyro/nn/module.py in __call__(self, *args, **kwargs)
    411     def __call__(self, *args, **kwargs):
    412         with self._pyro_context:
--> 413             return super().__call__(*args, **kwargs)
    414
    415     def __getattr__(self, name):

~/.local/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    725             result = self._slow_forward(*input, **kwargs)
    726         else:
--> 727             result = self.forward(*input, **kwargs)
    728         for hook in itertools.chain(
    729                 _global_forward_hooks.values(),

~/.local/lib/python3.7/site-packages/pyro/infer/autoguide/guides.py in forward(self, *args, **kwargs)
    483         # if we've never run the model before, do so now so we can inspect the model structure
    484         if self.prototype_trace is None:
--> 485             self._setup_prototype(*args, **kwargs)
    486
    487         plates = self._create_plates(*args, **kwargs)

~/.local/lib/python3.7/site-packages/pyro/infer/autoguide/guides.py in _setup_prototype(self, *args, **kwargs)
    438
    439     def _setup_prototype(self, *args, **kwargs):
--> 440         super()._setup_prototype(*args, **kwargs)
    441
    442         self._event_dims = {}

~/.local/lib/python3.7/site-packages/pyro/infer/autoguide/guides.py in _setup_prototype(self, *args, **kwargs)
    156         # run the model so we can inspect its structure
    157         model = poutine.block(self.model, prototype_hide_fn)
--> 158         self.prototype_trace = poutine.block(poutine.trace(model).get_trace)(*args, **kwargs)
    159         if self.master is not None:
    160             self.master()._check_prototype(self.prototype_trace)

~/.local/lib/python3.7/site-packages/pyro/poutine/messenger.py in _context_wrap(context, fn, *args, **kwargs)
     10 def _context_wrap(context, fn, *args, **kwargs):
     11     with context:
---> 12         return fn(*args, **kwargs)
     13
     14

~/.local/lib/python3.7/site-packages/pyro/poutine/trace_messenger.py in get_trace(self, *args, **kwargs)
    185         Calls this poutine and returns its trace instead of the function's return value.
    186         """
--> 187         self(*args, **kwargs)
    188         return self.msngr.get_trace()

~/.local/lib/python3.7/site-packages/pyro/poutine/trace_messenger.py in __call__(self, *args, **kwargs)
    169                 exc = exc_type(u"{}\n{}".format(exc_value, shapes))
    170                 exc = exc.with_traceback(traceback)
--> 171                 raise exc from e
    172             self.msngr.trace.add_node("_RETURN", name="_RETURN", type="return", value=ret)
    173         return ret

~/.local/lib/python3.7/site-packages/pyro/poutine/trace_messenger.py in __call__(self, *args, **kwargs)
    163                                       args=args, kwargs=kwargs)
    164             try:
--> 165                 ret = self.fn(*args, **kwargs)
    166             except (ValueError, RuntimeError) as e:
    167                 exc_type, exc_value, traceback = sys.exc_info()

~/.local/lib/python3.7/site-packages/pyro/poutine/messenger.py in _context_wrap(context, fn, *args, **kwargs)
     10 def _context_wrap(context, fn, *args, **kwargs):
     11     with context:
---> 12         return fn(*args, **kwargs)
     13
     14

~/.local/lib/python3.7/site-packages/pyro/poutine/messenger.py in _context_wrap(context, fn, *args, **kwargs)
     10 def _context_wrap(context, fn, *args, **kwargs):
     11     with context:
---> 12         return fn(*args, **kwargs)
     13
     14

/nfs/team205/vk7/sanger_projects/BayraktarLab/scvi-tools/scvi/external/cell2location/_module.py in model(self, x_data, idx, obs2sample)
    299         #print(l_s_add.shape)
    300         #print(alpha_g_inverse.shape)
--> 301         mu = torch.matmul(w_sf, self.cell_state) * m_g + torch.matmul(obs2sample, s_g_gene_add) + l_s_add
    302         theta = torch.matmul(obs2sample, torch.ones([1, 1]) / (alpha_g_inverse * alpha_g_inverse))
    303         # convert mean and overdispersion to total count and logits (input to NB from pyro)

RuntimeError: Tensor for 'out' is on CPU, Tensor for argument #1 'self' is on CPU, but expected them to be on GPU (while checking arguments for addmm)
                Trace Shapes:
                 Param Sites:
                Sample Sites:
           m_g_alpha_hyp dist       1     1 |
                        value       1     1 |
            m_g_beta_hyp dist       1     1 |
                        value       1     1 |
                     m_g dist       1 10228 |
                        value       1 10228 |
  N_s_cells_per_location dist    2019     1 |
                        value    2019     1 |
  Y_s_combs_per_location dist    2019     1 |
                        value    2019     1 |
      z_sr_combs_factors dist 50 2019     1 |
                        value 50 2019     1 |
   K_r_factors_per_combs dist 50    1     1 |
                        value 50    1     1 |
          x_fr_comb2fact dist 50    1    34 |
                        value 50    1    34 |
                    w_sf dist    2019    34 |
                        value    2019    34 |
           l_s_add_alpha dist       1     1 |
                        value       1     1 |
            l_s_add_beta dist       1     1 |
                        value       1     1 |
                 l_s_add dist    2019     1 |
                        value    2019     1 |
       s_g_gene_add_mean dist       1     1 |
                        value       1     1 |
  s_g_gene_add_alpha_hyp dist       1     1 |
                        value       1     1 |
s_g_gene_add_alpha_e_inv dist       1     1 |
                        value       1     1 |
            s_g_gene_add dist       1 10228 |
                        value       1 10228 |
         alpha_g_phi_hyp dist       1     1 |
                        value       1     1 |
         alpha_g_inverse dist       1 10228 |
                        value       1 10228 |
Trace Shapes:
 Param Sites:
Sample Sites:

I can give a more detailed response later, but it's necessary to write device agnostic code in your module.
For example:
https://github.com/vitkl/scvi-tools/blob/3c725f1f82fdc2fa716c749b6a6cfe27417c7ed0/scvi/external/cell2location/_module.py#L145
Everything that's a tensor defined in the init should be added to the torch nn modules parameters -- one way to do this is to use self.register_buffer.
Once all the necessary tensors are considered as part of the module, the .cuda() command will move everything.
It would also be super helpful if you made a PR so it's easier to read the code.

Also, out of curiosity, how much faster is JIT trace elbo?

Thanks @adamgayoso!
So every single prior value should be explicitly converted to tensor and registered with self.register_buffer?
Let's me try this and do a PR then.
Looking forward to your replies about other points.

Let's me try this and do a PR then.

There are other things that we have recommendations about. So no need really to wait for this.

What would be the reasons to choose scvi.external over a separate package that depends on scVI?

And regarding this, it probably makes sense for you to have your own package given that it seems like you're trying to build your own ecosystem. We have a template repo you can use to get started. But either way, making the PR makes it a lot easier to discuss the actual code.

Just made a PR! Thanks for feedback!
The template repo is great!

Also, out of curiosity, how much faster is JIT trace elbo?

I do not remember but it depends on data size and is less than 2x. Interestingly pyro seems to be slower than pymc3 and numpyro: https://github.com/pyro-ppl/numpyro/issues/869#issuecomment-766405228

I think that would be best to add scVI-Pyro port of cell2location to the original cell2location repo (merging the original repo with scVI template repo). Then import the original repo in scvi.external to simplify things for users. What do you think @adamgayoso?

@vitkl having your own package seems to make the most sense given my understanding of your goals with the project. We can further discuss importing your model into our external, but if your package is easy enough to run, we could also just link to you on our ecosystem page, which we could also put in our external docs. Happy to continue this conversation.

We decided to keep cell2location as a separate package that depends on scvi-tools but provided a contributed tutorial here https://github.com/YosefLab/scvi-tutorials/blob/master/cell2location_lymph_node_spatial_tutorial.ipynb.
Thanks for all the hard work to make this integration possible and a fantastic collaboration.


Archive API /bags/xx-xxx endpoint

This adds bag lookup capability to the archive API Flask app, which will fetch the contents of a bag using an identifier.
The endpoint on the API is /bags/xx-xxx  as defined in the RFC.

This will need one more rebase against master for the messaging fixes, and then I’ll merge it.


Wrapped sol for devnet too

Working on sollet.io usage of token-list i noticed that Wrapped SOL shows as Solana. This is so it shows the same as mainnet-beta or testnet.

@Arrowana thanks a lot! good catch.


Implements missing compacton selection functionality

Compaction selection functionality that opened files was not implemented because it opens user files in the manager.  This change implements the functionality inorder to get integration test passing.  There is already an open issue about finding a better way to do this. See #3526

This PR along with #3785 and #3784 are all part of an effort to get mvn verify -Psunny working in the elasticity branch.

Would this be another task type for #3801 ?

I wonder if we should get #3801 where it needs to be and merged, then implement this in that framework vs merging this to get sunny working just to have to back it out later.

I wonder if we should get https://github.com/apache/accumulo/pull/3801 where it needs to be and merged, then implement this in that framework vs merging this to get sunny working just to have to back it out later.

Its a small change.  I would like to get the verify sunny build working.  Would like to also look into expanding what is in the sunny tests to make sure all major features are well covered (ingest, scan, compact, split, merge, bulk import, etc).


[feature request] option to include module parameters in agent logs

The existing Empire agent logs have been an invaluable time-saver in keeping records during operations. One area for improvement would be to allow a user-configurable option (cli or global ) to include module parameters in agent log output when modules are executed instead of the current terse single-line output:
2017-05-18 09:24:41 :
Tasked agent to run module lateral_movement/invoke_wmi

Some modules could have fairly verbose and/or numerous options, so it might make sense to only log options that differ from their default value and perhaps concatenate them all onto a single line with separator characters:
2017-05-18 09:24:41 :
Tasked agent to run module lateral_movement/invoke_wmi
Options: Listener='http://myc2.net', UserName='User1', ComputerName='WIN1', Password='Terrible'

I'd be happy to submit a PR if this is deemed a useful feature.

Sure, PR's are always welcome.On Jun 8, 2017 6:47 PM, Chris Frohoff<EMAIL_ADDRESS>wrote:The existing Empire agent logs have been an invaluable time-saver in keeping records during operations. One area for improvement would be to allow a user-configurable option (cli or global ) to include module parameters in agent log output when modules are executed instead of the current terse single-line output:
2017-05-18 09:24:41 :
Tasked agent to run module lateral_movement/invoke_wmi
Some modules could have fairly verbose and/or numerous options, so it might make sense to only log options that differ from their default value and perhaps concatenate them all onto a single line with separator characters:
2017-05-18 09:24:41 :
Tasked agent to run module lateral_movement/invoke_wmi
Options: Listener='http://myc2.net', UserName='User1', ComputerName='WIN1', Password='Terrible'
I'd be happy to submit a PR if this is deemed a useful feature.
—You are receiving this because you are subscribed to this thread.Reply to this email directly, view it on GitHub, or mute the thread.


Where a are preferences stored?

I'm running an Avalonia Desktop application inside a Unix Docker container hosted on Windows. I'd like to persist the user preferences across container runs. To accomplish this, I should be able to save them on a volume. If the preferences are saved on file, a simple way to do this is by mapping (binding) a docker volume on an host's path. So... where are the preferences saved?

Internally it uses the IsolatedStorage API

Yep. On Windows, preferences are saved somewhere under %localappdata%\IsolatedStorage; in Linux it is under ~/.local/share/IsolatedStorage


Another exception inside Node examples/basics-multiTurn: [sprintf] mixing positional and named placeholders is not (yet) supported

$ ~/git/BotBuilder/Node/examples/basics-multiTurn$ node app.js
who founded ibm?
Sure... no problem.

Exception in detail:
 $ node app.js
who founded ibm?
I don't know '%(company)s'. Pick a company from the list or say 'nevermind'.
[ 'I don\'t know \'%(company)s\'. Pick a company from the list or say \'nevermind\'.%s',
  '\n   1. Microsoft\n   2. Apple\n   3. Google\n   4. Amazon' ]
{ resumed: 1,
  error:
   Error: [sprintf] mixing positional and named placeholders is not (yet) supported
       at Function.sprintf.parse (/home/ubuntu/git/BotBuilder/Node/node_modules/sprintf-js/src/sprintf.js:159:27)
       at sprintf (/home/ubuntu/git/BotBuilder/Node/node_modules/sprintf-js/src/sprintf.js:19:34)
       at Object.vsprintf (/home/ubuntu/git/BotBuilder/Node/node_modules/sprintf-js/src/sprintf.js:174:24)
       at Session.vgettext (/home/ubuntu/git/BotBuilder/Node/lib/Session.js:227:50)
       at Session.gettext (/home/ubuntu/git/BotBuilder/Node/lib/Session.js:59:21)
       at Message.setText (/home/ubuntu/git/BotBuilder/Node/lib/Message.js:21:52)
       at Prompts.sendPrompt (/home/ubuntu/git/BotBuilder/Node/lib/dialogs/Prompts.js:228:25)
       at Prompts.begin (/home/ubuntu/git/BotBuilder/Node/lib/dialogs/Prompts.js:130:14)
       at Session.beginDialog (/home/ubuntu/git/BotBuilder/Node/lib/Session.js:108:16)
       at beginPrompt (/home/ubuntu/git/BotBuilder/Node/lib/dialogs/Prompts.js:302:13),
  childId: 'BotBuilder.Dialogs.Prompts' }
Sure... no problem.

Good catch... That would have been introduced with a recent change I made. Let me look into that.

checked in a fix. it was actually a simple type-o. I used the wrong prompt when the entity is missing.


emulate-scroll-for-no-mouse-alternate-buffer not working?

Hi!
I installed this plugin today, but unfortunately, the option @emulate-scroll-for-no-mouse-alternate-buffer is not working properly for me. Here is my configuration:
run-shell ~/config/tmux/plugins/tmux-better-mouse-mode/scroll_copy_mode.tmux
set -g mouse on
set -g @emulate-scroll-for-no-mouse-alternate-buffer on

I'd like to be able to scroll in less using my mouse wheel, but it doesn't work.
Here's the output of tmux list-keys -T root:
bind-key -T root MouseDown1Pane   select-pane -t = ; send-keys -M
bind-key -T root MouseDown1Status select-window -t =
bind-key -T root MouseDown3Pane   if-shell -F -t = "#{mouse_any_flag}" "select-pane -t=; send-keys -M" "select-pane -mt="
bind-key -T root MouseDrag1Pane   if-shell -F -t = "#{mouse_any_flag}" "if -Ft= \"#{pane_in_mode}\" \"copy-mode -M\" \"send-keys -M\"" "copy-mode -M"
bind-key -T root MouseDrag1Border resize-pane -M
bind-key -T root WheelUpPane      if-shell -F -t = "#{mouse_any_flag}" "send-keys -M" "         if -Ft= ''           \"send-keys -t= up ; send-keys -t= up ; send-keys -t= up ; \"           \"             select-pane -t= ;             if -Ft= '#{pane_in_mode}'               'send-keys -M ; send-keys -M ; send-keys -M ; '               'copy-mode -e ; send-keys -M ; send-keys -M ; send-keys -M ; '           \"       "
bind-key -T root WheelUpStatus    previous-window
bind-key -T root WheelDownPane    if-shell -F -t = "#{mouse_any_flag}" "send-keys -M" "         if -Ft= \"\"           \"send-keys -t= down ; send-keys -t= down ; send-keys -t= down ; \"           \"select-pane -t= ; send-keys -M ; send-keys -M ; send-keys -M ; \"       "
bind-key -T root WheelDownStatus  next-window

Have I missed something? I'm running OpenBSD 6.3.

Unfortunately, tmux on OpenBSD has no version, as it’s a part of the operating system itself :-) The version, I suppose, is tmux as of OpenBSD 6.3.

Nathan Daly wrote:
Hmm, let's start with some small questions: which version of tmux do you have?

I have the same issue, here are all my deets:
Arch Linux (4.18.16.arch1-1)
tmux 2.8-1
terminal emulator: alacritty 0.2.1-1
relevant bits from ~/.tmux.conf
set -g @plugin 'tmux-plugins/tpm'
set -g mouse on
set -g @plugin 'nhdaly/tmux-better-mouse-mode'
set -g @scroll-without-changing-pane "on"
set -g @emulate-scroll-for-no-mouse-alternate-buffer "on"
set -g @scroll-speed-num-lines-per-scroll "3"

$ tmux list-keys -T root
bind-key -T root MouseDown1Pane   select-pane -t = ; send-keys -M
bind-key -T root MouseDown1Status select-window -t =
bind-key -T root MouseDown3Pane   if-shell -F -t = "#{mouse_any_flag}" "select-pane -t=; send-keys -M" "select-pane -mt="
bind-key -T root MouseDrag1Pane   if-shell -F -t = "#{mouse_any_flag}" "if -Ft= \"#{pane_in_mode}\" \"copy-mode -M\" \"send-keys -M\"" "copy-mode -M"
bind-key -T root MouseDrag1Border resize-pane -M
bind-key -T root WheelUpPane      if-shell -F -t = "#{mouse_any_flag}" "send-keys -M" "if -Ft= \"#{pane_in_mode}\" \"send-keys -M\" \"copy-mode -et=\""
bind-key -T root WheelUpStatus    previous-window
bind-key -T root WheelDownStatus  next-window
bind-key -T root M-Up             select-pane -U
bind-key -T root M-Down           select-pane -D
bind-key -T root M-Left           select-pane -L
bind-key -T root M-Right          select-pane -R

Hopefully I'm just doing something dumb

Solution!
I know this issue is old, but I found the solution (at least for myself on Linux Mint 19.x). I was using an older version of less and was using the following option in my LESS environment variable (set in zshrc):
 -X or --no-init
                Disables sending the termcap initialization  and  deinitializa‐
                tion  strings  to the terminal.  This is sometimes desirable if
                the deinitialization string does  something  unnecessary,  like
                clearing the screen.

I was doing this for a good reason (that I can't quite remember now), but it made it so less was not sent to an alternate buffer (alternate_on didn't register less as a valid alt process).
Luckily, the latest version of less has mouse support! You just need to do the following:

Install latest version of less from source (551 at this time): http://www.greenwoodsoftware.com/less/download.html
Update LESS config in zshrc to do what you want
Have the correct config in tmux.conf

I'll let you figure out 1 for yourself.
For LESS, here's my config:
LESS="--ignore-case --status-column --LONG-PROMPT --RAW-CONTROL-CHARS"
LESS="$LESS --HILITE-UNREAD --tabs=4 --quit-if-one-screen"
export LESS
export LESS_TERMCAP_mb=$'\E[1;31m'     # begin bold
export LESS_TERMCAP_md=$'\E[1;36m'     # begin blink
export LESS_TERMCAP_me=$'\E[0m'        # reset bold/blink
export LESS_TERMCAP_so=$'\E[01;44;33m' # begin reverse video
export LESS_TERMCAP_se=$'\E[0m'        # reset reverse video
export LESS_TERMCAP_us=$'\E[1;32m'     # begin underline
export LESS_TERMCAP_ue=$'\E[0m'        # reset underline
export PAGER=less

For tmux, here's the relevant config (assuming you've already configured your mouse correctly)
# Options for: nhdaly/tmux-better-mouse-mode
set -g @scroll-down-exit-copy-mode "on"
set -g @scroll-without-changing-pane "on"
set -g @scroll-in-moused-over-pane "on"
set -g @scroll-speed-num-lines-per-scroll "3"
set -g @emulate-scroll-for-no-mouse-alternate-buffer "on"

set -g @plugin 'nhdaly/tmux-better-mouse-mode'


refine region request debug log

Log region request round failed log when total backoff duration is more than 1s.

/merge


feat(select): add experimental select

Closes https://github.com/IBM/carbon-components/issues/1211
Adds in experimental Select
Changelog
New

Experimental Select, Experimental InlineSelect

Changed

Unnested selectors in select.scss
Updated text-input.scss and number-input.scss to use the focus outline mixin (Blocked by https://github.com/IBM/carbon-components/pull/1337)
Updated NumberInput height (40px --> 32px) and adjusted arrows accordingly

Removed

Old unnecessary styles

Testing / Reviewing
Will add a staging link once Date Picker styles have been merged in, as this relies on styles from that PR

The select is missing the border bottom but besides that looks good 👍 ✅


Gradle plugin software.amazon.smithy.rust.codegen.server.smithy:codegen-server:0.1.0 not found

I've used Smithy with Java services at Amazon and this is my first trying Smithy outside of Amazon. After cloning the application and running Gradle, I see the exception:
* What went wrong:
A problem occurred configuring project ':model'.
> Could not determine the dependencies of null.
   > Could not resolve all task dependencies for configuration ':model:classpath'.
      > Could not find software.amazon.smithy.rust.codegen.server.smithy:codegen-server:0.1.0.
        Searched in the following locations:
          - file:/home/home/.m2/repository/software/amazon/smithy/rust/codegen/server/smithy/codegen-server/0.1.0/codegen-server-0.1.0.pom
          - https://plugins.gradle.org/m2/software/amazon/smithy/rust/codegen/server/smithy/codegen-server/0.1.0/codegen-server-0.1.0.pom
        Required by:
            project :model

On reviewing the URL above, I see the exception:
{
  "errors" : [ {
    "status" : 404,
    "message" : "Could not find resource"
  } ]
}

This is the most recent version of the plugin on the smithy-rs repository as well. Do you know why I cannot see/access this Gradle plugin?

Hum.. this repository needs some love as something has probably changed inside smithy-rs. I'll try to reproduce and fix this in the next couple of days.

@PranavBhasin001 Have you pulled in the git submodule?
From the root of the package:
git submodule init
git submodule update

I am working on updating this to the latest version of smithy-rs and I am also updating the README.


Privacy related vulnerabilities

What would you like to be added:
I want horusec to point out vulnerabilities related to actual privacy related rules for countries.
Using Brazil as an Example, we actually have the LGPD (short for Lei Geral de Proteção de Dados, or General Data Protection Law), which prevents developers from exposing/keeping users data (Ex: CPF, RG -> Brazil IDs).
Those Rules should point out for example, the use of those words in concats with another strings:
This should be pointed out by Horusec:
console.log("CPF: " + client.document);

This not:
console.log("This string have a arbitrary word which uses cpf but it is not related to the document");

I though that maybe we could add a flag to be used by horusec start to specify the country to be used as base on the analysis something like:
horusec start --privacy-vuls="br"
Would use Brazilian LGPD Rules only.
horusec start --privacy-vuls="br,eu"
Would use Brazilian (LGPD) and european (GDPR) related Rules.
Not specifying the --privacy-vuls would ignore these privacy related rules.
Why is this needed:
Actually Privacy is having a huge amount of interest by the media, governments and companies.
A SAST that could in theory prevent those breaks would in thesis help developers avoid leaking users data.

Hi @guilhermepaulozup, Any references where LGPD can be obtained? I'm curious what the requirements are in the LGPD.

Hi @ne0z . Sure, most of the references are in PT-BR but I found some articles in english.
https://usercentrics.com/knowledge-hub/brazil-lgpd-general-data-protection-law-overview/#:~:text=The General Data Protection Law,effect on August 16%2C 2020.
I really expect, at first, only to stop the hardcoding/exposure of those documents on code bases.


Server Error when types wrong URL.

When you type wrong url which not refer to a existing user there is an error.
example : https://blocksocial.vercel.app/users/a-non-existing-user

I forgot to add commit message to close this issue.
ce398b7f280ed8341b34e2f24a4040663d78a931


of teams constraint

@DimaKudosh Is there an easy way to add a constraint for an exact number of teams? E.g., say I only want lineups that have 3 teams total - what's the easiest way to apply that constraint?

I've added set_total_teams method for optimizer


Jest@27 compatibility

<EMAIL_ADDRESS>Release notes

Issues

[ ] JSDOM is no longer the default test environment.
[x] setImmediate is not defined (#123)
[ ] TypeError [ERR_INVALID_ARG_TYPE]: The "options.agent" property must be one of Agent-like Object, undefined, or false. Received an instance of Object
[ ] Test functions cannot both take a 'done' callback and return something. Either use a 'done' callback, or return a promise. (previously less strict).

Invalid options.agent
The error originates from this Agent check in Node.js:
let agent = options.agent;
const defaultAgent = options._defaultAgent || Agent.globalAgent;
if (agent === false) {
  agent = new defaultAgent.constructor();
} else if (agent === null || agent === undefined) {
  if (typeof options.createConnection !== 'function') {
    agent = defaultAgent;
  }
  // Explicitly pass through this statement as agent will not be used
  // when createConnection is provided.
} else if (typeof agent.addRequest !== 'function') {
  throw new ERR_INVALID_ARG_TYPE('options.agent',
                                  ['Agent-like Object', 'undefined', 'false'],
                                  agent);
}

—Source

@kettanaito Could we get a release for these fixes?


Add rule names to compiled schema

Package version
1.7.0
Describe the bug
Opening this as a bug because there's no way to open feature request issues right now.
Add names to rules similar to the Symbol(schema_nme) on s, this will allow for tooling like vine to zod/vine to json schema to be more complete feature-wise. Including a Symbol(rule_name) in rules would do the trick
Reproduction repo
No response

Hey, sorry picking up it a bit late. Can you please share some examples of that exactly you need in the compiled output?

Closing since not actionable


Fix allowing null password and email

the formValidation middleware has refactored for check null or empty field like those ones


Remove label node from point annotations from init samples

Remove label node from point annotations, because not managed by poitns, from init samples.

Thank youuu @LeeLenaleee


feat!: add foundations for scaling mechanism

This PR adds the foundations for a scaling mechanism based on a fixed set of producer partitions that is split across the available consumers.

This PR should now be ready for an initial review.
Note that I adjusted the data type of the variables related to the partitioning mechanism due to the random number generator I used in the producer class — instead of uint64_t, the type of the respective variables is now uint32_t. I suppose the maximum value here should still be large enough, though.


Implement Authentication

@chriddyp I have implemented a barebones version of how authorization will take place in DB Connector.
It would be great if you could take a preliminary look and let me know if I am going in the right direction.. 😸
Meanwhile, I would be working on implementing the actual logic and the process.

@chriddyp This is ready for a review..
There is one small tweak needed for electron apps, to automatically redirect from login page to connections page. Right now it relies on checking cookies, which is not available in electron.. but I can handle it as a separate PR.

Update --  Added the electron tweak as well. It allows user to access electron app when not-logged in, but allows option to log-in as well:

Fantastic. So to confirm, you can use Falcom without logging in? People hate being forced to sign up for a platform when it's not necessary (ie they just want to query SQL and don't care about plotly), so I think this would be awesome.

So to confirm, you can use Falcom without logging in? People hate being forced to sign up for a platform when it's not necessary (ie they just want to query SQL and don't care about plotly), so I think this would be awesome.

For electron app --> Yes.
For headless/server version, we provide a configuration called AUTH_ENABLED which allows users/admins to switch off authentication as necessary (it is enabled by default)..

on server when AUTH_ENABLED=true  :

@chriddyp getting error on Mac Build:

Are there any specific steps we need to follow?
I am running:
$ yarn run package
$ ./zipItAll.sh

I have tried removing node_modules and re-installing + rebuilding everything as well..

@tarzzz - yarn run package looks like the correct command. I'm not sure why this error is cropping up all of a sudden

Headless build: https://plotly-connector-pr-180.herokuapp.com/
Still looking into the mac build issue..

@tarzzz It shouldn't happen.
source-map-support is listed as a package dependency.
Could you check there is source-map-support in release/[...]/Resources/app/node_modules?
If it doesn't have source-map-support, wipe the release folder, make sure you've got enough hard drive space (> 600 MB) and rerun yarn run package.

@n-riesco Even after trying yarn run package multiple times.. the release/[...]/Resources/app/node_modules is empty. There seems to be an issue with the package builder?

$ release/Mac-64bit/Plotly Database Connector-darwin-x64/Plotly Database Connector.app/Contents/Resources/app/node_modules pwd
release/Mac-64bit/Plotly Database Connector-darwin-x64/Plotly Database Connector.app/Contents/Resources/app/node_modules
$ release/Mac-64bit/Plotly Database Connector-darwin-x64/Plotly Database Connector.app/Contents/Resources/app/node_modules ls
$ release/Mac-64bit/Plotly Database Connector-darwin-x64/Plotly Database Connector.app/Contents/Resources/app/node_modules

@tarzzz Let's talk about this on Slack when you're around.

Update: Addressed the comments and fixed existing tests..
I am working on covering these test-cases:

making a request after the 5 min expiry token expires makes another request to plotly
making a request within the 5 min expiry doesn't make a request to plotly
if a user is revoked within 5 minutes then they can still access the connector but after the token expires they are denied access
loop through all of the GET endpoints and verify that 401s are returned. i believe that you can
access the list of routes dynamically with server.router.mounts
requests to /, /login/, /static/, and /oauth2/ return in 200s

The last one has already been covered.. so working on others..

@chriddyp Addressed the comments and added some tests related to accessTokens (as mentioned above).. Please let me know if it looks good to you !! 😸

woohoo! 💃  once my comments are addressed. alternatively, I'm happy to review it again if you'd like.

@chriddyp Addressed the comments. let me know if it, feel good to be merged.. 😸

@chriddyp -- Added updates:

Updated settings names,
Added ACCESS_TOKEN_EXPIRY logic while re-generating token, as well as while authenticating,
Fixes promises related issue in tests. Thanks for such an informative codepen example.. 😸

There are some test failures (which are working fine locally for me), working on a fix.. but let me know if code looks good otherwise..

looking good! just a couple of more small comments

woohoo.. merging..  🎉


A different approach for workspace management

Hey everyone! 👋
First and foremost, thank you for this amazing piece of work! I truly missed something like this on macos, since other alternatives are not usable in strict work environment, due to system integrity protection. This application is Godsend!
I was thinking about how to solve the visible application on the bottom right corner problem.
I read in the docs, that:

For better or worse, macOS doesn’t allow to place windows outside the visible area entirely. You will still be able to see a few pixels of "hidden" windows in the bottom right corner of your screen. That means, that if AeroSpace crashes badly you will still be able to manually "unhide" the windows by dragging these few pixels to the center of the screen.

And the reasoning behind that is:

Apple doesn’t provide public API to communicate with Spaces (create/delete/reorder/switch Space and move windows between Spaces)

I checked the state of this in hammerspoon, since I remembered, that I managed that somehow before and it's still doable:
https://github.com/Hammerspoon/hammerspoon/issues/3111
Do you think moving the currently invisible applications into a different space would do the trick? I don't mind to have two different spaces (1. the actual visible space where I manage my AeroSpace workspaces and 2. the space where all the non-visible windows are)
Thanks in advance!
If you are happy with the proposed solution, I'm more than happy to open a PR in the following weeks!

Moving windows between macOS Spaces requires private API usage, it goes against values of the project.
You may also heard that Apple broke those private APIs in a recent macOS 14.5 update and all the apps that were using them had been broken until they found a workaround.
I agree that the current approach of hiding windows can be improved though, see: #149 #219


CI: schedule daily build to detect breakages

This package hasn't been built in a while and locally I saw a failure

/home/alex/ros/h/foo/src/topic_based_ros2_control/test/topic_based_system_test.cpp:38:10: fatal error: ros2_control_test_assets/descriptions.hpp: No such file or directory
   38 | #include <ros2_control_test_assets/descriptions.hpp>
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

If you build against ros2_control from the humble branch this will fail.

Thanks! @moriarty Do you mind fixing it? I'll merge the PR after that

Someone from Picknik needs to disable clang-tidy. @MarqRazz Do you mind disabling clang-tidy? I don't have admin access


LGU 2.8.8 doesn't show modded items.

This isn't only affected by LGU. scpecific mods like "buyable shotgun shells" don't appear on the Store item list.
For example. I don't see peeper on the shop and when typing "peeper" says there is nothing. I already checked many times and my mods/libraries are up to date.
Seems like the issue is LethalLib. what version should I use then?

Latest version of LGU and LethalLib installed and I can buy the items mentioned in the issue.
Are you absolutely sure the mods are being loaded?
You can check by looking at either the console it opens upon launch or the file "LogOutput.log" generated in BepInEx folder.
Do these plugin folders indeed contain the .dll file?
I noticed that thunderstore sometimes does not download the .dll file, even though the mod manager say it has.

Mods loads (as this logs says.) such as LGU and buyable Shells.
Plugins constains their respective .dll and .pdb files.
Also I want to add that I added a JJK scraps theme but I haven't seem them on the map. Even in DanceTools console.
(No, DanceTools don't display lgu/buyable shells items store)
but I saw this message. not really sure what it means but it doesn't seems related to the shop in any way.
[Info   : LethalLib] LethalLib loaded!!
[Error  : Unity Log] TypeLoadException: Could not load type of field 'LethalLib.Modules.Enemies+<>O:<0>__RegisterLevelEnemies' (0) due to: Could not load file or assembly 'MMHOOK_Assembly-CSharp, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null' or one of its dependencies.
Stack trace:
LethalLib.Plugin.Awake () (at :IL_0078)
UnityEngine.GameObject:AddComponent(Type)
BepInEx.Bootstrap.Chainloader:Start()
UnityEngine.Rendering.HighDefinition.HDRenderPipelineAsset:OnEnable()
I even tried to uninstall some mods. I tried with only these but still nothing

Any help?

You also require the HookGenPatcher for LethalLib to function correctly.

 The patcher folder from hookgen's was on another place. Thank you so much for make me notice.


Subaru Crosstrek 2018 keeps saying "Car Unrecognized"

Describe the bug
Whenever I start the car up, the screen just say's "Car Unrecognized" and goes into Dashcam mode. It's worked twice, where I went into calibration mode and set it up the first time, and the car drove me the second time. Every other time it's just come up with this error.
How to reproduce or log data
Start up the car
Expected behavior
I'd expect the comma to behave as it should, taking over the controls when cruise control is activated.
Device/Version information (please complete the following information):

Device: [comma two]
Dongle ID: [44f74c0e55ca7c27]
Route: [44f74c0e55ca7c27 2020-11-26--16-46-32]
Timestamp: [When in the route the bug occurs (e.g. 4min 30s into the drive)]
Version: [0.7.6)]
Car make/model: [Subaru Crosstrek 2018 (UK)]

Additional context

I see you also ran a fork which does recognize your car. If you submit a PR with the fingerprint for your car we can add support.

Hi, I'm so sorry for the delay in getting back to you @pd0wm
How would I go about submitting a PF with the fingerprint? Would this mean I'd just use the official open pilot and it would work?
Unfortunately, for some reason, my vehicle is no longer recognised with he branch I was using, and the MartinL branch is also giving me errors - "White and grey panda are usupported. Upgrade to comma two or black panda"
Thank you in advance, and sorry again for the delay!

Hi, I'm so sorry for the delay in getting back to you @pd0wm
How would I go about submitting a PF with the fingerprint? Would this mean I'd just use the official open pilot and it would work?
Unfortunately, for some reason, my vehicle is no longer recognised with he branch I was using, and the MartinL branch is also giving me errors - "White and grey panda are usupported. Upgrade to comma two or black panda"
Thank you in advance, and sorry again for the delay!

I'm having the same problem with my Subaru Crosstrek 2019. Is it because my phone is not pairing with the dash cam?


US-73: Implement AI Shooting For Enemies

US-73:
As a player I want the enemies in the game to fire back at me so that the game is more interesting to play.
This user story is about implementing AI behavior in enemies for shooting at the player.
Tasks:

[ ] Extend AICommander with methods for shooting at the player

Acceptance Criteria:

[ ] Enemies shoot at the player when they are in range
[ ] Enemies fire projectiles in the direction of the player

This already exists dumdum💋🍻


Introduce NBNavigationSplitView as a backport of NavigationSplitView

Apple introduced NavigationSplitView in WWDC22, this is a backport implementation of it with limitation:

only support double colunm layout (sidebar + detail)
can not nested in NBNavigationStack

Pease see the demo for usage

Thanks @strangeliu for the PR. I haven't yet had a chance to look at it in detail but I will try to soon.

Any updates here? Looking forward to have NBNavigationSplitView

Sorry for the radio silence on this issue. I've been working on a slightly different implementation that would overcome the drawbacks that @strangeliu mentioned (thanks for your work on this @strangeliu!).   I'm just working through some remaining issues before sharing it. Thanks for your patience.

I've raised a PR #28, attempting to overcome the limitations @strangeliu described here. The API more closely resembles that of NavigationSplitView, in that you can have two or three pane layouts, and you can nest a NBNavigationStack inside a NBNavigationSplitView, though it must be within the detail pane only. Thanks @strangeliu for instigating this. If anyone feels like checking out that branch and trying it out, any feedback would be gratefully received. 🙏


Add a better HTTP test-bench interface

The current request interface does not correctly elevate exceptions thrown within it's request/response handlers. Re-implementing a similar interface around a promise fixes this problem and replaces some boiler-plate required in Mocha cases.

👍  This looks good. 🚢  it.


🍒 Cherry pick PR #4632 to staging 🍒

🍒 Cherry pick https://github.com/Expensify/App/pull/4632 to staging 🍒

🚀 Cherry-picked to staging by @OSBotify in version: 1.0.85-4 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by @isagoico in version: 1.0.85-9 🚀

platform
result

🤖 android 🤖
failure ❌

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


BFI should also do bit flip

New stuff added to this PR:

Inject transformer Byzantine faults
Faults are generated inductively, not up front
Using a generic accumulator in BFI
Implementation class and unit test for transformer BFI for strings
Using a different background colour for Byzantine faults
Implementing afterInvocationWithByzantineFault and using it instead of afterInvocationWithException for BFI

Closes #181 and #165

New stuff added to this PR:

Inject transformer Byzantine faults
Faults are generated inductively, not up front
Implementation class and unit test for transformer BFI for strings
Using a different background colour for Byzantine faults
Implementing afterInvocationWithByzantineFault and using it instead of afterInvocationWithException for BFI

in the UX, the injected response should look the same as the response without FI: so like 'example' int he normal execution and 'examp1e' in the BFI execution

I don't see coral.  What am I missing?

Oh, perhaps my monitor is just not good enough to distinguish the red from the coral, they look almost identical.

It looks better on the MacBook display, but not much different on my external widescreen.

Bah, OK, just leave it as it is for now.

update that changelog yo

no need to bump version, we'll target all of this at 1.8.0

This is looking good, only a few minor things left to address.

Please do not forget follow-up PR to remove use of byzantine with transformer in Filibuster libraries: Core, Instrumentor, etc.

Honestly, it would just be easier if you did that in this PR, actually.

I'll mark the files accordingly.

StatefulRedisConnection<String, String> myStatefulRedisConnection = new RedisInterceptorFactory<>(statefulRedisConnection, redisConnectionString).getProxy(StatefulRedisConnection.class);
        RedisCommands<String, String> myRedisCommands = myStatefulRedisConnection.sync();

        myRedisCommands.get(key);
        myRedisCommands.get(key);
        myRedisCommands.get(key);

This test will trigger that if statement to get hit, because there are more than one RPC, and when you hit endInvocation for the second RPC in the non-reference execution, you'll hit this to avoid injecting faults for the first RPC.  That seems to be working correctly.
However, suppressCombinations doesn't work and the UX is also busted.  If I run that above with suppressCombinations on, I see it run tests anyway where combinations are present and faults are not rendered correctly.
See:


Adds support for ignoreFailures to gradle plugin

Closes #1435
The ignoreFailures property inherited from  CodeQualityExtension now can be used to continue the build even if there are more findings than allowed.

Can you please also add tests showing ignoreFailures causes a detekt task to succeed even when there are failures over the threshold amount?
I forgot to mention that I did try that but for some reason the gradle runner seems to return exit code 1 instead of 2. I will have another look.

the gradle runner seems to return exit code 1 instead of 2.

I believe that's because the change introducing exit code 2 is only on master branch, and at the moment the Gradle plugin tests actually pull detekt-cli from an external repo during testing. Because exit code 2 feature isn't in any released version yet that test won't pass.
I'm going to try and improve that situation for the plugin tests, but in the meantime let's hope @arturbosch releases RC15 soon ;)
Then you can rebase and add that test.


No module named 'blinker._saferef' error when running generate_conf.py

🐛 Summary
When running python3 scripts/generate_conf.py, the following error occurs: ModuleNotFoundError: No module named 'blinker._saferef'
To reproduce
Steps to reproduce the behavior:

Follow set up instructions and run python3 scripts/generate_conf.py

Expected behavior
Config file should have been generated, instead got an error.
more info
This appears to be an issue with selenium. See this issue for more details: https://github.com/seleniumbase/SeleniumBase/issues/2782
Per that thread, I manually did pip install blinker==1.7.0 and was then able to generate the config file.

This script has been removed in the latest release of the tool and is no longer an issue. Thank you!


Pulling down image returning 503 error

Our builds started failing last night. Seeing this error
***[group]Pull down action image 'ghcr.io/rtcamp/action-slack-notify:v2.2.0'
  ***[command]/usr/bin/docker pull ghcr.io/rtcamp/action-slack-notify:v2.2.0
  Error response from daemon: Head "https://ghcr.io/v2/rtcamp/action-slack-notify/manifests/v2.2.0": received unexpected HTTP status: 503 Service Unavailable
  Warning: ***[warning]Docker pull failed with exit code 1, back off 5.589 seconds before retry.

Thoughts? Thanks!

I'm getting this error too. Weird

https://www.githubstatus.com packages seems to be down.

Looks resolved now :smile:


implement feature

implement mentioned feature in the project, the piano app view same as mentioned in the project.It's working  without any bug
Please accept my request  as contributing in hacktoberfest.

@Keertijanm  your code implements mentioned feature correctly and within time. I appreciate it.
Accepting  your pull-request regarding hacktoberfest-accepted.


k8sT: Make health test more robust

Previously we never retried attempting to find each cilium-health
endpoint IP in each Cilium pod, so if startup was delayed slightly then
it would be possible for the test to query the list of cluster nodes
before Cilium had a chance to find all of the k8s annotations for
cilium-health instances.
This patch makes this logic a bit more robust by retrying the fetch
logic multiple times, and only failing if particular cilium-health
instances are not known in all Cilium instances after a regular helper
timeout.
Fixes: #3802
Fixes: beff61ed313a ("test: Add basic cilium-health tests")
Signed-off-by: Joe Stringer<EMAIL_ADDRESS>

test-me-please

test-me-please

Hitting a bunch of test flakes, however the actual test that is being modified in this PR is passing:
https://jenkins.cilium.io/job/Cilium-PR-Ginkgo-Tests-Validated/2740/testReport/k8s-1/7/K8sValidatedHealthTest_checks_cilium_health_status_between_nodes/

@eloycoto it passed on both:
https://jenkins.cilium.io/job/Cilium-PR-Ginkgo-Tests-Validated/2749/execution/node/62/log/?consoleFull (search K8sValidatedHealthTest)
https://jenkins.cilium.io/job/Cilium-PR-Ginkgo-Tests-Validated/2749/testReport/k8s-1/7/

@ianvernon @eloycoto Given that this has passed, it doesn't make the affected test worse so I propose shifting this to ready-to-merge. It seems to be constantly hitting issues such as #3901, and IMHO it's just wasting CI time to keep re-running.

Sure, let's merge it.


Activate card confirmation mail per region

Is your feature request related to a problem? Please describe.
As a region i may not want to make the card earlier usable and not send an activation mail.
Describe the solution you'd like

add a section in the administration where region admins can activate/deactivate the card confirmation mail
send only card confirmation mail if the its activated
set the confirmationFeatureFlag true

@f1sh1918 description text for the checkbox is approved and should be in German
"Nach der Erstellung einer Karte verschickt das System eine E-Mail-Bestätigung an den Antragsstellenden mit einem Link zur Vorab-Aktivierung"


Differnt change set behavior for list and results

If there is no actual changes in the list like remove all elements from
list and added them again in the same transaction, the Results built
from List can detect there is no change actually happened thus no
notifications will be triggered. But in the same case, notification will
be triggered for the List with a change set of deletions and insertions.
Not sure if this is by design.

@tgoyne

The behavior is different because for Results it diffs the old and new result sets, while for List it just reports the changes from the transaction log. This means that List can report moves and is dramatically faster, but Results is better at only reporting things which resulted in actual changes.

Good to know it is by design


Verification reports failure but shows no errors or warnings.

Failing code

function filter<T(!new)>(xs: seq<T>, p: (T) -> bool): seq<T>
    ensures forall x: T :: x in xs && p(x) ==> x in filter(xs, p)
    ensures forall x: T :: x !in xs && p(x) ==> x !in filter(xs, p)
    ensures forall x: T :: x in filter(xs, p) ==> p(x)
    ensures forall x: T :: x in filter(xs, p) ==> x in xs[0..|xs|]
    ensures forall x: T :: x in filter(xs, p) ==> x in xs
    ensures forall x: T :: x in xs && !p(x) ==> x !in filter(xs, p)
    ensures forall i: nat :: i < |filter(xs, p)| ==> filter(xs, p)[i] in xs
{
    if xs == [] then [] else if p(xs[0]) then [xs[0]] + filter(xs[1..], p) else filter(xs[1..], p)
}

lemma filterSplit<T(!new)>(s: seq<T>, p: T -> bool, idx: nat)
  requires 0 <= idx <= |s|
  ensures filter(s, p) == filter(s[..idx], p) + filter(s[idx..], p)
{
  if idx == 0 {
  }
  else {
    filterSplit(s[1..], p, idx-1);
    assert filter(s[1..], p) == filter(s[1..][..(idx-1)], p) + filter(s[1..][(idx-1)..], p);
    if p(s[0]) {
      calc {
        filter(s, p);
        [s[0]] + filter(s[1..], p);
        [s[0]] + filter(s[1..][..(idx-1)], p) + filter(s[1..][(idx-1)..], p);
        {
          assert s[..idx] == [s[0]] + s[1..idx];
          assert s[1..idx] == s[1..][..(idx-1)];
        }
        filter(s[..idx], p) + filter(s[1..][(idx-1)..], p);
        {
          assert s[1..][(idx-1)..] == s[idx..];
        }
        filter(s[..idx], p) + filter(s[idx..], p);
      }
    }
    else {}
  }
}

lemma filterMultiSet<T(!new)>(s: seq<T>, p:  T -> bool)
  ensures multiset(filter(s, p)) <= multiset(s)
{
  if s == [] {
  }
  else {
    filterMultiSet(s[1..], p);
    calc <= {
      multiset(filter(s, p));
      multiset([s[0]]) + multiset(filter(s[1..], p));
      multiset([s[0]]) + multiset(s[1..]);
      {
        assert s == [s[0]] + s[1..];
      }
      multiset(s);
    }
  }
}

lemma multisetAndSeqs<T(!new)>(xs: seq<T>)
    ensures |multiset(xs)| == |xs|
{
}

predicate isNegatedBooleanFn<T(!new)>(xs: seq<T>, p: (T) -> bool, q: (T) -> bool) {
    forall x: T :: x in xs && p(x) ==> !q(x)
}

function filter_mset<T(!new)>(ms: multiset<T>, p: (T) -> bool): multiset<T>
    ensures forall x :: x in ms && p(x) ==> x in filter_mset(ms, p) && ms[x] == filter_mset(ms, p)[x]
    ensures forall x :: x in filter_mset(ms, p) ==> p(x)
    ensures forall x :: x in filter_mset(ms, p) ==> x in ms
    ensures forall x: T :: x !in ms && p(x) ==> x !in filter_mset(ms, p)
{
    if ms == multiset{} then multiset{} else
   var x :| x in ms; if p(x) then var result := multiset{}; result[x := ms[x]] + filter_mset(ms[x := 0], p) else filter_mset(ms[x := 0], p)

}

// lemma filterAndFilterMset<T>(ms: seq<T>, p: (T) -> bool)
//     ensures multiset(filter(ms, p)) == filter_mset(multiset(ms), p)
// {
//     assert forall x :: x in filter(ms, p) ==> x in multiset(filter(ms, p)) && p(x);
//     assert forall x :: x in filter(ms, p) ==> x in filter_mset(multiset(ms), p);
//     assert forall x :: x in filter_mset(multiset(ms), p) ==> x in filter(ms, p);
//     filterLemmaSizes(ms, filter(ms, p), p);
//     assert forall x :: x in filter(ms, p) ==> multiset(filter(ms, p))[x] == filter_mset(multiset(ms), p)[x];
// }

lemma filterMsetAndSum<T(!new)>(xs: seq<T>, ms: multiset<T>, p: (T) -> bool)
    requires ms == multiset(xs)
    ensures exists Q: (T) -> bool :: isNegatedBooleanFn(xs, p, Q) && (filter_mset(ms, p) + filter_mset(ms, Q)) == ms
{
    filterMS(xs, p);
    var Q :| isNegatedBooleanFn(xs, p, Q);
    var sum_ms := filter_mset(ms, p) + filter_mset(ms, Q);
    forall x | x in ms
        ensures ms[x] == sum_ms[x]
    {
        if p(x) {
            assert x in filter_mset(ms, p);
            assert filter_mset(ms, p)[x] == ms[x];
            assert x in sum_ms;
            assert sum_ms[x] == ms[x];
        }else {
            assert x in filter_mset(ms, Q);
            assert filter_mset(ms, Q)[x] == ms[x];
            assert x in sum_ms;
            assert sum_ms[x] == ms[x];
        }
    }
    assert sum_ms == ms;
}

lemma filterMS<T(!new)>(xs: seq<T>, p: (T) -> bool)
 ensures exists q: (T) -> bool :: isNegatedBooleanFn(xs, p, q)
{
  var q: (T) -> bool := y => !p(y);
  forall x | x in xs
    ensures x in xs && p(x) ==> !q(x)
  {
    if p(x) {
        assert !q(x);
    }
  }
  assert isNegatedBooleanFn(xs, p, q);
//   assert forall x: T :: x in xs && p(x) ==> !q(x);

}

Steps to reproduce the issue

Dafny version: <IP_ADDRESS>719
Dafny VSCode extension version: 2.7.0

Expected behavior
If verification fails, I expect it will tell me where it is failing...
Actual behavior
It reports that verification failed, but it does not show any problems, nor highlight any lines which are failing. The entire bar on the left is green.

Although there is clearly an error on VSCode, on my side the vertical bar isn't entirely green (gutter icons), but there is a yellow context without the assertion.
It looks like the faulty assertion is
assert x in filter_mset(ms, Q); in the else branch.
When I change this assert to assume, verification passes. It looks like the reporting is not indicating the position, and the diagnostic is not reported.

This has been fixed in the meanwhile, so closing.


Framebuffer is incomplete: Attachments are not all the same size.

Using PostProcessingSprite() sometimes the sprite just doesn't get rendered. It's than throwing these kinde of errors:

I'm not using any textures on the 3d model, so it seems like it's talking about the PostProcessingSprite itself as being a texture.
I believe I've seen other people with the same error message on the pixi github, but I can't find it nowhere anymore and even then to me and I believe also to them it was completely unclear what was causing this and how this could even be worked around. I'm trying to debug this at the moment.
Any clue on what could cause this issue to happen? Could it be related to using the PostProcessingSprite's resolution, width or height settings? Does this even have anything to do with textures? Or could it even be something else completely?

Could it perhaps be related to this issue from years ago?:
https://github.com/pixijs/pixijs/issues/746

Do you have an example which sometimes reproduces this error?

@jnsmalm Sorry, can't send you the project. When debugging I would try to create a new, small, project to isolate an issue. But the thing is that yesterday when I tried to debug this the issue didn't happen once and couldn't replicate it. While at other times it happens every two or third time the model is shown. It seems completely random.
I did change a little thing now tho, I was first changing the resolution of the sprite to a value that had a lot of decimals, because Javascript didn't seem to be able to do a simple calculation like 41*0.01. It didn't result in 0.41, but in 0.41000000000000003 instead. Which to me is weird (try it yourself in a Chrome dev console: console.log(41*0.01). Now I'm using toFixed(2) to round instead. As you understand, it would be weird if this would be the reason of the issue, but who knows.
Long story short, it's a little difficult to create a small demo if not sure what exactly to isolate and when exactly this issue appears so we can replicate it.


[PyTorch/C] Fix compiling warnings and backend selection logic for fused attention

This PR fixes the compiling warnings in SDPA flash after upgrading to cuDNN frontend v1. It also fixes the logic for selecting the SDPA flash backend on sm80 when cuDNN runtime version is >8.9.6.

/te-ci

Pipeline 11413511

Pipeline 11414037

/te-ci pytorch

Pipeline 11439646

With newer cuDNN, pipeline 11441309

/te-ci

Pipelines 11444120 (PyTorch and Paddle) and 11464931 (Jax with an older Jax container) passed.


Modularity in tremor script and query

I like this RFC. It tackles a problem that clearly exists, and does it in a principled way. The whole spectrum of the problem is well devised, from the language-theoretic aspects around the tricky issue of allowing recursion, to the practical issues like TREMOR_PATH.
In the interest of improving it even further, I provide some "C"s to the "RFC" below.
Detailed Comments
On the Preprocessor
I dislike the way that the preprocessor is made part of "the specification" in this RFC. Using a preprocessor as a transparent way of adding modules to Tremor Script/Trickle is great. However, the specification of a #!line and #!config preprocessor directive cements "the preprocesser" as a part of Tremor; as something that has to be supported in a backwards-compatible way forever.
In many cases, inlining a module is a great solution. On the other hand, I can envision situations where a different semantics would be desired. By the specification of this RFC, if a large (subtree of) module(s) is used from many scripts in the same Tremor deployment, the runtime has to duplicate all data pertaining to those modules, including constants and AST nodes, just because the specification says so. As far as I can tell, the only requirements that this RFC should impose are that

the use-chain is tree-shaped (i.e., no cycles),
the root of each tree is a non-module file, and that
the nodes of the tree do not change after initial resolve.

But it's likely that I'm missing some insight. Perhaps related to that, I don't fully get what is meant by

The latter constraint is to ensure that logic deployed into the runtime is always traceable to source loaded by a user.

On side-effects in Tremor Scripts
The RFC, including the statement

By design constraint at this time, tremor-script is biased towards pure side-effect free functional programming.

leaves open the possibility for side-effecting code in Tremor Script. In practice, only very constrained forms of side-effects exist in Tremor Script. I might be missing something, so to explain my reasoning: emit and drop act as early returns, not side-effects. Re-assignments to variables is logically equivalent to declaring new variables due to the limitations on loops. patches logically yield a new value. The only way to have proper side-effecting actions, is by linking custom natively implemented functions (= "intrinsic"/"built-in"?) into the runtime (and — if we're being pedantic — the functions in the std::random/tremor::origin/tremor::system namespaces are technically not referentially-transparent, yet still deterministic in the context of a certain sequence of events), but it's not clear to me to what extent adding side-effecting functions is supported (e.g., "you can technically do it, but if it breaks it's your problem").
The RFC also states that

side effecting operations such as with the emit or drop are also not allowed in functions.

As mentioned, I don't even really consider these to be side-effects: from the perspective of the script they're just fancy returns. All in all, the RFC leaves the status of side-effects in Tremor Script/Trickle underspecified. Is it the intent that the new functions are purely functional? More to the point: would, e.g., adding an optimizer that assumes that referential transparency holds, be backwards compatible with Tremor Script/Trickle code that is written to this RFC's specification?
On Partial Functions
The part of the RFC specifying "Matching Functions" states that

The match form supports partial functions.

but also

The matching function form imposes a default case requirement so that unmatches cases have error handling defined. Unlike match expressions the default case in user defined functions must not ( and can not ) be omitted.

Does the first quotation refer to match expressions? If not, both quotations appear to be at odds: if a default case must be present, then the matching function automatically becomes a total function, no?
On Recursion
Obviously, not allowing multiple recursion, limits the expressiveness. On the other hand, embedded Tremor scripts probably aren't the place to do a lot of tree-traversals anyway. I support the choice of limiting loops to only for comprehensions and tail recursion. Similarly, higher-order functions seem like a can of worms you shouldn't open unless you have a clear use-case (do make sure that the Tremor Script documentation makes clear that functions are not values that can be assigned to variables!).
A minor "paper-cut" with the current approach of defining functions only when their definition ends, is that the order in which functions are defined in a module matters. Do you know whether your users will expect that? (In any case, this behavior is not specified in the RFC document, and the usability could easily be improved at a later point by adding a specific error message, should it turn out users are confused by this.)
On Terminating when Depth > RECURSION_LIMIT
The hardcoded recursion depth feels like the roughest edge in the RFC. It is the only point where the pragmatism threatens the model. It's difficult to add backwards-compatible improvements later, when currently the language semantics boils down to "don't recurse too often, eh". Any theshold that's picked (at language-runtime-compile-time!) will be either too strict for simple recursive functions (say, a factorial), too loose to ensure good throughput with expensive recursive functions, or both. A function that works fine in a test environment might fail in production because the threshold there is lower, or even because the end-user-provided data gives rise to more recursive calls.
What's perhaps missing in the RFC is an exposition on the rationale behind this design decision. In decreasing order of strictness, the goal could be

to ensure that every event is processed in a specified time frame,
to ensure that every event is either processed, or dropped due to timeout, in a specified time frame,
to ensure that every event is processed in a specified number of steps,
to ensure that every event is either processed, or dropped due to timeout, in a specified number of steps,
to ensure that each function is either processed, or dropped due to timeout, in a specified number of steps, or
to ensure that each function terminates for all its inputs (either statically or by, e.g. run-time size-change termination).

As far as I understand the RFC and its implementation, it currently targets the fifth option. (Is my assessment correct, and) does this behavior align with the expectations of your users?
Summary of Comments

Specifying the preprocessor in this way burdens future evolution paths for little gain right now.
The RFC leaves the question unanswered whether Tremor Script functions are envisioned to be and remain purely functional.
The description on partial functions might be written in a confusing way, or I might have to turn in my CS degree...
The order in which functions are defined matters in the implementation, which is suboptimal but fine.
The recursion depth is arbitrary, but the RFC also does not state what exactly the goal is.

Thanks for the awesome work putting out this RFC and its implementation.

Heya thank you so much for the feedback :) I'll go over it fully tomorrow just a quickly reply here:

Specifying the preprocessor in this way burdens future evolution paths for little gain right now.

Good point! we should probably point out that this is an implementation detail and not to be dependent on!

The RFC leaves the question unanswered whether Tremor Script functions are envisioned to be and remain purely functional.

The description on partial functions might be written in a confusing way, or I might have to turn in my CS degree...

You are in luck you don't have to return your degree, this is confusing we need to rephrase that!

The order in which functions are defined matters in the implementation, which is suboptimal but fine.

That's partially based on the implementation we picked but it has the upside that it naturally protects against multi-function recursion that would potentially break the enforced recursion limit and tail-call recursion only. It's a good point that it might feel unnatural to users, I think since both Darach and I have a good bit of erlang background this felt 'natural'  to us. That all said this restriction does significantly simply the logic and code, it's a good question however if that's the right tradeoff and it at least should be discussed/spelled out in the RFC!

The recursion depth is arbitrary, but the RFC also does not state what exactly the goal is.

It is, we should make this at least configurable via a parameter to the runtime instead of hard coding it. An alternative would be a #!config directive that defines it per script?


First version of Turbina

Status

(WIP, Ready, Needs Review)

Additional Notes
This plugin will make available turbo streams. When all looks good i will make the documentation and tests.

I have a feeling this could be heavily simplified. In fact I've been curious about simplifying the cableready plugin as well but havent had time.
My initial thoughts are it feels weird to introduce a new data attribute for turbo instead of users just doing: data-turbo="true"
the other thing is I think we could just change default mimetypes for a plugin. I think the isTurboResponse and auto processing it is awesome.

I am confused as to the use case for this mechanism. Could you walk through a code path or scenario for why this would be useful?
TIA!

I have a feeling this could be heavily simplified. In fact I've been curious about simplifying the cableready plugin as well but havent had time.
My initial thoughts are it feels weird to introduce a new data attribute for turbo instead of users just doing: data-turbo="true"
the other thing is I think we could just change default mimetypes for a plugin. I think the isTurboResponse and auto processing it is awesome.

I think this too, but do you think its fine for a first version?
I didn't use data-turbo="true" because the documentation warns not to use it. What are the implications of having it?
I thought about it too, but i just follow the pattern of the cable ready. What i did before was just add turbo-streams headers on headers.ts

I appreciate your enthusiasm, but "fine for a first version" isn't going to cut it for actually shipping... although perhaps I am misunderstanding your qualifier.
A video would be super helpful!
Let me better explain: I don't fully understand why you need Turbo Stream support in mrujs, since the way TS works is via custom web components that don't care how they were added to the page. Can't you just return HTML normally that just happens to include TS elements? It doesn't seem like it's something mrujs needs to "support" since it should already just work.
We really care a lot about the API surface for this and I know that we're both pretty confused by the big picture idea of this. It's not the "how" so much as the "why".

@leastbad it is almost the case of CableCar, to facilitate the thigs, if i want to use in 10 form, then dont need to work with callbacks and other things, Turbina already handles everything.
Video: https://www.youtube.com/watch?v=Vu-eL30tnxU

I need you to answer my question. Turbo Stream is just web components in an HTML fragment. Why does mrujs need to do anything other than just update HTML?
CableReady/CableCar is JSON. It needs to be interpreted. That's the difference!

But I think I already answered you, i see this plugin more like a helper, instead of doing this by yourself, this plugin do it for you.


Question about the dimension of head

Hi, I notice that right now the dimension of the head is fixed as 32 because of the constraint of the Cuda kernel. I wonder what if I change the dimension of the head to 64 since that figure in some codebases is set as 64.

Hello,
We fixed the dimension for our purposes to get a minor speed improvement. You can modify the line #define DIM 32 and change it to 64 and recompile if you're interested in doing that.
We plan to release either a separate version of the kernel with dynamic dims, or merge that into the kernel directly.

Hi, I want to double check that no matter the value of dim and num_heads, the dim of head is always 32?

Hi,
No, we specifically kept the per-head dim at 32 for our 4 variants, and extended heads for larger variants. That's why we kept it fixed in the kernel.

Just an update,
You can now use arbitrary dims per head with v0.11.
@PeiqinZhuang If that resolves your question, feel free to close the issue.

Sorry, to what exactly are you referring by block size?

Closing this due to inactivity. If you still have questions feel free to open it back up.


Cannot tell difference between public and private channels

Imho, matterircd should indicate the "private" status of a channel by faking channel modes +i and +s.

Good idea


Update puppeteer to latest major version

The chrome-86 tag we are using is very old at this point and does not support arm-based Macs. Updating to the latest major version to get a handful of fixes and types directly in the package.

I'm looking into the WebGL compatibility test failure.

@calebmshafer @wgoehrig These tests assume the browser is using software rendering.
With new version, chrome is using GPU instead of software rendering on Windows.
I tried adding (among other flags), "--disable-gpu" to certa.json's chromeOptions.args. It still uses GPU.
If I launch my installed Chrome from cmd line with same option, it uses software rendering (if you close all Chrome windows first).
I tried editing ChromeTestRunner.initialize to always add --disable-gpu to options.args. After clean rebuild it still uses GPU.

@pmconne okay, I'll take a look. Maybe they change the flag for this in puppeteer or don't honor it anymore for some reason...

Failing core-frontend tests.
  1) RenderState
       should have expected initial WebGL context state:

      AssertionError: expected<PHONE_NUMBER> to equal<PHONE_NUMBER>
      + expected - actual

      -4294967295
      +2147483647

      at Context.call (D:\vsts_a\1\s\core\frontend\src\test\render\webgl\RenderState.test.ts:291:30)
      at callFn (@/lib/runnable.js:366:21)
      at Test.run (@/lib/runnable.js:354:5)
      at Runner.runTest (@/lib/runner.js:677:10)
      at fn (@/lib/runner.js:801:12)
      at next (@/lib/runner.js:594:14)
      at fn (@/lib/runner.js:604:7)
      at next (@/lib/runner.js:486:14)
      at http://localhost:3001/@/lib/runner.js:572:5
      at timeslice (@/browser-entry.js:100:5)

  2) Techniques
       WebGL 1
         should compile all shader programs:
     Error: Timeout of 95000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.
      at createTimeoutError (@/lib/errors.js:498:15)
      at Test._timeoutError (@/lib/runnable.js:431:10)
      at done (@/lib/runnable.js:308:18)
      at http://localhost:3001/@/lib/runnable.js:371:11

  3) Techniques
       WebGL 1
         should compile all shader programs without MRT:
     Error: Timeout of 95000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.
      at createTimeoutError (@/lib/errors.js:498:15)
      at Test._timeoutError (@/lib/runnable.js:431:10)
      at http://localhost:3001/@/lib/runnable.js:246:24

  4) Techniques
       WebGL 1
         should compile all shader programs without logarithmic depth:
     TypeError: Cannot read property 'capabilities' of undefined
      at Context.call (D:\vsts_a\1\s\core\frontend\src\test\render\webgl\Technique.test.ts:127:31)
      at callFn (@/lib/runnable.js:366:21)
      at Test.run (@/lib/runnable.js:354:5)
      at Runner.runTest (@/lib/runner.js:677:10)
      at fn (@/lib/runner.js:801:12)
      at next (@/lib/runner.js:594:14)
      at fn (@/lib/runner.js:604:7)
      at next (@/lib/runner.js:486:14)
      at http://localhost:3001/@/lib/runner.js:572:5
      at timeslice (@/browser-entry.js:100:5)

  5) Techniques
       WebGL 1
         should successfully compile surface shader with clipping planes:
     TypeError: Cannot read property 'techniques' of undefined
      at Context.call (D:\vsts_a\1\s\core\frontend\src\test\render\webgl\Technique.test.ts:137:38)
      at callFn (@/lib/runnable.js:366:21)
      at Test.run (@/lib/runnable.js:354:5)
      at Runner.runTest (@/lib/runner.js:677:10)
      at fn (@/lib/runner.js:801:12)
      at next (@/lib/runner.js:594:14)
      at fn (@/lib/runner.js:604:7)
      at next (@/lib/runner.js:486:14)
      at http://localhost:3001/@/lib/runner.js:572:5
      at timeslice (@/browser-entry.js:100:5)

  6) Techniques
       WebGL 1
         should produce exception on syntax error:
     TypeError: Cannot read property 'capabilities' of undefined
      at new ShaderBuilder (D:\vsts_a\1\s\core\frontend\src\render\webgl\ShaderBuilder.ts:501:25)
      at new VertexShaderBuilder (D:\vsts_a\1\s\core\frontend\src\render\webgl\ShaderBuilder.ts:714:5)
      at new ProgramBuilder (D:\vsts_a\1\s\core\frontend\src\render\webgl\ShaderBuilder.ts:1142:17)
      at createPurpleQuadBuilder (D:\vsts_a\1\s\core\frontend\src\test\render\webgl\Technique.test.ts:20:19)
      at Context.call (D:\vsts_a\1\s\core\frontend\src\test\render\webgl\Technique.test.ts:143:25)
      at callFn (@/lib/runnable.js:366:21)
      at Test.run (@/lib/runnable.js:354:5)
      at Runner.runTest (@/lib/runner.js:677:10)
      at fn (@/lib/runner.js:801:12)
      at next (@/lib/runner.js:594:14)

  7) Techniques
       WebGL 1
         should produce exception on unused uniform:
     TypeError: Cannot read property 'capabilities' of undefined
      at new ShaderBuilder (D:\vsts_a\1\s\core\frontend\src\render\webgl\ShaderBuilder.ts:501:25)
      at new VertexShaderBuilder (D:\vsts_a\1\s\core\frontend\src\render\webgl\ShaderBuilder.ts:714:5)
      at new ProgramBuilder (D:\vsts_a\1\s\core\frontend\src\render\webgl\ShaderBuilder.ts:1142:17)
      at createPurpleQuadBuilder (D:\vsts_a\1\s\core\frontend\src\test\render\webgl\Technique.test.ts:20:19)
      at Context.call (D:\vsts_a\1\s\core\frontend\src\test\render\webgl\Technique.test.ts:166:25)
      at callFn (@/lib/runnable.js:366:21)
      at Test.run (@/lib/runnable.js:354:5)
      at Runner.runTest (@/lib/runner.js:677:10)
      at fn (@/lib/runner.js:801:12)
      at next (@/lib/runner.js:594:14)

  8) Techniques
       WebGL 1
         Number of varying vectors
           does not exceed minimum guaranteed:
     TypeError: Cannot read property 'context' of undefined
      at Context.call (D:\vsts_a\1\s\core\frontend\src\test\render\webgl\Technique.test.ts:207:45)
      at callFn (@/lib/runnable.js:366:21)
      at Test.run (@/lib/runnable.js:354:5)
      at Runner.runTest (@/lib/runner.js:677:10)
      at fn (@/lib/runner.js:801:12)
      at next (@/lib/runner.js:594:14)
      at fn (@/lib/runner.js:604:7)
      at next (@/lib/runner.js:486:14)
      at http://localhost:3001/@/lib/runner.js:572:5
      at timeslice (@/browser-entry.js:100:5)

  9) Techniques
       WebGL 2
         "before all" hook for "should produce a simple dynamic rendering technique":
     Error: Class BisCore:Entity is already registered. Make sure static schemaName and className members are correct on class EntityState
      at Function.registerEntityState (D:\vsts_a\1\s\core\frontend\src\IModelApp.ts:309:13)
      at Function.startup (D:\vsts_a\1\s\core\frontend\src\IModelApp.ts:371:10)
      at Context.<anonymous> (D:\vsts_a\1\s\core\frontend\src\test\render\webgl\Technique.test.ts:57:9)

Failing core-full-stack-tests test.
  1) RenderTarget
       should clip using a single plane:

      AssertionError: expected 2 to be at least 3
      + expected - actual

      -2
      +3

      at D:\vsts_a\1\s\full-stack-tests\core\src\frontend\standalone\RenderTarget.test.ts:692:29
      at testOnScreenViewport (D:\vsts_a\1\s\full-stack-tests\core\src\frontend\TestViewport.ts:336:5)
      at testViewports (D:\vsts_a\1\s\full-stack-tests\core\src\frontend\TestViewport.ts:360:3)
      at testViewportsWithDpr (D:\vsts_a\1\s\full-stack-tests\core\src\frontend\TestViewport.ts:368:5)
      at Context.<anonymous> (D:\vsts_a\1\s\full-stack-tests\core\src\frontend\standalone\RenderTarget.test.ts:668:5)

closing in favor of #4017


Indicator for missing site access

In managed browswers, it is possible that certain sites are blocked for extension access. Currently, DemoMonkey does not explicitly indicate that that is the apparent reason why no DemoMonkey commands come through.
It would be good if, even though the current URL is included in an activated config and there is no access to the site, the user gets warned about that missing access so the user can expect DemoMonkey not to be working (i.e. there is a problem).
Background info
Site access properly configured:

Site access is working:

Site where access is not working:

You get a small red ! in the dev-release now, if the page access is not given.

This is part of the stable release already.


Idea for flag to make it easier to try SPAs in reactor

Alongside --port, I propose adding --entry (or --entrypoint) to the list of flags for elm reactor. A typical usage would look like elm reactor --entry=src/Main.elm, which would cause src/Main.elm to be mounted at / (more specifically, http://localhost:<port>/).
Why is this useful? Because if you use Browser.Navigation to change urls, refreshing the page should work rather than render a 404.

There is a pull request for (almost) this in #1814


Error occurred during Apollo Server startup

Hey everyone. I am trying to follow the Getting Started guide from the read me but it keeps throwing the following error when I try to test the API endpoint in Postman:

An error occurred during Apollo Server startup. All GraphQL requests will now fail. The startup error was: module.require is not a function

Code to reproduce
import { startServerAndCreateNextHandler } from '@as-integrations/next';
import { ApolloServer } from '@apollo/server';
import { gql } from 'graphql-tag';

const resolvers = {
  Query: {
    hello: () => 'world',
  },
};

const typeDefs = gql`
  type Query {
    hello: String
  }
`;

const server = new ApolloServer({
  resolvers,
  typeDefs,
});

const handler = startServerAndCreateNextHandler(server);

export async function GET(request) {
  return handler(request);
}

export async function POST(request) {
  return handler(request);
}

In Postman, I am sending a POST request to http://localhost:3000/api/graphql with the following body:
{hello}

I'm new to Apollo and fairly new to Next.js, so I'm not really sure where the error is coming from. Am I messing up by trying to ping the endpoint from Postman? Is this pointing to an issue deeper in my project? Am I just misunderstanding what I'm supposed to doing here?
Thanks in advance!

Where is your route file located?
Since your URL is /api/graphql I'm guessing you're using an API Route (pages/api/graphql.ts).
If you're using an API Route you should use a default export as shown here.

Sorry, I forgot to include that in my original post. I am using route handlers with the new App directory now that it's been moved to stable. The route file is located at app/api/graphql/route.js. The requests appear to have been reaching the route correctly so I figured that folder structure wasn't the issue, but maybe I'm wrong.

I was getting this error as well; I was also using next dev --turbo but after I removed --turbo the error went away. Not sure if this is your issue, but that was my fix.

@trevsm that solved the issue immediately. Can't believe I didn't think to try that, I guess I was just loving the build time reduction with Turbopack too much 😅. Thanks a bunch!


ADManagedServiceAccount: add TrustedForDelegation setting

This is a boolean value and specifies whether an AD Managed Service Account is trusted for Kerberos Delegation.  Default is $false

Pull Request (PR) description

This Pull Request (PR) fixes the following issues

Task list

[x] Added an entry to the change log under the Unreleased section of the file CHANGELOG.md.
Entry should say what was changed and how that affects users (if applicable), and
reference the issue being resolved (if applicable).
[ ] Resource documentation added/updated in README.md.
[ ] Resource parameter descriptions added/updated in README.md, schema.mof and comment-based
help.
[ ] Comment-based help added/updated.
[ ] Localization strings added/updated in all localization files as appropriate.
[x] Examples appropriately added/updated.
[ ] Unit tests added/updated. See DSC Community Testing Guidelines.
[x] Integration tests added/updated (where possible). See DSC Community Testing Guidelines.
[x] New/changed code adheres to DSC Community Style Guidelines.

This change is

Great work!


Selecting multiple social providers only generates one in code

Package version
"@adonisjs/ally": "^4.0.0",
"@adonisjs/auth": "^8.0.5",
"@adonisjs/core": "^5.1.8",
"@adonisjs/lucid": "^15.0.0",
"@adonisjs/repl": "^3.1.3",

Node.js and npm version
npm --version
7.11.2
node --version
v16.1.0
Sample Code (to reproduce the issue)
$ node ace configure @adonisjs/ally
❯ Select social providers you are planning to use · google, facebook
Description
After installing ally and running configuration, I selected facebook and google (I forget which order) and completed configurations. It only generated .env entries + config entries for google. I had to manually add facebook; looked like a bug.

The issue was just with the facebook one. Fixed