Exceptions aren't propagated from server to client

If the server raises an exception then those exceptions aren't propagated to the client. The normal thrift library automatically handles this by sending a ThriftApplicationException from the server to the client as part of the response message.
How can I propagate exceptions from the server to the client using aiothrift? Any help would be appreciated.

Instead of an application exception on the client-side, I get an IncompleteReadError exception followed by a ConnectionClosedError
Traceback (most recent call last):
File "/home/vaastav/anaconda3/lib/python3.8/site-packages/aiothrift/connection.py", line 119, in execute
result = await self._recv(api)
File "/home/vaastav/anaconda3/lib/python3.8/site-packages/aiothrift/connection.py", line 136, in _recv
fname, mtype, rseqid = await self._iprot.read_message_begin()
File "/home/vaastav/anaconda3/lib/python3.8/site-packages/aiothrift/protocol.py", line 469, in read_message_begin
api, ttype, seqid = await read_message_begin(
File "/home/vaastav/anaconda3/lib/python3.8/site-packages/aiothrift/protocol.py", line 159, in read_message_begin
data = await reader.readexactly(4)
File "/home/vaastav/anaconda3/lib/python3.8/asyncio/streams.py", line 721, in readexactly
raise exceptions.IncompleteReadError(incomplete, n)
asyncio.exceptions.IncompleteReadError: 0 bytes read on a total of 4 expected bytes
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "client.py", line 16, in
asyncio.run(create_connection())
File "/home/vaastav/anaconda3/lib/python3.8/asyncio/runners.py", line 43, in run
return loop.run_until_complete(main)
File "/home/vaastav/anaconda3/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "client.py", line 12, in create_connection
print(await conn.add(5, 6))
File "/home/vaastav/anaconda3/lib/python3.8/site-packages/aiothrift/connection.py", line 130, in execute
raise ConnectionClosedError("Server connection has closed") from e
aiothrift.errors.ConnectionClosedError: Server connection has closed

I'm not sure if it's appropriate to raise some kind of ThriftApplicationException. Maybe we could hijack the server exception and manually create a thrift exception and send that back to client, I just wonder if it's the best way to do it.
I think the most simple solution is to define an exception struct in your thrift file and throws that exception in your Service like
service PingPong {
    string ping() throws (1:SampleException error),

}
exception SampleException {
    1: list<string> failed
}


Archive KidneySingleCellAtlas data in ENA

This dataset has metadata archived at BioStudies and BioSamples but is yet to be archived in ENA
The latest spreadsheet is here:
https://drive.google.com/open?id=1vk5e2yPk6plMnoor1JvcLxzxwUbTmTrs
It has a related AUDR ticket here:
https://github.com/HumanCellAtlas/hca-data-wrangling/issues/341
But AUDR issues should have been fixed in the latest spreadsheet before the r-release.
The data is in DCP1 DSS
I am not sure if fastqs have already been converted to bams (@ESapenaVentura ?)
See this issue for previous work https://github.com/HumanCellAtlas/hca-data-wrangling/issues/427

Refer to this https://github.com/HumanCellAtlas/hca-data-wrangling/issues/427 for the current status
tl;dr: Only SequencingRuns are left to upload.

Update: Just ran the command
docker run -v $PWD:/app/output --env INGEST_API_URL=https://api.ingest.archive.data.humancellatlas.org/ --env DSP_API_URL=https://submission.ebi.ac.uk --env AAP_API_URL=https://api.aai.ebi.ac.uk/auth --env ONTOLOGY_API_URL=https://www.ebi.ac.uk/ols --env AAP_API_DOMAIN=subs.team-2 --env AAP_API_USER=<aap_user> --env AAP_API_PASSWORD=<aap_pass> --env VALIDATION_POLL_FOREVER=False --env SUBMISSION_POLL_FOREVER=False quay.io/ebi-ait/ingest-archiver:d2020-07-09.1_CLI --alias_prefix=HCA --ingest_submission_uuid=d5410c6e-612d-421a-a66f-2de5e04dd050 --exclude_types=project,study,samples,sequencingExperiment

Update: Ran the ingest-file-archiver, files are being converted and uploaded.

Submission with the 22 sequencingRuns: DSP submission

Update: 1 file left for upload!

Done https://www.ebi.ac.uk/ena/browser/view/PRJEB37165
AI before closing the ticket:

[ ] Make sure ingest retrieved the accessions successfully
[ ] Make sure the accessions are put back in the metadata spreadsheet

https://ui.ingest.archive.data.humancellatlas.org/submissions/detail?uuid=d5410c6e-612d-421a-a66f-2de5e04dd050&project=abe1a013-af7a-45ed-8c26-f3793c24a1f4
Seems like there is a problem with the accessions. Only the submittables from the latest DSP submission are recorded (20 SequencingRuns), and no accession has been retrieved for them. @aaclan-ebi ?

Sorry, I wasn't able to add pagination in the UI. It only shows the first 20 archive entities.
The sequencing runs weren't updated because the DSP submission processing result states values are 'Error' instead of 'Complete'. I've adjusted the archiver script to still update the archive entities and metadata in Ingest. The files were now updated with sequencing run accessions.
https://api.ingest.archive.data.humancellatlas.org/submissionEnvelopes/5daf405071fe4a0008e325ae/files
The other accessions (project, study, sequencing experiment) can be added by doing an update submission.  The archiver now uses the archive entities to track which metadata to update. Since those metadata were submitted to dsp before the archive entities records were added in Ingest, the archive entities were not created. We can do migration and add those archive entities by inspecting the dsp submission however that is not implemented yet. The easiest and fastest way is to do an update submission.

Thanks for looking into this @aaclan-ebi ! So, just to clarify, next steps are:

Create an update submission uploading the latest spreadsheet (With accessions) to ingest.
Hit submit on the archiving option

Please confirm so I can begin doing this! thanks

Please just do no. 1 for now. Don't hit the submit yet.
Can I get back to you on this? I think we need to do special case handling for DCP submissions which were archived and updated with accessions.

I am going to file issues to investigate the following:

How to export DCP datasets which have updates (new accessions) ? The submissions are already in Complete state.

For DCP and new datasets which were archived prior the archiver support for storing accessions in Ingest. Those have accessions but the accessions may not have been put back to Ingest yet. How should we put back accessions in Ingest?

I've created this issue to be able to put back the accessions in Ingest entities.
ebi-ait/hca-ebi-dev-team#226

There are already issues filled to investigate the steps left so I think this ticket has filled its purpose.
I'm closing the ticket but if you do not agree feel free to open.

With the new board, this is being reopened to reflect the fact the EMBL-EBI accessions from this archiving process need to be added back to the ingest record of this project
This ticket can be closed when said archive accessions are added back to the record so it can then be propagated to the DCP

Update release - VERY LOW PRIORITY, only run accessions are missing and it's a DCP1 dataset (Low value/High cost)

This is the same dataset as #81
Both experiment and run accessions are already present - I think this can be considered done


Maximum Price Limit

Description
Only send an alert if an in-stock card is at or below a defined target price. This will reduce the amount of false-positives from various marketplace sellers listing their cards for unreasonable prices. A few sellers on Amazon seem to be notorious for this.

This is going to be solved by parsing the selectors and getting prices. This will be an overall improvement on speed, price filtering and various other things.
This will be top priority on my workload.
Thanks!

Preferably this would have pricing set on each product with the option to set a price margin in the env so we can customize it.

A PR for a simliar concept has been submitted #173.  Instead of setting prices, it will allow sellers to be banned from the bot.  For example, Sports Authentics is a vendor that keeps triggering the bot.  I agree filtering by price limit is a better overall solution, but this PR should at least eliviate false-positives from Amazon.


[Bug]: Unable to change password again without exiting after changing password

ODC version
odc422
OB version
not have
What happened?

What did you expect to happen?
After completing the password modification, you can modify the password again
How can we reproduce it (as minimally and precisely as possible)?
After modifying the password, do not exit and change the password again
Anything else we need to know?
No response
Cloud
No response

Fixed In Commit 56afee5e5dc7b457f36c8001e436bb370c52a221
------
咏和
Thu Oct 26 17:37:55 2023 +0800

PASS


Props missing inside handle_event() socket.assigns map.

Hello - another good day using Surface here, though there is a rather perplexing case holding me up.
I am making an appt-booking application, and the scheduling logic needs to pair a number of customers demanding services alongside a number of employees who can help. These are the props, service_keys and employee_keys.
I use data possible_by_day to hold open hours for the customer to book, so these can be displayed on a calendar heatmap, and as clickable booking links. I load these from Shop.Schedule.check_blocks/4 in update/2, because the props seem to be missing in mount/1 - this could be a similar problem.
So @possible_by_day is loading properly on initial render, and then I see the case where a user changes the month on the calendar; I need to load new options for booking times. This is done in handle_event("schedule_month", params, socket). Here also, socket.assigns is missing the necessary props, :service_keys and :employee_keys. I mucked around in Context also and had no luck accessing them inside the event.
Checking the IO.inspect responses in the console, on the schedule_month event I see the keys from both params and socket.assigns - the assigns are missing the two props I need to use, although related docs imply they should be there:
["month"]
[:block, :day, :__changed__, :flash, :__context__, :__caller_scope_id__,
 :myself, :possible_by_day]

The pre-release application is running on https://assemble.codes/book -
I pre-recorded some of the user inputs so you can see the schedule component's issue on page load.
Here is the code I use (simplified in this code block).
The commented lines are causing the error.
defmodule ShopWeb.Components.Schedule do
  use Surface.LiveComponent
  import Shop.Schedule

  prop service_keys, :list, default: []
  prop employee_keys, :list, default: []

  data possible_by_day, :map, default: %{}
  data day, :string
  data block, :string, default: nil

  def mount(socket),
    do: {:ok, socket |> assign(:day, today() |> Date.to_string) }

  def update(assigns, socket) do
    {:ok, socket
    |> assign(:possible_by_day, message() |> check_blocks(
      assigns[:service_keys], assigns[:employee_keys], this_month()))
    }
  end

  def handle_event("schedule_day", params, socket) do
    IO.inspect params
    {:noreply, assign(socket, :day, params["day"])}
  end

  def handle_event("schedule_month", params, socket) do
    IO.inspect params |> Map.keys
    IO.inspect socket.assigns |> Map.keys

    {:noreply, socket }
    # {:noreply, socket |> assign(:possible_by_day,
    #   message() |> check_blocks(
    #   socket.assigns[:service_keys],
    #   socket.assigns[:employee_keys],
    #   month(params["month"]))
    # ) }
  end

  def render(assigns) do
    ~F"""

 Jason.encode! } >

Please schedule:

    """
  end
end

As is, I accomplished my goals using a sad hackaround:
  prop service_keys, :list, default: []
  prop employee_keys, :list, default: []

  data possible_by_day, :map, default: %{}
  data services, :list, default: []
  data employees, :list, default: []
  data day, :string
  data block, :string, default: nil

  def mount(socket),
    do: {:ok, socket |> assign(:day, today() |> Date.to_string) }

  def update(assigns, socket) do
    {:ok, socket
    |> assign(:services, assigns[:service_keys])
    |> assign(:employees, assigns[:employee_keys])
    |> assign(:possible_by_day, message() |> check_blocks(
      assigns[:service_keys], assigns[:employee_keys], this_month()))
    }
  end

  def handle_event("schedule_day", params, socket) do
    {:noreply, assign(socket, :day, params["day"])}
  end

  def handle_event("schedule_month", params, socket) do
    {:noreply, socket
    |> assign(:possible_by_day,
      message() |> check_blocks(
        socket.assigns[:services],
        socket.assigns[:employees],
        month("#{params["year"]}-#{params["month"]}"))
    )
    |> push_event("color", %{})
    }
  end

I don't know if assigning your assigns in update is a sad workaround... Generally, you assign(socket, assigns) as per the default callback that you're overriding.

ah, really? that sure is cleaner than doing them each one by one. if this is the price of admission for using the update callback, this makes sense. The Phoenix docs on hexdocs.pm glossed over this side effect, so I'd been unsure why some of my components that had no need for update/2 seemed to be running properly, and this one seemed broken.
I'll see how much I can add to the docs there.


Add FOG, RADIANCE, and IRRADIANCE shader overrides

As discussed with reduz on IRC.
This allows users to implement their own custom fog, radiance, and irradiance by overriding the default behaviour.
Closes https://github.com/godotengine/godot/issues/33616
In short, all 3 built ins just blend between the override and the internal value.
Documentation PR to follow.

Does this PR make it possible to implement https://github.com/godotengine/godot-proposals/issues/411 with a shader?

@Calinou probably! I am a little unclear on exactly what that effect is. but now you can do anything you want!


How to modify the loss to encourage exploration?

I'm trying to apply a3c.py to a different problem. However it quickly converge to the dominant action and never move again. My question is: whether there are any suggestions for modifying parameters or loss function to encourage exploration? Have you tried that in pensieve?  Thanks!

Thanks a lot Hongzi!
I actually read your paper and set ENTROPY_WEIGHT to 7 (should it must be no larger than 5?) But I still feel it does insufficient exploration. I also observed the gradients of critic network are all zero. I changed to leaky_relu, but the critic_gradient is still all zero. Have you experienced that in Pensieve? Any suggestions?
Best Regards,
Jia

That's weird. I don't think we had this issue before. Can you see if the loss of value network goes down?

In my last run, td_loss goes up from 26.xxx to 30.xxx and then drops to 26.xxx. I can rerun it and send a complete log of td_loss tomorrow. Thanks!
Best Regards,
Jia

Problem solved, it is because my neural network architecture is not so reasonable. Now it has been corrected, and I can see obvious exploration and critic gradients are not zero any more.


Guard against SCTPTransport being closed during start

Pass association to readLoop instead of accessing SCTPTransport
inside readloop. It is possible that before the read loop starts
the SCTPTransport assocation could be cleared, this would cause
us to pass a  nil assocation to pion/datachannel
Resolves #571

Codecov Report

Merging #799 into master will increase coverage by 0.03%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #799      +/-   ##
==========================================
+ Coverage   75.82%   75.85%   +0.03%
==========================================
  Files          59       59
  Lines        4220     4217       -3
==========================================
- Hits         3200     3199       -1
+ Misses        751      748       -3
- Partials      269      270       +1

Impacted Files
Coverage Δ

sctptransport.go
73.46% <100%> (-0.54%)
:arrow_down:

dtlstransport.go
68.02% <0%> (+1.16%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 63b2d4d...1aa103c. Read the comment docs.


[RFC] Fusion Models

[RFC] Fusion Models
TLDR

Fused Models are two+ pre-trained models joined together and further tuned to work as one model. This is the approach used for most SOTA Multimodal models currently.
This RFC proposes special modules and utils similar to our PEFT implementation to allow for easy manipulation of these fused model parameters and their state dicts.

WHY: To support a wide variety of multimodal models and multimodal recipes with a consistent single model API.
Context
The majority of SOTA multimodal models coming out these days are not trained end to end as uniform models, but instead a separate model is trained for each modality and then these models are fused together. In the simplest case, you have an LLM trained on text and an image encoder such as CLIP. You then need to combine these pre-trained models and finetune them so that the LLM can understand the embeddings from the image encoder.
There are many different ways to combine them, and a detailed breakdown of these approaches can be found here. At a high level, there are two overarching architectures that most models can be grouped into. EarlyFusion and DeepFusion. EarlyFusion defines a model where a mapping is learned from encoder outputs to token embeddings. Then encoded inputs are converted to tokens and fed into the LLM the same as text tokens. On the other hand, in DeepFusion, the output of the encoder is injected directly into the intermediate layers of the LLM, typically with cross attention, following the same approach for encoder decoder language models.
To work with these models, there are several considerations. If you have two+ pre-trained models, you may want to be able to fuse them and add some additional learnable parameters (we will call fusion parameters). For this you want to be able to define the fused model but be able to load in the individual checkpoints. You also want to be able to determine which parameters/sub-modules to make trainable and which to keep frozen. Apart from directly fusing models, there are a lot of open models (like Llava, PaliGemma, Idefics, Phi Vision, etc) that have already been fused, but you might want to further finetune. In this case you want to be able to load in the full fused checkpoint and further finetuning on the models, either on all the weights or using PeFT techniques. To handle all of this, and keep uniform recipes/utils, it would be ideal if the fused models could behave as a single model but then easily decompose into their individual parts.
TLDR

Multimodal models are primarily DeepFusion and EarlyFusion architectures
Modules/utils for fusing and un-fusing MM models can allow for fine-tuning a variety of MM architectures in a uniform way.

Proposal
High Level List:

Introduce a modules/model_fusion folder for handling fused models
fusion_models introduces two top level wrapper modules that combine a single encoder and decoder in two different ways. This allows the two models to be used as one model.
fusion_layer and fusion_embed are fusion modules (similar to PeFT modules) that introduce extra trainable parameters to the pre-trained fused models
fusion_utils is based on peft_utils and provides functions for accessing and modifying fusion parameters

Folder Structure for new modules/functions
torchtune/
	modules/
		model_fusion/
			fusion_models.py
				DeepFusionModel
				EarlyFusionModel
			fusion_layer.py
			fusion_embed.py
			fusion_utils.py
				register_fusion_module
				get_fusion_params
				set_trainable_params

Pseudo code demonstrating usage of fused models
def flamingo_builder():
	return DeepFusionModel(
		encoder=CLIP(...),
		decoder=Llama3(...))

>>> model = flamingo_builder()

# Load full checkpoints
>>> model.load_state_dict(...)

# Or load pretrained individual models
>>> model.encoder.load_state_dict(...)
>>> model.decoder.load_state_dict(...)

# set only the fusion parameters to be trainable
>>> fusion_params = get_fusion_params(model)
>>> set_trainable_params(model, fusion_params)

# Do inference with the model the same as a text model
# Fusion models have the same signatures as TransformerDeocder
>>> data = flamingo_transform(...) # Message formated text + images
>>> model(**data)
tensor([-1.0885, ..., .0231], grad_fn=<ViewBackward0>)

[!Note]
This RFC proposes a complete design for supporting fusion models, but not all of the design is required to be built to support any one given fusion style model. This RFC gives us an opportunity to take a holistic view over the space, but it is recommended that each module/component is built as need.

Fusion Models
DeepFusion requires that Decoder is already modified to have any additional layers for accepting encoder inputs (these should be wrapped with FusionLayer to be able to track these additional parameters and work with checkpoints). For inference all of the cache can be managed by the decoder, so these methods are simply passed through.
class DeepFusionModel(nn.Module):
	def __init__(self,
		decoder: TransformerDecoder,
		encoder: nn.Module,
		...
	):

	def setup_caches(self, batch_size: int, dtype: torch.dtype) -> None:
        """Setup key value caches for attention calculation.

        Args:
            batch_size (int): batch size for the caches.
            dtype (torch.dtype): dtype for the caches.
        """
        self.decoder.setup_caches(batch_size, dtype)

	def caches_are_enabled(self) -> bool:
        """Check if the key value caches are setup."""
        return self.decoder.caches_are_enabled()

	def reset_caches(self):
        """Reset the key value caches."""
        self.decoder.reset_caches()

	def forward(
		self,
        tokens: Tensor,
        *,
        mask: Optional[Tensor] = None,
        encoder_input: Optional[Tensor] = None,
        encoder_mask: Optional[Tensor] = None,
        input_pos: Optional[Tensor] = None,
    ) -> Tensor:
	if 	encoder_input is not None:
		# during decoding, the output of the encoder
		# is cached in the decoder cross attention kv_cache.
		# so encoder_input is only called when there's a new image
		encoder_embed = self.encoder(**encoder_input)
	# input_pos is tracked by kv_cache now, but left in for
	# now to support sample packing and can be used here
	if input_pos is not None and encoder_mask is not None:
		encoder_mask = encoder_mask[input_pos]
	output = self.decoder(tokens, mask, encoder_embed, encoder_mask, input_pos)
	return output

EarlyFusion is more complicated from the fusion side, but does not require a special Decoder. EarlyFusion needs to call decoder.tok_embeddings directly so it can merge the embeddings with the encoded embeddings. As to not modify the state_dict it employs two hooks to keep tok_embeddings inside the decoder for the state_dict.
class EarlyFusionModel(nn.Module):
	def __init__(self,
		decoder: TransformerDecoder,
		encoder: nn.Module,
		encoder_token: int,
	):
		self.decoder = decoder
		self.encoder = encoder
		self.encoder_token = encoder_token

		...

		# A little surgery in the decoder to give the
		# fusion module access to control the embeddings
		# The alternative is to pass a special tok_embeddings
		# module into TransformerDecoder builder that does the
		# merging there
		self.tok_embeddings = decoder.tok_embeddings
		decoder.tok_embeddings = nn.Identity()

		self.register_state_dict_post_hook(Test._state_dict_hook)
        # TODO: Switch to register_load_state_dict_pre_hook in v2.5
        self._register_load_state_dict_pre_hook(
	        Test._load_state_dict_hook,
	        with_module=True
	    )

	def _state_dict_hook(self, destination, prefix, keep_vars):
		"""
		Keep tok_embeddings inside of decoder state_dict

		[!Note] This update changes the order of the OrderedDict
		"""
		key = "tok_embeddings"
		decoder_key = "decoder.tok_embeddings"
		destination[decoder_key] = destination[key]
		del destination[key]

	def _load_state_dict_hook(self, state_dict, *args, **kwargs):
		""" Undo the change from _state_dict_hook """
		key = "tok_embeddings"
		decoder_key = "decoder.tok_embeddings"
		state_dict[key] = state_dict[decoder_key]
		del state_dict[decoder_key]

	def _merge_embeds(self, embeds, encoder_embeds, tok_mask):
		""" Combine embeds based on token mask """
		...

	def setup_caches(self, batch_size: int, dtype: torch.dtype) -> None:
        """Setup key value caches for attention calculation.

        Args:
            batch_size (int): batch size for the caches.
            dtype (torch.dtype): dtype for the caches.
        """
        self.decoder.setup_caches(batch_size, dtype)

	def caches_are_enabled(self) -> bool:
        """Check if the key value caches are setup."""
        return self.decoder.caches_are_enabled()

	def reset_caches(self):
        """Reset the key value caches."""
        self.decoder.reset_caches()

	def forward(
		self,
        tokens: Tensor,
        *,
        mask: Optional[Tensor] = None,
        encoder_input: Optional[Tensor] = None,
        input_pos: Optional[Tensor] = None,
        **kwargs, # no need for encoder_mask
    ) -> Tensor:

    if 	encoder_input is not None:
		encoder_embeds = self.encoder(**encoder_input)

	tok_mask = tok != self.encoder_token
	embeds = self.tok_embeddings(tokens[tok_mask])
	embeds = self._merge_embeds(embeds, encoder_embeds, tok_mask)

	output = self.decoder(tokens, mask, input_pos)
	return output

Fusion Layer
This module's primary use case is for DeepFusion modules where you need to add new layers within the LLM decoder. Instead of adding additional layers directly, which would change layer counts and break checkpoint keys for the text model, FusionLayer fuses the new layer to an existing layer that it modifies. This fused layer is now treated as a single layer. The new parameters are returned with fusion_params. Two state dict hooks are used here so that the original layer doesn't have it's name modified in the state_dict.

class FusionLayer(nn.Module):
    """Fusion layer as introduced in `Flamingo: a Visual Language Model for
    Few-Shot Learning <https://arxiv.org/abs/2204.14198>`_.

    Deep Fusion model architectures combine pretrained encoder models with
    pretrained language models by infusing the encoder outputs into the middle
    layers of the LLM. This allow the language model to interpret the encoder
    outputs as text, to "understand" different modalities that you can train an
    encoder for. To enable the language model to adapt to the encoder outputs,
    the FusionLayer inserts a new learnable layer between the decoder (language
    model) layers to learn to combine the encoder outputs and decoder
    activations. The fusion layer can be inserted before or after the decoder
    layer, in Flamingo they insert before.

    Args:
        layer (nn.Module): original decoder layer
        fusion_layer (nn.Module): new fusion layer
        fusion_first (bool): boolean to insert fusion layer before or after
	        the decoder layer.
    """

    def __init__(
        self,
        layer: nn.Module,
        fusion_layer: nn.Module,
        fusion_first: bool = True
    ):
        super().__init__()
        self.layer = layer
        self.fusion_layer = fusion_layer
        self.fusion_first = fusion_first
        self.register_state_dict_post_hook(Test._state_dict_hook)
        # TODO: Switch to register_load_state_dict_pre_hook in v2.5
        self._register_load_state_dict_pre_hook(
	        Test._load_state_dict_hook,
	        with_module=True
	    )

    def _state_dict_hook(self, destination, prefix, keep_vars):
	    """
		Remove "layer" from the original layer in the state_dict
		name. This keeps the orginal state dict name for the layer
		from before fusing with the second layer.

		[!Note] This update changes the order of the OrderedDict
		"""
        keys = list(destination.keys())
        for key in keys:
            if key.startswith("layer"):
                new_key = key.replace("layer.", "")
                destination[new_key] = destination[key]
                del destination[key]

    def _load_state_dict_hook(self, state_dict, *args, **kwargs):
	    """ Undo the change from _state_dict_hook """
        keys = list(state_dict.keys())
        for key in keys:
            if not key.startswith("fusion_layer"):
                new_key = "layer." + key
                state_dict[new_key] = state_dict[key]
                del state_dict[key]

	def setup_cache(self, batch_size: int, dtype: torch.dtype) -> None:
		self.layer.setup_cache(batch_size, dtype)
		self.fusion_layer.setup_cache(batch_size, dtype)

    def cache_enabled(self) -> bool:
        return self.layer.cache_enabled()

    def reset_cache(self):
        self.layer.reset_cache()
        self.fusion_layer.reset_cache()

    def fusion_params(self) -> List[str]:
        """
        Return parameters of fusion layer.
        """
        fusion_params = [
            f"fusion_layer.{k}" for k, v in self.fusion_layer.named_parameters()
        ]
        return fusion_params

    def forward(self, x: Tensor, **kwargs) -> Tensor:
        """
        Args:
            x (Tensor): input tensor with shape
                [batch_size x seq_length x embed_dim]

        Returns:
            Tensor: output tensor with same shape as input
                [batch_size x seq_length x embed_dim]`

        """
        if self.fusion_first:
            x = self.fusion_layer(x, **kwargs)
            x = self.layer(x, **kwargs)
        else:
            x = self.layer(x, **kwargs)
            x = self.fusion_layer(x, **kwargs)
        return x

Fusion Embedding
The reason for this layer is summarized in the docstring.
class FusionEmbedding(nn.Module):
    def __init__(self,
	    vocab_size: int,
	    additional_tokens: int,
	    embed_dim: int
	) -> None:
        """Fusion embedding traines additional special tokens while
        keeping the original embedding frozen. When fusing new models with a
        language model, there may be some additional tokens needed to support
        the fused language model. The FusionEmbedding keeps the original
        embedding frozen while learning a much smaller second embedding for the
        additional tokens. During forward the module routes the tokens to the
        appropriate embedding table.

        Args:
            vocab_size (int): language model vocab size
            additional_tokens (int)): additional tokens for the fused model
            embed_dim (int): embedding dimension of the two embedding tables
        """
        super().__init__()
        self.embedding = nn.Embedding(vocab_size, embed_dim)
        self.fusion_embedding = nn.Embedding(additional_tokens, embed_dim)
        self.dim = embed_dim
        # TODO: Support merging the embeddings after finetuning

    def fusion_params(self) -> List[str]:
        """
        Return fusion embedding parameters.
        """
        fusion_params = ["fusion_embedding.weight"]
        return fusion_params

    def _fused_embed(self, bs, seq_len):
        """
        Return an empty tensor the shape of the combined embedding.
        """
        device = self.embedding.weight.device
        dtype = self.embedding.weight.dtype
        return torch.empty(bs, seq_len, self.dim, device=device, dtype=dtype)

    def forward(self, input: Tensor) -> Tensor:
        """
        Args:
            input (Tensor): input integer tensor with shape
                [batch_size x seq_length]

        Returns:
            Tensor: output tensor embedding with shape
                [batch_size x seq_length x embed_dim]`

        """
        bs, seq_len = input.size()
        vocab_size = self.embedding.num_embeddings

        mask = input < vocab_size
        tokens = torch.masked_select(input, mask)
        additional_tokens = torch.masked_select(input, ~mask) - vocab_size

        embeds = self.embedding(tokens)
        additional_embeds = self.fusion_embedding(additional_tokens)

        out = self._fused_embed(bs, seq_len)
        mask = mask.unsqueeze(-1).expand(bs, seq_len, self.dim)
        out.masked_scatter_(mask, embeds)
        out.masked_scatter_(~mask, additional_embeds)
        return out

Fusion Utils
These are mostly pulled directly from peft_utils as the process of getting special marked parameters and freezing/unfreezing them and handling the checkpointing is very similar to peft. These are kept separate to peft though as it's likely that multimodal models would need to be combined with peft recipes and we need to control peft and fusion parameters separately.
def get_fusion_params(model: nn.Module) -> Dict[str, nn.Parameter]:
    """
    Return the subset of parameters from a model that correspond to fused
    modules. Assumes that any fusion class has defined the
    :func:`~torchtune.modules.model_fusion.FusionLayer.fusion_params` method.

    Args:
        model (nn.Module): Instance of model class containing some
        fusion params.

    Returns:
        Dict[str, nn.Parameter]: the subset of model's state dict containing
        only adapter parameters.

    """
    fusion_params = {}
    for k, v in model.named_modules():
        if hasattr(v, "fusion_params") and callable(v.fusion_params):
            current_fusion_params = v.fusion_params()
            for n, p in v.named_parameters(recurse=True):
                if n in current_fusion_params:
                    full_key = f"{k}.{n}" if k else n
                    fusion_params.update({full_key: p})
                    current_fusion_params.remove(n)
            assert (
                current_fusion_params == []
            ), f"Fusion params {current_adapter_params} not converted"
    return fusion_params

def set_trainable_params(
		model: nn.Module, fusion_params: Dict[str, Any]
	) -> None:
    """
    Set trainable parameters for an nn.Module based on a state dict of fusion
    parameters.

    Args:
        model (nn.Module): Instance of model class containing some adapter
	        params.
		fusion_params (Dict[str, Any]): State dict mapping adapter key names to
			their respective nn.Parameters (i.e. outputs of
			:func:`~torchtune.modules.model_fusion.get_fusion_params`.)

    Returns:
        None
    """
    for k, v in model.named_parameters():
        v.requires_grad_(k in fusion_params)

Additional function to convert any module into a fusion module. A primary use case would be for modifying the original encoder with additional layers to learn a mapping from the original encoder to the LLM space. These layers or defined as a full module, would be marked a fusion_module so their parameters could be controlled independently from the pre-trained encoder.
def regsiter_fusion_module(module):
	""" Add the method fusion_params to to an nn.Module
		to mark them as fusion params
	"""
	def fusion_params(self) -> List[str]:
        """
        Return parameters of fusion layer.
        """
        return list(self.named_parameters().keys())
    module.fusion_params = fusion_params

Discussion

DeepFusionModel could just be called EncoderDecoderTransformer but that doesn't cast it in the context of fusing pre-trained models
This design was intended to be robust over a large set of MM fusion designs, but it's a fast moving space and it's possible it's overfit to the current state of things. For example, there hasn't been any thought given to "MM in + MM out" models. Though I believe that the design makes minimal assumptions as to give us flexibility going forward. For example, the fusion_models all assume one encoder now for simplicity but can be updated pretty easily to support multiple encoders.
These concepts are primarily meant to be exposed at the model builder stage and remain hidden at the recipe level. The only area they get exposed would be in checkpoint logic if we introduce recipes stitching together multiple pretrained models
The name itself directly describes this architecture type, but is not a term that I'm aware of being used anywhere for these models.

done by #1338

opening to keep active while we explore EarlyFusion


Block Issues and PR for non members

Sorry guys
ref: #40

Great idea except that GitHub has no way to do that ☹


LocalBaseline: On change of x-axis unit, preserve true value of plugin.x

This is not needed right now.
To further clarify, this wasn't part of the original delivery of LocalBaseline.
The idea was that if x was on a peak at 400px and the user switches to wavelengths then the selection would stay at the peak (instead of jumping to 400 cm-1). While it's a neat idea, and it would make sense to be a general part of plugins with x-value selectors (Worek, LocalBaseline), I know for a fact that switching the x-axis mid-measurement is not a part of the workflow of our users of LocalBaseline.


Issue #226: init ndstf array elements

The following tests have been validated on Bionic for the fix of Issue: #226

[x] nosetests --with-doctest
[x] nosetests ftests/gasplus/*
[x] solvcon/sandbox/gas/tube/go run visualization

I did not bump the build system of Travis CI to be bionic for the validation of this fix on Xenial over Travis CI. If we are ok with the Bionic update, we could move forward to bionic anytime later.

x

According to the discussion of Issue #226 , we will deprecate the warning for this error message instead.

I am seeing GitHub action failing (https://github.com/solvcon/solvcon/runs/524118539?check_suite_focus=true#step:7:858).  Although the CI has not been fully configured, I'd like to merge this PR so that GHA may pass.
The initialization may incur additional runtime, but it shouldn't be too bad and the code is legacy anyway.


data.graphql returning undefined in getUserByUsername function

5 minutes ago I started noticing an odd error that stated that it cannot read property "user" of undefined, when I looked into it, it was a malfunction of the getUserByUsername function that caused data.graphql to return undefined.

Upon trying to use the getPhotosByUsername function I receive this error which is connected to the getUseByUsername function
UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'user' of undefined
    at C:\Users\epicd\Desktop\typescript discord bot\node_modules\instagram-web-api\lib\index.js:152:34
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async Instagram.getPhotosByUsername (C:\Users\epicd\Desktop\typescript discord bot\node_modules\instagram-web-api\lib\index.js:215:18)

Upon trying to use the getPhotosByUsername function I receive this error which is connected to the getUseByUsername function
UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'user' of undefined
    at C:\Users\epicd\Desktop\typescript discord bot\node_modules\instagram-web-api\lib\index.js:152:34
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async Instagram.getPhotosByUsername (C:\Users\epicd\Desktop\typescript discord bot\node_modules\instagram-web-api\lib\index.js:215:18)

Can you provide your code please?

Can you provide your code please?

This is the code used
        const instaClient = new Instagram({ instaName, instaPassword })
        await instaClient.login()

        setInterval(async () => {
            const cookies = JSON.parse(readFileSync('./cookies.json', 'utf8'))

            const posts = await instaClient.getPhotosByUsername({ username: 'rulesfortheboys' })

            if (posts.user.edge_owner_to_timeline_media.count > cookies.lastPostCount) {
                cookies.lastPostCount++;

                const shortcode = posts.user.edge_owner_to_timeline_media.edges[0].node.shortcode
                const link = `https://www.instagram.com/p/${shortcode}`

                const channel = await client.channels.fetch('794304862997315584') as TextChannel;
                channel.send(link)

                writeFile('./cookies.json', JSON.stringify(cookies, null, 4), (err) => {
                    if (err) console.log(err);
                })
            }
        }, 5000)

This is the code used
        const instaClient = new Instagram({ instaName, instaPassword })
        await instaClient.login()

        setInterval(async () => {
            const cookies = JSON.parse(readFileSync('./cookies.json', 'utf8'))

            const posts = await instaClient.getPhotosByUsername({ username: 'rulesfortheboys' })

            if (posts.user.edge_owner_to_timeline_media.count > cookies.lastPostCount) {
                cookies.lastPostCount++;

                const shortcode = posts.user.edge_owner_to_timeline_media.edges[0].node.shortcode
                const link = `https://www.instagram.com/p/${shortcode}`

                const channel = await client.channels.fetch('794304862997315584') as TextChannel;
                channel.send(link)

                writeFile('./cookies.json', JSON.stringify(cookies, null, 4), (err) => {
                    if (err) console.log(err);
                })
            }
        }, 5000)

Sorry about that, it seems like it was an interference between TypeScript's import and JavaScript's require which caused the code to malfunction.

Sorry about that, it seems like it was an interference between TypeScript's import and JavaScript's require which caused the code to malfunction.

Upon closer research, I have seen that it redirects me to the Instagram Login page, even if I login into the account first.

Upon closer research, I have seen that it redirects me to the Instagram Login page, even if I login into the account first.

@CyberShuri You can use the sendChallenge and then the verifyChallenge method to fix this

@CyberShuri You can use the sendChallenge and then the verifyChallenge method to fix this

Thank you, rmbh. Issue closed

Thank you, rmbh. Issue closed

Please can you elaborate on sendChallenge and verifyChallenge? I have this issue popping randomly

Please can you elaborate on sendChallenge and verifyChallenge? I have this issue popping randomly

did you fix ?


Extract remove_worker logic from MiqServer

https://github.com/ManageIQ/manageiq/issues/21716
This is focused on the "killing" of our MiqServer#miq_worker and our in memory store of them.
simplify the way that we traverse the various workers that we are going to stop/kill/quiesce.
CHANGE: cut down the scope of this to just focus around the miq_worker
Changes in the code
We had 2 code patterns for cleaning out the lists:

worker.destroy, miq_workers.reload
miq_workers.delete(workers)

If there is a reason to use one of the other, please share

I know it's still WIP, but there is too many disparate things going on in this PR that it's confusing to follow.  The smaller refactors like renaming methods (e.g. renaming remove_unknown_workers to kill_unknown_worker_processes) should probably move out to separate PRs otherwise they make it really hard to see what's going on.

push: cops

@Fryguy refactored this WIP PR and cut down the scope a lot.
If you go through the commits, I believe it shows the rename and the refactor. Let me know if you want me to put more commits into there.
I also put the high level goals and the steps into the issue

This refactor is a lot clearer now.  From what I can see, this LGTM.  @jrafanie and/or @agrare Please also review.

:+1: looks good @kbrock

@Fryguy Is this a good size for a PR?
I have at least 3x the amount of these changes but am trying to hold off so it can fit in the mind best as possible

Yes, this is perfect.  It's very clear here with a singular goal of DRYing up this pattern:
    workers.each do |w|
      yield(w)
      worker_delete(w.pid)
      w.destroy
    end

@jrafanie Please also review.

update:

switched from collection.delete to collection.reload. Since we are deleting most of the records anyway, refresh will bring back very few and may pickup blips a little better.
rebased


[no_merge] Test HMAC coverage with all pending PR

Check out this project: https://github.com/7AXEL/Web2APK


Contracts: StoredContractByNameVersioned

To execute a smart contract's entry point one can invoke a versioned contract by name. This is one of 6 ExecutableDeployItem variants.

Just has a look @ the docs for StoredVersionedContractByName in https://docs.casperlabs.io/en/latest/implementation/serialization-standard.html and am seeing non-standard JSON:
StoredVersionedContractByName { name: "lWJWKdZUEudSakJzw1tn", version: Some(1632552656), entry_point: "S1cXRT3E1jyFlWBAIVQ8", args: 9975e6957ea6b07176c7d8471478fb28df9f02a61689ef58234b1a3cffaebf9f303e3ef60ae0d8 }
is version really: version: Some(1632552656)? or is it version: "Some(1632552656)" if it is the former the Java JSON parser will not be able to read the JSON and we'll have to preprocess it. Can you add some real JSON examples to the ticket please.


Fixes URLResolversTransformer so that it transforms urlresolvers

This use-case was failing to transform:
from django.core import serializers, urlresolvers

Until I updated to the BaseFuncRenameTransformer class.

Ah yes that's the same issue as #63...

What I was seeing is that:
from django.core import serializers, urlresolvers

was getting skipped.
If I edited them by hand to be:
from django.core import serializer
from django.core import urlresolvers

Then running my version would produce:
-from django.core import urlresolvers
+from django import urls

Which is what I was expecting.
This is probably part of a more complex issue than a real fix.

I made a fix for this in #347 which is now released in 1.4.0. If you could try it out and confirm that it works, it would be amazing. I appreciate it might be too late and you gone down the manual route by now...

@browniebroke thank you for that. I will try them out this week. I paused work on the project that generated this PR, but I am picking it back up tomorrow.


Add unit tests

Added unit tests for core classes. The unit tests for now are just the start and will add more over time.

Why is there no assertion on the generated motion in terms of their coordinate/PoseWithCell/spline etc.? (to ensure the behaviour doesn't change over new commits).
Same goes for some other similar tests.
When there's no assertion inside the test case it only checks that it doesn't "crash" but not assurance of consistent behaviour.

I have added tests for the various types of motions, their associated splines / poses, and the computation of costs. Each motion type from Forward, Backward, Point-turn and Lateral has unique aspects which are individually tested.

The changes looks good to me


neurite_type filter not available in viewer.draw

In the past there was the possibility to view different parts of the neuron depending on a neurite_type.
How I can do that with the current API?
import neurom
import neurom.viewer
nrn = neurom.load_neuron('test_data/h5/v1/Neuron.h5')

neuron.viewer.draw(nrn, neurite_type=neurom.AXON)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-10-f0e17a29ff38> in <module>()
----> 1 viewer.draw(nrn, neurite_type=neurom.AXON)

/home/zisis/Projects/ARCH_NGV/ngv_venv/local/lib/python2.7/site-packages/neurom/viewer.pyc in draw(obj, mode, **kwargs)
    127
    128     output_path = kwargs.pop('output_path', None)
--> 129     plotter(ax, obj, **kwargs)
    130
    131     if mode != 'dendrogram':

TypeError: plot_neuron() got an unexpected keyword argument 'neurite_type'

Hi, I am getting the same issue, but I actually just want to plot anything but the axons (soma and dendrites) how can I do it?
I am trying with plotly viewer
Thanks,

Hi Orem,
Yes, as you can see, this is a known issue. We even have the pull request for it.
In the mean time you can use what @mgeplf suggested:
from neurom import load_neuron, NeuriteType
from neurom.viewer import draw
from neurom.core.types import tree_type_checker as is_type
nrn = load_neuron('blabla.swc')
axon = list(nm.iter_neurites(nrn, filt=is_type(NeuriteType.axon)))[0]
draw(axon)


Add Delete component

Hi @msaraiva 👋
This PR adds this bulma delete element to this project.
I started to play with bulma & surface so I decided to create some components, this is the first I've ever built - if that's ok for you others will come soon 🙈 .
Please, let me know what can be improved or if this Delete component worth to be added here at all.
Thanks for your work on surface, it looks amazing ✨

Hi @feliperenan!
It looks great!

if that's ok for you others will come soon

Awesome! The more, the better :)
I have a couple of questions:

Why did you need to have both, a link and a button version?

Can you provide one, maybe two examples for the online documentation? They can just mimic the ones in Bulma's page and you don't have to add them to the @moduledoc. You can just copy/paste here as a comment. In the near future, the site's source will be public and authors will be able to add the examples themselves.

Thanks for contributing!

Why did you need to have both, a link and a button version?

I started only with link but after a closer look at bulma documentation in the combinations part I realized that they use the button tag there. That's why I implemented both. But maybe we could keep only the button version 🤔 . Let me know what you think then I will update accordingly.

Can you provide one, maybe two examples for the online documentation? They can just mimic the ones in Bulma's page and you don't have to add them to the @moduledoc. You can just copy/paste here as a comment. In the near future, the site's source will be public and authors will be able to add the examples themselves.

Well, you mean the code example? There it is:
<Delete />
<Delete label="Hello word" size="small" />
<Delete label="Hello word" size="medium" />
<Delete as="button" size="large" />

They will produce the HTML described here: https://bulma.io/documentation/elements/delete/#sizes

I started only with link but after a closer look at bulma documentation in the combinations part I realized that they use the button tag there. That's why I implemented both.

Oh, I see. I missed that in the docs :)

But maybe we could keep only the button version 🤔 . Let me know what you think then I will update accordingly.

Yeah, I'm not sure if there's any practical difference so I think we could start with only one of them. Maybe <a> since it's the most used in Bulma's docs.  Then we can wait until someone needs it as a <button> and see how we should implement it. WDYT?

Agree. So let's start with link and wait for more use cases to introduce the button one 👍

@feliperenan thanks! ❤


[Feature Request] Open folder after downloading the video.

As it stands now YPD doesn't have any feature to let the user open the download directory.
add a setting to automatically open the download directory when YPD finishes downloading the files.
Remember to add this functionality only on the downloading page don't add this on setting page.
There are 2 versions of this feature so let's just make a todo list

[x] Add an option to automatically open destination folder on the downloading page.

[x] add an option to manually open the destination folder when downloading finishes

Done and done, sorry for the delay

add an option to manually open the destination folder when downloading finishes

I agree that it is missing the Open containing folder function for each download in the "Queue" tab. Currently, YouTube Playlist Downloader v1.924 does not have this feature. The only action you can do with a download is to delete it with the green cross on the left side.

add an option to manually open the destination folder when downloading finishes

I agree that it is missing the Open containing folder function for each download in the "Queue" tab. Currently, YouTube Playlist Downloader v1.924 does not have this feature. The only action you can do with a download is to delete it with the green cross on the left side.

It will open the containing folder if you double click the item in the queue. You can click anywhere to trigger it (except on the delete button of course)


fix: Make dark mode look better

Hey, the dark mode snippet just inverts the color, I made a change so that it looks more like a real dark mode filter

Thanks. It does looks much better 🚀


Change TransportClient to REST api in ElasticSearchSink

Instead of modularizing ElasticSearchSink to support various versions of ElasticSearch, we can use bulk REST api as elasticsearch-hadoop is doing. Jest will be good option.

#172


[RESTEASY-2294] Update to MP REST Client 1.3.3

@asoldano once this is merged, I think we're good for a 4.x release from Quarkus side


GitLab signin returns white screen if user has not authorized Mattermost

Summary
GitLab signin returns white screen if user has not authorized Mattermost as follows from web interface

Environment Information

Device Name: Android
OS Version: 7.0
Mattermost App Version: 1.8.0
Mattermost Server Version: 4.7.3

Steps to reproduce

Setup Mattermost with GitLab login
Install app
Try to login to Mattermost with GitLab from user account that has never accessed Mattermost via desktop

Expected behavior
Should prompt user to authorize
Observed behavior
Does not prompt user to authorize

Hi @bbodenmiller,
Thanks for the report 👍
Can you try upgrading to the latest server version of Mattermost to see whether this issue still reproduces for you?

That won't be possible for at least a month. Regardless this is the GitLab interface that the app doesn't handle properly, not seeing how a minor setver upgrade is going to help that.

@bbodenmiller What version of GitLab do you use?

10.6

Would you be able to help gather any logs from around the time the issue occurs, both from Mattermost and GitLab?

Have you recently upgraded to a new server or app version (is this is a new issue or is this the first time you are trying Gitlab sign-in)?

Have you set your Site URL field in System Console > Configuration? https://docs.mattermost.com/administration/config-settings.html#site-url and followed all other documentation https://docs.mattermost.com/deployment/sso-gitlab.html#gitlab-single-sign-on?

I think you're misunderstanding... These will not help in fixing the issue.

N/a
Not a new issue
Yes

The issue is that on the first login of Mattermost via GitLab, GitLab prompts the user to confirm they want to allow Mattermost to access their account. Mattermost mobile app does not know how to handle this. It's like when you login to a website using Facebook... Facebook has to verify with the user they want to allow.

My apologies for misunderstanding, I will have one of our engineers take a look.

Hey @bbodenmiller, I've filed a ticket to look into this further.
https://mattermost.atlassian.net/browse/MM-10573

Hi, I have the problem of the status ok screen, on two android devices: tablet Android 7.0, and S9plus with Android 8.0. The server is mattermost.cern.ch, I cannot access with https, but it works with http. I logon through gitlab (the only option I have) , use the single sign screen, and after signin and status ok, nothing more.
I don't know the version of gitlab, I tried both mattermost version 1.9.2 and the beta one. Can you help me?
thanks
Marina

Hi @passaseo! Thank you for letting us know 👍
We have a ticket open here to investigate this issue: https://mattermost.atlassian.net/browse/MM-10573. We'll let you know updates on its progress.

HI problem solved, together with the server administrator, I had to connect to a different (not the one for desktop) server, using https. thanks


Add the github url in the package.json

The npmjs site doesn't link the packages to our github repo. This needs to be fixed by updating the URL in the package.json

This repository is deprecated.


Fix the bug where the schedule list is not shown after adding a sched…

…ule task so have to type list to show.
Committed by @zachylimwl in Verbena's computer.

Coverage increased (+0.03%) to 53.848% when pulling 4e99179989066f9d3a5716f7702fd527fbf930f1 on Verbena:add-collated-code into a3a81446ca03ed035b080aea820f5464f51b82a7 on CS2103AUG2016-W15-C4:master.

Coverage increased (+0.03%) to 53.848% when pulling 1d164642e14aec12c5b47e8f2aa78976dc537192 on Verbena:add-collated-code into a3a81446ca03ed035b080aea820f5464f51b82a7 on CS2103AUG2016-W15-C4:master.


Custom Icons

Love the simplicity of this!
At risk of asking for more complexity and breaking a core principle of this project, would you consider adding the ability to use arbitrary images as icons? Either by URL or loaded from a local assets folder. Font Awesome is great but I've quickly found myself wanting to embed custom icons (like TP Link's icon for network portals, Traefik's icon for that dashboard, etc., etc.)

Thank you, I will take in consideration the custom icon feature if I will find a way to fit it in the project without compromising the simplicity of it. At the moment I do not think that loading from url/asset folder would be the best fit.

Custom icons are now available in release v1.2.0


Suggestion

Make the hover totem in guicursor module programable. What I mean by programmable is to be able to tell it what order of totems to use. It looks really obvious when an admin does /invsee and sees that your totems are neatly being used from left to right on each row.

@SeedCrackerXEnjoyer Please suggest this on the discord server, this page is for issues.

@ItziSpyder  please close this

Closing this issue as the hover cursor is being queued for next render draw call. Currently, the method I'm using is not efficient if done randomly. Render calls draw in order from left to right, then top to bottom.


feat: warn if statement has no effect

Closes #664
Summary of Changes
Show a warning if a statement has no effect.

:tada: This PR is included in version 0.4.0 :tada:
The release is available on:

npm package (@latest dist-tag)
npm package (@latest dist-tag)
v0.4.0
GitHub release

Your semantic-release bot :package::rocket:


create a component to handle the search bar with debounce

Signed-off-by: Augustin Husson<EMAIL_ADDRESS>

I'm wondering if we want to have this behavior directly in the v2.33. WDYT @juliusv ?

I'm wondering if we want to have this behavior directly in the v2.33.

I'm fine either way. Most pages will be small enough for the lack of debounce to cause a problem, and it's an optional new functionality.
@beorn7 do you already know if you're going to do an rc.1 anyway? Then maybe we could include this.

:+1: from a code perspective

Yes, I think we should have this in 2.33.0. @Nexucis could you change this PR to be on top of the release-2.33 branch?

I changed the based branch @beorn7 !


Two way data binding

Hello,
How can you use Binding(two way data communication) with this library from view model.

Hey @gsipic
that depends. A binding to the underlying ProcessState or LoadableState can be created just like you would with other properties in the view model:
import Processed
import SwiftUI

final class ViewModel: ObservableObject, LoadableSupport {
  @Published var loadable: LoadableState<Int> = .absent
}

struct OuterView: View {
  @StateObject private var viewModel = ViewModel()

  var body: some View {
    InnerView(loadable: $viewModel.loadable)
  }
}

struct InnerView: View {
  @Binding var loadable: LoadableState<Int>

  var body: some View {
    Text("")
  }
}

However, if you need/want to pass down the run/load methods, that's not really possible, by the nature of how this package works in classes. The view model owns these methods, so you'd have to pass down the view model itself (as an @ObservedObject for example).
When used in classes, these loadables and processes can't be complex objects themselves because value observation in SwiftUI would not work anymore (among other problems, like storing and handling the internal Task). That's why it is implemented through these LoadableSupport/ProcessSupport protocols.
With iOS 17 and the new @Observable mechanism, I can probably improve upon this quite a bit in a new version of this package.

In retrospect, I should have probably been more explicit about this current limitation in the README. Sorry about that!

Ok, thank you for response. Can you make data property in loadableState struct to have setter and getter. Because in current situation you need to pass all Loadable state in InnerView but a want only pass data part of loadableState.
Current data don't have setter
public var data: Value? {
    if case .loaded(let data) = self { return data }
    return nil
  }

Don't want
struct InnerView: View {
  @Binding var loadable: LoadableState<Int>

  var body: some View {
    Text("")
  }
}

a want this
struct InnerView: View {
  @Binding var loadable: Int

  var body: some View {
    Text("")
  }
}

This is how a make workaround a have added setter and getter on data. And with that inner view can only get part of wanted data. I don't need to pass all Loadable to InnerView.
extension LoadableState {
    var dataBinding: Value {
        get {
            if case .loaded(let data) = self { return data }
            fatalError()
        }
        set {
            setValue(newValue)
        }
    }
}

The problem is that there is no way to do this correctly, because there is not a possible, single definition for correct in such a case.
@Binding var loadable: Int

Using a non-optional type here makes the assumption that the loadable state can only ever be .loaded(value), which is not true in the general case. Also, if there always is a value, you don't need a loadable in the first place.
Secondly, what would it mean to set the data property to nil? A state of .absent, .loading or .error? All of these would make sense because for any of them, the getter of data would actually return nil.

That said, you might be facing a specific use case where your InnerView is guaranteed to be only shown when the loadable does have a value and you want it to just be able to manipulate what that value is, while always keeping the .loaded state of the loadable.
This is a use case that would have to rely on proper documentation (because its behavior is not exactly intuitive). There's still no single, correct way of doing this.
Would the following work for you?
extension Loadable.Binding {
  var loadedBinding: Binding<Value>? {
    guard let data = wrappedValue.data else {
      return nil
    }

    return Binding<Value> {
      data
    } set: { newValue in
      guard case .loaded = wrappedValue else {
        return // no-op? Not clear what should happen in this case.
      }
      wrappedValue = .loaded(newValue)
    }
  }
}

struct OuterView: View {
  @Loadable<Int> private var loadable

  var body: some View {
    if let binding = $loadable.loadedBinding {
      InnerView(loadable: binding)
    }
  }
}

struct InnerView: View {
  @Binding var loadable: Int

  var body: some View {
    Text(loadable.formatted())
  }
}

To be honest, I'm not entirely sure this would be a good addition to the package. I think it's just not clear enough in its behavior for people to easily understand and use this. But I need to think about this a bit more, or maybe you have some ideas or arguments on how to make it clearer 🙂
In any case, for now you can just use your existing workaround or copy my extension from above into your project and you will have support for this.

Ok, thank you for support about this issue. The library is very good thank you for open sourcing it.

Thank you 🙂
What might be doable is adding a second product to the package that users can import separately 🤔. That product could contain helper extensions for these "rarer" scenarios. This would keep the core functionality/package clean.
Would you be fine with that? Like, I could add a product called ProcessedUtility or so that would contain this.

Hey,
That is a good approach 🙂. I'm fine with that design. Making clean separation between core and extensions.
Thank you!

Awesome!
I'll prepare a new release this weekend then, hopefully 🙂

I might need a few more days, sorry 😅 Got lost in some other projects over the weekend

@SwiftedMind Yes that better naming convention.


Redirection of clicking on the own rankig

When you're browsing the first page (ranks 1-10) and you're clicking on your own profile(rank 145)
should your profile be opened - like every other one in the list, or should the ranking page, that contains your stats (rank 141-150) be opened?
You could then again click on your profile, which now would act like all others and redirect to your profile.

It should redirect to your profile, would be kinda confusing, everything else redirects to the specific profile too


[WIP] Add the horizontal elastic timetable overload checking rule to the cumulative constraint.

This PR adds an implementation of the Horizontal Elastic TimeTable Overload Checker presented in Kameugne et al. 2024 to the Cumulative Constraint.
The propagator is not activated by default and can be enabled by setting the SatParameters.use_horizontal_overload_checking_in_cumulative to true.
For Reviewers
The PR is not ready for merge. Rather, it is meant as a "snapshot" to start discussing the implementation and whether the propagator is a net-positive for OR-Tools.
Ultimately, I plan to keep working on this PR to implement the "IQHE Edge-finder" rule (from the same paper) which dominates the TimeTable Edge-finder filter rule in terms of propagation. That being said, it is not clear whether this will be a net-positive in an LCG solver. The reason is that the explanations tend to be big and not easy to generalize.

Thanks a lot for the PR!
I had a quick look at the CL and added some comments. I'm a new member of the team, so don't be surprised if I misunderstood something during the review.

Hi @vitor1001, thank you for the review and the comments! I tried to address most of them, PTAL.
As mentioned in the PR description, I'd like to keep working on this PR and update the propagator to actually filter the start/end variables. Though, I would need a way to evaluate that the code is actually driving improvements (in terms of #nodes first, then in terms of speed). Is there an c++ example I could use to compare the performance of different versions of the cumulative on a benchmark that matters to the team by any chance?

rcpsp.tar.gz
Here is a set of cp-sat protobufs for rcpsp models (single and multi mode)


Write full specs for WithdrawAndCall feature

Describe the Issue
Perform technical assessment and write full specs for the WithdrawAndCall feature.

The document should provide:

The API of the new component to integrate
Dependencies
A list of tasks to be performed for the completion of the functionality (should be broken down into small tasks)
Time estimation for the completion of the feature
Difficulties to be encountered

Is there a spec of how this would work?
Suppose i have a omnichain dapp on zevm that does a bunch of stuff and the call withdtawAndCall. It triggers a cctx that calls a contract on ethereum. Since its user specified contract, we should expect such call could revert. What happens then?

No longer relevant https://github.com/zeta-chain/node/issues/2402


phpunit5 support

I added PHPUnit 5 support.
The Driver signature is the same as 6.

Just to confirm, has this actually been tested with a project that uses 5 ?

Just to confirm, has this actually been tested with a project that uses 5 ?

Hello, yes I tested it locally (just sent a patch to match PHP 5 class names).
Currently running on my local machine, works perfectly with PHPUnit 5.

If you could just include a change for the readme, I think that's the last mention of versions anywhere in this repo ?

If you could just include a change for the readme, I think that's the last mention of versions anywhere in this repo ?

readme updated - thank you for this amazing coverage tool (and the wrapper) it really saves a lots of time. i'm glad to be able to run with phpunit 5 as well now :)


Add a static defaultInstance to the _StorageClasses.

Start each message with this static instance so there is no heap
allocation until the value is actually mutated. This should be
the biggest win when getting a message field that hasn't been
set as it avoids a heap allocation in that read only case.

I believe this is the last thing on https://github.com/apple/swift-protobuf/issues/457.

There is one potential follow up from a performance pov, but i'm not convinced the numbers will be more than noise, and it comes at the cost of a little more codegen.
Current generation is:
  fileprivate mutating func _uniqueStorage() -> _StorageClass {
    if !isKnownUniquelyReferenced(&_storage) {
      _storage = _StorageClass(copying: _storage)
    }
    return _storage
  }

Since every message starts off with the defaultInstance, the first mutation of every message is a _StorageClass(copying:).  But before this change, we started with an init since we create those heap instances.  So we are trading an init for the copy of those default values.  The copy has the potential to be more expensive because it requires reading from another set of addresses and writing into the new instance.  Likewise, the compiler could be smart, and if all the initial values were zero, it might bzero/etc. the new instance instead of individual assignments during init, where the copy has to be value lookups since they aren't known at compile time.
So we could generate something like:
  fileprivate mutating func _uniqueStorage() -> _StorageClass {
    if !isKnownUniquelyReferenced(&_storage) {
      if _storage === _StorageClass.defaultInstance {
        _storage = _StorageClass()
      } else {
        _storage = _StorageClass(copying: _storage)
     }
    }
    return _storage
  }

Thoughts?

I doubt the compiler can be that smart (the bzero example) because the fields in the storage class are optionals and optionals have their discriminator set to 1 when they aren't set.
It could be worth exploring the perf side effects of that change separately—I don't know off the top of my head what kind of gains it would have.

Regarding @thomasvl 's idea about restructuring _uniqueStorage, I played a while back with the idea of changing that function to not return _storage with the idea that it might be more efficient to assert uniqueness and then have clients just use the _storage variable directly.  I could find no performance impact at all when I measured it, though, so I've not explored it further.


feat: only add response header for sampled traces

This changes the behavior of WithTraceIDResponseHeader to only add the header if the trace was actually sampled. A trace-id for a trace that wasn't sent anywhere doesn't seem particularly useful?
This could be considered a breaking change. Alternatively, we could add an extra option to control that, but that seems a bit overkill IMHO?

Hello @costela,
Do you have any concern or specific needs to not add trace id as headers for non-sample trace?
As @riandyrn mentioned, trace id might be use by underlying middleware whether sampled or not, like #22

hi @riandyrn, @ilhamsyahids
We have a similar use case, where a middleware does some reporting on traces, and the "unused" traceID was throwing us off.
Your suggestion seems totally sufficient for our case as well. However, it doesn't quite fit with the current header customization in WithTraceIDResponseHeader. I just reworked the PR for a first pass, but if you'd like to make the samped header also customizeable, we'd probably need another separate option function (which feels a bit overkill :see_no_evil:).
PTAL :pray:

anyone get a chance to look at the proposed changes? :innocent:

Hi, @costela
Thanks for the update
Can you please also add test for sampled trace? You might need to check the example

Hello, @costela
Thanks for your latest commit. However, we figured out it's better to just let the end user modify their response. So we decided to create https://github.com/riandyrn/otelchi/pull/57.
Would you please give your feedback there? It is basically the general form of this PR.

deprecated by #57

superceded by https://github.com/riandyrn/otelchi/pull/59


Fix and update behavior of updateFilteredPersonList after every command

In ModelManager.java, I believe updateFilteredPersonList(PREDICATE_SHOW_ALL_PERSONS); should be moved to the AddPatientCommand instead, as it is part of a command.
Also, in general, perhaps we could deprecate model.updateFilteredPersonList(Model.PREDICATE_SHOW_ALL_PERSONS);. Instead,

If a patient is added, it is appended to the back of the currently displayed list.
If a patient is edited, the currently displayed person list should be unchanged, except reflecting the changes to the person on the person card. To achieve this behavior, we can simply remove the model.updateFilteredPersonList(PREDICATE_SHOW_ALL_PERSONS); line on EditPatientCommand, AddTaskCommand, EditTaskCommand, and DeleteTaskCommand.
If a patient is deleted, the currently displayed person list should remove that deleted person. The current implementation already has this behavior.

These behaviors are being resolved in the feature-ui branch. Will create PR once it is ready.


README.md: add v0.14.1 and v0.14.2 release, docs links

Changes

Updating the README :wink:
/cc @sbwsg @afrittoli @bobcatfish @pritidesai
Submitter Checklist
These are the criteria that every PR should meet, please check them off as you
review them:

[ ] Includes tests (if functionality changed/added)
[x] Includes docs (if user facing)
[x] Commit messages follow commit message best practices
[x] Release notes block has been filled in or deleted (only if no user facing changes)

See the contribution guide for more details.
Double check this list of stuff that's easy to miss:

If you are adding a new binary/image to the cmd dir, please update
the release Task to build and release this image.

Reviewer Notes
If API changes are included, additive changes must be approved by at least two OWNERS and backwards incompatible changes must be approved by more than 50% of the OWNERS, and they must first be added in a backwards compatible way.
Release Notes
NONE

/kind documentation

/lgtm


only lowercase md files and images

instead of renaming everything in the repo we should rename .md, and image files only

fixed on trunk


Cannot instantiate docker image using tomcat:9.0.65-jdk17-temurin

I am using official tomcat:9.0.65-jdk17-temurin image from https://hub.docker.com/layers/tomcat/library/tomcat/9.0.65-jdk17-temurin/images/sha256-cc64730eb8cc23761987927379cddfb440711e5c15c60ae4012a9ad62bb6fefb?context=explore
and following is my Dockerfile -

FROM tomcat:9.0.65-jdk17-temurin
LABEL APP_NAME="MYAPP"

#Copying server.xml file
COPY my-build/server.xml /usr/local/tomcat/conf
COPY my-build/context.xml /usr/local/tomcat/conf

#Create a directory for SSL certificates
RUN mkdir -p /home/my_secrets

#Copying SSL certificates
COPY my-build/MyCertTrustStore.jks /home/my_secrets

#Copying the WAR file
COPY my-build/libs/my-lib.war /usr/local/tomcat/webapps

ENTRYPOINT ["catalina.sh", "run"]

the deployment is done via Jenkins Pipeline to an RHEL server.
RHEL Server Information:
# uname -a
Linux aadi-myserver-02.w3-969.mds.com 3.10.0-1160.71.1.el7.x86_64 #1 SMP Wed Jun 15 08:55:08 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

# cat /etc/os-release
NAME="Red Hat Enterprise Linux Server"
VERSION="7.9 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.9"
PRETTY_NAME="Red Hat Enterprise Linux"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.9:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.9
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.9"

Docker Information (on RHEL Server):
# docker info
Client:
 Debug Mode: false

Server:
 Containers: 9
  Running: 8
  Paused: 0
  Stopped: 1
 Images: 22
 Server Version: 19.03.14
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: local
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: ea765aba0d05254012b0b9e595e995c09186427f
 runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
 init version: fec3683
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 3.10.0-1160.71.1.el7.x86_64
 Operating System: Red Hat Enterprise Linux
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 15.66GiB
 Name: aadi-myserver-02.w3-969.mds.com
 ID: HIPE:HYBZ:H3AQ:CO6B:VSM6:REO6:AAOD:BYGG:DTUO:2YVW:ZJWT:53WO
 Docker Root Dir: /monitoring/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  <IP_ADDRESS>/8
 Live Restore Enabled: true

Failure while starting Docker container is -
# docker logs dec0151cb928
Cannot find /usr/local/tomcat/bin/setclasspath.sh
This file is needed to run this program
Cannot find /usr/local/tomcat/bin/setclasspath.sh
This file is needed to run this program
Cannot find /usr/local/tomcat/bin/setclasspath.sh
This file is needed to run this program

NOTE:

This is working fine with
tomcat:9.0.58-jdk17-temurin,
tomcat:9.0.58-jre17-temurin and
tomcat:9.0.58-jdk17-openjdk

Surprisingly, above reported issue is not observed when we create an image using tomcat:9.0.65-jdk17-temurin and tomcat:9.0.65-jre17-temurin on Windows and Mac laptops.

Guys,
Before others jump to help, here is an update.
The issue is related to - https://github.com/docker-library/tomcat/issues/269
I have identified that my Windows and Mac laptops have Docker version 20.x whereas the RHEL machine had Docker 19.x
Therefore using a 'tomcat:9.0.58-jre17-temurin-focal' is a workaround for now but the larger goal ahead will be to upgrade Docker versions.
Kindly add comments and suggestions to increase my understanding..

Yeah looks like you're running into https://github.com/docker-library/tomcat/issues/269#issuecomment-1170461034

jammy based images definitely require newer versions of docker and libseccomp which is the issue here.  So either update those packages on the hosts or use the focal based variants. See also https://github.com/tianon/docker-brew-ubuntu-core/issues/236

The non-jammy variants should work fine for you 9.0.65-jdk17-corretto or 9.0.65-jdk17-temurin-focal


Disable comment char # when reading mzTab. Closes #21

Checked and fixes my example here locally.
I did NOT check if the mzTab-M specs somewhere define/allow # as comment char.
In that case, the fix would be a tad more complex ...
Yours, Steffen

Thanks for the fix! The # character is indeed not allowed in mzTab-M as a comment, instead, comment lines need to be started with COM in the first column, followed by arbitrary text in the second column.


Wrong color calucation in HS mode

I am using Echo Plus to connect to HA-Bridge directly. When I change the colors the HUE comes in values 0-65535. So the whole RGB calculation failes. This was already discussed in https://github.com/bwssytems/ha-bridge/issues/1060:

Some more color values,l tested in the Alexa app :
Green : {"hue":21845,"sat":254}
Blue : {"hue":43690,"sat":254}
Yellow : {"hue":10923,"sat":254}
Red : {"hue":0,"sat":254}
Magenta : {"hue":54613,"sat":254}
Cyan : {"hue":32768,"sat":254}
Orange : {"hue":7100,"sat":254}

This is what I got from the logs:
hue/sat request - Hue: 21845, Sat: 254, Bri: 0
09-09-2019 22:04:58.147 | DEBUG | hue/sat request - Hue: 21845, Sat: 254, Bri: 0
09-09-2019 22:04:58.183 | DEBUG | calculate HSL vars - var1: 0.0, var_2: 0.0, h2: 0.8333334, h2 + 1/3: 1.1666634, h2 - 1/3: 0.50000334 | com.bwssystems.HABridge.hue.ColorDecode
09-09-2019 22:04:58.184 | DEBUG | hue_2_rgb vh: 1.1666634 | com.bwssystems.HABridge.hue.ColorDecode
09-09-2019 22:04:58.185 | DEBUG | hue_2_rgb vh: 0.8333334 | com.bwssystems.HABridge.hue.ColorDecode
09-09-2019 22:04:58.185 | DEBUG | hue_2_rgb vh: 0.50000334 | com.bwssystems.HABridge.hue.ColorDecode
09-09-2019 22:04:58.186 | DEBUG | Color change with HSL: Hue: 21845, Sat: 254, Bri: 0. Resulting RGB Values: 0 0 0
or
09-10-2019 00:06:52.812 | DEBUG | hue/sat request - Hue: 43690, Sat: 254, Bri: 77
09-10-2019 00:06:52.830 | DEBUG | calculate HSL vars - var1: 0.0, var_2: 0.6039216, h2: 0.16666675, h2 + 1/3: 0.49999675, h2 - 1/3: -0.16666326 | com.bwssystems.HABridge.hue.ColorDecode
09-10-2019 00:06:52.831 | DEBUG | hue_2_rgb vh: 0.49999675 | com.bwssystems.HABridge.hue.ColorDecode
09-10-2019 00:06:52.832 | DEBUG | hue_2_rgb vh: 0.16666675 | com.bwssystems.HABridge.hue.ColorDecode
09-10-2019 00:06:52.832 | DEBUG | hue_2_rgb vh: -0.16666326 | com.bwssystems.HABridge.hue.ColorDecode
09-10-2019 00:06:52.833 | DEBUG | Color change with HSL: Hue: 43690, Sat: 254, Bri: 77. Resulting RGB Values: 154 154 0
since convertHSLtoRGB() calculates wrong RGB values the conversion RGBtoHSB() also produces wrong values.

No feedback on this yet? This is imho a serious bug as it's impossible to change colors of a Philips hue bulb.

I'll look into it

ok, thank you. What I did was changing color via Echo Plus Alexa and via Alexa app. What I'm going to try is checking out how the Philips App behaves.
From what I got from the internet it seems hue (on Philips Hue) is a value between 0 - 65535. Division by 182.04 results in 'regular' hue values. As you can see the value is send from the Alexa to the HA-Bridge.
I'm using a ioBroker and (Simple API, deCONZ adapter) with deCONZ, last one requires Hue

Will be writing a test case for this and will validate the calculation

Having a very hard time with this conversion as Philips does not publish their calculation.


RHEL-08-020040/41 needs additional configuration.

tmux is not fully implemented after "MEDIUM | RHEL-08-020041 | PATCH | RHEL 8 must ensure session control is automatically started at shell initialization" has been applied.  So, tmux does not get launched on remote sessions.
Expected Behavior
tmux should be running for session control.
Actual Behavior
tmux is not running.
Control(s) Affected
What controls are being affected by the issue: RHEL-08-020041
Possible Solution
Append the line: [ -n "$PS1" -a -z "$TMUX" ] && exec tmux at the end of the /etc/bashrc file to enable session contol for remote sessions.

I've attempted the presented solution above and when a non-elevated user session locked, I still run into the issue where user is unable to access the tty - error Permission Denied .
Now, if I log in as root and lock the TMUX session, I am able to log back in and access TMUX session.
I tried several other things in this venture, by turning off SELINUX, attempted to set permissions of the tty sessions (not sure if that's the right approach) and few other items. In the end, non-elevated accounts cannot access TMUX sessions after the session is locked.
This bug right now is quite annoying and I currently have this STIG ID turned off in the meantime.

@uk-bolly I think I found the resolve.
The error that kept showing up was the 'vlock' did not have the permissions to open /etc/init.d/system-auth since system-auth was included /etc/pam.d/vlock file.
Upon further checking, I noticed that both password-auth and system-auth are set to 0640 vs 0644. I may have missed it somewhere, but is 0640 the default permission scheme for these files? As soon as I set 0644, restarted sssd, I was successful in locking/unlocking with a user non-elevated credentials.
Thoughts?

#122 Recommend this issue is closed. TMUX sessions work after updating permissions on /etc/pam.d/system-auth file.

hi @jmalpede
Thank you as always for raising issues. I believe we may have found a fix for this as noted by @matthew-willis. This should now be included in the latest devel release.
Please let us know if this is not working for you?
Many thanks again
uk-bolly

This issue is fixed in release 2.6.0, thank you for opening the issue!
-George


Create CDS profile Reader and Blueprint Creator

Background
The CRTDL (https://github.com/medizininformatik-initiative/clinical-resource-transfer-definition-language) defines groups of data to be extracted which reference specific fhir profiles.

    ** Click to expand JSON Examplegroup which references an extraction of all laboratory values and for these the code and value**
{
  "attributeGroups": [
    {
      "groupReference": "https://www.medizininformatik-initiative.de/fhir/core/modul-labor/StructureDefinition/ObservationLab",
      "attributes": [
        {
          "attributeRef": "Observation.code",
          "mustHave": false
        },
        {
          "attributeRef": "Observation.value",
          "mustHave": true
        }
      ],
      "filter": []
    }
  ]
}

Resources are then extracted based on the CRTDL and the linked profile.
Task
The aim of the extraction is to extract resources that only contain the fields the researcher has selected in the CRTDL, but also to ensure that the resulting resource is still CDS compatible.
To achieve this the groupReference should be used to resolve and load the respective profile and then generate a blueprint resource for each profile, which pre-fills all fields for the resource in such a way that the resulting resource is CDS compatible.
To this end a general blueprint generator should be created, which can read in a profile and generate a respective resource using a java library.
The Blueprints should consider the following:

Cardinality 1.. fields should be filled with an mii compatible data absent reason
the blueprint should create a minimal version of a CDS compatible resource, therefore all fields with cardinality 0...1 should not be set
It might be possible to create the generator resourceType specific (e.g. Observation, Condition), potentially avoiding some casting, as the number of FHIR resourceTypes is limited.
An example use of the HAPI lib for resource creation can be seen here:
4.1 https://github.com/geloro94/FHIRTestResourceGenerator/blob/main/src/test/java/ExtractTest.java
4.2. https://github.com/geloro94/FHIRTestResourceGenerator/blob/085607904eabd8188f6523c1dcf215834cbd6695/src/main/java/FhirResourceFactory.java#L217

@Lucas0T, @alexanderkiel  - I closed this issue, as we essentially decided to do the blueprint generation on the fly as the resource to be extracted is generated.


Session Issue #4

Solved Issue #4 I guess....

I'm so sorry :-;

Hello, solved all the issues above. Please re-check. I apologise for not understanding well.

Looks better! Let's see what @hum has to say.

Alright, added presences intents, used correct bot object and cleaned check member status.

LGTM


A Python background process stays opened after closing Spyder in master

Issue Report Checklist

[x] Searched the issues page for similar reports
[x] Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description
After opening Spyder in master with the bootstrap.py or from the Anaconda navigator and closing it, a Python  process remains opened in Windows. Introspection becomes broken by this until I close these background processes and restart Spyder. So this is probably related with the problems I reported in Issue #7726.
Note that this is not occurring with the 3.x branch.

What steps reproduce the problem?

Open Spyder
Close Spyder

What is the expected output? What do you see instead?
All Python processes started by Spyder should be closed
Versions

Spyder version: Spyder 4.0.0.dev0 https://github.com/spyder-ide/spyder/commit/3465e77f077a346b71dfdf85d5f7eea98e64162b
Python version: Python 3.6.5 64bits
Qt version: Qt 5.9.3
PyQt version: PyQt5 5.9.2
Operating System name/version: Windows 10

Dependencies

pyflakes >=0.6.0  :  1.6.0 (OK)
pycodestyle >=2.3 :  2.3.1 (OK)
pygments >=2.0    :  2.2.0 (OK)
sphinx >=0.6.6    :  1.7.1 (OK)
rope >=0.9.4      :  0.10.7 (OK)
jedi >=0.11.0     :  0.12.1 (OK)
nbconvert >=4.0   :  5.3.1 (OK)
pandas >=0.13.1   :  0.22.0 (OK)
numpy >=1.7       :  1.13.0 (OK)
sympy >=0.7.3     :  1.1.1 (OK)
cython >=0.21     :  0.27.3 (OK)
qtconsole >=4.2.0 :  4.3.1 (OK)
IPython >=4.0     :  6.2.1 (OK)
matplotlib >=2.0.0:  2.2.0 (OK)
pylint >=0.25     :  1.8.3 (OK)

This is probably related to the fact that the old code completion architecture has not been removed yet. It's in our TODO list: #7724.

I just want to confirm that without killing these background processes in Windows, introspection stop working.
I wish I could help more, but I do not understand enough what is going on.

@ccordoba12 Note that, this does not occur when running in debug mode.
I'm under the impression that the LSP client do not have time to close correctly when Spyder is not run in debug mode.

@ccordoba12 This is still occurring even after the removal of the old architecture. But when running in debug, it is not. Sometimes, the LSP Client takes a long time to close. Like 15-20 sec.

@ccordoba12 This problem is not happening if I run Spyder from my Anaconda env with pyls==0.19.

Ok, so the problem is with the latest pyls. Thanks for confirming it.

With the latest pyls (>0.19), introspection simply doesn't work at all or make Spyder crash, whether in pip or Anaconda..
The problem I've reported here happens on Windows with pyls==0.19, but I think this is an issue that is specific to my pip install. The problem doesn't occur with Anaconda also using pyls==0.19.

The problem I've reported here happens on Windows with pyls==0.19, but I think this is an issue that is specific to my pip install. The problem doesn't occur with Anaconda also using pyls==0.19.

As part of #8153 I believe I likely reproduced the same problem, since I was upgrading and downgrading with pip, got errors related to pyls.exe being in use, and with all other variables (other than background processes, which I speculated were the cause but wasn't able to isolate a specific one since I had so many Spyders and Pythons open at the time) being equal it would not work after some of the installs downgrades, but when I came back and tried a day later it did. It occurred on 0.19 and 0.20, the two versions I tested.

@dalthviz, are you able to reproduce this with PyLS 0.19?

I have been able to, although it doesn't always make completion stop working at least in my limited testing.

Ok, thanks! I'm moving this to beta3 because I think we need to address this in PyLS itself.

All - this still happens. I am using Spyder 4 and after closing had 6 python.exe processes open (that appeared to have each accumulated after separate Spyder instances).

@pbower Which Spyder 4 beta or dev version are you using? What is your OS and PyLS versions and how did you install them (pip, conda-forge, spyder-ide channel, etc))?
As of Beta 6 (and consistently in previous betas since this issue was first reported), I indeed have had 3 processes stick around indefinitely each time after Spyder crashes, but I haven't noticed it happen if Spyder exits cleanly. The three are Spyder's LSP transport (I assume the client), the PyLS server, and the Jedi subprocess of the server, and depending on use occupy anywhere from 100 MB to well over half a GB of RAM per triplet.
I cannot be 100% sure this occurs on the latest Beta 7 as I'm not sure if it has crashed yet, but next time Spyder crashes I will check. From what I can tell, it looks like the Beta 6 processes are running under the window-mode pythonw while the Beta 7 processes appear to be running under the standard command-line python. I'm running PyLS 0.28.3 with all deps from spyder-ide, no pip installs like before.

Hi there,
Thanks for looking into this issue.
I am using 4.0.0.b5
Installed via conda
OS is windows 10 and python is 3.7.3
It did appear to be crash related - as screens completely locked up and froze until I closed via the task manager. This is when I noticed it.
If you require any more info please let me know.
Cheers
Get Outlook for iOShttps://aka.ms/o0ukef

From: CAM Gerlach<EMAIL_ADDRESS>Sent: Thursday, October 24, 2019 11:41:17 AM
To: spyder-ide/spyder<EMAIL_ADDRESS>Cc: Peter<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [spyder-ide/spyder] A Python background process stays opened after closing Spyder in master (#7796)
@pbowerhttps://github.com/pbower Which Spyder 4 beta or dev version are you using? What is your OS and PyLS versions and how did you install them (pip, conda-forge, spyder-ide channel, etc))?
As of Beta 6 (and consistently in previous betas since this issue was first reported), I indeed have had 3 processes stick around indefinitely each time after Spyder crashes, but I haven't noticed it happen if Spyder exits cleanly. The three are Spyder's LSP transport (I assume the client), the PyLS server, and the Jedi subprocess of the server, and depending on use occupy anywhere from 100 MB to well over half a GB of RAM per triplet.
I cannot be 100% sure this occurs on the latest Beta 7 as I'm not sure if it has crashed yet, but next time Spyder crashes I will check. From what I can tell, it looks like the Beta 6 processes are running under the window-mode pythonw while the Beta 7 processes appear to be running under the standard command-line python. I'm running PyLS 0.28.3 with all deps from spyder-ide, no pip installs like before.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/spyder-ide/spyder/issues/7796?email_source=notifications&email_token=AI27BYT7CPCRQWGFYF2QRMDQQDVK3A5CNFSM4FSVYHIKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECDJYXI#issuecomment-545692765, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AI27BYWGGSOFF5VFRGP4LXLQQDVK3ANCNFSM4FSVYHIA.

Also I should add even normally python runs Spyder runs with 6 python.exe processes? Is there a reason for this? No ones put a bitcoin miner in there that persists after shutdown have they 😂
Get Outlook for iOShttps://aka.ms/o0ukef

From: Peter Bower<EMAIL_ADDRESS>Sent: Thursday, October 24, 2019 2:08:10 PM
To: spyder-ide/spyder<EMAIL_ADDRESS>spyder-ide/spyder<EMAIL_ADDRESS>Cc: Mention<EMAIL_ADDRESS>Subject: Re: [spyder-ide/spyder] A Python background process stays opened after closing Spyder in master (#7796)
Hi there,
Thanks for looking into this issue.
I am using 4.0.0.b5
Installed via conda
OS is windows 10 and python is 3.7.3
It did appear to be crash related - as screens completely locked up and froze until I closed via the task manager. This is when I noticed it.
If you require any more info please let me know.
Cheers
Get Outlook for iOShttps://aka.ms/o0ukef

From: CAM Gerlach<EMAIL_ADDRESS>Sent: Thursday, October 24, 2019 11:41:17 AM
To: spyder-ide/spyder<EMAIL_ADDRESS>Cc: Peter<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [spyder-ide/spyder] A Python background process stays opened after closing Spyder in master (#7796)
@pbowerhttps://github.com/pbower Which Spyder 4 beta or dev version are you using? What is your OS and PyLS versions and how did you install them (pip, conda-forge, spyder-ide channel, etc))?
As of Beta 6 (and consistently in previous betas since this issue was first reported), I indeed have had 3 processes stick around indefinitely each time after Spyder crashes, but I haven't noticed it happen if Spyder exits cleanly. The three are Spyder's LSP transport (I assume the client), the PyLS server, and the Jedi subprocess of the server, and depending on use occupy anywhere from 100 MB to well over half a GB of RAM per triplet.
I cannot be 100% sure this occurs on the latest Beta 7 as I'm not sure if it has crashed yet, but next time Spyder crashes I will check. From what I can tell, it looks like the Beta 6 processes are running under the window-mode pythonw while the Beta 7 processes appear to be running under the standard command-line python. I'm running PyLS 0.28.3 with all deps from spyder-ide, no pip installs like before.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/spyder-ide/spyder/issues/7796?email_source=notifications&email_token=AI27BYT7CPCRQWGFYF2QRMDQQDVK3A5CNFSM4FSVYHIKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECDJYXI#issuecomment-545692765, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AI27BYWGGSOFF5VFRGP4LXLQQDVK3ANCNFSM4FSVYHIA.

Oh ok great thanks. So it’s just that on crash something isn’t quitting.
Thanks for that.
Get Outlook for iOShttps://aka.ms/o0ukef

From: CAM Gerlach<EMAIL_ADDRESS>Sent: Thursday, October 24, 2019 9:08:32 PM
To: spyder-ide/spyder<EMAIL_ADDRESS>Cc: Peter<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [spyder-ide/spyder] A Python background process stays opened after closing Spyder in master (#7796)
Yes, completely normal, like any modern multiprocess application and this is A Good Thing™. Looking at them in a task manager you can figure out what each is: Python launcher, main Spyder process, LSP server, LSP client, Jedi completion engine, and the IPython kernel that powers the default Console. An additional Python process will be launched for each kernel you start, and there are usually at least a few QtWebEngine processes that render the documentation in the help pane for the Editor and each Console.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/spyder-ide/spyder/issues/7796?email_source=notifications&email_token=AI27BYUYYUZ46TLV52OCNFTQQFX2BA5CNFSM4FSVYHIKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECEPRXQ#issuecomment-545847518, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AI27BYWBEHSH5RHQB2L57CTQQFX2BANCNFSM4FSVYHIA.

I indeed have had 3 processes stick around indefinitely each time after Spyder crashes

For the record, this is being worked on in PR #10111.


Logout removes all accounts on the phone...

We are currently removing all the account on the phone when logging out. At least for my phone all of my email accounts are removed when logging out.
Device: Nexus 5
Android version: 5.1.1
Account[] allAccounts = AccountManager.get(this).getAccounts();
    for (Account account : allAccounts) {
        AccountManager.get(this).removeAccount(account, null, null);
    }

Above we get all the accounts on the device instead of getting them by type

I think we should distinguish between switching accounts, and removing them.
Take a look at the gmail app. There you can easily switch between different mail accounts in the Navigation drawer

See the PR here: https://github.com/pockethub/PocketHub/pull/898

#898 is merged, so we should close this issue