Fidelity of encoding for arbitrary unitaries

Greetings there,
Hope all are well. I came across your great repo and have been experimenting with it for a few days now. I have been using the Floki00/qc_unitary_3qubit for the moment and noticed that the fidelity of the encoding for arbitrary unitaries almost never reaches above 0.7, which is a considerable error for 3 qubits.
I am aware that the model used may need fine-tuning, however, I'd like to ensure that my use of it is correct and whether the infidelity mentioned is expected.
Here is my current wrapper of genQC:
from __future__ import annotations

__all__ = ["Diffusion"]

from genQC.pipeline.diffusion_pipeline import DiffusionPipeline
from genQC.inference.infer_compilation import generate_comp_tensors, convert_tensors_to_circuits
import genQC.util as util
import numpy as np
from numpy.typing import NDArray
from qiskit.quantum_info import Operator as QiskitOperator
import torch
from typing import Sequence, SupportsIndex, TYPE_CHECKING

import qickit
if TYPE_CHECKING:
    from qickit.circuit import Circuit
from qickit.primitives.operator import Operator
from qickit.synthesis.unitarypreparation import UnitaryPreparation

# Set the random seed for reproducibility
np.random.seed(0)
torch.manual_seed(0)

class Diffusion(UnitaryPreparation):
    """ `qickit.synthesis.unitarypreparation.Diffusion` is the class for performing unitary
    compilation using diffusion models (DMs).
    ref: https://arxiv.org/abs/2311.02041

    Notes
    -----
    The default pre-trained model used in this class is for 3-qubit unitaries only.

    Parameters
    ----------
    `output_framework` : type[qickit.circuit.Circuit]
        The quantum circuit framework.
    `model` : str, optional, default="Floki00/qc_unitary_3qubit"
        The pre-trained model to use.
    `prompt` : str, optional, default="Compile using: ['h', 'cx', 'z', 'ccx', 'swap']"
        The prompt to use for the compilation.
    `max_num_gates` : int, optional, default=12
        The maximum number of gates to use in the compilation.
    `num_samples` : int, optional, default=128
        The number of samples to use in the compilation.
    `min_fidelity` : float, optional, default=0.9
        The minimum fidelity to accept the solution.

    Attributes
    ----------
    `output_framework` : type[qickit.circuit.Circuit]
        The quantum circuit framework.
    `model` : str
        The pre-trained model to use.
    `prompt` : str
        The prompt to use for the compilation.
    `max_num_gates` : int
        The maximum number of gates to use in the compilation.
    `num_samples` : int
        The number of samples to use in the compilation.
    `pipeline` : genQC.pipeline.diffusion_pipeline.DiffusionPipeline
        The pre-trained model pipeline.

    Raises
    ------
    TypeError
        If the output framework is not a subclass of `qickit.circuit.Circuit`.
    ValueError
        If the minimum fidelity is not in the range [0, 1].
    """
    def __init__(
            self,
            output_framework: type[Circuit],
            model: str="Floki00/qc_unitary_3qubit",
            prompt: str="Compile using: ['h', 'cx', 'z', 'ccx', 'swap']",
            max_num_gates: int=12,
            num_samples: int=128,
            min_fidelity: float=0.9
        ) -> None:

        super().__init__(output_framework)
        self.model = model
        self.prompt = prompt
        self.max_num_gates = max_num_gates
        self.num_samples = num_samples

        if not (min_fidelity >= 0 and min_fidelity <= 1):
            raise ValueError("The minimum fidelity should be in the range [0, 1].")
        self.min_fidelity = min_fidelity

        # Determine the device to use (CPU or GPU)
        device = util.infer_torch_device()

        # Clean the memory
        util.MemoryCleaner.purge_mem()

        # Load the pre-trained model
        self.pipeline = DiffusionPipeline.from_pretrained(repo_id=model, device=device)

    def apply_unitary(
            self,
            circuit: Circuit,
            unitary: NDArray[np.complex128] | Operator,
            qubit_indices: int | Sequence[int]
        ) -> Circuit:
        """ Apply the unitary to the circuit.

        Parameters
        ----------
        `circuit` : qickit.circuit.Circuit
            The quantum circuit to apply the unitary to.
        `unitary` : NDArray[np.complex128] | qickit.primitives.Operator
            The unitary to apply.
        `qubit_indices` : int | Sequence[int]
            The qubit indices to apply the unitary to.

        Returns
        -------
        circuit : `qickit.circuit.Circuit`
            The quantum circuit with the unitary applied.

        Raises
        ------
        ValueError
            If the unitary is not a 3-qubit unitary.
            No solution found with fidelity > 0.9.
        """
        if isinstance(unitary, np.ndarray):
            unitary = Operator(unitary)

        if isinstance(qubit_indices, SupportsIndex):
            qubit_indices = [qubit_indices]

        if self.model == "Floki00/qc_unitary_3qubit" and unitary.num_qubits != 3:
            raise ValueError("The default pre-trained model is for 3-qubit unitaries only.")

        num_qubits = unitary.num_qubits

        # As the neural network works only with real numbers, we first separate
        # the two components and create a 2 dimensional tensor for the magnitude
        # of each component
        U_r, U_i = torch.Tensor(np.real(unitary.data)), torch.Tensor(np.imag(unitary.data))
        U_tensor = torch.stack([U_r, U_i], dim=0)

        # Now we generate a tensor representation of the desired quantum circuit using the DM based on the prompt and U
        # This is also known as inference
        out_tensors = generate_comp_tensors(
            pipeline=self.pipeline,
            prompt=self.prompt,
            U=U_tensor,
            samples=self.num_samples,
            num_of_qubits=num_qubits,
            system_size=num_qubits,
            max_gates=self.max_num_gates,
            g=10
        )

        # Find the best solution in terms of the number of gates and fidelity
        qc_list, _ = convert_tensors_to_circuits(out_tensor=out_tensors, gate_pool=self.pipeline.gate_pool)

        # Find the best solution in terms of the number of gates and fidelity
        depths = []

        for qc in qc_list:
            qc_unitary = QiskitOperator(qc).data

            fidelity = np.abs(
                np.dot(
                    np.conj(qc_unitary.flatten()), # type: ignore
                    unitary.data.flatten()
                )
            )/2**num_qubits

            if fidelity > self.min_fidelity:
                depths.append(qc.depth())

        if len(depths) == 0:
            raise ValueError(f"No solution found with fidelity > {self.min_fidelity}.")

        # Find the shortest circuit with fidelity > `self.min_fidelity`
        best_qc = qc_list[depths.index(min(depths))]

        circuit = qickit.circuit.Circuit.from_qiskit(best_qc, self.output_framework)

        return circuit

I tried cuda-quantum's notebook too for safety, and the infidelity (almost 0.5 infidelity) is present there as well. I have played around with larger number of samples as well as longer max number of gates (even beyond the theoretical limit of most exact encoders which is $2^{N+1}$ where N is the number of qubits, i.e., 30, 40, and even 50) to no avail. I'd deeply appreciate some guidance if possible.
Oh, FYI, qickit is my package which is not published yet, so if you like to test it yourself, please let me know.

Hello @ACE07-Sev 👋, thanks for your interest in this project.
Regarding the limitations of the Floki00/qc_unitary_3qubit weights, there are some restrictions this model was trained on: 3 qubits, a maximum of 12 gates and the (discrete) gate set ['h', 'cx', 'z', 'x', 'ccx', 'swap'] . When one wants to compile a unitary U, one should be aware that the model can only possibly find solutions if there exists a (theoretical) physical solution, within these constraints, for that given U.  Hence, the set of compile-able unitaries is quite smaller than the whole unitary group.
Within these constraints we show the expected compilation distribution in our paper Quantum circuit synthesis with diffusion models in Figure 4. Also, we found “the model successfully identifies the correct exact unitary for 92.6 % of the 3100 tested unitaries”, where exact means a fidelity of 1.
However, we are working on bigger and better models, which we plan to release alongside future publications. Stay tuned for that 😄.
Out of curiosity, what kind of unitaries are you testing on and are interested in? Can you give an example of a unitary matrix and maybe a corresponding circuit which the model should find? This could help us benchmark and tune the next models. We are also interested in what tasks a user may be looking for, which we can take into account for future considerations of the project.

Greetings there,
Hope you are well. Thank you very much for the prompt response, and apologies for the delay on my side. So, I tried random gates within the gate set (nothing beyond a depth of 5 or 6) as well as random generated unitaries using scipy.linalg's unitary_group for 3 qubits.
Results are mostly 65 percent fidelity or so at best. I'll send a comprehensive report here today or tomorrow for your kind reference.
So, as for what users would be looking for I'd say high fidelity compilation with as low of a depth as possible. I have been working on this task for a year now, and found MPS approach to be more suitable (i.e., encoded a 22 qubit state with ~12111 depth instead of ~8M with 97 percent fidelity). Whilst they are very effective for lowering depth, they take a bit of time to run given the low level QR and SVD operations needed. If RL models can become as robust with lower inference times, that would certainly be useful.
I would overall suggest focusing more on general unitary compilation. Given that's what I'm actively working on, I'd love to contribute and even collaborate if you're interested.

The low accuracy on unitaries from scipy.stats.unitary_group is expected, it is very unlikely that such a random U can be decomposed (exactly) into ['h', 'cx', 'z', 'x', 'ccx', 'swap'] using a maximum of 12 gates. Note that there are no parametrized gates (e.g. rx or ry). While the gate set is universal (already h and ccx is) this only holds for a very large number of gates.
You can also check the example notebook on unitary compilation [doc] [notebook].
When you mention “high fidelity compilation; ... MPS approach” do you refer to quantum state preparation or unitary matrix to circuit compilation?
In your case, where would these “general unitary compilation” tasks stem from (e.g. a unitary evolution from a particular Hamiltonian)?

The low accuracy on unitaries from scipy.stats.unitary_group is expected, it is very unlikely that such a random U can be decomposed (exactly) into ['h', 'cx', 'z', 'x', 'ccx', 'swap'] using a maximum of 12 gates. Note that there are no parametrized gates (e.g. rx or ry). While the gate set is universal (already h and ccx is) this only holds for a very large number of gates.
You can also check the example notebook on unitary compilation [[doc]]
(https://florianfuerrutter.github.io/genQC/examples/unitary_compilation.html) [[notebook]]
(https://github.com/FlorianFuerrutter/genQC/blob/main/src/examples/2_unitary_compilation.ipynb).

Yes, I instead used random circuits made of the gate set mentioned within the maximum number of gates constraint and it worked better. I get the gist now, though I would like to see the scaling of this for an arbitrary U compared to exact approaches such as QSD.
My hope would be that the RL approach would fare better in terms of circuit depth, otherwise, it would be better to use Shende/QSD for exact encoding, or MPS/MPO for approximate encoding. Each approach has its pros and cons. For instance, whilst Shende is much faster, it also produces circuits that are exponential in depth. On the other hand, whilst the MPS approach produces significantly shallower circuits, it also consumes a considerable amount of time with low-level QR and SVD operations.
The appeal of MPS/MPO is that it proposes an answer to "Can vs Can't". With the emergence of other alternatives, we can later contemplate the "Should vs Shouldn't" question.

When you mention “high fidelity compilation; ... MPS approach” do you refer to quantum state preparation or unitary matrix to circuit compilation?

So, I have currently done state preparation (using an MPS) and am actively working on adding the unitary preparation (using an MPO).

In your case, where would these “general unitary compilation” tasks stem from (e.g. a unitary evolution from a particular Hamiltonian)?

So, I interpret quantum computing as simply $A\ket{\psi}$ where $A$ is a unitary matrix over N qubits, and $\ket{\psi}$ is a vector over N qubits (simply a tensor contraction backend). These two primitives form our high-level code. Now, to perform actual quantum computing they need to be compiled to low-level code, aka quantum circuits. The main focus of my work is enabling this conversion to be done as shallowly as possible.
The reason this is useful is because it's domain agnostic. Regardless of whether you do quantum chemistry, physics simulation, optimization, or QML, at the end of the day you are working with these primitives and need to represent them as quantum circuits so that you may run them on actual quantum hardware. Given the exponential depth scaling for both state preparation and unitary preparation, the scale of problems we can meaningfully implement on actual hardware is often limited to trivial ones. The goal with the compiler I am making is to reduce the cost and allow for implementing larger, more practical problems to be implementable on current hardware. So, for this to be possible I must cover arbitrary vectors as well as unitary matrices up to an adequate fidelity.
A good analogy for this would be the impact of making a much more efficient compiler for python on codes that are developed using python. Regardless of what code you write (or its use-case), you will observe the improvement. So, that's where the need for efficient general unitary compilation comes from.

By the way, I also found a rather silly bug in the code I sent before. It's supposed to find the shortest circuit with 0.9+ fidelity, and it was missing the circuit_solutions list.
# Find the best solution in terms of the number of gates and fidelity
qc_list, _ = convert_tensors_to_circuits(out_tensor=out_tensors, gate_pool=self.pipeline.gate_pool)

# Find the best solution in terms of the number of gates and fidelity
depths = []
solution_circuits = []

for qc in qc_list:
    qc_unitary = QiskitOperator(qc).data

    fidelity = np.abs(
        np.dot(
            np.conj(qc_unitary.flatten()), # type: ignore
            unitary.data.flatten()
        )
    )/2**num_qubits

    if fidelity > self.min_fidelity:
        depths.append(qc.depth())
        solution_circuits.append(qc)

if len(depths) == 0:
    raise ValueError(f"No solution found with fidelity > {self.min_fidelity}.")

# Find the shortest circuit with fidelity > `self.min_fidelity`
best_qc = solution_circuits[depths.index(min(depths))]

circuit = qickit.circuit.Circuit.from_qiskit(best_qc, self.output_framework)

return circuit

Thank you for the clarification.
I will close this issue now, as it is a limitation of this particular pre-trained model weights and more a feature request for better models.


Is there a way to override Tab functionality with fallback?

monaco-editor version: 0.10.1
Browser: Chrome 61
OS: Windows 10
Steps or JS usage snippet reproducing the issue:
I'm working on emmet support for the Monaco editor.
I want to use emmet when code matches a certain condition, otherwise fallback to the old tab functionality (e.g. suggest completion or insert a tab character).
Is there a way to achieve this?

got answer from #165


Question about use of serialize(., map {}) in https://github.com/xspec/xspec/blob/master/src/reporter/format-utils.xsl#L260

I am currently exploring whether I can use Saxon-JS 2.3 and its xslt3 command line tool instead of Saxon Java to run XSLT specific XSpec tests.
On my way to doing this I found some oddity in the way Java based Saxon and Node.js based Saxon-JS handle the use of serialize(., map {}) in https://github.com/xspec/xspec/blob/master/src/reporter/format-utils.xsl#L260, while Saxon Java doesn't output any XML declaration with serialize($someTextNode, map{}) but seems to treat it as serialize($someTextNode) which the spec defines as serialize($someTextNode, ()) Saxon-JS outputs an XML declaration for the code used in the XSpec code (i.e. serialize(., map {})) while not doing this for a simple serialize(.) or serialize(., ()).
Before "patching" my local copy of https://github.com/xspec/xspec/blob/master/src/reporter/format-utils.xsl#L260 to use serialize(.) instead of serialize(., map{}), I was wondering whether anyone here knows why the empty map{} is passed as the second argument to the serialize function.
I will also look into filing a bug on Saxon-JS as the spec https://www.w3.org/TR/xpath-functions/#func-serialize seems to suggest the default values for an empty map are to omit-xml-declaration.

map{} is to make sure that all the parameters take their default values.
If I understand https://www.w3.org/TR/xpath-functions-31/#func-serialize correctly, serialize(.) or serialize(., ()) leaves the parameters implementation-defined. In fact, serialize(.) on Saxon <IP_ADDRESS> was turning off omit-xml-declaration (#40).

Thanks for the (air) quick response. As Saxon-JS currently doesn't use the default for the empty map, I will temporarily go with serialize(., map { 'omit-xml-declaration' : true() }). Saxonica seems to be busy this week with some other project but Norm has promised to look at the issues I have raised so far later on. So hopefully some future maintenance release of Saxon-JS 2 doesn't need me "patching" up XSpec files to get them to run with xslt3 or in this case to get some normal output without lots of pesky XML declarations showing up in all places where you don't want them.

I also gave Saxon-JS 2.3 a try and encountered these problems, all or some of which you're aware of:

FORG0006 at xpath.xsl#786 and wrong resolve-uri() (https://saxonica.plan.io/boards/5/topics/8286)
Whitespace-only text nodes are lost in some cases (https://saxonica.plan.io/issues/5001)
inherit-namespaces="no" doesn't set xmlns="" (https://saxonica.plan.io/issues/5082)


AP: Remove callback_url from client config

callback_url has been deprecated in favor of SEP-specific URLs.

Preview is available here:http://developers-pr958.previews.kube001.services.stellar-ops.com


Custom page size

Sorry this is more a question than an issue, are only standard page sizes supported or can this generate pdfs with custom dimensions?

Pages with custom dimensions are supported, see this example.

Awesome, thank you!


Bump to k8s.io 1.24.16 dependencies

This is the last planned 1.24 release.

We’ll get this automatically during the release.


[APERO] mini data are not consistent

We see inconsistencies between minidata2 reductions in spirou.

APERO was run on different machines

in theory with the same git branch, same installed python modules

LBL was run on a single machine

rms of difference spirou@rali-cook@jupiter = 0.06417200529342021 m/s
rms of difference spirou@rali-cook@nb19 = 0.1346402795169976 m/s
rms of difference spirou@rali-spirou@maestria = 0.14235515258684286 m/s
rms of difference spirou@rali-lam = 0.06417200529342021 m/s
rms of difference spirou@rali-cfht = 0.9038638383785134 m/s

Note this issue is also mentioned here: https://docs.google.com/document/d/1C8dp_5KuBfiumSGRzSTI1guZFvJkILOGYjmmvuK-6_c/edit?usp=sharing

here's a new link to Toulouse data for this minidataset: https://mycore.core-cloud.net/index.php/s/3cIsdql4lH3JqhV
thanks!

So newworlds (as with previous versions) agrees very well with one group of the machines.

And a zoom in:

rms of difference spirou@rali-cook@jupiter = 0.06417200529342021 m/s
rms of difference spirou@rali-cook@nb19 = 0.1346402795169976 m/s
rms of difference spirou@rali-spirou@maestria = 0.14235515258684286 m/s
rms of difference spirou@rali-newworlds = 0.13698797905443139 m/s
rms of difference spirou@rali-lam = 0.06417200529342021 m/s
rms of difference spirou@rali-cfht = 0.9038638383785134 m/s
So we have:

cook@juptier, spirou@rali and lam agreeing very well
cook@nb19, spirou@maestria and newworlds agreeing very well
cfht not agreeing with anyone

Interesting results! I wonder what this bimodality reflects.
Can you please remind us on which machine the full reduction is run at UdM (rali, jupiter, maestria, nb19?)?

The full run is on rali. And yep I'm equally interested/perplexed by this bimodality.
At one point @eartigau suggested it could be the CCF we do at the start in LBL "jumping" onto a certain solution for the sys vel but I'm not sure this could still be true.
I think the best way would be to have someone (with time) take a tar of the full mini data sets from all of these and work backward (start at the tcorr, then the extract, then the calibs, then pp) and see where the difference comes from.
If the tcorr are the same to ~numerical accuracy it has to come from LBL.

Neil I just sent you a link to a new set of blaze+tcorr from a new mini_run2 test I did yesterday night after a apero_reset and apero_precheck.
I plotted the difference between blaze or tcorr files from tests before the reset and after and I see differences, so maybe now the cfht data fits the results from other machines.

Interesting, I'll add this data today. If it does agree with the other machines we really need to understand what happened between the restart. I'll post here when I have the data added to the graph.

Yep so CFHT agrees well now. So the question is what was the difference? We might have to ask Chris what could have changed as that is really perplexing.
Here is the new graph:

rms of difference spirou@rali-cook@jupiter = 0.06417200529342021 m/s
rms of difference spirou@rali-cook@nb19 = 0.1346402795169976 m/s
rms of difference spirou@rali-spirou@maestria = 0.14235515258684286 m/s
rms of difference spirou@rali-newworlds = 0.13698797905443139 m/s
rms of difference spirou@rali-lam = 0.06417200529342021 m/s
rms of difference spirou@rali-cfht = 0.06658835428246165 m/s
I'm glad there are now only two sets, I still don't understand that or why CFHT data changed after a reset! :man_shrugging:

Here is a map of the difference between a given blaze taken before and after the reset.  The difference is not 0 and increases towards the K-band, where the blazes have a shape which is less symetrical.  I understand that the polynomial involved into the blaze flux model is only up to 3rd order, but maybe the non-orthogonality between terms is the culprit for differences we see from one reduction to the next?

Differences are low frenquecies, and likely to be described well with a polynomial up to degree 3.

What I'll do today is another reset on apikipiha and run again the mini_run2, to verify that what I get is consistent.

The results after this second apero-reset suggest that a given machine finds always the same fit for a the blaze. But what about different machines ?   Maybe Neil you could check for any differences in the blazes you got from the different reductions in marseille, toulouse, montreal and cfht ?

Yer I'll try checking these tomorrow. Correct me if I'm wrong though but your two results are from the same machine (before and after reset?) so do we have any idea what the difference was?

So I went through all 7 machines. And the difference is tiny.
The fluxes we are talking for the blaze are ~80000 to 100000.
The biggest difference I see is on order 10 with a max diff of 0.6 at the edge of the blaze (around a flux of 5000)
Seems to only be cfht out this far on this particular order and all others agree much better:

All other orders look like this with all not agreeing exactly but all agreeing quite well (within 0.03 flux on 60000)

So yer most blazes are at numerical levels (fractionally ~10^-7) with the largest deviation being fractional = 10^-5 on order 10.

This is with v0.7.288  with the four machines in Montreal:

rms of difference spirou@rali-cook@jupiter = 0.15466742511421885 m/s
rms of difference spirou@rali-cook@nb19 = 0.016303989383939683 m/s
rms of difference spirou@rali-spirou@maestria = 0.044794374568119605 m/s

If you can just drop your GL699_GL699 files in here: https://drive.google.com/drive/folders/1XpgLS_mD0-vZXMFIJCblZPoKBsC0xoC1?usp=sharing
I'll add to the comparison
@clairem789 turns out my code needs the .fits files too (sorry)

@njcuk9999 I have uploaded my lbl files to your google drive. Just to let you know.

Comparison between CFHT and NW minidataset .288  @njcuk9999 @clairem789
night 2020-05-14
Gl699  *o_pp.fits:   difference = exactly 0
A difference is visible in blaze, order 10.  Also in tcorr.

This isn't really a good way to plot this. You need to make the difference fractional or relative to the flux in one of the images.
Instead of a direct difference you need difference/mean value of the two images at each pixel... you'll see its tiny (at the numerical accuracy level). The values of the blaze are huge compared to a value of "10".


Flowify promisify

Add flow annotations to jest-cli/lib/promisify.js

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please sign up at https://code.facebook.com/cla - and if you have received this in error or have any questions, please drop us a line at<EMAIL_ADDRESS>Thanks!

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Facebook open source project. Thanks!

I don't know why eslint reports a syntax error. I can't find a workaround. Even the simple case
type Pair = [A, A];
results in a syntax error.


update carvel github action

update carvel github actions due to repository migration

Codecov Report

:exclamation: No coverage uploaded for pull request base (v0.3.x@675a56d). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             v0.3.x     #363   +/-   ##
=========================================
  Coverage          ?   69.88%
=========================================
  Files             ?       17
  Lines             ?      943
  Branches          ?        0
=========================================
  Hits              ?      659
  Misses            ?      263
  Partials          ?       21

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Issue when producing a message with multiple headers

Versions
Please specify real version numbers or git SHAs, not just "Latest" since that changes fairly regularly.
Sarama Version: 6a8d89d71d22bbdc14b6cb52778889a710be19a7
Kafka Version: 1.0.0
Go Version: go1.9.2 linux/amd64
Configuration
What configuration values are you using for Sarama and Kafka?
Kafka: wurstmeister/kafka:1.0.0 docker image with default config.
Sarama: 	cfg.Version = sarama.V1_0_0_0
Logs
When filing an issue please provide logs from Sarama and Kafka if at all
possible. You can set sarama.Logger to a log.Logger to capture Sarama debug
output.
Producer logs:
2017/12/03 16:04:57 Initializing new client
2017/12/03 16:04:57 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
2017/12/03 16:04:57 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
2017/12/03 16:04:57 client/metadata fetching metadata for all topics from broker localhost:9092
2017/12/03 16:04:57 Connected to broker at localhost:9092 (unregistered)
2017/12/03 16:04:57 client/brokers registered new broker #1 at <IP_ADDRESS>:9092
2017/12/03 16:04:57 Successfully initialized new client
2017/12/03 16:04:57 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
2017/12/03 16:04:57 producer/broker/1 starting up
2017/12/03 16:04:57 producer/broker/1 state change to [open] on my_topic/0
2017/12/03 16:04:57 Connected to broker at <IP_ADDRESS>:9092 (registered as #1)

Consumer logs:
2017/12/03 16:04:56 Initializing new client
2017/12/03 16:04:56 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
2017/12/03 16:04:56 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
2017/12/03 16:04:56 client/metadata fetching metadata for all topics from broker localhost:9092
2017/12/03 16:04:56 Connected to broker at localhost:9092 (unregistered)
2017/12/03 16:04:56 client/brokers registered new broker #1 at <IP_ADDRESS>:9092
2017/12/03 16:04:56 Successfully initialized new client
2017/12/03 16:04:56 ClientID is the default of 'sarama', you should consider setting it to something application-specific.
2017/12/03 16:04:56 Connected to broker at <IP_ADDRESS>:9092 (registered as #1)
2017/12/03 16:04:56 consumer/broker/1 added subscription to my_topic/0
2017/12/03 16:04:57 Key: header-3, Value: value-3
2017/12/03 16:04:57 Key: header-3, Value: value-3
2017/12/03 16:04:57 Key: header-3, Value: value-3

Problem Description
When producing a message with multiple headers the last header overwrites all other headers.
Code examples:
Producer:
package main

import (
	"log"
	"os"

	"github.com/Shopify/sarama"
)

func main() {
	sarama.Logger = log.New(os.Stdout, "", log.LstdFlags)

	cfg := sarama.NewConfig()
	cfg.Version = sarama.V1_0_0_0
	cfg.Producer.Return.Successes = true
	cfg.Producer.Return.Errors = true

	producer, err := sarama.NewSyncProducer([]string{"localhost:9092"}, cfg)
	if err != nil {
		panic(err)
	}

	msg := &sarama.ProducerMessage{
		Topic: "my_topic",
		Value: sarama.StringEncoder("testing 123"),
		Headers: []sarama.RecordHeader{
			sarama.RecordHeader{
				Key:   []byte("header-1"),
				Value: []byte("value-1"),
			},
			sarama.RecordHeader{
				Key:   []byte("header-2"),
				Value: []byte("value-2"),
			},
			sarama.RecordHeader{
				Key:   []byte("header-3"),
				Value: []byte("value-3"),
			},
		},
	}
	_, _, err = producer.SendMessage(msg)
	if err != nil {
		panic(err)
	}
}

Consumer:
package main

import (
	"log"
	"os"

	"github.com/Shopify/sarama"
)

func main() {
	sarama.Logger = log.New(os.Stdout, "", log.LstdFlags)

	cfg := sarama.NewConfig()
	cfg.Version = sarama.V1_0_0_0

	consumer, err := sarama.NewConsumer([]string{"localhost:9092"}, cfg)
	if err != nil {
		panic(err)
	}

	partitionConsumer, err := consumer.ConsumePartition("my_topic", 0, sarama.OffsetNewest)
	if err != nil {
		panic(err)
	}

	msg := <-partitionConsumer.Messages()
	for _, h := range msg.Headers {
		log.Printf("Key: %s, Value: %s", h.Key, h.Value)
	}
}

Hmm, the encode/decode logic looks right. Can you take a look on the network and see if this is a bug happening on encode or on decode? Or maybe (though unlikely) it's a bug on the broker side?
cc @wladh on the off-chance you know what could be causing this.

Oh right. This is one of the more annoying subtleties of Go, it's bitten me before too.

Merge fixed the issue. Thanks.


Remote php interpreter  Laravel root directory problem

Stange problem here :
My setup is as follow : my php interpreter is "remote" : ssh connection to a local running VM
It so happens that when i try to run the tinker console, I got this phpstorm notification :
"The configured laravel root is not a directory
Please check if it's the right path. Using a guessed vendor path for now."
The strange thing is that if I take the path in "Laravel root folder", and put "cd path" in my .bashrc, it actually works, which means that the directory definetely exists.
I looked in the repository and came to this  (PhpArtisanTinkerUtil.kt line 56)
if (projectSettings.laravelRoot.isNotEmpty()) {
    val customLaravelRoot = File(projectSettings.laravelRoot)

    if (customLaravelRoot.exists() && customLaravelRoot.isDirectory) {
        val customComposerDir = File(customLaravelRoot.path + "/vendor")

        if (customComposerDir.exists() && customComposerDir.isDirectory) {
            laravelRoot = projectSettings.laravelRoot
        } else {
            LaravelRootDoesNotHaveVendorBalloon(project).show()
        }
    } else {
        LaravelRootDoesNotExistBalloon(project).show()
    }
}

As far as I understand it (not that much), this is for local php interpreter, and I didn't find anything related to remote :
did I missed something ?

This issue is now released and accepted! You can update via your IDE plugins settings :)


pull request for e2e test at 11/18/2016 10:26:04

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

optest/docs/updated_at.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


settings.gradle refactor

remove list of projects, just individually include each project since most projects were being included that way.

well, I don't have a solution to make the new project starter work if this is merged, so I am closing for now, and will add some comments to make things more clear what is going on.


Failing to start IE with 1.3.0

2017/06/07 12:44:21 [NEW_REQUEST]
2017/06/07 12:44:21 [NEW_REQUEST_ACCEPTED]
2017/06/07 12:44:21 [6] [LOCATING_SERVICE] [internet explorer-]
2017/06/07 12:44:21 Using default version: 11
2017/06/07 12:44:21 [6] [USING_DRIVER] [internet explorer-11]
2017/06/07 12:44:21 [6] [ENVIRONMENT_NOT_AVAILABLE] [unknown] [internet explorer-11]

%%
2017/06/07 12:44:21 [6] [LOCATING_SERVICE] [internet explorer-]
%%
No browser version in this line.

https://github.com/aerokube/selenoid/commit/f1e33ae39f2e4bbe938a564b48426f55815b4136#diff-eeade751807b5e894d1f9a5b21390878
Accidentally removed &Driver statement.


Add topic field in .dockstore.yml for GitHub app tools, workflows, and notebooks

Description
Corresponding UI PR: https://github.com/dockstore/dockstore-ui2/pull/1859
This PR adds a topic field to the .dockstore.yml that hooks up to the existing Entry topicManual field, allowing users to specify a topic for each workflow in the .dockstore.yml.
The existing Entry topicManual and topicAutomatic fields are entry-level properties, which was a bit hard for me to wrap my head around because most fields in the .dockstore.yml are version-level. This topic .dockstore.yml field operates similar to the other entry-level properties in the .dockstore.yml like publish and latestTagAsDefault by using the value from the latest push.
Behaviour:

The entry topicManual is set using the topic from the latest push.
If topic is non-empty, the topicManual is updated and the topicSelection is automatically set to MANUAL.
If topic is empty, i.e. topic: '', then the topicManual is reset to null and the topicSelection is automatically set to AUTOMATIC. This is to provide the user a way to reset their topicManual since they can't edit it in the UI.
If topic doesn't exist in the .dockstore.yml at all, nothing is changed. This is to prevent a version without a topic from clearing the entry's topicManual.
The user is still able to change the topic selection in the UI for GitHub app entries if they want to.

Review Instructions
Create a .dockstore.yml with more than two workflows and specify a topic for each workflow. Check on QA that these two workflows have the topic you specified and that the topic selection is MANUAL.
Set the topic to an empty string for both workflows, i.e. topic: ''. Check on QA that these two workflows have no Manual Topic value and that the Topic Selection is AUTOMATIC.
Issue
#5625
Security and Privacy
If there are any concerns that require extra attention from the security team, highlight them here.
e.g. Does this change...

Any user data we collect, or data location?
Access control, authentication or authorization?
Encryption features?

Please make sure that you've checked the following before submitting your pull request. Thanks!

[x] Check that you pass the basic style checks and unit tests by running mvn clean install
[x] Ensure that the PR targets the correct branch. Check the milestone or fix version of the ticket.
[x] Follow the existing JPA patterns for queries, using named parameters, to avoid SQL injection
[x] If you are changing dependencies, check the Snyk status check or the dashboard to ensure you are not introducing new high/critical vulnerabilities
[x] Assume that inputs to the API can be malicious, and sanitize and/or check for Denial of Service type values, e.g., massive sizes
[x] Do not serve user-uploaded binary images through the Dockstore API
[x] Ensure that endpoints that only allow privileged access enforce that with the @RolesAllowed annotation
[x] Do not create cookies, although this may change in the future
[ ] If this PR is for a user-facing feature, create and link a documentation ticket for this feature (usually in the same milestone as the linked issue). Style points if you create a documentation PR directly and link that instead.

The existing Entry topicManual and topicAutomatic fields are entry-level properties, which was a bit hard for me to wrap my head around because most fields in the .dockstore.yml are version-level. This topic .dockstore.yml field operates similar to the other entry-level properties in the .dockstore.yml like publish and latestTagAsDefault by using the value from the latest push.

Yeah, it's easy to convince myself that the topic, like the README/description, should be "per version".  If the topic is entry-level, and a workflow has multiple versions that specify different topics in the .dockstore.yml, the entry-level topic will change as the different versions are pushed...

The existing Entry topicManual and topicAutomatic fields are entry-level properties, which was a bit hard for me to wrap my head around because most fields in the .dockstore.yml are version-level. This topic .dockstore.yml field operates similar to the other entry-level properties in the .dockstore.yml like publish and latestTagAsDefault by using the value from the latest push.

Yeah, it's easy for me to convince myself that the topic, like the README/description, should be "per version". If the topic is entry-level, and a workflow has multiple versions that specify different topics in the .dockstore.yml, the entry-level topic will change as the different versions are pushed...

I convinced myself of that at first but then I started thinking that it might be repetitive if the user has to edit the topic for each version in an entry when the topic is meant to be a general description of the workflow.
We have specified in the docs before that a entry-level property should only be set in a main branch (see publish comment here), so we could do something similar to help prevent the topic from changing.
I agree that it's difficult to manage an entry-level topic if multiple versions can change it, so please weigh in if you have strong feelings on entry-level vs version-level topics in the .dockstore.yml

I agree that it's difficult to manage an entry-level topic if multiple versions can change it, so please weigh in if you have strong feelings on entry-level vs version-level topics in the .dockstore.yml

I had similar concerns about the publish property when that was added and was never entirely comfortable with it; whether an entry is published comes down to which branch/tag is processed last.

Behaviour:
The entry topicManual is set using the topic from the latest push.
If topic is non-empty, the topicManual is updated and the topicSelection is automatically set to MANUAL.
If topic is empty, i.e. topic: '', then the topicManual is reset to null and the topicSelection is automatically set to AUTOMATIC. This is to provide the user a way to reset their topicManual since they can't edit it in the UI.
If topic doesn't exist in the .dockstore.yml at all, nothing is changed. This is to prevent a version without a topic from clearing the entry's topicManual.
The user is still able to change the topic selection in the UI for GitHub app entries if they want to.

This behavior is pretty subtle. Want to topic automatic? -- specify topic: '', but don't omit the property entirely, etc. On the other hand, it's unlikely that people are going to be switching the value very often. It's worth documenting; if there isn't a followup doc ticket, could you please create one.
Despite my above concerns, the property is on the entry and belongs there, and I can't think of any better alternatives.

This behavior is pretty subtle. Want to topic automatic? -- specify topic: '', but don't omit the property entirely, etc. On the other hand, it's unlikely that people are going to be switching the value very often. It's worth documenting; if there isn't a followup doc ticket, could you please create one.

Created https://github.com/dockstore/dockstore/issues/5720, I want to put the 1.15 milestone but not sure since we wanted to keep it slim :slightly_smiling_face: @coverbeck please add it if it makes sense


put mlir sha-1 into file

Handle MLIR sha-1 in our docker file similar to that of how we handle the onnxruntime commit we track.
Currently Ubuntu22.04 and 20.04 dockers are out of sync. Will be an issue in the future with other Docker containers used.

Superceeding #1988 this with https://github.com/ROCmSoftwarePlatform/AMDMIGraphX/pull/2037

Already handled by #2037  and closed PR #1988


Add live preview feature.

Hii :wave: @smaranjitghose, It would be great for users if they can see the live preview of pdf or image adjacent to the editor. That will help the user to decide the right font-size quickly. Currently, the user can't estimate the right font-size due to the inconsistency between the page and font-size ratio in the writing editor and the generated pdf/png.

Please assign this to me, I would like to work on this feature as a GSSOC 21 participant.

Hii 👋 @smaranjitghose, It would be great for users if they can see the live preview of pdf or image adjacent to the editor. That will help the user to decide the right font-size quickly. Currently, the user can't estimate the right font-size due to the inconsistency between the page and font-size ratio in the writing editor and the generated pdf/png.

Assigned for 4 days.

@smaranjitghose Would you mind labelling this issue with GSSOC 21? I think that is the way my PR will be considered for the same.

@smaranjitghose Would you mind labelling this issue with GSSOC 21? I think that is the way my PR will be considered for the same.

Sure. However, the PRs are counted based upon the labels in the PR itself. Make sure you don't miss out on that

@smaranjitghose Would you mind labelling this issue with GSSOC 21? I think that is the way my PR will be considered for the same.

Sure. However, the PRs are counted based upon the labels in the PR itself. Make sure you don't miss out on that

Yeah sure, thanks for reminding me. Also, I checked it again and it is mandatory to label issues with gssoc21 and level. It has been stated in the MOM for the 9 March meeting. Would you mind labeling the issue with the level as well?

@smaranjitghose Would you mind labelling this issue with GSSOC 21? I think that is the way my PR will be considered for the same.

Sure. However, the PRs are counted based upon the labels in the PR itself. Make sure you don't miss out on that

Yeah sure, thanks for reminding me. Also, I checked it again and it is mandatory to label issues with gssoc21 and level. It has been stated in the MOM for the 9 March meeting. Would you mind labeling the issue with the level as well?

Already done

Closed due to inactivity


Bug: ztm evict user sometimes not work (can not reproduce consistently )

I want to reproduce case mentioned in https://github.com/flomesh-io/ztm/issues/31 consistently but failed.
But other case happened again when I invite the second endpoint's with a wrong IP adderss, evict it and then invite with right IP adderss mutiple times. Each time get same json file say user already exists. But i have run evict. I tried deleted exited json file, run evict and invite again and again but it still generate file say user already exists.
But finally, I run evict and invite again, json file generated corretly, which makes me so consufed. Mabybe it takes a while to let evict effect on db file?

@ChaiByte The invite command can only be executed once for the same user. The error Username already exists is returned from CA due to duplicated user, instead of json file exist. The content recorded in ca db file are user name and certificate.  The private key is only generated in the first invitation for specified user.
There is no delay for evict action. You can try with process below:
ztm invite home --bootstrap <IP_ADDRESS>:8888 # output correct json content
ztm invite home --bootstrap <IP_ADDRESS>:8888 # return Username already exists

ztm evict home && ztm invite home --bootstrap <IP_ADDRESS>:8888 #output correct json content

Got it. I wrote a blog post to document my usage process. Feel free to close this issue.

@ChaiByte Is this issue resolved?

@ChaiByte Is this issue resolved?

Sure.


nri-bundle has CVEs on multiple container images

Bug description
Containers images that we use from the nri-bundle have been failing security scans with trivy (https://github.com/aquasecurity/trivy) due to open CVEs. We are currently more interested in the ones listed as critical
Version of Helm and Kubernetes
helm version
version.BuildInfo{Version:"v3.5.1", GitCommit:"32c22239423b3b4ba6706d450bd044baffdcf9e6", GitTreeState:"clean", GoVersion:"go1.15.7"}
Which chart?
nri-bundle-4.8.11
The chart name and version.
What happened?
These are the images that we use from the nri-bundle which have open CVEs
newrelic/infrastructure-k8s:2.9.0
newrelic/newrelic-fluentbit-output:1.14.0
newrelic/nri-kube-events:1.8.2
What you expected to happen?
We expect the security scan to pass on all the images which we scan from the nri-bundle
How to reproduce it?
$ trivy image
Run the trivy commands listed above and inspect the CVEs listed by trivy
Anything else we need to know?
Could you upgrade the bundle to use any newly available containers images which have zero critical CVEs? Or provide mitigations for the open critical CVEs that we can use.

Hi @aroommen,
I have created a PR to add a security pipeline to the release process (#966)
The workflow run did not raise any issue: https://github.com/newrelic/helm-charts/actions/runs/3189253653/jobs/5202851197
Could you provide any report of them?

Thanks @kang-makes I see that your run is passing the following option to trivy - "--ignore-unfixed" - which could suppress some CVEs. Here is my report
trivy image newrelic/nri-kube-events:1.8.2 | grep CRITICAL
│ zlib    │ CVE-2022-37434 │ CRITICAL │ 1.2.12-r1         │ 1.2.12-r2     │ zlib: heap-based buffer over-read and overflow in inflate() │
trivy image registry.hub.docker.com/newrelic/infrastructure-k8s:2.13.2  | grep CRITICAL
│ expat   │ CVE-2022-40674 │ CRITICAL │ 2.4.8-r0          │ 2.4.9-r0      │ expat: a use-after-free in the doContent function in │
│ github.com/emicklei/go-restful │ CVE-2022-1996       │ CRITICAL │ v2.9.5+incompatible                │ 2.16.0                            │ go-restful: Authorization Bypass Through User-Controlled Key │
trivy image --skip-update   newrelic/newrelic-fluentbit-output:1.14.0 | grep CRITICAL
│ libssl1.1        │ CVE-2022-2068    │ CRITICAL │ 1.1.1n-0+deb11u2        │ 1.1.1n-0+deb11u3        │ openssl: the c_rehash script allows command injection        │
│ openssl          │ CVE-2022-2068    │ CRITICAL │ 1.1.1n-0+deb11u2        │ 1.1.1n-0+deb11u3        │ openssl: the c_rehash script allows command injection        │
│ zlib1g           │ CVE-2022-37434   │ CRITICAL │ 1:1.2.11.dfsg-2+deb11u1 │ 1:1.2.11.dfsg-2+deb11u2 │ zlib: heap-based buffer over-read and overflow in inflate()  │

@kang-makes Given that its not possible to fix it could you direct us on how to mitigate the risk from these CVEs?


Anilist error: Failed to bypass cloudflare

Steps to reproduce
Try to add or change a series' Anilist tracking.
Expected behavior
Series should be able to be edited without giving an error.
Actual behavior
Error is given out, and cannot change or add a series tracking, manually or automatically.
Crash logs
No response
TachiyomiSY version
Stable 1.10.5
Android version
Android 9
Device
Samsung Galaxy Note 8
Other details

Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a short but informative title.
[X] I have gone through the FAQ and troubleshooting guide.
[X] I have updated the app to version 1.10.5.
[X] I have updated all installed extensions.
[X] I will fill out all of the requested information in this form.

This is not related to extensions.


Texture embed wrong

Hi
i try to export from CodeWalker then import to blender but seem texture is wrong.
Export model not same color as CodeWalker model viewer
Even mesh net is not the same.
Did i do something wrong?
Prop name: stt_prop_stunt_tube_crn_5d

As you can see on the shaders, seems to be a tint value ( wrong one, but it is 👍 )

File > External Data > Find Missing Files > Select the textures folder from the object

As you can see on the shaders, seems to be a tint value ( wrong one, but it is 👍 )

Thank you may be it use tint, but i fogot to Find Missing Textures too.

File > External Data > Find Missing Files > Select the textures folder from the object

Work like a charm, thank you!


Question about SemiPriori

Hello author!
First of all, thank you for the constructive work in Category discovery.
I have a question about the way of applying SemiPriori which adds sample-wise class labels as pairwise constraints to G˜ d.
As the dataset includes labeled and unlabeled data, unlabeled data cannot add class labels. How could SemiPriori work?
Thanks in advance.

Hi, Sorry for my late reply. First of all, thanks for your interest in our work.
After we obtain the diffused graph after affinity propagation, affinities between labeled samples are noisy. Therefore, we convert the label information of labeled data into pairwise affinities and calibrate the predicted positive/negative entries on the diffused graph. Note that this operation only affects the entries of pairs of labeled data. You can see more details in our paper and codes.

Hi, Sorry for my late reply. First of all, thanks for your interest in our work.
After we obtain the diffused graph after affinity propagation, affinities between labeled samples are noisy. Therefore, we convert the label information of labeled data into pairwise affinities and calibrate the predicted positive/negative entries on the diffused graph. Note that this operation only affects the entries of pairs of labeled data. You can see more details in our paper and codes.


Posix code coverage

Added some  APIs into posix test cases to incease code coverage into :

POSIX  timer test
POSIX pthread  RW Lock
there was a bug in timing calculation for timer_gettime, it was also fixed .

@youvedeep is this required for 1.11?

Codecov Report

Merging #6499 into master will increase coverage by 0.08%.
The diff coverage is 72.72%.

@@            Coverage Diff             @@
##           master    #6499      +/-   ##
==========================================
+ Coverage   53.24%   53.32%   +0.08%
==========================================
  Files         435      435
  Lines       41292    41307      +15
  Branches     7919     7923       +4
==========================================
+ Hits        21986    22028      +42
+ Misses      16099    16066      -33
- Partials     3207     3213       +6

Impacted Files
Coverage Δ

kernel/posix/timer.c
55.42% <100%> (+8.43%)
:arrow_up:

tests/posix/pthread_rwlock/src/posix_rwlock.c
85.52% <61.53%> (-5.52%)
:arrow_down:

tests/posix/timer/src/posix_timer.c
87.8% <83.33%> (-0.77%)
:arrow_down:

kernel/posix/pthread_rwlock.c
81.05% <0%> (+27.36%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1566d0f...d40f6f1. Read the comment docs.

This fixes one bug in POSIX implementation. If we can pull in for rel 1.11 it will be good.
Else it is ok for 1.12 too as it is mostly for getting better code coverage.


cloudflare-speedtest: Add version 2.2.5

[x] I have read the Contributing Guide.

/verify

https://github.com/chawyehsu/dorado/pull/930


Collect custom attributes and tags

Hi again!
I've been working around issue #106.

Unfortunately, I was not able to keep only using pyVmomi and I had to use Vmware Python SDK as well so as to collect tags.

That is in the past. I changed every code snippet using vmware SDK to direct requests to the vsphere rest api. I hope that is ok.
Please take a look.

Full disclosure - the project i was using vmware_exporter for is now k8s bare metal w/ kata, so I don't have a test cluster for this and my head is out of the game.
It's a big diff and haven't given't it a thorough review but seems like a reasonable change. I'd probably want to test it against a cluster before merging it, if I still had one.
My biggest concern would be vmware's code, TBH. It's really grim that vmware can't manage to put an wheel on pypi for their library in something like 2.5 years (see https://github.com/vmware/vsphere-automation-sdk-python/issues/38). My gut reaction is if they can't manage that, then what other python basics are they not doing? Anyway I poked around and apparently they feel like they can't list it on pypi cause its not open source enough. The community seems dubious about the validity of this, but i can't see them give any other rationale for not doing it. If they think it's not open source enough to list on pypi maybe we need to consider whether it's an appropriate dependency for vmware_exporter.
If you don't think the licencing is an issue, then the next thing would be pinning the github dependency to a known sha so if they push a regression or deprecation we don't pick it up immediately by surprise.
I'm also not sure about referencing a github commit from setup.py (requirements.txt gets hoovered into setup.py). I generally avoid doing depending on unreleased stuff, so I have no idea if it actually works. So we should probably test that it can generate valid wheels still.

I've been testing on a DC with more than 6k vms,  60 hosts, 8 clusters and
around 260 ds. Runs pretty good to me. I can't see much difference.
Em sáb, 25 de abr de 2020 12:44, Daniel Pryor<EMAIL_ADDRESS>escreveu:

@pryorda commented on this pull request.
This looks pretty good from my angle.  Did you see much of a difference in
performance while fetching attributes?
Thanks for our contributions and feature adds :)
In vmware_exporter/helpers.py
https://github.com/pryorda/vmware_exporter/pull/194#discussion_r415086469
:

from pyVmomi import vmodl

+def get_unverified_session():

"""
create session to connect to vmware API
from Vmware Samples
"""
session = requests.session()
session.verify = False

Can you make this a defaulted variable?
In vmware_exporter/helpers.py
https://github.com/pryorda/vmware_exporter/pull/194#discussion_r415086635
:

@@ -15,6 +27,19 @@ def batch_fetch_properties(content, obj_type, properties):
recursive=True
)

"""

   Gathering all custom attibutes names are stored as key (integer) in CustomFieldsManager

   We do not want those keys, but the names. So here the names and keys are gathered to

   be translated later

"""
allCustomAttributesNames = dict(

Can you add an example of what this looks like as a metric?
In vmware_exporter/vmware_exporter.py
https://github.com/pryorda/vmware_exporter/pull/194#discussion_r415086863
:

     self.host = host

     self.username = username
     self.password = password
     self.ignore_ssl = ignore_ssl
     self.collect_only = collect_only
     self.specs_size = int(specs_size)

   # Custom Attributes

   # flag to wheter fetch custom attributes or not

   self.fetch_custom_attributes = fetch_custom_attributes

   # vms, hosts and datastores custom attributes must be stored by their moid

   self._vmsCustomAttributes = {}

   self._hostsCustomAttributes = {}

   self._datastoresCustomAttributes = {}

   # Tags

   # flag to wheter fetch tags or not

   self.fetch_tags = fetch_tags

   # label names and ammount will be needed later to insert labels from custom attributes

   self._labelNames = {

I like what you did here.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/pryorda/vmware_exporter/pull/194#pullrequestreview-400404701,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AO5BSPNTSXGJNMUOE6W7EVTROMAMLANCNFSM4MQOFKKQ
.

About the license:
https://github.com/vmware/vsphere-automation-sdk-python/blob/<IP_ADDRESS>/LICENSE
It is MIT.
Em sáb, 25 de abr de 2020 13:33, Jc2k<EMAIL_ADDRESS>escreveu:

Full disclosure - the project i was using vmware_exporter for is now k8s
bare metal w/ kata, so I don't have a test cluster for this and my head is
out of the game.
It's a big diff and haven't given't it a thorough review but seems like a
reasonable change. I'd probably want to test it against a cluster before
merging it, if I still had one.
My biggest concern would be vmware's code, TBH. It's really grim that
vmware can't manage to put an wheel on pypi for their library in something
like 2.5 years (see vmware/vsphere-automation-sdk-python#38
https://github.com/vmware/vsphere-automation-sdk-python/issues/38). My
gut reaction is if they can't manage that, then what other python basics
are they not doing? Anyway I poked around and apparently they feel like
they can't list it on pypi cause its not open source enough. The community
seems dubious about the validity of this, but i can't see them give any
other rationale for not doing it. If they think it's not open source enough
to list on pypi maybe we need to consider whether it's an appropriate
dependency for vmware_exporter.
If you don't think the licencing is an issue, then the next thing would be
pinning the github dependency to a known sha so if they push a regression
or deprecation we don't pick it up immediately by surprise.
I'm also not sure about referencing a github commit from setup.py
(requirements.txt gets hoovered into setup.py). I generally avoid doing
depending on unreleased stuff, so I have no idea if it actually works. So
we should probably test that it can generate valid wheels still.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/pryorda/vmware_exporter/pull/194#issuecomment-619405929,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AO5BSPIUN37UC2JLJYIXKOLROMGGJANCNFSM4MQOFKKQ
.

Well that’s exactly why the community is confused about the lack of a pypi release. They say it is MIT just as you do, but the VMware team themselves also say it is not open source enough to go on pypi. They haven’t provided much clarity on what the problem is. Just that it’s a legal one. So maybe only a part of the release is MIT?

I guess I will code some direct access to VMware api.
Em sáb, 25 de abr de 2020 14:49, Jc2k<EMAIL_ADDRESS>escreveu:

Well that’s exactly why the community is confused about the lack of a pypi
release. They say it is MIT just as you do, but the VMware team themselves
also say it is not open source enough to go on pypi. They haven’t provided
much clarity on what the problem is. Just that it’s a legal one. So maybe
only a part of the release is MIT?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/pryorda/vmware_exporter/pull/194#issuecomment-619416494,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AO5BSPN3CPCPSXAYANO5633ROMPB3ANCNFSM4MQOFKKQ
.

Well that’s exactly why the community is confused about the lack of a pypi release. They say it is MIT just as you do, but the VMware team themselves also say it is not open source enough to go on pypi. They haven’t provided much clarity on what the problem is. Just that it’s a legal one. So maybe only a part of the release is MIT?

You're right. Definitely there is something weird. They say there might be a license issue with the code generator they use but it does not seem to be really. They would not share the code on github if it were not "open" enough. And 2 years is quite enough term to at least propose a solution.
Anyways... I've replaced their python sdk by direct request to vsphere rest api. I hope it is a bettter approach.

Hey! I had to make some minor adjustment:

I missed the get_env_bool;
I replaced vmware sdk by rest api direct requests for tag fetching;
I sent an email about this third change. The collector deferred methods, when the result is anything that can be coerced to False, hangs the method. I made some change on that, and I would like you to check.

With those change above, you will need to make a new release, maybe v0.12.1 ? :-P

nice job

I am no git expert. Please check if commits are squashed as requested.

@pryorda Am I missing something? How can I help?

@pryorda Am I missing something? How can I help?

Sorry about the delay. I'm testing this out now.


Too many false positives

Is there a way to disable errors for words it does not recognize?
For example when I am writing technical content, with peoples names I get hundreds of occurrences that I would have to add to my dictionary or ignore list. Is there a way to only check words that he has confidence or suggestions?

@rtfpessoa See the options described in the readme: personal and ignore seem relevant.

Like I mentioned in my question:

I get hundreds of occurrences that I would have to add to my dictionary or ignore list

In each text I write I need to add more that 10 words to the dictionary. And I would just prefer to have only warning for words that retext-spell knows are wrong or at least has some confidence and not just all the words it does not know.

@rtfpessoa That seems to be an issue with the dictionary you use.

only warning for words that retext-spell knows are wrong or at least has some confidence and not just all the words it does not know.

How else do you propose spell checkers / dictionaries work? 🤔

Hmm, well, I don’t think other spell checkers accept these either (they all essentially have a big list of valid words). All these words are jargon or names of things, not “normal american english”. This is not a bug, it’s a feature. And if you don’t want to get warned by them, place them in a personal dictionary! 🤷♂

Ok, thank you.


"Reference in new issue" original-poster tagging not working

Gitea Version
1.15.0
Git Version
2.27.0
Operating System
Windows Server 2019
How are you running Gitea?
We used the download from the Gitea website and are running it as a service on Windows, as described in the installation instructions.
Database
SQLite
Can you reproduce the bug on the Gitea demo site?
No
Log Gist
No response
Description
When using the "Reference this issue"-feature for an existing issue, it creates a new issue by quoting the previous one. At the bottom of the issue, there is a reference to the original author. However, in our case, it does not tag it correctly: it takes the display name (so in my case it would print @Jelle%20Hulter instead of @jellehulter using my username).
I was unable to reproduce it on the demo site, somehow it shows the username there for issues instead of the display name/full name which is attached to the account. Tried committing a commit which used the same email as the Gitea account but that didn't help either.
After this, I noticed that our Gitea configuration had the DEFAULT_SHOW_FULL_NAME setting set to true which caused this change. Apparently, the "Reference this issue"-window uses this setting to determine if it should use either the username or the full name, while this should be just the display name.
Maybe this has already been fixed in the meanwhile, I tried taking a look in the PR and issue history but I was unable to find a similar issue.
Screenshots

Here, the username would be expected instead of the full name (and hence still the percentage-sign encoding of a space)

After cloning and building it myself, I can confirm it is still the case in Gitea version 1.16.0+dev-252-gf96d0d3d5:


Revert "Create collate file for yew onn"

Reverts CS2103AUG2017-F10-B2/main#65
Reason: Code did not pass all three test.

Changes Unknown when pulling 8ed1578e8cb0cd1cb1a0c00578f5e3912ee674d7 on revert-65-Create_collate_file_for_Yew_Onn into ** on master**.


Feature Request: Improved ChatGPT Integration

I'm currently using ObsidianCustomFrames to embed OpenAI's ChatGPT website in Obsidian.
It works nicely as far as it goes.
However, they have recently added the facility to save the contents of multiple conversations in their "cloud".
It occurs to me that it would be very nice to be able to trigger some mechanism (perhaps something like running a snippet in a regular browser) which would take the contents of the currently displayed conversation and write them (nicely formatted in markdown) to a markdown file in the Obsidian vault named for the current sidebar title of the conversation.
That would allow for convenient offline backup of valuable conversations and assist with weeding out completed or throwaway conversations from the "cloud" store.
It seems that this would probably require the ability to run custom Javascript in the embedded iFrame, which looks to be on your roadmap.

Sorry, but I'm not interested in adding additional compatibility for specific websites.
As you mentioned, the custom JavaScript feature on the roadmap will allow you to add a custom snippet or button to a custom frame, which might be able to help with what you're trying to achieve.


add support for Aliyun images

This adds support for images uploaded to the Aliyun cloud.
Mostly a copy/paste of the existing AWS types and methods.
Closes: #29

I imagine there is a better way to handle the similarities between AWS + Aliyun, but my Go isn't that great these days, so I brute forced it

(Marked as Draft in case there is a better solution that should be pursued)

cc @bgilbert who may have opinions

Next step is a PR to bump this vendored version in coreos-assembler btw.


[release-1.3] mem-hotplug: fix target pod requests calculation with hugepages

Manual backport of https://github.com/kubevirt/kubevirt/pull/12168.
This fixes a bug where you cannot hotplug successfully when using hugepages due to virt-launcher being tricked into thinking that the target pod does not have enough memory to accomodate the new VMI with additional hotplugged memory.

What this PR does
Before this PR:
Doing memory hotplug with hugepages gets the hotplug migration stuck indefinitely.
After this PR:
Doing memory hotplug with hugepages works as expected.

Fixes #
Why we need it and why it was done in this way
The following tradeoffs were made:
The following alternatives were considered:
Links to places where the discussion took place:
Special notes for your reviewer

Checklist
This checklist is not enforcing, but it's a reminder of items that could be relevant to every PR.
Approvers are expected to review this list.

[ ] Design: A design document was considered and is present (link) or not required
[ ] PR: The PR description is expressive enough and will help future contributors
[ ] Code: Write code that humans can understand and Keep it simple
[ ] Refactor: You have left the code cleaner than you found it (Boy Scout Rule)
[ ] Upgrade: Impact of this change on upgrade flows was considered and addressed if required
[ ] Testing: New code requires new unit tests. New features and bug fixes require at least on e2e test
[ ] Documentation: A user-guide update was considered and is present (link) or not required. You want a user-guide update if it's a user facing feature / API change.
[ ] Community: Announcement to kubevirt-dev was considered

Release note

NONE

/hold
Have to wait for https://github.com/kubevirt/kubevirt/pull/12146 first.

@acardace Can you have a look at the rebase?

@xpivarc I will, but after https://github.com/kubevirt/kubevirt/pull/12146 gets in.

/unhold
@xpivarc PR is rebased now.

/retest


Poor MathML output

Issue Summary
The mchem packages produces very ugly MathML when there is a much better (and better for assistive technology) solution.  Solving this would solve #22. Technically, this is not a bug report, but the upshot is that with the current output, recognition and proper voicing of the chemistry output is extremely difficult.
At the end of this issue is a relatively simple example that generates poor MathML output. It has (at least) four problems:

there are extra mrows,
multiple elements are in a single mi
mphantom and mpadded are needlessly used
the sup/super scripts are not attached to the chemical element they adorn

Let me go through these one by one...
The first one (extra mrows) is pretty obvious. In the output below, there is:
      <mrow data-mjx-texclass="ORD">
        <mi data-mjx-auto-op="false">SO</mi>
      </mrow>

There is no need for the mrow.
The second is also pretty obvious -- "SO" is in a single mi. That's a misuse of mi and is equivalent to putting x and y into a single mi and then making that italic (via mathvariant="italic") so that it looks ok. Given this is a chemistry package, is it hard to recognize an element and put it in its own mi? If it is a single letter, then the default italics are overridden with mathvariant as in
    <mi mathvariant="normal">S</mi>

It is valid to add mathvariant="normal" to <mi>Na</mi> if that makes generation simpler (i.e., you can always add it to any chemical element).
The third problem's solution is maybe less obvious. mphantom and mpadded are being used to force the scripts to all have the same vertical position. But MathML has a much simpler solution for this usage: mmultiscripts.  From the MathML spec:

This element allows the representation of any number of vertically-aligned pairs of subscripts and superscripts, attached to one base expression. It supports both postscripts and prescripts. Missing scripts must be represented by the empty element none. All of the upper scripts should be baseline-aligned and all the lower scripts should be baseline-aligned.

Since all chemical elements have a capital letter, all scripts will be horizontally aligned.
An example of the better output, which also attaches the scripts to the element (and therefore solves the last problem) is:
<math>
    <mi>S</mi>
    <mmultiscripts>
        <mi>O</mi>
        <mn>4</mn>
        <mrow>
            <mn>2</mn>
            <mo>−</mo>
        </mrow>
    </mmultiscripts>
</math>

I've removed the MathJaX generated attributes for clarity. Compare this to the mchem/MathJax output below.
This is also a better solution for the prescripts used in nuclear chemistry.
There is a problem for electron, proton, etc., and scripts because the base doesn't have a capital letter.
The mphantom/mpadded trick could be used here (inside an mrow around the base element), but is not ideal.
This leads to an alternative to mmultiscripts: use msub/msup/msubsup with the attributes subscriptshift and superscriptshift. There are two problems with this approach:

you need to figure out the amount of shift that is appropriate (use font relative units).
MathML Core does not support these attributes (MathJaX does). This means that when someday MathJaX makes use of MathML in an output jax, it will need to do something (probably ugly) to make this work.

Finally, here is the example: \ce{SO4^2−}
<math xmlns="http://www.w3.org/1998/Math/MathML">
  <mstyle mathcolor="#a33e00">
    <mrow data-mjx-texclass="ORD">
      <mrow data-mjx-texclass="ORD">
        <mi data-mjx-auto-op="false">SO</mi>
      </mrow>
      <msub>
        <mrow data-mjx-texclass="ORD">
          <mrow data-mjx-texclass="ORD">
            <mpadded width="0">
              <mphantom>
                <mi>A</mi>
              </mphantom>
            </mpadded>
          </mrow>
        </mrow>
        <mrow data-mjx-texclass="ORD">
          <mrow data-mjx-texclass="ORD">
            <mpadded height="0">
              <mn>4</mn>
            </mpadded>
          </mrow>
        </mrow>
      </msub>
      <msup>
        <mrow data-mjx-texclass="ORD">
          <mrow data-mjx-texclass="ORD">
            <mpadded width="0">
              <mphantom>
                <mi>A</mi>
              </mphantom>
            </mpadded>
          </mrow>
        </mrow>
        <mrow data-mjx-texclass="ORD">
          <mn>2</mn>
          <mo>−</mo>
        </mrow>
      </msup>
    </mrow>
  </mstyle>
</math>

Thanks, @NSoiffer, for your long elaboration. But I'd like to contradict in some important points.
Your "example of the better output" (JsFiddle) is nonsense. It renders as

and that is wrong on many levels.

"SO" must be upright (otherwise it would be variables)
the "2–" must not be above the "4", according to the rules, e.g. IUPAC.
It looks semantically wrong, because it attaches "2–" to "O" in the XML structure

Just because JAWS has a bug and reads out phantom content (#22), is not a good reason to resort to sizes that are font-dependent and will lead to wrong renderings with different fonts.
I agree that the generated MathML looks a bit convoluted, like double mrows. Maybe we could improve a bit there. But please be aware that we have a mhchem→TeX→MathML chain, i.e. we do not go from mhchem syntax to MathML directly.
For assistive technologies, I would make the bold claim that using the mhchem syntax for Braille and read-out would be much better. It gets complicated when mhchem and real math are mixed, though, like in $m_{\ce{H2O}}$ or $\ce{Fe(CN)_{$\frac{6}{2}$}}$.

I'm not sure how I ended up with that mmultiscripts example as it is wrong (as you point out). The proper one (using the mathvariant attr I said in the text should be used) is:
<math>
    <mi mathvariant="normal">S</mi>
    <mmultiscripts>
        <mi mathvariant="normal">O</mi>
        <mn>4</mn>
        <none/>
        <none/>
        <mrow>
            <mn>2</mn>
            <mo>−</mo>
        </mrow>
    </mmultiscripts>
</math>

This also pushes the 2- out as it should have been. See the codepen rendering.

For assistive technologies, I would make the bold claim that using the mhchem syntax for Braille and read-out would be much better.

I'm a little dubious about that. It can certainly be done, but MathML has more structure which simplifies things and one doesn't have to have a full TeX interpreter to deal with all the hacks people do in TeX. What TeX has are macro names which can give semantic information for speech (e.g., binom). MathML 4 is adding an intent attr which some more semantic TeX implementations such as PreTeXt  has said they will experiment with generating.

Thanks. That looks much better.
I would even claim that semantical grouping should be
<math>
    <mmultiscripts>
        <mrow>
            <mi mathvariant="normal">S</mi>
            <mmultiscripts>
                <mi mathvariant="normal">O</mi>
                <mn>4</mn>
                <none/>
            </mmultiscripts>
        </mrow>
        <none/>
        <mrow>
            <mn>2</mn>
            <mo>−</mo>
        </mrow>
    </mmultiscripts>
</math>

but that would require semantic knowledge that the mhchem syntax does not contain. (JsFiddle)

The typographic subtleties, that the phantom solves, are not done here. I want to have all subscripts and superscripts on the same height. Not even with A and O, the superscripts are the same height. (JsFiddle)

For assistive technologies, I would make the bold claim that using the mhchem syntax for Braille and read-out would be much better.

I'm a little dubious about that.

My counter argument: There is a reason that chemistry books in Braille use a different (shorter) notation for chemistry than they do for generic math.

Okay, back to the question if we can simplify the MathML output, e.g. remove the double rmow. An improved rendering would require a change in how MathJax works. We would need to change the mhchem→TeX math→MathML chain into something that skips TeX math. Or maybe change the TeX math interface, so that it includes a special \multiscripts{base}{left-sub}{left-sup}{right-sup-presub}{right-sub}{right-sup-stacked}{right-sup-postsub} that has a simpler MathML output. I don't know how easily this would be possible. Including @dpvc.

First, about the extra mrow elements:  MathJax's internal representation keeps information about the original TeX input that is needed in order to get the spacing right (among other things), and when MathJax produced MathML output, it includes nodes and classes that carry that information.  That way, if MathJax's MathML output is used as input to its MathML input jax, it can reproduce the internal format that would have come from the original TeX input.  The <mrow data-mjx-texclass="ORD"> and similar mrow elements are used for that purpose, so are not redundant from that standpoint.  If these are removed, that could lead to incorrect layout, since the needed information will be missing.  It is true that some of them could be removed without problems, so MathJax could prune some of these when it generates MathML output; mhchem should not have to worry about such things.
In terms of the example you have been looking at, while I am not a chemist, by understanding of the sulphate ion given by \ce{SO4^2-} is that the subscript 4 goes with the O, as there are 4 oxygen atoms and one sulphur atom, and that the superscript 2- indicates that the ion as a whole has a charge of -2.  That is, the superscript is for the whole ion, while the subscript is for the oxygen alone, as in (S(O_4))^2-.  Neil's representation using mmultiscripts with a base of O would put the superscript and subscript both on the oxygen, which is a different meaning:  that there are 4 oxygen and each has charge -2.  Again, not being a chemist, I don't know which of these is correct, but it seems to me that the offset superscript is meant to signify that the 2- goes with the whole ion, without having to use parentheses.
In that case, the LaTeX markup naturally would be
{\mathrm{S}\mathrm{O}_4}^{2-}

yielding
$$\Large {\mathrm{S}\mathrm{O}_4}^{2-}$$
with MathML (simplified):
<math xmlns="http://www.w3.org/1998/Math/MathML" display="block">
  <msup>
    <mrow>
      <mi mathvariant="normal">S</mi>
      <msub>
        <mi mathvariant="normal">O</mi>
        <mn>4</mn>
      </msub>
    </mrow>
    <mrow>
      <mn>2</mn>
      <mo>−</mo>
    </mrow>
  </msup>
</math>

This corresponds to your last example with nested mmultiscripts nodes, though there is no need for mmultiscripts in this case, as it is being used just to get individual subscripts and superscripts, where msub and msup would do just fine (with one caveat that I will discuss shortly).  If you were using prescripts, then mmultiscripts would be useful for that.  MathJax does have two macros that handle that:  \sideset in the AMS extension, and \prescript in the mathtools extension.  Note that \sideset is supposed to be used only for a base that is a large operator, though MathJax doesn't enforce that, and that the prescripts are left-justified for this macro (with \prescript they are right-justified).  Neither of them supports more than one pre- or postscript of each type.  A more general mechanism for producing mmultiscripts elements could certainly be implemented.
The caveat is that the positioning of a subscript when there is no superscript may be higher than when there is a superscript.  That is the position of the 4 in {\mathrm{0}_4}^{2-} is different from that in \mathrm{O}_4^{2-} as shown in:
$$\Large {\mathrm{0}_4}^{2-}\quad \mathrm{0}_4^{2-}$$ whereas the positioning using mmultiscripts will always be the lower position, regardless of whether there is a superscript.  One can get the same effect visually with {\mathrm{O}_4^{}}^{2-}, though this will probably mess up accessibility.
In terms of placement of superscripts, if the base is a single letter (or a letter with certain accents), the placement should be the same regardless of the letter.  In your example of \mathrm{A}^8\mathrm{O}^8, the CHTML output is
<mjx-math display="true" style="margin-left: 0px; margin-right: 0px;" class="MJX-TEX" aria-hidden="true">
  <mjx-msup>
    <mjx-mi class="mjx-n">
      <mjx-c class="mjx-c41"></mjx-c>
    </mjx-mi>
    <mjx-script style="vertical-align: 0.413em;">
      <mjx-mn class="mjx-n" size="s">
        <mjx-c class="mjx-c38"></mjx-c>
      </mjx-mn>
    </mjx-script>
  </mjx-msup>
  <mjx-msup>
    <mjx-mi class="mjx-n">
      <mjx-c class="mjx-c4F"></mjx-c>
    </mjx-mi>
    <mjx-script style="vertical-align: 0.413em;">
      <mjx-mn class="mjx-n" size="s">
        <mjx-c class="mjx-c38"></mjx-c>
      </mjx-mn>
    </mjx-script>
  </mjx-msup>
</mjx-math>

Note that both mjx-script tags have the save 'vertical-align' value and content, and so they should be positioned the same.  Unfortunately, not all browsers are very accurate about these things, and so they don't always produce exactly the same positioning.  There is very little MathJax can do about that.  Here's what I see for this example in your jsFiddle as well as in my own pages:

which is nicely aligned for me.  When the base is not a single letter (like if it were a letter with a superscript already), then the position may be different but for plain letters, they should all line up.  That should also be true for subscripts, as long as they either all have no superscript or all have a superscript.

Let me lead with that I'm not a chemist and haven't taken a chemistry course in decades, so I blew it with the structure/semantics of \ce{SO4^2-} and mmultiscripts.
@mhchem wrote:

My counter argument: There is a reason that chemistry books in Braille use a different (shorter) notation for chemistry than they do for generic math.

The only reference I found for changes to Nemeth for chemistry is one that shortens some arrows and adds some symbols. It doesn't change the rules for sub/superscripts (Nemeth does have a special rule/shortcut for numeric subscripts). For UEB, the guidelines don't change the output for chemistry at all. I've implemented both in MathCAT, but not the arrow modifications for chemistry. Recognizing chemistry is needed first, and that is what I'm working on. If you saw a reference that I missed, please let me know.
As with @dpvc, I don't see a difference for the A^8 O^8 example you provided -- the superscripts are aligned on both Chrome and Firefox in windows for me.
@dpvc  wrote:

One can get the same effect visually with {\mathrm{O}_4^{}}^{2-}, though this will probably mess up accessibility.

Works fine for MathPlayer, but your mileage may vary with other math-to-speech transforms as they tend to be more simplistic (as is apparently JAWS with mphantom).
@dpvc: thanks for you detailed response. The point of using mmultiscripts vs msub (etc) is the alignment of the scripts even in the absence of a sub/superscript, but your solution of an empty script might work. I'm not clear though on whether you are suggesting that @mhchem can use some of the simpler markup you showed or not.

The point of using mmultiscripts vs msub (etc) is the alignment of the scripts even in the absence of a sub/superscript

Yes, that was why I gave my caveat above.  It would certainly be possible to make a macro that produces mmultiscripts for a subscript rather than msub.

I'm not clear though on whether you are suggesting that @mhchem can use some of the simpler markup you showed or not.

Yes, I think that notation is already available and can be used without additional macros being defined.  Although the empty superscript hack works visually, I would probably recommend the mmultisctipts approach if you always want the lower positions for the subscripts.  This would require a new macro, as \sidesset and \prescript both end up producing plain msub/msup/msubsup if there are no prescripts.
@NSoiffer, I'm not sure if you are aware of the work-flow for mhchem, which is that it takes the contents of \ce{...} and converts it to another TeX string and then that is parsed by MathJax.  The mhchem package does not produce MathML directly itself, so most of the MathML suggestions are not something mhchem would do directly, other than by changing the TeX that it uses in order to end up with the better MathML (e.g., using \mathrm{S}\mathrm{O} rather than \mathrm{SO}).
Note also that @mhchem is the author of the original LaTeX package as well as MathJax's extension.  When he first wrote the extension (replacing my crappy one from long ago), he wanted to make sure that both LaTeX and MathJax were consistent, and so rewrote the LaTeX package so that both would use the same algorithms internally.  Since the LaTeX package must produce LaTeX (not MathML), keeping the package entirely in terms of TeX notation is an important factor, here.  While it would be possible to make new macros that produce mmultiscripts for MathJax output, Martin would have to provide LaTeX-based definitions of the same macros in order to have mhchem work in actual LaTeX.  So using as much standard LaTeX as possible is also an advantage.
My concern about using ^{} is that Volker's SRE currently will end up with "to the power" with nothing afterward.  That can probably be fixed, but it currently doesn't handle this well.

Going the side-path of "accessible chemistry".

The only reference I found for changes to Nemeth for chemistry is one that shortens some arrows and adds some symbols.

Erm, if you jump from section 3 to section 7, you will see that this is a completely different notation.
\ce{5 C2O4^{2-}(aq)} = #5,c2,o4"^2-"(aq)
This notation omits the indication of uppercase letters, subscripts, etc. (It's not quite clear how exactly this will translate to Braille characters. 6-dot Braille needs two characters for numbers, 8-dot braille (Eurobraille) needs just one.)
Maybe you could provide the Braille representation of the corresponding MathML, but I am sure it will be quite different and more lengthy.
Besides special mathematical Braille systems (English-speaking countries: Nemeth Braille, German-speaking countries: 6 dot Braillemathematikschrift or 8 dot SMSB), LaTeX seems to be the choice for a lot of people. Searching with English terms is tricky, but German return several pages of spot-on search results (search for: latex als mathematikschrift für blinde). If LaTeX is used for mathematics, it would also work well for chemistry, in particular with the mhchem (\ce) syntax, I'd reckon.
Disclaimer: While I have a good knowledge about chemical typography, I forgot everything about what these notations actually mean or how you would pronounce it. Also, what I know about Braille is second-hand knowledge from the Internet.

@dpvc

he wanted to make sure that both LaTeX and MathJax were consistent, and so rewrote the LaTeX package so that both would use the same algorithms internally.

I have to admit this isn't true. mhchem for LaTeX and mhchem for MathJax work differently internally. The MathJax version is much more forgiving for sloppy input (partly because of your first implementation :-)) and – blame on me – it supports a few more special notation than the LaTeX version. Nobody complained yet and motivated me.

I have to admit this isn't true.

I stand corrected, and apologize for the misinformation.  Thanks for the correction!

@NSoiffer I am confused what you mean by poor MathML. If two representations
generate the same typeset expression are they not equivalent? There might be
good vs bad styles, similar Syntax of programming languages. If you are trying
to infer semantics exclusively on the structure of syntax, you will always be at
the mercy of whatever software has generated that syntax. For example, I would
assume mrow elements have only grouping functions, similar to groups in SVG,
and while they might add styling information they should not carry any
"meaning". SRE would simply strip superfluous mrows, rewrite multiscipts
without left indices into sub/superscripts, etc.
Also why don't you simply use the more informative LaTeX notation here, instead
of throwing out the knowledge, and then try to regain it by interpreting the
MathML. A few year's back I experimented with some chemistry rules on the basis
of mhchem in SRE and ultimately wrote some for the LaTeX representatinos
directly rather than any intermediary MathML. Hence they are not yet in use, and
the maths interpretation is what you currently see, as @dpvc has pointed out.
However, I hope to use them soon, as we are currently moving towards using LaTeX
more directly in SRE (the [latex-to-speech] package is a first step).
This is in particular in order to support accessibility. As @mhchem pointed out
the Nemeth use case is rather small, given that over the last decades in a
number of European countries, visually impaired children are being taught LaTeX
directly, starting in primary school, which particularly helps communicating
mathematics with non-specialist teachers and thus furthers inclusive
education. In some of these countries (notably Germany and Slovakia, but others
are sure to follow) specialist Braille systems have been superseded entirely by
translating LaTeX code into 8-dot Euro Braille. I feel it would be a terrible
precedent to ignore these stakeholders in order to enable some rare symbols in
Nemeth that will -- let's be honest -- ultimately only be used and understood by
very few people.
Now, whether LaTeX will also be used for chemistry and if so if it will be
mhchem I would not know... But I am currently working with some people in
Baden Wuerttemberg, so I am sure going to find out.

Apologies, the earlier replies got buried in my mail stream. So a little catch up:
@dpvc: thanks for the history/description of how mhchem and mathjax work together. The correction by @mhchem indicates that maybe some other macros would produce simpler MathML.
@mhchem re braille:

\ce{5 C2O4^{2-}(aq)} = #5,c2,o4"^2-"(aq)
This notation omits the indication of uppercase letters, subscripts, etc.

Maybe you were a little hasty in looking at the RHS: dot 6 ("," in ASCII braille) is the capitalization char in Nemeth (and UEB). You'll notice that it is before the 'c' and the 'o'.
Also, simple numeric subscripts in Nemeth (regardless of it being chemistry) skip the subscript indicator. It's an optimization in Nemeth but not UEB. It does lead to a complication though when the subscript is followed by a number, so a special rule is needed for that case. Many translators that I tried fail to handle that special case correctly. If you look at example 7.1-3, you see that there is a subscript indicator (dots 5-6/";" in ASCII braille). Nemeth code always states the nesting of the sub/superscripts, so you'll two subscript indicators before the "s" and "p". So again, I don't see anything special for Nemeth code for chemistry here.
@zorkow was on a call today with me and others. I was aware Germany and a few other European countries were teaching LaTeX in grade school. However, I learned from @zorkow today that there is a move to use (a predefined subset of) LaTeX for the braille math and that is replacing the old braille math code. With 8 dot braille, that makes a lot of sense. I do agree that assuming they expand that to include mhchem, that would be a real boon for accessibility since your design is clear and concise.
As @zorkow also pointed out, maintaining the TeX via semantics/annotation in MathML would make it easier to produce the braille for the LaTeX assuming it started life as TeX (and also that it didn't have unknown macros/weird redefinition of the syntax).

@NSoiffer I am confused what you mean by poor MathML. If two representations generate the same typeset expression are they not equivalent?

@zorkow: by "poor" I mean significantly more complicated than needed. It is definitely not the case that two notations that produce the same rendering are equivalent. Display may be by far the most important usage, but accessibility and search are other usages. It is sort of like saying two programs are the same if they produce the same result, One might take a linear amount of time and the other an exponential amount of time or one might be 20 lines of code and the other 1,000 lines. But I think this type of discussion is not productive and hopefully you don't feel compelled to extend it.
I do agree that there is good semantic info in the macro being used. The Math WG is adding an intent arg to MathML to capture exactly this type of information ("author intent"). I have focused the design of MathCAT around the idea of "intent" and others are working on generating intent from TeX, so we should have examples of the workflow that goes from authors to AT by the time the MathML 4 spec become a recommendation (which will be a while). If mhchem/MathJax eventually generates intent, it would make it much easier to reliably generate good speech for all AT.

Sorry, for replying so late, I was on holidays.
I agree with @Nsoiffer, we should not protract this discussion; so here are a few, final remarks from my site:

Display may be by far the most important usage, but accessibility and search are other usages.

All three points are debatable:

I am not aware of many systems that actually use MathML as a basis for display.

MathML is NOT accessible out of the box. Otherwise, why the need for specialised systems (or intent) so preprocessing has to happen, no matter what. In fact, the design decision I regretted the most in SRE was to base it's input on MathML rather than something more powerful.

While I doubt that there are many use cases for formula search, beyond secondary school math before exams, all half-way working search engines (e.g. searchonmath) are based on LaTeX. And even for MathML I would assume some canonicalization would be important.

It is sort of like saying two programs...

Poor comparison, in my opinion, as any "mess" added to a MathML expression will only add constant overhead to the XML parsing. But I am not an expert on the MathML spec and their might be hidden traps.
What I find more worrying is that this good XML property is now discarded with intent. From what I have learned so far about the intent attribute is that now strings with holes (variables) have to be understood by AT, adding unpredictable complexities via

a need for additional string parsing
evaluation of variables
lookup in the DOM, which is not compatible with recursive descent
potential risk of circularity

In many ways we already have a version of this idea in MathJax, using the semantic embeddings via O(1) data attributes. When we first published
that we had some severe pushback as "this was not in line with the MathML spec".
But now MathML is trying to reinvent that via programming in strings?
And, if they are serious about intent, then surely that could be applied to any DOM structure as we do in MathJax. Why the insistence on a single input language like MathML?
In the call @NSoiffer mentions, @NSoiffer suggested that I would try to break things to "retain job security" with MathJax. I do find this a tad insulting; none of my math work was ever done for commercial gain. Everything was always open source and transparent.
As such I strongly object to the idea of enforcing a particular format simply because it serves current commercial technologies, instead of facilitating a paradigm shift in math accessibility, which might be far more helpful for the actual target audience (i.e., the next generation of visually impaired students).  In particular, pushing a "standard" as a requirement for accessibility, even if that is detrimental to accessibilty efforts in non-Anglosaxon parts of the world, is in my opinion unethical, but unfortunately seems to be in line with a US centric view that is all too common in W3C working groups.

My turn to apologize for a late reply -- I was on the road for four weeks...
Let me start by saying I thought that you (@zorkow) know me well enough that any mention of retaining job security was meant as a joke. You know I respect your work and all you have done for the community. I'm sorry to hear you were offended by it and I apologize for that.
As for the rest of your comments, we agree it is not productive to extend this discussion, so I won't go through a point by point rebuttal and just say I disagree with a number of your comments. If you want to discuss them in more detail, I suggest opening a different issue, perhaps in the MathML issue tracker.


Add Code highlighting

Adding highlight.js code highlighting (same as Design system)
Added renderer to marked to add correct classes
Inline default highlight.js css
Trello ticket: https://trello.com/c/IIEvTvmy/681-fix-code-style-in-the-prototype-kit

@alex-ju split function further?

bumped up to 19

thanks @alex-ju


Name attribute set.

How we can set "name" attribute?

Since 1.3.7 you can pass name property to component


Issue with axios browser dependencies

I am seeing following in the documentation.

Getting a little confused about safari browser support here. According to above chart it seems like it does not work on safari 9 however it works fine on safari 6 or 7.
So my concern is that is axios supported on safari 9?

The chart actually shows the opposite: axios has been tested with Safari 9. What makes you think it does not work in Safari 9?


trace db Attributes

What type of this PR
/kind feature
What this PR does / why we need it:
trace db Attributes
Specified Reivewers:
/assign @liuhaoyang

Closed for non-active.
Reopen it if you need.


Fixing squid:S2259 - Null pointers should not be dereferenced

This pull request is focused on resolving occurrences of Sonar rule squid:S2259 - “Null pointers should not be dereferenced”. You can find more information about the issue here: https://dev.eclipse.org/sonar/rules/show/squid:S2259
Please let me know if you have any questions.
Sameer Misger

@misgersameer Thanks!


🐛 Fix for human_readable_label_name not defined

When running a job that attached a file set we'd receive errors because the private methods haven't been pulled over into this project.

@laritakr I'm not sure what you mean, I copied it from here: https://github.com/scientist-softserv/adventist-dl/blob/main/app/indexers/hyrax/file_set_indexer.rb
We created a decorator but didn't pull over all of the methods. When we run jobs currently, it fails because of it.

@laritakr I'm not sure what you mean, I copied it from here: https://github.com/scientist-softserv/adventist-dl/blob/main/app/indexers/hyrax/file_set_indexer.rb

@ShanaLMoore
Ah ok, I missed that you were just pulling in existing code... I thought you were overriding something new and was very confused.
Can you add back the comment explaining what is being overridden?