Incorrect expectation values of separate Hamiltonian terms

Environment

Qiskit Nature version: 0.4.4
Python version: 3.9.7
Operating system: Kubuntu 20.04.4 LTS

What is happening?
The expectation values obtained via ExpectationFactory and CircuitSampler classes are incorrect in this case.
How can we reproduce the issue?
The following snippet reproduces the problem, where it can be seen, that evaluation of the expectation value of separate Hamiltonian terms w.r.t. Hartree-Fock state always equals 0, while the expectation value w.r.t. ground state obtained via VQE is incorrect for +_0 -_0 and +_2 -_2 terms.
Due to a slightly wordy syntax necessary to obtain expectation values, the functionality itself is enclosed in classes OperatorEvaluatorBase and HermitianOperatorEvaluator.
#!/usr/bin/env python3
from abc import ABC
from typing import Callable, Union
import numpy as np
from qiskit import QuantumCircuit
from qiskit.circuit import Parameter
from qiskit.circuit.parametervector import ParameterVectorElement
from qiskit.opflow import ExpectationFactory, StateFn, CircuitStateFn, CircuitSampler
from qiskit.providers.aer import Aer
from qiskit.utils import QuantumInstance
from qiskit.utils.backend_utils import is_aer_provider
from qiskit_nature.algorithms import GroundStateEigensolver, VQEUCCFactory
from qiskit_nature.circuit.library import UCC, HartreeFock
from qiskit_nature.converters.second_quantization import QubitConverter
from qiskit_nature.drivers import Molecule
from qiskit_nature.drivers.second_quantization import ElectronicStructureMoleculeDriver, ElectronicStructureDriverType
from qiskit_nature.mappers.second_quantization import JordanWignerMapper
from qiskit_nature.operators.second_quantization import FermionicOp
from qiskit_nature.problems.second_quantization import ElectronicStructureProblem
from qiskit_nature.transformers.second_quantization.electronic import ActiveSpaceTransformer

class OperatorEvaluatorBase(ABC):
    def __init__(self, operator, backend, include_custom: bool = False) -> None:
        self._backend = backend
        self._expectation_obj = ExpectationFactory.build(operator=operator,
                                                         backend=backend,
                                                         include_custom=include_custom)
        self._measurable_operator = self._expectation_obj.convert(StateFn(operator,
                                                                          is_measurement=True)).reduce()

    def get_evaluation_func(self, circuit: QuantumCircuit) -> Callable:
        pass

class HermitianOperatorEvaluator(OperatorEvaluatorBase):
    def __init__(self, operator, backend, include_custom: bool = False) -> None:
        super(HermitianOperatorEvaluator, self). \
            __init__(operator, backend, include_custom)

    def get_evaluation_func(self, circuit: QuantumCircuit) -> Callable:
        expectation_operator = self._measurable_operator.compose(CircuitStateFn(circuit)).reduce()
        circ_sampler = CircuitSampler(self._backend, param_qobj=is_aer_provider(self._backend))

        def evaluation_func(circ_params: Union[np.array, list, tuple,
                                               dict[Union[ParameterVectorElement, Parameter], float]]) -> float:
            param_binding = None
            if isinstance(circ_params, dict):
                param_binding = circ_params
            else:
                param_binding = {p: circ_params[i] for i, p in enumerate(circuit.parameters)}
            return circ_sampler.convert(expectation_operator, params=param_binding).eval().real

        return evaluation_func

if __name__ == '__main__':

    backend = Aer.get_backend('statevector_simulator')

    geometry = [('N', [0.000000000000, 0.000000000000, 0.000000000000]),
                ('C', [0.000000000000, 0.000000000000, 1.498047000000]),
                ('H', [0.000000000000, -0.938765985000, 2.004775984000]),
                ('H', [0.000000000000, 0.938765985000, 2.004775984000]),
                ('H', [-0.744681452, -0.131307432, -0.634501434])]

    # Obtaining ground state
    driver = ElectronicStructureMoleculeDriver(Molecule(geometry),
                                               basis="sto3g",
                                               driver_type=ElectronicStructureDriverType.PSI4)

    as_transformer = ActiveSpaceTransformer(num_electrons=2, num_molecular_orbitals=2)
    es_problem = ElectronicStructureProblem(driver, transformers=[as_transformer])
    converter = QubitConverter(JordanWignerMapper())

    ansatz = UCC(qubit_converter=converter, num_particles=(1, 1), num_spin_orbitals=4, excitations='sd',
                 alpha_spin=True, beta_spin=True, max_spin_excitation=None, generalized=True, preserve_spin=True,
                 reps=1, initial_state=None)

    solver = GroundStateEigensolver(converter, VQEUCCFactory(quantum_instance=QuantumInstance(backend), ansatz=ansatz))

    ground_state = solver.solve(es_problem)
    optimal_circ = ansatz.bind_parameters(ground_state.raw_result.optimal_parameters)

    # Obtain 1-body terms of Hamiltonian and rewrite the coefficients to 1
    ham_op = es_problem.second_q_ops()[0]
    one_body_terms = [(e[0], 1) for e in ham_op.to_list() if len(e[0].split()) == 2]

    # Obtain expectation values for every term separately
    exp_funcs_vqe = [HermitianOperatorEvaluator(converter.convert(FermionicOp(e)),
                                                backend,
                                                include_custom=True).get_evaluation_func(optimal_circ)
                     for e in one_body_terms]

    # Expectation values for Hartree-Fock state
    hf_circ = HartreeFock(num_spin_orbitals=4,
                          num_particles=(1, 1),
                          qubit_converter=converter)

    exp_funcs_hf = [HermitianOperatorEvaluator(converter.convert(FermionicOp(e)),
                                               backend,
                                               include_custom=True).get_evaluation_func(hf_circ)
                    for e in one_body_terms]

    print('Expectation values for separate 1-body Hamiltonian terms with HF-state circuit:')
    for t, f in zip(one_body_terms, exp_funcs_hf):
        print(f'{t[0]}: {f([])}')

    print('Expectation values for separate 1-body Hamiltonian terms with VQE-ground-state:')
    for t, f in zip(one_body_terms, exp_funcs_vqe):
        print(f'{t[0]}: {f([])}')

The output
Expectation values for separate 1-body Hamiltonian terms with HF-state circuit:
+_0 -_0: 0.0
+_0 -_1: 0.0
+_1 -_0: 0.0
+_1 -_1: 0.0
+_2 -_2: 0.0
+_2 -_3: 0.0
+_3 -_2: 0.0
+_3 -_3: 0.0
Expectation values for separate 1-body Hamiltonian terms with VQE-ground-state:
+_0 -_0: 0.08137254743300243
+_0 -_1: 0.12130051705413182
+_1 -_0: 0.12130051705413182
+_1 -_1: 0.08137254743300243
+_2 -_2: 0.08137254743300243
+_2 -_3: 0.12130051705413182
+_3 -_2: 0.12130051705413182
+_3 -_3: 0.08137254743300243

What should happen?
The expectation values obtained w.r.t. HF state should be nonzeros, while the output for the ground state should look like this:
+_0 -_0: 0.919251665
+_0 -_1: 0.12130051705413182
+_1 -_0: 0.12130051705413182
+_1 -_1: 0.08137254743300243
+_2 -_2: 0.919251665
+_2 -_3: 0.12130051705413182
+_3 -_2: 0.12130051705413182
+_3 -_3: 0.08137254743300243

Any suggestions?
If it helps, I can write more specific and brief example - this one was created with an intention of demonstrating the problem in the full context.

The title of your issue seems to suggest that this is only dealing with aspects of Qiskit Terra, rather than Nature. Please produce a minimal example which concisely summarizes the issue you are trying to seek out here.

@mrossinek  I put the problem here, as I'm trying to obtain the expectation values of separated Hamiltonian terms via the abovementioned classes and I'm obtaining incorrect results even in simple cases.
To be specific, e.g. I want to take the first term of Hamiltonian +_0 -_0 and obtain its expectation value with a Hartree-Fock state, but I'm obtaining zero. That's the same for all the one-body operators. Are they not supposed to be used like this?
I've tried to simplify the code to focus on this one case only, without the ground state results.
#!/usr/bin/env python3
from qiskit.opflow import ExpectationFactory, StateFn, CircuitStateFn, CircuitSampler
from qiskit.providers.aer import Aer
from qiskit.utils.backend_utils import is_aer_provider
from qiskit_nature.circuit.library import HartreeFock
from qiskit_nature.converters.second_quantization import QubitConverter
from qiskit_nature.drivers import Molecule
from qiskit_nature.drivers.second_quantization import ElectronicStructureMoleculeDriver, ElectronicStructureDriverType
from qiskit_nature.mappers.second_quantization import JordanWignerMapper
from qiskit_nature.operators.second_quantization import FermionicOp
from qiskit_nature.problems.second_quantization import ElectronicStructureProblem
from qiskit_nature.transformers.second_quantization.electronic import ActiveSpaceTransformer

if __name__ == '__main__':

    backend = Aer.get_backend('statevector_simulator')

    geometry = [('N', [0.000000000000, 0.000000000000, 0.000000000000]),
                ('C', [0.000000000000, 0.000000000000, 1.498047000000]),
                ('H', [0.000000000000, -0.938765985000, 2.004775984000]),
                ('H', [0.000000000000, 0.938765985000, 2.004775984000]),
                ('H', [-0.744681452, -0.131307432, -0.634501434])]

    driver = ElectronicStructureMoleculeDriver(Molecule(geometry),
                                               basis="sto3g",
                                               driver_type=ElectronicStructureDriverType.PSI4)

    as_transformer = ActiveSpaceTransformer(num_electrons=2, num_molecular_orbitals=2)
    es_problem = ElectronicStructureProblem(driver, transformers=[as_transformer])
    converter = QubitConverter(JordanWignerMapper())

    # Obtain 1-body terms of Hamiltonian and rewrite the coefficients to 1
    ham_op = es_problem.second_q_ops()[0]
    one_body_terms = [(e[0], 1) for e in ham_op.to_list() if len(e[0].split()) == 2]

    # Expectation values for Hartree-Fock state
    hf_circ = HartreeFock(num_spin_orbitals=4,
                          num_particles=(1, 1),
                          qubit_converter=converter)

    op = converter.convert(FermionicOp(one_body_terms[0]))

    measurable_op = ExpectationFactory.build(operator=op,
                                             backend=backend,
                                             include_custom=True).convert(StateFn(op, is_measurement=True)).reduce()

    expec_op = measurable_op.compose(CircuitStateFn(hf_circ)).reduce()

    circ_sampler = CircuitSampler(backend, param_qobj=is_aer_provider(backend))
    print(f'<{one_body_terms[0][0]}> = {circ_sampler.convert(expec_op).eval().real}')

TL;DR: supply the correct register_length value when initializing your FermionicOp instances, e.g. like so:
FermionicOp(one_body_terms[0], register_length=4)

Explanation
Alright the issue which you are running into here is two-fold:

You are initializing your custom FermionicOp instances using the sparse label notation without providing a value for register_length. This behavior results in an automatic computation of this register length which is why this does not raise an error. What this means specifically for you, is that the fermionic operators have length 1 which does not match your expectation of length 4. You can see this when printing e.g. op which is the resulting qubit operator, also only of length 1.
The second problem is endianess. Terra orders their qubits like computer scientists, in little endian format. On the contrary, in Nature our operators are big endian, since is the more wide-spread standard in chemistry/science-oriented software packages. This is the reason, why the above is actually causing issues for you at all, since the the HartreeFock state you initialize, has no X applied on the "last" qubit (from Nature's point of view), which is where Terra will apply the single-length qubit operator evaluation (since it considers this the "first" qubit).

This has been brought up before here: https://github.com/Qiskit/qiskit-nature/issues/609


fix(cron): allow unresolved variables in when

This PR allows for unresolved variables to be present in cronworkflows during the evaluation of the when clause.
This is needed because workflow variables are not accessible/unresolved when the clause is evaluated.

This is needed because workflow variables are not accessible/unresolved when the clause is evaluated.

I've provided an example in the test. The variables that a workflow may have, for example: {{workflow.name}} will not be available during the call to evalWhen, this means we must allow for unresolved variables when calling the templating engine.

Oh you mean inside the workflowSpec? Can you limit the eval to just the when field instead?

Oh you mean inside the workflowSpec? Can you limit the eval to just the when field instead?

Yup, unfortunately, this isn't really possible. Otherwise, that would have been my preferred solution.

Can you elaborate why?
If I'm not mistaken, this would also allow invalid variables in when, which is a source of constant confusion in Argo templating.

Can you elaborate why?
If I'm not mistaken, this would also end up allowing invalid variables in when, which is a source of constant confusion in Argo templating.

The current templating engine doesn't care, it looks for any string with the "{{" pattern and attempts to replace, and it does this at the workflow level.
We absolutely need a refactor of how all this works.

The functions used here, NewTemplate and Replace, take an arbitrary string. So if I'm not mistaken, you can make that work on a single field without any refactoring of the templating logic

The functions used here, NewTemplate and Replace, take an arbitrary string. So if I'm not mistaken, you can make that work on a single field without any refactoring of the templating logic internals.

Yeah you are correct, I just wanted to keep the logic consistent across wherever we use templating but I suppose this is better.

@agilgur5 alright, just did that.

LGTM now, just one stylistic comment


Adding init_drupal to sudo-allowed commands

Problem / Motivation
We use the container images in Amazon ECS and to setup the containers we use the provided init_drupal script when the container starts up, however to do so we had to include this command in the sudoers file.
Is there a rationale why /usr/local/bin/init_drupal was not included in /etc/sudoers.d/wodby like the other scripts in wodby/php?

Your link shows init_container being included in sudoers file

Oh sorry, I see now that I did a mistake in typing the issue. 😓
The correct script was init_drupal (as mentioned in the title) and not init_container, which as you say is already included. 😃

it wasn't included in the base image because init_drupal added in the drupal-php image. Also it seems it doesn't require sudo permissions to do anything, otherwise we would update sudoers in the drupal-php

You are absolutely right, not sure what was the issue I encountered 🤔
I can confirm that there is no need for sudo, I have used this successfully in several places.
thanks for the feedback.


ESP8266 resets to CWMODE=1 on every power cycle

Hi, I set CWMODE = 2 or 3 and when I power off and on the module, it defaults to CWMODE = 1, is this the intended behaviour?

That is correct. This is set in user_init():
`//Promiscuous works only with station mode
//Code will error if not in station mode.
//AT+CWMODE=1
//AT+CWAUTOCONN=0
if(at_wifiMode != STATION_MODE)
{
#ifdef DEBUG
    at_port_print("\r\nStation mode not set, setting station mode.\r\n");
#endif

    ETS_UART_INTR_DISABLE();
    wifi_set_opmode(1);
    ETS_UART_INTR_ENABLE();
}`

Thanks @bleckers for the interesting project, wiring everything up today. will let you know how it goes.

Good stuff :)
Just to note, I forgot to mention that all settings and changes you make will revert to default on reboot. I have updated the readme to reflect this and explained it a little more here - https://github.com/subpos/esp_firmware_v2/issues/2


(nuget.commandline) Update failing

Current Behavior
Update of nuget.commandline throws an error: https://gist.github.com/gep13/bd2eaa76f2a9ab739ca0544c502dca6e#nugetcommandline

I wrote the AU updater for that package.
It looks like site was changed in the meantime so the scraper should be changed to reflect that. It should be trivial, but keep in mind that this is intended as reference point for other Nuget packages so that it should be general enough (here only package name needed to be changed).
Xplat package on my machine returns this when executed:

.\NuGet.CommandLine.XPlat.exe

Unhandled Exception: System.IO.FileNotFoundException: Could not load file or assembly 'NuGet.Common, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.
   at NuGet.CommandLine.XPlat.Program.Main(String[] args)

It has bunch of dependencies while CommandLine has none. Its funny that it does so, as I have VS2017 installed.
The current AU updater doesn't handle dependencies.

It looks like site was changed in the meantime so the scraper should be changed to reflect that.

@majkinetor Why do you still want to scrape the nuget.org website? Since the AU script uses the NuGet package it should also check the latest available version of the NuGet package instead to determine if it should update instead of checking a website which might be out-of-date (eg. the latest mentioned version on nuget.org is 4.3 for which no nuget.commandline package exists but only a nuget.commandline.xplat)

I don't want to. It should definitely be changed to use something else
For example nuget itself could be used if nothing more appropriate exists (such as REST API) -  nuget list id:<package_name> in which case AU should have nuget.commandline preinstalled :+1:
PS> nuget list id:nuget.commandline -Prerelease
NuGet.CommandLine 4.3.0-preview3-4168
NuGet.CommandLine.XPlat 4.3.0

JSON is way better, I didn't see you already used it.


Delete large disk fails on devstack environment

'can delete large disk' test case failed during the validation, here is the log:
Your OpenStack
  validating configuration
    cpi
      can attach floating IP to a VM... passed
      can create a VM with static IP... passed
      can create large disk... passed
      can delete large disk... failed
...

Failures:

  1) Your OpenStack validating configuration cpi can delete large disk
     Large disk '21fe3586-bd3d-4886-bd18-d4c8b92fa6ba' could not be deleted. OpenStack error: CPI error 'Unknown' with message 'read timeout reached' in 'delete_disk' CPI method

@voelzmo thanks for your info, will have a try.

Dear @animationzl,
I'm not sure where I can find the logs of our tests in the link you pasted. All I could find is logs from the setup of the buildjob. What's the  directory where the test logs are stored in?

Hey @voelzmo ,
with the value of read_timeout set to 360, the output of ./validate --stemcell bosh-openstack-qemu-ubuntu-trusty-go_agent.tgz --config validator.yml looks like this
2018-07-03 09:24:07.206169 | ubuntu-xenial | Your OpenStack
2018-07-03 09:24:07.206896 | ubuntu-xenial |   validating configuration
2018-07-03 09:25:17.590223 | ubuntu-xenial |     cpi
2018-07-03 09:25:24.979557 | ubuntu-xenial |       can create large disk... passed
2018-07-03 09:31:24.224190 | ubuntu-xenial |       can delete large disk... failed
...
...
2018-07-03 09:39:56.845538 | ubuntu-xenial | Failures:
2018-07-03 09:39:56.845553 | ubuntu-xenial |
2018-07-03 09:39:56.845615 | ubuntu-xenial |   1) Your OpenStack validating configuration cpi can delete large disk
2018-07-03 09:39:56.845814 | ubuntu-xenial |      Large disk '3e83ca5f-e38b-43fc-a36e-5572bd8d0eb0' could not be deleted. OpenStack error: CPI error 'Bosh::Clouds::CloudError' with message 'Timed out waiting for Volume `3e83ca5f-e38b-43fc-a36e-5572bd8d0eb0' to be deleted' in 'delete_disk' CPI method

note that I found the CREATE and DELETE actions for volume 3e83ca5f-e38b-43fc-a36e-5572bd8d0eb0 succeeded in cinder logs
root@ubuntu:/opt/stack/new# grep '3e83ca5f-e38b-43fc-a36e-5572bd8d0eb0' screen-logs/screen-c-api.log
2018-07-03 09:25:19.461 INFO cinder.volume.api [req-1e603339-ff88-48ea-94f0-d991b07a679d demo] [volume-3e83ca5f-e38b-43fc-a36e-5572bd8d0eb0] Volume created successfully.
...
...
2018-07-03 09:25:26.314 INFO cinder.api.openstack.wsgi [req-9bd3c3b7-9a3d-4372-b301-9f0808a76733 demo] DELETE http://<IP_ADDRESS>:8776/v2/5cc595a20761422d9613375e53ee630e/volumes/3e83ca5f-e38b-43fc-a36e-5572bd8
d0eb0
2018-07-03 09:25:26.315 INFO cinder.api.v2.volumes [req-9bd3c3b7-9a3d-4372-b301-9f0808a76733 demo] Delete volume with id: 3e83ca5f-e38b-43fc-a36e-5572bd8d0eb0
2018-07-03 09:25:26.363 INFO cinder.volume.api [req-9bd3c3b7-9a3d-4372-b301-9f0808a76733 demo] [volume-3e83ca5f-e38b-43fc-a36e-5572bd8d0eb0] Volume info retrieved successfully.
2018-07-03 09:25:26.393 INFO cinder.volume.api [req-9bd3c3b7-9a3d-4372-b301-9f0808a76733 demo] [volume-3e83ca5f-e38b-43fc-a36e-5572bd8d0eb0] Delete volume request issued successfully.
2018-07-03 09:25:26.393 INFO cinder.api.openstack.wsgi [req-9bd3c3b7-9a3d-4372-b301-9f0808a76733 demo] http://<IP_ADDRESS>:8776/v2/5cc595a20761422d9613375e53ee630e/volumes/3e83ca5f-e38b-43fc-a36e-5572bd8d0eb0 r
eturned with HTTP 202
2018-07-03 09:25:26.394 INFO eventlet.wsgi.server [req-9bd3c3b7-9a3d-4372-b301-9f0808a76733 demo] <IP_ADDRESS> "DELETE /v2/5cc595a20761422d9613375e53ee630e/volumes/3e83ca5f-e38b-43fc-a36e-5572bd8d0eb0 HTTP/1.1"
 status: 202  len: 187 time: 0.0959902

according to the logs, the volume has been deleted successfully, the error maybe other reasons.


Fix CI by running linter & replacing deprecated fn

Replace deprecated .copy -> .copy_mount to fix PyTest

Run black to fix my earlier PR

Before

After

Type of Change

[ ] New Example
[ ] Example updates (Bug fixes, new features, etc.)
[x] Other (changes to the codebase, but not to examples)

Checklist

[x] Example is testable in synthetic monitoring system, or lambda-test: false is added to example frontmatter
[x] Example does not require third-party dependencies to be installed locally
[x] Example is documented with comments throughout, in a Literate Programming style.

 Thanks for your contribution.

forgot that github actions arent enabled by default when forking so i missed the black linter check, so definitely wanted to fix that. figured id be a good samaritan by also fixing PyTest

Nice, thanks for doing this!!


[Target] Remove Process::GetCPPLanguageRuntime

Summary:
I want to remove this method because I think that Process should be
language agnostic, or at least, not have knowledge about specific language
runtimes. There is "GetLanguageRuntime()" which should be used instead. If the
caller a CPPLanguageRuntime, they should cast it as needed. Ideally, this
should only happen in plugins that need C++ specific knowledge.
The next step I would like to do is remove "GetObjCLanguageRuntime()" as well.
There are a lot more instances of that function being used, so I wanted to
upload this one first to get the general reception to this idea.
Reviewers: compnerd, davide, JDevlieghere, jingham, clayborg, labath, aprantl
Subscribers: lldb-commits
Differential Revision: https://reviews.llvm.org/D62755
cc @compnerd @JDevlieghere @dcci
git-svn-id: https://llvm.org/svn/llvm-project/lldb/trunk@362544 91177308-0d34-0410-b5e6-96231b3b80d8

This is a cherry-pick, so I guess its okay.  I really don't understand the point of the helper method to do the cast.  Why not have the single site caller do the explicit cast rather than add the GetCPPLanguageRuntime method?

@swift-ci please test

@compnerd: This is a request from @jimingham. If we add more instances of GetCPPLanguageRuntime it might be a bigger deal, kind of like what is happening with the ObjC case today.

Right, the equivalent from ObjC is used many times, and so the Get function is more clearly motivated.  But if we're going to use the Get pattern for ObjC, which I presume @xiaobai will get to at some point, then we should also use it for CPP - which is what he got to first...


[BugFix] getColumnStatistic on dangling auto_fill_col triggers NPE

What type of PR is this：

[x] BugFix
[ ] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Which issues of this PR fixes ：

Fixes #
Problem Summary(Required) ：

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr will affect users' behaviors
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[x] I have checked the version labels which the pr will be auto backported to target branch

[x] 2.5
[ ] 2.4
[ ] 2.3
[ ] 2.2

@Mergifyio backport branch-2.5


Verify need of includes in already coded modules

It finally appears that some includes are not useful for the compilation of some already coded modules.
Let's check which ones may be commented (right now) then removed (later) in which modules!

Completed; tested.


support io-ts for runtime schema parsing

Implement an io-ts based schema parser as an alternative to joi and zod

Unlikely to ever get time for this, and I'm personally very happy with zod - won't do


Bugfix: do not match dead process IDs in compilation table

@hmmr discovered a scenario where frequently stopping and starting nodes while testing DDL compilation can result in an error due to lingering process IDs from the previous instantiation of the node.
Clean up PIDs after the compile is finished, and also explicitly match only compiling records so that users of prior versions with legacy data do not hit this bug.
Also extend commentary and add a riak_test intercept point to make system testing for this bug easier.

Brett will be creating a test for this.

riak_kv_compile_tab.erl:172: The call riak_kv_compile_tab:insert(BucketType::any(),DDLVersion::any(),DDL::any(),'undefined',State::'compiled' | 'compiling' | 'failed' | 'retrying') breaks the contract (BucketType::binary(),DDLVersion::riak_ql_component:component_version(),DDL::term(),CompilerPid::pid(),State::compiling_state()) -> 'ok' | 'error'

Sadly enough, not only did I not run dialyzer, I didn't even try to compile it. Got too enthusiastic about the fix.

Fix submitted

+1 72e17e6

Thanks @hmmr. Let's wait until we have a test, though, before we merge this.

This PR conflicts with the changes I have made in #1500 which changes the responsibilities of the dets compile table. It now stores versions of the DDL as different DDL record types, e.g. ddl_v1, ddl_v2.
If you anyone writes the test I will run it against the changes in my PR.

This is irrelevant with Andy's changes.


Iceberg and Delta target: sync schema field comments

Important Read
Aim to address https://github.com/apache/incubator-xtable/issues/525
What is the purpose of the pull request
It is expected to capture the comments and sync the comments of field when syncing schema. The PR makes the comments of schema fields are captured and synced correctly.
Brief change log

For Delta target, extract the comment from internal schema and add to Spark schema representative.
For Iceberg target, make sure any difference of comments is captured during syncSchema() and the schema is updated with new comments.

Verify this pull request
This change added tests and can be verified as follows:

Added unit tests to test the comments in Iceberg schema sync test class
Added comments in Delta schema extractor test class

Thank you for fixing this @emilie-wang!


Does not work on windows 8

Hi
I am trying your sample code on windows 8.
I does not seem to work. I works on windows 10 though.

I'll take a look. Thank you!

hi, ran into this as well, it seems related to transparent: true
will post when/if we find a suitable solution
cheers

hi, it seems it s a combination of flags that make this not appear on win 8
i've added
resizeable :false
and removed
minimizable: false, maximizable: false,
and it seems ok on windows 8. did not test on other systems yet, but it shouldn t be a problem. will test win 10 and macos just to be sure.
would you prefer a pull request after?
cheers

in our tests it seems to work both on win 10 and macOS (except this which is still an issue: https://github.com/blainesch/electron-notifications/issues/12 )


DOC: OracleDB 23ai demo link brokern

Checklist

[X] I added a very descriptive title to this issue.
[X] I included a link to the documentation page I am referring to (if applicable).

Issue with current documentation:
Oracle AI Vector Search End-to-End Demo Guide is broken and throws 404.
(https://github.com/langchain-ai/langchain/tree/master/cookbook/oracleai_demo.md)
Idea or request for content:
No response

okay I am fixing it.

This has been fixed....

When is this getting pushed? I still the link is broken


Mast: Watch clariwell.in

Mast requests the watch of the watch_keyword clariwell\.in. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

clariwell\.in has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by tripleee in Charcoal HQ


[2] Link player's ID with stat.ink ID to display a user icon and link to his/her profile

For example, as of now, not linked Eli's icon with stat.ink's user info.
https://stat.ink/@frozenpandaman/spl2/61241#players
We should be able to use that ID to tie user information.
And we can also link to their profile page.

f9fdfe9344b4bd8567c4c27250ae9c54451af97a


Query result was required, but an empty Object((Weak)) was returned instead

Bug description
I'm getting this error a lot when upsert-ing to Postgresql. Happens on different models. Certain bits of data seem to cause it to have this error although I can't figure out why. Majority of items upsert fine, but 7 out of 100 say will fail with this error.
PrismaClientUnknownRequestError2:
Invalid `prisma.keyword.upsert()` invocation:

  Invalid response data: the query result was required, but an empty Object((Weak)) was returned instead.
    at PrismaClientFetcher.request (/Users/xx/Documents/boilerplate/node_modules/@prisma/client/runtime/index.js:78786:15)
    at processTicksAndRejections (node:internal/process/task_queues:93:5)
    at /Users/xx/Documents/boilerplate/packages/xx/lib/xxx.ts:424:25 {
  clientVersion: '2.11.0'
}

How to reproduce
              await prisma.keyword.upsert({
                create: {
                  id: item.keywordId,
                  bid: keywordStatus.bid,
                  state: keywordStatus.state,
                  keywordText: item.keywordText,
                  matchType: item.matchType,
                  updated: new Date(),
                  Campaign: { connect: { id: item.campaignId } },
                },
                update: {
                  bid: keywordStatus.bid,
                  state: keywordStatus.state,
                },
                where: { id: item.keywordId },
              });

Data Example 1:
{
  keywordId:<PHONE_NUMBER>9505,
  cost: 0,
  attributedSales1d: 0,
  campaignId:<PHONE_NUMBER>4050,
  matchType: "EXACT",
  clicks: 0,
  impressions: 2,
  campaignName: "wood paint pens",
  keywordText: "wood paint pens",
  attributedUnitsOrdered1d: 0,
}

Data Example 2:
{
  keywordId:<PHONE_NUMBER>291,
  cost: 0,
  attributedSales1d: 0,
  campaignId:<PHONE_NUMBER>32250,
  matchType: "EXACT",
  clicks: 0,
  impressions: 43,
  campaignName: "glass paint",
  keywordText: "glass paint",
  attributedUnitsOrdered1d: 0,
}

Expected behavior
Upsert object to DB
Prisma information
model Keyword {
  id          Float           @id
  keywordText String
  matchType   MatchType
  bid         Float?
  state       State?
  rangeStart  Float?
  rangeEnd    Float?
  suggested   Float?
  updated     DateTime        @updatedAt
  reports     KeywordReport[]
  Campaign    Campaign        @relation(fields: [campaignId], references: [id])
  campaignId  Float
}

Environment & setup

OS: Mac OS,
Database: PostgreSQL
Node.js version:-v15.2.0
Prisma version:2.11.0

Reverted to Prisma 2.10.0 and everything works again.

Hey @Stevenje
Sorry for the delay here, I was kinda on holiday. Can you please share KeywordReport, MatchType and Campaign models/enums. I wasn't able to reproduce this so please share these models.

Sure, here you go. Hope you had a nice break.
enum MatchType {
  BROAD
  PHRASE
  EXACT
}

model Campaign {
  id                   Float      @id
  name                 String
  campaignType         String?
  targetingType        String?
  dailyBudget          Float?
  placementTop         Float?
  placementProductPage Float?
  strategy             String?
  state                State?
  adGroups             AdGroup[]
  Portfolio            Portfolio? @relation(fields: [portfolioId], references: [id])
  portfolioId          Float?
}

model KeywordReport {
  date                     DateTime
  clicks                   Int
  cost                     Float
  attributedSales1d        Float
  attributedUnitsOrdered1d Float
  impressions              Int
  Keyword                  Keyword  @relation(fields: [keywordId], references: [id])
  keywordId                Float

  @@id([keywordId, date])
}

Still present in 2.12.1
PrismaClientUnknownRequestError2:
Invalid `prisma.keyword.upsert()` invocation:

  Invalid response data: the query result was required, but an empty Object((Weak)) was returned instead.
    at PrismaClientFetcher.request (/Users/steven/Documents/boilerplate/versiontwo/node_modules/@prisma/client/runtime/index.js:79359:15)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:93:5)
    at /Users/steven/Documents/boilerplate/versiontwo/packages/amazon/lib/amazon.ts:636:15 {
  clientVersion: '2.12.1'
}

@pantharshit00 Any idea what could be causing this?
I tried with a clean DB today, using AWS RDS Postgres and spent a couple of hours playing around with the code to see if anything might happen to fix it and came up short.

It also seems to be unable to connect items sometimes.
Create adgroup: PrismaClientUnknownRequestError2:
Invalid `prisma.adGroup.create()` invocation:

  Error occurred during query execution:
ConnectorError(ConnectorError { user_facing_error: None, kind: ForeignKeyConstraintViolation { constraint: Index("AdGroup_campaignId_fkey") } })
    at PrismaClientFetcher.request (/Users/steven/Documents/boilerplate/versiontwo/node_modules/@prisma/client/runtime/index.js:79359:15)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:93:5)
    at /Users/steven/Documents/boilerplate/versiontwo/packages/amazon/lib/amazon.ts:497:45 {
  clientVersion: '2.12.1'
} {
  keywordId:<PHONE_NUMBER>1333,
  cost: 0,
  attributedSales1d: 0,
  adGroupName: 'CAM0009 Manual - Low Search Volume 0-1k',
  campaignId:<PHONE_NUMBER>66800,
  matchType: 'EXACT',
  clicks: 0,
  impressions: 45,
  campaignName: 'CAM0009 Manual - Low Search Volume 0-1k',
  adGroupId:<PHONE_NUMBER>56720,
  keywordText: 'paint marker',
  attributedUnitsOrdered1d: 0
}

Although the item 100% is in the DB

Fixed by switching all ID values from FLOAT -> STRING although this was not idea.

@Stevenje There is an open issue which covers this floats ids issue which is confirmed: https://github.com/prisma/prisma/issues/3665
So I am going to close this issue in favor of that.


Create milestones to define a roadmap

You have some ideas for the future of this project.
We should create milestones to define some goals.

Milestones would be something like:

allow the app to separate morphs into head + body morphs.
allow user to rate head + body separately
allow/force user to only use standard clean_install vam morphs (a prerequisite for the neural net)

(1-3) would be a major release.

add neural net which trains on user ratings.

(4) would be another major release.

Since you are maintainer of this product I suggest that you define the milestones. You can do it right here in Github (and I am convinced you already know that), and then we can discuss what is viable and what not.

Check this: https://github.com/pinosante/VAM-Evolutionary-Character-Creation/milestones

Milestone should be associated to a (sub)version number like 2.6.7
Please don't ask about details, because I know little about this, since we didn't use this at my company until recently. I am going to read about it, but appreciate your thoughts about it.

I suggest to define milestones by version numbers, and then assign features to them.


Livewire select searchable, disabled with live and dependencies isn't working

Package
filament/filament
Package Version
v3.1.18
Laravel Version
v10.35.0
Livewire Version
v3.2.6
PHP Version
8.2.12
Problem description
I'm trying to create multiple selects with dependecies between them, I want the selects to be searchable, all of them.
When I make them searchable they simply don't work anymore, but as long as they are not searchable they work as expected

WITH SEARCHABLE
Before selecting anything

After selecting main you notice it's NOT disabled but the values are no populated, or the placeholder

WITHOUT SEARCHABLE
Before selecting anything

After selecting main notice it's NOT disabled and the placeholder is populated as well as the values

From debugging I can see in ray the values when I change the values using
I've achieved this by having an updated method to hook into livewire lifecycles
After select the main:
$this->data

$this->form->getState()

Expected behavior
I'm expending that when the selects are searchable to behave in the same way as when they are not but with the nice html and search input AKA native false
Steps to reproduce
Clone repo
npm install
npm run build
artisan serve
Check web.php for component
Reproduction repository
https://github.com/eduardichim/filament-bug
Relevant log output
No response
PS:
After some more testing it looks like that the disable is causing this, removing the disabled logic works as expected

Duplicate of https://github.com/filamentphp/filament/issues/3068


Document warnings on using remote source for

I think there should be better warnings documented about using a remote url for the config.xml tag <content src="..." />
It should be noted that using remote urls can be dangerous for security and is also against the terms of service of both the Apple App & Google Play stores. (Text of interest is in bold).
Apple App Store policy at Section 4.7 HTML5 Games, Bots, etc[1] reads:

Apps may contain or run code that is not embedded in the binary (e.g. HTML5-based games, bots, etc.), as long as code distribution isn’t the main purpose of the app, the code is not offered in a store or store-like interface, and provided that the software (1) is free or purchased using in-app purchase; (2) only uses capabilities available in a standard WebKit view (e.g. it must open and run natively in Safari without modifications or additional software); your app must use WebKit and JavaScript Core to run third-party software and should not attempt to extend or expose native platform APIs to third-party software; (3) is offered by developers that have joined the Apple Developer Program and signed the Apple Developer Program License Agreement; (4) does not provide access to real money gaming, lotteries, or charitable donations; (5) adheres to the terms of these App Review Guidelines (e.g. does not include objectionable content); and (6) does not support digital commerce. Upon request, you must provide an index of software and metadata available in your app. It must include Apple Developer Program Team IDs for the providers of the software along with a URL which App Review can use to confirm that the software complies with the requirements above.

Google Play Store policy "Malicious Behaviour" reads:

An app distributed via Google Play may not modify, replace, or update itself using any method other than Google Play's update mechanism. Likewise, an app may not download executable code (e.g. dex, JAR, .so files) from a source other than Google Play. This restriction does not apply to code that runs in a virtual machine and has limited access to Android APIs (such as JavaScript in a webview or browser).

While the text isn't as explicit as Apple, and the examples are mostly native file examples, it does say that this does not apply to code that has limited access to Android APIs such as Javascript but in Cordova, the javascript does have full access to android APIs via the cordova bridge.
Currently the documentation does not warn the users of this, and I see on a rather frequent basis of this kind of usage, which I think is completely improper.
[1] https://developer.apple.com/app-store/review/guidelines/
[2] https://play.google.com/about/developer-content-policy-print/

1-ish)

your app must use WebKit and JavaScript Core to run third-party software and should not attempt to extend or expose native platform APIs to third-party software;

As long as third-party software doesn't touch it, it should be fine.  If you only go to your website, (and we are 1st or 2nd party in our app right?  no definition of third party to be found), it should be fine?  And they can't just mean all third party software.... That would mean no jQuery (remote or local).
Most reasonably, the intent, (baring monetary reasons), is probably to prevent malicious third party code from having access to native APIs.  So probably the best way to handle this is not load any script tags except for those tags which you explicitly control.  Eg. no dynamic facebook/jquery, only the static versions that you host yourself or have loaded to the app yourself (ensuring no third party has the ability to change their code and gain access through your app).
2-ish)

only uses capabilities available in a standard WebKit view (e.g. it must open and run natively in Safari without modifications or additional software)

Is it correct to say Cordova uses extended capabilities provided by the standard webview?  Does that count as safari modifications?  If so, I would say that is the end of Cordova.
Conclusion:
I don't really see much here the applies anymore specifically to remote urls than local ones.
But if Apple is serious about # 2.  Well poop.

As long as third-party software doesn't touch it, it should be fine.

If it is exposed, it's already a problem (and a security vulnerability)

And they can't just mean all third party software.... That would mean no jQuery (remote or local).

In this context, I believe third-party software just simply means remote software. if jQuery is bundled with the app, then I believe it's okay.

Is it correct to say Cordova uses extended capabilities provided by the standard webview? Does that count as safari modifications? If so, I would say that is the end of Cordova.

Cordova provides this functionality via cordova.js and the cordova bridge that allows javascript to send messages to the native environment, which in then turns into native API calls.

Cordova does not support injecting native code into the app (as far as I am aware.)
Cordova's codebase itself does not inject native code, though technically speaking a plugin could offer that can feature, however that isn't really Cordova's concern. Cordova does give native access to javascript via the cordova bridge, and the bigger problem is accessing these APIs using remote sources.

(PS. Why do you think this kind of usage is improper? I think it has incredible potential. Eg. write one website that works for desktop and apps. Great for teams low on man power.)

You can still do this without relying on a remote sources. You simply bundle your sources with your cordova app. This however does require some thought and planning on how to architect your product(s) so that code can be shared between two apps.
Generally speaking, if you bundle your sources with your app then you're fine. These policies I'm sure are mostly to keep your app secure. An insecure app hurts both you as the developer, and the app provider (because people will blame Apple/Google for allowing your insecure app). Additionally, Apple who wants to review every single app update you publish, they don't like mechanisms where your app can significantly change in behaviour and bypass their review process. Hence the policy "code distribution isn’t the main purpose of the app".
Loading remote content still has their uses. Webpack hot reload server for example, which is useful for development. Enterprise users can build iOS apps that don't get distributed via the apple store so they don't have to obey apples policies is another reason why it's okay to keep the functionality. It's just something users should be aware of if they intend to submit their cordova app to the app stores.

Apple who wants to review every single app update you publish, they don't like mechanisms where your app can significantly change in behaviour and bypass their review process. Hence the policy "code distribution isn’t the main purpose of the app".

I see.  That makes sense mostly.
But then I find it a bit strange that they even allow webviews.  How are custom browser apps even allowed to exist?  Browsers may not violate # 2, but they definitely violate at least a few other clauses.

Custom browsers aren't allowed on iOS. Apps must use the webviews provided by the iOS SDK. Chrome for iOS for example is not Chromium, but just the standard ios webview with a chrome skin ui wrapped around the webview.
Google doesn't have such a policy afaik and do allow developers to implement their own webviews.
Webviews are inheritly secure due to their limited
and controlled access to native APIs, but the cordova bridge bypasses this. Not sure on the iOS specific details but on android, this is done by overriding the native method that handles the javascript prompt api, where the javascript will send a serialized string prefixed with a token that signals that "this is a native call", which the native code can intercept and read then call the appropriate API, then return a response. There are multiple bridging strategies I believe they will work in a similar fashion.
Again there is nothing wrong with this, as long as you use it in a proper way.

Ah, yes.  By “custom browser” I did mean ones such as chrome.
I absolutely understand why cordova could be used to violate # 2 and chrome could not.
I said

Browsers may not violate # 2, but they definitely violate at least a few other clauses.

I am curious why browsers, such as chrome, would be allowed when they violate at least  # 3, 4, 5, and 6 for sure? (eg. I can go to whatever inappropriate gambling site I desire.)

I am curious why browsers, such as chrome, would be allowed when they violate at least # 3, 4, 5, and 6 for sure? (eg. I can go to whatever inappropriate gambling site I desire.)

Obviously I cannot speak for Apple but probably is a mixture of, 1) they understand it's a web browser, 2) Google is a reputable company with just as much legal power as Apple.
None of this really is a concern of Cordova though.

Sure, but it appears that it is still violating the rules.  Maybe cordova could use a similar exception.

There are tons of other less well known (or not known at all) browsers on the app store (plenty of shady ones too I’m sure).  I don’t think they can all get the same big powerful corporation pass as google.

None of this really is a concern of Cordova though.

True, not directly.  But it helps in understanding the intent and purpose of this rule and how it is actually applied.  (Which does matter when trying to guess how cordova apps will be judged.)
Thank you answering my questions!

Any warnings about this in the docs should also mention the security aspects of allowing uncontrolled remote content to load in the Cordova WebView (as in https://cordova.apache.org/news/2020/09/29/cve-2020-6506.html)


[New Feature]:  Create About page

🎮 feature Request
i want to add a about page in your website
Point down the features
I would like to create the about page of the website so that it will be easy for users to know the details / information of the website
Select program in which you are contributing
GSSoC24
Code of Conduct

[X] I follow CONTRIBUTING GUIDELINE of this project.

If you have any doubt you can directly send message on our discord (https://discord.gg/rZb46cCMmK), we have created seprate chennals for all projects...
Thank you :)


Clean up React 0.14 Example

Address @dprince's suggestions here: https://github.com/kisenka/webpack-svg-sprite-loader/pull/5#discussion_r41553206

Sooo React 0.14, thank you!


Closes #79 | Add dataloader for Yunshan-Cup-2020 Lao POS

Please name your PR after the issue it closes. You can use the following line: "Closes #79" where you replace the ISSUE-NUMBER with the one corresponding to your dataset.
Checkbox

[x] Confirm that this PR is linked to the dataset issue.
[x] Create the dataloader script seacrowd/sea_datasets/my_dataset/my_dataset.py (please use only lowercase and underscore for dataset naming).
[x] Provide values for the _CITATION, _DATASETNAME, _DESCRIPTION, _HOMEPAGE, _LICENSE, _URLs, _SUPPORTED_TASKS, _SOURCE_VERSION, and _SEACROWD_VERSION variables.
[x] Implement _info(), _split_generators() and _generate_examples() in dataloader script.
[x] Make sure that the BUILDER_CONFIGS class attribute is a list with at least one SEACrowdConfig for the source schema and one for a seacrowd schema.
[x] Confirm dataloader script works with datasets.load_dataset function.
[x] Confirm that your dataloader script passes the test suite run with python -m tests.test_seacrowd seacrowd/sea_datasets/<my_dataset>/<my_dataset>.py.
[ ] If my dataset is local, I have provided an output of the unit-tests in the PR (please copy paste). This is OPTIONAL for public datasets, as we can test these without access to the data files.

@sabilmakbar
Thank you for the nits!
Fixed according to the comments. Please check.

LGTM, tested! Thanks @jen-santoso and @ryanignatius. Merging this.


Add "-L" option to cURL cmdline example

It looks like an http POST to https://requestbin.net/ returns status 301 (Moved Permanently)
udooer@udooneogm01:~$ curl -X POST -d "fizz=buzz" -v http://requestbin.net/r/4l9847t6
* Hostname was NOT found in DNS cache
*   Trying <IP_ADDRESS>...
* Connected to requestbin.net (<IP_ADDRESS>) port 80 (#0)
> POST /r/4l9847t6 HTTP/1.1
> User-Agent: curl/7.35.0
> Host: requestbin.net
> Accept: */*
> Content-Length: 9
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 9 out of 9 bytes
< HTTP/1.1 301 Moved Permanently
< Date: Sun, 25 Apr 2021 04:59:43 GMT
< Transfer-Encoding: chunked
< Connection: keep-alive
< Cache-Control: max-age=3600
< Expires: Sun, 25 Apr 2021 05:59:43 GMT
< Location: https://requestbin.net/r/4l9847t6
< cf-request-id: 09a8fee86d00004c68cbaf0000000001
< Report-To: {"max_age":604800,"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report?s=qpZdu%2FY9VKAjNH09PSSYaFROLX4V2sidbWt0M3VqASQZWAfiOBeXzMUNyAGrjmLNYqlhxUZY4uisDDMwEZYyse13WvYTK5tvubOQ3o4V1Q%3D%3D"}],"group":"cf-nel"}
< NEL: {"report_to":"cf-nel","max_age":604800}
* Server cloudflare is not blacklisted
< Server: cloudflare
< CF-RAY: 645500ed79cf4c68-AMS
< alt-svc: h3-27=":443"; ma=86400, h3-28=":443"; ma=86400, h3-29=":443"; ma=86400
<
* Connection #0 to host requestbin.net left intact
udooer@udooneogm01:~$

The "-L" option to curl allows the http POST to complete successfully with status 200 OK
udooer@udooneogm01:~$ curl -X POST -d "fizz=buzz" -L http://requestbin.net/r/4l9847t6
ip:<IP_ADDRESS>
udooer@udooneogm01:~$

Please refer to "man curl" for details.
Signed-off-by: Gianpaolo Macario<EMAIL_ADDRESS>

As a matter of fact the http POST executes successfully when using the https version of the URL - example
udooer@udooneogm01:~$ curl -X POST -d "fizz=buzz" https://requestbin.net/r/4l9847t6
ip:<IP_ADDRESS>
udooer@udooneogm01:~$

Code changed, thanks


Code for obtaining the baseline results.

Hi @cmhungsteve
Thanks for the great work. I wanted to execute and obtain the baseline results mentioned in the paper (DANN + [TempPooling/TempRelation]), can you please let me know if you can make the code public?
Thanks

DANN + TemPooling:
frame_aggregation=avgpool
use_target=uSv
adv_DA=RevGrad
DANN + TemRelation:
frame_aggregation=trn-m
use_target=uSv
adv_DA=RevGrad
and set other XXX_DA as none.

DANN + TemPooling:
frame_aggregation=avgpool
use_target=uSv
adv_DA=RevGrad
DANN + TemRelation:
frame_aggregation=trn-m
use_target=uSv
adv_DA=RevGrad
and set other XXX_DA as none.

Thanks a lot @cmhungsteve
It would be great if you could also share the arguments for obtaining the 'source-only' and 'target-only' results. :)

Thanks a lot @cmhungsteve
It would be great if you could also share the arguments for obtaining the 'source-only' and 'target-only' results. :)

Just set use_target=none, and then modify dataset_source.
For example, if you are working on hmdb_ucf and want to train and test on ucf101 only, then set dataset_source=ucf101, otherwise set dataset_source=hmdb51.
In another word, this all depends on how you define your source and target datasets.

Just set use_target=none, and then modify dataset_source.
For example, if you are working on hmdb_ucf and want to train and test on ucf101 only, then set dataset_source=ucf101, otherwise set dataset_source=hmdb51.
In another word, this all depends on how you define your source and target datasets.

Do I need to set use_attn to none when reproducing DANN results from 3, 4, 5?

Yes


[android_intent] Adds mapping to "Intent.ACTION_CALL"

Sorry if this is the wrong way of proposing a PR, I am a web developer and don't know how things are done in Flutter / Android.
Description
This PR only adds support for the action : Intent.ACTION_CALL from the android_intent plugin. The proposed name is "action_call".
Related
Is there a plan to map all android intents to counterpart names in this flutter plugin? I am guessing that other users could need other actions which are unavailable for now.

I just signed the Contributor License Agreement

Look at this: #1923. It also implements call and many more intents. You can close this pull request in favor of the other one.

closing in favor of https://github.com/flutter/plugins/pull/1923


Add encipher.toml

hi I'm an individual validator.
I've been in crypto since 2017. I like everything around web3, IT and crypto. In addition, I have a higher technical education with a specialization in low-level programming. Have an experience in Linux and Windows for more than twenty years.
24/7 Support
With the help of my skills, I will try to continue to contribute to the technical part of project. And also through the graphic design of marketing and communication articles in different communities in which I am a member, to attract new users to the ecosystem.
https://github.com/encipher88 , https://twitter.com/encipher4836 , <@440387851561992192> - discord, https://t.me/enbit88 - TG
I am the current Aleo, Masa, Sei Ambassador !
2nd place in Stake Wars NEAR – mainnet delegation
Participated in testnet:
APTOS AIT2 , AIT3 (200 people out of 10,000 were selected according to the degree of qualification)
Everscale, Aleo, Archway, Humanode, Kyve, Stratos, Quicksilver, teritori, Massa, Cosmic Horizon, Stafihub, Subspace, Deweb, Kira, Sei, Masa finance, Ironfish, Sui, NEAR stakewars-iii, Starknet, golden, bundlr, bifrost, neutron, Namada.

I've also contributed to NamadaTrustedSetup at round #2105 with the contribution hash f72cee067c3030392fb89616644b24cd2ff6e1fcbb0dfc9e0f14e7ff1ebd839ee0b74fa45306fbba60ca246aa9653f5adf02bd777cb1903d1085044196bc8929

Merged manually, please be ready at 17:00 UTC!


The skills in the "skill cards" are not in center

Describe the bug
There positioning is not in centre for each one.
Possible solution
Putting the name of skill above and the logo in the next line and then set the padding!
Screenshots

If applicable, add screenshots to help explain your problem.

Please assign me this issue @harikesh409

Proceed @MonalikaKapoor

Not to be merged?
On Tue, Jun 1, 2021 at 7:33 PM Harikesh Pallantla @.***>
wrote:

Closed #533
https://github.com/Algo-Phantoms/Algo-Phantoms-Frontend/issues/533 via
#545 https://github.com/Algo-Phantoms/Algo-Phantoms-Frontend/pull/545.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/Algo-Phantoms/Algo-Phantoms-Frontend/issues/533#event-4825631241,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQDKGV4IDO6VYV55YXV2Y43TQTSE7ANCNFSM45ZRAUWA
.

Not to be merged?
…
On Tue, Jun 1, 2021 at 7:33 PM Harikesh Pallantla @.***> wrote: Closed #533 <#533> via #545 <#545>. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#533 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQDKGV4IDO6VYV55YXV2Y43TQTSE7ANCNFSM45ZRAUWA .

It is merged


Some MolStandardize Pipeline suggestions

I got most of the way through my review and did my suggestions as direct suggest code changes since that was quicker (and I could make sure that they actually work!).
The biggest change is to make all the operations in the standardization pipeline configurable.
You've created something tailor-made for Novartis' requirements and that will make some folks happy, but we can really increase the utility of the system by making it configurable. I think I managed to make it reasoanbly lightweight to customize the pipeline without changing the default behavior of the current code.
Some of the more minor stuff:

The Pipeline class itself is now pretty minimal and doesn't have any virtual functions
The code is all more explicit about where the symbols from the enums are coming from, and I switched to scoped enums for PipelineStage. I don't think that switch is possible for PipelineStatus while still allowing you to have things like VALIDATION_ERROR as part of the enum itself (and this seems like a useful thing)

All C++ and Python tests pass for me on linux and the mac, but I haven't looked at what this does to the java builds yet.

@greglandrum sorry, is this one ready to merge, or are you considering making changes to the PipelineOptions part too?

@greglandrum sorry, is this one ready to merge, or are you considering making changes to the PipelineOptions part too?

I think it's ready to merge. I may make some suggestions about PipelineOptions later after the functionality is on master

Thank you, @greglandrum !


doc: gsg: Windows: Clarify cmd.exe language

In the windows world, cmd.exe is refered to as a command prompt and not a windows terminal. This can result in initial installation confusion.

@carlescufi I added you because you changed the initial paragraph I'm changing. Trying to get some insight.

ping @carlescufi @kartben


Documentation on vSAN capacity overhead for first AVS cluster should be updated

This page publishes "Typical Raw vSAN Datastore Usage (GB)" for first AVS cluster.
According to this documentation, customers expect around 4TB consumed by AVS management/control pane components.
However, in practice "Typical Raw vSAN Datastore Usage (GB)" is observed to be taking upto 7TB.
Customers expect AVS documentation provide correct information on "Typical Raw vSAN Datastore Usage (GB)".
Update will help customers to continue trusting in their Azure Migrate based assessment recommendation.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: e1a0a489-f920-2b04-6949-cdd17aef16ca
Version Independent ID: f70f6c39-1df2-a1ac-e371-3313734832a8
Content: Concepts - Private clouds and clusters - Azure VMware Solution
Content Source: articles/azure-vmware/concepts-private-clouds-clusters.md
Service: azure-vmware
GitHub Login: @jjaygbay1
Microsoft Alias: jacobjaygbay

@Mahesh-MSFT
Thanks for the feedback! I have assigned the issue to content author to check and update the document as appropriate.

@jjaygbay1
Can you please check and add your comments on this doc update request as applicable.

Looks like this is updated as part of commit https://github.com/MicrosoftDocs/azure-docs/commit/f6f2ef99e7b3d83ff23b18a99afe06d9b4401354


 This causes major traffic jams and brings the city to a halt. I'd very much appreciate having some way to despawn just the car / train causing the problems. Here is an example of such a situation. The car in the blue circle is stuck. It's destination is the building to the right of it (opposite to the one demanding for police). Maybe something is wrong with the advanced parking AI?

Pressing the reset stuck vehicles / cims button doesn't help.
I've enabled all features, very high simulation accuracy, all improved KI options + parking KI, as well as realistic speeds.
Let me know if there is more I can do to help!

Yes, I noticed this as well. However it is really hard to find out what is going wrong. After the Mass Transit DLC is released I will get on that!

I think I identified the bug. Should be fixed in version 1.9.0 (coming out on Thursday).

Great, thank you!
I'll test and report once the new version is out.

Fixed and updated right now!

Fixed and updated right now!

Tested things a bit - so far, so good! 👍


copr: implement collation aware Vec-SlowHashAggr

What have you changed?
Based on https://github.com/tikv/tikv/pull/6608
Let Vec-SlowHashAggr executor regard the collation of ordering expressions.
What is the type of the changes?

New feature (a change which adds functionality)

How is the PR tested?

Unit test

/test

/run-tests

/run-all-tests

/bench +sysbench +tpch

/run-all-tests

The collation API change is awesome

/bench +sysbench +tpch

@TennyZhuang @breeswish PTAL

/run-all-tests

/merge


[#2729] Changing mvnw file permissions

Changing mvnw file permissions (+ executable permission)

Codecov Report

Merging #2846 into master will increase coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #2846      +/-   ##
==========================================
+ Coverage   40.86%   40.87%   +0.01%
==========================================
  Files        1783     1783
  Lines       61331    61331
  Branches     8623     8623
==========================================
+ Hits        25060    25068       +8
+ Misses      36271    36263       -8

Impacted Files
Coverage Δ

.../profiler/sender/StandbySpanStreamDataStorage.java
79.66% <0%> (-1.7%)
:arrow_down:

...ofiler/sender/StandbySpanStreamDataSendWorker.java
88.05% <0%> (-1.5%)
:arrow_down:

...rp/pinpoint/rpc/server/PinpointServerAcceptor.java
82.42% <0%> (+3.03%)
:arrow_up:

...m/navercorp/pinpoint/rpc/packet/RequestPacket.java
94.87% <0%> (+5.12%)
:arrow_up:

...vercorp/pinpoint/rpc/packet/ServerClosePacket.java
80% <0%> (+12%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a496f89...1947974. Read the comment docs.


Documentation for arrguments for various indicators

Where can I find documentation for the arguments for indicators?
Thanks

Hello @arainboldt,
By using help(ta.indicator_name) or checking the open source code.
Kind Regards,
KJ

Thanks.
Out of curiosity. Why haven't you set up sphinx (or alternative) autodocs for readthedocs?

@arainboldt
No worries.

Why haven't you set up sphinx (or alternative) autodocs for readthedocs?

I am a beginner with Sphinx v7.0.1 and unfortunately have not achieved the results I would like. However it is something I plan to do when I am at the point to merge development into main (v0.3.14b).
Do you have any experience with Sphinx (or an alternative)? If so, that would be helpful. 😎
KJ

@twopirllc that's understandable. I don't have much experience either.
Just played around with it a bit.


Empty arc in circuit constraint

I think we currently do not handle the case where one machine is empty. This can be done by adding self arcs for the source/sink node, see the following example:
https://github.com/google/or-tools/blob/main/ortools/sat/docs/scheduling.md#transitions-in-a-no_overlap-constraint
We should also add a test to check if this is indeed a problem.

While we're at it, also assign the explicit -1 source to a variable.


[DO NOT MERGE]LG: Optimize exception handling

Target: Can throw friendly error prompts at runtime.

This is a pretty big change, I can't be sure about all of this. I suggest we break this up, and merge this step by step.
The steps i visioned are

Make sure the exception is not intercepted by BailourErrorPolicy, and add test cases for this, in step 1, no LG grammar rule is changed.
Loose only one step of the grammar, let's say allow empty file. This would requires you have a StaticChecker(let's call it this way) after you parse file. This requires no changes in Evaluator.
3-n. then in step 3...N, each step you will do 3 things basicly

loose a little bit in grammar
add a little bit in StaticChecker
remove a little bit in Evaluator (maybe, just maybe, maybe we keep everything the same in Elevator)

I would like to see step 1 is in one PR, 2 is another, and 3-N can be in the 3rd PR.

That make sense


Fix #343

Fixes #343 #344

Changes proposed in this Pull Request:

Fix the missing backdrop click event.
Allow a user to manually control open state via @inputs

An important note about this fix is that the open property on the <pxb-user-menu> is now required instead of being optional.  I don't think this is a huge issue since v6.0.0 is already a breaking change and all of our documentation & storybook examples already assumed the open property was provided.
To test, I got the mat-chip menu overlay example to work using this version of the library.
https://pxblue-angular-design-patterns.web.app/

Thinking about this a little more, maybe I can tweak some things to make the open prop optional again... Looking into this today.

I was going to see if there's a way I can refactor this to make open not required, but then I realized that a usermenu (most of the time) is going to have clickable items in it that'll close the menu when clicked.

An important note about this fix is that the open property on the <pxb-user-menu> is now required instead of being optional. I don't think this is a huge issue since v6.0.0 is already a breaking change and all of our documentation & storybook examples already assumed the open property was provided.
To test, I got the mat-chip menu overlay example to work using this version of the library.
https://pxblue-angular-design-patterns.web.app/

Mat-chip inside User Menu is working perfectly fine now.

The functionality looks good. Not an Angular person so I trust you on the code change :P

I need code owner to check out this branch so we can potentially publish 6.0.0 tomorrow.  @jeffvg @daileytj


unable to use LightGBMClassifier in R due to wrong datatype

I have an R data.frame object which I would like to pass to the ml_light_gbmclassifier function using the following code:
model= ml_light_gbmclassifier(maxDepth=20, x=training, featuresCol=feat, labelCol=lab,numIterations=10)
where trainDF is an R data.frame
this throws an error saying that it cannot convert object data.frame to Spark DataFrame.
Error : Unable to retrieve a Spark DataFrame from object of class data.frame
I even tried using SparkR and creating a SparkDataFrame
so adding this line of code
trainDF = as.DataFrame(training, numPartitions=20)
his is still not accepted, and I receive the following:
Error : Unable to retrieve a Spark DataFrame from object of class SparkDataFrame

@elenaterenzi I added an example to clear things up here:
https://github.com/Azure/mmlspark/pull/377
I also added the ability to create "unfit" or "untrained" models since that functionality was missing and wouldn't work with some estimators that took another estimator as a parameter (eg train regressor/train classifier).

@imatiach-msft  can this issue be resolved?

@mhamilton723 @elenaterenzi  yes, we resolved over skype, I think we can close this now.  There are some issues with default params that need to be fixed in the R wrappers though that I need to look into more.


Godot 4 Beta AnimatedTexture Tilemap does not working

Godot version
4 Beta
System information
Windows 11
Issue description
I have created Tilemap by animated texture and create a small map but animated tilemap does not working
Steps to reproduce
No code
Minimal reproduction project
Need password to upload

Tile animations are done differently in Godot 4, you can see here for details: #52539
Not sure if any tutorial exists yet.

Thank you for your answer, I have tried to achieve that by sprite sheet and animation texture but could not do that. do you know what kind of object should be use to drag and drop into tileset ?!

Animations are created with spritesheets. You specify number of columns, separation and then add frames and it happens automatically. Here's a short video guide:
https://user-images.githubusercontent.com/2223172/193011163-7faf7964-fcf4-4573-9d48-f8ac95389b54.mp4

Thanks a lot it works perfect :))

Btw when creating a res as animated texture dosn't work anymore ?! if not what is the use case of it? Im new to Godot thats why I am asking this question

Yes, AnimatedTexture is no longer supported in TileMaps. The old workflow was that you'd assign an AnimatedTexture for each tile separately, so it required lots of setup to get your TileMap animated. The new workflow is much easier, because you can add animation for all tiles at once.
Closing, as this is intended behavior.


adds an API function to get information about upcoming events

I tried to make this as non-invasive as possible - but i did change the signature for a very old function to return an index instead of being declared void.  Added doxy markup for an explanation of the new API.

@samhatchett , in line 647 the control index is not returned, is it because its not needed there?

@eladsal that's right - the old behavior doesn't change within that calling code, this PR just hijacks the control timestep function get information back about which element caused the control action so it can be used by the new calling code.

Thank you. Can you please also add the single-threaded version of the function?

@eladsal yes, I would be happy to add s/t versions.

@eladsal I have made the suggested changes, if you wouldn't mind taking a look. thanks!

@samhatchett my concern is with the eventType in EN_timetonextevent. Such a type may be difficult to use in other programming languages. For example, in EN_getlinktype we use an Int for the link type. Is there a function we use a user-defined type?

@eladsal Looking over this again, I'm re-discovering that the C API relies on client code to handle the implicit casting from enum types to integers. I will change the new functions to reflect that convention. Thanks for the comments, and i regret that it's taken me this long to circle back!

Thank you @samhatchett, I think we are good to go.


Another robux phishing scam

What: This website falls into the umbrella category of “Scams and Phish”, but more specifically, both “Roblox scams” and “Scams targeting children”.
How: On Roblox Discord servers, warnings were being issued about this.
Here's a warning posted on the Databrawl Discord advising people to secure their accounts.
Example of a compromised individual's account.
Where: This scam lives at https://rbx.best/.

@CyanoTex this doesn't seem actionable.

@CyanoTex this doesn't seem actionable.

Understandable. At least I tried to save some unfortunate young souls from being phished. ☹

Maybe one of our curators will list this?
@hawkeye116477 you list rbxfree.com and www.rbxfree.com in KADhosts.  Is rbx.best related?

https://github.com/PolishFiltersTeam/KADhosts/commit/8740ba8409e1fd4726f5f430c5b0a91810d1adbb#diff-92d3b2a6b099788b883d18eb574d874bR878
https://github.com/PolishFiltersTeam/KADhosts/commit/8740ba8409e1fd4726f5f430c5b0a91810d1adbb#diff-92d3b2a6b099788b883d18eb574d874bL573
We are probably rbxfree.com on the list because of the popularity of the Roblox game, because now the site has no polonization.

Oher story is that young people mostly learn English as a second language, less frequently learn German; probably part as a third learn Spanish language (rbxfree.com).

Thanks Krystian @krystian3w.
Closing.


pharo interpreter cant handle the result of String new

image := PowertalkRingImage fromSpec
  wordSize: 8;
  genesis;
  bootstrap;
  fillClasses;
  fillSymbols.
bootstrapper := image generateLMR.
bootstrapper useHostModuleLoader.
runtime := bootstrapper result.

str1 := (runtime newString: '').
strClass := runtime sendLocal: #class to: str1.
str2 := runtime sendLocal: #new to: strClass.

If I try to inspect str2 my image ends up freezing.
This kind of instance is created by CharacterArray>>replaceAll:with:.
https://github.com/powerlang/egg/blob/faec18e724f67d062ff0dfae0003e077c170b15e/modules/Kernel/CharacterArray.st#L504

This was my fault due to some changes I did on LMRObject (an implementation of asLocalString)


Make it easier to import a trait "shadowed" by a derive macro

Write &dyn Debug.
Get compiler error E0404 "not a trait", because the standard library prelude contains the derive macro named Debug but not the trait named Debug.
Ask rust-analyzer to assist.

Right now, no assists are available. It would be nice if instead it offered to add use std::fmt::Debug;. The compiler does provide a suggestion: "consider importing one of these items instead: use serde::__private::fmt::Debug;, use std::fmt::Debug;, use core::fmt::Debug;" but evidently this doesn't translate into rust-analyzer assists.

Duplicate of https://github.com/rust-analyzer/rust-analyzer/issues/7408


example/drag: significant delay of a sprite move with Metal

Ebitengine Version
f6ffd86ca82ef6f9b25d25293750b31a3461ccf0
Operating System

[ ] Windows
[X] macOS
[ ] Linux
[ ] FreeBSD
[ ] OpenBSD
[ ] Android
[ ] iOS
[ ] Nintendo Switch
[ ] PlayStation 5
[ ] Xbox
[ ] Web Browsers

Go Version (go version)
go version go1.21.1 darwin/amd64
What steps will reproduce the problem?

Run examples/drag
Drag a sprite quickly

What is the expected result?
The sprite keeps up with the cursor position
What happens instead?
The sprite failed to keep up with the cursor position.
https://github.com/hajimehoshi/ebiten/assets/16950/23be3eee-ce1c-4fe0-93f2-bc871db18321
Anything else you feel useful to add?
OpenGL also has a similar delay, but this is much smaller.
https://github.com/hajimehoshi/ebiten/assets/16950/aeee0789-3ab3-4861-80fb-f5c913f1bd3a
I have not tested DirectX yet.
Handling events (#1704) didn't improve this situation significantly.

I found that asynchronous rendering (#2664) caused delays.

From the above figure, the time lag between the game's updatings and the rendering is increased in theory, and apparently this actually happened.
I'll disable this when vsync is on.

Now the situation with Metal was quite improved. Windows has a similar issue with both OpenGL and DirectX, but the delay seems almost the same as the current Metal.
An experimental implementation to handle events (#1704) didn't solve this issue.
Apparently disabling vsync improved the delay very much. For GUI applications or some special game applications, disabling vsync would be an option to suppress delays.

Reopen this. "2" didn't work well with fullscreen.

Apparently disabling vsync improved the delay very much. For GUI applications or some special game applications, disabling vsync (+ SetScreenClearedEveryFrame(false)) would be an option to suppress delays.

This conclustion is not changed.

I've observed this problem quite a bit on my MacBook Air M1 2020.  So I always run my games with OpenGL.
This is a screenshot from the drag demo running on my external monitor.  I wonder if the FPS is a contributor to the problem?  As you can see in the examples below, the metal FPS is always lower than the OpenGL FPS.
External monitor, Metal:
EBITENGINE_GRAPHICS_LIBRARY=metal go run .

External monitor, OpenGL:
EBITENGINE_GRAPHICS_LIBRARY=opengl go run .

When I run it on my laptops built in screen, I can still see the drag issue, but it's not nearly as obvious.
Builtin screen, Metal:
EBITENGINE_GRAPHICS_LIBRARY=metal go run .

Builtin screen, OpenGL:
EBITENGINE_GRAPHICS_LIBRARY=opengl go run .

The only change I made to the drag example was to replace the Debug info with one that includes the ActualFPS and ActualTPS.
% git rev-parse HEAD
6cdabf09d1ef3d04933c7a707f05a0339961a9d3

-       ebitenutil.DebugPrint(screen, "Drag & Drop the sprites!")
+       // ebitenutil.DebugPrint(screen, "Drag & Drop the sprites!")
+       msg := fmt.Sprintf("ActualFPS: %0.2f\nActualTPS: %0.2f", ebiten.ActualFPS(), ebiten.ActualTPS())
+       ebitenutil.DebugPrint(screen, msg)

As suggested, I also tried without VSYNC enabled.  This made the drag feel perfect for openGL, but I don't necessarily want my game running at an insanely high FPS.  For metal, it started out with a high FPS (400) but quickly dropped down to 30, making the drag feel bad again.
External monitor, Metal, With VSYNC enabled:
EBITENGINE_GRAPHICS_LIBRARY=meta go run .

External monitor, OpenGL, With VSYNC enabled:
EBITENGINE_GRAPHICS_LIBRARY=opengl go run .

I hope this helps a bit.  Thanks for Ebitengine!

@philhassey Which version of Ebitengine have you tested?

Well, I've sometimes noticed FPS is unstable with Metal (more unstanble than OpenGL, as we see), and the behavior is often flaky so I'm confused how to fix this issue. Especially if this happens with an external monitor, this is quite hard to reproduce this in my environment.

The version I was using was a clone from today off this hash:
% git rev-parse HEAD
6cdabf09d1ef3d04933c7a707f05a0339961a9d3

I've seen this in practice for the last few months since I started using Ebiten (~November 2023.)
I think the external display only makes it more obvious, so not required to reproduce.
On the built-in laptop screen, you can see in my tests Builtin screen, Metal: and Builtin screen, OpenGL: that the FPS in OpenGL is 120hz, whereas with Metal it's 60hz.  My guess is that the lower the FPS, the less "fresh" the events are that my game is fed before drawing the screen.  By having a higher FPS in OpenGL, it looks that much more up-to-date.
With my external monitor (which is running at 30hz as per the Display settings on the Mac), this becomes very obvious.  It almost seems like a bug that OpenGL runs at 90hz on my external display.  However, since that makes the event handling work better, it accidentally fixes it.
That's my guess anyways!  If you want me to try it out on a specific version of Ebiten, I'm glad to do whatever.

I failed to reproduce 60 FPS w/ Metal in my MacBook M3 Pro.

Actually FPS heavily depends on whether the application window is focused or not. When unfocused, the FPS dropped to 60.

Thanks for checking it out.  Could be just an early M1 issue.
Did you notice if
	ebiten.SetVsyncEnabled(false)

was effective when using Metal?
On my machine the FPS would spike to 400 or so then drop down after about 30 seconds to whatever the screen's FPS was (60 or 30 on mine.).
I also found that if I changed ebiten.SetTPS(240) the behavior was a bit stranger, never really settling anywhere, but FPS would vary wildly between 60 and 2000.

Did you notice if [...] was effective when using Metal?

Yes, this worked with 500-1000 FPS. I confirmed this with examples/windowsize and pressing V. This worked even after 30 seconds.

Could you file your issue as a new issue? Unstable FPS seems a different issue than this. Thanks,

Filed issue https://github.com/hajimehoshi/ebiten/issues/2931


Add internal support for SuperRecyclerView 1.1.0

@wKovacs64 not sure how much work it is. but can you do this on the develop branch? so i can stay with git flow ;)

Closing and resubmitting to develop branch.

@wKovacs64 thank you so much


[Docs] RPC port numbers

It seems there is no docs for port numbers of RPCs such as web3, grpc, etc. Also, I'm not sure if they are configurable or not.

We'd love a quick PR on this if you were able.
The ports are, indeed, configurable, via the config file you use. We dump defaults into the default config file provided when burrow config is ran.
If you'd like to see the defaults quickly in your terminal you can run burrow spec -f1 | burrow configure -s-

I would like to work on this if it nobody started working on this.

Please go ahead @deepakchethan!


Module not found: Can't resolve 'ajv/dist/core'

To Reproduce
I've a monorepo, after upgrading to 14.6.0, and fumadocs-openapi to 5.8.2
I'm no longer able to build
looks the issue is more on the @scalar/openapi-parser package than on yours, but just to register
Current vs. Expected behavior
After downgrading to
    "fumadocs-core": "14.5.4",
    "fumadocs-docgen": "^1.3.2",
    "fumadocs-mdx": "11.1.2",
    "fumadocs-openapi": "^5.7.5",
    "fumadocs-twoslash": "^2.0.1",
    "fumadocs-ui": "14.5.4",

everything works as expected
Provide environment information
    "fumadocs-core": "14.6.0",
    "fumadocs-docgen": "^1.3.2",
    "fumadocs-mdx": "11.1.2",
    "fumadocs-openapi": "^5.8.2",
    "fumadocs-twoslash": "^2.0.2",
    "fumadocs-ui": "14.6.0",

Which area(s) are affected? (Select all that apply)
Integrations (e.g OpenAPI, Typescript DocsGen)
Additional context
@rkt/web:build: ../../node_modules/ajv-draft-04/dist/vocabulary/draft4.js
@rkt/web:build: ../../node_modules/ajv-draft-04/dist/index.js
@rkt/web:build<EMAIL_ADDRESS>@rkt/web:build<EMAIL_ADDRESS>@rkt/web:build: ../../node_modules/fumadocs-openapi/dist/server/index.js
@rkt/web:build: ./app/source.ts
@rkt/web:build: ./app/(llms)/llms.txt/route.tsx
@rkt/web:build:
@rkt/web:build: ../../node_modules/ajv-draft-04/dist/vocabulary/draft4.js
@rkt/web:build: Module not found: Can't resolve 'ajv/dist/vocabularies/applicator'

You can install ajv-draft-04 to the project, that can probably fix the issue

I think I can work on another PR next month, for now I will add the dependency to Fumadocs itself as a temporary fix


Implement an ad-hoc client

Let's create a new binary in src/backend/bin/psql so we can leverage existing psql code.

A good way to test this before wiring it up into the in-process CQ is to manually send it INSERT, UPDATE, and DELETEs and verifying that it visualizes the continuous query result properly.


fix(store): fixed audit log query created before wrong unix time

Thanks a lot for this! Can you please add the same for deployments: https://github.com/NorthernTechHQ/nt-gui/blob/main/packages/store/src/deploymentsSlice/thunks.ts#L60-L61 and device troubleshoot sessions: https://github.com/NorthernTechHQ/nt-gui/blob/main/packages/store/src/devicesSlice/thunks.tsx#L775-L776 ? Thank you!

added. deployments will be handled in mender as the start point is here https://github.com/mendersoftware/mender-server/blob/main/frontend/src/js/components/deployments/pastdeployments.js#L90-L91


Forseti Explainer data-discrepancy

Looking at IAM, a user has been removed as project/owner (in fact has no privileges, inherited or otherwise). However, Forseti Explainer is reporting that a user does have the role/owner.

Issue OBE.  User indicated they are no longer receiving the notifications.  I've asked for additional information if available.  Will post the info here.


Implement open telemetry integration - client/server and explict token latency

Description
Integrate OpenTelemetry (metrics only) and expose it using Promteheus exporter for:

Server (incomming connection) (request/reponse size, duration) with http_route label for each handler:

http_server_duration_milliseconds (histogram: bucket/total/sum)
http_server_request_size_bytes_total (counter)
http_server_response_size_bytes_total (counter)

Client (all outgoing connection to services) with net_peer_name label to distinguish target:

http_client_duration_milliseconds (histogram: bucket/total/sum)
http_client_request_size_bytes_total (counter)
http_client_response_size_bytes_total (counter)

Explict Float64 historgrams for first and all next tokens lantecy:

llm_first_token_latency_milliseconds (histogram: bucket/total/sum)
llm_next_token_latency_milliseconds (histogram: bucket/total/sum)

Other default Go language metrics like: go_gc_duration_seconds, go_goroutines and others.

Additional feature is integrated opentelemetry trace context propagation (not tested).
Metrics are exposed on /metric with Prometheus handler. Example output for metrics:
1) server metrics example
# HELP http_server_duration_milliseconds Measures the duration of inbound HTTP requests.
# TYPE http_server_duration_milliseconds histogram
http_server_duration_milliseconds_bucket{http_method="POST",http_route="/",http_scheme="http",http_status_code="200",net_host_name="<IP_ADDRESS>",net_host_port="8080",net_protocol_name="http",net_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_version="0.49.0",le="0"} 0
http_server_duration_milliseconds_bucket{http_method="POST",http_route="/",http_scheme="http",http_status_code="200",net_host_name="<IP_ADDRESS>",net_host_port="8080",net_protocol_name="http",net_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_version="0.49.0",le="5"} 0
http_server_duration_milliseconds_bucket{http_method="POST",http_route="/",http_scheme="http",http_status_code="200",net_host_name="<IP_ADDRESS>",net_host_port="8080",net_protocol_name="http",net_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_version="0.49.0",le="+Inf"} 2
http_server_duration_milliseconds_sum{http_method="POST",http_route="/",http_scheme="http",http_status_code="200",net_host_name="<IP_ADDRESS>",net_host_port="8080",net_protocol_name="http",net_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_version="0.49.0"} 25597.877786
http_server_duration_milliseconds_count{http_method="POST",http_route="/",http_scheme="http",http_status_code="200",net_host_name="<IP_ADDRESS>",net_host_port="8080",net_protocol_name="http",net_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_version="0.49.0"} 2
# HELP http_server_request_size_bytes_total Measures the size of HTTP request messages.
# TYPE http_server_request_size_bytes_total counter
http_server_request_size_bytes_total{http_method="POST",http_route="/",http_scheme="http",http_status_code="200",net_host_name="<IP_ADDRESS>",net_host_port="8080",net_protocol_name="http",net_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_version="0.49.0"} 202
# HELP http_server_response_size_bytes_total Measures the size of HTTP response messages.
# TYPE http_server_response_size_bytes_total counter
http_server_response_size_bytes_total{http_method="POST",http_route="/",http_scheme="http",http_status_code="200",net_host_name="<IP_ADDRESS>",net_host_port="8080",net_protocol_name="http",net_protocol_version="1.1",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_version="0.49.0"} 134

2) client metrics example
# HELP http_client_duration_milliseconds Measures the duration of outbound HTTP requests.
# TYPE http_client_duration_milliseconds histogram
http_client_duration_milliseconds_bucket{http_method="POST",http_status_code="200",net_peer_name="embedding-svc.chatqa.svc.cluster.local",net_peer_port="6000",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_version="0.49.0",le="5000"} 2
...
http_client_duration_milliseconds_bucket{http_method="POST",http_status_code="200",net_peer_name="llm-svc.chatqa.svc.cluster.local",net_peer_port="9000",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_version="0.49.0",le="0"} 0
...
# HELP http_client_request_size_bytes_total Measures the size of HTTP request messages.
# TYPE http_client_request_size_bytes_total counter
http_client_request_size_bytes_total{http_method="POST",http_status_code="200",net_peer_name="embedding-svc.chatqa.svc.cluster.local",net_peer_port="6000",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_version="0.49.0"} 202
http_client_request_size_bytes_total{http_method="POST",http_status_code="200",net_peer_name="llm-svc.chatqa.svc.cluster.local",net_peer_port="9000",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_version="0.49.0"} 3670
http_client_request_size_bytes_total{http_method="POST",http_status_code="200",net_peer_name="reranking-svc.chatqa.svc.cluster.local",net_peer_port="8000",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_version="0.49.0"} 14076
http_client_request_size_bytes_total{http_method="POST",http_status_code="200",net_peer_name="retriever-svc.chatqa.svc.cluster.local",net_peer_port="7000",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_version="0.49.0"} 19498
# HELP http_client_response_size_bytes_total Measures the size of HTTP response messages.
# TYPE http_client_response_size_bytes_total counter
http_client_response_size_bytes_total{http_method="POST",http_status_code="200",net_peer_name="embedding-svc.chatqa.svc.cluster.local",net_peer_port="6000",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_version="0.49.0"} 19422
http_client_response_size_bytes_total{http_method="POST",http_status_code="200",net_peer_name="llm-svc.chatqa.svc.cluster.local",net_peer_port="9000",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_version="0.49.0"} 480
http_client_response_size_bytes_total{http_method="POST",http_status_code="200",net_peer_name="reranking-svc.chatqa.svc.cluster.local",net_peer_port="8000",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_version="0.49.0"} 3640
http_client_response_size_bytes_total{http_method="POST",http_status_code="200",net_peer_name="retriever-svc.chatqa.svc.cluster.local",net_peer_port="7000",otel_scope_name="go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp",otel_scope_version="0.49.0"} 13972

3) custom metrics example
# HELP llm_first_token_latency_milliseconds Measures the first token generate duration of.
# TYPE llm_first_token_latency_milliseconds histogram
llm_first_token_latency_milliseconds_bucket{otel_scope_name="entrag-telemetry",otel_scope_version="",le="1"} 0
llm_first_token_latency_milliseconds_bucket{otel_scope_name="entrag-telemetry",otel_scope_version="",le="64"} 0
...
llm_first_token_latency_milliseconds_bucket{otel_scope_name="entrag-telemetry",otel_scope_version="",le="+Inf"} 2
llm_first_token_latency_milliseconds_sum{otel_scope_name="entrag-telemetry",otel_scope_version=""} 9550.203300000001
llm_first_token_latency_milliseconds_count{otel_scope_name="entrag-telemetry",otel_scope_version=""} 2
# HELP llm_next_token_latency_milliseconds Measures the next token generate duration of.
# TYPE llm_next_token_latency_milliseconds histogram
llm_next_token_latency_milliseconds_bucket{otel_scope_name="entrag-telemetry",otel_scope_version="",le="1"} 4
llm_next_token_latency_milliseconds_bucket{otel_scope_name="entrag-telemetry",otel_scope_version="",le="64"} 4
...
llm_next_token_latency_milliseconds_bucket{otel_scope_name="entrag-telemetry",otel_scope_version="",le="+Inf"} 36
llm_next_token_latency_milliseconds_sum{otel_scope_name="entrag-telemetry",otel_scope_version=""} 14365.578998999998
llm_next_token_latency_milliseconds_count{otel_scope_name="entrag-telemetry",otel_scope_version=""} 36

4) Prometheus client default Go language metrics:
skl212 ➜  ~ curl -sL -N http://<IP_ADDRESS>:8080/metrics G go_ | grep -v HELP | grep -v TYPE
go_gc_duration_seconds{quantile="0"} 9.7515e-05
go_gc_duration_seconds{quantile="0.25"} 0.000102628
go_gc_duration_seconds{quantile="0.5"} 0.000134259
go_gc_duration_seconds{quantile="0.75"} 0.000141445
go_gc_duration_seconds{quantile="1"} 0.000191476
go_gc_duration_seconds_sum 0.00130944
go_gc_duration_seconds_count 10
go_goroutines 8
go_info{version="go1.22.4"} 1
go_memstats_alloc_bytes 2.795128e+06
....
go_memstats_sys_bytes 2.276456e+07
go_threads 23

Issues
n/a
Type of change

[x] New feature (non-breaking change which adds new functionality)

Dependencies

open telemetry go sdk (+prometheus exporter)

go.opentelemetry.io/otel
go.opentelemetry.io/otel/exporters/prometheus
go.opentelemetry.io/otel/metric
go.opentelemetry.io/otel/sdk/metric
go.opentelemetry.io/otel/metric

prometheus client:

github.com/prometheus/client_golang/prometheus/promhttp

open telemetry go contrib http client/server intrumentation

go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp

Tests
Manually using curl (examples show above) after running only with ChatQnA sample graph.
TODO:

[ ] better or new label and attributes (e.g. otel_scope_name...)
[ ] better buckets boundries for token latencies
[ ] reconfigure default bucket boundries for server/client latnecy metrics
[ ] test OTEL trace propagation
[ ] write unit tests
[ ] document
[x] remove some comments with examples

Hi @ppalucki, thanks for your effort on this PR!
In mcGraphHandler :
responseBody, statusCode, err := routeStep(defaultNodeName, *mcGraph, inputBytes, inputBytes, req.Header)

I think there should be a measurement after routeStep returns the responseBody because this produce contains the duration for going through all microservice in the pipeline. And this information can calculate and evaluate the handling time for GMC router. It really is helpful for GMC router.
I think you also can support the measurement metrics in mcDataHandler which is focusing on data embedding and storage in VectorDB.

Hi @ppalucki, thanks for your effort on this PR! In mcGraphHandler :
responseBody, statusCode, err := routeStep(defaultNodeName, *mcGraph, inputBytes, inputBytes, req.Header)

I think there should be a measurement after routeStep returns the responseBody because this produce contains the duration for going through all microservice in the pipeline. And this information can calculate and evaluate the handling time for GMC router. It really is helpful for GMC router.
I think you also can support the measurement metrics in mcDataHandler which is focusing on data embedding and storage in VectorDB.

I quickly added all_token_latency, which is essenatialy sum of first and "all"  next tokens, but still not sure that it covers what you wanted e.g. "user facing" latency of all steps in pipeline + network.
I need to test it more carefully.
llm_all_token_latency_milliseconds_sum{otel_scope_name="entrag-telemetry",otel_scope_version=""}<PHONE_NUMBER>84
llm_first_token_latency_milliseconds_sum{otel_scope_name="entrag-telemetry",otel_scope_version=""} 85542.49316200001
llm_next_token_latency_milliseconds_sum{otel_scope_name="entrag-telemetry",otel_scope_version=""}<PHONE_NUMBER>5300014

Hi @ppalucki, I think you may need to fix the DCO error by adding the Signed-off-by with your github id and email.

See also: https://github.com/opea-project/GenAIComps/pull/845

This PR is going to be superseded by new work - please check this #644
Also some issues were found:

first token latency should be be renamed to TTFT (time-to-first-token) or better "time to first bytes"
next token latency should be called TPOT (time-per-output-token) or ITL (inter-token-latency)
first token should measure not just "model server" first token generation but also pipeline latency (before request even is redirected to model llm-server) and first token should reflect that time

Here is proposed change semantic to generated metrics:
Currently (wrong):
[ embed]->[retrive]->[reranking]->[vllm]
                                  "first  token"   "token1"  "token2"

<-------- pipeline latency -->    <-------- all token latency ------>
                                  <-first token->  <--ntl--> <--ntl-->

and should be (fixed):
[ embed]->[retrive]->[reranking]->[vllm]
                                  "first  token"   "token1"  "token2"

<------------------------- e2e latency ------------------------------>
<-------- pipeline latency -----> <---------- model server latency -->
<-----------------------------------first token->  <--ntl--> <--ntl-->

where ntl stands for "next token latnecy"


Building on Raspberry Pi fails with "unrecognized command line option"

I'm getting an error when trying to build te project on raspberry pi:
g++: error: unrecognized command line option ‘-m32’
Environment:
Raspberry Pi model v2
bgfx, bx and GENie cloned in the same workspace folder
GENie compiled on rpi and copied in bx/tools/bin/linux
I've used the following command to create project files
../bx/tools/bin/linux/genie --gcc=rpi gmake
Note:
I did a quick dirty hack and removed the option from toolchain.lua, and regenerated the project files again with GENie. The error is gone but then there are a lot of other issues during compilation.

Use just config=debug/release instead specifying bitness.

Thanks Branimir,
that was real fast reponse, using make config=debug/release - j 4 instead of simple make -j 4 seems to solve the "unrecognized command line option" error.
I still get some compilation error in glcontext_egl.cpp, someting about x11SetDisplayWindows being declared void.
I'll investigate further and come back with findings.
Maybe it is worth to note that I'm not using X11 windows server at all but relying on dispmanx/EGL using the Broadcom drivers. I'm also pretty lame when it comes to linux toolchains and environments so I might make mistakes.
Thanks,
Catalin

I tried compiling with no success on Raspberry Pi. It seems that bgfx is assuming EGL 1.4 or above version of headers for EGL. From what I've encountered it seems that the current Raspbian Jessie image available from raspberrypi.org contains older EGL headers (version 1.3?? not sure) but it is clear that some defines are missing. Like: EGL_CONTEXT_MAJOR_VERSION_KHR, EGL_CONTEXT_MINOR_VERSION_KHR, EGL_CONTEXT_OPENGL_DEBUG_BIT_KHR, EGL_CONTEXT_FLAGS_KHR.
In order to get to these missing declarations errors I had to comment out first the definition of x11SetDisplayWindow ( void x11SetDisplayWindow(::Display* _display, ::Window _window) ) in glcontext_egl.cpp as it seems a different declaration of the function was already available in bgfxplatform.h ( inline void x11SetDisplayWindow(void* _display, uint32_t _window, void* _glx = NULL) )
Right now i'm trying to merge Khronos official EGL headers 1.5 with the ones from Raspberry Pi. Completely replacing rpi EGL headers is a no go due to custom rpi stuff in those headers.
Am I missing something about building bgfx on rpi? According to Readme, rpi should be a supported platform but so far I wasn't able to build it. I didn't see any platform specific build instructions for rpi either.
Cheers,
Catalin

Haven't check in a while, will try sometime this week.

It should be fixed now. Grab latest.

It works, thanks