3.0 breaks connections using ansible

I'm trying out version 3.0.1 with --connection=ansible and it breaks on ansible_runner. The same command works fine with 2.1.0:
pytest -vvv --connection=ansible --ansible-inventory=hosts test_remote.py
collected 0 items / 1 errors

=================================== ERRORS ====================================
_______________________ ERROR collecting test_remote.py _______________________
../../../../.virtualenvs/ceph-volume/lib/python3.6/site-packages/pluggy/hooks.py:289: in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
../../../../.virtualenvs/ceph-volume/lib/python3.6/site-packages/pluggy/manager.py:68: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
../../../../.virtualenvs/ceph-volume/lib/python3.6/site-packages/pluggy/manager.py:62: in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
../../../../.virtualenvs/ceph-volume/lib/python3.6/site-packages/_pytest/python.py:238: in pytest_pycollect_makeitem
    res = list(collector._genfunctions(name, obj))
../../../../.virtualenvs/ceph-volume/lib/python3.6/site-packages/_pytest/python.py:414: in _genfunctions
    self.ihook.pytest_generate_tests(metafunc=metafunc)
../../../../.virtualenvs/ceph-volume/lib/python3.6/site-packages/pluggy/hooks.py:289: in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
../../../../.virtualenvs/ceph-volume/lib/python3.6/site-packages/pluggy/manager.py:68: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
../../../../.virtualenvs/ceph-volume/lib/python3.6/site-packages/pluggy/manager.py:62: in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
../../../../.virtualenvs/ceph-volume/lib/python3.6/site-packages/testinfra/plugin.py:113: in pytest_generate_tests
    ansible_inventory=metafunc.config.option.ansible_inventory,
../../../../.virtualenvs/ceph-volume/lib/python3.6/site-packages/testinfra/host.py:143: in get_hosts
    for backend in testinfra.backend.get_backends(hosts, **kwargs):
../../../../.virtualenvs/ceph-volume/lib/python3.6/site-packages/testinfra/backend/__init__.py:91: in get_backends
    for name in klass.get_hosts(host, **kw):
../../../../.virtualenvs/ceph-volume/lib/python3.6/site-packages/testinfra/backend/ansible.py:59: in get_hosts
    return AnsibleRunner.get_runner(inventory).get_hosts(host)
../../../../.virtualenvs/ceph-volume/lib/python3.6/site-packages/testinfra/utils/ansible_runner.py:124: in get_hosts
    groupmatch = fnmatch.fnmatch(group, pattern)
../../../../.virtualenvs/ceph-volume/lib/python3.6/fnmatch.py:35: in fnmatch
    pat = os.path.normcase(pat)
../../../../.virtualenvs/ceph-volume/lib/python3.6/posixpath.py:54: in normcase
    s = os.fspath(s)
E   TypeError: expected str, bytes or os.PathLike object, not NoneType
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!

With 2.1.0 is fine:
pytest -vvv --connection=ansible --ansible-inventory=hosts test_remote.py
============================= test session starts =============================
platform linux2 -- Python 2.7.16, pytest-4.4.1, py-1.8.0, pluggy-0.10.0 -- /tmp/27/bin/python2
cachedir: .pytest_cache
rootdir: /home/alfredo/python/
plugins: testinfra-2.1.0
collected 2 items

test_remote.py::test_release_file[ansible://node2] PASSED               [ 50%]
test_remote.py::test_release_file[ansible://node3] PASSED               [100%]

The test file:
def test_release_file(host):
    release_file = host.file("/etc/os-release")
    assert release_file.contains('CentOS')
    assert release_file.contains('VERSION="7 (Core)"')

And the hosts file:
[nginx]
node2
node3

If I try passing --hosts=all testinfra assumes an SSH connection:
transport.py              1746 ERROR    Exception: key cannot be used for signing
transport.py              1744 ERROR    Traceback (most recent call last):
transport.py              1744 ERROR      File "/home/alfredo/.virtualenvs/ceph-volume/lib/python3.6/site-packages/paramiko/transport.py", line 2036, in run
transport.py              1744 ERROR        handler(self.auth_handler, m)
transport.py              1744 ERROR      File "/home/alfredo/.virtualenvs/ceph-volume/lib/python3.6/site-packages/paramiko/auth_handler.py", line 298, in _parse_service_accept
transport.py              1744 ERROR        sig = self.private_key.sign_ssh_data(blob)
transport.py              1744 ERROR      File "/home/alfredo/.virtualenvs/ceph-volume/lib/python3.6/site-packages/paramiko/agent.py", line 418, in sign_ssh_data
transport.py              1744 ERROR        raise SSHException("key cannot be used for signing")
transport.py              1744 ERROR    paramiko.ssh_exception.SSHException: key cannot be used for signing
transport.py              1744 ERROR
transport.py              1746 ERROR    Exception: key cannot be used for signing
transport.py              1744 ERROR    Traceback (most recent call last):
transport.py              1744 ERROR      File "/home/alfredo/.virtualenvs/ceph-volume/lib/python3.6/site-packages/paramiko/transport.py", line 2036, in run
transport.py              1744 ERROR        handler(self.auth_handler, m)
transport.py              1744 ERROR      File "/home/alfredo/.virtualenvs/ceph-volume/lib/python3.6/site-packages/paramiko/auth_handler.py", line 298, in _parse_service_accept
transport.py              1744 ERROR        sig = self.private_key.sign_ssh_data(blob)
transport.py              1744 ERROR      File "/home/alfredo/.virtualenvs/ceph-volume/lib/python3.6/site-packages/paramiko/agent.py", line 418, in sign_ssh_data
transport.py              1744 ERROR        raise SSHException("key cannot be used for signing")
transport.py              1744 ERROR    paramiko.ssh_exception.SSHException: key cannot be used for signing
transport.py              1744 ERROR

If I explicitly use --hosts='ansible://node1 (node1 is a VM) it thinks it is a local connection (the test fails because the local machine is not CentOS):
pytest -vvv --hosts='ansible://node1' test_remote.py
============================= test session starts =============================
platform linux -- Python 3.6.8, pytest-4.4.1, py-1.8.0, pluggy-0.9.0 -- /home/alfredo/.virtualenvs/ceph-volume/bin/python3.6
cachedir: .pytest_cache
rootdir: /home/alfredo/python/
plugins: testinfra-3.0.0, xdist-1.28.0, forked-1.0.2
collected 1 item

test_remote.py::test_release_file[ansible://localhost] FAILED           [100%]

================================== FAILURES ===================================
___________________ test_release_file[ansible://localhost] ____________________

host = <testinfra.host.Host object at 0x7fc4beeb7400>

    def test_release_file(host):
        release_file = host.file("/etc/os-release")
>       assert release_file.contains('CentOS')
E       AssertionError: assert False
E        +  where False = <bound method File.contains of <file /etc/os-release>>('CentOS')
E        +    where <bound method File.contains of <file /etc/os-release>> = <file /etc/os-release>.contains

test_remote.py:4: AssertionError
------------------------------ Captured log call ------------------------------
base.py                    268 INFO     RUN CommandResult(command=b'uname -s', exit_status=0, stdout=b'Linux\n', stderr=None)
base.py                    268 INFO     RUN CommandResult(command=b'lsb_release -a', exit_status=0, stdout=b'Distributor ID:\tUbuntu\nDescription:\tUbuntu 19.04\nRelease:\t19.04\nCodename:\tdisco\n', stderr=b'No LSB modules are available.\n')
base.py                    268 INFO     RUN CommandResult(command=b'grep -qs -- CentOS /etc/os-release', exit_status=1, stdout=None, stderr=None)
========================== 1 failed in 0.79 seconds ===========================

Hi, from traceback code, it looks you're still using testinfra 3.0.0 and not 3.0.1
Can you retry with testinfra 3.0.1 ?
That being said, the behavior you get is very weird even with bogus 3.0.0, I cannot reproduce locally:
$ virtualenv -p python3 v
$ source v/bin/activate
$ pip install testinfra ansible
$ echo "$(hostname)" > inventory
$ cat << EOF > t.py
def test_dummy(host):
    assert host.run('false').rc == 1
EOF
# euler is my hostname
$ python -m pytest -v --connection=ansible --ansible-inventory=inventory t.py
t.py::test_dummy[ansible://euler] PASSED
$ python -m pytest -v --connection=ansible --ansible-inventory=inventory --hosts=all t.py
t.py::test_dummy[ansible://euler] PASSED
$ python -m pytest -v --ansible-inventory=inventory --hosts=ansible://all t.py
t.py::test_dummy[ansible://euler] PASSED
$ python -m pytest -v --ansible-inventory=inventory --hosts=ansible://euler t.py
t.py::test_dummy[ansible://euler] PASSED
$ python -m pytest -v '--hosts=ansible://euler?ansible_inventory=inventory' t.py
t.py::test_dummy[ansible://euler] PASSED
$ python -m pytest -v --hosts=ansible://localhost t.py
t.py::test_dummy[ansible://localhost] PASSED
$ python -m pytest -v --hosts=ansible://euler t.py
t.py::test_dummy[ansible://localhost] PASSED

Only the latest test case is wrong, because when inventory is empty (not specified), it return a local connection to "localhost" which doesn't match the given pattern. We need to fix this.

I've upgraded to 3.0.1 with similar results :(
The output from pytest is very long, but these two makes me assume that keys aren't being loaded properly:
test_remote.py::test_release_file[ansible://node3]
  /home/alfredo/.virtualenvs/ceph-volume/lib/python3.6/site-packages/paramiko/client.py:822: UserWarning: Unknown ssh-ed25519 host key for node3: b'52cdb9e883c2548b6ea68fe292db0eeb'
    key.get_name(), hostname, hexlify(key.get_fingerprint())
---

And this one is the same as before with 3.0.0

transport.py              1746 ERROR    Exception: key cannot be used for signing
transport.py              1744 ERROR    Traceback (most recent call last):
transport.py              1744 ERROR      File "/home/alfredo/.virtualenvs/ceph-volume/lib/python3.6/site-packages/paramiko/transport.py", line 2036, in run
transport.py              1744 ERROR        handler(self.auth_handler, m)
transport.py              1744 ERROR      File "/home/alfredo/.virtualenvs/ceph-volume/lib/python3.6/site-packages/paramiko/auth_handler.py", line 298, in _parse_service_accept
transport.py              1744 ERROR        sig = self.private_key.sign_ssh_data(blob)
transport.py              1744 ERROR      File "/home/alfredo/.virtualenvs/ceph-volume/lib/python3.6/site-packages/paramiko/agent.py", line 418, in sign_ssh_data
transport.py              1744 ERROR        raise SSHException("key cannot be used for signing")
transport.py              1744 ERROR    paramiko.ssh_exception.SSHException: key cannot be used for signing
transport.py              1744 ERROR

Hmm, where do you define ssh keys ? In inventory, ansible.cfg, environment variables ?
How can I reproduce the issue ? Any link to a test suite on github or travis/jenkins build ?

My SSH keys are in $HOME/.ssh/config and 2.1.0 just works without specifying anything. I will try to get you a reproducer - these are all reproducible locally with new virtualenvs.
I will try with a vanilla VM and see if I can reproduce. Not sure what is special about my machine

Ok I have a bit of a reproducer for at least one of the problems. CentOS 7, Python 2.7:
$ cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

Here are the versions installed in the virtualenv:
ansible (2.7.10)
asn1crypto (0.24.0)
atomicwrites (1.3.0)
attrs (19.1.0)
bcrypt (3.1.6)
cffi (1.12.3)
cryptography (2.6.1)
enum34 (1.1.6)
funcsigs (1.0.2)
ipaddress (1.0.22)
Jinja2 (2.10.1)
MarkupSafe (1.1.1)
more-itertools (5.0.0)
paramiko (2.4.2)
pathlib2 (2.3.3)
pip (9.0.1)
pluggy (0.11.0)
py (1.8.0)
pyasn1 (0.4.5)
pycparser (2.19)
PyNaCl (1.3.0)
pytest (4.4.1)
PyYAML (5.1)
scandir (1.10.0)
setuptools (28.8.0)
six (1.12.0)
testinfra (3.0.1)
wheel (0.29.0)

The test:
$ cat test_remote.py

def test_release_file(host):
    release_file = host.file("/etc/os-release")
    assert release_file.contains('CentOS')
    assert release_file.contains('VERSION="7 (Core)"')

The invocation shows I am specifying an IP which is remote (not the local one), and yet it connects to localhost:
$ pytest -v --hosts='ansible://<IP_ADDRESS>' test_remote.py
============================= test session starts =============================
platform linux2 -- Python 2.7.5, pytest-4.4.1, py-1.8.0, pluggy-0.11.0 -- /home/vagrant/tmp/bin/python
cachedir: .pytest_cache
rootdir: /home/vagrant/tmp
plugins: testinfra-3.0.1
collected 1 item

test_remote.py::test_release_file[ansible://localhost] PASSED           [100%]

Yes, you have to pass an ansible inventory here (with --ansible-inventory or by adding ?ansible_inventory=... to the --host option) or testinfra will use the default from ansible-inventory --list.
I'll fix this behavior to have the same behavior than testinfra 2.X which skip the test.

Ok.
            - Any key we can find through an SSH agent (if allowed).
            - Any "id_rsa", "id_dsa" or "id_ecdsa" key discoverable in ~/.ssh/
              (if allowed).
            - Plain username/password auth, if a password was given.

        (The password might be needed to unlock a private key [if 'passphrase'
        isn't also given], or for two-factor authentication [for which it is
        required].)
        """
        saved_exception = None
        two_factor = False
        allowed_types = set()
        two_factor_types = {"keyboard-interactive", "password"}
        if passphrase is None and password is not None:
            passphrase = password

        # If GSS-API support and GSS-PI Key Exchange was performed, we attempt
        # authentication with gssapi-keyex.
        if gss_kex and self._transport.gss_kex_used:
            try:
                self._transport.auth_gssapi_keyex(username)
                return
            except Exception as e:
                saved_exception = e

        # Try GSS-API authentication (gssapi-with-mic) only if GSS-API Key
        # Exchange is not performed, because if we use GSS-API for the key
        # exchange, there is already a fully established GSS-API context, so
        # why should we do that again?
        if gss_auth:
            try:
                return self._transport.auth_gssapi_with_mic(
                    username, gss_host, gss_deleg_creds
                )
            except Exception as e:
                saved_exception = e

        if pkey is not None:
            try:
                self._log(
                    DEBUG,
                    "Trying SSH key {}".format(
                        hexlify(pkey.get_fingerprint())
                    ),
                )
                allowed_types = set(
                    self._transport.auth_publickey(username, pkey)
                )
                two_factor = allowed_types & two_factor_types
                if not two_factor:
                    return
            except SSHException as e:
                saved_exception = e

        if not two_factor:
            for key_filename in key_filenames:
                for pkey_class in (RSAKey, DSSKey, ECDSAKey, Ed25519Key):
                    try:
                        key = self._key_from_filepath(
                            key_filename, pkey_class, passphrase
                        )
                        allowed_types = set(
                            self._transport.auth_publickey(username, key)
                        )
                        two_factor = allowed_types & two_factor_types
                        if not two_factor:
                            return
                        break
                    except SSHException as e:
                        saved_exception = e

        if not two_factor and allow_agent:
            if self._agent is None:
                self._agent = Agent()

            for key in self._agent.get_keys():
                try:
                    id_ = hexlify(key.get_fingerprint())
                    self._log(DEBUG, "Trying SSH agent key {}".format(id_))
                    # for 2-factor auth a successfully auth'd key password
                    # will return an allowed 2fac auth method
                    allowed_types = set(
                        self._transport.auth_publickey(username, key)
                    )
                    two_factor = allowed_types & two_factor_types
                    if not two_factor:
                        return
                    break
                except SSHException as e:
                    saved_exception = e

        if not two_factor:
            keyfiles = []

            for keytype, name in [
                (RSAKey, "rsa"),
                (DSSKey, "dsa"),
                (ECDSAKey, "ecdsa"),
                (Ed25519Key, "ed25519"),
            ]:
                # ~/ssh/ is for windows
                for directory in [".ssh", "ssh"]:
                    full_path = os.path.expanduser(
                        "~/{}/id_{}".format(directory, name)
                    )
                    if os.path.isfile(full_path):
                        # TODO: only do this append if below did not run
                        keyfiles.append((keytype, full_path))
                        if os.path.isfile(full_path + "-cert.pub"):
                            keyfiles.append((keytype, full_path + "-cert.pub"))

            if not look_for_keys:
                keyfiles = []

            for pkey_class, filename in keyfiles:
                try:
                    key = self._key_from_filepath(
                        filename, pkey_class, passphrase
                    )
                    # for 2-factor auth a successfully auth'd key will result
                    # in ['password']
                    allowed_types = set(
                        self._transport.auth_publickey(username, key)
                    )
                    two_factor = allowed_types & two_factor_types
                    if not two_factor:
                        return
                    break
                except (SSHException, IOError) as e:
                    saved_exception = e

        if password is not None:
            try:
                self._transport.auth_password(username, password)
                return
            except SSHException as e:
                saved_exception = e
        elif two_factor:
            try:
                self._transport.auth_interactive_dumb(username)
                return
            except SSHException as e:
                saved_exception = e

        # if we got an auth-failed exception earlier, re-raise it
        if saved_exception is not None:
            raise saved_exception
>       raise SSHException("No authentication methods available")
E       SSHException: No authentication methods available

lib/python2.7/site-packages/paramiko/client.py:750: SSHException
================================================================================================ warnings summary =================================================================================================
test_remote.py::test_release_file[ansible://node3]
  /home/vagrant/tmp/lib/python2.7/site-packages/paramiko/kex_ecdh_nist.py:39: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
    m.add_string(self.Q_C.public_numbers().encode_point())

test_remote.py::test_release_file[ansible://node3]
  /home/vagrant/tmp/lib/python2.7/site-packages/paramiko/kex_ecdh_nist.py:96: CryptographyDeprecationWarning: Support for unsafe construction of public numbers from encoded data will be removed in a future version. Please use EllipticCurvePublicKey.from_encoded_point
    self.curve, Q_S_bytes

test_remote.py::test_release_file[ansible://node3]
  /home/vagrant/tmp/lib/python2.7/site-packages/paramiko/kex_ecdh_nist.py:111: CryptographyDeprecationWarning: encode_point has been deprecated on EllipticCurvePublicNumbers and will be removed in a future version. Please use EllipticCurvePublicKey.public_bytes to obtain both compressed and uncompressed point encoding.
    hm.add_string(self.Q_C.public_numbers().encode_point())

test_remote.py::test_release_file[ansible://node3]
  /home/vagrant/tmp/lib/python2.7/site-packages/paramiko/client.py:822: UserWarning: Unknown ssh-ed25519 host key for node3: 52cdb9e883c2548b6ea68fe292db0eeb
    key.get_name(), hostname, hexlify(key.get_fingerprint())

-- Docs: https://docs.pytest.org/en/latest/warnings.html
====================================================================================== 1 failed, 4 warnings in 1.05 seconds =======================================================================================
(tmp) [vagrant@node2 tmp]$ ssh node3
Warning: Permanently added '<IP_ADDRESS>' (ECDSA) to the list of known hosts.
Last login: Tue May  7 19:36:02 2019 from <IP_ADDRESS>
[vagrant@node3 ~]$ logout
Connection to <IP_ADDRESS> closed.

So I can ssh without problems to node3 (<IP_ADDRESS>) but can't with pytest/testinfra even with --ansible-inventory. I tried both the IP and the hostname with the same behavior.
Would that setup be enough for a reproducer? Have you tried a fresh/vanilla CentOS VM?

Oh I see, this is because testinfra with paramiko backend doesn't read ~/.ssh/config, does it works if you change "paramiko" by "ssh" here https://github.com/philpep/testinfra/blob/master/testinfra/utils/ansible_runner.py#L81 (or just drop the "if" statement).
I think reading user ~/.ssh/config by default in paramiko connection seems more reasonable than using plain ssh calls, I'll make this change.

if I change that line to 'ssh' it works like a champ!
I think reading ~/.ssh/config makes sense, but I would also like to add that using --ssh-config=~/.ssh/config didn't had any effect
Thank you so much for your patience while trying to fix this!

Yes I make sense to support --ssh-config and --ssh-identity-file here as well. I'll make this.
Thanks for reporting this bug, I this will save time to a lot of users using testinfra+ansible !

I just released 3.0.2 with these fixes, tell me if it works for you.

3.0.2 works! Thank you so much


Add firecracker vsock at instance startup

This commit adds the init code to create a vsock per instance. This is related to Issue #87

@m-kat this is code I had created prior you taking over issue #87 The code here only adds the creation of the vsock per instance and not more. I found that firecracker does not cleanup the socket file on the host when the instance terminates. So firecracker-pilot needs to this. Other than that if you register /bin/bash as a flake you can test the socket communication with this patch as follows:

In the guest shell
socat VSOCK-LISTEN:42,fork -

On the host
socat - UNIX-CONNECT:/run/sci_cmd_24965.sock
CONNECT 42

So some tasks are still open

[ ] implement socket file cleanup in firecracker-pilot
[ ] implement sending of socket "CONNECT ID" string in firecracker-pilot
[ ] implement sending of command to socket in firecracker-pilot
[ ] implement listen on socket in sci

@m-kat feel free to merge this when starting your implementation


build: run e2e tests with bazel

Builds the e2e app with Bazel
Serves the e2e app with Bazel
Switches e2e tests completely to async/await. The promise manager from Selenium has been removed in the latest version. See https://github.com/SeleniumHQ/selenium/issues/2969
Runs the e2e tests with Bazel protractor rules

Note: This already simplifies our setup a lot and also makes it easier for people to run e2e tests locally also w/ AOT.
I want to speed up these e2e tests in a follow-up by moving them into individual web_tests rather than into a suite. This enables better concurrency

This PR is ready for review and works as expected, but is blocked on #14622.


Update and clean up Ubuntu docker files

This issue collects a number of updates and cleanups that have been pending for some time. See "possible follow-ups" below for a non-exhaustive list of things that are out of scope of this PR.

[x] Remove the Ubuntu 17.10 image - this has never been used on the CI and is no longer supported. Just remove the file. (https://github.com/ARMmbed/mbedtls-test/pull/16)
[x] Remove the Debian images - they're not used by the CI and probably very out of date. (They're referenced by two variables in vars/common.groovy but those variables are no longer used - remove them too.) )(https://github.com/ARMmbed/mbedtls-test/pull/16)
[ ] Update the Docker Readme and run.sh script to include --security-opt seccomp=unconfined (see below).
[ ] Update the value of the ARMLMD_LICENSE_FILE environment variable in the two remaining images. The correct value can be found in vars/gen_jobs.groovy which can then be adapted to no longer set that variable (and the corresponding comments removed). (See on the old repo: 147 to apply and 149 to undo.)
[ ] Align the structure and contents of the Ubuntu 18.04 image with that of the Ubuntu 16.04 image (see below).

Seccomp: In dev_envs/docker_files/run.sh, add --security-opt seccomp=unconfined to the docker run line, and add an explanation about it in dev_envs/docker_files/README.md in the section **run**, for example (from a previous PR - 130 in the old repo):
Note, to run the `all.sh "test_zeroize"` tests you need to pass the additional parameters ` --cap-add=SYS_PTRACE --security-opt seccomp=unconfined` to allow ASLR to be disabled for the zeroize test. Without these parameters, this test will always fail.

Apparently this is necessary with Ubuntu 18.04.
Align 18.04 with 16.04: The 16.04 docker file has been cleaned up recently (192 in the old repo). Before that clean up, the differences between the 16.04 and 18.04 files were as follows:

the base version obviously
inconsequential formatting changes - formatting has been re-done anyway
differences in handling yotta dependencies - yotta and all its dependencies have been removed anyway
the 16.04 image supports installing arm-linux-gnueabi-gcc and (the effect of) gcc-multilib and the 18.04 image doesn't - it should
the 16.04 image support abi-compliance-checker and the 18.04 image doesn't - it should

So, the 18.04 doesn't have anything extra compared to 16.04 that we'd need to preserve. As a consequence, the following strategy can be applied to align the images:

Replace the 18.04 file with a copy of the 16.04 file and replace 16.04 with 18.04 up to an including the FROM ubuntu:16.04 line but not beyond. (One commit.)
Go over the occurrences of 16.04 and [Uu]buntu in the rest of the file - those are comments indicating that we're working around some limitation (generally, packages version of some software too old) of 16.04. For each occurences, check if it also applies to 18.04 and update the comment and possibly the corresponding code. One commit for each of the following groups of occurrences:
a. abi-compliance-checker and abi-dumper
b. pip version
c. libtasn1 for GnuTLS 3.6 (or 3.7 by the time with get there).

Note: making sure all.sh runs fully on the 18.04 image is out of scope for this issue.
Possible follow-ups: (no issues created for those as we might need to discuss the strategy first).

Start using 18.04 as the default for all.sh job in the Jenkns, and only keep a few selected jobs on 16.04 for compatibility. Chicken-and-egg problem: ideally, need to make sure all.sh passes on the Ubuntu 18.04 first, but since all.sh is so slow to run locally (8h+ on my laptop) it's better let the CI run it. Might need a companion PR to mbedtls if bugs are revealed (for example, new warnings from newer toolchains.)
Create ubuntu-20.04 file with the same procedure as for aligning the 18.04 image above: copy from 18.04, then look for occurrences of 18.04 or [Uu]buntu to adapt. Extend the dockerfile-builder to handle a 20.04 image as well. Ensure the image passes all.sh (see comments about using 18.04 on the CI above.)

For reference, here's how to test the new Dockerfiles on the CI: https://confluence.arm.com/display/BSGSoftware/Continuous+Integration+(CI)+Systems#ContinuousIntegration(CI)Systems-UpdatingDockerfilesusedinJenkins


Try heroku provisioning

Also offers a free layer more than enough for running the server.
https://devcenter.heroku.com/articles/php-support
PHP Built-in Web server

For testing purposes, you may start PHP’s built-in Web server by using php -S <IP_ADDRESS>:$PORT as the entry for “web” in your Procfile:

web: php -S <IP_ADDRESS>:$PORT

Some caveats:

[ ] Keep alives reset the internal IP (annoying new session messages)
[ ] Requires to commit composer.lock file
[ ] Requires to detect HTTPS #56


Transfer: update "available" after empty

Assume I have 10 IOU and 100 BTS in my account.
Steps:

click on "Send"
input "to", select "IOU" from "amount" box: the "available" amount is updated
click "available", click "send": asset in the amount box is updated to "BTS" and unable to change, "available" is updated to "0 IOU" but didn't change to "XXX BTS"

This issue was moved to bitshares/bitshares-ui#45


Use Object UIDs as Result/Record names.

This uses Object UIDs as the Record name in order to accomplish a few
different things:

Fixes an issue where k8s object names (<= 253 chars) didn't fit in to Result name
parts (<= 63 chars). UIDs
Sidesteps the original reason why we tried prepending the type in the
first place to distinguish TaskRuns/PipelineRuns with the same name.
(UIDs are unique for the cluster).
Allows us to detect and distinguish cases where users delete then
recreate objects with the same name. Using UIDs will cause us to
create new Results/Records for each new instance, instead of trying to
update the existing data.

The downside of this change means that Result/Record names are no longer
human readable, which means that users will need to rely more on
Record data to distinguish entries.
Fixes #48

/cc @ImJasonH @afrittoli in case you have any thoughts.

/cc @ImJasonH @afrittoli in case you have any thoughts.


Nesting selectors within other functions doesn't work

Thank you for the great article! I’ve got a quick question if you don’t mind.
It looks like the currying that enables the omission of the styled-components prop bag arg doesn’t work if you nest a getter within another function(s):
// Simple example unrelated to test output below

import styled from 'styled-components'
import { rgba } from 'polished' // Some utility helper function

const Example = styled.div`
  color: ${rgba(getColor('primary'))}
`

Note the resulting output in this snapshot test (because the test nests a getter within another helper function):

Am I right to think there isn’t really a way around this? I'm struggling to find a solution to this problem.
Any pointers or ideas would be greatly appreciated! :)

I'm stuck trying to figure out a way to change whether the createSelector helper conditionally curries the getter or not.


Ticket/1.6.x/9599 nexenta operatingsystem

Adds support for Nexenta to the operatingsystem and osfamily facts.
Since Nexenta is binary compatible with solaris, it has also been added
into the Solaris osfamily fact.
Tests and osfamily support added by Adrien Thebo<EMAIL_ADDRESS>

So we lose: hardwareisa, macaddress, uniqueid with this patch. Maybe we should backport mkincaid's confine patch to 1.6.x so we don't have to go through this nightmare again.

Alternately, target this at 1.7.0. I think it was reasonable to target the confine patch at 1.7.0, and I don't feel like constantly fighting this so perhaps we should just have all operatingsystem patches targeted at 1.7.0. More pressure to get that release out of the door eh? :)

Hmm. Up until now OS changes have been bugs on 1.6.x - is it wise to change the policy late in the day? Nexenta exists now - and has an existing bug - I think the fix is trivial enough to fix now to warrant a fix sooner rather then later I guess.
If you really don't think the confine patch should be back-ported then we need to do the confine changes to hardwareisa, macaddress, uniqueid in that case.
Its a little late ... lets discuss it later.

I ran though and looked at the confine patch, and it really does seem that it was just targeted at the wrong branch. I'm fine with backporting it.

Yep - this looks good now after we merged in #9708.

Merged manually:
https://github.com/puppetlabs/facter/commit/c3a8a9448a0eaa28519587a0de68773bb30b69a3


line height improperly calculated for scaling other than 100%

Environment
Microsoft Windows [Version 10.0.18362.295]
Windows Terminal (Preview)
Version: 0.4.2382.0

Steps to reproduce
Switch the "Change the size of text, apps, and other items" to something other than 100%
Open up Terminal and colorize multiple contiguous lines on the screen
(For me, I'm running WSL and run vim followed by :color blue)
Expected behavior
There should not be any gaps between the characters.
Here's the terminal on a 2560x1440 resolution monitor (Landscape) at 100% scale setting

Actual behavior
On 125% zoom, the same window now looks like this (tiny black horizontal marks between what should be contiguous lines of text all colored blue).

At 150% scale, we have a similar issue, though now instead of every line being affected, every other line has the horizontal artifact.

175%, and 225% zoom seem to have the same artifact line pattern as 125%
200% zoom has no artifact.
This looks similar to the artifact reported by @shanselman in https://github.com/microsoft/terminal/issues/455#issuecomment-518455051, but the issue I see is present independent of the fontFace setting (i see it both with fontFace set to Fira Code, and with fontFace setting removed thus falling back to the default fontFace).
Note: I initially came across this while using multiple monitors, and the artifact presented itself when dragging between two monitors. It turns out that the Scale setting is on a per-monitor basis, so if anyone working on a fix has multiple monitors, they can test several scale settings in parallel, as dragging the window between monitors will dynamically apply the scaling to the window.

This is actually still just /dup #455, even if the context is different. Thanks!


Not compatible with latest version of Neo4j driver

There has been a backwards incompatible change to the Neo4j drivers which changed the package structure. As a result, installing and then importing regraph fails.
>>> import regraph
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/benbc/src/opensafely-core/databuilder/.venv/lib/python3.9/site-packages/regraph/__init__.py", line 6, in <module>
    from regraph.backends.neo4j.graphs import Neo4jGraph
  File "/home/benbc/src/opensafely-core/databuilder/.venv/lib/python3.9/site-packages/regraph/backends/neo4j/__init__.py", line 3, in <module>
    from regraph.backends.neo4j.graphs import Neo4jGraph
  File "/home/benbc/src/opensafely-core/databuilder/.venv/lib/python3.9/site-packages/regraph/backends/neo4j/graphs.py", line 10, in <module>
    from neo4j.v1 import GraphDatabase
ModuleNotFoundError: No module named 'neo4j.v1'

The fix needed is to pin the neo4j-driver dependency in setup.py to < 4.0.0.
As a workaround you can provide this dependency yourself alongside the dependency on regraph so that the latest version of neo4j-driver doesn't get installed.
(This project doesn't look like it's maintained, so I'm mostly putting this here as documentation so that other people can find the workaround.)

I discovered that this only applies to the latest published version on PyPI. The latest code in this repo is working fine, so installing directly from here is an even better workaround. Since there haven't been any releases in over two years, I imagine that anyone who is still using this library is installing from here already; so I'll close this issue.
Just in case the library authors see this: it's a really helpful addition to NetworkX -- thanks!


Do not create loader if args.createLoader is false

createLoader is always true when calling compileFile with object as args. I wish it not to create a loader if args.createLoader is false.

I just pulled in the Typescript request. So this change will need to modify index.ts now.

OK, I rebased the commit

I was about to make the same change, plus adding a parameter to the cli (i.e. --no-loader) to turn off the loader creation.
If you can make it too it will be great, otherwise I will add the cli part later.

I made it, but I found that the program won't create a loader by default (If without -l).

I was about to make the same change, plus adding a parameter to the cli (i.e. --no-loader) to turn off the loader creation.
If you can make it too it will be great, otherwise I will add the cli part later.

I made it, but I found that the program won't create a loader by default (If without -l).

You are right. That flag is unnecessary.
I will merge your pull request now. I will do some typo fixes (in readme.md and other places), then I will publish it to npm.
Thank you for your contribution.


Fix crate READMEs for core, execution, physical-plan

Which issue does this PR close?

Closes #.
Rationale for this change

Apply https://github.com/apache/arrow-datafusion/pull/7964 for new READMEs in https://github.com/apache/arrow-datafusion/pull/7970
What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Thanks @Jefffrey


TypeScript type definitions

Hello,
I might be wrong, but I wasn't able to find any TS definition for @reactioncommerce/components.
Is this a problem on my side or are there no TS definitions to be found?
I really like the Reaction Commerce component library, but having to manually declare a definition for each component is really slowing down my development process.
In case no definition is available would you be interested in adding them? I could probably try to contribute.
Thanks.

@mikemurray @willopez Thoughts on this?

There aren't any yet but we would welcome any PRs to add some

I think there are ways to use the current proptypes in combination with type definitions. Perhaps you could find a tool that converts them?

I will try to find something and report back.


docker network gateway and dhcp aren't mentioned early enough

File: engine/userguide/networking/index.md, CC @mstanleyjones
The network notion discussions would seem to benefit from a mention of it's gateway and dhcp support as an organizing model for the discussion.

Bear with me, I'm not the networking expert 😅, but could you explain a bit more in detail what kind of changes you'd like to see (what's missing, or what wasn't clear from reading the docs)?
The docker networks have a default "IPAM" driver; alternative drivers are possible, but the default doesn't use DHCP (for the containers).
Happy to help, but it would be welcome if you could add a bit more information 👍

Simply put, the discussion appeared to lack a guiding principal. However, I am thinking that establishing such a thing may be premature.
In any event, a network may be considered something like a room that connected members inhabit and can talk to each other. Each is given a name/label by a tool call the DHCP. Also a designated member hangs by a window to allow occupants to talk to outsiders by relaying messages into the room or to specific members and vise versa.

Ok, I think I see what you're thinking of; more describe the "concept" of networks (why would I use a custom networks), as an introduction, more "story telling"
I'm not sure we should bring DHCP in here, though; docker networking works a bit different then a regular network. Usually, a device is connected to a network, and asks for an IP address; the DHCP server hands it that IP address. Docker networks have an IPAM (IP Address Management); this is pluggable, and could use a DHCP, but by default it's just enumerating IP-addresses from the available pool.
When creating a docker network, a pool of IP-addresses is reserved, and a network "namespace" created (we can call that the "room", but that may get more complicated later). When a container is started, an IP-address is issued, a network "endpoint"  is created (envision a "networking cable" being connected to the network) and given to the container. Inside the container, a new network-interface is created when the endpoint is attached.
If a container is connected to two networks, it gets handed two endpoints (two cables), and only containers being plugged in to the same network are able to chat with each other.

Yes, I understood the behavior upon reflection but the description as it existed was not as clear as it could be. Thanks for your attention.


Add a Note Warning of AAD Policy Which can Cause Import Failures

Hi team,
I've seen a couple of incidents recently where the AAD Policy that allows admins to prevent users from creating Azure DevOps Orgs attached to their AAD has invisibly caused import failures for users.
I'd suggest that we place a note here: https://docs.microsoft.com/en-us/azure/devops/migrate/migration-import?view=azure-devops#run-an-import
with a link out to this doc: https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/azure-ad-tenant-policy-restrict-org-creation?view=azure-devops
I.E.
NOTE:
Azure Administrators can forbid users from creating new Azure DevOps Organizations.  If the following Policy is set up your import will fail to complete:  https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/azure-ad-tenant-policy-restrict-org-creation?view=azure-devops.   Please verify that this is not set or that you have an exception for the user performing the migration before you begin.
So that there is at least something on the import side that points to this scenario, as more people become aware of and use this policy.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 4fa0ebd3-8437-8508-5a8d-462186fcce80
Version Independent ID: efe892fe-387e-0a61-fe7d-33d2ac99dfa9
Content: Import migrate process from on-premises to Azure DevOps - Azure DevOps
Content Source: docs/migrate/migration-import.md
Product: devops
Technology: devops-migrate
GitHub Login: @KathrynEE
Microsoft Alias: kaelli

@KathrynEE -- please review private PR 1331.


Removed Pipeline API reference

Removed Pipeline API reference since it's no longer used in the tutorials.

Thanks, @rpetrusha!!!!


Performance with file uploads

a simple 12Mb file takes 1.6 sec  to upload with reproxy
With reproxy:
time curl -X POST "http://localhost/api/upload" -H  "accept: application/json" -H  "Content-Type: multipart/form-data" -F<EMAIL_ADDRESS>
0.01s user 0.01s system 1% cpu 1.614 total

without :
time curl -X POST "http://localhost:8000/api/upload" -H  "accept: application/json" -H  "Content-Type: multipart/form-data" -F<EMAIL_ADDRESS>
0.00s user 0.01s system 29% cpu 0.062 total

conf:
  reproxy:
    image: umputun/reproxy:master
    container_name: reproxy
    hostname: reproxy
    ports:
      - "80:8080"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./files/static:/static
    environment:
      - LISTEN=<IP_ADDRESS>:8080
      - DOCKER_ENABLED=true
      - DEBUG=true
      - MAX_SIZE=20000000

I'm not sure how reproxy can add such a significant delay, there is no path here involving anything that heavy. Have you tried to repeat the test to see if the results consistent? Another good idea is to run reproxy without docker to eliminate this part because on some os running docker in the internal VM, any file access to a volume can be slow

volumes seems not the case, here is my setup without any volume:
services:
    reproxy:
      image: umputun/reproxy:master
      container_name: reproxy
      hostname: reproxy
      ports:
        - 80:8080
      volumes:
        - /var/run/docker.sock:/var/run/docker.sock:ro
      environment:
        - LISTEN=<IP_ADDRESS>:8080
        - DOCKER_ENABLED=true
        - DEBUG=true
        - LOGGER_STDOUT=true
        - MAX_SIZE=20000000

    whoami:
      image: 'containous/whoami'
      container_name: whoami
      labels:
        reproxy.server: 'localhost'

test:
time curl -X POST -D - "http://localhost/whoami/" -H  "accept: application/json" -H  "Content-Type: multipart/form-data" -F<EMAIL_ADDRESS>HTTP/1.1 100 Continue

HTTP/1.1 200 OK
Content-Type: text/plain; charset=utf-8
Date: Thu, 13 May 2021 06:51:09 GMT
Transfer-Encoding: chunked

Warning: Binary output can mess up your terminal. Use "--output -" to tell
Warning: curl to output it to your terminal anyway, or consider "--output
Warning: <FILE>" to save to a file.

0.01s user 0.03s system 2% cpu 1.566 total

note it gave 2 http responses actually:
HTTP/1.1 100 Continue
<pause>
HTTP/1.1 200 OK

well, it gives exactly the same without reproxy
curl -X POST -D - "http://<IP_ADDRESS>:8888" -H  "accept: application/json" -H  "Content-Type: multipart/form-data" -F "file=@12MB"
HTTP/1.1 100 Continue

HTTP/1.1 200 OK
Date: Thu, 13 May 2021 21:07:25 GMT
Content-Type: application/octet-stream
Connection: close
Transfer-Encoding: chunked

Warning: Binary output can mess up your terminal. Use "--output -" to tell
Warning: curl to output it to your terminal anyway, or consider "--output
Warning: <FILE>" to save to a file.

and this container also complains
dk run -it --rm -p 8888:80 containous/whoami
Starting up on port 80
2021/05/13 21:07:25 http: superfluous response.WriteHeader call from main.whoamiHandler (app.go:179)
2021/05/13 21:08:27 http: superfluous response.WriteHeader call from main.whoamiHandler (app.go:179)

@umputun
what do you suggest ? containous/whoami is not good ?
ok.. let's take another one:
services:
    reproxy:
      image: umputun/reproxy:master
      container_name: reproxy
      hostname: reproxy
      ports:
        - 80:8080
      volumes:
        - /var/run/docker.sock:/var/run/docker.sock:ro
      environment:
        - LISTEN=<IP_ADDRESS>:8080
        - DOCKER_ENABLED=true
        - DEBUG=true
        - LOGGER_STDOUT=true
        - MAX_SIZE=20000000

    svc2:
      image: ghcr.io/umputun/echo-http
      hostname: svc2
      container_name: svc2
      command: --message="hello world from svc2"
      ports:
        - 9999:8080
      labels:
        reproxy.server: 'localhost'

with reproxy = 1.6seconds
time curl -X POST -D - "http://localhost/svc2/" -H  "accept: application/json" -H  "Content-Type: multipart/form-data" -F<EMAIL_ADDRESS>HTTP/1.1 100 Continue

HTTP/1.1 200 OK
Content-Length: 432
Content-Type: application/json; charset=utf-8
Date: Fri, 14 May 2021 08:02:52 GMT

{"message":"hello world from svc2","request":"POST /","host":"<IP_ADDRESS>:8080","headers":{"Accept":"application/json","Accept-Encoding":"gzip","Content-Length":"12511436","Content-Type":"multipart/form-data; boundary=------------------------8908f9526570e656","Expect":"100-continue","User-Agent":"curl/7.64.1","X-Forwarded-For":"<IP_ADDRESS>","X-Forwarded-Host":"localhost","X-Real-Ip":"<IP_ADDRESS>"},"remote_addr":"<IP_ADDRESS>:56914"}
curl -X POST -D - "http://localhost/svc2/" -H "accept: application/json" -H

0.01s user 0.03s system 2% cpu 1.645 total

without 0.017 seconds
time curl -X POST -D - "http://localhost:9999/svc2/" -H  "accept: application/json" -H  "Content-Type: multipart/form-data" -F<EMAIL_ADDRESS>HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Fri, 14 May 2021 08:04:19 GMT
Content-Length: 324
Connection: close

{"message":"hello world from svc2","request":"POST /svc2/","host":"localhost:9999","headers":{"Accept":"application/json","Content-Length":"12511436","Content-Type":"multipart/form-data; boundary=------------------------8976497ade0607f1","Expect":"100-continue","User-Agent":"curl/7.64.1"},"remote_addr":"<IP_ADDRESS>:60852"}
curl -X POST -D - "http://localhost:9999/svc2/" -H "accept: application/json"

0.00s user 0.01s system 67% cpu 0.017 total

what do you suggest ? containous/whoami is not good ?

No clue. All I'm saying dbl headers you mentioned above had nothing to do with reproxy

I think this is related to the max size request check. Try to set MAX_SIZE=0 and see if it fix the issue

yep, MAX_SIZE=0 helps
( but would be nice to have MAX_SIZE option available :) )

and even better - to be able to set like MAX_SIZE=100MB

size limiter (a part of go-pkgz/rest) seems to have some performance issue due to the way it checks the content size. I'll take a look and see what can be done.

this is the ticket if you want to follow the progress https://github.com/go-pkgz/rest/issues/16

@vitalik have you ever tried reproxy with any real upstream that actually supports file upload? The test you provided with echo-http is not that real, as nothing here actually reads the body. With the limiter in place, this read enforced and this could be a valid reason explaining the time difference you see.

Ну да... :)
I replaced Nginx with reproxy on a pet project and got wild latency on file uploads endpoints

latency actually expected here as the read happens right away in the limiter. the question is - what about the total time (throughput)? Did you see that significant difference like in the tests above?

I think it should be better now. pls check the master image as you have a chance

I'm going to close this one as the problem seems to be solved, at least according to my tests. Size units added as well.
if it still making trouble in your tests feel free to reopen.

well... now  each requests is performing fast enough
BUT instead - now I'm getting random connection drops with curl tests:

and it is NOT happening if I set MAX_SIZE=0
(docker-compose.yml form this comment: https://github.com/umputun/reproxy/issues/71#issuecomment-840355956 )

as noted earlier - this testing doesn't provide much information and much confidence, because the destination you used is not even capable of uploading files.  This is a more realistic one (see attached)
time curl -X POST -D - "http://localhost/upload" -H  "accept: application/json" -H  "Content-Type: multipart/form-data" -F "file=@12MB"
HTTP/1.1 100 Continue

HTTP/1.1 200 OK
Content-Length: 22
Content-Type: text/plain; charset=utf-8
Date: Mon, 17 May 2021 17:01:45 GMT

successfully uploaded
curl -X POST -D - "http://localhost/upload" -H "accept: application/json" -H   0.01s user 0.01s system 5% cpu 0.278 total

I did this test a dozen times and it worked just fine.
to run it - unpack in any place, run docker-compose build && docker-compose up
Archive.zip


fix potential fd leakage for SegmentProcessorFramework

Try to fix https://github.com/apache/pinot/issues/9415: potential fd leakage in SegmentProcessorFramework. The fileManager has to call cleanup to release the mmap'ed intermediate files.

Codecov Report

Merging #9797 (291b5c1) into master (dc49b2c) will decrease coverage by 38.81%.
The diff coverage is 100.00%.

@@              Coverage Diff              @@
##             master    #9797       +/-   ##
=============================================
- Coverage     64.08%   25.27%   -38.82%
+ Complexity     4970       44     -4926
=============================================
  Files          1903     1944       +41
  Lines        102553   104622     +2069
  Branches      15605    15854      +249
=============================================
- Hits          65722    26441    -39281
- Misses        32062    75472    +43410
+ Partials       4769     2709     -2060

Flag
Coverage Δ

integration1
25.27% <100.00%> (?)

unittests1
?

unittests2
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...rocessing/framework/SegmentProcessorFramework.java
90.78% <100.00%> (-7.90%)
:arrow_down:

...in/java/org/apache/pinot/spi/utils/BytesUtils.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...java/org/apache/pinot/spi/trace/BaseRecording.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...java/org/apache/pinot/spi/trace/NoOpRecording.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...ava/org/apache/pinot/spi/config/table/FSTType.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...ava/org/apache/pinot/spi/config/user/RoleType.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...ava/org/apache/pinot/spi/data/MetricFieldSpec.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...ava/org/apache/pinot/spi/stream/StreamMessage.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...java/org/apache/pinot/common/tier/TierFactory.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...a/org/apache/pinot/spi/config/table/TableType.java
0.00% <0.00%> (-100.00%)
:arrow_down:

... and 1571 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


[Crates] Added crate size badge

Proposed in https://github.com/badges/shields/issues/6011
Didn't make the issue but last message was 2021 so I figured it was safe and I needed this.
Routes:
Crates.io Size: /crates/size/:crate
Crates.io Size (version): /crates/size/:crate/:version

Also, while doing this I realised that shields don't use **bibytes but divide by 1000 instead. I just followed what the rest of the badges do but are there any plans of adding support for this? Since there's now an inconsistency between the badge's number and the one on crates.io/docker/github repo size

Also, while doing this I realised that shields don't use **bibytes but divide by 1000 instead. I just followed what the rest of the badges do but are there any plans of adding support for this? Since there's now an inconsistency between the badge's number and the one on crates.io/docker/github repo size

Will point back to https://github.com/badges/shields/issues/9703#issuecomment-1789630736 as I believe our position is still the same.
Our main goal is consistency within the Shields ecosystem, but we can certainly consider offering support for these types of variances when it makes sense

Also, while doing this I realised that shields don't use **bibytes but divide by 1000 instead. I just followed what the rest of the badges do but are there any plans of adding support for this? Since there's now an inconsistency between the badge's number and the one on crates.io/docker/github repo size

Will point back to https://github.com/badges/shields/issues/9703#issuecomment-1789630736 as I believe our position is still the same.

Thanks for digging that comment out. I knew it had come up before. Lets keep the scope of this PR to just merging this in its current form. I am going to write up another issue about this topic. I do think it is worth doing though.

I've raised https://github.com/badges/shields/issues/10437 and laid out my thoughts on this.


React-Dates Broken Style

react-dates version
react-dates": "^21.8.0
Bug Description
I use React-Dates with Tailwind CSS Component Library in NextJs which is deployed on Vercel.
When I tested things. The Date-Pick looks good locally but not in production, The CSS file in production also loads but the date picker is broken for some reason.
Source code (including props configuration)
<SingleDatePicker
  date={purchase_date}
  required
  block
  regular
  numberOfMonths={1}
  placeholder="Purchase Date"
  isOutsideRange={() => false}
  onDateChange={(date) => setpurchaseDate(date)}
  focused={purchase_dateFocus}
  onFocusChange={() => setpurchaseDateFocus(!purchase_dateFocus)}
  id="purchase_date"
/>

Screenshots/Gifs

Desktop (please complete the following information):

OS: Ubuntu
Browser: Chrome
Version: 85.0.4183.83 (Official Build) (64-bit)

Additional context
I don't know if there is a conflict between tailwind and react-dates CSS.
Any help would be much appreciated! Thanks

I was using Tailwind CSS with NextJS and I had to include my react-date CSS file in the build at postcss.config.js file to get it compiled and it worked!
module.exports = {
  plugins: {
    tailwindcss: {},
    autoprefixer: {},
    ...(process.env.NODE_ENV === 'production'
      ? {
          '@fullhuman/postcss-purgecss': {
            content: ['./components/**/*.js', './pages/**/*.js', './public/css/*.css'],
            defaultExtractor: (content) => content.match(/[\w-/:]+(?<!:)/g) || [],
          },
        }
      : {}),
  },
}

Issue Closed


Deploying on virtual machines?

It's possible that there might be issues running certain software or programs on Virtual Machines, depending on the specific software, hardware configuration, and virtualization platform. In your case, it seems like you're encountering an issue with the "illegal instruction" error when trying to use a specific repository. To troubleshoot this, you can try the following step

Check the compatibility of the software with Virtual Machines: Make sure the software you're trying to run is compatible with Virtual Machines. Some software might not work properly due to limitations in the virtual environment.

Update your Virtual Machine's hardware configuration: Ensure that your Virtual Machine has the necessary hardware resources to run the software smoothly. This includes sufficient CPU, RAM, and storage space.

Verify the virtualization platform: Check if your host machine is using the correct virtualization platform for your Virtual Machine. Some platforms might not support certain software or have known issues.

Update the Virtual Machine's guest operating system: Make sure your guest operating system is up-to-date with the latest patches and security updates. This can help resolve potential compatibility issues.

Reinstall the software: If the issue persists, try reinstalling the software on your Virtual Machine. This can sometimes resolve any conflicts or errors that might have occurred during the initial installation.

Seek support from the software developer: If the issue remains unresolved, contact the software developer for assistance. They might be able to provide guidance on how to resolve the "illegal instruction" error or if there are any known issues with their software in Virtual Machines.

Consider using a different Virtualization platform: If you're still unable to resolve the issue, you might want to try using a different virtualization platform to see if it improves compatibility with the software.

Remember to always follow the instructions provided by the software developer and be cautious when making changes to your Virtual Machine's configuration.
If you need help, please @NeuralChat

@NeuralChatBot

To troubleshoot the "illegal instruction" error when using a specific repository in a Virtual Machine, you can try the following steps:

Check the compatibility of the software with Virtual Machines: Make sure the software you're trying to run is compatible with Virtual Machines. Some software might not work properly due to limitations in the virtual environment.

Update your Virtual Machine's hardware configuration: Ensure that your Virtual Machine has the necessary hardware resources to run the software smoothly. This includes sufficient CPU, RAM, and storage space.

Verify the virtualization platform: Check if your host machine is using the correct virtualization platform for your Virtual Machine. Some platforms might not support certain software or have known issues.

Update the Virtual Machine's guest operating system: Make sure your guest operating system is up-to-date with the latest patches and security updates. This can help resolve potential compatibility issues.

Reinstall the software: If the issue persists, try reinstalling the software on your Virtual Machine. This can sometimes resolve any conflicts or errors that might have occurred during the initial installation.

Seek support from the software developer: If the issue remains unresolved, contact the software developer for assistance. They might be able to provide guidance on how to resolve the "illegal instruction" error or if there are any known issues with their software in Virtual Machines.

Consider using a different Virtualization platform: If you're still unable to resolve the issue, you might want to try using a different virtualization platform to see if it improves compatibility with the software.

Remember to always follow the instructions provided by the software developer and be cautious when making changes to your Virtual Machine's configuration. If you need help, please @NeuralChat for assistance.


Flow Diagram step0

Is your feature request related to a problem? Please describe.
According to meeting 15.12.2023
add a flow diagramm of step0 process
Describe the solution you'd like
add a screenshot of your flow, so I can adapt https://github.com/swisstopo/topo-satromo/blob/main/satromo_processor.drawio

flow_step0.pdf

Let me know if you want to PPTX version, or any other change

@gilliM
I updated the data flow in satromo_processor.drawio  as I understand it.  it is work in progress regarding the layout.
I
Regarding clouds and shadow I have questions
: "clouds" are treated in  def maskCloudsAndShadow
https://github.com/swisstopo/topo-satromo/blob/33711a8269dd15a28174594c6a1a2fda2437d336/step0_processors/step0_processor_s2_sr.py#L170
Questions

[ ] : cloud threshold: we ommit all sceenes which have more than 50% of cloud coverage: https://github.com/swisstopo/topo-satromo/blob/33711a8269dd15a28174594c6a1a2fda2437d336/step0_processors/step0_processor_s2_sr.py#L177, is this correct? 50% seems quite low to me, @HendrikWulf @gilliM : what would make  sense from your perspective? 90% ?

[ ] IF condition: if the image is cloudy : I  do not understand the code: where  in def maskCloudsAndShadows
 do you exit  the  processor def generate___mosaic_for_single_date if there is too much cloud&shadow above the threshold? or ist it done in https://github.com/swisstopo/topo-satromo/blob/1bda99f45d7d5e01823e25d3d4169cef5387c7c7/step0_processors/step0_processor_s2_sr.py#L366 ? where does the threshold kick in?

[ ] EXPORT cloudy:  Does it make sense to export an asset if it is "cloudy" as ashown in your docu? the image is then not used or do I understand the code wrong? As I understand the code  is not exporting cloudy assets. which makes sense

from the code: https://github.com/swisstopo/topo-satromo/blob/cc77fe58e28fdadf2c47101f65a3368d7f9f0a47/step0_processors/step0_processor_s2_sr.py#L367 we ignore images with > 98% clouds. I would go lower to at max 80% personally·

it's here: https://github.com/swisstopo/topo-satromo/blob/cc77fe58e28fdadf2c47101f65a3368d7f9f0a47/step0_processors/step0_processor_s2_sr.py#L370 + the return just the line after the end the step0 script. It's after the mosaic because the threshold (98% here) is applied on the whole mosaic and not per scene: we decide together for the whole swath if we want to process it or not.

the code does not export cloudy asset. The idea behind is to save some money by not processing and storing useless cloud pixels.

thanks, did not understand  ",2"  now i do fixed with 8a5e0c82a190915868e502030f872f5ee3d911a7

thanks clear now for me
celar to me, the image was irritating me


Adding prettier command and running it for the first time

Description
Adding prettier command and running it for the first time.
Type of change

This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional).

Fixes: #issue_number

Lets have a discussion before adding new required tools


New EU4 governments

As mentioned in the Development Diary, EU4 is gonna lose its many governments and having them merged alltogether, as the only difference will be in "reforms" and "absolutism". The old system should still be supported (in case of old-patched saves) and used any time the file is found to be pre-1.26
Such system would allow a better conversion to Vic2 governments and political reforms, if applied.

EU Monarchy, EU Tribal --> (Absolute Monarchy)/(Prussian Const.)/(HM's Gov.)
EU Republic --> (Democracy)/(Pres. Dict.)/(Bourg. Dict.)
EU Theocracies --> Theocracy

The monarchical reforms have been revealed:

Feudalism vs Autocracy: this should lead to upper house policy and type of monarchy
Hereditary Nobility: this should lead, maybe, to trade unions policy
Bureaucracy: this should lead to political parties policy
Growth of Administration: this should lead to upper house composition policy
Deliberative Assembly: this should lead to voting franchise policy
Absolute Rule vs Constitutional: this should lead to the type of monarchy
Separation of Powers: this should lead to press rights and public meetings policies

Seeing Republic reforms:

Oligarchy vs Merchantry vs Nobility: trade union policy
Republican Virtues: public meetings policies
Elections frequency: political parties policies
Federalism vs others: type of republican regime
Parliamentary vs Presidential: upper house composition
Consolidation of Power: (none)
Guiding Principles: press rights
Electorate: voting system
Office selections: voting franchise
Question of Dictatorship: type of republican regime (again)

Seeing Theocratic reforms, which convert to the government of theocracy:

Leadership: voting franchise (although no voting actually)
Internal vs External Mission: trade union policy
Divine Cause: press rights
Clergy in Administration: upper house composition
Secularize: voting system

Seeing Tribal reforms, which convert to the government of absolute_monarchy, yet with voting franchise of "Only Landed" (to simulate the clans):

Tribal Administration: upper house composition
Cultural Values: press rights
Religion vs Secular: trade union policy
Modernization: voting system

The governmental reforms are free to be seen in the Dharma Manual (page 24~30): https://forumcontent.paradoxplaza.com/public/387428/PDX_EUIV_Dharma_Manual_ENG.pdf

As #517 pulls, a new file is added: DharmagovernmentMapping.txt.
It is supposed to check out what reforms a country has and then calculating the probability to choose a government or the other. Each reform has a government mapped. If the country has that reform, a +1 is added to the probability of the government.
In a hypothetical case, having 5 points for Absolute Monarchy, 4 for HM's, the country will become an Absolute Monarchy. If no points could be found, the converter should default to Absolute Monarchy.

Resurrecting this issue, it seems that in Dharma saves a country history will feature this, regardless of using the DLC or not:
	NOV={
		government_rank=1
		government_name="russian_republic"
		subject_focus=0
		...

The government_name field could be used to determine what to map in saves that show to be of at least 1.26 version, as stated inside them at line 6
savegame_version={
	first≥1
	second≥26
...

Such new mappings are as in the pastebin following: https://pastebin.com/dFB3YgG2


Battery calibrate does not work!

I liked to calibrate my MBP '23's battery using the battery calibrate command. It didn't work.
So, I used the command in the terminal (shown by the battery command). The electric cord was attached, and its light remained dimmed, as expected. As I was working with MBP, I noticed that the charge dropped to 15%, but stayed there for an unexpected long time. Charging was not started. I disconnected and connected the cord, and charging began, according to the orange light in the connector. However, for some 30 min, the percentage of charge did not rise.
In this phase, I quitted the battery app, disconnected/connected the cord, and charging began very soon. Now it is 85%.
In conclusion, I was not able to use Battery for calibrating my MBP's battery. I hope that the MBP's battery is calibrated by just charging the device normally up to 100%.  In all, I like the app for its basic use, although after the latest update, it is frustrating that I need to insert my password after every booting.

Don't use their calibrate function because there are severe bugs and it's detrimental to your battery. Apparently it was released without testing first. Just use their maintain function.

+1 exact same issue, drains down to 15% doesn't charge up


UN WPP step has incorrect folder structure

Problem
There are a bunch of files in etl/steps/data/garden/un/2022-07-11/ which are used in the un_wpp step but which are not counted as its dependencies.
Background
The ETL keeps track of every file used in every step, so that it knows when it needs to rebuild the data.
There are two types of ways you can make steps and keep track of their dependencies:

<channel>/<namespace>/<version>/<dataset>.py (all files matching <dataset>.* are dependencies)
<channel>/<namespace>/<version>/<dataset>/__init__.py (all files in <dataset>/ are dependencies)

If you have many files, you must use the folder version of a step.
Impact
You can make changes to one of those files, e.g. demographics.py, and the production data will not change, because it isn't aware that any of the ingredients of the un_wpp dataset have changed.
Solution

Move un_wpp.py -> un_wpp/__init__.py
Move un_wpp.* files -> un_wpp/
Move *.py in that folder -> un_wpp/
Fix paths in the step so that it still builds cleanly

(Just a side note that <channel>/<namespace>/<version>/shared* also count as dependencies, but are not encouraged as a best practice)

solved in https://github.com/owid/etl/pull/521


"native" app support

Running as a native app would be nice, like openlens or aptakube.
Since its pretty easy i have created a DEMO for you using wails (wails.io) in the tree: https://github.com/mschneider82/kubewall/tree/wails
If you want to try this, maybe you adopt it into main.
Just install wails and run wails dev   or with linux wails dev -tags webkit2_40  in the wails tree in the /backend/ui folder.
the yarn build should be executed before that ( yarn --cwd client install && yarn --cwd client run build && rm -rf backend/routes/static && mv client/dist backend/routes/static  )

In this tree i just forwarded the wails app to the localhost:7080 this is hacky but works.

Hello @mschneider82 , many thanks for using the app and trying out wails.
We were also considering developing a native app using Wails. There are a few things we need to think about regarding running the UI in-cluster, using SSH, Docker and native apps.
Maybe we can provide a separate build for a native app. I will try out a few things in an upcoming release, I will take your updates as a reference.
Once again thanks so much 😀  🙏


onPurchasedProductsFetched is always called twice

onPurchasedProductsFetched is called twice on connect().
One time with purchases empty and once with purchases as expected (in case the user purchased).
The worst thing is that there's a race condition so you can't even write a logic based on skipping the first time as sometimes this is the correct data, followed by the false data.
It doesn't help to connect before setting the listener or after.

onPurchasedProductsFetched is called twice on connect(). One time with purchases empty and once with purchases as expected (in case the user purchased). The worst thing is that there's a race condition so you can't even write a logic based on skipping the first time as sometimes this is the correct data, followed by the false data.
It doesn't help to connect before setting the listener or after.

I had the same issue. If your device supports subscriptions, then the code sends first callback to onPurchasedProductsFetched(..) and second one containing the actual purchase history you are interested in. This is how the library is designed currently.
In my case, I am only interested in INAPP purchases so I skipped all redundant callbacks by checking productType == ProductType.INAPP.

I've never had any problems restoring purchases with my app, but I'm only using INAPP
What's the status of the issue? Checking the productType is a workaround or still having issues?

It helps for sure. I still have some reports from users saying they "lose" their Pro purchase experience but I didn't check yet if it's a lib issue or my own implementation issue. I can say that it started happening since I switched to this lib and only when the user opens the app after it was in the background for a while (Don't keep activities doesn't reproduce the same conditions unfortunately).

The onPurchasedProductsFetched is MEANT to be called twice, once for each product type (IN-APP and SUBS).
Try to follow the logic from the RemoveAdsExampleActivity...
Also, it's good to know that Google Billing API returns purchases details for currently owned items without a network request using the Google Play Store cache...


[BUG] Confusing settings in Readme.

In the Readme file:
Command hooks exist in the format: {hook_name}
{presave}: executes _before a session is saved
{saveextra}: execute _after a session is saved, return string will save to *x.vim, reference :help mks
{postsave}: executes _after a session is saved
{prerestore}: executs _before a session is restored
{postrestore}: executs _after a session is restored
{predelete}: executs _before a session is deleted
{postdelete}: executs _after a session is deleted

For example, {presave} should be literally be {pre_save}. This is so confusing. I tried to set like this:
presave_cmds={"NERDTreeClose"}
this actually won't work and give zero error message... and it should be:
pre_save_cmds={"NERDTreeClose"}
================ Please Correct Those {hook_name}s by Adding a "_" For Each of Them ===========================
Finally I figured out by:

open a file;
opening NERDTree;
run :SessionSave;
then I found the NERDTree is still open. It has be known that the NERDTree won't be saved by auto-session. If you keep it on before :SessionSave, you will get errors when you try to :SessionRestore since auto-session can't restore the NERDTree. ( see details here: https://github.com/rmagatti/auto-session/pull/179) So, it's kind important to automatically close NERDTree before :SessionSave to avoid those annoying error messages.

I don't know why, but the tiagofumo/vim-verdtree-syntax-highlight seems don't get along well with auto-session. It will cause errors too at :SessionRestore, I simply keep it away while using auto-session.
Just leave this message in case someone else need.


Unable to get the sub folder specific imports on main file

Sub folders modules are not added to file. We need to go to each sub folder and run the command, We might need to do some recursive use case where if the user is holding multiple folder as sub projects the a main requirement.txt can be produced via main folder

If I'm understanding this issue correctly, I can't reproduce it. Here's what I tried:
$ tree
.
├── app1
│   └── app.py
├── app2
│   └── app.py
└── app3
    └── app.py

$ cat app1/app.py
import boto3
import flask

$ cat app2/app.py
import psycopg2
import sqlalchemy

$ cat app3/app.py
import requests

$ pipreqs . --print
requests==2.18.4
boto3==1.4.7
Flask==0.12.2
psycopg2==<IP_ADDRESS>
SQLAlchemy==1.2.0b3
INFO: Successfully output requirements

Is this the kind of setup you're trying to accomplish?

Absolutely but I am on Ubuntu 16.04 Lts, python 2.7 and have the same
structure of folders but I was not able to get the details.
On 13 Nov 2017 8:55 pm, "Jon Banafato"<EMAIL_ADDRESS>wrote:

If I'm understanding this issue correctly, I can't reproduce it. Here's
what I tried:
$ tree
.
├── app1
│   └── app.py
├── app2
│   └── app.py
└── app3
└── app.py
$ cat app1/app.py
import boto3
import flask
$ cat app2/app.py
import psycopg2
import sqlalchemy
$ cat app3/app.py
import requests
$ pipreqs . --print
requests==2.18.4
boto3==1.4.7
Flask==0.12.2
psycopg2==<IP_ADDRESS>
SQLAlchemy==1.2.0b3
INFO: Successfully output requirements
Is this the kind of setup you're trying to accomplish?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/bndr/pipreqs/issues/106#issuecomment-343953262, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AESGI-GWW1iJF87lwhAd8DqdN4lIaqVAks5s2F9pgaJpZM4QbXU8
.

Which version of pipreqs are you using? Can you provide a minimal example that fails?

0.4.9
#Cheers,
[image: photo]
Shivdhwaj Pandey
Coder, Developer, Programmer, Consultant
+917387767716 |<EMAIL_ADDRESS>http://in.linkedin.com/in/shivdhwajpandey http://twitter.com/shivdhwaj
http://www.facebook.com/shivdhwaj http://github.com/shivdhwaj
http://opentechlabs.blogspot.com/
On 14 November 2017 at 09:17, Jon Banafato<EMAIL_ADDRESS>wrote:

Which version of pipreqs are you using? Can you provide a minimal example
that fails?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/bndr/pipreqs/issues/106#issuecomment-344138080, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AESGI-_fWMgVCK4ud5gqcTfco0B59080ks5s2Q1kgaJpZM4QbXU8
.

Hm, I can't reproduce your issue with 0.4.9 or master. If you can provide an example of the issue, I'm happy to help debug.

I'm seeing the same issue on 0.4.13


util: add PROXY protocol generation functions

Description: Add utility functions for generating v1 and v2 PROXY protocol headers. This is the first step to adding Envoy PROXY proto generation support. These functions can also be used in other places that would also like to generate the header.
Risk Level: Low
Testing: Unit
Docs Changes: None
Release Notes: None
Related to https://github.com/envoyproxy/envoy/issues/1031
@alyssawilk

/retest

I saw the "envoy-linux (bazel clang_tidy)" build fail because it failed to fetch a repo. Thought I could re-run that with the retest thing but I guess not?

Thanks for the feedback! I'll move the regression tests over to this branch 🙂.

LGTM pending CI cleanup :-)

@mattklein123 or @zuercher @snowp any of you up for second pass?


refactor: Satisfy linter in zonesToRegion

Change Overview
There was annotation which disables gocritic. This linter could be enabled now, since we do have test which will detect panic.
Also it fixes order of regions returned by zonesToRegions
Pull request type
Please check the type of change your PR introduces:

[ ] :construction: Work in Progress
[x] :rainbow: Refactoring (no functional changes, no api changes)
[x] :hamster: Trivial/Minor
[ ] :bug: Bugfix
[ ] :sunflower: Feature
[ ] :world_map: Documentation
[ ] :robot: Test

Issues

fixes #issue-number

Test Plan

[ ] :muscle: Manual
[x] :zap: Unit test
[ ] :green_heart: E2E

I've accidentally pushed original PR under wrong branch name. After renaming branch via GH, it has automatically closed existing PR, so I had to create new one.


chronos may redirect to itself

Sometimes, a slave server may try to be the leader. It wrote the log as this:

[INFO] [JobScheduler STARTING] 03-21 12:25:24,958 [JobScheduler] - Trying to become leader.

In fact, it wasn't selected as leader, because it didn't write any log as this:

[JobScheduler] - Elected as leader.

But after that, when I do some http request to this node, it start to redirect to itself. I can't receive any answer. And this slave server will print a lot of log, "[RedirectFilter] - Proxying request to ", to itself.
Then the slave server will block. After printed more than 200 redirect logs, it stop printing any log.
I restarted the chronos service on that slave server, then it became normal.

I hopes to know why this will happen. How to prevent it. It has been occured twice this month. QAQ

The version of Mesos is 0.28.0
The version of Chronos is 2.4.0


Workspace trust prevents opening codespace remote

Testing #122252

Try to open a codespace instance in VS Code

An error message prompts and also choosing "No I don't trust" doesn't work at all.

Trusting the workspace doesn't work either, I have to reload the window for it to work.

closing as you must use extensions.supportUntrustedWorkspaces until remotes onboard properly

As @sbatten said, in order to unblock this scenario you need to add the following to your user settings:
  "extensions.supportUntrustedWorkspaces": {
    "GitHub.codespaces": {
      "supported": true
    }
  }


Add hint for rest parameter subdirectory necessity

Highlight the necessity to use a rest parameter when using subdirectories to organize collections.
What kind of changes does this PR include?

New or updated content

Description
I updated the docs to include a hint to highlight the necessity of a rest parameter when using subdirectories to organize collections.
This wasn't really obvious to me since the error that was thrown (Expected "slug" to match "[^\/#\?]+?", but got "foo/bar") when starting dev and trying to build only gave some kind of hint to what went wrong.
The blog starter template helped me clarify. In hindsight it was clear that this was necessary but I thought it would be a good addition to the docs since others might run into the same issue.
Thanks btw. for the amazingly wonderful great work you guys are doing 🥳

Thanks @chimp1nski! I wonder if it actually would make sense to use the rest parameter for the main example instead of in a separate call-out. It's probably a common scenario and I don't think there are major drawbacks.
Paging @bholmesdev @sarah11918 to see what they think.

Completely agree with everyone here! A wonderful addition, @chimp1nski . Thank you and welcome to Team Docs! 🥳


Translation Overhaul #2

This concerns these other translations that were not upto-date: it-it, nl-nl, pt-br, pt-pt, ro-ro, th-th
Note: No actual Translations were made.
These translations are now on par with the en-us one. Old strings weren't overwritten.
Its now a shitton easier for the translators of their respective country to translate the strings now since all the mess of old unorganized strings is properly aligned to the english translation for better future translations.

Coverage remained the same at 24.341% when pulling 7df1341f411085e0f701d4f240b73fb4e478965f on Lamppostmanufacturer:patch-5 into 2cca143121c19d5b1fe45c690b973b7465ef7aa9 on NyaaPantsu:dev.

Coverage remained the same at 24.341% when pulling 17ae92a751bb1c605d4452cd9b064c8535978380 on Lamppostmanufacturer:patch-5 into 81a8f1c65cbe3405a638b28ec5227c9e4919294a on NyaaPantsu:dev.

Its just a feature. Some of those translations were missing 1000+ lines and some changes are in the middle of other strings. Totally messed ordererd etc. This is to ensure all translations are on par with each other and since empty ID's will revert to the original file anyway it makes no difference other than easier management.

Coverage remained the same at 24.341% when pulling 0cb9c1bb0f486b4f30b62d266fe77c9081bb0971 on Lamppostmanufacturer:patch-5 into e1cf80169c1c06dfc0b78bb0f76f920c59c38b3b on NyaaPantsu:dev.

Coverage remained the same at 24.341% when pulling 225080004bd9089bc853f323263b7ab7b9940ef0 on Lamppostmanufacturer:patch-5 into 00e092372f8339bc8f7fd135fb4fc0784403ce71 on NyaaPantsu:dev.


move all irreplaceable content to openshift-config

The installer provides a set of certificates that we cannot replace, cannot remove, and cannot rotate.  These are the only irreplaceable content in our operator and target namespaces. This pull moves that content into openshift-config since we cannot manage it.  In theory, this will make it possible to delete the entire operator and target namespace and recover from it.
/assign @sttts @mfojtik

discussed in slack last week.
/retest


MP GO:0004665	prephenate dehydrogenase (NADP+) activity

child of
GO:0008977	prephenate dehydrogenase activity?

According to the definition, GO:0008977 uses NAD+. GO:0004665 uses NADP+

OK I didn't read the def ;( rule 1
Can the term name be altered....I am picking up a mapping to the "NAD" term on my NADP
(or my annotation is incorrect....)

id: GO:0008977
-name: prephenate dehydrogenase activity
+name: prephenate dehydrogenase (NAD+) activity


8357- update otc schema with aoes

This PR adds the remaining covid AOE questions to the csv-otc-covid-19 schema so they are properly handled if received from the sender. They are not required by the schema. Currently only "sympomatic for disease" and "illness onset date" were present as the schema was developed from specs for NIH's "MakeMyTestCount" RADx project .
If you are suggesting a fix for a currently exploitable issue, please disclose the issue to the prime-reportstream team directly outside of GitHub instead of filing a PR, so we may immediately patch the affected systems before a disclosure. See SECURITY.md/Reporting a Vulnerability for more information.
Test Steps:

run ./prime data command on test data (attached)
ensure that output has expected values for aoe questions that were valued in input
run ./prime test or ./gradlew testSmoke locally

Changes

added 7 AOE questions to schema as optional.
pregnant
date of first test
employed in healthcare
hospitalized
icu
resident congregate setting
residence type

Testing

[x ] Tested locally?
[x ] Ran ./prime test or ./gradlew testSmoke against local Docker ReportStream container?
[ ] (For Changes to /frontend-react/...) Ran npm run lint:write?
[ ] Added tests?

Process

[ ] Are there licensing issues with any new dependencies introduced?
[ ] Includes a summary of what a code reviewer should test/verify?
[ ] Updated the release notes?
[ ] Database changes are submitted as a separate PR?
[ ] DevOps team has been notified if PR requires ops support?

Linked Issues

Fixes #8357

OTC-COVID-19-SCHEMA-TEST.csv

Looking good.


Bits and pieces...

This is what I could salvage from my recent attempt on bridging limited equivocation results. I would like this to be approved by @palmskog before merging it, as it can definitely be postponed until after the release.

@traiansf Also, before merging make sure to prepare fixes for casper-cbc-proofs, as some things break there.


the final representation of the code

Hello! I want to ask, will the ply file be generated after the "python saga_gui.py --model_path <path to the pre-trained 3DGS model>" step? I also want to ask, what is the final representation of the code after it has run? “python render.py -m <path to the pre-trained 3DGS model> --precomputed_mask  --target  scene --segment
"Will the code re-produce the point cloud after running, why do I only have pictures?"

Yeah. Rendering the results can only get images. In the SAGA GUI, it generate a 0-1 bitmap, which depict the Gaussians belong to the target of interest. The point cloud (3D Gaussians) is still stored in the original 3D-GS model. You can use the bitmap to cut off the original Gaussian model to get the target.


Connection: close - Modsecurity Blocks the site on load because cdn-loop: cloudflare

A fresh install of ubuntu 20.04, nginx version 1.23.2 and Modsecurity 3.3.4. When I visit www.example.com I get 403 Forbidden. When I change:
http {
	modsecurity on;

to
http {
	modsecurity off;

the site loads normally. The reason is CDN-Loop by cloudflare. CDN-Loop is a typical Cloudflare header but Modsecurity closes the connection. The error log shows the following:
Logs and dumps

---LxD3aa9l---A--
[17/Dec/2022:15:58:03 +0000] 1671292XXXXX.5XXXX6 192.xxx.a.xx 38XXX 192.xxx.b.xx 80
---LxD3aa9l---B--
GET / HTTP/1.1
cf-visitor: {"scheme":"https"}
accept-encoding: gzip
cdn-loop: cloudflare
Connection: close
sec-fetch-mode: navigate
cf-ipcountry:
Host: example.com
X-Forwarded-For: 1x3.111.ax.45, 172.111.bx.136
X-Real-IP: 172.111.bx.136
cf-ray: 77b0dba33ddb9b8f-xyz
cache-control: max-age=0
accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
X-Forwarded-Proto: https
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36 Edg/108.0.1234.56
sec-fetch-site: none
sec-ch-ua: "Not?A_Brand";v="8", "Chromium";v="108", "Microsoft Edge";v="108"
sec-fetch-user: ?1
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"
upgrade-insecure-requests: 1
sec-fetch-dest: document
accept-language: en-US,en;q=0.9,fr;q=0.8
cf-connecting-ip: 1x3.111.ax.45

---LxD3aa9l---E--
\x0d\x0a<title>403 Forbidden</title>\x0d\x0a\x0d\x0a
403 Forbidden
\x0d\x0a
nginx\x0d\x0a\x0d\x0a\x0d\x0a\x0d\x0a\x0d\x0a\x0d\x0a\x0d\x0a\x0d\x0a\x0d\x0a

---atvQ9f4R---H--
ModSecurity: Warning. Matched "Operator `Rx' with parameter `^(?:(?:\*|[^\"(),\/:;<=>?![\x5c\]{}]+)\/(?:\*|[^\"(),\/:;<=>?![\x5c\]{}]+))(?:\s*+;\s*+(?:(?:charset\s*+=\s*+(?:\"?(?:iso-8859-15?|windows-1252|utf-8)\b\"?))|(?:(?:c(?:h(?:a(?:r(?:s(?:e[^t\"(),\/:;<=> (714 characters omitted)' against variable `REQUEST_HEADERS:accept' (Value: `image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8' ) [file "/etc/nginx/modsec/coreruleset-3.3.4/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"] [line "1147"] [id "920600"] [rev ""] [msg "Illegal Accept header: charset parameter"] [data "image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8"] [severity "2"] [ver "OWASP_CRS/3.3.4"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [hostname "192.xxx.2.3x"] [uri "/favicon.ico"] [unique_id "167129268381.104766"] [ref "v559,53t:lowercase"]
ModSecurity: Warning. Matched "Operator `Rx' with parameter `^(?i:(?:[a-z]{3,10}\s+(?:\w{3,7}?://[\w\-\./]*(?::\d+)?)?/[^?#]*(?:\?[^#\s]*)?(?:#[\S]*)?|connect (?:\d{1,3}\.){3}\d{1,3}\.?(?::\d+)?|options \*)\s+[\w\./]+|get /[^?#]*(?:\?[^#\s]*)?(?:#[\S]*)?)$' against variable `REQUEST_LINE' (Value: `GET /favicon.ico HTTP/1.1' ) [file "/etc/nginx/modsec/coreruleset-3.3.4/rules/REQUEST-920-PROTOCOL-ENFORCEMENT.conf"] [line "48"] [id "920100"] [rev ""] [msg "Invalid HTTP Request Line"] [data "GET /favicon.ico HTTP/1.1"] [severity "4"] [ver "OWASP_CRS/3.3.4"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-protocol"] [tag "paranoia-level/1"] [tag "OWASP_CRS"] [tag "capec/1000/210/272"] [hostname "192.xxx.2.3x"] [uri "/favicon.ico"] [unique_id "167129268381.104766"] [ref "v0,25"]
ModSecurity: Access denied with code 403 (phase 2). Matched "Operator `Ge' with parameter `5' against variable `TX:ANOMALY_SCORE' (Value: `8' ) [file "/etc/nginx/modsec/coreruleset-3.3.4/rules/REQUEST-949-BLOCKING-EVALUATION.conf"] [line "81"] [id "949110"] [rev ""] [msg "Inbound Anomaly Score Exceeded (Total Score: 8)"] [data ""] [severity "2"] [ver "OWASP_CRS/3.3.4"] [maturity "0"] [accuracy "0"] [tag "application-multi"] [tag "language-multi"] [tag "platform-multi"] [tag "attack-generic"] [hostname "192.xxx.2.3x"] [uri "/favicon.ico"] [unique_id "167129268381.104766"] [ref ""]

How can I solve this issue?

Hello @jessicana ,
If you have a well-understood false positive, a common way to handle it is to disable the problematic rules using one the various tools within ModSecurity -- preferably under as narrow conditions as practical.
If you wish to inquire about the specific rules being triggered in your case, I'll note that this github is for the engine itself, and that the CRS rules are managed by a separate group.

Using one of the below rules resolved the problem:
SecRule &REQUEST_HEADERS:Transfer-Encoding "@eq 0" "id:1017,phase:2,msg:'POST without Content-Length or Transfer-Encoding headers',pass,nolog,noauditlog,ctl:ruleRemovebyID=949110"
SecRule &REQUEST_HEADERS:Content-Length "@eq 0" "id:1018,phase:2,msg:'POST without Content-Length or Transfer-Encoding headers',pass,nolog,noauditlog,ctl:ruleRemovebyID=949110"
Would you please connect me with the group that manages the CRS rules to confirm if this was the best solution?
Many thanks for all of your support.


Better experience when running with nodeshift

I noticed that when you deploy this from your local machine, we are not creating the config-map yaml or updating the roles.
I'm pretty sure nodeshift knows how to create configMaps,  but i don't think it knows about role bindings yet.
When someone launches this from the launcher,  all this is setup for them.  maybe we can make this a better experience when running locally

closing for now,  as it might be out of scope for nodeshift to handle
If we feel we want to re-visit this in the future,  we can re-open


Report Viewer images are broken again

FastReport.OpenSource.Web 2020.3.0 has broken the Report Viewer images again as per Issue #151 in ASP.NET Core MVC. I am unable to use a newer version due to Issue #192
The report is show but the loading spinner and toolbar icons are broken links.
 Do you still have problems with this?


Hashes should get included with url dependencies in exported requirements.txt

Poetry version: Poetry (version 1.2.2)
Python version: 3.11.0
OS version and name: Arch Linux
pyproject.toml: -

[x] I am on the latest stable Poetry version, installed using a recommended method.
[x] I have searched the issues of this repo and believe that this is not a duplicate.
[x] I have consulted the FAQ and blog for any relevant entries or release notes.
[x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue
Given something along django-fsm-admin = { url = "https://github.com/infarm/django-fsm-admin/archive/38f2719935be16a7c01d110651ad8ea8383bbe1d.zip" } in [tool.poetry.dependencies] it results in django-fsm-admin @ https://github.com/infarm/django-fsm-admin/archive/38f2719935be16a7c01d110651ad8ea8383bbe1d.zip ; python_version >= "3.11" and python_version < "4.0" when using poetry export -f requirements.txt -o "requirements-main.txt" --only=main.
When using pip install -r requirements-main.txt it causes the following error:

ERROR: Hashes are required in --require-hashes mode, but they are missing from some requirements. Here is a list of those requirements along with the hashes their downloaded archives actually had. Add lines like these to your requirements files to prevent tampering. (If you did not enable --require-hashes manually, note that it turns on automatically when any package has a hash.)
https://github.com/infarm/django-fsm-admin/archive/38f2719935be16a7c01d110651ad8ea8383bbe1d.zip --hash=sha256:32bc3205cec3ec83a78dd0fd0b5f02f25d81a9689493c2580c8fdb4e02c6f4ec

I think with "url" requirements hashes can and should get included in the exported file.
For reference: this was fixed in PDM in https://github.com/pdm-project/pdm/commit/1a1f8748 (via https://github.com/pdm-project/pdm/issues/1103), where the output in requirements.txt looks as follows:
django-fsm-admin @ https://github.com/infarm/django-fsm-admin/archive/38f2719935be16a7c01d110651ad8ea8383bbe1d.zip \
    --hash=sha256:32bc3205cec3ec83a78dd0fd0b5f02f25d81a9689493c2580c8fdb4e02c6f4ec

duplicate #146
and as there IMO this belongs in poetry proper rather than here: if poetry were to include hashes in the lockfile then this plugin would automatically export them


Upgrade to failure 0.1.2 idioms

Fixes some deprecation warnings in Cargo

r? @matklad
(rust_highfive has picked a reviewer for you, use r? to override)

@bors: r+

:pushpin: Commit adbd625e35f15d3a8c5a0851d8226bcdf4195184 has been approved by alexcrichton

:hourglass: Testing commit adbd625e35f15d3a8c5a0851d8226bcdf4195184 with merge d06fe7b169447756abb41f51aed7700b5268e8e6...

:sunny: Test successful - status-appveyor, status-travis
Approved by: alexcrichton
Pushing d06fe7b169447756abb41f51aed7700b5268e8e6 to master...


Need to define IITB toil

How to measure reduction rate of IITB toil?
Rate = number of production release / number of personnel?
Rate = number of service desk requests (ITSM) / number of operations personnel?

Number of automated process * human interventions / Number of manual
process * human interventions?
Le lun. 8 juill. 2019 17 h 33, Rémy B<EMAIL_ADDRESS>a écrit :

How to measure reduction rate of IITB toil?
Rate = number of production release / number of personnel?
Rate = number of service desk requests (ITSM) / number of operations
personnel?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/sara-sabr/ITStrategy/issues/51?email_source=notifications&email_token=AAM4TZOSAAX7EM4LWMCHVZTP6OXELA5CNFSM4H67VMV2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4G56QG5Q,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAM4TZOPCVCNQUDMB5YUU4LP6OXELANCNFSM4H67VMVQ
.

Perhaps we need Lead Time analysis (or value stream analysis, whatever phrasing) first.
Then we could map increases in efficiency to any hiring that took place.

I think this is critical.
Le mar. 16 juill. 2019 11 h 18, Jayson<EMAIL_ADDRESS>a écrit :

Perhaps we need Lead Time analysis (or value stream analysis, whatever
phrasing) first.
Then we could map increases in efficiency to any hiring that took place.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/sara-sabr/ITStrategy/issues/51?email_source=notifications&email_token=AAM4TZMFZX4LBJA6TJG5DFLP7XRFDA5CNFSM4H67VMV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2BGK7I#issuecomment-511862141,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAM4TZO4URT32W2F52KW5T3P7XRFDANCNFSM4H67VMVQ
.

That should probably be part of our strategies. To be reviewed.


Test Prime V2 branch to go along with the API Versioning ADR

Do Not Merge
Jira ticket
Summary
This is a test PR of possible approaches when creating a version 2 of our Prime API. It has added one new List Prime Moves endpoint to the version 2 api.
There are a couple of options in going about this, so feel free to branch off or make your own suggestions.

Warnings

:warning:
Please add the JIRA issue key to the PR title (e.g. MB-123)

  Generated by :no_entry_sign: dangerJS against 376984598ba2fd6b950b2af308501c17ff0870e2


The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.

Have you changed the setting.py? There is an GAME_API_PREFIX option.

啊？setting.py没看过

已经解决了，谢谢


OIDC support for sovereign clouds

Added OIDC support for AzureUSGovernment and AzureChinaCloud
ISSUE #257- https://github.com/Azure/login/issues/257

@arnoldna  Thanks for the PR. Can you add more details on the tests that were done for this PR?

I have access to an AzureUSGovernment subscription and tested connectivity utilizing the following yaml:
`name: Run Azure Login with OpenID Connect
on: [push]
permissions:
id-token: write
contents: read
jobs:
build-and-deploy:
runs-on: ubuntu-latest
steps:
- name: 'Az CLI login'
uses: azure/login@v1
with:
client-id: ${{ secrets.AZURE_CLIENT_ID }}
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
environment: 'AzureUSGovernment'
- name: 'Run Azure CLI commands'
  run: |
      az account show
      az group list
      pwd`

If there are other tests you would like to run then let me know.

@BALAGA-GAYATRI Any updates on this PR?

@BALAGA-GAYATRI Any updates on this PR?

We are following up on the same. Will get back with the info.

@BALAGA-GAYATRI - Can we get an update on when this PR will be approved? Thanks

friendly bump on this ❤

Looks like the linting is failing on README - can someone please fix this and merge? PR has been open for months

Closing this pr, the feature is done in https://github.com/Azure/login/pull/321.


added beta app release 2023 page

Can you share how it looks on mobile? I won't be near a computer for awhile