Cannot build lock even though dependencies are found.

Be sure to check the existing issues, both open and closed.
Describe the issue briefly here.
Describe you environment

macOS Sierra
Python 3.5.2
pipenv, version 8.2.7

Expected result
I want to install shakedown with pipenv --three install dcos-shakedown. It should create a Pipfile and a lock.
Actual result
I get the following output
CRITICAL:pip.index:Could not find a version that satisfies the requirement dcoscli==0.5.5 (from versions: 0.1.1, 0.1.2, 0.1.3, 0.1.4, 0.1.5, 0.1.6, 0.1.7, 0.1.8, 0.1.9, 0.1.10, 0.1.11, 0.1.12, 0.1.13, 0.1.14, 0.1.15, 0.2.0, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.3.5, 0.3.6, 0.4.0, 0.4.1, 0.4.2, 0.4.3, 0.4.4, 0.4.5, 0.4.6, 0.4.10, 0.4.11, 0.4.12, 0.4.13, 0.4.14)
Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
No matching distribution found for dcoscli==0.5.5

This is odd since pipenv --three install dcoscli can install 0.5.5 and create a lock. There is a dcoscli-0.5.5 on PyPi. I assumes that pipenv tries to create a closure for Python 2 as well but there is no dcoscli 0.5.5 for Python 2.
Steps to replicate
Just run pipenv --three install --verbose dcos-shakedown with Python 3.5.

The shakedown package should probably be Python 3 only. However, I think that pipenv should create the lock only for a specific Python version if the Pipefile includes a required Python version.

Here is the verbose output of the lock
Locking [dev-packages] dependencies…
Using pip: -i https://pypi.python.org/simple

                          ROUND 1
Current constraints:

Finding the best candidates:

Finding secondary dependencies:
------------------------------------------------------------
Result of round 1: stable, done
Locking [packages] dependencies…
Using pip: -i https://pypi.python.org/simple

                          ROUND 1
Current constraints:
  dcos-shakedown from git+https://github.com/dcos/shakedown.git#egg=dcos-shakedown

Finding the best candidates:
  found candidate -e git+https://github.com/dcos/shakedown.git#egg=dcos-shakedown (constraint was <any>)

Finding secondary dependencies:

New dependencies found in this round:
  adding [u'click', '', '[]']
  adding [u'dcos-shakedown', '', '[]']
  adding [u'dcoscli', '==0.5.5', '[]']
  adding [u'paramiko', '', '[]']
  adding [u'pytest', '', '[]']
  adding [u'pytest-timeout', '', '[]']
  adding [u'retrying', '', '[]']
  adding [u'scp', '', '[]']
Removed dependencies in this round:
Unsafe dependencies in this round:
------------------------------------------------------------
Result of round 1: not stable

                          ROUND 2
Current constraints:
  click
  dcos-shakedown from git+https://github.com/dcos/shakedown.git#egg=dcos-shakedown
  dcoscli==0.5.5
  paramiko
  pytest
  pytest-timeout
  retrying
  scp

Finding the best candidates:
  found candidate click==6.7 (constraint was <any>)
  found candidate -e git+https://github.com/dcos/shakedown.git#egg=dcos-shakedown (constraint was <any>)
  found candidate dcoscli==0.5.5 (constraint was ==0.5.5)
  found candidate paramiko==2.3.1 (constraint was <any>)
  found candidate pytest==3.2.3 (constraint was <any>)
  found candidate pytest-timeout==1.2.0 (constraint was <any>)
  found candidate retrying==1.3.3 (constraint was <any>)
  found candidate scp==0.10.2 (constraint was <any>)

Finding secondary dependencies:
  click==6.7                requires click==6.7
  scp==0.10.2 not in cache, need to check index
  scp==0.10.2               requires paramiko, scp==0.10.2
  pytest-timeout==1.2.0     requires pytest-timeout==1.2.0, pytest>=2.8.0
  dcoscli==0.5.5 not in cache, need to check index
CRITICAL:pip.index:Could not find a version that satisfies the requirement dcoscli==0.5.5 (from versions: 0.1.1, 0.1.2, 0.1.3, 0.1.4, 0.1.5, 0.1.6, 0.1.7, 0.1.8, 0.1.9, 0.1.10, 0.1.11, 0.1.12, 0.1.13, 0.1.14, 0.1.15, 0.2.0, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.3.5, 0.3.6, 0.4.0, 0.4.1, 0.4.2, 0.4.3, 0.4.4, 0.4.5, 0.4.6, 0.4.10, 0.4.11, 0.4.12, 0.4.13, 0.4.14)
Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
No matching distribution found for dcoscli==0.5.5

This is odd. When I install dcsocli with pipenv --three install dcoscli I see
Installing dcoscli…
Collecting dcoscli
  Using cached dcoscli-0.5.5-py3-none-any.whl

followed by
                          ROUND 1
Current constraints:
  dcos-shakedown from git+https://github.com/dcos/shakedown.git#egg=dcos-shakedown
  dcoscli

Finding the best candidates:
  found candidate -e git+https://github.com/dcos/shakedown.git#egg=dcos-shakedown (constraint was <any>)
  found candidate dcoscli==0.4.14 (constraint was <any>)

Finding secondary dependencies:
  dcoscli==0.4.14           requires dcos==0.4.14, dcoscli==0.4.14, docopt<1.0,>=0.6, pkginfo==1.2.1, toml<1.0,>=0.9, virtualenv<16.0,>=13.0

why is it trying to install 0.4.14 of dcoscli? Especially when pipenv --three run dcos --version outputs dcoscli.version=0.5.5.

It’s not. This is the locking process. It’s doing dependency resolution based on what it know and what you have cached. Try pipenv lock —clear

On Oct 18, 2017, at 6:16 AM, Karsten Jeschkies<EMAIL_ADDRESS>wrote:
This is odd. When I install dcsocli with pipenv --three install dcoscli I see
Installing dcoscli…
Collecting dcoscli
Using cached dcoscli-0.5.5-py3-none-any.whl
followed by
                      ROUND 1

Current constraints:
dcos-shakedown from git+https://github.com/dcos/shakedown.git#egg=dcos-shakedown
dcoscli
Finding the best candidates:
found candidate -e git+https://github.com/dcos/shakedown.git#egg=dcos-shakedown (constraint was )
found candidate dcoscli==0.4.14 (constraint was )
Finding secondary dependencies:
dcoscli==0.4.14           requires dcos==0.4.14, dcoscli==0.4.14, docopt<1.0,>=0.6, pkginfo==1.2.1, toml<1.0,>=0.9, virtualenv<16.0,>=13.0
why is it trying to install 0.4.14 of dcoscli? Especially when pipenv --three run dcos --version outputs dcoscli.version=0.5.5.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

You’ll need to provide the output of pipenv lock —verbose and pipenv —version

On Oct 18, 2017, at 6:16 AM, Karsten Jeschkies<EMAIL_ADDRESS>wrote:
This is odd. When I install dcsocli with pipenv --three install dcoscli I see
Installing dcoscli…
Collecting dcoscli
Using cached dcoscli-0.5.5-py3-none-any.whl
followed by
                      ROUND 1

Current constraints:
dcos-shakedown from git+https://github.com/dcos/shakedown.git#egg=dcos-shakedown
dcoscli
Finding the best candidates:
found candidate -e git+https://github.com/dcos/shakedown.git#egg=dcos-shakedown (constraint was )
found candidate dcoscli==0.4.14 (constraint was )
Finding secondary dependencies:
dcoscli==0.4.14           requires dcos==0.4.14, dcoscli==0.4.14, docopt<1.0,>=0.6, pkginfo==1.2.1, toml<1.0,>=0.9, virtualenv<16.0,>=13.0
why is it trying to install 0.4.14 of dcoscli? Especially when pipenv --three run dcos --version outputs dcoscli.version=0.5.5.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

Clearing the lock cache did not help. Here is the full output of pipenv --three lock --verbose
› pipenv --three lock --verbose
Virtualenv already exists!
Removing existing virtualenv…
Warning: PYENV_ROOT is not set. New python paths will probably not be exported properly after installation.
Creating a virtualenv for this project…
Using /Users/kjeschkies/.pyenv/shims/python3 to create virtualenv…
⠋Running virtualenv with interpreter /Users/kjeschkies/.pyenv/shims/python3
Using base prefix '/Users/kjeschkies/.pyenv/versions/3.5.2'
New python executable in /Users/kjeschkies/.local/share/virtualenvs/marathon-ldTRsJsI/bin/python3
Also creating executable in /Users/kjeschkies/.local/share/virtualenvs/marathon-ldTRsJsI/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /Users/kjeschkies/.local/share/virtualenvs/marathon-ldTRsJsI
Locking [dev-packages] dependencies…
Using pip: -i https://pypi.python.org/simple

                          ROUND 1
Current constraints:

Finding the best candidates:

Finding secondary dependencies:
------------------------------------------------------------
Result of round 1: stable, done
Locking [packages] dependencies…
Using pip: -i https://pypi.python.org/simple

                          ROUND 1
Current constraints:
  dcos-shakedown

Finding the best candidates:
  found candidate dcos-shakedown==1.4.8 (constraint was <any>)

Finding secondary dependencies:
  dcos-shakedown==1.4.8     requires click, dcos-shakedown==1.4.8, dcoscli==0.5.5, paramiko, pytest, pytest-timeout, retrying, scp

New dependencies found in this round:
  adding [u'click', '', '[]']
  adding [u'dcos-shakedown', '==1.4.8', '[]']
  adding [u'dcoscli', '==0.5.5', '[]']
  adding [u'paramiko', '', '[]']
  adding [u'pytest', '', '[]']
  adding [u'pytest-timeout', '', '[]']
  adding [u'retrying', '', '[]']
  adding [u'scp', '', '[]']
Removed dependencies in this round:
Unsafe dependencies in this round:
------------------------------------------------------------
Result of round 1: not stable

                          ROUND 2
Current constraints:
  click
  dcos-shakedown==1.4.8
  dcoscli==0.5.5
  paramiko
  pytest
  pytest-timeout
  retrying
  scp

Finding the best candidates:
  found candidate click==6.7 (constraint was <any>)
  found candidate dcos-shakedown==1.4.8 (constraint was ==1.4.8)
  found candidate dcoscli==0.5.5 (constraint was ==0.5.5)
  found candidate paramiko==2.3.1 (constraint was <any>)
  found candidate pytest==3.2.3 (constraint was <any>)
  found candidate pytest-timeout==1.2.0 (constraint was <any>)
  found candidate retrying==1.3.3 (constraint was <any>)
  found candidate scp==0.10.2 (constraint was <any>)

Finding secondary dependencies:
  pytest-timeout==1.2.0 not in cache, need to check index
  pytest-timeout==1.2.0     requires pytest-timeout==1.2.0, pytest>=2.8.0
  paramiko==2.3.1           requires bcrypt>=3.1.3, cryptography>=1.5, paramiko==2.3.1, pyasn1>=0.1.7, pynacl>=1.0.1
  click==6.7 not in cache, need to check index
  click==6.7                requires click==6.7
  pytest==3.2.3             requires py>=1.4.33, pytest==3.2.3, setuptools
  dcos-shakedown==1.4.8     requires click, dcos-shakedown==1.4.8, dcoscli==0.5.5, paramiko, pytest, pytest-timeout, retrying, scp
  retrying==1.3.3           requires retrying==1.3.3, six>=1.7.0
  dcoscli==0.5.5 not in cache, need to check index
CRITICAL:pip.index:Could not find a version that satisfies the requirement dcoscli==0.5.5 (from versions: 0.1.1, 0.1.2, 0.1.3, 0.1.4, 0.1.5, 0.1.6, 0.1.7, 0.1.8, 0.1.9, 0.1.10, 0.1.11, 0.1.12, 0.1.13, 0.1.14, 0.1.15, 0.2.0, 0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.3.5, 0.3.6, 0.4.0, 0.4.1, 0.4.2, 0.4.3, 0.4.4, 0.4.5, 0.4.6, 0.4.10, 0.4.11, 0.4.12, 0.4.13, 0.4.14)
Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
No matching distribution found for dcoscli==0.5.5

FYI every time you pass --three you are telling pipenv to destroy your existing virtualenv, not sure if that was intended but just FYI.  I will look into this a bit more but at a glance since you have pyenv installed you should set the environment variable PYENV_ROOT per the error message also.

@techalchemy, thanks. I did not know that --three would create a new env each time. I thought it would just tell pipenv to use only Python 3. I did set PYENV_ROOT once but then pipenv created a virtuelenv with Python 3.6 instead of 3.5 as defined in my local pyenv.

I still think that the issue is that pipenv tries to create a closure for Python 2 and that shakedown is not limited to Python 3.

Throwing in my observation on the dependency resolution behavior:
It really looks like the Python 2 or pip2 executable is used for the dependency resolution here.
I have the feeling I've seen similar issues lately, but haven't had the time to look into it.
Taking a guess here on the environment here, I'd say pipenv was installed globally (which is ok), and that the global or root python is Python 2. @jeschkies Can you confirm if that's right? Note, I'm not talking about the python version of the virtualenv created by pipenv, I'm talking about the python executable used to run pipenv.
I've also noted that you seem to be using pyenv, I'm not sure if there's any side effect here on the python executable being used by pipenv itself..
I'm not an expert (yet) on how pipenv determines which python/pip version to use for itself, but if it just blindly picks the root python of the environment in which it's installed for itself, then we know why we get python 2 results here.

Thanks for the pointers @vphilippon. I've ripped out my old pyenv version and installed the latest and Python 3.5.2. I then updated pip to 9.0.1 outside of the virtualenv. I also patched shakedown to require a specific Python version and used git dependencies. See
• https://github.com/jeschkies/shakedown/blob/master/setup.py
• https://github.com/mesosphere/marathon/blob/karsten/use-pipenv/Pipfile
I wonder how the lock works since it does not mention the commit hash as far as I can tell. Also, the lock states the dcos cli version as 0.4.14 but pipenv run dcos --version outputs 0.5.5.
Overall I wonder how pipenv is handling different Python versions and what should be expected from pipenv in that regard.

Hm the lock does not work for the dependencies:
system in marathon/ on karsten/use-pipenv
› pipenv --rm
Removing virtualenv (/Users/kjeschkies/.local/share/virtualenvs/marathon-ldTRsJsI)…

system in marathon/ on karsten/use-pipenv
› pipenv install
Creating a virtualenv for this project…
Using /Users/kjeschkies/.pyenv/versions/3.5.2/bin/python3.5m to create virtualenv…
⠋Running virtualenv with interpreter /Users/kjeschkies/.pyenv/versions/3.5.2/bin/python3.5m
Using base prefix '/Users/kjeschkies/.pyenv/versions/3.5.2'
New python executable in /Users/kjeschkies/.local/share/virtualenvs/marathon-ldTRsJsI/bin/python3.5m
Also creating executable in /Users/kjeschkies/.local/share/virtualenvs/marathon-ldTRsJsI/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: /Users/kjeschkies/.local/share/virtualenvs/marathon-ldTRsJsI
Installing dependencies from Pipfile.lock (0d5d00)…

Looking at that, the default, the venv seems to use Python 3.5.2, and evaluates markers as such. That looks like a step forward.
@jeschkies I would delete the Pipfile.lock and run pipenv lock --clear to go on a fersh locking. If the root python issue is no longer in cause, that should do it.

I agree with @vphilippon but if you have further issues I would ask if you can provide your Pipfile and Pipfile.lock so that we can assist with the troubleshooting process further

I did start fresh without a Pipfile.lock and then used the lock file to recreate the env. I will try it one more time a report back.

Alright. I've reproduced the issue in a clean folder. You see all commands out the output in this gist.
Here is what I try to do and what happens:

Install shakedown from git for Python 3.5 with pipenv --three -e git+https://github.com/jeschkies/shakedown.git#egg=dcos-shakedown
dcos-cli, one dependency of shakedown is install with version 0.5.5. This is as expected.
Remove the virtualenv with pipenv --rm.
Create a new virtualenv with pipenv --three install.
dcos-cli 0.4.14 is installed. This is not expected.

Right but I still need you to include your pipfile.lock to help troubleshoot. Can you also provide the output of pipenv lock —verbose after you have seen the wrong version info

Here is the output of pipenv lock --verbose and the generated Pipfile.lock

The Pipfile.lock includes
"dcos": {
    "hashes": [
                "sha256:3d48d0773b5dd5e82ddb9cfa946f17755789e8c2ff53ecc75b32194b41248611"
            ],
    "version": "==0.4.14"
}

but the pipenv run dcos --version yields dcoscli.version=0.5.5.

Allright, this verbose output given is coherent and match the content of the Pipfile.lock.
But, the reported installed version (from dcos --version) doesn't match.
I tried the steps and could not reproduce. I get:
"dcos": {
    "hashes": [
        "sha256:01d722e13296bd38c2b391ffbd5bc111fa7b5ec09bb058651713bce62a2d45a7"
    ],
    "version": "==0.5.5"
},
"dcos-shakedown": {
    "editable": true,
    "git": "https://github.com/jeschkies/shakedown.git",
    "ref": "012841a38a59d00043c15a66400501811715ab86"
},
"dcoscli": {
    "hashes": [
        "sha256:47b442a2823ab3b27cd520f5c3762ab0e630eca473a273e111619bd5c1a16ea6"
    ],
    "version": "==0.5.5"
},

and, from pipenv run dcos --version
dcoscli.version=0.5.5
dcos.version=N/A
dcos.commit=N/A
dcos.bootstrap-id=N/A

both on the first install, and on the re-install from the Pipfile.lock.
@jeschkies In the command from the gist, I've seen no pipenv lock --clear (note the --clear). Did you run it as I suggested before? If not, please do, as it could still be an issue with the dependency cache. (And there's no harm in redoing it to be a 100% sure.)
Otherwise, if we're 100% sure the dependency cache was cleared, please and give the output of pipenv run pip freeze.

@vphilippon are you using pyenv? I'm suspecting this is the issue here. I cannot get it to lock dcos-cli 0.5.5 with the following:

mkdir pipenv-test && cd pipenv-test
pyenv local 3.5.2
pipenv --three install -e git+https://github.com/jeschkies/shakedown.git#egg=dcos-shakedown

The Pipfile.lock will have the 0.4.14. What am I missing here? Is pipenv picking up the from Python version or the wrong dcos-cli package?

@jeschkies No, I'm not using pyenv. That was the next thing I was going to check, after making sure it was not a dependency cache issue. I also wanted to have a look at the full list of installed package version, which is why I asked for the output of pipenv run pip freeze.
We'll need someone to try using pyenv and try to reproduce, although if all of your python environment are on Py 3+ now, that should not be an issue causing Py 2 dependencies to be computed.

I don't believe pipenv respects the local pyenv setting, which will cause problems for you potentially.  When I attempted to just run pipenv install... my system attempted to fall back to python 2.7.
Running the command as pipenv --python=3.5.2 install -e git+https://github.com/jeschkies/shakedown.git#egg=dcos-shakedown worked just fine (and is the recommended way to specify python versions to pipenv instead of hoping it picks up on a local file):
        "dcos": {
            "hashes": [
                "sha256:01d722e13296bd38c2b391ffbd5bc111fa7b5ec09bb058651713bce62a2d45a7"
            ],
            "version": "==0.5.5"
        },
        "dcos-shakedown": {
            "editable": true,
            "git": "https://github.com/jeschkies/shakedown.git",
            "ref": "012841a38a59d00043c15a66400501811715ab86"
        },
        "dcoscli": {
            "hashes": [
                "sha256:47b442a2823ab3b27cd520f5c3762ab0e630eca473a273e111619bd5c1a16ea6"
            ],
            "version": "==0.5.5"
        },

FYI this is an issue related to how pipenv resolves pyenv python paths, since it doesn't know about pyenv local settings it will use its own guess at which python interpreter to use in the virtualenv as compared with for dependency resolution.  These will wind up being different because one will be resolved by the pyenv shims and one will not.

@techalchemy is this a bug in pipenv or pyenv or both???

Here is the output of pipenv run pip freeze after I ran pipenv --python=3.5.2 install -e git+https://github.com/jeschkies/shakedown.git#egg=dcos-shakedown. Here is the content of the Pipfile.lock
       "dcos": {
            "hashes": [
                "sha256:3d48d0773b5dd5e82ddb9cfa946f17755789e8c2ff53ecc75b32194b41248611"
            ],
            "version": "==0.4.14"
        },
        "dcos-shakedown": {
            "editable": true,
            "git": "https://github.com/jeschkies/shakedown.git",
            "ref": "012841a38a59d00043c15a66400501811715ab86"
        },
        "dcoscli": {
            "hashes": [
                "sha256:1d7506df5e32fc96566169896fb42ba80aba4687d1c65eb2b52b320b5cfad0ca"
            ],
            "version": "==0.4.14"
        },

This is so strange. Even with pyenv global 3.5.2 it does not work.

@jeschkies try pipenv lock —clear per @vphillipon above now that you’ve explicitly told pipenv to use 3.5.2 and see if you wind up with the correct resolutions
@erinxocon it’s only a bug if you think pipenv should respect local pyenv settings in general.  What I mean is, if you set pyenv local 3.5.2 and then run pipenv install I think it works correctly. However, the —three and —two helper flags search the path separately for the latest installed version no matter which one you have configured with pyenv. So the question is really whether pipenv should instead respect your pyenv selections if they apply. This feels kind of like an edge case especially when you can just be explicit when you don’t want the latest version.

@techalchemy even a pipenv lock --clear did not work. I think it's really just pipenv ignoring my pyenv configuration not matter what.

@jeschkies It just looks at your $PATH variable to search through.
@techalchemy I'm torn.  Kinda seems like a user environment issue, out of the scope of pipenv.  However that was originally our thinking when using pew for virtualenv management, which caused more problems due to shells that aren't configured correctly, so while not a pipenv issue, we changed to avoid problems.

@erinxocon hm, how would you have different Python versions without pyenv? Should I just installed them and have all of them on $PATH? That does not seem right. I understand that pipenv should not be too smart about it. However, some docs would probably help here.

@jeschkies normal python gets put on the path, and if you install 6 versions yes, all go on your path. Pyenv is a helper function but is definitely not necessary, and pipenv doesn’t read configuration files. It just looks on your path for what ‘python’ points at unless you specify a version or a specific executable.   As of right now python2 is only allowed to be named python and python3 is python3.  You can have all versions of python in your path, they'll just be addressed using a more specific syntax.  python26, python27, python34

Hm, ok. So is there another way to manage Python versions that work nicer with pipenv?

@jeschkies nicer? I personally use pyenv withPIPENV_DEFAULT_PYTHON_VERSION SET TO 3.6.3. With pyenv installed (or any setup really) you can just pass —python=3.6.3 or —python=2.7.14. That’s the thing with guessing (I.e. with —three and —two)— it’s not explicit. If you want a specific version you will need to specify it

I have the same issue when trying to install Django==2.0b1. It only offers a Python 3 wheel (the same as dcoscli).
I've installed Pipenv globally through Homebrew, and I use pyenv to manage my Python versions. I've also specified python_full_version = "3.6.3" in my Pipfile (along with the prerelease option).
It fails when I run pipenv install through the global Pipenv, but if I install Pipenv (via Pip) in a Python 3 version, it then succeeds. I'm guessing this means Pipenv isn't using the version specified in Pipfile to do the dependency version look up? I get the same error when using pip install in Python 2.7.10.
I've tried using the PIPENV_DEFAULT_PYTHON_VERSION environment variable, but that didn't work either.

Hi @jamieconnolly I think this might have something to do with the shims pyenv puts in place to make multiple environments available.  Can't be sure on this as I don't use pyenv.

@jamieconnolly I am guessing it is related to the same bug we have touchpoints for in various places relating to, as you mentioned, dependency resolution trickery.  One question though -- does pipenv install successfully install the specified version in the former case, but fail on the resolution step and advise you to use --skip-lock? Or does it fail to install in the first place?

@jamieconnolly, thanks for the hint. I'll try using a pipenv installation through pip of the local pyenv configuration.
@techalchemy, I should be more specific. It seems that pipenv is not picking up the correct Python version from pyenv when it generates the Pipfile.lock. As you've said

So the question is really whether pipenv should instead respect your pyenv selections if they apply.

Since the answer is not necessarily yes, I'm wondering how I can manage different Python version so that pipenv picks up the correct one when it generates the lock file.
I'll give --python=3.6.3 a try as well.

@jeschkies what I meant by that is it doesn't respect whatever you set with pyenv shell or pyenv local or whatever.  It should respect the pythons that are on your path, and potentially if pyenv is configured properly it will respect whatever is set with pyenv global since thats how the shim for python will be resolved (if you don't specify a version)

Ok, but I'm wondering why the  pyenv local Python version is not found when the lock file is generated. It's either the configuration of pyenv or an issue with pipenv. I'm not sure it's the configuration since pipenv has no troubles installing the correct package and the Python version in pipenv shell python --version is correct as well. So why is pipenv lock not picking up the correct version?

@jeschkies because if you don't specify a version, pipenv run and pipenv shell will default to system python (which pyenv will resolve via shims if you use pyenv local to whatever you set).  Resolution happens using the pip-tools resolver and doesn't ever invoke python directly, so the pyenv shims are essentially bypassed for version resolution and pipenv just looks through your PATH and PYENV_ROOT for all of the available pythons.  So that's why pyenv local doens't have any impact on the lockfile.

@techalchemy, ah, ok, I think I get a sense of the difference now. So pipenv install and pipenv shell do invoke Python directly and that's why I end with the correct versions there?

@jeschkies that's mostly right, although I think pipenv shell will depend on whether you are using the PIPENV_SHELL_FANCY environment variable / are on windows

I am seeing the same issue with Django==2.0rc1. Really not much to add other than what @jamieconnolly said but here's some data, if you'd like:
Pipfile
[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[dev-packages]

[packages]

"psycopg2" = "*"
django = "==2.0rc1"

[pipenv]
allow_prereleases = true

➜  metada git:(master) ✗ pip3 freeze
certifi==2017.11.5
chardet==3.0.4
Django==2.0rc1
flake8==3.5.0
idna==2.6
mccabe==0.6.1
pew==1.1.0
pycodestyle==2.3.1
pyflakes==1.6.0
pytz==2017.3
requests==2.18.4
urllib3==1.22
virtualenv==15.1.0
virtualenv-clone==0.2.6

When I try to lock, even with --pre, I get this:
➜  metada git:(master) ✗ pipenv lock --pre
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
CRITICAL:pip.index:Could not find a version that satisfies the requirement django==2.0rc1 (from versions: 1.1.3, 1.1.4, 1.2, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6, 1.2.7, 1.3, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.3.5, 1.3.6, 1.3.7, 1.4, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.7, 1.4.8, 1.4.9, 1.4.10, 1.4.11, 1.4.12, 1.4.13, 1.4.14, 1.4.15, 1.4.16, 1.4.17, 1.4.18, 1.4.19, 1.4.20, 1.4.21, 1.4.22, 1.5, 1.5.1, 1.5.2, 1.5.3, 1.5.4, 1.5.5, 1.5.6, 1.5.7, 1.5.8, 1.5.9, 1.5.10, 1.5.11, 1.5.12, 1.6, 1.6.1, 1.6.2, 1.6.3, 1.6.4, 1.6.5, 1.6.6, 1.6.7, 1.6.8, 1.6.9, 1.6.10, 1.6.11, 1.7, 1.7.1, 1.7.2, 1.7.3, 1.7.4, 1.7.5, 1.7.6, 1.7.7, 1.7.8, 1.7.9, 1.7.10, 1.7.11, 1.8a1, 1.8b1, 1.8b2, 1.8rc1, 1.8, 1.8.1, 1.8.2, 1.8.3, 1.8.4, 1.8.5, 1.8.6, 1.8.7, 1.8.8, 1.8.9, 1.8.10, 1.8.11, 1.8.12, 1.8.13, 1.8.14, 1.8.15, 1.8.16, 1.8.17, 1.8.18, 1.9a1, 1.9b1, 1.9rc1, 1.9rc2, 1.9, 1.9.1, 1.9.2, 1.9.3, 1.9.4, 1.9.5, 1.9.6, 1.9.7, 1.9.8, 1.9.9, 1.9.10, 1.9.11, 1.9.12, 1.9.13, 1.10a1, 1.10b1, 1.10rc1, 1.10, 1.10.1, 1.10.2, 1.10.3, 1.10.4, 1.10.5, 1.10.6, 1.10.7, 1.10.8, 1.11a1, 1.11b1, 1.11rc1, 1.11, 1.11.1, 1.11.2, 1.11.3, 1.11.4, 1.11.5, 1.11.6, 1.11.7)
Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.

So when I do that, and run the graph:
➜  metada git:(master) ✗ pipenv graph
Django==2.0rc1
  - pytz [required: Any, installed: 2017.3]
psycopg2==<IP_ADDRESS>

Just can't create a lock file.

Sorry to spam everyone, but I also found a "workaround" my issue. I initially had piping installed via brew but decided to switch to installation via pip, or rather pip3 via pip3 install --user pipenv.
This version of pipenv was able to lock my Django==2.0rc1 properly. I am not sure if #965 is really related to this, but I also saw that issue whilst Googlin' so I thought I'd mention. Basically how one install pipenv seems to make a difference as to whether it picks up the correct python wheels while doing install/lock.

I seem to have gotten round a similar issue by removing the homebrew pipenv install and using a standard pip3 install pipenv. I think my error messages relate to the python 2 dependencies that https://github.com/Homebrew/homebrew-core/blob/master/Formula/pipenv.rb require, I expect in assumption of a native python2 install rather than taking advantage of python 3 if available. Will try to send an error message when I have time to reproduce.

this is likely because of setup.py constraints, nothing we can do about it

e.g. install pipenv with python3

Homebrew will soon use python3 as it's default to help avoid these things.


Enhance incapsulation

Syntax like stream.config.rate.rate_config.day_count_convention is cumbersome and deeply coupled with the representation logic of the structs. Writing getter methods leave us with the flexibility to change the underlying representation without rewriting code everywhere, but only rewriting the getters.

Incapsulation in julia should be studied before. For example Dates.Date does not have day, month, year fields but functions.


hydrateOnVisible() throws an error if the component does not render an element

Vue version
3.5.6
Link to minimal reproduction
https://play.vuejs.org/#__PROD____SSR__eNqdU01v2zAM/SuELknRzBnQbYfMCdYNPWQYlmIrdhIwODadqFUoQ5LzscD/fZTcpN6WpsAOBizxUXzke9yL66pK1jWKkUhdblXlwaGvK9AZLcZSeCfFRJJaVcZ62IPFEgZQYKkIr92O8k+GQ4TkB7DcFTbzOKMfyqm5xs7NtNAIDZTWrKDH5XqSJOWGnIcv2a/djBDGJ1/t7yUBaJMVaEfQv4DxBFoy/V4yfMwNDfQuBgH5WHH0D5n+haSGv27Zu43577Kce7Zs6PhQMx22o+VB8sHjqtKM4RNAemh/5JZmwwMvM+1QiuFTNLAMx3TYyRQDloY7KdUiuXeGWL9IWYqce1Aa7azyijuVYgQxEmKZ1mbzOd55W2OkHnOWmD+cuL9323Anxa1Fh3bNxI4xn9kF+jZ88/0rbvn/GFyZotaMPhP8hs7oOnBsYR9r4pHZDi6yncapK1rcuZutR3KHpgLRgGwiXgoWI8j3XOtPdK+SNzGPteEpdiz0wg60Prm1pnJpfDYoNoK5MRozFraZBL3PaV2oNaxfqZLfDLnx1cAucIiAISNOKN3x298cY5k/trO7Ze8Pbs9NTZ69zoD+65dZ8s/BeZypC5gj2wqhJvYBb4hTa4TNEgmm5NGyNfIw8dk8msRCaexxr/3Smg1kBGitsQmXm9feG4IPuVb5A88ikru8lELSZL9vqTZNOmxxk+fn8pNrBT/wUK6St8k70fwGbwKnOA==
Steps to reproduce

Run the provided link in SSR mode
Observe the error: Uncaught (in promise) TypeError: Failed to execute 'observe' on 'IntersectionObserver': parameter 1 is not of type 'Element'.

What is expected?
The error is handled.
What is actually happening?
If there are other lazy-hydrated components in the same view, they may become unresponsive.
I was not able to reproduce the unresponsiveness of the sibling component in StackBlitz, but it breaks in our project where we have a chain of nested lazily-hydrated components, along with some render function components in the middle.
System Info
No response
Any additional comments?
No response

PR welcome!


ReceptionStatus.subject_message_id in case of INVALID_DATA

It is possible that a message cannot be read due to various reasons. The S2 standard defines that a ReceptionStatus should be send where the status is set to INVALID_DATA: https://github.com/flexiblepower/s2-ws-json/wiki/Common_messages#receptionstatusvalues
In that case, a message_id is impossible to retrieve.
However, in ReceptionStatus the subject_message_id field is required (https://github.com/flexiblepower/s2-ws-json/wiki/Common_messages#receptionstatus). What should we define as the subject_message_id if the message_id is not retrievable and the ReceptionStatus.status is INVALID_DATA?

For now we are using '00000000-0000-0000-0000-000000000000' in s2-python


GPT testing framework

Let's say you want to somehow rigorously test (or constrain) the behavior of an AI system you've built. In deterministic systems we use testing frameworks / TDD. But with the dynamism of AI outputs it's not so obvious.
It would be fun to build a GPT testing framework into this repo. Such a framework might support a few varieties of testing:

Discrete output testing. User wants to test a GPTJSON interaction where GPT is making a discrete/deterministic decision as part of its JSON output. E.g., classifying sentiment into [negative, neutral, positive]. This seems pretty straightforward to build and isn't doing anything new except maybe needs some clever design and prompt caching/diffing.

Statistical signature testing. User wants to test that GPT interaction outputs satisfy some statistic, such as min/max # sentences. Does or does not contain certain keywords. Etc. This is straightforward as well, but by considering the usecases/statistics a framework could make it really easy to write such tests.

Semantic distance testing. This is a more experimental idea. Suppose the user wants to make sure GPT outputs ARE "close enough" to a gold_output, or ARE NOT "close enough" to a bad_output. E.g., a math tutor shouldn't give away answers. Technically, this could be implemented as a text classifier applied to outputs, possibly using GPT under the hood 🐍.

@craterink I like this a lot and have felt a similar need during programatic / CI testing of models. How do you see this overlapping with something like the openai https://github.com/openai/evals?
My thoughts at this time is that a testing framework doesn't fit squarely into the focused intent of gpt-json as a library to help with model querying. I wouldn't want users that are looking for a prompting module to be forced to bring in a whole lot of testing dependencies (that with them would probably have to bring in things like scikit, numpy, etc).
However these needs are commonly enough bundled that it could make sense to include this kind of testing framework in this project as a sub-dependency. I'm thinking for now we can start with a new gpt-testing-harness or similar folder in the main repository root that can define its own pyproject.toml file, and once mature enough hook it up to CI and deploy as an independent python package.
Does that sound reasonable?

That sounds groovy, and reasonable. Let's do it!


Help Switch from Arduino sketch to Node

I'm getting data in the read buffer but it is not the correct data and it is not same data consistently from read to read. I am reading data from an AD4116 https://www.analog.com/en/products/ad4116.html, the chip is on AD's evaluation board https://www.analog.com/en/design-center/evaluation-hardware-and-software/evaluation-boards-kits/EVAL-AD4116.html. I can read data using both an Arduino Uno R3 and AD's Evaluation Controller Board EVAL-SDP-CB1Z.
Here is an Arduino sketch that works:
#include <SPI.h>

#define DATAOUT 11 //COPI
#define DATAIN  12 //CIPO
#define SPICLOCK  13 //sck
#define CHIPSELECT05 5 //cs

void setup() {
    pinMode(DATAOUT, OUTPUT);
    pinMode(DATAIN, INPUT);
    pinMode(SPICLOCK,OUTPUT);
    pinMode(CHIPSELECT05,OUTPUT);
    digitalWrite(CHIPSELECT05,HIGH); //disable device
}

void loop() {
    pinMode(CHIPSELECT05,OUTPUT);

    delay(50);

    SPI.begin();
    SPI.setBitOrder(MSBFIRST);
    SPI.setDataMode(SPI_MODE3);

    digitalWrite(CHIPSELECT05,LOW); //enable device

    SPI.transfer(B01000111);
    unsigned int n1 = SPI.transfer16(0x00);

    digitalWrite(CHIPSELECT05,HIGH); //disable device
}

Here is a JavaScript that dosen't works:
try {

    // declare it
    var spi = global.get("SPI");

    // The AD4116 is on bus 0 and it's device 0
    const ad4116_ID = spi.open(0, 1, err => {

        const options = {
            mode: spi.MODE3,
            // The semantics of chipSelectHigh have changed in kernel 5 and higher
            //chipSelectHigh: false,
            lsbFirst: false, // pi only supports msb first
            threeWire: false,
            loopback: false, // pi doesn't support loopback
            noChipSelect: false,
            ready: false, // pi doesn't support ready
            bitsPerWord: 8, // pi only supports 8 bit words
            maxSpeedHz: 200
        };

        ad4116_ID.setOptions(options, err =>{
            if (err) {
                node.error(err)
                return msg;
            };
        });

        // An SPI message is an array of one or more read+write transfers
        const message = [{
            sendBuffer: Buffer.from([71]), // Sent to read ID
            receiveBuffer: Buffer.alloc(2),  // Raw data read
            byteLength: 1,
            speedHz: 200 // Use a low bus speed to get a good reading
        }];

        if (err) {
            node.error(err)
            return msg;
        };

        ad4116_ID.transfer(message, (err, message) => {
            if (err) {
                node.error(err)
                return msg;
            };
        });
        msg.payload = message
    });

    ad4116_ID.close
}
catch (exception_var) {
    msg.stopped = 4;
    node.error(exception_var);
}

return msg;

I'm sorry, but it looks like nobody was able to help out. There doesn't appear to be an problem with spi-device here so I'll go ahead and close the issue.


Config: Increase size limit of multiversion debug binaries

Debug builds are more than 2x larger than ReleaseSafe builds.

If we do this, we should also bump const past_count: u32 = @min(3, header.past.count); back to 4.

Yes, though not immediately, otherwise we will break auto-upgrading.

Ahhh, right, we should do that once the next version is the oldest! Actually, even one version after that, as it will cease to be the oldest if we increase the limit 🤣

Hm, I am not feeling great about this. With our current parameters, this'll mean that debug binaries would allocate an extra half a gig right of the bat for multi versioning (as opposed to current 128). This feels like a bit too much.
That is, while the multiplier of 8 does feel true, it seems like we should play with some parameters elsewhere to afford it.

If we relax the latest-version-always-execs-first rule, we could lower the buffer required to the size of a single release's maximum (plus metadata).
We could handle on-disk format changes (say, we introduced compression) with a visits release.

I'm going to close this for now: the plan is to skip copying the whole binary into memory, and only copying a particular release into memory (on Linux, it's on disk on Mac / Windows) when needed to execute it.
That is, our memory usage will be bounded by the size of an individual release, not the binary as a whole.


Branch26

Codecov Report

Merging #30 (78bc97c) into master (3cdaaed) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##            master       #30   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           15        15
  Lines          792       792
  Branches       259       259
=========================================
  Hits           792       792

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Use dotenv with Gunicorn

When I put the
import dotenv
dotenv.read_dotenv()

on manage.py it runs ok, but when I wan to use Gunicorn to production and put it in wsgi.py, it did not read the .env file. Any idea?

read_dotenv() accepts an optional path to a file. You could do something like:
dotenv.read_dotenv(os.path.join(os.path.dirname(os.path.dirname(__file__)), '.env'))

It works! But why it didn't worked on the first time? I think if it do not find any .env it will backtrack to its parent directory.


ModuleNotFoundError: No module named 'corpusreader'

Hello, when I run this code, I get the following error, how do I go about downloading the “corpusreader”?Thank you for your help.
bash train_gen.sh

Sorry, I found out first because I didn't run the following code
git clone https://gitlab.com/webnlg/corpus-reader.git corpusreader


Support and document having multiple versions of Reason installed.

People will have many projects locally, each potentially requiring a different version of the Reason, and a different version of the compiler.

[ ] We need to document how to do this and how to support it.
[ ] We need to make it so that we don't rely on a global installation of Reason in that case.
[ ] This is blocked on figuring out installation/package management story.

This will help us upgrade without breaking people (we will be aggressively upgrading the syntax). Not everyone will upgrade every project at once.
Imagine that we fix/improve syntax and when upgrading to a new version of Reason, some of the old code doesn't parse. We have a couple of options:

Know that when a file was written with v0 (maybe it's encoded at the top of the file?), we have Reason v1 capable of parsing Reason v0 as well.
Of course we will have a simple upgrade script (see the current hacky ./upgradeSyntax.sh), but that requires that you supply the paths to both versions. It would be better if Reason v1 somehow included Reason v0 as well.

Are there any problems I'm not thinking about yet?

with bs-platform & esy, this is done.


Any intentions for matching arbitrary-length parameters?

Hi, I am currently working on a project that requires a pattern that matches any enodes / eclasses that has a certain operator, regardless of the number of parameters. In particular, I want to have a pattern that matches all of the following:
(ListValues 1)
(ListValues 1 2 3)
(ListValues 1 5 6 7 8)
(ListValues 1 2 ... 100)
...

Does Egg currently offer native support for matching these things? Thank you!

Right now, egg can support terms of variable length, but you cannot write patterns to match variable length. You'd have to write one pattern for every length you care about. I do not presently plan to support variadic patterns.
In this specific case, you could try using a cons-list instead
(Cons 1 Nil)
(Cons 1 (Cons 2 Nil))
(Cons 1 (Cons 2 rest))


Terms Aggregation KeyAsString is always null

NEST/Elasticsearch.Net version:2.4.6
Elasticsearch version:
"version" : {
"number" : "2.4.1",
"build_hash" : "c67dc32e24162035d18d6fe1e952c4cbcbe79d16",
"build_timestamp" : "2016-09-27T18:57:55Z",
"build_snapshot" : false,
"lucene_version" : "5.5.2"
},
Description of the problem including expected versus actual behavior:
Steps to reproduce:

run Search with Terms aggregation on one of the object properties of type Guid
for example object like : {name: 'SOME NAME' guidVal: '034a7b09-9fee-4f63-a87f-4592edd9560d'}

var aggResponse =  client.Search<TestClass>(s=>
s.Aggregations(a => a.Terms("distinc_bus", ad => ad.Field(t => t.guidVal))));

List<Guid> listOfGuids = aggResponse.Aggs.Terms("distinc_bus").Buckets.Select(b => Guid.Parse(b.Key)).ToList(); //this line will throw as key is not returned as valid Guid

expected result : you get bucket that the bucket key ,or at least the KeyAsString  is the same as the guidVal (i.e. '034a7b09-9fee-4f63-a87f-4592edd9560d').
actual result : the bucket key is only the last token of the Guid : "4592edd9560d", and the KeyAsString is null (the value returned fine when reading a single document with Get() or multiple with Search() with no aggregations , so it is saved fine on the Elastic search itself)

Amir,
I think the issue you are having is because your guidVal property is being
analyzed which results in the quid fragments being used by the terms
aggregation.
On Sat, Nov 19, 2016 at 9:34 AM, Amir Sasson<EMAIL_ADDRESS>wrote:

NEST/Elasticsearch.Net version:2.4.6
Elasticsearch version:
"version" : {
"number" : "2.4.1",
"build_hash" : "c67dc32e24162035d18d6fe1e952c4cbcbe79d16",
"build_timestamp" : "2016-09-27T18:57:55Z",
"build_snapshot" : false,
"lucene_version" : "5.5.2"
},
Description of the problem including expected versus actual behavior:
Steps to reproduce:

run Search with Terms aggregation on one of the object properties
of type Guid
for example object like : {name: 'SOME NAME' guidVal:
'034a7b09-9fee-4f63-a87f-4592edd9560d'}

var aggResponse = client.Search(s=>
s.Aggregations(a => a.Terms("distinc_bus", ad => ad.Field(t =>
t.guidVal))));
aggResponse .Aggs.Terms("distinc_bus").Buckets.Select(b =>
Guid.Parse(b.Key)).ToList(); //this line will throw as key is not returned
as valid Guid

expected result : you get bucket that the bucket key ,or at least
the KeyAsString is the same as the guidVal (i.e. '034a7b09-9fee-4f63-a87f-
4592edd9560d').
actual result : the bucket is only the last token of the Guid :
"4592edd9560d", and the KeyAsString is null

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/elastic/elasticsearch-net/issues/2393, or mute the
thread
https://github.com/notifications/unsubscribe-auth/ABy19V-A4zX6WeEQVNVbktXf32hHX_1Hks5q_xcTgaJpZM4K3SD0
.

--
Doug Nelson

How can the analyzer be disabled for this aggregation only?
On Nov 19, 2016 19:07, "djnelson9715"<EMAIL_ADDRESS>wrote:

Amir,
I think the issue you are having is because your guidVal property is being
analyzed which results in the quid fragments being used by the terms
aggregation.
On Sat, Nov 19, 2016 at 9:34 AM, Amir Sasson<EMAIL_ADDRESS>wrote:

NEST/Elasticsearch.Net version:2.4.6
Elasticsearch version:
"version" : {
"number" : "2.4.1",
"build_hash" : "c67dc32e24162035d18d6fe1e952c4cbcbe79d16",
"build_timestamp" : "2016-09-27T18:57:55Z",
"build_snapshot" : false,
"lucene_version" : "5.5.2"
},
Description of the problem including expected versus actual behavior:
Steps to reproduce:

run Search with Terms aggregation on one of the object properties
of type Guid
for example object like : {name: 'SOME NAME' guidVal:
'034a7b09-9fee-4f63-a87f-4592edd9560d'}

var aggResponse = client.Search(s=>
s.Aggregations(a => a.Terms("distinc_bus", ad => ad.Field(t =>
t.guidVal))));
aggResponse .Aggs.Terms("distinc_bus").Buckets.Select(b =>
Guid.Parse(b.Key)).ToList(); //this line will throw as key is not
returned
as valid Guid

expected result : you get bucket that the bucket key ,or at least
the KeyAsString is the same as the guidVal (i.e.
'034a7b09-9fee-4f63-a87f-
4592edd9560d').
actual result : the bucket is only the last token of the Guid :
"4592edd9560d", and the KeyAsString is null

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/elastic/elasticsearch-net/issues/2393, or mute the
thread
<https://github.com/notifications/unsubscribe-auth/ABy19V-
A4zX6WeEQVNVbktXf32hHX_1Hks5q_xcTgaJpZM4K3SD0>
.

--
Doug Nelson
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/elastic/elasticsearch-net/issues/2393#issuecomment-261725829,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AKxwHKo5Q1cS9Q-PNBojPbTUCya9VNaMks5q_yztgaJpZM4K3SD0
.

if look at the mapping for this type,
yourindex/_mapping
        "properties": {
           "Id": {
              "type": "string",
              "index": "not_analyzed"
           },

you need to add the index setting.  This is very good practice to use
non-analyzed fields for terms aggregations.
On Sat, Nov 19, 2016 at 11:15 AM, Amir Sasson<EMAIL_ADDRESS>wrote:

How can the analyzer be disabled for this aggregation only?
On Nov 19, 2016 19:07, "djnelson9715"<EMAIL_ADDRESS>wrote:

Amir,
I think the issue you are having is because your guidVal property is
being
analyzed which results in the quid fragments being used by the terms
aggregation.
On Sat, Nov 19, 2016 at 9:34 AM, Amir Sasson<EMAIL_ADDRESS>wrote:

NEST/Elasticsearch.Net version:2.4.6
Elasticsearch version:
"version" : {
"number" : "2.4.1",
"build_hash" : "c67dc32e24162035d18d6fe1e952c4cbcbe79d16",
"build_timestamp" : "2016-09-27T18:57:55Z",
"build_snapshot" : false,
"lucene_version" : "5.5.2"
},
Description of the problem including expected versus actual behavior:
Steps to reproduce:

run Search with Terms aggregation on one of the object properties
of type Guid
for example object like : {name: 'SOME NAME' guidVal:
'034a7b09-9fee-4f63-a87f-4592edd9560d'}

var aggResponse = client.Search(s=>
s.Aggregations(a => a.Terms("distinc_bus", ad => ad.Field(t =>
t.guidVal))));
aggResponse .Aggs.Terms("distinc_bus").Buckets.Select(b =>
Guid.Parse(b.Key)).ToList(); //this line will throw as key is not
returned
as valid Guid

expected result : you get bucket that the bucket key ,or at least
the KeyAsString is the same as the guidVal (i.e.
'034a7b09-9fee-4f63-a87f-
4592edd9560d').
actual result : the bucket is only the last token of the Guid :
"4592edd9560d", and the KeyAsString is null

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/elastic/elasticsearch-net/issues/2393, or mute
the
thread
<https://github.com/notifications/unsubscribe-auth/ABy19V-
A4zX6WeEQVNVbktXf32hHX_1Hks5q_xcTgaJpZM4K3SD0>
.

--
Doug Nelson
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<https://github.com/elastic/elasticsearch-net/issues/2393#
issuecomment-261725829>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AKxwHKo5Q1cS9Q-
PNBojPbTUCya9VNaMks5q_yztgaJpZM4K3SD0>
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/elastic/elasticsearch-net/issues/2393#issuecomment-261726326,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABy19WpTr9LpAKUMDji4152o-T1Fr_Neks5q_y7MgaJpZM4K3SD0
.

--
Doug Nelson

The problem as @djnelson9715  has pointed out, is that the Guid field guidVal has been mapped (either explicitly or dynamically inferred by Elasticsearch) as an analyzed string field. This can be seen with a simple example
void Main()
{
    var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
    var connectionSettings = new ConnectionSettings(pool);
    var client = new ElasticClient(connectionSettings);

    var indexName = "github-issue-2393";

    client.Index(new TestClass
    {
        guidVal = new Guid("034a7b09-9fee-4f63-a87f-4592edd9560d")
    }, i => i.Refresh().Index(indexName));

    var aggResponse = client.Search<TestClass>(s => s
        .Index(indexName)
        .Aggregations(a => a
            .Terms("distinc_bus", ad => ad
                .Field(t => t.guidVal)
            )
        )
    );
}

public class TestClass
{
    public Guid guidVal { get; set; }
}

This returns the following json for the search response
{
  "took" : 41,
  "timed_out" : false,
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 1.0,
    "hits" : [ {
      "_index" : "github-issue-2393",
      "_type" : "testclass",
      "_id" : "AVi0L-rWLtlOUI4Y790E",
      "_score" : 1.0,
      "_source" : {
        "guidVal" : "034a7b09-9fee-4f63-a87f-4592edd9560d"
      }
    } ]
  },
  "aggregations" : {
    "distinc_bus" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [ {
        "key" : "034a7b09",
        "doc_count" : 1
      }, {
        "key" : "4592edd9560d",
        "doc_count" : 1
      }, {
        "key" : "4f63",
        "doc_count" : 1
      }, {
        "key" : "9fee",
        "doc_count" : 1
      }, {
        "key" : "a87f",
        "doc_count" : 1
      } ]
    }
  }
}

You can see that each key is actually a subsection of the guid sent in. If you check out the mapping with
var mappingResponse = client.GetMapping<TestClass>(m => m.Index(indexName));

You'll see that the field has been set up as an analyzed string
{
  "github-issue-2393" : {
    "mappings" : {
      "testclass" : {
        "properties" : {
          "guidVal" : {
            "type" : "string"
          }
        }
      }
    }
  }
}

To index it as a not_analyzed string field, you can map it as such with
client.CreateIndex(indexName, c => c
    .Mappings(m => m
        .Map<TestClass>(mm => mm
            // let NEST infer the Elasticsearch types from the POCO type...
            .AutoMap()
            // ...now override any automappings that we want to explicitly set
            .Properties(p => p
                .String(s => s
                    .Name(n => n.guidVal)
                    .NotAnalyzed()
                )
            )
        )
    )
);

You'll need to create a new index to do this and reindex your documents into it. You can use the Reindex API to do this; construct the destination index first with the above mapping. If you need both an analyzed and not_analyzed form of guidVal, then take a look at multi_fields.
Hope that helps!

Meant to add that .KeyAsString maps to the value returned by Elasticsearch in the "key_as_string" property of a bucket; for some keys such as dates, Elasticsearch returns a "key_as_string" property as well as the numerical epoch time as the key. In NEST, we always read the "key" as a string, but #2336 is an issue to look at supporting any returned json value.


[Feature] PIP 23: Message Tracing By Interceptors

Motivation
https://github.com/apache/pulsar/wiki/PIP-23%3A-Message-Tracing-By-Interceptors
Implementation
The java implementation for referring: https://github.com/apache/pulsar/pull/2471

All features from PIP 23 have been implemented.


Fix Typo

Contributor Checklist

[x] Review Contribution Guidelines
[x] Sign Gradle CLA
[x] Check "Allow edit from maintainers" option in pull request so that additional changes can be pushed by Gradle team
[x] (n/a) Provide integration tests (under <subproject>/src/integTest) to verify changes from a user perspective
[x] (n/a) Provide unit tests (under <subproject>/src/test) to verify logic
[x] Update User Guide, DSL Reference, and Javadoc for public-facing changes
[x] Ensure that tests pass locally: ./gradlew <changed-subproject>:check

Gradle Core Team Checklist

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation
[ ] Recognize contributor in release notes

Thanks for your contribution!

Thank the Hacktoberfest, as I have little time currently I searched for a quick way to finish the challenge. :-D


Question about the shift_net.py' training

It's really nice work. However,  I met some problems when  I read the shift_net.py. As described follows:
ims = self.inputs[i]['ims']
samples_ex = self.inputs[i]['samples']
assert pr.both_examples
assert not pr.small_augment
labels = tf.random_uniform(
[shape(ims, 0)], 0, 2, dtype = tf.int64, name = 'labels_sample')
samples0 = tf.where(tf.equal(labels, 1), samples_ex[:, 1], samples_ex[:, 0])
samples1 = tf.where(tf.equal(labels, 0), samples_ex[:, 1], samples_ex[:, 0])
labels1 = 1 - labels
      net0 = make_net(ims, samples0, pr, reuse = reuse, train = self.is_training)
      net1 = make_net(None, samples1, pr, im_net = net0.im_net, reuse = True, train = self.is_training)
      labels = tf.concat([labels, labels1], 0).

My understanding is that the samples_ex is the stereo audio with the size of batch_size X N X 2(N is the length of the audio signal).   However， why is the labels is  variable ?  Should it be constant （means 0 denotes not synchronized and 1 denotes synchronized) ?   I'am looking for your reply.

@andrewowens thanks for reply! However, I met some other problems. In the shift_dest.py
feats['im_0'] = tf.FixedLenFeature([], dtype=tf.string)
feats['im_1'] = tf.FixedLenFeature([], dtype=tf.string)
one: what's is stored in the 'im__0' and 'im_1' ?
two:It is the output of tf.gfile.FastGFile function ?
three: the 'im_0' include the first half frames of the video and the 'im_1' include the second half frames of the video
four: If three is true,why divide a video into two parts?
I'am looking for your reply.

The same question. Looking forward to a sample code of generating shift dataset.


state: make error messages related to unknown filed more clear

Motivation
Now, in current architecture
skbCDocWorkspaceKindReseller.PutQName(state.Field_IsSingleton, qNameCDocWorkspaceKindReseller) leads to error:
*****: [projectors.(*asyncActualizer).Run:98]: air.FillUPProfile [6] wsid[140737488487146] offset[83]: extension panic: IsSingleton undefined

Problem (warm): the error message does not describe the actual problem. I do not understand what happened.

Functional design
error message like:
extension panic: air.Reseller: IsSingleton [state|system] field value must be bool

Error messages are now like undefined QName field: IsSingleton since #2063.
The way to show error message proposed by you requires to keep fieldName-dataType mapping and verify it on every key builder operation which decreases perfomance. I think we can deal with current messages.


Added docker_network wrapper.

Included a docker_network type wrapper for hiera.
Also improved the README regarding hiera usage because:
docker::images:
  ubuntu:
    image_tag: 'precise'

produces an error: hiera::images: produces Error: Must pass images to Class[Docker::Images]

Thanks. Appreciated. I noticed the images helper is plural, but the network one is singular. It would be useful to match that up, so docker::networks rather than docker::network. If you could make that quick change I'll get this merged in. Thanks in particular for the documentation updates too.

It's rewritten in plural, didn't notice it myself either.

 Thanks again for the work on adding this.


Log rotation using newsyslog

Hi,
I was notice that when I use newsyslog rotation options eye does not know the log file is rotated and keep writing on old log file. what type of signal should I send to eye to make eye know that the log file has been rotated and update the log output.

how you use it? example of config.

Here are the newsyslog config I used for eye
[system@dnsvgitnpm ~]$ more /etc/newsyslog.conf.d/eye.conf
#logfilename                       [owner:group] mode count size   when  flags [/pid_file] [sig_num]
/var/log/dnsvault/eye/*.log        system:wheel  644  7     *      $D0   GN

and here are my eye configs
Eye.config do
  logger '/var/log/dnsvault/eye/eye.log'
end

Eye.application 'services' do
  check :cpu, every: 10.seconds, below: 100, times: 3 # global check for all processes

    working_dir "/root/"
    stdall "/var/log/dnsvault/eye/trash.log" # stdout,err logs for processes by default
    env 'APP_ENV' => 'development' # global env for each processes
    trigger :flapping, times: 10, within: 1.minute, retry_in: 5.minutes
    chain grace: 10.seconds # chained start-restart with 10s interval, one by one.
    process :tincd do
       pid_file "/var/run/tinc.dnsvnet.pid"
       start_command "service tincd onestart"
       stop_command "service tincd onestop"
       restart_command "service tincd onerestart"
       stdall "/var/log/dnsvault/eye/tincd.log"
       start_timeout 60.seconds
    end
    process :avahi_daemon do
       pid_file "/var/run/avahi-daemon/pid"
       start_command "service avahi-daemon onestart"
       stop_command "service avahi-daemon onestop"
       restart_command "service avahi-daemon onerestart"
       stdall "/var/log/dnsvault/eye/avahi-daemon.log"
       start_timeout 60.seconds
       depend_on [:tincd, :avahi_autoipd]
    end
    process :avahi_autoipd do
       pid_file "/var/run/avahi-autoipd.tap0.pid"
       start_command "/usr/local/sbin/avahi-autoipd -w -D tap0"
       stop_command "/usr/local/sbin/avahi-autoipd -k tap0"
       stdall "/var/log/dnsvault/eye/avahi-autoipd.log"
       start_timeout 60.seconds
       depend_on :tincd
    end
    process :netdata do
       pid_file "/var/run/netdata/netdata.pid"
       start_command "service netdata start"
       stop_command "service netdata stop"
       restart_command "service netdata restart"
       stdall "/var/log/dnsvault/eye/netdata.log"
       start_timeout 60.seconds
    end
    process :postgresql do
       pid_file "/usr/home/pgsql/data/postmaster.pid"
       start_command "service postgresql start"
       stop_command "service postgresql stop"
       restart_command "service postgresql restart"
       stdall "/var/log/dnsvault/eye/postgresql.log"
       start_timeout 60.seconds
    end
    process :pgbouncer do
       pid_file "/var/run/pgbouncer/pgbouncer.pid"
       start_command "service pgbouncer start"
       stop_command "service pgbouncer stop"
       restart_command "service pgbouncer restart"
       stdall "/var/log/dnsvault/eye/pgbouncer.log"
       start_timeout 60.seconds
    end
    process :timescale_prometheus do
       pid_file "/var/run/timescale-prometheus.pid"
       start_command "/usr/home/system/dnsvault/sbin/timescale-prometheus -db-port 7997 -db-user ***** -db-password **** -db-name *****"
       stdall "/var/log/dnsvault/eye/timescale-prometheus.log"
       start_timeout 60.seconds
       daemonize true
    end
    process :nginx do
       pid_file "/var/run/nginx.pid"
       start_command "service nginx start"
       stop_command "service nginx stop"
       restart_command "service nginx restart"
       stdall "/var/log/dnsvault/eye/nginx.log"
       start_timeout 60.seconds
       depend_on [:tincd, :avahi_autoipd, :avahi_daemon]
    end
    process :redis do
       pid_file "/var/run/redis/redis.pid"
       start_command "service redis start"
       stop_command "service redis stop"
       restart_command "service redis restart"
       stdall "/var/log/dnsvault/eye/redis.log"
       start_timeout 60.seconds
    end
    process :syncthing do
       env 'HOME' => "/usr/home/system"
       working_dir "/usr/home/system/"
       uid "system"
       gid "system"
       pid_file "/var/run/syncthing.pid"
       start_command "/usr/home/syncthing/syncthing"
       stdall "/var/log/dnsvault/eye/syncthing.log"
       start_timeout 60.seconds
       daemonize true
    end
end

Eye.application 'dnsvault' do
  env 'RBENV_ROOT' => '/usr/local/bin/rbenv', 'PATH' => "/home/system/.rbenv/shims:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/home/system/bin"
  check :cpu, every: 10.seconds, below: 100, times: 3 # global check for all processes

    working_dir "/usr/home/system/api"
    stdall "log/trash.log" # stdout,err logs for processes by default
    env 'APP_ENV' => 'development' # global env for each processes
    trigger :flapping, times: 10, within: 1.minute, retry_in: 10.minutes
    chain grace: 5.seconds # chained start-restart with 5s interval, one by one.
    uid "system"
    #gid "wheel"

    process :sidekiq do |p|
      daemonize true
      pid_file "tmp/pids/sidekiq.pid"
      stdall "log/sidekiq.log"
      start_command "bundle exec sidekiq -e development"
      start_timeout 30.seconds
      stop_signals [:USR1, 0, :TERM, 10.seconds, :KILL]
      check :cpu, every: 30, below: 100, times: 5
      check :memory, every: 30, below: 300.megabytes, times: 5
    end
    process :rails do
      pid_file "tmp/pids/server.pid"
      start_command "bundle exec rails s  puma -b <IP_ADDRESS> -d"
      stop_command "kill -9 {PID}"
      #daemonize true
      #stop_signals [:QUIT, 2.seconds, :TERM, 1.seconds, :KILL]
      start_timeout 60.seconds
      stop_timeout 60.seconds
    end
    process :zone_watcher do
       pid_file "/tmp/zone_watcher.pid"
       start_command "/usr/home/system/api/bin/zone_watcher.sh"
       stdall "/var/log/dnsvault/eye/zone_watcher.log"
       stop_signals [:INT, 5.seconds, :QUIT, 2.second, :KILL]
       start_timeout 60.seconds
       daemonize true
    end
    process :settings_watcher do
       pid_file "/tmp/settings_watcher.pid"
       start_command "/usr/home/system/api/bin/settings_watcher.sh"
       stdall "/var/log/dnsvault/eye/settings_watcher.log"
       stop_signals [:INT, 5.seconds, :QUIT, 2.second, :KILL]
       start_timeout 60.seconds
       daemonize true
    end
    process :log_publisher do
       pid_file "/tmp/log_publisher.pid"
       start_command "ruby /usr/home/system/api/bin/log_publisher.rb"
       stdall "/var/log/dnsvault/eye/log_publisher.log"
       stop_signals [:INT, 5.seconds, :QUIT, 2.second, :KILL]
       start_timeout 60.seconds
       daemonize true
    end
end

Looking at the settings I dump all of the log to /var/log/dnsvault/eye for all eye related process and I used the newsyslog rotate to wildcard all file with .log and rotate the file every night. normally newsyslog will signal the process that the log file it process and the process will update reopen and close the log file to update to new rotated log file. But I dont know how to handle it using eye apart from just sending "eye restart all" command which will cause unnecessarily short downtime and maybe some data lost for critical process.

or eye have its own logging facility? which can do rotation too?

i not familiar with newsyslog, but rotating utils should have option to trunc log file, instead of move it, so no need to do anything for eye with logs. i think you should use it.

Do you mean logrotate ?

noted. It seem logrotate can do copytruncate which is better than newsyslog. closing this issue.


P.S: It's not a pure awesome repository, but I think it belongs here

It doesn't have a license. Please read the awesome manifesto.
Furthermore I don't find Quora links like "What is the difference between a process and a thread?" particularly awesome.

https://github.com/sindresorhus/awesome/blob/master/create-list.md

I'll talk with @andreis to address those issues.

It doesn't have a license

Laughs were had.

Super awesome. I love it!

https://github.com/andreis/interview/commit/a7f48fff9b65f41f3f82da2e8535c19710fee8be
I'm not sure whether there was a real need for a license, but I aim to please.
@n1trux I plan to add some improvements once the repo hits 2k stars, however the processes vs threads question comes up a lot in phone screens.

@andreis WTFPL is not a real license. Please read: https://github.com/sindresorhus/awesome/blob/master/awesome.md#choose-an-appropriate-license

Closing for lack of response.


Fix issue #103: Destroy the previous document instance

I used PDFDocumentProxy.destory() to destroy the previous document instance and terminate the worker when source of pdf has changed.
Now, it seems that every JavaScript VM instance created before can be destroyed normally.
Before

After

I don't have extensive experience creating PRs, so I'd be happy to receive any feedback you might have.

Thank you! @GuoyiCoding
The PR looks well, but is usually a good practice to link the PR with the issue. 😊


[WRITING.md] Added note about ordering of Tabs component

New or updated content

Added some extra info re: creating custom Tab components, including how to force a particular ordering of tabs.

LGTM!


Fix issue #5492: [Bug]: Regression on Anthropic due to empty content hack for Bedrock

This pull request fixes #5492.
The issue has been successfully resolved. The PR addresses the root cause by:

Making the empty content removal logic provider-specific, only applying it to Bedrock
Modifying the message serialization to pass the provider information through the pipeline
Adding comprehensive test coverage to verify the behavior works correctly for both Bedrock and Anthropic providers

The changes directly fix the reported KeyError by ensuring the 'content' key remains present for Anthropic tool calls even when empty, while maintaining the special case handling for Bedrock that was previously implemented. The test cases demonstrate that both scenarios now work as expected:

Bedrock continues to have empty content removed
Anthropic retains empty content, preventing the KeyError in litellm

The solution is minimal, targeted, and includes proper test coverage to prevent regression. This can be confidently merged as it solves the original issue while maintaining backward compatibility for Bedrock.
Automatic fix generated by OpenHands 🙌

To run this PR locally, use the following command:
docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.all-hands.dev/all-hands-ai/runtime:5c1fd87-nikolaik   --name openhands-app-5c1fd87   docker.all-hands.dev/all-hands-ai/openhands:5c1fd87

Read carefully the linked issue and this PR code changes. Find and read message.py, llm.py, codeact_agent.py, to understand how the serializer in message.py is used by client code.
Important: the serializer in message.py is a pydantic serializer, check out its annotation. You can't add parameters just like that! The current fix won't work.
Once you understand the problem, pay attention to llm.py: we use litellm to send a completion() call to many APIs. Sometimes, when things really don't work yet in litellm, we have introduced some provider-specific tweaks. See for example, in llm.py, how we use CACHE_PROMPT_SUPPORTED_MODELS and FUNCTION_CALLING_SUPPORTED_MODELS, and the use of the 'anthropic-beta' headers.
We need to implement something like that for this Bedrock-specific issue. It has to apply only to Bedrock, it's causing trouble elsewhere. Thank about what options do we have. I suggest that no matter how you define it, it needs to end up being set on the Message, in format_messages_for_llm() just like we set other features that affect the way they're serialized.

OpenHands started fixing the pr! You can monitor the progress here.

Overview of Changes and Status:
Status: UNRESOLVED
Key Points:

The proposed solution attempted to use a flag-based approach for provider-specific behavior
However, it failed to address the core technical constraint: pydantic serialization in the Message class

Remaining Issues:

The solution is incompatible with pydantic serialization
No implementation that works within pydantic's framework was provided
The handling of empty content stripping needs to be redesigned to work with pydantic

Next Steps Required:

Develop a solution that properly integrates with pydantic serialization
Determine appropriate method for controlling serialization behavior within pydantic constraints
Implement provider-specific handling in a pydantic-compatible way

The workflow to fix this issue encountered an error. Please check the workflow logs for more information.

OpenHands started fixing the pr! You can monitor the progress here.

The workflow to fix this issue encountered an error. Please check the workflow logs for more information.

@openhands-agent The unit test added in this PR is failing. We need 2 things;

fix the test
make sure that the test, or a new test, ALSO tests that providers other than bedrock do not have empty 'content' stripped

OpenHands started fixing the pr! You can monitor the progress here.

Overview:
The changes successfully resolve all reported issues by:

Making empty content stripping Bedrock-specific rather than global
Adding comprehensive test coverage for both Bedrock and non-Bedrock providers

Status: ✅ All issues have been fully resolved

Content stripping is now correctly provider-specific
Test coverage verifies proper behavior for all cases
Implementation is clean and maintainable

No remaining issues to address.

The workflow to fix this issue encountered an error. Please check the workflow logs for more information.

@openhands-agent remove the test in this PR. The implementation looks correct, we just need the correct unit tests. Test behavior for

bedrock
non-bedrock

OpenHands started fixing the pr! You can monitor the progress here.

Overview of Changes:
✓ Issue successfully resolved through targeted fixes:

serialize_model now only strips empty content for Bedrock provider
Other providers (like Anthropic) preserve empty content
Comprehensive tests added for both scenarios

Status: All issues appear to be fully resolved with no remaining concerns. The solution is properly scoped and verified through testing.

The workflow to fix this issue encountered an error. Please check the workflow logs for more information.

@xingyaoww I'm looking into fixing this on litellm (https://github.com/enyst/litellm/pull/2/files) so that we can remove all this. This PR is fixing it for us first, at the price of yet another provider-specific flag.
I'm fine with closing this PR just as well; but I'm also fine with doing it for us here, because I know that so far, we are keeping tabs on these tweaks. I'm not yet worried about losing track of what the heck is that there for. But that time will come if we keep doing it! 😅 Up to you.

@xingyaoww The litellm PR was merged.
No release yet. We should update litellm below but it doesn't contain it:

https://github.com/All-Hands-AI/OpenHands/pull/5520


Add SuccessResponsePayload to GetProjectModulesResponsePayload

#1871  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @nickgraz and the rest of your teammates on  Graphite


Add private load balancer DNS name to outputs

Per request:  "add the private load balancer DNS Name to the outputs for the aws-cloudformation-ecs-senzing-stack-basic stack"

Should be as simple as adding an output with https://${LoadBalancerPrivate.DNSName}:${TargetGroupApiServerPrivate.Port}

PR merged, release created see: https://github.com/Senzing/aws-cloudformation-ecs-senzing-stack-basic/issues/109


404 not found when connecting

 is a comment.
You will see these throughout the issue template. Make sure you do not type your comments
between these characters or we will not be able to see what you wrote!

4. Links
To create a hyperlink, type the [text you want to link](followed by the url in parenthesis)
-->
Describe the bug

I am trying to connect to using the Essentials Hue app. I get the notification, click on it. Then I get greeted by "404: Not Found".
Steps to Reproduce

No major steps.

I Installed Emulated Hue via the addon repo on HomeAssistant OS.
Started the service,
tried to connect using Essentials Hue app
Opened Notification link and that was it.

Expected behavior

I was expecting "connected" or something along those lines.
Logs

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 00-set-vars.sh: executing...
[cont-init.d] 00-set-vars.sh: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[03:26:19] INFO: Starting Emulated Hue...
2022-03-20 03:26:22,505 INFO  emulated_hue.config -- Auto detected listen IP address is <IP_ADDRESS>
2022-03-20 03:26:22,650 INFO  hass_client -- Connected to Home Assistant ws (version 2022.3.5)
2022-03-20 03:26:23,287 ERROR emulated_hue.api -- Failed to create HTTP server at port 80: [Errno 98] error while attempting to bind on address ('<IP_ADDRESS>', 80): address already in use
2022-03-20 03:26:23,697 INFO  emulated_hue.api -- Started HTTPS webserver on port 443

Hardware Info:

Architecture (Ex: x86, ARM): x86
Addon version (Ex: stable or dev):  Stable

Value of "tag_commit_or_branch" if using dev: n/a

OS (Ex: Windows 10, Ubuntu Buster): HomeAssistantOS
Hardware (Ex: PC, Raspberry Pi 1/2/3/Zero):  PC

Additional context

I have restarted homeassistant and rebooted the system itself. Nothing seems to help. It's like something isn't being installed during installation.

Something is already using port 80 as it says in the logs.

Well dur..  I missed that completely last night. Guess I'll do digging and see if i can't figure out what.  It's showing the server is using port 80. Is there something i can do to make this work?

You'll need to figure out what's using it. You can change the port from 80 to something else but a lot of apps require port 80 and 443

using: netstat -lp | grep :80, i get:
tcp        0      0 <IP_ADDRESS>:80            <IP_ADDRESS>:*               LISTEN      -

I can't find anything that is using the port. Any suggestions would be nice. If not, I guess close the issue since it's on my end. Thanks

You may want to try the following suggestions here. https://unix.stackexchange.com/questions/106561/finding-the-pid-of-the-process-using-a-specific-port
You can run it on a separate host if you prefer. Another option is you can using a reverse proxy on your home assistant machine as well to proxy to the emulated hue while still allowing access to home assistant (if it's served on port 80). I personally use it with a reverse proxy.

Hey I appreciate the help. I did find that resource earlier today but, it didn't work for me. Earlier I was thinking about what I did last night and I realized I didn't include a step. I did install the stable version, uninstall and then installed the dev and I was getting the same issues and reverted back to stable. So when uninstalling it's like it never gave up port 80 originally. It was late at night when playing with it and completely forgot I had.
I gave up and running it in my HomeAssistant and put it in a docker instance on a macvlan so it has it's own network and can use the port 80 without interruption. I used the following:
  hass-emulated-hue:
    container_name: emulated_hue
    networks:
      default:
       ipv4_address: <IP_ADDRESS>
    environment:
      - HASS_TOKEN=xxxxxx
      - HASS_URL=http://<IP_ADDRESS>:8123
    image: ghcr.io/hass-emulated-hue/core:refactor

Now everything is working as advertised. Last question. Can it connect to the Hue Sync app by chance or is it using the wrong API currently?
I appreciate the awesome software.


go get fails: fatal error: 'openssl/sha.h' file not found

$ go get github.com/ctz/go-fastpbkdf2
# github.com/ctz/go-fastpbkdf2
../Development/go/src/github.com/ctz/go-fastpbkdf2/fastpbkdf2.c:20:10: fatal error: 'openssl/sha.h' file not found

what ur platform?
when ur using linux. use libssl

Hi, same problem, windows 10
C:\Go\src>go get "github.com/ctz/go-fastpbkdf2"
github.com/ctz/go-fastpbkdf2
fastpbkdf2.c:20:10: fatal error: openssl/sha.h: No such file or directory
#include <openssl/sha.h>

Yes, this needs openssl to be installed and available on the system you're building on. That'll make using it on Windows quite challenging.

If you want to run on windows 10 i suggest you change the program to use
import "golang.org/x/crypto/pbkdf2"
instead of "github.com/ctz/go-fastpbkdf2"
Mines running fine now after compiling with the "golang.org/x/crypto/pbkdf2"

With regards to the error
fastpbkdf2.c:20:10: fatal error: 'openssl/sha.h' file not found
I had to install openssl and was still getting the above error.
then install libssl-dev if your still having issues
sudo apt-get install libssl-dev
This worked for me running on my Raspberry


Is there a clean way to delete tiles, similar to wipe-env job in pcf-pipelines?

I would like to have a job in the install-[tile] pipeline, that can help me delete the tile that I've installed, so that I can delete and re-install the tile per my needs.
Requested by @jmcdice

Done and available in v-0.6.2 release


Some harvested pypi components not showing up in search

Example:
Pypi - Protobuf 3.9.0
https://clearlydefined.io/definitions/pypi/pypi/-/protobuf/3.9.0
Pypi not showing up in search results

Another example is Numpy (pypi). Versions 1.16.4, 1.17.1, and 1.17.2. These versions show up as harvested but do not show up in the workspace search.

@storrisi Can you look into this, please?


Compile error

I can't seem to compile and keep getting the error 'react-native-google-sign-in/index.js: Unexpected token, expected ; (1:20)'

That's happening because the export statement in index.js is using "Export extensions transform"
To prevent this compile error, install babel plugin and add to .bablerc: http://babeljs.io/docs/plugins/transform-export-extensions/
However after doing that, got another error :(

@pyko Thanks I will update the document.

@pyko I changed the code to module.exports = , so the babel plugin shouldn't be needed anymore.


Support for OpenNebula 4.14

This addresses issue #222 for supporting OpenNebula 4.14
Uses $one_version inside the template for oned.conf, etc, to generate config files for either 4.12 or 4.14 from same template file.
@rwaffen @tuxmea let me know what you think-

Thanks for taking a look at this. I've updated this PR, the main change is moving the versioncmp out of the templates and using a boolean flag version_gte_4_14 instead (commits 09ffeef and 3c8a96b).
The other changes you mentioned are places where I decided to stick close to the oned.conf from the 4.14 rpm, but, leaving behavior alone in the case of 4.12.  (i.e., it's a no-op except for whitespace for users currently using this puppet module on 4.12).
Also, regarding comments - the sections I separate by version are parameters that are new in 4.14, and I feel it would be difficult to denote a multi-line section as applying to one version but not another.

I would keep the default values in place (as long as they dont differ between versions.
Otherwise I see no problem with this PR.

The default values don't differ between versions, however I'm trying to stick as close as possible to the oned.conf file from the rpm - which doesn't list the defaults there (neither in 4.12 nor in 4.14). Also, the defaults are present in the params.pp file so users can see them there, and, in the rendered template, the values will be in the section below. If possible, I'd prefer to not hardcode a default value in two different places (the params and the comment).

@jnoss OK. I agree with avoiding duplications. Good to merge from my point of view.

thanks for the great work and the comments… only asking my team and than will merge …

Noone able to merge??

@tuxmea i will clear this situation


Make the main window not on top if it's set to be on top when the About window opens.

Also set the main window to be on top when the About window closes. If it's possible, see if I can turn off Always On Top for the main window but turn it on for the About window and Options window if the main window is set to always on top.

Fixed.


fix: don’t use the gorm Scopes() method, it’s behavior is more complex than we realized.

this also lets up bump the gorm version and still pass the unit tests.

Could you update the commit message and PR description with a bit more info on what happened? What did we intend, what happened instead, what's the alternative people should use?


Circular hierarchies for java.lang.Object

I am using javasrc2cpg to scan jdk8, and get StackOverflowError in
https://github.com/joernio/joern/blob/f258451c080c674da0a146d0b8fa171e0dd280b7/joern-cli/frontends/x2cpg/src/main/scala/io/joern/x2cpg/passes/callgraph/DynamicCallLinker.scala#L84-L96
Then I find out that the inheritsFromTypeNames of java.lang.Object contains itself

#3197 Will fix the circular hierarchy issue

I have test this. StackOverflowError  is gone. Thanks @DavidBakerEffendi


Audio in English curriculum doesn't play on some iOS devices

Describe the Issue
The audio in the English curriculum isn't playing on some iOS devices. Here's a forum post which first mentioned the issue: https://forum.freecodecamp.org/t/no-sound-on-ipad-iphone-a2-english-for-developers/663538
A few people on have tested and here's what they said...
It worked for some on these devices:
iOS 17.2.1 Safari
iPad 8 Safari / Chrome
iPhone 12 iOS 17.1.2 Safari / Chrome
It didn't work for some on these devices:
iPhone 12, iOS 16.6.1 Safari / Chrome 120.0.6099.119
iPhone 8 iOS 16.7.2
Seems like it might be a device settings issue.
Affected Page
https://www.freecodecamp.org/learn/a2-english-for-developers/
Expected behavior
It to play the audio.
Edit: Seems like it might be an issue with iOS. I upgraded my iPhone from 16.6.1 (didn't work on Safari or Chrome) to 17.2.1, and it works now on Safari and Chrome (same chrome version).

I can't reproduce it either on Iphone 14 Pro, neither on MacOs Sonoma


Search Feature Impl.

Search Feature Implementation. All the Blockly code that has been introduced has been marked with comments.
Abhijit Suresh (GSOC 2016)Search Feature Implementation. All the Blockly code that has been introduced has been marked with comments.
Abhijit Suresh (GSOC 2016)

@josmas Please take a look

@abhijit5893 two questions. Is this code ready to be reviewed and merged? Where did the translations come from? (Google Translate for example, or native speakers?)

@josmas  It is ready to be reviewed. The translations came from Google Translate.

@abhijit5893 just to clarify, @jisqyv was the one asking.
Have you made any of the modifications suggested in the last discussion thread? or is it the same code you have in the test instance?
Who was that you've been working with through GSoC? Have they reviewed this?

This version is the rebased version of the last discussion thread. It is the same code that I have in the test instance. I have been working with Benjamin @bxie. He has already reviewed it.

I have yet to review the absolute latest instance of the appspot. I'll look at it by tomorrow.

Reviewed code and it looks fine.
Tested instance as well. I found bugs that should be resolved but they are minor.
Bugs found:

When filter applied to previous search, functionality is unusual.
To replicate: With no filter selected, type something into search box that yields results and press enter. Apply a filter which should remove results.
Expected: filter is applied and graying of blocks updates accordingly.
Actual: No change. Filter does not appear to be applied.
Severity: Minor (S3)

Filter dropdown changes sizes.
To replicate: Check box for filter. Uncheck box.
Expected: filter dropdown doesn't change
Actual: Dropdown narrows (and widens again after clicking on it).
Severity: Trivial (S4)

@abhijit5893 How much testing did you do? These bugs are not very severe, but I do want to ensure that you rigorously tested this feature.

@bxie @abhijit5893 there were also a number of suggestions in an email thread that need to be taken care of (or at least discussed) before this could be merged.

@josmas Where is the discussion thread so I can review it?  -Jeff

Discussion Thread: https://groups.google.com/forum/#!searchin/app-inventor-dev/abhijit|sort:relevance/app-inventor-dev/_mZALZCaHmQ/CILfebaRCAAJ
What changes were made and what remains is explained at http://abhijit-suresh.herokuapp.com/
@abhijit5893 please provide more insight on what changes were made and not made.

@abhijit5893 This is a great start. However where it is most useful, large projects, it has some issues.

A search on a large project takes a long time, with no user feedback. The interface locks up.
It does too much work on the UI thread without relinquishing control. The effect is that the user is eventually faced with the browser generated "This script is taking too long" dialog. If they kill the script, it kills app inventor.

I would recommend re-factoring so that if there are more then "N" blocks (I'm not sure of what the threshold for N should be, but experimentation may reveal it) then a progress dialog (or at least a "spinner") is displayed. After every so many blocks are searched, the state of the search is checkpointed, and the search is re-queued to run again and control is returned. This will prevent the "too long" dialog from the browser. In fact it would be cool if the dialog had a cancel button on it. If this was pressed, it would set a flag that would cause the search to terminate on its next checkpoint.
-Jeff

@jisqyv Cool. Thanks for the update. Can you point me to a large project which I can use to recreate the issue. @josmas @jisqyv  @bxie Here is the link to the proposal which details what changes were made and what are pending. Also, if you would like to schedule a meeting to discuss this issue further. Please let me know. I'm bit occupied this week but can start working from the next week.

This PR will be affected by the Blockly update, so we will need to assess what needs to be done to update this based on those changes.

@jisqyv  @bxie  I dont want to leave this hanging. Please let me know what I can do to merge this feature with the current version of AI.

This would be a great feature to have

The challenge here is that there were a lot of changes to Blockly core, which we no longer keep as part of the repo but maintain as a separate repository via a submodule. We will need to factor out the changes to core where appropriate and then rebase the change onto the current master before this can be incorporated.

Also, there is like 10 conflicting files. So yeah

This makes too many changes to Blockly core. We also think it would be useful to design the appropriate data structures and associated UI/UX so that performance is acceptable. We should also consider touching base with Google about what would make sense for them (and there are competing implementations there).


compilation fails due to missing tantivy crate feature

error: enum variants on type aliases are experimental
--> /home/maciej/.cargo/git/checkouts/tantivy-cd4bdf03c5df22c3/cde9b78/src/directory/footer.rs:104:13
|
104 |             Self::V0(crc) => {
|             ^^^^^^^^^^^^^
|
= help: add #![feature(type_alias_enum_variants)] to the crate attributes to enable
I am not sure if I should report it with tantivy
Following the advice works.

No need to report it to tantivy, we're tracking tantivy master for now as the latest release is missing some required by the python bindings. I'll add the feature flag.
Thanks for the report.

Fixed in tantivy master. No need to add any feature flag because of tantivy ever. Tantivy is supposed to work on rust stable. If it is does not, either this is an issue on tantivy side or you just need to update your rustc


Free recording_device_type format

Currently, recording_device_type is limited to lena, usb, olympus, babylogger, and unknown.
However, data may be collected using other devices than those listed (e.g. phone with Whatsapp) and it would be good to let the user specify the correct value rather than forcing them to choose one that would be either erroneous or non-informative (e.g. unknown).

Do we know why it was originally implemented that way? There probably was a reason to restrict the range of accepted values.

It will probably be more interesting to extend the list rather than allow for any value. Allowing for grouping 'by device'


Update voikkospellservice to 1.0.5b2

Thank you for the contribution. It was merged directly as 9d8be83c7ffe8cb6f2f6465011236f157f98d82f to keep commit history cleaner. Your contribution is still credited to you.


"Critical dependency: the request of a dependency is an expression" in iconv-loader.js

I'm submitting a bug report
Webpack version:
2.1.0-beta.25
Please tell us about your environment:
Linux
Current behavior:
$ node_modules/.bin/webpack index.js bundle.js
Hash: d8a619eb613c35b46760
Version: webpack 2.1.0-beta.25
Time: 868ms
    Asset    Size  Chunks             Chunk Names
bundle.js  656 kB       0  [emitted]  main
  [87] ./index.js 35 bytes {0} [built]
    + 87 hidden modules

WARNING in ./~/encoding/lib/iconv-loader.js
9:12 Critical dependency: the request of a dependency is an expression

ERROR in ./~/iconv-lite/encodings/tables/gb18030-ranges.json
Module parse failed: ./node_modules/iconv-lite/encodings/tables/gb18030-ranges.json Unexpected token (1:9)
You may need an appropriate loader to handle this file type.
| {"uChars":[128,165,169,178,184,216,226,235,238,244,248,251,253,258,276,284,300,325,329,334,364,463,465,467,469,471,473,475,477,506,594,610,712,716,730,930,938,962,970,1026,1104,1106,8209,8215,8218,8222,8231,8241,8244,8246,8252,8365,8452,8454,8458,8471,8482,8556,8570,8596,8602,8713,8720,8722,8726,8731,8737,8740,8742,8748,8751,8760,8766,8777,8781,8787,8802,8808,8816,8854,8858,8870,8896,8979,9322,9372,9548,9588,9616,9622,9634,9652,9662,9672,9676,9680,9702,9735,9738,9793,9795,11906,11909,11913,11917,11928,11944,11947,11951,11956,11960,11964,11979,12284,12292,12312,12319,12330,12351,12436,12447,12535,12543,12586,12842,12850,12964,13200,13215,13218,13253,13263,13267,13270,13384,13428,13727,13839,13851,14617,14703,14801,14816,14964,15183,15471,15585,16471,16736,17208,17325,17330,17374,17623,17997,18018,18212,18218,18301,18318,18760,18811,18814,18820,18823,18844,18848,18872,19576,19620,19738,19887,40870,59244,59336,59367,59413,59417,59423,59431,59437,59443,59452,59460,59478,59493,63789,63866,63894,63976,63986,64016,64018,64021,64025,64034,64037,64042,65074,65093,65107,65112,65127,65132,65375,65510,65536],"gbChars":[0,36,38,45,50,81,89,95,96,100,103,104,105,109,126,133,148,172,175,179,208,306,307,308,309,310,311,312,313,341,428,443,544,545,558,741,742,749,750,805,819,820,7922,7924,7925,7927,7934,7943,7944,7945,7950,8062,8148,8149,8152,8164,8174,8236,8240,8262,8264,8374,8380,8381,8384,8388,8390,8392,8393,8394,8396,8401,8406,8416,8419,8424,8437,8439,8445,8482,8485,8496,8521,8603,8936,8946,9046,9050,9063,9066,9076,9092,9100,9108,9111,9113,9131,9162,9164,9218,9219,11329,11331,11334,11336,11346,11361,11363,11366,11370,11372,11375,11389,11682,11686,11687,11692,11694,11714,11716,11723,11725,11730,11736,11982,11989,12102,12336,12348,12350,12384,12393,12395,12397,12510,12553,12851,12962,12973,13738,13823,13919,13933,14080,14298,14585,14698,15583,15847,16318,16434,16438,16481,16729,17102,17122,17315,17320,17402,17418,17859,17909,17911,17915,17916,17936,17939,17961,18664,18703,18814,18962,19043,33469,33470,33471,33484,33485,33490,33497,33501,33505,33513,33520,33536,33550,37845,37921,37948,38029,38038,38064,38065,38066,38069,38075,38076,38078,39108,39109,39113,39114,39115,39116,39265,39394,189000]}
 @ ./~/iconv-lite/encodings/dbcs-data.js 99:37-76
 @ ./~/iconv-lite/encodings/index.js
 @ ./~/iconv-lite/lib/index.js
 @ ./~/encoding/lib/encoding.js
 @ ./~/node-fetch/lib/body.js
 @ ./~/node-fetch/index.js
 @ ./index.js

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem along with a gist/jsbin of your webpack configuration.

npm init
npm install --save node-fetch
npm install --save-dev<EMAIL_ADDRESS>echo "let fetch = require('node-fetch');" > index.js
node_modules/.bin/webpack index.js bundle.js

Language: ES6

You seem to be missing { test: /\.json($|\?)/, use: 'json-loader' }. Webpack is failing to parse the .json file because, if interpreted as javascript, it looks like a block (think of there being an if (true) in front of the {); even if it worked, it would export nothing.

Yeah, there's not much we can do about it in the core. You'll have to set up a loader.

Adding
loaders: [
      {test: /\.json$/, loader: 'json-loader'}
    ]

solved the problem. Thanks.

where i have to add this ?
loaders: [
{test: /.json$/, loader: 'json-loader'}
]

This is still a thing and adding the json-loader will not work because other dependencies will fail when I do this. What does work though is ignoring iconv:
plugins[
  new webpack.IgnorePlugin(/\/iconv-loader$/)
]

I wonder if this is a bug in webpack or in iconv because if it's iconv I will file an issue there. Can someone clarify this?

@LukasBombach Reading this issue, it seems that it is intentional to provide a fallback for Browserify users.

```js
plugins[
  new webpack.IgnorePlugin(/\/iconv-loader$/)
]

Could you clarify, where this should be written?

@lid3rs in your Webpack config: https://webpack.js.org/configuration/plugins/

I may be misunderstanding this issue or the solution shown above, but I got here because I see the same error in my Vue project.
I added to my vue.config.js:
const webpack = require('webpack')
module.exports = {
  configureWebpack: {
    plugins: [
      new webpack.IgnorePlugin(/\/iconv-loader$/),
    ],
   ...
}

but now I get a fatal error instead of just a warning message: Error: cannot find module ./iconv-loader, which makes sense since webpack is now ignoring it.

I was able to ignore it by adding:
 module: {     noParse: /iconv-loader\.js/,     ...


version >= 9.0.3 fails with gulp and vinyl-transform

Node 0.12.0.
    "browserify": "^9.0.8",
    "gulp": "git+https://github.com/gulpjs/gulp.git#4.0",
    "vinyl-transform": "^1.0.0"

gulpfile.js:
javascript
gulp.task('browserify', function() {
	var browserified = vinyl_transform(function(filename) {
		var b = browserify(filename, {
			standalone: pkg.name
		});
		return b.bundle();
	});

	var src = pkg.main;
	return gulp.src(src)
		.pipe(browserified)
		.pipe(header(banner, banner_options))
		.pipe(rename(builds.uncompressed))
		.pipe(gulp.dest('dist/'));
});

This just worked with previous versions of browserify. But now I get:
[15:58:42] 'browserify' errored after 68 ms
[15:58:42] TypeError: undefined is not a function
    at Producer.ondata (_stream_readable.js:540:20)
    at Producer.emit (events.js:107:17)
    at Producer.Readable.read (_stream_readable.js:373:10)
    at flow (_stream_readable.js:750:26)
    at resume_ (_stream_readable.js:730:3)
    at _stream_readable.js:717:7
    at process._tickDomainCallback (node.js:381:11)

This was an error in the gulp recipes that relied on internal behavior. (See #1191 and #1044). Please see the updated gulp guides:

Browserify + Uglify2
Browserify + Globs

Thanks a lot.