pipenv install is very slow

Running pipenv install after changing one dependency takes about ~5 minutes for me, on a windows 10 machine with an SSD.
The vast majority of that time is spent inside Locking [packages] dependencies...
It seems like there might be some quadratic-or-worse complexity in this step?
I've included most of our pipfile below, but I had to remove some of our private repo dependencies:
[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true

[packages]
alembic = "==0.8.4"
amqp = "==1.4.7"
analytics-python = "==1.2.5"
anyjson = "==0.3.3"
billiard = "==<IP_ADDRESS>"
braintree = "==3.20.0"
celery = "==3.1.18"
coverage = "==4.0.3"
docopt = "==0.4.0"
eventlet = "==0.19.0"
flake8 = "==3.0.4"
Flask-Cors = "==2.1.2"
Flask-Login = "==0.3.2"
Flask = "==0.12.1"
funcsigs = "==0.4"
fuzzywuzzy = "==0.12.0"
gcloud = "==0.14.0"
html2text = "==2016.9.19"
itsdangerous = "==0.24"
Jinja2 = "==2.8"
jsonpatch = "==1.15"
jsonschema = "==2.5.1"
PyJWT = "==1.4.2"
kombu = "==3.0.30"
LayerClient = "==0.1.9"
MarkupSafe = "==0.23"
mixpanel = "==4.3.0"
mock = "==1.3.0"
nose-exclude = "==0.4.1"
nose = "==1.3.7"
numpy = "==1.12.1"
pdfrw = "==0.3"
Pillow = "==4.1.0"
pusher = "==1.6"
pycountry = "==1.20"
pycryptodome = "==3.4.5"
pymongo = "==3.2"
PyMySQL = "==0.7.4"
python-dateutil = "<=2.5.1"
python-Levenshtein = "==0.12.0"
python-magic = "==0.4.6"
python-coveralls = "==2.9.0"
pytz = "==2015.6"
raygun4py = "==3.1.2"
"repoze.retry" = "==1.3"
requests = "==2.8.1"
sendgrid = "==2.2.1"
slacker = "==0.7.3"
SQLAlchemy-Enum34 = "==1.0.1"
SQLAlchemy-Utils = "==0.31.6"
SQLAlchemy = "==1.1.9"
typing = "==<IP_ADDRESS>"
twilio = "==5.6.0"
Unidecode = "==0.4.19"
voluptuous = "==0.8.11"
Wand = "==0.4.4"
watchdog = "==0.8.3"
Werkzeug = "==0.12.1"
wheel = "==0.24.0"
WTForms = "==2.0.2"
xmltodict = "==0.9.2"
zeep = "==0.24.0"

Hey again @Diggsey, this is due to the way we're writing changes right now. I have these changes ready to merge but they're breaking for the projects.py API so we're going to hold off until the next major release. Hopefully we'll have this up and ready in the next few weeks. I'll leave this open to track the issue for now.

We sprinted on this together at PyCon. It'll be faster soon.

Right now for me it is not slow, it is freezing...
A pipenv install my_package or a simple pipenv install does not give me any output, after 20 minutes.

Hey @NicolasWebDev, what version of pipenv are you using? Also do you have delegator.py installed on your system separately? If so, what version is that at? This was an issue that should have been resolved in v3.6.0.
If everything above is up to date, could you please provide the contents of your Pipfile? Thanks!

Hi @nateprewitt, you were right, I was on v3.5.x. Updating to 4.1.1 resolved the freezing issue. Now it is still taking 5 minutes, but at least it is usable!
Sorry for the noise, I always forget that packages installed through pip are not automatically updated.
Thanks!

Glad you got things resolve @NicolasWebDev! We're working on getting this sped up more, hopefully #373 will be a step closer in the next release.

@Diggsey @NicolasWebDev, I've just released 4.1.2 which should have these speed improvements added. There's still some work to do here, but this will at least expedite the initial bootstrap time for pipenv.

@nateprewitt Thanks for the update, pipenv does seem faster for me now, but it still takes several minutes just to run pipenv lock, even when nothing has changed - it still waits in Locking [packages] dependencies... for the vast majority of that time.

@Diggsey, a lot of that time is because you're downloading a huge number of files in that Pipfile. It seems like you're pinning all of your dependencies too. Are you directly importing all of these into your project or are some dependency requirements of the others?

@nateprewitt We could remove some of them, but the majority are direct dependencies - why does it need to download all of the dependencies very time it generates the lock file?

We need to determine everything that it installs as dependencies. In order to get that, we download each package and determine what an installation looks like at lock time. This allows us to appropriately pin everything in the Pipfile.lock. Without downloading, there isn't a reliable way of checking sub-dependencies and resolving range dependency declarations.

Given that most of the packages are going to stay the same over time, would it be possible to cache the downloaded packages?

This can be closed, isn't it? @kennethreitz

@Diggsey want to look into that for us?

This might be a silly question, but doesn’t Pip already do package caching?

I'm under the impression that it does.

Can pipenv use the pip cache system or it has to be implemented from the scratch?

Pipenv just runs pip, so the cache should automatically be used.

fixed! lock is wicked fast now.

Oh, thank you. I think that was the only thing holding my back from pushing everyone over to pipenv at work.

Wow, nice, that was literally more than a 100x speedup for me, and it also caught a dependency conflict that the previous version didn't catch!
What would be useful is a verbose flag for pipenv lock - I was only able to diagnose the dependency conflict by editing piptools/logging.py to enable verbose logging, but once I did that it gave a very clear indication of what was going on.

I'm probably missing something :) Where is it fixed? Latest release is 4 days ago, so I installed latest version from master. However, pipenv install is still slow.
I tried:

install pipenv the fancy way ⚡ 🍰 ⚡
use both latest published version of pipenv and latest version from master
install single package

Using latest stable version (5.3.5.), it takes 3:40 to install one package:
∙ time pipenv install --dev raven
Installing raven...
Collecting raven
  Using cached raven-6.1.0-py2.py3-none-any.whl
Collecting contextlib2 (from raven)
  Using cached contextlib2-0.5.5-py2.py3-none-any.whl
Installing collected packages: contextlib2, raven
Successfully installed contextlib2-0.5.5 raven-6.1.0

Adding raven to Pipfile's [dev-packages]...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
Updated Pipfile.lock!
pipenv install --dev raven  10,11s user 2,77s system 5% cpu 3:40,04 total

Using version from master, it still hangs (one package, +10 minutes)
Any ideas? Thanks a lot!

Sometimes dependencies take a while to install, especially if they have c compilations. Want to share your Pipfile?

I understand sometimes it takes a while, but it was slow from the beginning. Just curious if it's a problem at my side.
Here's my Pipfile:
[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true

[dev-packages]
pytest = "*"
pytest-django = "*"
pytest-testmon = "*"
pytest-watch = "*"
django-debug-toolbar = "*"
raven = "*"

[packages]
dj-database-url = "*"
Django = "*"
djangorestframework = "*"
gunicorn = "*"
newrelic = "*"
psycopg2 = "*"
requests = "*"
whitenoise = "*"
graphene-django = "*"

psycopg2 might be taking a while. Everything else should be nice and fast. Try removing it and see how much your speed increases.

$ pipenv install raven just took like 1s for me.

$ pipenv install raven just took like 1s for me.

That's what I would expect! Can I turn on verbose output somehow?
I tried remove psycopg2, but it doesn't affect much. Running pipenv install raven hangs for a while.
I have:

Python 3.6.2
macOS 10.12.6

i see no reason why raven shouldn't be instantaneous.
do pip install raven inside of pipenv shell and tell me if it's slow there too.

All pipenv is doing is running pip, so that's effectively the "verbose mode"

That's instant:
∙ time pip install raven                                                                                19:38  tricoder@issac
Requirement already satisfied: raven in /Users/tricoder/.envs/lingui-api-VE1OToiy/lib/python3.6/site-packages
Requirement already satisfied: contextlib2 in /Users/tricoder/.envs/lingui-api-VE1OToiy/lib/python3.6/site-packages (from raven)
noglob pip install raven  0,54s user 0,15s system 76% cpu 0,900 total

Running pipenv hangs about 2-3 minutes (inside/outside pipenv shell).
∙ time pipenv install raven                                                                                19:39  tricoder@issac
Installing raven...
Requirement already satisfied: raven in /Users/tricoder/.envs/lingui-api-VE1OToiy/lib/python3.6/site-packages
Requirement already satisfied: contextlib2 in /Users/tricoder/.envs/lingui-api-VE1OToiy/lib/python3.6/site-packages (from raven)

Adding raven to Pipfile's [packages]...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
Updated Pipfile.lock!
pipenv install raven  4,49s user 0,46s system 2% cpu 3:21,17 total

@Diggsey can you open a new issue about --verbose and provide an example of what you'd like it to look like?

@tricoder42 is the slow part the locking step or the install step? Locking was vastly improved in the latest releases.

time pipenv install raven
Installing raven...
Collecting raven
  Using cached raven-6.1.0-py2.py3-none-any.whl
Requirement already satisfied: contextlib2 in /Users/kennethreitz/.local/share/virtualenvs/pipenv-u9yqWeFK/lib/python3.6/site-packages (from raven)
Installing collected packages: raven
Successfully installed raven-6.1.0

Adding raven to Pipfile's [packages]...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
Updated Pipfile.lock!
        9.30 real         5.49 user         0.42 sys

It's like 50:50 installing:locking

@tricoder42 and you're using latest?

Let me replicate with your exact pipfile.

I guess so:
∙ pipenv --version                                                                                19:42  tricoder@issac
pipenv, version 5.3.5

∙ pipsi upgrade git+https://github.com/kennethreitz/pipenv.git#egg=pipenv                                                                                19:45  tricoder@issac
Collecting pipenv from git+https://github.com/kennethreitz/pipenv.git#egg=pipenv
  Cloning https://github.com/kennethreitz/pipenv.git to /private/var/folders/g9/1wbckv154mbby3tm411z_m340000gn/T/pip-build-se4ao5/pipenv
...
Installing collected packages: pipenv
  Found existing installation: pipenv 5.3.5
    Uninstalling pipenv-5.3.5:
      Successfully uninstalled pipenv-5.3.5
  Running setup.py install for pipenv ... done
Successfully installed pipenv-5.3.5

$ time pipenv install
No package provided, installing all dependencies.
Pipfile found at /Users/kennethreitz/pipenv/testapp/Pipfile. Considering this to be the project home.
Pipfile.lock not found, creating...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
Updated Pipfile.lock!
Installing dependencies from Pipfile.lock...
[================================] 22/22 - 00:00:37
       58.94 real        40.51 user         8.62 sys

It does even when I'm installing first package into fresh new pipenv. I'll try to create pipenv --three instead of pipenv --python python3.6

@tricoder42 want to hop on a google hangouts in like an hour?

Or we can use apple screen sharing if you use Messages.app.

I'm about to join a meeting, but I'll be available after that.

Cool! I'll try clean & reinstall everything from the scratch and we'll see. I'm available in one hour

Awesome — we'll figure this out then. Add me on messages.app :)

If anyone is experiencing extremely slow Locking behavior with v11.9.0, i found that downgrading to v9.0.0 takes a 5m30s install down to 1m36s.

@ryantuck I'd recommend if you're going to pin an old version to pin 9.0.3 but you lose a significant amount of added security for the speed, you might as well just use --skip-lock at that point

--skip-lock definitely sped things up. I went down this path because pipenv install --system --python=3.6 wasn't actually installing to the correct system python, and i was under the assumption that i needed to generate the Pipfile.lock before attempting the system install. It still didn't work, so I went back to using regular old pip, but will comment back on this thread if I ever make any progress along these lines.

—system and —python are mutually exclusive — the latter option always needs a virtualenv

It should be a little bit faster the second or third time because of
caching tho. Do you see an improvement?
On Thu, Apr 12, 2018 at 10:23 AM Alexander Kavanaugh <
<EMAIL_ADDRESS>wrote:

yep, locking takes a while for me too. v11.10.0. Ubuntu on WSL.
[[source]]url = "https://pypi.python.org/simple"verify_ssl = truename = "pypi"
[packages]babel = "==2.5.3""boto3" = "==1.7.3"colorama = "==0.3.9"coreapi = "==2.3.3"dj-database-url = "==0.5.0"djangorestframework = "==3.7.7"django-axes = "==4.0.2"django-clever-selects = "==0.8.2"django-crispy-forms = "==1.7.2"django-choices = "==1.6.0"django-extra-views = "==0.10.0"django-filter = "==1.1.0"django-hijack = "==2.1.7"django-hijack-admin = "==2.1.7"django-js-reverse = "==0.8.1"django-model-utils = "==3.1.1"django-phonenumber-field = "==2.0.0"django-polymorphic = "==2.0.2"django-redis-cache = "==1.7.1"django-role-permissions = "==2.2.0""django-s3direct" = "==1.0.4"django-static-precompiler = {extras = ["libsass"], version = "==1.8.2"}django-storages = "==1.6.6""django-tables2" = "==1.21.2"django-webpack-loader = "==0.6.0"django-widget-tweaks = "==1.4.2"facebookads = "==2.11.4"googleads = "==11.0.1"markdown = "==2.6.11"phonenumbers = "==8.9.3"pillow = "==5.1.0""psycopg2-binary" = "==2.7.4"pygments = "==2.2.0"pyssim = "==0.4"python-dotenv = "==0.8.2"pytz = "==2018.4"raven = "==6.6.0"sendgrid-django = "==4.2.0"slacker = "==0.9.65"termcolor = "==1.1.0"tqdm = "==4.21.0"twitter-ads = "==3.0.0"brotlipy = "==0.7.0"waitress = "==1.1.0"whitenoise = "==3.3.1"Django = "==2.0.4"
[dev-packages]coverage = "==4.5.1"selenium = "==3.11.0"tblib = "==1.3.2""flake8" = "==3.5.0"django-debug-toolbar = "==1.9.1"django-extensions = "==2.0.6"
[requires]python_version = "3.6"
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (7a535c)!
Installing dependencies from Pipfile.lock (7a535c)…
🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 92/92 — 00:01:01
real    8m1.993s
user    5m32.406s
sys     7m15.203s
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/pypa/pipenv/issues/356#issuecomment-380882203, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABhjqwIPyHtX0NTVoV1UPYR7HcwYm-2kks5tn42SgaJpZM4NbeoN
.

I had already installed the packages before that. I just removed the lockfile and ran install again. I'll do it again to get another data point

Woah. Super interesting... what causes the caching to kick in only after the third+ time?
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (7a535c)!
Installing dependencies from Pipfile.lock (7a535c)…
An error occurred while installing coreapi==2.3.3! Will try again.
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 92/92 — 00:00:58
Installing initially–failed dependencies…
Success installing coreapi==2.3.3!▉▉▉ 0/1 — 00:00:00
  ☤  ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 1/1 — 00:00:01

real    2m50.218s
user    2m19.438s
sys     5m44.797s

Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (7a535c)!
Installing dependencies from Pipfile.lock (7a535c)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 92/92 — 00:00:55

real    2m32.042s
user    2m6.516s
sys     5m10.219s

@kavdev @jtratner introduced a feature to cache the hashes as well so that should make a sizeable improvement

Here I am... 15 mins later

@giantas not helpful. Please provide pipfile, output, and duration with pip install.  Also whether you can do invidual packages.

It takes quite a lot to run on my machine. macOS 10.13.4, pipenv, version 11.10.0
Download runs almost immediately, but then it get's stuck on Locking [packages] dependencies…. Here's taking half a minute for two dependencies, and then 6 minutes for another 3 dependencies, if I throw all my project's dependencies at it, it just hangs indefinitely at the locking step
pablo@batman scanr (develop) $ time pipenv install flask flask_pymongo
Installing flask…
Looking in indexes: https://pypi.python.org/simple
Collecting flask
  Using cached https://files.pythonhosted.org/packages/77/32/e3597cb19ffffe724ad4bf0beca4153419918e7fa4ba6a34b04ee4da3371/Flask-0.12.2-py2.py3-none-any.whl
Collecting Werkzeug>=0.7 (from flask)
  Using cached https://files.pythonhosted.org/packages/20/c4/12e3e56473e52375aa29c4764e70d1b8f3efa6682bef8d0aae04fe335243/Werkzeug-0.14.1-py2.py3-none-any.whl
Collecting itsdangerous>=0.21 (from flask)
Collecting Jinja2>=2.4 (from flask)
  Using cached https://files.pythonhosted.org/packages/7f/ff/ae64bacdfc95f27a016a7bed8e8686763ba4d277a78ca76f32659220a731/Jinja2-2.10-py2.py3-none-any.whl
Collecting click>=2.0 (from flask)
  Using cached https://files.pythonhosted.org/packages/34/c1/8806f99713ddb993c5366c362b2f908f18269f8d792aff1abfd700775a77/click-6.7-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23 (from Jinja2>=2.4->flask)
Installing collected packages: Werkzeug, itsdangerous, MarkupSafe, Jinja2, click, flask
Successfully installed Jinja2-2.10 MarkupSafe-1.0 Werkzeug-0.14.1 click-6.7 flask-0.12.2 itsdangerous-0.24

Adding flask to Pipfile's [packages]…
Installing flask_pymongo…
Looking in indexes: https://pypi.python.org/simple
Collecting flask_pymongo
  Using cached https://files.pythonhosted.org/packages/fa/71/ab920741dedd605ef4adbd57d0c7d9f43a6b6fe4068604fffbc6f64b2c9c/Flask_PyMongo-0.5.1-py3-none-any.whl
Requirement already satisfied: Flask>=0.8 in /Users/pablo/.local/share/virtualenvs/scanr-2m6AW0PB/lib/python3.6/site-packages (from flask_pymongo) (0.12.2)
Collecting PyMongo>=2.5 (from flask_pymongo)
  Using cached https://files.pythonhosted.org/packages/5c/7f/1f7240883ec3fa768d7e066c9cbd42ceb42d699ba1a0fb9d231c098a542d/pymongo-3.6.1-cp36-cp36m-macosx_10_6_intel.whl
Requirement already satisfied: click>=2.0 in /Users/pablo/.local/share/virtualenvs/scanr-2m6AW0PB/lib/python3.6/site-packages (from Flask>=0.8->flask_pymongo) (6.7)
Requirement already satisfied: itsdangerous>=0.21 in /Users/pablo/.local/share/virtualenvs/scanr-2m6AW0PB/lib/python3.6/site-packages (from Flask>=0.8->flask_pymongo) (0.24)
Requirement already satisfied: Werkzeug>=0.7 in /Users/pablo/.local/share/virtualenvs/scanr-2m6AW0PB/lib/python3.6/site-packages (from Flask>=0.8->flask_pymongo) (0.14.1)
Requirement already satisfied: Jinja2>=2.4 in /Users/pablo/.local/share/virtualenvs/scanr-2m6AW0PB/lib/python3.6/site-packages (from Flask>=0.8->flask_pymongo) (2.10)
Requirement already satisfied: MarkupSafe>=0.23 in /Users/pablo/.local/share/virtualenvs/scanr-2m6AW0PB/lib/python3.6/site-packages (from Jinja2>=2.4->Flask>=0.8->flask_pymongo) (1.0)
Installing collected packages: PyMongo, flask-pymongo
Successfully installed PyMongo-3.6.1 flask-pymongo-0.5.1

Adding flask_pymongo to Pipfile's [packages]…
Pipfile.lock (c202d3) out of date, updating to (3068be)…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (3068be)!
Installing dependencies from Pipfile.lock (3068be)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 32/32 — 00:00:03
To activate this project's virtualenv, run the following:
 $ pipenv shell

real	0m37.816s
user	0m34.556s
sys	0m4.517s

pablo@batman scanr (develop) $ time pipenv install gunicorn h5py joblib
Installing gunicorn…
Looking in indexes: https://pypi.python.org/simple
Collecting gunicorn
  Using cached https://files.pythonhosted.org/packages/64/32/becbd4089a4c06f0f9f538a76e9fe0b19a08f010bcb47dcdbfbc640cdf7d/gunicorn-19.7.1-py2.py3-none-any.whl
Installing collected packages: gunicorn
Successfully installed gunicorn-19.7.1

Adding gunicorn to Pipfile's [packages]…
Installing h5py…
Looking in indexes: https://pypi.python.org/simple
Collecting h5py
  Using cached https://files.pythonhosted.org/packages/69/d5/dff2a8f7658fd87ab3330a0ab47e4363681d8bdf734a495add65a347f5e3/h5py-2.7.1-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Requirement already satisfied: six in /Users/pablo/.local/share/virtualenvs/scanr-2m6AW0PB/lib/python3.6/site-packages (from h5py) (1.11.0)
Collecting numpy>=1.7 (from h5py)
  Using cached https://files.pythonhosted.org/packages/a0/df/fa637677800e6702a57ef09e1d62e42aec3f598fb235f897155d146f2f59/numpy-1.14.2-cp36-cp36m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl
Installing collected packages: numpy, h5py
Successfully installed h5py-2.7.1 numpy-1.14.2

Adding h5py to Pipfile's [packages]…
Installing joblib…
Looking in indexes: https://pypi.python.org/simple
Collecting joblib
  Using cached https://files.pythonhosted.org/packages/4f/51/870b2ec270fc29c5d89f85353da420606a9cb39fba4747127e7c7d7eb25d/joblib-0.11-py2.py3-none-any.whl
Installing collected packages: joblib
Successfully installed joblib-0.11

Adding joblib to Pipfile's [packages]…
Pipfile.lock (0d514f) out of date, updating to (a4d15f)…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (a4d15f)!
 Do note that this is partly due to the installation of numpy, which I'm sure we are probalby compiling from source to lock or something stupid. would help if we provided useful output here

Is there anything I can do about it? or am I just out of luck at using pipenv and having these slow lock file times? I guess I could --skip-lock but I sort of wanted that feature

@pablote can you check if you rerun the same locking operation, is it still slow?

If I rerun it finishes almost immediately, so I guess it's just the first time some new dependency gets added.

so it's actually caching the hashes locally, so for whatever reason it's the hash generation process that is incredibly slow here...

Let me know if there's any info I can provide to help diagnose. For the moment I'll just go back to pip + virtualenv + pip-tools :/

@pablote once you have the hashes once, locking again won't be that slow

Please, please provide useful output.  I just upgraded my pipenv from 9.1.0 to 11.10.0 in order to resolve the Invalid Marker failure at package lock step as per, e.g., #1622 --- now, I have a pipfile with ipykernel, pandas, jupyter, numpy, and matplotlib in there and with my latest attempt to use pipenv install to get the lock file going, I've been sitting in locking [packages] dependencies… for upwards of 10 minutes.
Because there's no output, I can't tell whether there is something actually going (like building numpy from source) or if it's just hanging.  The best I can do is sort of squint at top and conclude that maybe it's doing something because a python process seems to be hanging around...
I'm happy to contribute to some work on this if needed.

Please, please provide useful output. I just upgraded my pipenv from 9.1.0 to 11.10.0 in order to resolve the Invalid Marker failure at package lock step as per, e.g., #1622 --- now, I have a pipfile with ipykernel, pandas, jupyter, numpy, and matplotlib in there and with my latest attempt to use pipenv install to get the lock file going, I've been sitting in locking [packages] dependencies… for upwards of 10 minutes.

Totally understandable complaint and I've been stewing in the back of my mind trying to think about how we can provide more useful feedback to the user (because I agree it's a little bit confusing that it doesn't have any output). I'd give up after 15 minutes. As @techalchemy pointed out, each time your cache should get populated a bit more, so it should get consistently faster over time.
One question: are you using public PyPI?

@jtratner yes, using public PyPi --- and I eventually did end up giving up, trashing the virtualenv and building a new one; I did manage to get a lockfile built by installing my dependencies one at a time.  (Interestingly, matplotlib was the slowest---even worse than numpy!)

Perhaps a message such as This may take a long time would help to reassure people until a clearer solution is decided upon.

15 mins is still insanely long I doubt I’d sit and wait for it. @paultopia you can run pipenv lock —verbose for more output

It does have the general feel of being slower than it should be, but maybe I'm underestimating the issue. If I look at the running processes in my computer, I can see the python one running the whole time pipenv is running, and it never goes above ~15%, it should probably use more of it if it's doing cpu intensive work like hashing files. Also, I've used other package managers that hash dependencies, like yarn, and they are pretty fast.

We need to figure out what it’s doing...

I've created a github project showing slowness when using locking. Please have a look at https://github.com/AndreasPresthammer/slow-pipenv . I'm not a 100% sure that this is the same problem though. Pull the repo down and run the slow.sh script, and observe the difference in execution time.

This is definitely still a problem (5+ mins), with latest python3.6, pip, and pipenv versions, and installing a simple package like torch. I don't think this issue should be marked as closed.

For anyone who wants to comment on this issue being closed: provide the output of pipenv lock --verbose. We know resolution is slow for packages that get compiled from source and that take a long time to build under normal conditions. We need logs to gain insight into the cause if this is slower than normal.

I'm also using Docker for my dev and production environment and think that the installation inside of Docker feels slow, compared to installation on the host.
Here is the log of pipenv lock --verbose, including the Pipfile and Pipfile.lock:
https://gist.github.com/mimischi/6270b7ece566cc571b427baaf1c331d4

@mimischi The resolution results are cached in the host, but Docker cannot access them. It is slow because it needs to redo all the resolution from scratch. It would help if you mount the host’s cache directory into Docker. There’s a comment in an issue mentioning the path you should mount. I don’t have the time to look it up now, but you can try find it. (And please contribute it to the doc’s FAQ page when you do!)

FWIW, locking on my dev box now takes less than 30s. Much better than before -- appreciate it.

Hi, getting the same problem.
here is the verbose
`
pipenv lock --verbose
Locking [dev-packages] dependencies…
Using pip: -i https://pypi.python.org/simple
                      ROUND 1

Current constraints:
pylint
Finding the best candidates:
found candidate pylint==1.9.1 (constraint was )
Finding secondary dependencies:
pylint==1.9.1             requires astroid<2.0,>=1.6, isort>=4.2.5, mccabe, six
New dependencies found in this round:
adding ['astroid', '<2.0,>=1.6', '[]']
adding ['isort', '>=4.2.5', '[]']
adding ['mccabe', '', '[]']
adding ['six', '', '[]']
Removed dependencies in this round:
Unsafe dependencies in this round:
Result of round 1: not stable
                      ROUND 2

Current constraints:
astroid<2.0,>=1.6
isort>=4.2.5
mccabe
pylint
six
Finding the best candidates:
found candidate astroid==1.6.4 (constraint was >=1.6,<2.0)
found candidate isort==4.3.4 (constraint was >=4.2.5)
found candidate mccabe==0.6.1 (constraint was )
found candidate pylint==1.9.1 (constraint was )
found candidate six==1.11.0 (constraint was )
Finding secondary dependencies:
pylint==1.9.1             requires astroid<2.0,>=1.6, isort>=4.2.5, mccabe, six
astroid==1.6.4            requires lazy-object-proxy, six, wrapt
isort==4.3.4              requires -
mccabe==0.6.1             requires -
six==1.11.0               requires -
New dependencies found in this round:
adding ['lazy-object-proxy', '', '[]']
adding ['wrapt', '', '[]']
Removed dependencies in this round:
Unsafe dependencies in this round:
Result of round 2: not stable
                      ROUND 3

Current constraints:
astroid<2.0,>=1.6
isort>=4.2.5
lazy-object-proxy
mccabe
pylint
six
wrapt
Finding the best candidates:
found candidate astroid==1.6.4 (constraint was >=1.6,<2.0)
found candidate isort==4.3.4 (constraint was >=4.2.5)
found candidate lazy-object-proxy==1.3.1 (constraint was )
found candidate mccabe==0.6.1 (constraint was )
found candidate pylint==1.9.1 (constraint was )
found candidate six==1.11.0 (constraint was )
found candidate wrapt==1.10.11 (constraint was )
Finding secondary dependencies:
astroid==1.6.4            requires lazy-object-proxy, six, wrapt
wrapt==1.10.11            requires -
lazy-object-proxy==1.3.1  requires -
six==1.11.0               requires -
pylint==1.9.1             requires astroid<2.0,>=1.6, isort>=4.2.5, mccabe, six
mccabe==0.6.1             requires -
isort==4.3.4              requires -
Result of round 3: stable, done
Locking [packages] dependencies…
`
Here is the pipenv --version: pipenv, version 2018.05.18
Also i don't know why this is happening no specific reason/ error is occurring.
In my case when I do pipenv lock it starts but it never ends as far as I know. I have been waiting for 2 hrs now still no sign of completion. And has given me ReadTimeoutError twice this is the third time I am doing this.
Using Python 3.6.4
Any help would be of great help as my projects due date is closer.

Also vote for this issue to be reopened. It is far from solved....take somewhere between 10-20 minutes to lock my project within a Docker container.  It also uses an insane amount of memory - such that I had to increase the allocation to Docker to stop it from killing the process.

Please provide example Pipfilesmand details about your environments if you want help with issues with resolution speed. We are cutting a release this week and don’t have time to track individual one off comments on closed threads but can test against Pipfiles if you provide one

@jkp Allow me to assure you that each and every core developer of this project (not that there are many of us to begin with) is very well aware of this issue, and is as troubled by it as you are, if not more. This is, however, by no means an easy problem, and we have already throw everything we could at it to make it as usable as possible, without having to tear everything down in Python packaging. We do also have a lot on our plate at the moment, and need to also focus on those other issues. The inevitable decision me must make, then, is to prioritise issues we actually know how to solve, and only start thinking about our next steps after those are done, to maximise the effect of our effort.
Now, I fully acknowledge that your priority may be different from ours. This performace issue can be the single most largest problem in your workflow, and you want to put it up as the most important thing in this project. Please bear in mind, however, that you are not the only user of this tool, and we need to prioritise the need of all, even the need of our own, in front of yours. I acknowledge that. I urge, therefore, anyone sharing the situation to join minds on this issue, and try to think of a way to solve it. Once we know what to actually do, we can do it.
The issue is kept closed because we do not know what we can do, and would serve only as noise in the issue tracker when we try to manage it. There is no point, as least in our workflow, to have an issue that nobody can work on.

Appreciate your workflow so if that is how you manage issues that's fine.  I'll try to add any information I can to help track down the problem.
I did some debugging by installing mitmproxy between pipenv and the net to trace the requests being made.  I found a couple of interesting things.

We are using a private pypi index that doesn't support the json-api yet. This slows things down significantly since it looks like the fallback is to bruteforce and download everything listed in http index in order to extract metadata etc.  One suggestion here would be just to add some simple logging to warn that this fallback method is being used - it might save someone else having to dig deeper to figure this out.

Using the brute-force method it seems that the code downloads packages that aren't relevant to the architecture in use.  For example on a linux machine it was downloading win32 or osx specific wheel packages. This feels like something that could be detected and avoided since it's clear that binary packages built for other architectures are not going to be of any use.

I will continue to debug and report back any useful info as I find it.

It seems like even with the json interface in use pipenv is making unnecessary requests for wheel files that relate to different architectures. The implementation is currently pretty naive in that it checks all files listed against a given release, regardless of the target platform / arch.

I feel like we must be downloading the same things multiple times, right?

We should probably move the discussion to #2284. It is actually the locking part that is slow (install is essentially TOML manipulation + lock + sync), not installing.

Regarding locking down to a single-arch. I understand the design choice; perhaps there could be an option to pass a flag to allow a user to lock only for the host architecture? This would be a pretty significant optimisation both in terms of time, and bandwidth for some use-cases.

@techalchemy thanks for your response. The packaging.version.parse() find seems like a good lead.  Could you put some more colour on this statement:

Regarding the JSON api, it's not actually enabled to hit directly in the current release, and I am planning to disable it in the codebase again before we release.

I didn't understand why you are planning to disable it?

@jkp Regarding the JSON API, let’s just say it is not the best designed thing out there. The simple API is much more suitable for us. By disabling it we’re using the simple API, not using no APIs altogether.

It's taking too long for me to install Pyspark.
My Pipfile -
[[source]]
name = "pypi"
verify_ssl = true
url = "https://pypi.org/simple"

[dev-packages]
pylint = "*"
pyspark = "*"

[requires]
python_version = "3.5"

shell output -
Pipfile.lock not found, creating...
Locking [dev-packages] dependencies...

Process is stuck on the last line above.
Finishes after 15-20 mins
pipenv, version 2018.7.1

@keshavkaul PySpark is very large, and can take quite some time just to download. Give it some time, it will be much better afterwards (because Pipenv caches the result).
(Or you can urge the developers to release a wheel distribution. That would help a bit.)

Note for future visitors: Please refrain from posting your slow installation result. We know it can be slow. We know why it is slow. Your result does not add anything to this topic.

Would it be possible to have some information or progress bar like apt-get or wget (download speed, size downloaded, total size) during the libraries downloads?
I guess this is the issue here, pipenv seemed slow for me but it was just the library download, I had to open a system monitor to understand that pipenv was downloading the files and how much was already downloaded, what speed etc

hava same issue: Locking [packages] dependencies... hang forever
my environment:

macOS High Sierra 10.13.6
Python: Python 3.6.4

@crifan there is no need to post the same message on every issue open or closed which mentions locking speed. We will see your comment no matter how many times you say the same thing. If you want to be helpful, you will need to provide a reproducible example case. Chiming in to say “me too” simply doesn’t add anything besides extra traffic on the issue tracker. Please be mindful of that.

Same here. Pipenv very slow.
Takes an hour to lock and install.

I think this issue has been answered well here: https://github.com/pypa/pipenv/issues/1914#issuecomment-378846926

Python dependencies require us to fully download and execute the setup files of each package to resolve and compute. That's just the reality, it's a bit slow. If you can't wait 2 minutes or you feel it's not worth the tradeoff, you can always pass --skip-lock.

by @techalchemy

Would it be possible to get the list of hashes from the PyPI API, rather than compute them ourselves?

pipenv is awesome, but this issue seems to be still existing. will be glad to see any progress. --skip-lock did not work.

hava same issue: Locking [packages] dependencies... hang forever
my environment:
CentOS 7.5
Python: Python 3.6.5
pipenv: version 2018.10.13
Can not use pipenv in virtualenv ? if yes,  I think it is  so bad.

pipenv is awesome, but this issue seems to be still existing. will be glad to see any progress. --skip-lock did not work.

Worked for me

I found that using Git Bash on Windows was very slow compared to Powershell. I don't have any statistics or data on this, but PS seemed faster. So if you are using Git Bash you may want try the native PS for pipenving.

The following docker image takes more than 30 minutes to build on my laptop (i7/16Gb):
Dockerfile
FROM python:3.7-alpine

# Update package list.
RUN set -ex && apk update

# Install apk dependencies.
RUN set -ex && apk add --no-cache musl-dev gcc libffi-dev openssl-dev make

# Install Pipenv.
RUN set -ex && pip install pipenv --upgrade

# Copy Pipfiles.
RUN mkdir /website
COPY Pipfile /website

# Install Pipenv dependencies.
WORKDIR /website
RUN set -ex && pipenv install --system --skip-lock --verbose

Pipfile
[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[requires]
python_version = "3.7"

[packages]
sanic = "*"
jinja2 = "*"
asyncpg = "*"
uvloop = "*"
munch = "*"

[dev-packages]

[pipenv]
allow_prereleases = true

Is this normal? Can someone reproduce?

+1

I have long left pipenv and whenever i need to create virtual env in use "venv" or other options.

Having a weird slowness issue as well, only 2 modules for some scripting I am doing.
boto3
click

Took 15/20 min to install, internet tests at over 60Mbps down and running on an up to date MacBook Pro 2019 (not my choice of hardware).

Please use virtualenv for time being. until better solution available

My goodness this is slow.
I'm on a work project, so I can't really change the list of packages or switch from Pipenv right now.
It's taking about 20 minutes every time, which is really interrupting my flow as I setup a project I've never used before, so am doing this quite a bit as I get things going.
Apologies for this non-useful feedback, but basically +1 on this.

99% of the time I do this, the dependencies will resolve to the same one in my lock file, this is because it's part of my dev pipeline.
In the case where there are no new upstream packages since the last run, surely the process could be skipped ?

We have done some performance enhancements to pipenv in recent times, including a big install optimization which released 2022.8.31. For an independent comparison of benchmarks, please have a look at: lincolnloop.github.io/python-package-manager-shootout


Support settings custom search path

Add search_path configuration open to shards.
Fix showing incorrect shard when setting a shard to an unsupported number (just a display error bug).

MR have conflicts

2024-08-21T07:31:04.106968Z ERROR ThreadId(04) pgcat::pool: Shard 0 down or misconfigured: TimedOut
2024-08-21T07:31:04.107054Z ERROR ThreadId(04) pgcat::pool: Could not validate connection pool

I've tried to run the code but Timeout Error shown

The code seems like sample code GPT generated

Sorry it's my bad I add the options code to set search_path after the end of \0

Any news about that? 🥲


Head sort on C, C++ and Python

Hey @ming-tsai,
I wanted to add heap sort in C, C++ and, python, further want to add docs/en of heap sort please assign me to do that.
Thanks

Hi @Sranu2109,
When you finished one of them, please make it checked.

[x]   C
[x]   C++
[x]   Python

Done with all of em'

Nice work @Sranu2109


fixing bug in determining synapse type in multinode #99 #minor

Solves the bug introduced in #99.

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (6e00b05) 84.02% compared to head (7294e0e) 84.07%.
Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #101      +/-   ##
==========================================
+ Coverage   84.02%   84.07%   +0.04%
==========================================
  Files          76       76
  Lines        5178     5192      +14
==========================================
+ Hits         4351     4365      +14
  Misses        827      827

Files
Coverage Δ

bluecellulab/synapse/synapse_factory.py
90.47% <100.00%> (+0.60%)
:arrow_up:

tests/test_synapse/test_synapse_factory.py
100.00% <100.00%> (ø)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


change property type of styles to object like the doc says

might fix this https://github.com/gmaps-marker-clusterer/gmaps-marker-clusterer/issues/20

@XmlmXmlmX did you test this? I'm not entirely sure about this change. Check this comment https://github.com/gmaps-marker-clusterer/gmaps-marker-clusterer/issues/20#issuecomment-279131811
If you check from line 44 on in this file https://googlemaps.github.io/js-marker-clusterer/examples/advanced_example.html you can see that google used an array for individual cluster sizes. My change might break this functionality. Clearly the documentation wasn't good, but I think we should run some tests and maybe create some more examples.

@Remo Sorry, this was to fast :/


Variant 'apolloCommonMainServiceProducer' in project does not match the consumer attributes

Hey,
I want to use the graphql client in a KMM and generate via gradle plugin the graphql query classes.
But I cant build my project with the dependency apollo-runtime-kotlin in commonMain.
First I got a problem like this, so I added their solution snippet, but now i get this error while building:

Could not determine the dependencies of task ':checkCommonMainServiceApolloDuplicates'.
Could not resolve all task dependencies for configuration ':apolloCommonMainServiceDuplicatesConsumer'.
Could not resolve project :.
Required by:
project :
Variant 'apolloCommonMainServiceProducer' in project : does not match the consumer attributes
Variant 'apolloCommonMainServiceProducer' capability org.example:my-project:unspecified declares a component, as well as attribute 'com.apollographql.service' with value 'service', attribute 'com.apollographql.variant' with value 'commonMain':
- Incompatible because this component declares a usage of 'apollo-metadata' of a component and the consumer needed a runtime of a component

Here is my build.gradle.kts:
build.txt

Hi 👋  Thanks for reaching out! That part (from https://github.com/Kodein-Framework/Kodein-DI/issues/264):
configurations.all {
    afterEvaluate {
        if (isCanBeResolved) {
            attributes {
                attribute(usage, project.objects.named(Usage::class.java, "java-runtime"))
            }
        }
    }
}

will confuse Apollo configurations as the Apollo plugin relies on a apollo-metadata usage. What you can do is not overwrite the attributes if it's already there:
configurations.all {
    afterEvaluate {
        if (isCanBeResolved && !attributes.contains(usage)) {
            attributes {
                attribute(usage, project.objects.named(Usage::class.java, "java-runtime"))
            }
        }
    }
}

Thanks a lot that error is gone :)
But now I got the next problem.
 Cannot add a configuration with name 'jsNpm' as a configuration with that name already exists.
Do you have an idea?

Not really sure, sorry. That seems related to the JS targets/configuration. Maybe try to ask on the Kotlin slack?

I'm going to close this one as the original issue is fixed. Please let me know if you want me to reopen.


Update lint_test.go

@dexhorthy I think the new phrasing is much clearer so will work with the team to get at least this part adjusted (since it should just be error copy change)


Doors open/close status get desync'd across client/server

Probably the biggest issue behind solving this is that a client currently can open a door and go through it immediately without lag.  Implementing this would add a lag time to that.

My best thought to resolving this is that when a door is "triggered" by a client, the client tells the server "this door was triggered", and the server tells everyone to open the door for however long doors are opened for.  If a door is "triggered" by a server, it simply tells everyone to open the door for however long doors are opened for.  Clients would simply close doors when instructed.  If a "open door" is triggered while the door is open, it stays open longer.
This gets into ID'ing doors, and figuring out how to sync this all across client/server.  Shouldn't be terribly difficult, but I don't know the first thing about where to look for this code.


Add constructor method "new" for Input types

Depends on #253
Depends on #258
Depends on #259
Add constructor method "new" for Input types.
The constructor method provides a simpler way to initialize a type with
many or only optional fields. Assuming an input type like this:
    input CustomerInput {
      company: String
      customFields: [CustomFieldInput!]
      email: String
      firstName: String
      lastName: String
      phoneNumber: String
    }

The generated code looks like so:
    pub struct CustomerInput {
      pub company: Option<String>,
      ...
    }
    impl CustomerInput {
      pub fn new() -> Self {
        Self { company: None, custom_fields: None, ... }
      }
    }

And with mandatory fields it looks like this:
    input CustomerInput {
      firstName: String
      lastName: String!
    }

    pub struct CustomerInput {
      pub firstName: Option<String>,
     pub lastName: String,
    }
    impl CustomerInput {
      pub fn new(lastName: String) -> Self {
        Self { firstName: None, lastName: lastName }
      }
    }

Because not used code is removed by the linker, those extra generated methods do not add to the final binary size if not used.
Usecase
We can't derive from Default for input types, because they potentially contain mandatory fields like enums not derived from default themselves. An input type with many optional fields therefore requires to name every single field, even if it is just for assigning None. With this PR however I can do the following:
let query = CreateCustomer::build_query(create_customer::Variables {
    customer: create_customer::CustomerInput {
        first_name: Some("first".to_owned()),
        last_name: Some("last".to_owned()),
        email: Some("test@abc.de".to_owned()),
        ..create_customer::CustomerInput::new()
    },
});

Will review this next time I get a chance to work on this project. Sorry for the wait!

Could you please rebase on master for the conflict resolution and CI fixes? Thanks!

Hmm. Maybe we can somehow divine that Default could be implemented for the Variables struct? Maybe this is better part of adding separate derives for input and results? That seems more flexible to me personally because it would only apply where it matters rather than making new enum types and such. Thoughts?

I will rebase when I find time, sure.

Hmm. Maybe we can somehow divine that Default could be implemented for the Variables struct?

I think I don't get you. Because of possible mandatory fields in Variables, that the API user should set in contrast to relying on default created values.

Maybe this is better part of adding separate derives for input and results? That seems more flexible to me personally because it would only apply where it matters ....

I'm not against a separation for input and results. But it doesn't change the fact outlined above, does it?

rather than making new enum types and such. Thoughts?

In this PR at least no new types are created. Only an impl is generated.

That's why I said "divine". We can only do so when there are actually no required fields (String has a impl Default, but is probably not too relevant here.
The separation would allow you to say variables_derives = "Debug, Default" and you just get it for free. If Default isn't implementable, the compiler tells you. If it is, you get it (though you can add it for things like String-using variables structures, this might actually be wanted).
Once it's rebased, it should be easier to review.


NoClassDefFoundError: Failed resolution of: Lkotlin/jvm/internal/Intrinsics

I tried to use your lib, but I had this issue when I was trying to launch by this code:
  Intent intent = new PlacePicker.IntentBuilder()                     .setLatLong(latLng.latitude, latLng.longitude)                       .showLatLong(true)                     .setAddressRequired(true)                     .hideMarkerShadow(true)                     .setMarkerImageImageColor(R.color.colorPrimary)                     .setMapRawResourceStyle(R.raw.map_style)                     .disableBootomSheetAnimation(true)                     .onlyCoordinates(true)                     .build(getBaseActivity());             startActivityForResult(intent, Constants.PLACE_PICKER_REQUEST);
and the err logs:

Caused by: java.lang.ClassNotFoundException: Didn't find class "kotlin.jvm.internal.Intrinsics" on path: DexPathList[[zip file "/system/framework/org.apache.http.legacy.boot.jar", zip file "/data/app/chat.social.free.text.messaging-sYeyXSfWWb0WtUfIEHp01Q==/base.apk"],nativeLibraryDirectories=[/data/app/chat.social.free.text.messaging-sYeyXSfWWb0WtUfIEHp01Q==/lib/arm64, /system/lib64]]

I have the same problem. What was your solution? Thanks


[WIP] (docs) Dart Developer guides/attribute directives example

@kwalrath @kevmoo Please review.

To make a variable private, you just name it with an underscore prefix. But it seems weird to use a constructor to initialize a class variable. Is it really an instance variable?

Actually, the leading underscore has no meaning to TS/JS at all. It is a convention that lets the readers (and the persons debugging the code) know that this variable is NOT part of the public API and may not be referenced externally (at your peril).
TS enforces the  private and internal access modifiers within the TS world; it has no runtime effect.
It is an instance variable, not a class variable.

Ah. Thanks @wardbell @kwalrath. I think see where my disconnect was. So if add Renderer _renderer; and ElementRef _el; then those will both be private, correct?
Then the constructor can be Highlight(ElementRef this._el, Renderer this._renderer);

Yep, that looks right. Dart style is to avoid abbreviations, so we'd probably use _element instead.

PS to Ward: Dart has no private/internal access modifiers. Just using _ makes the variable library-private.

LGTM. @kevmoo, any comments?

See comments!

Thanks @kevmoo. I made those changes. @kwalrath updating to beta.1 broke this example. ElementRef says it no longer has a getter for 'style'. Also referenced here  it suggest to use ElementRef with caution. It looks like they are still using it in TS though. Thoughts?

@yjbanov do you know the answer to Tom's question?

...updating to beta.1 broke this example. ElementRef says it no longer has a getter for 'style'. Also referenced here it suggest to use ElementRef with caution. It looks like they are still using it in TS though.

@kwalrath @tomplusplus
You now need to pass elementRef.nativeElement to the Renderer. That's a recent change.

Thanks very much, @yjbanov!

Ugh!  This is the kind of breaking change we were NOT supposed to see. It affects almost every attribute directive.  The API guide is no help on the subject as it describes the first parameter as  renderElement: any w/ no further comment.
When we went to beta we promised people that we would be cautious about breaking changes. We're supposed to have a high bar for breaking changes.
As a matter of policy, we should always update API docs when making a breaking change.   I do not think that noting it in the CHANGES.MD is sufficient.
See issue I raised: https://github.com/angular/angular/issues/6627

PLEASE DO NOT MERGE!
We cannot have samples running on different versions of A2! We have to all be on the same version across all samples and all language tracks.
I'm grateful that Tom discovered the issue though!

I plan to convert this back to b0 and merge it, so we can have something to point people to.


First choko world

@ldemk I assembled choko world from materials from Jonny007-patch-1 branch.
Merge it with master, please.

@abench looks great! Merged!


How to distinguish the source of getSignatures and setSignatures

Question
I don't know the corresponding relationship between them
"sources": [
	{
		"fileName": "a.ts",
		"line": 6,
		"character": 12
	},
	{
		"fileName": "a.ts",
		"line": 12,
		"character": 12
	}
],
"getSignature": { ... },
"setSignature": { ... }

a.ts content:
interface DelegateInterface<T> {
        /**
         * @description lalala
         * @effect 1.0
         */
        set useUpdate(v: boolean);

	/**
         * @description lalala
         * @effect 1.0
         */
        get useUpdate(): boolean;
}

The order in the source seems to be stored from top to bottom, but the writing order of set and get is uncertain, so I cannot determine the corresponding order of set and get

interface DelegateInterface<T> {

        /**
         * @description 是否使用更新set
         * @effect 调用端生效
         */
        set useUpdate(v: boolean);

		/**
         * @description 是否使用更新get
         * @effect 调用端生效
         */
        get useUpdate(): boolean;

}

The order in the source seems to be stored from top to bottom, but the writing order of set and get is uncertain, so I cannot determine the corresponding order of set and get

In 0.23, since sources aren't added to signature reflections, this isn't possible cleanly - I've addressed this in the 0.24 beta branch already, which I intend to release before the end of the year.
It should be stable to check getSignature.id and setSignature.id, these should be ordered in the same way as the sources... but that's really ugly. You could also use a plugin to listen to EVENT_CREATE_SIGNATURE, and use that to get the source information yourself (likely also using addSerializer to make it show up in the JSON)

thanks


TrainingJobAnalytics

Is there any way we can output a csv file for the new TrainingJobAnalytics features with pytorch & the python SDK in our code?

Sorry I see this is actually just a regex thing given in the article https://aws.amazon.com/blogs/machine-learning/easily-monitor-and-visualize-metrics-while-training-models-on-amazon-sagemaker/
Closing


unfilled hours on trainingofficer might put form index out of order

Ensure empty inputs come in with a zero, ie 5 fields but 4 entries returns 5 key/value pairs.

"required" added to all input fields.


aleph-message does not expose a version value

aleph_message.__version__
Traceback (most recent call last):
  File "/home/odesenfans/pycharm-2021.3.1/plugins/python/helpers/pydev/pydevconsole.py", line 364, in runcode
    coro = func()
           ^^^^^^
  File "<input>", line 1, in <module>
AttributeError: module 'aleph_message' has no attribute '__version__'

When does this happen ?

When installing aleph-message in a clean virtualenv. I just reproduced it using the following steps:
python -m virtualenv venv
source venv/bin/activate
pip install aleph-message==0.4.0

python
>>> import aleph_message
>>> print(aleph_message.__version__)

results in

Traceback (most recent call last):
File "", line 1, in
AttributeError: module 'aleph_message' has no attribute __version__


Add examples

[ ] redux-form

+1 for validation with regex

Hi, @rytyr.
Do you mean something like this?
const mySchema = schema({
  title: {
    type: String,
    match: /^[A-Z0-9]/ // should be capitalized
  }
})

@diegohaz exactly. I believe such use cases should be described in documentation.


[BUG] TabPage should extend StatefulPage

Currently, with v0.9.3, this route definition is not possible:
'/books/:id': (route) {
  final id = RouteUtils.asInt(route.pathParameters['id']!);

  return (id != null)
      ? TabPage(child: BookPage(id: id), paths: [...])
      : NotFound();
},

The return type 'RouteSettings' isn't a 'Page<dynamic>', as required by the closure's context.

@bdremynoel this pattern should work in the meantime:
if (id == null) {
  return NotFound();
}

return TabPage(child: BookPage(id: id), paths: [...]);

I don't really understand what's going on here with Dart - this pattern is fine without null safety enabled, but won't compile with it.
I've filed an issue to get clarification before making a change to Routemaster here: https://github.com/dart-lang/sdk/issues/46333
In the meantime you can use if instead of the ternary operator.

Yes, it's ok, there's no reason to change the template of TabPage and associates.

I think the solution here is to change PageBuilder to return RouteSettings, but then check at runtime that it's a Page.
That could lead to runtime exceptions if you tried to return something other than Page, but at least there isn't any other object that extends from RouteSettings in Flutter.
Really annoying limitation of Dart :-/


Write not available for S3 compatible - Cloudian Objectstorage

Mountpoint for Amazon S3 version
mount-s3 1.0.1
AWS Region
We use an on premise S3 compatible Objectstorage, Cloudian
Describe the running environment
We use a S3 compatible objectstorage called Cloudian Hyperstore.
The Cloudian cluster is running on premise with a cluster of multiple CentOS Linux release 7.9.2009 (Core).
We installed the mount-s3 on our personal VMs running Red Hat Enterprise Linux release 8.8 (Ootpa).
What happened?
On our Cloudian Objectstorage we can create an account and generate the aws_access_key_id and the aws_secret_access_key. Then configure everything properly on our VMs system on ~/.aws/credentials and ~/.aws/config
I use aws_cli and all the functionalities are working correctly but when i use the mount-s3 I'm only able to list the files on a bucket and delete them if the flag --allow-delete is set when mounting.
Copying or moving files on the mount will result in a 403 error, saying to "The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method."
Relevant log output
Sep 14 11:20:00 my_vm mount-s3[3584971]: [WARN] lookup{req=18 ino=1 name="access.json"}: mountpoint_s3::fuse: lookup failed: inode error: file does not exist
Sep 14 11:20:00 my_vm mount-s3[3584971]: [WARN] create{req=20 parent=1 name="access.json"}: mountpoint_s3::fuse: create failed: operation not supported by Mountpoint
Sep 14 11:20:01 my_vm mount-s3[3584971]: [WARN] getxattr{req=26 ino=2 name="security.capability"}: mountpoint_s3::fuse: getxattr failed: operation not supported by Mountpoint
Sep 14 11:20:01 my_vm mount-s3[3584971]: [WARN] flush{req=30 ino=2 fh=2}: mountpoint_s3::fuse: flush failed: operation not supported by Mountpoint
Sep 14 11:20:01 my_vm mount-s3[3584971]: [WARN] open{req=24 ino=2}:put_object{id=10 bucket=testing key=access.json}: mountpoint_s3_client::s3_crt_client: request failed request_type=UploadPart http_status=403 range=None duration=305.900674ms ttfb=Some(300.559527ms) request_id=xxx
Sep 14 11:20:02 my_vm mount-s3[3584971]: [WARN] open{req=24 ino=2}:put_object{id=10 bucket=testing key=access.json}: mountpoint_s3_client::s3_crt_client: meta request failed duration=1.493716688s error=ClientError(Forbidden("The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. For more information, see REST Authentication and SOAP Authentication for details."))
Sep 14 11:20:02 my_vm mount-s3[3584971]: [WARN] release{req=32 ino=2 fh=2}: mountpoint_s3::fuse: release failed: put failed: Client error: Forbidden: The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. For more information, see REST Authentication and SOAP Authentication for details.

This one might be hard for us to offer much help for, since we're not familiar with Cloudian. You might be able to debug this by using verbose logging, and specifically the --debug-crt option, which will log the signing strings and signatures that Mountpoint calculates for requests. Then (at least for Amazon S3) the verbose version of the error message you provided usually also includes what the server calculated for signing strings and signatures. The verbose error message should also be logged with the --debug-crt option, so you can compare the two.

We are closing this issue. Please feel free to reopen if you have further information.

I'm now working with mount-s3 1.4.0.
I have some more information that might help defining the issue.
Feb 15 16:09:01 my-vm mount-s3[1571818]: [INFO] release{req=92 ino=5 fh=6 name=file.txt}: log: (id=0x7f0dd809b9c0) Signing successfully built canonical request for algorithm SigV4, with contents #012PUT#012/a.txt#012partNumber=1&uploadId=MmUxM2E2MzI5MTE3MDgwMDk3NDE3NTI#012accept:application/xml#012content-encoding:aws-chunked#012content-length:37#012host:bucket.ourhost#012x-amz-content-sha256:STREAMING-UNSIGNED-PAYLOAD-TRAILER#012x-amz-date:20240215T150901Z#012x-amz-decoded-content-length:0#012x-amz-trailer:x-amz-checksum-crc32c#012#012accept;content-encoding;content-length;host;x-amz-content-sha256;x-amz-date;x-amz-decoded-content-length;x-amz-trailer#012STREAMING-UNSIGNED-PAYLOAD-TRAILER

Feb 15 16:09:01 my-vm mount-s3[1571818]: [INFO] release{req=92 ino=5 fh=6 name=file.txt}: log: (id=0x7f0dd809b9c0) Signing successfully built string-to-sign via algorithm SigV4, with contents #012AWS4-HMAC-SHA256#01220240215T150901Z#01220240215/myregion/s3/aws4_request#012c248c4e2b29f55c094b50691a55fba6b2091b2a9846797e63fba517e8d10fa46

Feb 15 16:09:01 my-vm mount-s3[1571818]: [INFO] release{req=92 ino=5 fh=6 name=file.txt}: log: (id=0x7f0dd809b9c0) Http request successfully built final authorization value via algorithm SigV4, with contents #012AWS4-HMAC-SHA256 Credential=myaccessid/20240215/myregion/s3/aws4_request, SignedHeaders=accept;content-encoding;content-length;host;x-amz-content-sha256;x-amz-date;x-amz-decoded-content-length;x-amz-trailer, Signature=9095dcb27e05077937de47f2331af86538c2c64ba65b8879c3fc378125d2f1aa

Feb 15 16:09:01 my-vm mount-s3[1571818]: [INFO] log: id=0x559a889005d0 Requests-in-flight(approx/exact):1/1  Requests-preparing:0  Requests-queued:0  Requests-network(get/put/default/total):0/1/0/1  Requests-streaming-waiting:0  Requests-streaming-response:0  Endpoints(in-table/allocated):1/1

Feb 15 16:09:01 my-vm mount-s3[1571818]: [ERROR] log: id=0x7f0dd802afb0 Meta request cannot recover from error 14343 (Invalid response status from request). (request=0x7f0df801be20, response status=403)

Feb 15 16:09:01 my-vm mount-s3[1571818]: [INFO] log: id=0x7f0df0025fb0: Shutting down connection with error code 0 (AWS_ERROR_SUCCESS).

Those appear to be the recurring steps, followed by this error
Feb 15 16:09:02 my-vm mount-s3[1571818]: [WARN] release{req=92 ino=5 fh=6 name=file.txt}:put_object{id=28 bucket=mybucket key=a.txt}: mountpoint_s3_client::s3_crt_client: meta request failed duration=1.235436649s error=ClientError(Forbidden("The request signature we calculated does not match the signature you provided. Check your AWS Secret Access Key and signing method. For more information, see REST Authentication and SOAP Authentication for details."))
Do you think this is due to a bug of Cloudian in handling the signature check?

I'm not super familiar with Cloudian, but if it's only writes that are failing like this, one possibility is that it may not support S3's additional checksums feature, which Mountpoint requires for writes. They're specified with the x-amz-checksum-crc32c header. We're tracking the ability to turn off additional checksums in #516, but it's not something we're currently working on.

Hi @jamesbornholt I checked on the documentation of Cloudian and indeed this header it's not mentioned anywhere.

Hi all!
I wanted to inform you that this bug is now fixed on Cloudian's side. There was indeed the x-amz-checksum-crc32c header missing.
Now put operations work.
Tested on mountpoint version 1.5

Hi all! I wanted to inform you that this bug is now fixed on Cloudian's side. There was indeed the x-amz-checksum-crc32c header missing.
Now put operations work.
Tested on mountpoint version 1.5

Thanks for sharing, Angelo! Glad to hear you're having success with your use case.


Program type already present: androidx.versionedparcelable.ParcelImpl$1

I have an issue about the new version

What went wrong:

Execution failed for task ':app:transformDexArchiveWithExternalLibsDexMergerForDebug'.
  Learn how to resolve the issue at https://developer.android.com/studio/build/dependencies#duplicate_classes.
  Program type already present: androidx.versionedparcelable.ParcelImpl$1

What should I do here?

Hmm, no idea. I've never seen that.


Add Le Vu Trung Duong to Contributors list

Add my name to Contributors list and SE-class-2021 for lecture 4 assignment

Congratulations on the Pull Request! See you in class


Document how to bring in the SDKs from Maven Central

Follows up on the work done in #596.
We'll need to break up the current GitHub Packages section as some of it can be shared between instructions for both GitHub Packages and Maven Central.
┆Issue is synchronized with this Jira Task by Unito

@QuintinWillison I think that this has been resolved in the https://github.com/ably/ably-asset-tracking-android/pull/598, right? If yes then please close this issue :wink:

Indeed. Thanks! 😄


Amendment to flow-img to cope with video or audio

Hi I have made some changes to the directive of "flow-img" by changing it to "flow-src"
being able to cope with audio and video. How can I commit these changes to this repository? Perhaps could the author give me an email or other steps?
Cliff

Create a gist at https://gist.github.com/ and share your ideas, thanks


Signal-to-Noise Ratio calculation

Hi!
I'm currently working on a denoising CNN for EOG artifacts. I am implementing a similar procedure to the one described in your paper which cleverly parametrizes the artifact contaminations in the signal by computing the signal with the desired SNR levels to match the physiological ones.
However I couldn't understand why in the EEGdenoiseNet/code/benchmark_networks/data_input.py script you computed the SNR using the RMS instead of the MSE in the ratio.
Line 56 shows:
coe=get_rms(eeg)/(get_rms(noise)*SNR_train[i])

meaning that
SNR_train[i]=get_rms(eeg)/(get_rms(noise)*coe)

Being the definition of SNR the ratio of powers of signal and noise  shouldn't the SNR be estimated with MSE as  or alternatively using RMS as .
Please let me know if I am missing something. Thank you again for your work.
BR

As I know they all can be used to calculate SNR, for example in paper "Canonical Correlation Analysis Applied to Remove Muscle Artifacts From the Electroencephalogram'' and paper "Independent Vector Analysis Applied to Remove Muscle Artifacts in EEG Data"

Hi, thank you for the reply! I will try to be more concise and clear with my question.
I'm using the following additive relationship to describe how noise affects the final signal:    y = x + k*n
where y is the noisy EEG signal, x is the clean EEG signal, n is the noise (EOG or EMG) and k is a coefficient to parametrize how much n affects y.
By substituting this relationship inside the definition of SNR we get:
SNR = MSE(x) / MSE(k*n) = MSE(x) / ((k^2) * MSE(n))
If we solve the equation for k we get:
k = sqrt(MSE(x) / (SNR * MSE(n)) = RMS(x) / (RMS(n) * sqrt(SNR))
Which is different from the equation used in EEGdenoiseNet/code/benchmark_networks/data_input.py line 56 where there is no  square root of the SNR in the denominator.
coe=get_rms(eeg)/(get_rms(noise)*SNR_train[i])

# Shoud be changed into this:

coe=get_rms(eeg)/(get_rms(noise)*math.sqrt(SNR_train[i]))

Please let me know if it is an error on your side or I'm missing something.
BR

Thanks for your reply!
In this two papers, SNR is defined by:

different with

We also feel surprise for this difference, because this 2 paper are important in the area of EEG denoise, so we use the first defination.
Best regards

Thank you, I finally understand now. Have a nice day.
Best regards,
LG


winusb: 3.1.4 -> 3.1.5

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools.
This update was made based on information from https://repology.org/metapackage/woeusb/versions.
These checks were done:

built on NixOS
ran /nix/store/8j0k3dnx7vc5wcmayficjdsk02ix55va-woeusb-3.1.5/bin/woeusb -h got 0 exit code
ran /nix/store/8j0k3dnx7vc5wcmayficjdsk02ix55va-woeusb-3.1.5/bin/woeusb --help got 0 exit code
ran /nix/store/8j0k3dnx7vc5wcmayficjdsk02ix55va-woeusb-3.1.5/bin/woeusb -V and found version 3.1.5
ran /nix/store/8j0k3dnx7vc5wcmayficjdsk02ix55va-woeusb-3.1.5/bin/woeusb --version and found version 3.1.5
ran /nix/store/8j0k3dnx7vc5wcmayficjdsk02ix55va-woeusb-3.1.5/bin/woeusb -h and found version 3.1.5
ran /nix/store/8j0k3dnx7vc5wcmayficjdsk02ix55va-woeusb-3.1.5/bin/woeusb --help and found version 3.1.5
found 3.1.5 with grep in /nix/store/8j0k3dnx7vc5wcmayficjdsk02ix55va-woeusb-3.1.5
directory tree listing: https://gist.github.com/a1e026683073657b8127fe93d50cdb18

cc @bjornfor @gnidorah for review

Applied to master (af53e7599798106904f07e217e3792482b8130b1). (I changed commit prefix from winusb to woeusb.)


Images with transparency imported as Texture2DArray result in white textures.

Godot version
v4.1.3.stable.official [f06b6836a]
System information
Godot v4.1.3.stable - Windows 10.0.22621 - Vulkan (Forward+) - dedicated NVIDIA GeForce RTX 2070 SUPER (NVIDIA; <IP_ADDRESS>13) - AMD Ryzen 9 3900X 12-Core Processor (24 Threads)
Issue description
Issue:
Attempting to import an image with transparency as a Texture2DArray only results in an array of white textures.
Expectation:
The array should consist of textures made from slices of the original image, with transparency intact.
Steps to reproduce
Reproduce from scratch:

Add an image with transparency (png/webp/tga) to your project.
Change the import settings of said image to Texture2DArray, press Reimport, and in the confirmation modal press Save Scenes, Re-Import, and Restart.
After Godot has restarted and reopened the project, view the image/Texture2DArray in the inspector, all the indices of the array display as white textures:

Reproduce using reproduction project:

Opening the project should cause the supplied images (icon_opaque.png and icon_transparent.png) to be imported as Texture2DArrays.
View the images/Texture2DArrays in the inspector.

Minimal reproduction project
Reproduction project:
texture_2d_array.zip
The project contains the following resources:

example.tscn

A scene containing a side by side comparison of two different Texture2DArrays, made from Godot logo images, displayed on two 2D quads.
The left quad (Correct behavior) displays a Texture2DArray made from an opaque Godot logo image
The right quad (Incorrect behavior) displays a Texture2DArray made from a Godot logo image with transparency.

icon_opaque.png

An opaque Godot logo image, imported as a Texture2DArray.

icon_transparent.png

A Godot logo image with transparency, imported as a Texture2DArray.

M_Opaque_Logo.tres

Material using S_TextureArray.gdshader, inputting icon_opaque.png to its Texture Array parameter.

M_Transparent_Logo.tres

Material using S_TextureArray.gdshader, inputting icon_transparent.png to its Texture Array parameter.

S_TextureArray.gdshader

Custom shader taking a Texture2DArray as a parameter and "restitch" it back to a single image. It makes the assumption that the supplied Texture2DArray is made from an image sliced in an 8x8 grid.

The issue seems to be with how Lossless/Lossy options save the data as WebP by default. The other options work fine.
WebP ends up saving layers that are fully opaque as just RGB, while layers with transparency will end up RGBA.
This causes an exception when then trying to load it as a TextureArray, since there are 2 different formats of texture stored.
I tried forcing RGBA format on the WebP encode, but it seems the WebP library itself does alpha channel scanning and discards it if it feels it's not necessary, even with Lossless compression...
As a workaround for now, in Project Settings > Rendering > Textures, you can select "Force PNG", and it will avoid this issue for the Lossless mode.

Thank you for the workaround!
Enabling "Force PNG", reimporting the Texture2DArray and reloading the project caused the right logo to be displayed as intended in the reproduction project:

I ran into the exact same problem and noticed one interesting detail: the import fails if at least one frame is fully opaque.
For example, this 8x8 array contains 26 opaque frames:

After adding at least one transparent pixel in all these frames, the import is successful.


Fix readthedocs formatting issues

As noticed by @pllim a recent PR (probably #1093) broke the readthedocs formatting. This PR attempts to fix.

Nope... See rendered doc at https://halotools--1095.org.readthedocs.build/en/1095/

Maybe you are not using the theme that you think you are using. Raw log might give you more info. Example: https://readthedocs.org/api/v2/build/25571238.txt

BTW, if you want to save CPU cycles for CI, you can put [ci skip] on your commit message. Actions respects that and won't run. RTD does not, so it would run anyway.

Looks good now. Glad you figured it out! 🎉
https://halotools.readthedocs.io/en/latest/

Yes! Mysteriously, it was just this one-liner: e94ce2aea7d87f21d830c98c93718451c40342e9. Thanks again for all the help @pllim!


Explain unique replaces chain requirement in channel sort errors.

The "multiple channel head" error that can be returned from channel
sorting has been a source of confusion for users. The error message
now explains that a unique replacement chain is required in order to
define the relative order between channel entries, and it also
provides an abbreviated list of all replacement chains identified, in
the form "head...tail" or "singleton".

/hold
This is a WIP, and may have also identified a bug with one of the new tests.

/hold cancel

@benluddy @kevinrizza Will it be cherry-picked to the downstream repo?  If yes, when? I checked the latest commits(1dc76f08ed05a635458420ffa979aebbe59a3890) of the downstream repo, but not found this PR.  I am really interested in the process of picking up upstream PRs for the downstream.  Could you help explain more about it? Or any doc? Thanks!


How can I use it with PDF instead of txt?

I'm trying to use this for PDF files but I don't see any PDF examples.

Thanks for your attention. Currently, we don't offer support for PDF files. However, you can extract the content using OCR and convert it into a TXT file. We will consider adding OCR support in future updates.

yeah, that's what I did XD, but it can be used with LLama 3.2, I'm trying and it just stays in Loop XD. Terminal: INFO:lightrag:Logger initialized for working directory: ./
DEBUG:lightrag:LightRAG init with param:
working_dir = ./,
chunk_token_size = 1200,
chunk_overlap_token_size = 100,
tiktoken_model_name = gpt-4o-mini,
entity_extract_max_gleaning = 1,
entity_summary_to_max_tokens = 500,
node_embedding_algorithm = node2vec,
node2vec_params = {'dimensions': 1536, 'num_walks': 10, 'walk_length': 40, 'window_size': 2, 'iterations': 3, 'random_seed': 3},
embedding_func = {'embedding_dim': 384, 'max_token_size': 5000, 'func': <function  at 0x7936c8cfa9e0>},
embedding_batch_num = 32,
embedding_func_max_async = 16,
llm_model_func = <function hf_model_complete at 0x7936c95ac4c0>,
llm_model_name = meta-llama/Llama-3.2-1B-Instruct,
llm_model_max_token_size = 32768,
llm_model_max_async = 16,
key_string_value_json_storage_cls = <class 'lightrag.storage.JsonKVStorage'>,
vector_db_storage_cls = <class 'lightrag.storage.NanoVectorDBStorage'>,
vector_db_storage_cls_kwargs = {},
graph_storage_cls = <class 'lightrag.storage.NetworkXStorage'>,
enable_llm_cache = True,
addon_params = {},
convert_response_to_json_func = <function convert_response_to_json at 0x7936c95960e0>
INFO:lightrag:Load KV full_docs with 0 data
INFO:lightrag:Load KV text_chunks with 0 data
INFO:lightrag:Load KV llm_response_cache with 0 data
INFO:lightrag:[New Docs] inserting 1 docs
INFO:lightrag:[New Chunks] inserting 24 chunks
INFO:lightrag:Inserting 24 vectors to chunks
/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils_base.py:1601: FutureWarning: clean_up_tokenization_spaces was not set. It will be set to True by default. This behavior will be depracted in transformers v4.45, and will be then set to False by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884
warnings.warn(
INFO:lightrag:[Entity Extraction]...
/usr/local/lib/python3.10/dist-packages/transformers/generation/configuration_utils.py:615: UserWarning: num_beams is set to 1. However, early_stopping is set to True -- this flag is only used in beam-based generation modes. You should set num_beams>1 or unset early_stopping.
warnings.warn(
Setting pad_token_id to eos_token_id:128001 for open-end generation.
Setting pad_token_id to eos_token_id:128001 for open-end generation.
WARNING:accelerate.big_modeling:Some parameters are on the meta device because they were offloaded to the cpu.
Setting pad_token_id to eos_token_id:128001 for open-end generation.
/usr/local/lib/python3.10/dist-packages/transformers/generation/configuration_utils.py:615: UserWarning: num_beams is set to 1. However, early_stopping is set to True -- this flag is only used in beam-based generation modes. You should set num_beams>1 or unset early_stopping.
warnings.warn(
Setting pad_token_id to eos_token_id:128001 for open-end generation.
WARNING:accelerate.big_modeling:Some parameters are on the meta device because they were offloaded to the cpu.
Setting pad_token_id to eos_token_id:128001 for open-end generation.
Setting pad_token_id to eos_token_id:128001 for open-end generation.
I'll leave the code here:
`from lightrag.llm import hf_model_complete, hf_embedding
from transformers import AutoModel, AutoTokenizer
from lightrag import LightRAG, QueryParam
from lightrag.utils import EmbeddingFunc
import os
import sys
import nest_asyncio  # Import nest_asyncio
WORKING_DIR = "./"
Apply nest_asyncio to allow nested event loops
nest_asyncio.apply()
Initialize LightRAG with Hugging Face model
rag = LightRAG(
working_dir=WORKING_DIR,
llm_model_func=hf_model_complete,  # Use Hugging Face model for text generation
llm_model_name='meta-llama/Llama-3.2-1B-Instruct',  # Model name from Hugging Face
# Use Hugging Face embedding function
embedding_func=EmbeddingFunc(
embedding_dim=384,
max_token_size=5000,
func=lambda texts: hf_embedding(
texts,
tokenizer=AutoTokenizer.from_pretrained("sentence-transformers/all-MiniLM-L6-v2"),
embed_model=AutoModel.from_pretrained("sentence-transformers/all-MiniLM-L6-v2")
)
),
)
with open("./Ley-de-Bienestar-Animal.txt") as f:
rag.insert(f.read())
Perform naive search
print(rag.query("Haz un resumen de las leyes", param=QueryParam(mode="naive")))
Perform local search
#print(rag.query("Haz un resumen de las leyes", param=QueryParam(mode="local")))
Perform global search
#print(rag.query("Haz un resumen de las leyes", param=QueryParam(mode="global")))
Perform hybrid search
#print(rag.query("Haz un resumen de las leyes", param=QueryParam(mode="hybrid")))`

and if I comment this line: #nest_asyncio.apply()
I get this error: INFO:lightrag:Logger initialized for working directory: ./
DEBUG:lightrag:LightRAG init with param:
working_dir = ./,
chunk_token_size = 1200,
chunk_overlap_token_size = 100,
tiktoken_model_name = gpt-4o-mini,
entity_extract_max_gleaning = 1,
entity_summary_to_max_tokens = 500,
node_embedding_algorithm = node2vec,
node2vec_params = {'dimensions': 1536, 'num_walks': 10, 'walk_length': 40, 'window_size': 2, 'iterations': 3, 'random_seed': 3},
embedding_func = {'embedding_dim': 384, 'max_token_size': 5000, 'func': <function  at 0x7f82cc1df7f0>},
embedding_batch_num = 32,
embedding_func_max_async = 16,
llm_model_func = <function hf_model_complete at 0x7f81fb9b91b0>,
llm_model_name = meta-llama/Llama-3.2-1B-Instruct,
llm_model_max_token_size = 32768,
llm_model_max_async = 16,
key_string_value_json_storage_cls = <class 'lightrag.storage.JsonKVStorage'>,
vector_db_storage_cls = <class 'lightrag.storage.NanoVectorDBStorage'>,
vector_db_storage_cls_kwargs = {},
graph_storage_cls = <class 'lightrag.storage.NetworkXStorage'>,
enable_llm_cache = True,
addon_params = {},
convert_response_to_json_func = <function convert_response_to_json at 0x7f81fb99add0>
INFO:lightrag:Load KV full_docs with 0 data
INFO:lightrag:Load KV text_chunks with 0 data
INFO:lightrag:Load KV llm_response_cache with 0 data
RuntimeError                              Traceback (most recent call last)
 in <cell line: 31>()
30
31 with open("./Ley-de-Bienestar-Animal.txt") as f:
---> 32     rag.insert(f.read())
33
34 # Perform naive search
2 frames
/usr/lib/python3.10/asyncio/base_events.py in _check_running(self)
582     def _check_running(self):
583         if self.is_running():
--> 584             raise RuntimeError('This event loop is already running')
585         if events._get_running_loop() is not None:
586             raise RuntimeError(
RuntimeError: This event loop is already running

I'm trying to use this for PDF files but I don't see any PDF examples.

yeah so am i working on this while ensuring the LLM can understand inherent math and code logic within these PDFs  in the database

and if I comment this line: #nest_asyncio.apply()
I get this error: INFO:lightrag:Logger initialized for working directory: ./ DEBUG:lightrag:LightRAG init with param: working_dir = ./, chunk_token_size = 1200, chunk_overlap_token_size = 100, tiktoken_model_name = gpt-4o-mini, entity_extract_max_gleaning = 1, entity_summary_to_max_tokens = 500, node_embedding_algorithm = node2vec, node2vec_params = {'dimensions': 1536, 'num_walks': 10, 'walk_length': 40, 'window_size': 2, 'iterations': 3, 'random_seed': 3}, embedding_func = {'embedding_dim': 384, 'max_token_size': 5000, 'func': <function at 0x7f82cc1df7f0>}, embedding_batch_num = 32, embedding_func_max_async = 16, llm_model_func = <function hf_model_complete at 0x7f81fb9b91b0>, llm_model_name = meta-llama/Llama-3.2-1B-Instruct, llm_model_max_token_size = 32768, llm_model_max_async = 16, key_string_value_json_storage_cls = <class 'lightrag.storage.JsonKVStorage'>, vector_db_storage_cls = <class 'lightrag.storage.NanoVectorDBStorage'>, vector_db_storage_cls_kwargs = {}, graph_storage_cls = <class 'lightrag.storage.NetworkXStorage'>, enable_llm_cache = True, addon_params = {}, convert_response_to_json_func = <function convert_response_to_json at 0x7f81fb99add0>
INFO:lightrag:Load KV full_docs with 0 data
INFO:lightrag:Load KV text_chunks with 0 data
INFO:lightrag:Load KV llm_response_cache with 0 data
RuntimeError Traceback (most recent call last) in <cell line: 31>() 30 31 with open("./Ley-de-Bienestar-Animal.txt") as f: ---> 32 rag.insert(f.read()) 33 34 # Perform naive search
2 frames /usr/lib/python3.10/asyncio/base_events.py in _check_running(self) 582 def _check_running(self): 583 if self.is_running(): --> 584 raise RuntimeError('This event loop is already running') 585 if events._get_running_loop() is not None: 586 raise RuntimeError(
RuntimeError: This event loop is already running

Could you provide more details about the specific runtime environment?

and if I comment this line: #nest_asyncio.apply()
I get this error: INFO:lightrag:Logger initialized for working directory: ./ DEBUG:lightrag:LightRAG init with param: working_dir = ./, chunk_token_size = 1200, chunk_overlap_token_size = 100, tiktoken_model_name = gpt-4o-mini, entity_extract_max_gleaning = 1, entity_summary_to_max_tokens = 500, node_embedding_algorithm = node2vec, node2vec_params = {'dimensions': 1536, 'num_walks': 10, 'walk_length': 40, 'window_size': 2, 'iterations': 3, 'random_seed': 3}, embedding_func = {'embedding_dim': 384, 'max_token_size': 5000, 'func': <function at 0x7f82cc1df7f0>}, embedding_batch_num = 32, embedding_func_max_async = 16, llm_model_func = <function hf_model_complete at 0x7f81fb9b91b0>, llm_model_name = meta-llama/Llama-3.2-1B-Instruct, llm_model_max_token_size = 32768, llm_model_max_async = 16, key_string_value_json_storage_cls = <class 'lightrag.storage.JsonKVStorage'>, vector_db_storage_cls = <class 'lightrag.storage.NanoVectorDBStorage'>, vector_db_storage_cls_kwargs = {}, graph_storage_cls = <class 'lightrag.storage.NetworkXStorage'>, enable_llm_cache = True, addon_params = {}, convert_response_to_json_func = <function convert_response_to_json at 0x7f81fb99add0>
INFO:lightrag:Load KV full_docs with 0 data
INFO:lightrag:Load KV text_chunks with 0 data
INFO:lightrag:Load KV llm_response_cache with 0 data
RuntimeError Traceback (most recent call last) in <cell line: 31>() 30 31 with open("./Ley-de-Bienestar-Animal.txt") as f: ---> 32 rag.insert(f.read()) 33 34 # Perform naive search
2 frames /usr/lib/python3.10/asyncio/base_events.py in _check_running(self) 582 def _check_running(self): 583 if self.is_running(): --> 584 raise RuntimeError('This event loop is already running') 585 if events._get_running_loop() is not None: 586 raise RuntimeError(
RuntimeError: This event loop is already running

Are you running your code in Jupyter notebook? If so, do not comment  #nest_asyncio.apply().
This function is for 1. Allowing the execution of asynchronous code in Jupyter Notebook.& 2.Solving the problems that arise when initiating a new event loop within an existing event loop.

Yeah it was in Jupyter Notebook, but it always stays in the loop, and never responds

Maybe you can use a library such as Marker to parse the document first? Then pass the text output to LightRAG


"Logs/TrainingData.csv" has confusing naming

The name sounds like it logs events, when it actually just logs loss over time
A more suitable naming probably exists

Perhaps making the name a part of the config file would solve this?


Refactoring of many things

Trust me. This is better.
Probably still a few mistakes in the re-factoring.
You guys have really gotta give contributes a way to setup the database

No - why delete OpenSprites is Illuminati.mp3? :(

Lets not be hasty now. ;)

@The-Cheese-Eater Thanks.
Overall, this repo is really just cluttered and unorganized. (Sorry about admitting this)
This is really just one of my first PR's. I plan on creating separate classes for all functions for database access and user functions. Probably half the size of the repo while doing so.
@liam4 So..
I, and others, dislike your attitude.

Yep. Bad idea ever asking to join any Scratch based community..
People are full of prejudice here :(
Forget my issues and this PR request

It is with that attitude.
Bye then.

@robinp7720 Your refactor and clean up is great except for one thing: you formatted library code which should have stayed minified. Just make sure to format only non-library code and we could accept your PR.
And yes, a lot of the code is messy and everything. Classes might be nice but using classes in PHP is kind of annoying.

I also am the only experienced php programmer here (As far as I can tell). You want me, trust me.

Let's not jump to conclusions. Remember that you're trying to make a case for why your PR should be accepted. Don't assume that we want your code really badly or anything, because we don't.

thx


Overflow from RFC

I realize the purpose of this project is lean, commonplace useful RFC actions candidates for Svelte inclusion. Personally I think this would be great. However I think it is also an opportunity to take svelte actions submissions which aren't appropriate for inclusion into svelte for various reasons (too specific, too custom, too large) and then they would break out their own searchable curated list. For example, I am writing an action which allows you to click images and they open up in a modal showing the image fullscreen. Considering press to zoom also. IMO, surely too custom for RFC consideration, but probably would be useful to someone else.
Something like Svelte Showcase but for actions only.

interesting, i'd want to centralize those in the Showcase tho, for maximum discoverability? but still please feel free to link them here and i'll do my best to increase the visibility wherever appropriate


Add --mcdc flag, and handle MC/DC fields correctly in LLVM JSON

This adds a new unstable --mcdc flag, similar to --branch.
Apart from that, the LLVM JSON parser also needed changes to recognize the mcdc-related fields, as it would otherwise silently drop those.

Clippy now rightfully complains that Workspace::new takes too many boolean parameters, so there surely is some more opportunity for improvement.

CI is segfaulting somewhere in the LLVM bindings, probably because mcdc support is broken in the compiler right now because of llvm 19 changes:
https://github.com/rust-lang/rust/issues/126672 / https://github.com/rust-lang/rust/pull/126733
I had no problem running things locally with a slightly older nightly (presumably pre llvm-19 update).
I’m okay with holding off until this is properly fixed in the compiler.

(https://github.com/rust-lang/rust/pull/126733 has been merged and will be available in the next nightly.)

Published in 0.6.14. Thanks @Swatinem!


[Code style] .idea/codestyles/Project.xml occasionally changes by itself, blocking pull/merge

@shobhitagarwal1612, is allowing this file in git recommended, or is there another/better way to share the Android Studio configs?

There is an option of settings repository in Android Studio for sharing settings across the team.
https://www.jetbrains.com/help/idea/sharing-your-ide-settings.html#settings-repository
I can take a first stab at this.

Great, thanks for taking a look!

Hi @gino-m
I tried creating a sample settings repository and it worked as expected. (Also sent you an invitation to check out the repo)
So, the next steps should be:

Create a new repository (preferably private) where we'll keep our IDE settings.
Enable settings repository under Preferences > Plugin
Set the URL from step 1 under File > Settings Repository > Upstream URL. Then select 'Overwrite Local'

From this point onwards, the plugin should auto update the local settings whenever there is an update.

@gino-m Please ignore my last comment, it is still in progress. Some files were excluded during the sync which are a part of codeStyle.

If we save the codestyle in the IDE rather than project then it works!
https://github.com/shobhitagarwal1612/test_settings_repository/blob/master/codestyles/ground-android.xml
To apply the settings:
Preferences > Editor > Code Style > Click of "Scheme" drop-down > Select "ground-android" under Stored in IDE.
@gino-m
I have pushed the changes to my test settings repository. To test the changes:

Create a fresh clone of the project for testing
File > Settings Repository > Set url: https://github.com/shobhitagarwal1612/test_settings_repository/  and click overwrite local
Then select "ground-android" as preferred code style under Preferences (steps mentioned above)

We spoke - we can't rely on repos owned by individuals or other orgs, and the inconvenience this issue causes is minor, so I think we will skip this one for now. Thanks for looking into it!


I cant run

Hello, I am quite a novice. I follow the commands but I can't run them. What should I do? sorry for taking your time

Seems you did not run npm install first.

Hello, thank you for your answer.
Things I did:

I download the file and open the folder it is in with visual studio code.
i open terminal in folder
"npm install"
"npm run build"
"cd public"
"npm install"
and I close the terminal
I run windows live server and the result is:

i just want to make it executable and i am very newbie.

The simplest way is mentioned in README - just run npm run serve after npm install. It will run node.js built-in HTTP server. If you want to run it standalone, you just need to serve dist directory content by some HTTP server. The dist directory is created by npm run build. As I see from your screenshot, you are trying to serve public directory content which is wrong. This is directory with source files, not build product.


Truncate breaks with some html tags

the tag <input type="file" id="file" multiple /> specifically is known to break the truncate class throwing a Error when rendering template... error when the tag is displayed inline (rather than embedding)
Don't know if it's possible to assign me to this but I'm looking at it.

thanks!

Any more progress thoughts on this?  Is the issue that the 'summary' is truncating in the middle of a juicy HTML tag?  Is a simple fix to have a custom summary delimited by the summary marker above the form?

It's a really elusive issue to say the least. Some tags display and render while others break it. This is if they are within the summary length (usually first 200 characters).
Known breaking tags
<input>
<canvas>
Though in recent testing it seems to have been fixed. I've been unable to reproduce it since Thursday evening.

What is the usage, where is it truncated? In the page summary?
It should use Utils::truncateHtml().
Can you replicate it on tests? I tried https://github.com/getgrav/grav/commit/a683f661ed46a5f873bca5298a58ea7adcda554f which pass

Alright, so the issue is with code displaying by using tab (still only select elements)
---
title : Input Truncate Fail
---

# Input Fails

    <input type="file" id="file" multiple></input>

Regarding your test Flavio, that passes as it should. When it's not indented in, there is no problem.


Blackjack/casino auto-advances through menus

Environment

OS version: w7
Elona Foobar verision: 0.2.6

Details
Blackjack does not allow choices, instead the game seems to press Enter immediately, thus shutting the player out. Mashing Down Arrow has a chance to disrupt this, allowing a player to choose a bet. However, this fails after cards are dealt. You will always choose Stay. Since the AI cheats I chose to grab Lucky Day to see how far I could go with this glitch. Quite amusing.
A terrible video of it in action
https://mega.nz/#!69JHnAJb!Lm6nC7esXXDdjgBYPuR0pp6cThIsxc5-2etaoOnyrRI
How to reproduce
Enter blackjack/casino

Seems to be caused by the enter key not being delayed in all circumstances. I can fix this.


Jon/webhook headers

Adds support for header and payload definitions for alert channels. Also supports the use of kubernetes secrets as values for headers.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Update meta.yaml for 1.1.3 (#24)

update version and hash according to https://pypi.org/project/deepforest/1.1.3/

Checklist

[x] Used a personal fork of the feedstock to propose changes
[ ] Bumped the build number (if the version is unchanged)
[ ] Reset the build number to 0 (if the version changed)
[ ] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[ ] Ensured the license file is being packaged.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.


Improved handling of format specifications

Handling of field width for %s/%S and %d/%D
Corrected handling of field width and padding for %h/%H
Handling of undefined bits for %d/%D and %h/%H
Handling of %x/%X as alias for %h/%H
Handling of %o/%O and %b/%B

I wasn't aware of #2459 when I wrote this.


docs: Fix limit swap example

Proposed fix for limit swap example

:tada: This PR is included in version 2.10.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Deisctl Cluster Health

As a pre-req of the update service client, deisctl requires a means by which to quickly determine if the cluster is in a consistent state.
This could make use of the deis version work, and should also determine if the clusters generally healthy enough for update (ceph, dead components, etcd quorum, etc).
The result should be useful to run by a human as a deisctl command, as well as for the update client to consume.

This is not something we'll be able to get to for the LTS release (#4776) and the architecture in deis v2 has dramatically changed such that this is no longer relevant (deisctl no longer exists) so I'm closing this.