Django connector seems broken

From discord, from @paulzakin:
error connecting in 'pool-1': connection failed: could not receive data from server: Connection refused
error connecting in 'pool-1': connection failed: could not receive data from server: Connection refused
error connecting in 'pool-1': connection failed: could not receive data from server: Connection refused
error connecting in 'pool-1': connection failed: could not receive data from server: Connection refused
Traceback (most recent call last):
  File "/Users/code/server/worker.py", line 39, in <module>
    main()
  File "/Users/code/server/worker.py", line 35, in main
    x.run_worker()  # pyright: ignore [reportUnknownMemberType]
    ^^^^^^^^^^^^^^
  File "/Users/code/.venv/lib/python3.11/site-packages/procrastinate/app.py", line 270, in run_worker
    asyncio.run(f())
  File "/Users/foo/.local/share/mise/installs/python/3.11.7/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/Users/foo/.local/share/mise/installs/python/3.11.7/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/Users/foo/.local/share/mise/installs/python/3.11.7/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/code/.venv/lib/python3.11/site-packages/procrastinate/app.py", line 267, in f
    async with self.open_async():
  File "/Users/code/.venv/lib/python3.11/site-packages/procrastinate/utils.py", line 194, in __aenter__
    await self._open_coro()
  File "/Users/code/.venv/lib/python3.11/site-packages/procrastinate/psycopg_connector.py", line 170, in open_async
    await self._async_pool.open(wait=True)  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/code/.venv/lib/python3.11/site-packages/psycopg_pool/pool_async.py", line 382, in open
    await self.wait(timeout=timeout)
  File "/Users/code/.venv/lib/python3.11/site-packages/psycopg_pool/pool_async.py", line 169, in wait
    raise PoolTimeout(f"pool initialization incomplete after {timeout} sec")
psycopg_pool.PoolTimeout: pool initialization incomplete after 30.0 sec

I invoke the entry with poetry run python server/worker.py
And the entry looks like this

import django

django.setup()

import procrastinate
from procrastinate.contrib.django import app

from server.env import EnvService
from server.services.db import ProcrastinateService

ENV = EnvService.get_env()
env = EnvService.get_env_variables(ENV)

connector = procrastinate.PsycopgConnector(
    kwargs={
        "host": env["POSTGRES_HOST"],
        "user": env["POSTGRES_USER"],
        "password": env["POSTGRES_PASSWORD"],
        "port": env["POSTGRES_PORT"],
        "dbname": env["POSTGRES_DB"],
        **EnvService.get_ssl(),
    }
)

def main() -> None:
    x = app.with_connector(connector)
    blueprints = ProcrastinateService.get_blueprints(show_warnings=True)
    for blueprint, namespace in blueprints:
        x.add_tasks_from(blueprint, namespace=namespace)
    x.run_worker()  # pyright: ignore [reportUnknownMemberType]

if __name__ == "__main__":
    main()

And the way I used to run it was poetry run procrastinate --app=server.worker.app worker
And I used to define app = procrastinate.App(connector=connector) in the settings.py basically and then import that into worker.py
Even more interesting: when all I do is change the version to v2 and leave all the code as it is (no django stuff changed), app defined in settings.py and then invoked in worker.py, I get that same error pool initialization incomplete after 30.0 sec.

So (my guess) when I invoke poetry run procrastinate --app=server.worker.app worker --concurrency=1 with the imported app on the new version, something changed.

Suggestion: could it be that you're hitting this ? What version of psycopg are you using ?

I'm trying the following script:
from __future__ import annotations

import django

django.setup()

from os import environ

import procrastinate
from procrastinate.contrib.django import app

connector = procrastinate.PsycopgConnector(
    kwargs={
        "host": environ["POSTGRES_HOST"],
        "user": environ["POSTGRES_USER"],
        "password": environ["POSTGRES_PASSWORD"],
        "port": environ["POSTGRES_PORT"],
        "dbname": environ["POSTGRES_DB"],
    }
)

def main() -> None:
    x = app.with_connector(connector)
    x.run_worker()  # pyright: ignore [reportUnknownMemberType]

if __name__ == "__main__":
    main()

And it works as expected with the following envvars:
export POSTGRES_HOST=$PGHOST POSTGRES_USER=$PGUSER POSTGRES_PASSWORD=$PGPASSWORD POSTGRES_PORT=5432 POSTGRES_DB=$PGDATABASE DJANGO_SETTINGS_MODULE=procrastinate_demos.demo_django.project.settings

(from the dev env, so PG variables are set to match the docker connection defined here.

OK this is great, I can look at this on Monday! I am using 3.1.18. What are you using? This will be a good way for us to sanity check.

I'll triple check bit I'm using the dev env so all my versions are following the poetry lock file.

Yep, 3.1.18

Ok, very odd. Now when I install 2.0.0b1 and test it out, I get this error
Traceback (most recent call last):
  File "/foo/erms/manage.py", line 23, in <module>
    main()
  File "/foo/erms/manage.py", line 19, in main
    execute_from_command_line(sys.argv)
  File "/foo/erms/.venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/foo/erms/.venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/foo/erms/.venv/lib/python3.11/site-packages/django/core/management/base.py", line 413, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/foo/erms/.venv/lib/python3.11/site-packages/django/core/management/base.py", line 459, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/foo/erms/.venv/lib/python3.11/site-packages/django/core/management/base.py", line 107, in wrapper
    res = handle_func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/foo/erms/.venv/lib/python3.11/site-packages/django/core/management/commands/migrate.py", line 117, in handle
    executor = MigrationExecutor(connection, self.migration_progress_callback)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/foo/erms/.venv/lib/python3.11/site-packages/django/db/migrations/executor.py", line 18, in __init__
    self.loader = MigrationLoader(self.connection)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/foo/erms/.venv/lib/python3.11/site-packages/django/db/migrations/loader.py", line 58, in __init__
    self.build_graph()
  File "/foo/erms/.venv/lib/python3.11/site-packages/django/db/migrations/loader.py", line 276, in build_graph
    self.graph.validate_consistency()
  File "/foo/erms/.venv/lib/python3.11/site-packages/django/db/migrations/graph.py", line 198, in validate_consistency
    [n.raise_error() for n in self.node_map.values() if isinstance(n, DummyNode)]
  File "/foo/erms/.venv/lib/python3.11/site-packages/django/db/migrations/graph.py", line 198, in <listcomp>
    [n.raise_error() for n in self.node_map.values() if isinstance(n, DummyNode)]
     ^^^^^^^^^^^^^^^
  File "/foo/erms/.venv/lib/python3.11/site-packages/django/db/migrations/graph.py", line 60, in raise_error
    raise NodeNotFoundError(self.error_message, self.key, origin=self.origin)
django.db.migrations.exceptions.NodeNotFoundError: Migration procrastinate.0025_initial dependencies reference nonexistent parent node ('procrastinate', '0024_job_id_bigint')

It creates a <procrastinate migrations virtual path> folder in the root with migration 0025, which does not seem right.
Is this a weird thing on my machine?

And looking at https://github.com/procrastinate-org/procrastinate/releases/tag/2.0.0b1 - it looks like it is behind on main but a bunch of commits - is it force cutting a new beta release maybe with those commits?

And as a sanity check, I re-cloned out repo and resinstalled everything with v1.1.2 and that worked fine.

I’m not on my computer to flesh out a complete answer but I’m pretty sure it’s a __pycache__ issue. We should open a different issue for that

Ya, no rush obviously - 1.1.2 is working great currently!

Ok, the migration issue is solved in https://github.com/procrastinate-org/procrastinate/pull/941.
Now back to the connection issue. Anything you can share to help investigation is highly appreciated :)

And looking at 2.0.0b1 (release) - it looks like it is behind on main but a bunch of commits - is it force cutting a new beta release maybe with those commits?

Sure ! I thought you had mentionned that it was the same to you testing on main and testing on a tag. Either way work for me.
https://github.com/procrastinate-org/procrastinate/releases/tag/2.0.0b2

BTW, instead of:
connector = procrastinate.PsycopgConnector(
    kwargs={
        "host": environ["POSTGRES_HOST"],
        "user": environ["POSTGRES_USER"],
        "password": environ["POSTGRES_PASSWORD"],
        "port": environ["POSTGRES_PORT"],
        "dbname": environ["POSTGRES_DB"],
    }
)
x = app.with_connector(connector)

it might be worth trying:
x = app.with_connector(app.connector.get_worker_connector())

OK, excited to debug this!
Dumb question - I don't see a release on pypi for the next version?
Also, what is the difference between pypi and "main" of procrastinate. If it is the same I can just have poetry install from a git tag, rather than pypi, but I'd like to be sure :)

Awesome, I'll check this out tomorrow now

Traceback (most recent call last):
  File "/Users/foo/manage.py", line 23, in <module>
    main()
  File "/Users/foo/manage.py", line 19, in main
    execute_from_command_line(sys.argv)
  File "/Users/foo/.venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/Users/foo/.venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 416, in execute
    django.setup()
  File "/Users/foo/.venv/lib/python3.11/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/Users/foo/.venv/lib/python3.11/site-packages/django/apps/registry.py", line 91, in populate
    app_config = AppConfig.create(entry)
                 ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/foo/.venv/lib/python3.11/site-packages/django/apps/config.py", line 123, in create
    mod = import_module(mod_path)
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/.local/share/mise/installs/python/3.11.7/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/foo/.venv/lib/python3.11/site-packages/procrastinate/contrib/django/apps.py", line 12, in <module>
    from . import django_connector
  File "/Users/foo/.venv/lib/python3.11/site-packages/procrastinate/contrib/django/django_connector.py", line 13, in <module>
    from procrastinate.contrib.aiopg import aiopg_connector
  File "/Users/foo/.venv/lib/python3.11/site-packages/procrastinate/contrib/aiopg/__init__.py", line 3, in <module>
    from .aiopg_connector import AiopgConnector
  File "/Users/foo/.venv/lib/python3.11/site-packages/procrastinate/contrib/aiopg/aiopg_connector.py", line 8, in <module>
    import aiopg
ModuleNotFoundError: No module named 'aiopg'

Are you expected to install aiopg to get this to work? I thought we were switching to psycopg3 because it has a good async interface? I'm happy to install aiopg, just wanted to check.

want to point out 2 things:

from procrastinate.contrib.django import app throws an error, no variable named "app" is exported, there is "apps" though
from procrastinate.contrib.aiopg import AiopgConnector will throw a confusing error that aiopg module is not found, which made me think that it's procrastinate aiopg but in reality installing aiopg separately made it work

@paulzakin

Are you expected to install aiopg to get this to work?

Ah indeed, I could have made it clearer in the docs, and you're right that it might not be the best choice:
It's all in https://github.com/procrastinate-org/procrastinate/blob/main/procrastinate/contrib/django/django_connector.py#L223
I've added the psycopg3 compat but I haven't removed Aiopg, however flawed may it be. When determining which one to use, I look at the is_psycopg3 bool from Django that tells me whether Django is configured to use psycopg2 or 3. If psycopg2, I use the psycopg2-based Aiopg, if psycopg3, I use psycopg3. If your procrastinate is aiming for aiopg, it means you're using a pre-psycopg3 django version.
But then, given that people might have psycopg2 installed but not aiopg, and that I'm going to make a different connection altogether, maybe I should rather:

try to see which one of psycopg3 and aiopg is installed and use it
provide a parameter to choose explicitly

I'll fix this and make it clearer in the docs.

@hyusetiawan your comment makes me think that you're probably using the last stable release v1.1.2, while this issue is (sorry, unclearly) about the v2 beta.
As far as I can tell, the docs are configured to still point to 1.1.2, so maybe you're looking at the docs on the main branch ?
That being said, you're more than welcome to try and integrate v2.0.0b2 (it's worth re-reading the Django docs completely) and report any problem here. The v2 had the django integration completely re-done, and in theory it should be much smoother. That said, @paulzakin can attest that so far we're still experiencing road bumps.

ah okay, apologies for the confusion! it's stated in the title even that it's for v2 beta :)

Well... It was not stated on the title at the time of your comment :)

v2.0.0b3 is out

Just tested - no dice :(
Traceback (most recent call last):
  File "/foo/manage.py", line 23, in <module>
    main()
  File "/foo/manage.py", line 19, in main
    execute_from_command_line(sys.argv)
  File "/foo/.venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/foo/.venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 416, in execute
    django.setup()
  File "/foo/.venv/lib/python3.11/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/foo/.venv/lib/python3.11/site-packages/django/apps/registry.py", line 91, in populate
    app_config = AppConfig.create(entry)
                 ^^^^^^^^^^^^^^^^^^^^^^^
  File "/foo/.venv/lib/python3.11/site-packages/django/apps/config.py", line 123, in create
    mod = import_module(mod_path)
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/foo/.local/share/mise/installs/python/3.11.7/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/foo/.venv/lib/python3.11/site-packages/procrastinate/contrib/django/apps.py", line 12, in <module>
    from . import django_connector
  File "/foo/.venv/lib/python3.11/site-packages/procrastinate/contrib/django/django_connector.py", line 13, in <module>
    from procrastinate.contrib.aiopg import aiopg_connector
  File "/foo/.venv/lib/python3.11/site-packages/procrastinate/contrib/aiopg/__init__.py", line 3, in <module>
    from .aiopg_connector import AiopgConnector
  File "/foo/.venv/lib/python3.11/site-packages/procrastinate/contrib/aiopg/aiopg_connector.py", line 8, in <module>
    import aiopg
ModuleNotFoundError: No module named 'aiopg'

Haha. I forgot to plug the good ol' brain.
If you look at the diff, it's facepalm material.

https://github.com/procrastinate-org/procrastinate/releases/tag/2.0.0b4
https://pypi.org/project/procrastinate/2.0.0b4/

No worries - I'll look at tomorrow :)

Ok, good news! We have passed the aiopg problem. Bad news, other errors!
Running code like this leads
from procrastinate.contrib.django import app
app.configure_task(name="foo", queueing_lock="bar").defer(report_id="baz")

Leads to this error:
'Blueprint' object has no attribute 'configure_task'
And I used the  PROCRASTINATE_ON_APP_READY method to load the blueprints. I know it is being run, because FOO is being printed in the terminal but no dice after that.
def on_app_ready(app: procrastinate.App) -> None:
    print("FOO")
    blueprints = ProcrastinateService.get_blueprints(show_warnings=True)
    for blueprint, namespace in blueprints:
        app.add_tasks_from(blueprint, namespace=namespace)

Also, in the documentation: https://procrastinate.readthedocs.io/en/main/howto/django.html. I get an error with the load_tasks_from method that it does not exist. Are you sure it exists?
import procrastinate
def on_app_ready(app: procrastinate.App):
    app.load_tasks_from(some_blueprint)

Either way, forward progress!

Are you sure it exists?

Woops, add_tasks_from. Good catch.

'Blueprint' object has no attribute 'configure_task'

This one is interesting. I'm willing to bet this is happening when you're loading tasks at import time (so even before Django has setup everything). At this point, the DB isn't ready yet.
This is probably something I should make clearer in the doc:

When you do something with Django, here's what happens:

You launch some kind of script. Either manage.py (which has DJANGO_SETTINGS_MODULE hardcoded), or wsgi.py/asgi.py (same) or django-admin (which is equivalent and reads it from your env) or you launch your own script which calls django.setup() which reads DJANGO_SETTINGS_MODULE from the environment.
This all comes down to calling django.setup() which will import the settings defined at DJANGO_SETTINGS_MODULE. Because of how python imports work, importing x.y.z will import x/__init__.py, x/y/__init__.py and then x/y/z.py. If those __init__.py have imports, they'll be imported too
Once Django has read the settings, it takes INSTALLED_APPS and imports all the corresponging code. Here again, a lot of code might be imported by virtue of being referenced in the __init__.py of a parent file of an app path referenced in INSTALLED_APPS. For each app, Django loads apps.py inside the path
Everything is setup internally
Finally, Django launches each app's AppConfig.ready() method. At this point, everything is usable and it's your chance to execute some code "on startup".
And then django.setup() is finished, and whatever needs to happen next happens. This might be calling a management command, launching a wsgi server, calling your custom script etc.

Everything that wants to do DB calls using the Django DB before app.ready() (at import time) may or may not crash, because Django hasn't started yet (Runtime error: App registry isn't ready yet.)
As part of the Procrastinate django refactor, I'm defining procrastinate.contrib.django.app. It felt right to define and open the app in <procrastinate AppConfig>.ready(). But I thought people would be trying to use the app at import time. While deferring tasks at import time should be discouraged (for the reason mentioned above), registering blueprints at import time makes sense, so I made the choice that procrastinate.contrib.django.app is initialized as a Blueprint, and in <procrastinate AppConfig>.ready(), I'm creating a new app that loads tasks from this blueprint, and then replacing the instance at procrastinate.contrib.django.app with the new app. I thought it was likely to be seamless.
Of course, I was wrong :D
I see multiple paths here:

Documenting this and considering people will read the doc
Using a special Blueprint that would make a better error message if we try to configure jobs on it at import time
Use a special object that doesn't even act like a Blueprint, and instructs people who want to do anything to the app at import time to do it either in their own AppConfig.ready() or in  PROCRASTINATE_ON_APP_READY

I'm tempted by solution 2. Any opinion?

I think #2 is the best solution for sure, agree with you!

https://procrastinate.readthedocs.io/en/main/howto/django.html#startup-actions
https://github.com/procrastinate-org/procrastinate/releases/tag/2.0.0b5
https://pypi.org/project/procrastinate/2.0.0b5/

Ok, I'll check this out on Monday - forward progress!

So it is odd - everything works, except when I defer a task, it says it is not found. But the task exists!
# {'foo_study:healthz': <procrastinate.tasks.Task object at 0x10b029810>}
print(app.tasks)
if report.task_name:
  app.configure_task(name="foo_study:healthz", queueing_lock="foo_study:healthz").defer(foo=1)

And here is the error from the running worker
Launching a worker on all queues
Starting worker on all queues
Task was not found:
    Task cannot be imported.

Traceback (most recent call last):
  File "/Users/foo/.venv/lib/python3.11/site-packages/procrastinate/worker.py", line 236, in find_task
    return self.app.tasks[task_name]
           ~~~~~~~~~~~~~~^^^^^^^^^^^
KeyError: 'foo_study:healthz'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/foo/.venv/lib/python3.11/site-packages/procrastinate/worker.py", line 196, in process_job
    await self.run_job(job=job, worker_id=worker_id)
  File "/Users/foo/.venv/lib/python3.11/site-packages/procrastinate/worker.py", line 243, in run_job
    task = self.find_task(task_name=task_name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/foo/.venv/lib/python3.11/site-packages/procrastinate/worker.py", line 238, in find_task
    raise exceptions.TaskNotFound
procrastinate.exceptions.TaskNotFound:
    Task cannot be imported.

It looks like the task exists in the process where you defer, but not in the process where you run the worker.
I'm trying to come up with ideas of how that may happen. Can you confirm that:

On both processes you start from the procrastinate.contrib.django.app instance (and then on the worker you use app.with_connector(app.connector.get_worker_connector()) which ensure you have the same tasks)
In both processes, Django is started (either ./manage.py or django.setup() so that we know the AppConfig.ready() has run
Both processes have the same settings (especially PROCRASTINATE_ON_APP_READY)

One thing you can test is go to the AppConfig (in apps.py) of any of your apps that is after procrastinate in INSTALLED_APPS and add:
def ready(self):
	from procrastinate.contrib.django import app
	print(app.tasks)

On my side, I'm adding some logs, but from your pasted output, it looks like you're not using debug logs, and not formatting them to get the most of them. (structured element).
In this PR, I'm providing concrete examples of how you can get all the structured log values without having to add a structured logging library.
Output of those logs look like:
2024-03-02 10:33:13,008 INFO    procrastinate.blueprints Adding tasks from blueprint {'action': 'loading_blueprint', 'namespace': 'bp', 'tasks': ['bp:procrastinate_demos.demo_django.demo.tasks.set_indexed']}
2024-03-02 10:33:13,039 INFO    procrastinate.blueprints Adding tasks from blueprint {'action': 'loading_blueprint', 'namespace': 'builtin', 'tasks': ['builtin:builtin:procrastinate.builtin_tasks.remove_old_jobs']}
Launching a worker on all queues
2024-03-02 10:33:13,086 DEBUG   procrastinate.app All tasks imported {'action': 'imported_tasks', 'tasks': ['builtin:procrastinate.builtin_tasks.remove_old_jobs', 'procrastinate.builtin_tasks.remove_old_jobs', 'procrastinate_demos.demo_django.demo.tasks.index_book', 'bp:procrastinate_demos.demo_django.demo.tasks.set_indexed']}
2024-03-02 10:33:13,086 INFO    procrastinate.worker Starting worker on all queues {'action': 'start_worker', 'worker': {'name': 'worker', 'id': None, 'queues': None}, 'queues': None}

Which is more informative :)

Ok, so oddly enough - this all works now!
Basically, if I define the custom app in settings.py and then import it throughout the Django app - it all works beautifully now. The app we have built may be "weird enough" that using our app as good test case for the Django integration may be leading to more complexity that not. Especially as we define custom blueprints, do not use the Django command, etc.
connector = procrastinate.PsycopgConnector(
    kwargs={
        "host": env["POSTGRES_HOST"],
        "user": env["POSTGRES_USER"],
        "password": env["POSTGRES_PASSWORD"],
        "port": env["POSTGRES_PORT"],
        "dbname": env["POSTGRES_DB"],
        **EnvService.get_ssl(),
    }
)

command = " ".join(sys.argv)
is_django = "runserver" in command or "gunicorn" in command
app = procrastinate.App(connector=connector)
if is_django:
    app.open()

Our "custom" setup works now, but I thought I'd do some more digging to see if I can help you out for other Django people.
(I added structured logging, did not really seem to help much)
The code below is the crux of the issue. As an experiment, I went "all in" on the Django integration, including running it though the worker. And everything worked great - except for this.  Basically, PROCRASTINATE_ON_APP_READY works. The tasks were registered in both the Django and Procrastinate process.  So example #2 works great. However, example #2 requires a helper function (below) that imports the app at the time of invocation. If you do not do that, and just run it off the imported app, the you get that If this message appears at import time, the app is not ready yet. If the answer is, just do the code like example #2 or put from procrastinate.contrib.django import app INSIDE the function (also works) that is totally fine with me! I just want to understand if this behaviour makes sense to you. Otherwise, the Django integration seems to work great to me!
import procrastinate
class WorkerService:
    @classmethod
    def app(cls) -> procrastinate.App:
        from procrastinate.contrib.django import app
        return app

from procrastinate.contrib.django import app
# This function is executed in a Django request
def schedule_report_to_be_run(cls, report_name: str, study_id: int) -> None:
    app.configure_task(name=name, queueing_lock=name).defer(report_id=report.id) # 1. Does not work
    WorkerService.app().configure_task(name=name, queueing_lock=name).defer(report_id=report.id) # 2. Works

Aaaaah yes of course.
It will also work if you keep the import outside but do from procrastinate.contrib import django and django.app.
The problem is that when we do contrib_django.app = create_app(blueprint=contrib_django.app), whoever has already imported the app and kept a reference will continue to use the old instance.
Let's see if I can do something about that. Might have to use a proxy instance.

I've added a proxy object, so now, importing the object before it's ready and keeping a reference to it, it will work after the real app is swapped in place.
https://github.com/procrastinate-org/procrastinate/releases/tag/2.0.0b7
https://pypi.org/project/procrastinate/2.0.0b7/

Our application may be "weird enough" that using our app as good test case for the Django integration may be a bad idea...

If you had to do weird stuff, everyone had to do weird stuff :)

Ok excited to test this!
Also is there a way to add a flag to disable this? For example, we use our Procrastinate/ Django models to insert stuff during tests - I guess switching to the Procrastinate managed connection or models prevents this? Also I feel like I asked about the third Procrastinate / Django model - did that ever get exposed?
Procrastinate models exposed in Django, such as ProcrastinateJob are read-only. Please use the procrastinate CLI to interact with jobs.

I guess I can add a setting to control that, and maybe a pytest fixture too.
That said, in many cases, I'd say it should be easier to run tests with the InMemoryConnector that is specifically made for testing, exposed and documented.
I'd say it depends if we're talking about unit tests or integration test.
Btw, I could even expose FactoryBoy factories.
So:

PROCRASTINATE_READONLY_MODELS that can be updated on specific tests (showing the code on how to make a fixture should be nice)
maybe a fixture that would replace the contrib app connector with an InMemoryConnector (giving the code may be more usable than providing the object)
maybe a set of factories for the models (same, and would avoid a dependency. Downside: not testable)
yes ProcrastinatePeriodicDefer is exposed, and in the PR I forgot to mention it. Maybe add a small reference doc too, would be nice.
the django documentation should be a whole section, not just a page.

They are integration tests - so having access to the whole thing would be nice! I can make due without them - but it would be helpful

Good news - the proxy works!
Other news:
Procrastinate models exposed in Django, such as ProcrastinateJob are read-only. Please use the procrastinate CLI to interact with jobs. Only seems to apply to procrastinate_jobs and procrastinate_events. It does not trigger for procrastinate_periodic_defers. Is that by design?

Is that by design?

Nope, will fix that too.

(I'm removing the DB router from the next beta release. The more I use it, the less I'm convinced it's useful and it will make it harder for some of the changes. If you did set it up, you can remove it.)
(PR is ready but I'll need a bit more time before merging and releasing)

Sounds good!

Beta numbers are cheap
https://github.com/procrastinate-org/procrastinate/releases/tag/2.0.0b8
https://pypi.org/project/procrastinate/2.0.0b8/
The django doc was split into individual pages: https://procrastinate.readthedocs.io/en/main/howto_index.html
There are now ways to relax the readonly feature for tests, plus, it's explained how to get a testing connector for the Django app

Ok, more progress and those docs look great.
Two things:

Typo on PROCASTINATE_READONLY_MODELS
Is there a way to run healthchecks via the django command? https://procrastinate.readthedocs.io/en/stable/howto/monitoring.html

Otherwise PROCRASTINATE_READONLY_MODELS solved the problem with tests, so I think with just that healthcheck fix we are good to go!

Is there a way to run healthchecks via the django command?

Hm, healthchecks look at 3 things:

App config (actually it's a noop, it's just that if the app wasn't configured correctly, we would have crashed before reaching this point)
DB connection
The presence of the procrastinate_jobs table

Within Django:

The app config is mostly the responsibility of Procrastinate itself
Checking the DB configuration is the responsibility of Django. I fell that if Django's DB wasn't configured properly, it wouldn't be Procrastinate's decision to raise this.
The presence of the procrastinate_jobs table is the responsibility of Django migrations. This is checked by ./manage.py check. (Technically, it's even better with Django because it checks that all migrations were applied whereas Procrastinate only checks the 1st migration)

Because of all that, I didn't include the healthchecks command to what was exposed. I could re-add it back but I'm not sure that it would add benefits vs a false sense of security ?
I could, though, be easily convinced to add it back if you have ideas of things that would make sense checking ? Is your usecase more of a sanity-check for your integration, or a runtime check used as liveness/readiness check as a basis for killing dead pods ? Also, should that be a healthcheck validating whether we're able to defer tasks, or to run the worker ?
For now I have more questions than answers :D

I've fixed PROCASTINATE_READONLY_MODELS once, but I guess I had copy-pasted the error elsewhere. Will fix it.

All reasonable points!
So if you imagine a Procrastinate & Django application living in a container or pod, and the command is python manage.py procrastinate worker, in your mind what is the best way to judge if the worker is running correctly?  So, yup, in your words, a "runtime check used as liveness/readiness check as a basis for killing dead pods".
I only ever used it to check whether I could run the worker at all, defering tasks would not be needed for us. Would be good though!
I agree though, overall, that the Django connection does obviate some of the healthcheck stuff, as a lot of it is now handled by Django!

Also, side point, is there any way to get on_app_ready to only run once - it seems to run multiple times - once during make migrations, then during migrate, then during app load. That may just be a Django quirk, in which case ignore this :)
import procrastinate
def on_app_ready(app: procrastinate.App) -> None:
    print("FOO")
    # Do stuff with app

It seems to run multiple times - once during make migrations, then during migrate, then during app load

From my understanding of your setup, the issue is that the 3 things you mention are 3 separate processes. The things you do in on_app_ready are in-memory changes (such as registering a blueprint, this is not persisted anywhere) so each process will do its own django.setup(). This is less a Django quirk and more a consequence of your setup?
There may be multiple things you can do:

Adjust the logging level of Procrastinate so that it won't be so noisy when you don't need it to be. (I'm guessing the reason you don't want to run it multiple times is that it clutters your logs?)
Potentially adjust you settings based on sys.stdin so that your setup is different when you run runserver vs other commands. Alternately, use environment variables to explicitly trigger different setup steps ?
Launch everything is a single process: a single script that launches multiple call_command() calls?

As a side note, it's perfectly reasonable that your container launches ./manage.py migrate on startup (though it might get complicated if you have multiple containers launching at once: if they try to migrate the same DB in parallel you'll have issues. But makemigrations is definitely not something you should run automatically as part of your runtime. It should be made during development, and migrations should be committed alongside with the modifications of the model they relate to.

Yup, that answers my 2nd comment. And yup, the command makemigrations is just run on development!
As for the first comment, the healthcheck, does that answer your question? Otherwise, I think the integration is done, at least from my perspective?

As for the first comment, the healthcheck, does that answer your question? Otherwise, I think the integration is done, at least from my perspective?

Ah yes, still pondering on what to do about that (if anything). I think I understand your usecase and it makes sense, but I think it might be better answered than just by activating the healthcheks as they exist today, on Django.
I see 2 choices, and we can have both:

A <something> that tells you if the worker process is doing right. I'd tend to say that the most appropriate <something> that comes to mind would be an HTTP endpoint with /alive, /ready and potentially /metrics (from Prometheus)
A process that tells you if another process could theoretically work. It's kinda what the healthcheck command is. I think it's more about checking that the app is configured correctly and the code is right rather than checking the state of the setup right now, though it might be a side effect. I'm thinking that integrating the Django checks framework and doing our own checks might be the best way to do that, so that ./manage.py checks (maybe with --tags procrastinate) would do the trick. I'll look at that.

Ya, option #2 seems the best.
This is a really cool integration BTW, I think it will really help with people using the library :)

So, I tried integrating with Django checks framework and... It just didn't match what I wanted to do. Ended up just doing a custom ./manage.py procrastinate healthchecks with some specific Django tests. Probably isn't 100% useful, but it shakes the whole tree and see if anything falls.
https://github.com/procrastinate-org/procrastinate/releases/tag/2.0.0b9
https://pypi.org/project/procrastinate/2.0.0b9/
https://procrastinate.readthedocs.io/en/main/howto/django/basic_usage.html#checking-proper-configuration
Hopefully, we're good to go :shipit:

It works beautifully! Closed, as far as I am concerned :)

https://github.com/procrastinate-org/procrastinate/releases/tag/2.0.0


Use copy-pr-bot

This PR replaces the copy_prs functionality from the ops-bot with the new dedicated copy-pr-bot GitHub application.
Thorough documentation for the new copy-pr-bot application can be viewed below.

https://docs.gha-runners.nvidia.com/apps/copy-pr-bot/

Important: copy-pr-bot enforces signed commits. If an organization member opens a PR that contains unsigned commits, it will be deemed untrusted and therefore require an /ok to test comment. See the GitHub docs here for information on how to set up commit signing.
Any time a PR is deemed untrusted, it will receive a comment that looks like this: https://github.com/rapidsai/ci-imgs/pull/63#issuecomment-1688973208.
Every subsequent commit on an untrusted PR will require an additional /ok to test comment.
Any existing PRs that have unsigned commits after this change is merged will require an /ok to test comment for each subsequent commit or the PR can be rebased to include signed commits as mentioned in the docs below: https://docs.gha-runners.nvidia.com/cpr/contributors.
This information is all included on the documentation page linked above.
I've skipped CI on this PR since it's not a change that is tested.
[skip ci]

/merge


Package needs updated - glob package needs updated

Getting warnings from npm that the glob 7.2.3 used isn't supported and needs to be updated to > 9, not sure if everyone is getting this warning.

@georgefromohio Thanks for reporting. I haven't been able to duplicate this using the latest version of Vite and this plugin, so I'll need as much information as possible to help track down what might be going on.
Can you please share your package.json and the Node.js version you're using?

Thanks for the quick reply... here are a few outputs
>_ node -v
# v22.8.0
>_ npm install
# npm warn deprecated<EMAIL_ADDRESS>This module is not supported, and leaks memory. Do not use it. Check out lru-cache # if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful.
# npm warn deprecated<EMAIL_ADDRESS>Glob versions prior to v9 are no longer supported
>_ npm ls inflight glob
# osg-sx@ ~/osg-sx
#└─┬<EMAIL_ADDRESS>#  └─┬<EMAIL_ADDRESS>#    └──<EMAIL_ADDRESS>
My package.json
{
    "private": true,
    "type": "module",
    "scripts": {
        "dev": "vite",
        "build": "vite build"
    },
    "devDependencies": {
        "axios": "^1.6.4",
        "laravel-vite-plugin": "^1.0.0",
        "sass": "^1.70.0",
        "vite": "^5.3.4",
        "vite-plugin-sass-glob-import": "^3.0.2"
    }
}

Thank you very much @georgefromohio. Very helpful. I'll look into it.

@georgefromohio When you have a moment could you please try updating your plugins to 4.0.0-next.1 and let me know if you have any issues?

Works beautifully! Thank you for being super responsive and over all great! Appreciate you and your contributions! Will there be a new release? Let me know!
>_ npm install

# found 0 vulnerabilities

@georgefromohio Thanks for testing. Hope to have a release out in the next day.

This was released as https://github.com/cmalven/vite-plugin-sass-glob-import/releases/tag/4.0.0


[NTV-262] Fetch and apply translations for new project page

📲 What + 🤔 Why
Fetching and applying new translations that are required for the new project page. Kickstarter natively translates in several different languages and its important that we apply all translations before shipping a feature.
✅ Acceptance criteria

[ ] Switch the language on your device and open up a project.
[ ] Navigate through the new project page and verify all strings that can be translated are being translated.
[ ] Do this for each of the languages we support and check that there isn't anything unusual going on with the constraints, truncation of labels etc.

⏰ TODO

[ ] Wait for the final String to be translated and deployed from this PR
[ ] Once that String is available on iOS, replace line 114 of the ProjectEnvironmentalCommitmentDisclaimerCell with the new string. Feel free to remove the TODO and/or re-factor the implementation as you see fit.
[ ] Additionally, check if the "Something else" string is available and apply it to line 63 of ProjectEnvironmentalCommitmentCellViewModel

SwiftFormat found issues:

File
Rules

Library/ViewModels/ProjectEnvironmentalCommitmentCellViewModel.swift:49:1
warning: (spaceInsideComments) Add leading and/or trailing space inside comments.

  Generated by :no_entry_sign: Danger


filter, record_assessor, grep

Check CONTRIBUTING guideline first and here is the list to help us investigate the problem.
Describe the bug
I've read through the documentation and am unable to find an example.
is there something wrong with this syntax?
@type grep
  <regexp>
     key category
     pattern /\bAUDIT\b/
</regexp>
<regexp>
    key $.details.description
   pattern /\bAuthentication activity\b/
</regexp>
</filter>

The second regexp, seems to have no effect.
To Reproduce

Expected behavior

Your Environment

Fluentd or td-agent version: fluentd --version or td-agent --version
Operating system: cat /etc/os-release
Kernel version: uname -r

If you hit the problem with older fluentd version, try latest version first.
Your Configuration
<!-- Write your configuration here -->

Your Error Log
<!-- Write your **ALL** error log here -->

Additional context

Please write reproduce step.


WIP: Services Deleters

fixes #1155 for services module

Won't be able to work on this till after December due to some reasons


Not able to bind the socket to a port less than 1024, but it is acomplished in native applications

i am trying to convert my native app, to react-native for code sharing between android and ios. in my application, i use react-native-udp to detect the devices connected to the wifi. In my native application, i bind to a port 137, to detect a specfic type of device. When i try to recreate the same in react-native-udp, during bind, 137 it gives me error "EACCESS". the only difference i see from the native application code and react native udp code is, native application uses DatagramSockets(), where as react-native-udp is using MulticastSocket().
//new MulticastSocket(port);
Can you please make it work for binding to a port less than 1024?

@VijayLogitech does it work when you use a MulticastSocket natively (not via react-native-udp -> GCDAsyncUdpSocket)?

@mvayngrib hi, sorry for the late reply, need ur help here.
i tried with multicastSocket natively to bind to port 137, and it gives the same EACCESS error which we get in react native. So multigram socket is not working for port 137 both in RN and natively.
Also the ip range supported in Multicast socket address range only <IP_ADDRESS> to <IP_ADDRESS>.
So, i suggest having datagramsocket, for ip's outside the multicast socket address, as well as port number less than 1024.
will it be possible for you to change the instance creation to Datagramsocket, only if the port range is less than 1024, in the below line.
filename : android/src/main/java/com/tradle/react/UdpSocketClient.javaandroid/src/main/java/com/tradle/react/UdpSocketClient.java
public void bind(Integer port, @Nullable String address) throws IOException {
if(port <1024)
mSocket = new DatagramSocket(null);

To bind to a port less than 1024, you need to be root. On Android, there are two ways of doing that: being System app or su binary. (More info on this StackOverflow answer).


Remove time convert

This does not fit TS' use cases - the time calculator on the other hand will be useful.

Keeping this, will add the time calculator in addition


Revert stacked tabs removal (Request)

I apologize if this is out of scope. Feel free to close it if so.
One of the things that really annoys me upstream is the inability to see tab titles once you open too many. This used to be available in the form of a preference to use stacked tabs, which means that tabs pile up to the side rather than shrink to nigh-invisible sizes. Combined with something like The Great Suspender, tab stacks allowed to have a rather firefox-like experience for tab-heavy users.
And then google removed the preference with the justification of "it's not intended for desktops" even though the GUI code can handle it perfectly fine and there was no reason to block it from use. There was quite a bit of outcry.
The relevant code changes: https://codereview.chromium.org/291093005
I've made a hacky patch to force stacked tabs, but I have no doubt that this is unacceptable for inclusion anywhere and really should be behind a preference like before the removal:
--- chromium-62.0.3202.89/chrome/browser/ui/views/tabs/browser_tab_strip_controller.cc
+++ chromium-62.0.3202.89/chrome/browser/ui/views/tabs/browser_tab_strip_controller.cc
@@ -80,12 +80,7 @@
 bool DetermineTabStripLayoutStacked(PrefService* prefs, bool* adjust_layout) {
   *adjust_layout = false;
   // For ash, always allow entering stacked mode.
-#if defined(USE_ASH)
-  *adjust_layout = true;
-  return prefs->GetBoolean(prefs::kTabStripStackedLayout);
-#else
-  return false;
-#endif
+  return true;
 }

 // Get the MIME type of the file pointed to by the url, based on the file's

Is there any possibility you guys would think about reverting (or reimplementing; you can't revert it cleanly, so it's likely one would have to redo the pref code) this change from upstream with a patch? It'd be a nice enhancement to the user experience.

Sure, I can add this in. Even though it's not a high-priority feature, it looks pretty simple and straight-forward to patch. Plus, your post contains pretty much all the info I need to write the patch (thanks for the well-written post BTW). But, if Chromium significantly changes the code and breaks this patch, I won't go out of my way to rewrite it.
I haven't looked at the source tree for 62 yet, so if you already know the answers to these questions it'll speed things up a bit:

Is kTabStripStackedLayout configurable by the user at all? I don't see any configuration UI changes in the Issue you linked. You mentioned the pref code would have to be redone?
What does adjust_layout do?

Hey, I'm fine with that. I agree effort is better spent removing tracking. :P
Before the removal, there was an option in chrome://flags that allowed to toggle it as well as a console argument - upstream removed both, so kTabStripStackedLayout is right now purely used internally to keep track of the state from what I can tell.
Don't quote me on this: I think adjust_layout is used on tablets/touchscreens. From reading the issue tracker for chromium, it seems tablets are normally in equal-width mode until one hovers/touches the tabs bar, which causes it to switch to stacked. I'm assuming adjust_layout triggers this, but I'm not sure.

Hmm alright. I'll add options for both and see what happens.

Added in that referencing commit. Closing.


Consecutive Firebase.getInt returns wrong value

I have two functions, GetFirstInt and GetSecondInt as follow
FirebaseData firebaseData;
void setup(){
  Firebase.begin(FIREBASE_HOST, FIREBASE_AUTH);
  Firebase.reconnectWiFi(true);
}
String GetFirstInt(){
  if (Firebase.getInt(firebaseData, firstPath))
  {
    if(firebaseData.dataType() == "int"){
      return firebaseData.intData();
    }
  }
  return "";
}
String GetSecondInt(){
  if (Firebase.getInt(firebaseData, secondPath))
  {
    if(firebaseData.dataType() == "int"){
      return firebaseData.intData();
    }
  }
  return "";
}

These functions are called one after the other in regular intervals of 10 seconds.
Sometimes, the GetSecondInt returns the value of first int that is stored on Firebase Database (in "firstPath"), when it should be returning the value of second int stored in "secondPath".
I even tried to make FirebaseData firebaseData a local variable to these functions instead of a global one. I thought some kind of an overwrite issue might be causing this. But turning it into local variables led the ESP32 to crash for some reason.
Does someone know a fix for this?
Thanks

The normal usage of Firebase Data object does not cause problems.
Use Firebase Data object as local variable is not recommend which may lead to unreleased memory used by WiFi core.
Return integer inside String returned function getFirstInt() and getSecondInt() can cause memory leak or give the unexpected result.
You can help me to address the root causes by do the following.
Create example sketch that causes the problems and post it here.
Choose board as Generic ESP32 Module can show the debug message on serial, copy all serial debug message and post it here.

Your above code is error and can't compile.
And you're not provide the WiFi connection in setup too.

Your above code is error and can't compile due to return integer inside String returned function getFirstInt() and getSecondInt()
And you're not provide the WiFi connection in setup too.
The normal usage of Firebase Data object does not cause problems.
Use Firebase Data object as local variable is not recommend which may lead to unreleased memory used by WiFi core.
You can help me to address the root causes by do the following.

Create example of working sketch that causes the problems and post it here.
Choose board as Generic ESP32 Module can show the debug message on serial, copy all serial debug message and post it here.

@mobizt Thanks for the response. I have updated the code in the original question, and added some additional information. I will state those here again:

The flag variable (in CheckFlag function) should return the flag value stored on the Firebase Database in path "/flag".
BUT Sometimes, the flag (in CheckFlag function) actually returns the value of the date variable stored on firebase database (which should be fetched by prevDate variable in Sync function). But instead prevDate returns 0 (i.e, it fails to fetch a value).

I'll give an example of the above statement. Yesterday, the value stored in my Firebase database at /date was 21, and at /flag was 0.
The code looped and every 10 seconds, it showed the following output:
prevDate = 21 flag = 0
It ran fine for around 8-9 hours. After that, it suddenly shows the following output:
prevDate = 0 flag = 21
I guess prevDate was 0 because it couldn't fetch properly from online (in which case the function FirebaseGetInt returns 0). But what I don't get is why flag variable in code showed the value at /date instead of the value at /flag?
P.S Sorry I couldn't give the credentials of the Firebase Database, it belongs to the company I am currently interning at. But if required, please inform me so and I will create a new dummy database for your testing purpose.
Thanks

My library is rather give the exact what type of data store and read from database.
Please check your free heap and stack size that may lead to heap corruption problem because your application use large of memory for all time.


Remove format_header.sh

This PR removes the format_header.sh in favor of purely relying on clang-format. The .clang-format is updated to allow regrouping of includes. In total, out of the affected 1000 files, only ~200 have non-whitespace changes. Many of those are regarding the ginkgo main header, or previously missing
Include groups (ordered by their prority)

<oneapi/... since we used force-top with that a lot.
<hip/hip_runtime.h> same
'main' include
STL
backend library, i.e. cuda/hip/omp/mpi/...
third party library
Ginkgo main header
Ginkgo headers
Other <...> includes
All "..." includes

This is mostly based on the already used regrouping in format_header.sh.
Notable Changes

The 'main' header of an implementation file now uses "...", instead of <...>. This is required, since clang-format can't deduce the main header otherwise.
force-top is removed and instead handling by using negative priorities in .clang-format. In very few cases this is not sufficient, and there clang-format on/off achieves the same results.
Only 1 empty line between separate groups of includes. It is not possible to adjust that with a clang-format setting.
The <ginkgo/ginkgo.hpp> header is now ordered after standard library includes
Some 'main' headers are not recognized. This concerns especially backend headers, which add stuff to code headers, for example cuda/matrix/batch_struct.hpp will not have core/matrix/batch_struct.hpp as its main header anymore.
Split up implementations, like for the Jacobi preconditioner, don't recognize the 'main' header correctly. One solution could be to use the same splitting approach as for CSR/FBCSR kernels.

Since the format_header.sh script is very complex, I think these changes are a small price to removing it.

I like it, if it performs well, this should make IDE integration of our header ordering much smoother

The non-whitespace diff can be obtained with:
git diff -b -B -w -z --ignore-space-at-eol --ignore-blank-lines 195e5bf3..195e5bf3~1

The diff is also attached here.
diff.txt

<oneapi/.. -> <oneapi/dpl... only the dpl need to be on the top. the other oneapi before dpl may also introduce issues.
clang format does not take care of the "" for local and <> from global, so it may still destroy the regroup
hip_runtime should be in hip group. only a few hip tests request it on top which can be handled by clang-format off
9 can be merged into 6?
It changes a lot of the format rules in the documentation
I would say this pr changes the format style not just remove format_header

The main goal of the PR is to remove format_header.sh. The formatting changes are a by product of that.

9 can be merged into 6?

I don't follow, can you elaborate?

the regroup does not take care the header. for example cb_gmres example

Do you mean the wrong comment on the main ginkgo header file?

clang format does not take care of the "" for local and <> from global, so it may still destroy the regroup

I think it does. Any <> is priority 1-4, the <ginkgo/ headers are priority 5-6, and everything else, which includes "" is priority 7.

For me, removing format_header means that we have the equivalent way to handle it.
Removing something that drives the guideline is a bit dangerous to me although it may be the same thing in the end.

9 can be merged into 6?

Do we consider all the other library from outside (<...> not "...") is third party? I think it should be the case for ginkgo.

the regroup does not take care the header. for example cb_gmres

It was shown in my first review, but it is not there anymore.
// ginkgo main header
#include <ginkgo/ginkgo.hpp>

#include <....>

->
// ginkgo main header
#include <...>

#include <ginkgo/ginkgo.hpp>

regroup does not move the header with the comments.
It should be checked for this pr to ensure that the comments are also moved to correct place.

clang format does not take care of the "" for local and <> from global, so it may still destroy the regroup
I think it does. Any <> is priority 1-4, the <ginkgo/ headers are priority 5-6, and everything else, which includes "" is priority 7.

I mean if we put "ginkgo/..." it will be priority 7 because the script does not correct <>/""  usage now.

Another argument to remove the script is that it's slow. The clang and reuse pre-commit check each take ~1s when applying to all files, but the format-header check takes more than a minute. The reason is most likely that the script can only be run sequentially, while the other checks can run in parallel.

I've rebased it after the changes from #1616. It went pretty smoothly. I don't think there are additional changes that were not reviewed before, so I will go a head and merge it when the tests pass.


Complete Calculator Implementation --basic

What does this PR do?
This PR introduces a fully functional calculator component made with TypeScript and React .
What changes were made?

Implemented the entire calculator user interface.
Added logic for numeric input and arithmetic operations.

Additional Notes:

This PR is based on the 'calculator' branch.
Extensive testing has been performed to ensure the calculator works correctly.
The code adheres to coding standards and best practices.

@Kritika30032002  pls look

add readme containing steps for local setup.

add readme containing steps for local setup.

pls see

Thanks for your valuable contribution.
Don't forget to star the repository.


Add Laravel Oh Generators

Sorry I didn’t notice that it must >200 stars

This is a good find. I can understand the 200-star/10k download requirement to make it to the 'awesome' list, but how about another 'rising' or 'new' list which could include things like these?


Integrating additional plugin

I have tried for days to integrate a plugin to this package, but it does not seem to be working. I want to try and add in support for math/chem equations using the following rpm package (https://github.com/isaul32/ckeditor5-math).
I have managed to install and adjust the config, but the custom plugin is not being loaded into the config. Any pointers?
Maybe this could be added as a feature - it is extremely useful.

Thanks! Where is the myCustom.plugin syntax coming from? I managed to do the above but the plugin was never accessible from the CKEditor5 window object?

Thanks! Where is the myCustom.plugin syntax coming from? I managed to do the above but the plugin was never accessible from the CKEditor5 window object?

It's a naming convention from CKEditor plugins which use Webpack DLL plugin for being plugged in to the main editor module.
I implemented (basically for testing the capabilities) a custom widget following the tutorial and I started with their packaging tool which is still beta atm, and sets up a project that uses Webpack DLL plugin + their conventions (which I advice to follow).
In my experience, at least using TypeScript, neither approach worked straight away. But hammering a bit the project structure and declaring some globals such as a bit, you get out with a working plugin.

Thanks I'll check it out!

Thanks I'll check it out!
Hi @oliverkidd, i have same issue with you. How to fix this? Can you help me?

@lamuertepeluda Hello. I fixed the problem above but I want to add a cdn at the top to format the plugin's output. I have found the answer but haven't found it yet. Can you help me? "How to add a cdn to the head tag".
Ex: https://jsfiddle.net/isaul32/qktj9h7x/

@lamuertepeluda Hello. I fixed the problem above but I want to add a cdn at the top to format the plugin's output. I have found the answer but haven't found it yet. Can you help me? "How to add a cdn to the head tag".
Ex: https://jsfiddle.net/isaul32/qktj9h7x/

Hello @iamtun, I'm not sure the CDN is the correct approach here, since the way strapi adds dependencies is via yarn/npm.
Once you install a package, you should import it into the src/admin/app.tsx file (I use the strapi TypeScript template, you will have app.js or app.jsx if you use plain JavaScript). Now you have to rebuild the frontend. Check also my first answer to this thread.
If you cannot download your plugin from npm, then you can create a plugins folder in the strapi project root directory and put your plugin code there in a subfolder and modify the config/plugin.(ts|js) file to resolve your plugin from the root plugins sub folder (check this repository development guide for examples).
Have also a look to https://docs.strapi.io/dev-docs/admin-panel-customization for how the admin panel customization works.

Thanks for your feedback. After hours of searching, I solved my problem another way add import 'mathjax/es5/tex-svg-full.js' in admin/src/app .  Problem solved!

@lamuertepeluda @iamtun could anyone of you maybe give a quick guide on how to add plugins to your ckeditor in strapi? I followed the instructions in this thread here for hours, but it didn't work

@exniwe all I did and I know is described in this thread or in #100. Is it perhaps some differences in our setups that prevent you from achieving the result?

@lamuertepeluda Okay I did what you described in #100 and it works. But now I am trying to add this math plugin following your instruction, but I always get "ckeditor-duplicated-modules" error. So how would you add the math plugin?

@exniwe

make sure you are installing your plugin version that matches the CKeditor version in this strapi plugin (was 36.0.0, if I am not wrong - pls check the package.json - I also have a PR open for upgrading, which in turn needs to be updated to 38.0.1). Having different CKEditor runtime versions can lead to that error.
check that the version of the plugin you will choose is DLL-compatible. Not all CKEditor plugins were ported to the relatively new DLL build of CKEditor, which makes it easy to have modular plugins.

Okay, thanks @lamuertepeluda The installation as described by you worked. However, the latex expressions are not yet formatted. So, for example, when I enter an expression like x=\frac{-b\pm\sqrt{b^2-4ac}}{2a} in the input field of the math plugin, it is displayed unformatted. The documentation says that you can adjust the plugin options here, but I don't know where exactly in the plugin itself you can do that.

Hi @exniwe I don't know in the case on the plugin you are trying to add, since I never used, but generally speaking plugin options have to be provided via config using a declarative syntax.
As an example, consider how the link plugin options are provided in the base config file. Hope it helps.

Okay, thanks @lamuertepeluda The installation as described by you worked. However, the latex expressions are not yet formatted. So, for example, when I enter an expression like x=\frac{-b\pm\sqrt{b^2-4ac}}{2a} in the input field of the math plugin, it is displayed unformatted. The documentation says that you can adjust the plugin options here, but I don't know where exactly in the plugin itself you can do that?

Hi @exniwe, don't know if you have mathjax installed? If not then install mathJax and import 'mathjax/es5/tex-svg-full.js' in app.js. It works for my case

Hey guys!
I have a problem importing custom plugins (mine for testing or existing markdown gfm) into the /src/admin/app.tsx. The issue is in this line:

it says that CKEditor is undefined.

However if i import code directly into the ckeditor.ts config it works..

Any idea how to fix this?
It also works in this same file if i will paste full code of the plugin before defining CKEditor5, but changing CKEditor5 to window.CKEditor5. (it works like this only in ckeditor.ts)

When I import the plugin, I instantly get duplicate ckeditor error, on two separate plugins, all I want is some math in my editor...


Detecting Home

I found today that since latest update this integration is marking me and my wife at home when for sure we are not home.. I have reloaded the integration and I will keep an eye on the situation.

Be sure that integration is configured with http protocol. You can check from diagnostics or you can just reconfigure enabling advanced option so that you have the option to choose the communication protocol.
Wrong home detection is a typical issue related to old ssh library.

I have reconfigured to unsure I'm using https, but it's the same..
I have downgraded the integration version to test..

Problem still present even with old version..
Maybe related with latest asus firmware?

I'm testing another asus integration and the problem is present.. For sure is a router firmware problem..

For starting attach integration diagnostics.

config_entry-asuswrt_custom-98bb00b0d9b4a10f7a48a67c90ddb5fb.json.txt

After a lot of fight with my asus devices I think the problem is solved.. Made some reboots and resets..


Marmaid error

System info
System: Mac OS
Obsidian: Latest
Plugin 0.2.2
Error
Error parsing Mermaid diagram!

Parse error on line 7:
...ew      :10-30, 60mm    Reading      :1
-----------------------^
Expecting 'taskData', got 'NL'

Diagram
```mermaid
gantt
    title Day Planner for 7th November 2020
    dateFormat  HH-mm
    axisFormat %H:%M
    section Tasks
    Review notes from yesterday      :09-45, 45mm
    Create new notes for #article review      :10-30, 60mm
    Reading      :12-00, 25mm
    Reading      :12-30, 90mm
    Review notes and update daily note [[20201103]]      :15-00, 45mm
    Walk      :15-45, 45mm
    Reading      :16-30, 50mm
    Prep for tomorrow's meetings      :17-20, 340mm
    END      :23-00, 0mm
    section Breaks
    BREAK      :11-30, 30mm
    BREAK      :12-25, 5mm
    BREAK      :14-00, 60mm

Thanks, could be the # from #article on that line causing the issue. Does removing it render the chart?
I will look at escaping.

Yes the # is making the problem! Anyway please remove the heading, there is no need to write it down as we see daily note date in the Obsidian header (the name of file), it just occupy space and provide no extra information.

Fix and update available in latest release 0.2.3.


file watchers page isn't formatted properly

this page has several instances of what should be bulleted lists that are just smooshed together into a paragraph. Why? fix it.

ugh, this is intended behavior for python-markdown, and for the markdown spec, john gruber version. I should probably switch to commonmark or something


EOFError: Compressed file ended before the end-of-stream marker was reached

Hi,
I was using script same as always I do to download files from Saved and that bug happened:
Retrieving saved posts...
Traceback (most recent call last):
  File "/usr/local/bin/instaloader", line 8, in <module>
    sys.exit(main())
  File "/usr/lib/python3/dist-packages/instaloader/__main__.py", line 471, in main
    _main(loader,
  File "/usr/lib/python3/dist-packages/instaloader/__main__.py", line 178, in _main
    instaloader.download_saved_posts(fast_update=fast_update, max_count=max_count,
  File "/usr/lib/python3/dist-packages/instaloader/instaloader.py", line 81, in call
    return func(instaloader, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/instaloader/instaloader.py", line 1112, in download_saved_posts
    self.posts_download_loop(node_iterator, ":saved",
  File "/usr/lib/python3/dist-packages/instaloader/instaloader.py", line 1001, in posts_download_loop
    with resumable_iteration(
  File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__
    return next(self.gen)
  File "/usr/lib/python3/dist-packages/instaloader/nodeiterator.py", line 283, in resumable_iteration
    fni = load(context, resume_file_path)
  File "/usr/lib/python3/dist-packages/instaloader/structures.py", line 1877, in load_structure_from_file
    json_structure = json.load(fp)
  File "/usr/lib/python3.10/json/__init__.py", line 293, in load
    return loads(fp.read(),
  File "/usr/lib/python3.10/lzma.py", line 200, in read
    return self._buffer.read(size)
  File "/usr/lib/python3.10/_compression.py", line 118, in readall
    while data := self.read(sys.maxsize):
  File "/usr/lib/python3.10/_compression.py", line 99, in read
    raise EOFError("Compressed file ended before the "
EOFError: Compressed file ended before the end-of-stream marker was reached

Instaloader Version
latest (4.9.2)
I tried to update script, software reloading session cookies, login to account using terminal - but nothing of that helped.
In that case even count limit (-c) didn't help.
Also is important that I used that script after a long break - meanwhile adding more files to Saved.
I think that it looks like I break some kind of limit of total files there and even with set up count limit - it still shows the same error. Like it needs some kind of divide of total data

You missed important things like your commandline/script snippet.
Anyway it looks like this error doesn't come from instaloader.

Sorry for not including such important thing. Here it comes:
sudo instaloader -l <USERNAME> :saved
Session file does not exist yet - Logging in.
Enter Instagram password for <USERNAME>:
Logged in as <USERNAME>.
Retrieving saved posts...
Traceback (most recent call last):
  File "/usr/local/bin/instaloader", line 33, in <module>
    sys.exit(load_entry_point('instaloader==4.9.2', 'console_scripts', 'instaloader')())
  File "/usr/local/lib/python3.10/dist-packages/instaloader-4.9.2-py3.10.egg/instaloader/__main__.py", line 471, in main
    _main(loader,
  File "/usr/local/lib/python3.10/dist-packages/instaloader-4.9.2-py3.10.egg/instaloader/__main__.py", line 178, in _main
    instaloader.download_saved_posts(fast_update=fast_update, max_count=max_count,
  File "/usr/local/lib/python3.10/dist-packages/instaloader-4.9.2-py3.10.egg/instaloader/instaloader.py", line 81, in call
    return func(instaloader, *args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/instaloader-4.9.2-py3.10.egg/instaloader/instaloader.py", line 1111, in download_saved_posts
    self.posts_download_loop(node_iterator, ":saved",
  File "/usr/local/lib/python3.10/dist-packages/instaloader-4.9.2-py3.10.egg/instaloader/instaloader.py", line 1000, in posts_download_loop
    with resumable_iteration(
  File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.10/dist-packages/instaloader-4.9.2-py3.10.egg/instaloader/nodeiterator.py", line 283, in resumable_iteration
    fni = load(context, resume_file_path)
  File "/usr/local/lib/python3.10/dist-packages/instaloader-4.9.2-py3.10.egg/instaloader/structures.py", line 1877, in load_structure_from_file
    json_structure = json.load(fp)
  File "/usr/lib/python3.10/json/__init__.py", line 293, in load
    return loads(fp.read(),
  File "/usr/lib/python3.10/lzma.py", line 200, in read
    return self._buffer.read(size)
  File "/usr/lib/python3.10/_compression.py", line 118, in readall
    while data := self.read(sys.maxsize):
  File "/usr/lib/python3.10/_compression.py", line 99, in read
    raise EOFError("Compressed file ended before the "
EOFError: Compressed file ended before the end-of-stream marker was reached

Error related to reading the file.

Bug report lacks how to reproduce the error.


feat: allow skill to be installed via pip

Mike, what is with all the new stuff in .gitignore?  Files that get created now when you run tests?

Mike, what is with all the new stuff in .gitignore? Files that get created now when you run tests?

I appended a generic Python gitignore that will ignore any files that get built by pip or other build systems, so those don't accidentally get committed. I can take it out if you'd like, but I figured it would help keep your repository tidy.

No, that's fine.  I just wanted to understand.  Thanks again.

BTW, does the "incompatible skills" part that you fixed have any effect that avoid having to manually change neon.yaml?

BTW, does the "incompatible skills" part that you fixed have any effect that avoid having to manually change neon.yaml?

I'm hoping so, but I'm not exactly sure how well-supported that file is in Neon


Dropdown Menus don't close if opening another dropdown menu

I have multiple dropdown menus on a page for contextual menus (delete/export/etc individual items) and I can close the dropdowns when clicking anywhere on the body, however if I open a different dropdown menu, the previous one doesn't close, as it would if clicking anywhere else on the page.
Is this an option I'm missing, a bug, or intended behaviour?

Hi, which version of foundation are you using? i was seen this template: http://foundation.zurb.com/templates-previews-sites-f6/news-magazine.html and there the dropdowns work fine.

@andrewhinde I'm closing this out, feel free to reopen if it is still an issue. Thanks.

@andrewhinde I'm closing this out, feel free to reopen if it is still an issue. Thanks.

Was still open ;-)


Embedded syntax with `@ is parsed incorrectly

See  https://github.com/FStarLang/steel/blob/0f3aa42fc7d205a51dc56eaee8205d91ce4ee854/share/steel/examples/pulse/bug-reports/ExistsSyntax.fst

Pulse has a custom syntax overlaid on top of the syntax of F* terms.
There are two parts of the syntax that Pulse actually interprets:

The syntax of vprops, the separation logic propositions
The syntax of imperative computation

This issue concerns point 1 only.
I would like to have a syntax for vprops where there is no ambiguity as to which parts of a separation logic spec contain terms which Pulse should interpret vs. the parts of a spec that are just underlying F* terms. i.e., the concrete syntax should be in direct correspondence with Pulse's internal abstract syntax of vprops.
Towards that end, I have been revising the syntax of PulseVprop to be as follows, removing the actions for clarity (I would eventually add cases for the other connectives of Pulse, including FORALL, WAND-like connectives etc.):
pulseVprop:
  | t=atomicVprop

  | EXISTS bs=nonempty_list(pulseMultiBinder) DOT body=pulseVprop

  | l=pulseVprop starOp r=pulseVprop

The starOp is defined to be either "**" or "star", (the latter just for backwards compat, I would like to remove it):
%inline
starOp:
  | o=OPINFIX3
    { if o = "**" then () else raise_error (Fatal_SyntaxError, "Unexpected infix operator; expected '**'") (rr $loc) }
  | BACKTICK id=IDENT BACKTICK
    { if id = "star" then () else raise_error (Fatal_SyntaxError, "Unexpected infix operator; expected '**'") (rr $loc) }

And atomicVprops are:
atomicVprop:
  | LPAREN p=pulseVprop RPAREN // parenthesized pulseVprop

  | head=qlident args=list(argTerm) // application terms, like R.pts_to a #x v

  | BACKTICK_AT p=atomicTerm // embedded host terms

The most unusual part of this is the last case: if you want to write an arbitrary F* term for a vprop, you can do so using a syntax similar to antiquotation, i.e., you can write @( e ),
where e is a
I think the syntax has improved, such as in this case:
You can write this now:
exists m s.
     pts_to i full_perm m  **
     pts_to sum full_perm s **
     pure (s == sum_spec m /\ b == (m <> n))

where previously you had to write (note the additional parentheses and the "star" notation)
exists m s. (
    pts_to i full_perm m `star`
    pts_to sum full_perm s `star`
    pure (s == sum_spec m /\ b == (m <> n)))

On the other hand, when writing embedded vprops, the syntax is more explicit, as seen in this diff, where the embedded F* conditional is now explicitly tagged with `@ marker to embed it as a Pulse vprop.
-  requires (if true
-            then pts_to r full_perm 0ul
-            else pts_to r full_perm 1ul)
-  ensures  (if true
-            then pts_to r full_perm 1ul
-            else pts_to r full_perm 0ul)
+  requires `@(if true
+              then pts_to r full_perm 0ul
+              else pts_to r full_perm 1ul)
+  ensures  `@(if true
+              then pts_to r full_perm 1ul
+              else pts_to r full_perm 0ul)

And, of course, this solves the problem with the issue itself, where this is now interpreted correctly:
  R.pts_to r0 full_perm s **
  exists (s1:U8.t). R.pts_to r1 full_perm s1

Cf. 9fe0724

The embedded syntax notation `@ is handled incorrectly:

In #23 writing it around rec_perm causes a weird implicits cannot be inferred
Writing it around an arg causes F* to interpret it as an antiquotation, when it actually isn't

Revised my position on this completely. The syntax of vprops in F* and Pulse now coincide. We no longer need `@


Overhaul tax handling

So... https://github.com/ocdtrekkie/xrf_billing_sandstorm/blob/main/modules/taxes/acpm_viewrecords.php#L30 survived into my release version... Obviously inventory needs to specify whether it counts as income or an expense. I use this distinction primarily to highlight between work done (which is straight income), and parts or other expenses passed along on the bill (which are part of total receipts, but not net income).
Inventory should also specify whether the sales tax calculation should apply to it or not.

Database format for this change is done, but neither the income or sales tax handling is updated to use it yet.

Income handling is now fixed (that hardcoding linked above is gone). Sales tax overhaul is up next!


add get user oauth access tokens to clerk sdk

Type of change

[ ] 🐛 Bug fix
[x] 🌟 New feature
[ ] 🔨 Breaking change
[ ] 📖 Refactoring / dependency upgrade / documentation
[ ] other:

Packages affected

[ ] @clerk/clerk-js
[ ] @clerk/clerk-react
[ ] @clerk/nextjs
[ ] @clerk/remix
[ ] @clerk/types
[ ] @clerk/clerk-expo
[x] @clerk/backend-core
[x] @clerk/clerk-sdk-node
[ ] @clerk/edge
[ ] build/tooling/chore

Description

[ ] npm test runs as expected.

Still can't npm install locally since I'm on node 14 which is the latest AWS Lambda currently supports

[ ] npm run build runs as expected.

I needed to use this API but couldn't find it on the SDK, so I added it to UsersApi based on the docs.

tagging @nikosdouvlis @SokratisVidros for review

Thanks for the feedback! Committed the edit, but don't have permission to merge


Tutorials need to show ReactDOM.createRoot() to match React 18

Both the "Essentials" and "Fundamentals" tutorials are currently showing ReactDOM.render(), and that needs to be update to match React 18.

Both the "Essentials" and "Fundamentals" tutorials are currently showing ReactDOM.render(), and that needs to be update to match React 18.

I'm happy to work on this and update the tutorials for React

@marksist300 unfortunately it's more complicated than just updating a couple of render() calls. The example repos need to be reworked, and that requires actually doing a bunch of Git history surgery to alter the first commits, rebase all the following commits, and update all the tags used by the example sandboxes.

@marksist300 unfortunately it's more complicated than just updating a couple of render() calls. The example repos need to be reworked, and that requires actually doing a bunch of Git history surgery to alter the first commits, rebase all the following commits, and update all the tags used by the example sandboxes.

Ah, ok. I thought rebasing public repos was generally a 'no no'?

OK. Probably best left to someone who knows exactly what should and shouldn't be included from the git history then 🙂.
I was looking through the Redux fundamentals last night and noticed the docs are using e.which to register keystrokes:

Wasn't sure it would warrant a separate issue, but it should probably be added to the list of code snippets to update as it's deprecated. It's used on several pages.

I take that back, the "Fundamentals" tutorial repo still needs to be updated.


Introduce monorepo to manage packages

What would you like to be added:
Introduce a monorepo structure to manage the frontend and backend packages efficiently.
Why is this needed:
Implementing a monorepo in Coedepair will simplify and streamline package management across the project. Given that both the frontend and backend are built using TypeScript, adopting a monorepo will allow us to maximize code sharing and consistency across the codebase.
Additional Information:
There are several options available for setting up a monorepo, including lerna, turborepo, and pnpm. Among these, pnpm might be great option due to several reasons:

Given that our sibling project, Yorkie-JS-SDK, is already using pnpm to manage its monorepo, it would be advantageous to align our approach for consistency.
"Turborepo is a good one but its key feature "task cache" requires Vercel or a self-hosted cache server. If we think the benefit of having a remote build cache is significant, then Turborepo would be good, if not, pnpm is good to go." by @blurfx

Related to: https://github.com/yorkie-team/yorkie-js-sdk/issues/648

Considering the consistency in the management area, pnpm seems like a good choice.


Implement a hard retention period

The current retention period implementation is a soft one. Garbage collection will still keep segments based on the smallest channel tail.
It'd be interesting to have a hard retention period that will forcefully remove stalled segments.

Implemented at commit ff378f390258318836192a717a6037305ddffed5


Fix Issue 1180 - the GC failes to handle large allocation requests propperly

report out-of-memory for allocation a bit below size_t.max
even though reported as a regression, this doesn't seem like anything new, so it might not be necessary to merge into stable.

LGTM

Auto-merge toggled on

Auto-merge toggled off

Re-enable auto merge after fix for win64 build

Auto-merge toggled on


add new parameter 'action'

Allow specifying alternative commands to run other than idf.py build
Useful for using other esp-idf tools inside the docker container as part of Github CI builds.
Defaults to idf.py build
fixes #2

Example: Building custom flash images as part of Github actions
      - name: Create Final Merged Binary
        uses: espressif/esp-idf-ci-action
        with:
          esp_idf_version: v4.4
          path: './build/'

          # 'command' is the new thing this PR adds
          command: esptool.py --chip esp32 merge_bin --fill-flash-size 4MB -o ../your_custom_file.bin @flash_args

notes:

Based on previous work by @BrianPugh
I tested this with just my local projects, not sure if there's other test suites that it should be run through/etc
Thanks everyone

@binary1230  Thank you so much for the contribution!
I would only suggest naming the new parameter command instead of action`
Also, it looks like this closes https://github.com/espressif/esp-idf-ci-action/issues/2  could you please add fixes #2 to the PR description?

@binary1230 Thank you for your contribution!


Hardware cannot set color space "YCbCr444"

On ubuntu 14.04 on an Asus G75VW with Nvidia GTX 660M, updated to latest Nvidia Drivers 361.42. I cannot adjust the "color space" as you suggest to get rid of the green tint to the image(I only have RGB value). This might be a hardware limitation of my system, potentially GPU or Laptop display is the issue I think, referencing FreeBSD docs. Just wanted to share, not necessarily looking for a fix.
ftp://download.nvidia.com/XFree86/FreeBSD-x86/304.32/README/README.txt
"YCbCr444": sets color space to YCbCr 4:4:4. YCbCr supports only
limited color range. It is not possible to set this color space if
the GPU or display is not capable of limited range.
If the ColorSpace option is not specified, or is incorrectly specified,
then the color space is set to RGB by default.

This is also true for my monitor, HP ZR22W. However, you can set color space per display. So, you can e.g. set YCbCr for the Elgato GCHD and RGB for your monitor, if you have them both connected at the same time.
Or, you can set your Elgato GCHD to YCbCr and let Elgato's HDMI-Out (which isn't a 1:1 HDMI-passthrough btw.) handle your monitor.
However, since the green tint issue is finally fixed (not published yet), this won't be an issue anymore.

Closing this issue, since the original issue #8 has been fixed. You can now switch colorspaces using -c rgb or -c yuv, according to your source. More information can be found in the original issue.


yum install failed vespa-libtorrent-devel and vespa-zookeeper-c-client-devel

I have successfully install vespa-6.192.4 with this script serveral days before. I write this script based on this dockerfile.
But today I tried to install vespa in another machine, and There is a error that Error: No Package found for vespa-libtorrent-devel >= 1.0.11-6 Error: No Package found for vespa-zookeeper-c-client-devel >= 3.4.9-6
Is vespa-libtorrent-devel and vespa-zookeeper-c-client-devel not available recently?

vespa-libtorrent-devel  and vespa-zookeeper-c-client-devel are not used anymore, you should remove them from your script (dockerfile has been updated). Thanks for the report!


Bug: Spec Export - HEX not exporting.

the blank document bug was fixed but now the hex value is not coming up on the right hand side.

click color text to change color format.


Can't make rollings with a partition size smaller than the requested window

Right now, it's impossible to make any rolling operations (and probably other operations that I do not remember now) with a partition size smaller than the window requested.
Example:
import numpy as np
import pandas as pd
import dask.dataframe as dd
import random
import fastparquet

#generate dummy df
def generate_dask_df(size):
    d = {
        'dates':
        pd.date_range('1980-01-01 00:00:00', periods=size, freq='1h'),
        'signal1':
        np.random.normal(0, 0.3, size=size).cumsum() + 50,
    }

    df = pd.DataFrame(d)

    return dd.from_pandas(df.set_index('dates'), npartitions=5)

small_df = generate_dask_df(20).repartition(freq='3h')
small_df.divisions

(Timestamp('1980-01-01 00:00:00', freq='3H'),
 Timestamp('1980-01-01 03:00:00', freq='3H'),
 Timestamp('1980-01-01 06:00:00', freq='3H'),
 Timestamp('1980-01-01 09:00:00', freq='3H'),
 Timestamp('1980-01-01 12:00:00', freq='3H'),
 Timestamp('1980-01-01 15:00:00', freq='3H'),
 Timestamp('1980-01-01 18:00:00', freq='3H'),
 Timestamp('1980-01-01 19:00:00'))

This works nicely
small_df['signal1'].rolling('3h').mean().head()

this doesn't
small_df['signal1'].rolling('4h').mean().head()

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-73-995569d34a2d> in <module>
----> 1 small_df['signal1'].rolling('4h').mean().head()

/usr/local/lib/python3.7/site-packages/dask/dataframe/rolling.py in mean(self)
    259     @derived_from(pd_Rolling)
    260     def mean(self):
--> 261         return self._call_method('mean')
    262
    263     @derived_from(pd_Rolling)

/usr/local/lib/python3.7/site-packages/dask/dataframe/rolling.py in _call_method(self, method_name, *args, **kwargs)
    247         return map_overlap(pandas_rolling_method, self.obj, before, after,
    248                            rolling_kwargs, method_name, *args,
--> 249                            token=method_name, meta=meta, **kwargs)
    250
    251     @derived_from(pd_Rolling)

/usr/local/lib/python3.7/site-packages/dask/dataframe/rolling.py in map_overlap(func, df, before, after, *args, **kwargs)
    112         deltas = pd.Series(df.divisions).diff().iloc[1:-1]
    113         if (before > deltas).any():
--> 114             raise ValueError(timedelta_partition_message)
    115         dsk.update({(name_a, i): (_tail_timedelta, (df_name, i), (df_name, i + 1), before)
    116                     for i in range(df.npartitions - 1)})

ValueError: Partition size is less than specified window. Try using ``df.repartition`` to increase the partition size

This is something that would be doable and that would make sense to look into it and try to change it, or it will change the internals of dask?

The challenge here is that Dask dataframe won't know how large your
partitions are likely to be when constructing the task graph for this
operation, so it will be difficult to do this automatically.
One option would be to allow the user to define the number of partitions
they want to participate in every rolling output chunk, but in practice I
think that asking them to repartition ahead of time with an informative
error message might be a good compromise.  I'm curious, are there
improvements that you think could be made to the error message that would
have helped you further?
On Tue, Nov 27, 2018 at 6:05 AM Rodrigo Neves<EMAIL_ADDRESS>wrote:

Right now, it's impossible to make any rolling operations (and probably
other operations that I do not remember now) with a partition size smaller
than the window requested.
Example:
import numpy as npimport pandas as pdimport dask.dataframe as ddimport randomimport fastparquet
#generate dummy dfdef generate_dask_df(size):
d = {
'dates':
pd.date_range('1980-01-01 00:00:00', periods=size, freq='1h'),
'signal1':
np.random.normal(0, 0.3, size=size).cumsum() + 50,
}
df = pd.DataFrame(d)

return dd.from_pandas(df.set_index('dates'), npartitions=5)

small_df = generate_dask_df(20).repartition(freq='3h')
small_df.divisions
(Timestamp('1980-01-01 00:00:00', freq='3H'),
Timestamp('1980-01-01 03:00:00', freq='3H'),
Timestamp('1980-01-01 06:00:00', freq='3H'),
Timestamp('1980-01-01 09:00:00', freq='3H'),
Timestamp('1980-01-01 12:00:00', freq='3H'),
Timestamp('1980-01-01 15:00:00', freq='3H'),
Timestamp('1980-01-01 18:00:00', freq='3H'),
Timestamp('1980-01-01 19:00:00'))
This works nicely
small_df['signal1'].rolling('3h').mean().head()
this doesn't
small_df['signal1'].rolling('4h').mean().head()
---------------------------------------------------------------------------ValueError                                Traceback (most recent call last) in ----> 1 small_df['signal1'].rolling('4h').mean().head()
/usr/local/lib/python3.7/site-packages/dask/dataframe/rolling.py in mean(self)
259     @derived_from(pd_Rolling)
260     def mean(self):--> 261         return self._call_method('mean')
262
263     @derived_from(pd_Rolling)
/usr/local/lib/python3.7/site-packages/dask/dataframe/rolling.py in _call_method(self, method_name, *args, **kwargs)
247         return map_overlap(pandas_rolling_method, self.obj, before, after,
248                            rolling_kwargs, method_name, *args,--> 249                            token=method_name, meta=meta, **kwargs)
250
251     @derived_from(pd_Rolling)
/usr/local/lib/python3.7/site-packages/dask/dataframe/rolling.py in map_overlap(func, df, before, after, *args, **kwargs)
112         deltas = pd.Series(df.divisions).diff().iloc[1:-1]
113         if (before > deltas).any():--> 114             raise ValueError(timedelta_partition_message)
115         dsk.update({(name_a, i): (_tail_timedelta, (df_name, i), (df_name, i + 1), before)
116                     for i in range(df.npartitions - 1)})
ValueError: Partition size is less than specified window. Try using df.repartition to increase the partition size
This is something that would be doable and that would make sense to look
into it and try to change it, or it will change the internals of dask?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/dask/dask/issues/4248, or mute the thread
https://github.com/notifications/unsubscribe-auth/AASszPYcbBxyTW8-PBedLxTuKG4kf4-zks5uzRxvgaJpZM4Y1P1C
.

In my opinion, the error message is very informative to the user.
Regarding only TimeDealta operations, having the knowledge of divisions we can know a priori the number of partitions necessary for the operation that we want to do. Regarding Integer operations where divisions are unknown that a whole different story because there we don't know the size of each partition. But probably it's possible to have different behaviours depending on the operation(Integer/TimeDelta)?
That is another thing that dask could maybe leverage from FastParquet metadata, that is, store the size/min/max and other statistics for each partition directly from metadata. This is probably impossible because then we would have to update this after each operation over the initial dask graph right?

#4796

Seems to me like you can't make rolling windows with a partition size smaller than the requested window either?

Partition size is less than overlapping window size. Try using df.repartition to increase the partition size.

ddf.repartition(partition_size=10000).rolling(1000).apply(wma, raw=False).compute()

Actually, I don't think that this issue is still relevant. There was a PR https://github.com/dask/dask/commit/d53b5608781da50fffff11179f9f6e466e6315d9 that implemented the desired logic for TimeDelta operations (that it does not seem to be what you are trying to do). Since implementing this logic for other operations types seems impossible I will close this for now


[opentelemetry] Build Monitoring for OpenTelemetry

Context
We need to observe the behavior of OpenTelemetry itself and eventually determine faulty at quality, infrastructure, processing or computation. For this we need a to expose own metrics to Prometheus and build dashboard that allows to observe such behavior.
Acceptance Criteria

[x] expose OTel metrics to prometheus in internal cluster
[x] build Plutono dashboard to observe stability for OpenTelemetry

References

The otel-logs collector is exposing internal metrics about the logs collection by default without any useful labels. That's why we scrape these metrics with the otel logs collector itself, add in a second step labels like region,  cluster and nodename and removing unused labels. These enriched metrics are exposed again on a different port. Prometheus is picking them up via podmonitor.

https://github.com/monitoringartist/opentelemetry-collector-monitoring could be a starting point for dashboard and alerts for the OTEL collector.

Enhance monitoring #409 follow up ticket


Can't login via cli to appwrite 0.15.x

appwrite login
? Enter your email **************
? Enter your password **********
? Enter the endpoint of your Appwrite server  **************
And got:
✗ Error Param "userId" is not optional.

@squallsama we're making some final updates, so the Appwrite CLI will have support for v0.15.x soon.
Keep an eye out on our Discord server for more info on this.

@squallsama we're making some final updates, so the Appwrite CLI will have support for v0.15.x soon.
Keep an eye out on our Discord server for more info on this.

any updates??

@squallsama we're making some final updates, so the Appwrite CLI will have support for v0.15.x soon.
Keep an eye out on our Discord server for more info on this.

any updates??

@Meldiron has been working on the CLI and we're currently doing more QA to ensure that you don't face any issues in the updated CLI
You can keep an eye out for these here: https://github.com/appwrite/sdk-generator/pull/484

any progress? we stucked on cli on new release.

Any clue on when will the bug be fixed ?? Stuck on CLI 0.17.1 with appwrite 0.15.1

We'll be sharing all updates regarding this issue on the PR here: appwrite/sdk-generator#484
It is just undergoing final QA, and should be merged soon. Closing this issue for now as it is redundant.

But it's still valid. Maybe it's better to close issue after releasing fix ?


Disable using local datasource on the server

Context
The command COPY my_table TO './local_path.csv' would work and might overload the server. Similarly, a user can read a local file on the server which shouldn't be possible.
Expected
Error when running local commands on the server.
Actual
Command works as expected.
Impact
Security concerns.

Hello. I'm having trouble setting up GlareDB for CI in PRQL/prql#3658.
The ability to read local files is definitely useful.
Could you consider a workaround such as allowing only certain directories like ClickHouse?
https://clickhouse.com/docs/en/operations/server-configuration-parameters/settings#user_files_path


Fe homework 10

https://video.drift.com/v/abOzShPT1cR8UHdlo7aMmqo5sWKxyHWYMM49AfZOh731A/
Added Items

Searching

Removing Item

Metric
Value

alert_status
:white_check_mark: OK

bugs
0

code_smells
0

reliability_rating
A (0 Bugs)Note: A being best, E being worst

security_rating
A (0 Vulnerabilities)Note: A being best, E being worst

vulnerabilities
0

Metric
Value

alert_status
:white_check_mark: OK

bugs
0

code_smells
0

reliability_rating
A (0 Bugs)Note: A being best, E being worst

security_rating
A (0 Vulnerabilities)Note: A being best, E being worst

vulnerabilities
0

Nice work on the UI, keep it up
Commit messages must be proper


Make a better footer![FEATURE]

Is your feature request related to a problem? Please describe.
The footer looks kind of very basic and unfinished, also the hero section text is simple .
Describe the solution you'd like.
I would like to change the footer and remove the red color after effect with some matching colors. Change the  position of social  media buttons and edit the font in hero section text.
Application Version
v1.1.0
Screenshots
No response
Additional information
No response

@frypan01 Ok assigning you with the task. Please link this issue to your PR.

Can you assign to me?

Can you assign to me?

@frypan01 is already assigned. So, after @frypan01's PR is merged, @manishgupta416, you may open another PR to modify those changes.

I guess the one working on it is inactive so can u assign this to me ?

I guess the one working on it is inactive so can u assign this to me ?

Yeah sure @Aditi-1709.

Can you exactly specify what changes you expect?

Can you exactly specify what changes you expect?

@Aditi-1709 Already @frypan01 has made a PR changing some button transition colours. I shall review that now. If you think you want to bring some good changes to the footer such that it looks visually appealing, you can do so through another PR.

@Aditi-1709 Are you working on this issue? Else I may close this.

You said the @frypan01 did it so I wasn't working. If it's still available, I can do it .

And yes, it will be better if you specify all your requirements.

You said the @frypan01 did it so I wasn't working. If it's still available, I can do it .

Ok @Aditi-1709 , you need to make the footer visually more appealing and beautiful ensuring that it should be responsive. You can change the colour or add glow or colourful shadow to the icons and such other things of your choice to make the footer look beautiful.
You are already assigned so you may start the work. If you face any problem, let me know and I shall try to help you.

Closing this as inactive.


Use Perplexica's API

Great project, would like to let you know that there's an API for Perplexica now that you can make use of instead of scraping contents from the web app or using the web socket. You can read more over here: https://github.com/ItzCrazyKns/Perplexica/blob/master/docs/API/SEARCH.md

Thanks for the update! We will work on incorporating the API. Meanwhile, we welcome and appreciate open source contributions!