Heroku sqlalchemy error

Dear Miguel,
Am trying with the deployment via heroku platform, however I ran into an error I couldn't understand, after pushing the code and run it remotely on heroku, the app is giving internal server errors (500) due to uninitialized database (no tables), so after running the command:
heroku run python manage.py deploy
it gave me the following trace:
(EMU_app_venv) C:\Users\Mohamad\PycharmProjects\EMU_app>heroku run python manage.py deploy
Running python manage.py deploy on emu-app... up, run.3978
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade  -> a07ae8f8165b, empty message
INFO  [alembic.runtime.migration] Running upgrade a07ae8f8165b -> cfadf15f7921, empty message
INFO  [alembic.runtime.migration] Running upgrade cfadf15f7921 -> 27046ebd3882, empty message
INFO  [alembic.runtime.migration] Running upgrade 27046ebd3882 -> 8f2f7828bf1d, empty message
INFO  [alembic.runtime.migration] Running upgrade 8f2f7828bf1d -> 7ffc818a24c1, empty message
Traceback (most recent call last):
  File "manage.py", line 42, in <module>
    manager.run()
  File "/app/.heroku/python/lib/python2.7/site-packages/flask_script/__init__.py", line 412, in run
    result = self.handle(sys.argv[0], sys.argv[1:])
  File "/app/.heroku/python/lib/python2.7/site-packages/flask_script/__init__.py", line 383, in handle
    res = handle(*args, **config)
  File "/app/.heroku/python/lib/python2.7/site-packages/flask_script/commands.py", line 216, in __call__
    return self.run(*args, **kwargs)
  File "manage.py", line 25, in deploy
    upgrade()
  File "/app/.heroku/python/lib/python2.7/site-packages/flask_migrate/__init__.py", line 235, in upgrade
    command.upgrade(config, revision, sql=sql, tag=tag)
  File "/app/.heroku/python/lib/python2.7/site-packages/alembic/command.py", line 174, in upgrade
    script.run_env()
  File "/app/.heroku/python/lib/python2.7/site-packages/alembic/script/base.py", line 397, in run_env
    util.load_python_file(self.dir, 'env.py')
  File "/app/.heroku/python/lib/python2.7/site-packages/alembic/util/pyfiles.py", line 93, in load_python_file
    module = load_module_py(module_id, path)
  File "/app/.heroku/python/lib/python2.7/site-packages/alembic/util/compat.py", line 79, in load_module_py
    mod = imp.load_source(module_id, path, fp)
  File "migrations/env.py", line 87, in <module>
    run_migrations_online()
  File "migrations/env.py", line 80, in run_migrations_online
    context.run_migrations()
  File "<string>", line 8, in run_migrations
  File "/app/.heroku/python/lib/python2.7/site-packages/alembic/runtime/environment.py", line 797, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/app/.heroku/python/lib/python2.7/site-packages/alembic/runtime/migration.py", line 312, in run_migrations
    step.migration_fn(**kw)
  File "/app/migrations/versions/7ffc818a24c1_.py", line 20, in upgrade
    op.create_index(op.f('ix_users_email'), 'users', ['email'], unique=True)
  File "<string>", line 8, in create_index
  File "<string>", line 3, in create_index
  File "/app/.heroku/python/lib/python2.7/site-packages/alembic/operations/ops.py", line 849, in create_index
    return operations.invoke(op)
  File "/app/.heroku/python/lib/python2.7/site-packages/alembic/operations/base.py", line 318, in invoke
    return fn(self, operation)
  File "/app/.heroku/python/lib/python2.7/site-packages/alembic/operations/toimpl.py", line 88, in create_index
    operations.impl.create_index(idx)
  File "/app/.heroku/python/lib/python2.7/site-packages/alembic/ddl/impl.py", line 206, in create_index
    self._exec(schema.CreateIndex(index))
  File "/app/.heroku/python/lib/python2.7/site-packages/alembic/ddl/impl.py", line 118, in _exec
    return conn.execute(construct, *multiparams, **params)
  File "/app/.heroku/python/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 914, in execute
    return meth(self, multiparams, params)
  File "/app/.heroku/python/lib/python2.7/site-packages/sqlalchemy/sql/ddl.py", line 68, in _execute_on_connection
    return connection._execute_ddl(self, multiparams, params)
  File "/app/.heroku/python/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 968, in _execute_ddl
    compiled
  File "/app/.heroku/python/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
    context)
  File "/app/.heroku/python/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
    exc_info
  File "/app/.heroku/python/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/app/.heroku/python/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
    context)
  File "/app/.heroku/python/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) relation "ix_users_email" already exists
 [SQL: 'CREATE UNIQUE INDEX ix_users_email ON users (email)']

(EMU_app_venv) C:\Users\Mohamad\PycharmProjects\EMU_app>

The error says that there is an index that already exists. You need to review your migrations, you'll probably find that the index has been created already.

I have opened the shell remotely and drop_all then create_all then run the deploy function and it gave
Running python manage.py deploy on emu-app... up, run.4714
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.

So then I entered the application to register a new user via browser and opened the logs on cmd, it gave me again an error (although it worked on local dev server)

2016-06-19T10:04:30.879788+00:00 app[web.1]: Traceback (most recent call last):
2016-06-19T10:04:30.879801+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/logging/handlers.py", line 934, in emit
2016-06-19T10:04:30.879830+00:00 app[web.1]:     ",".join(self.toaddrs),
2016-06-19T10:04:30.879842+00:00 app[web.1]: TypeError: sequence item 0: expected string, NoneType found
2016-06-19T10:04:30.879842+00:00 app[web.1]: Logged from file app.py, line 1587
2016-06-19T10:04:30.880028+00:00 app[web.1]: Exception on /auth/register [POST]
2016-06-19T10:04:30.880036+00:00 app[web.1]: Traceback (most recent call last):
2016-06-19T10:04:30.880037+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1988, in wsgi_app
2016-06-19T10:04:30.880038+00:00 app[web.1]:     response = self.full_dispatch_request()
2016-06-19T10:04:30.880039+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1641, in full_dispatch_request
2016-06-19T10:04:30.880039+00:00 app[web.1]:     rv = self.handle_user_exception(e)
2016-06-19T10:04:30.880040+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1544, in handle_user_exception
2016-06-19T10:04:30.880051+00:00 app[web.1]:     reraise(exc_type, exc_value, tb)
2016-06-19T10:04:30.880052+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1639, in full_dispatch_request
2016-06-19T10:04:30.880052+00:00 app[web.1]:     rv = self.dispatch_request()
2016-06-19T10:04:30.880052+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/flask/app.py", line 1625, in dispatch_request
2016-06-19T10:04:30.880053+00:00 app[web.1]:     return self.view_functions[rule.endpoint](**req.view_args)
2016-06-19T10:04:30.880053+00:00 app[web.1]:   File "/app/app/auth/views.py", line 33, in register
2016-06-19T10:04:30.880054+00:00 app[web.1]:     if form.validate_on_submit():
2016-06-19T10:04:30.880054+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/flask_wtf/form.py", line 166, in validate_on_submit
2016-06-19T10:04:30.880055+00:00 app[web.1]:     return self.is_submitted() and self.validate()
2016-06-19T10:04:30.880055+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/wtforms/form.py", line 310, in validate
2016-06-19T10:04:30.880055+00:00 app[web.1]:     return super(Form, self).validate(extra)
2016-06-19T10:04:30.880056+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/wtforms/form.py", line 152, in validate
2016-06-19T10:04:30.880056+00:00 app[web.1]:     if not field.validate(self, extra):
2016-06-19T10:04:30.880057+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/wtforms/fields/core.py", line 204, in validate
2016-06-19T10:04:30.880057+00:00 app[web.1]:     stop_validation = self._run_validation_chain(form, chain)
2016-06-19T10:04:30.880058+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/wtforms/fields/core.py", line 224, in _run_validation_chain
2016-06-19T10:04:30.880058+00:00 app[web.1]:     validator(form, self)
2016-06-19T10:04:30.880059+00:00 app[web.1]:   File "/app/app/auth/forms.py", line 28, in validate_email
2016-06-19T10:04:30.880059+00:00 app[web.1]:     if User.query.filter_by(email=field.data).first():
2016-06-19T10:04:30.880059+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2659, in first
2016-06-19T10:04:30.880060+00:00 app[web.1]:     ret = list(self[0:1])
2016-06-19T10:04:30.880060+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2457, in __getitem__
2016-06-19T10:04:30.880060+00:00 app[web.1]:     return list(res)
2016-06-19T10:04:30.880061+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2761, in __iter__
2016-06-19T10:04:30.880061+00:00 app[web.1]:     return self._execute_and_instances(context)
2016-06-19T10:04:30.880062+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2776, in _execute_and_instances
2016-06-19T10:04:30.880062+00:00 app[web.1]:     result = conn.execute(querycontext.statement, self._params)
2016-06-19T10:04:30.880063+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 914, in execute
2016-06-19T10:04:30.880063+00:00 app[web.1]:     return meth(self, multiparams, params)
2016-06-19T10:04:30.880064+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 323, in _execute_on_connection
2016-06-19T10:04:30.880064+00:00 app[web.1]:     return connection._execute_clauseelement(self, multiparams, params)
2016-06-19T10:04:30.880065+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
2016-06-19T10:04:30.880065+00:00 app[web.1]:     compiled_sql, distilled_params
2016-06-19T10:04:30.880065+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1146, in _execute_context
2016-06-19T10:04:30.880066+00:00 app[web.1]:     context)
2016-06-19T10:04:30.880066+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1341, in _handle_dbapi_exception
2016-06-19T10:04:30.880067+00:00 app[web.1]:     exc_info
2016-06-19T10:04:30.880067+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
2016-06-19T10:04:30.880067+00:00 app[web.1]:     reraise(type(exception), exception, tb=exc_tb, cause=cause)
2016-06-19T10:04:30.880068+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
2016-06-19T10:04:30.880068+00:00 app[web.1]:     context)
2016-06-19T10:04:30.880068+00:00 app[web.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 450, in do_execute
2016-06-19T10:04:30.880069+00:00 app[web.1]:     cursor.execute(statement, parameters)
2016-06-19T10:04:30.880069+00:00 app[web.1]: ProgrammingError: (psycopg2.ProgrammingError) relation "users" does not exist
2016-06-19T10:04:30.880070+00:00 app[web.1]: LINE 2: FROM users
2016-06-19T10:04:30.880070+00:00 app[web.1]:              ^

it appears to me that PostgreSQL refuse the validate_email after registering?
please I need help..
is this errors platform dependent ? if so ( is there any easier platform for deploying python)

When you use drop_all you are only dropping the tables the application knows about. In particular, you are not dropping the little table that stores the migration version. For this reason, when you then run manage.py deploy, the system thinks it is already migrated to latest revision so it does nothing.
I still claim that there is some sort of inconsistency between the migrations that you have created and the state of your models.

Start with an empty database. Then apply the migrations and see where is the error. That will point to the migration that is incorrect.

Mine is really disturbing
'''
Exception on /auth/login [POST]
Traceback (most recent call last):
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
psycopg2.ProgrammingError: relation "user" does not exist
LINE 2: FROM "user"
^
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
response = self.full_dispatch_request()
File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/app/.heroku/python/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
raise value
File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
rv = self.dispatch_request()
File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
return self.view_functionsrule.endpoint
File "/app/app/auth/routes.py", line 23, in login
user = User.query.filter_by(username=username).first()
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3215, in first
ret = list(self[0:1])
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3007, in getitem
return list(res)
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3317, in iter
return self._execute_and_instances(context)
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3342, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
return meth(self, multiparams, params)
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1107, in _execute_clauseelement
distilled_params,
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
e, statement, parameters, cursor, context
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
raise value.with_traceback(tb)
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) relation "user" does not exist
LINE 2: FROM "user"
^
[SQL: SELECT "user".id AS user_id, "user".username AS user_username, "user".email AS user_email, "user".password_hash AS user_password_hash, "user".about_me AS user_about_me, "user".last_seen AS user_last_seen
FROM "user"
WHERE "user".username = %(username_1)s
LIMIT %(param_1)s]
[parameters: {'username_1': 'paphra', 'param_1': 1}]
(Background on this error at: http://sqlalche.me/e/f405)
'''
I do not know what to do.

@Paphra the error indicates that you do not have a user table in your database.

Mine is really disturbing
Exception on /auth/login [POST]
Traceback (most recent call last):
  File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
    cursor, statement, parameters, context
  File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
    cursor.execute(statement, parameters)
psycopg2.ProgrammingError: relation "user" does not exist
LINE 2: FROM "user"
             ^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/app/.heroku/python/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/app/.heroku/python/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/app/app/auth/routes.py", line 23, in login
    user = User.query.filter_by(username=username).first()
  File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3215, in first
    ret = list(self[0:1])
  File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3007, in __getitem__
    return list(res)
  File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3317, in __iter__
    return self._execute_and_instances(context)
  File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3342, in _execute_and_instances
    result = conn.execute(querycontext.statement, self._params)
  File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
    return meth(self, multiparams, params)
  File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1107, in _execute_clauseelement
    distilled_params,
  File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
    e, statement, parameters, cursor, context
  File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
    util.raise_from_cause(sqlalchemy_exception, exc_info)
  File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
    raise value.with_traceback(tb)
  File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
    cursor, statement, parameters, context
  File "/app/.heroku/python/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) relation "user" does not exist
LINE 2: FROM "user"
             ^

[SQL: SELECT "user".id AS user_id, "user".username AS user_username, "user".email AS user_email, "user".password_hash AS user_password_hash, "user".about_me AS user_about_me, "user".last_seen AS user_last_seen
FROM "user"
WHERE "user".username = %(username_1)s
 LIMIT %(param_1)s]
[parameters: {'username_1': 'paphra', 'param_1': 1}]
(Background on this error at: http://sqlalche.me/e/f405)

I do not know what to do.

Have you solved this issue? I'm having the same... The User table does exists and works locally... so I don't know why in Heroku it doesn't work @miguelgrinberg @Paphra

@parismollo the fact that you have the table in your local database does not indicate anything about the state of the Postgres database that you use with your Heroku app. The error claims the user table does not exist there.

@miguelgrinberg  Fixed with heroku run flask db upgrade.
Thank you !

tried the above solution, but threw me this error.
Traceback (most recent call last):
File "/app/.heroku/python/bin/flask", line 8, in
sys.exit(main())
File "/app/.heroku/python/lib/python3.6/site-packages/flask/cli.py", line 967, in main
cli.main(args=sys.argv[1:], prog_name="python -m flask" if as_module else None)
File "/app/.heroku/python/lib/python3.6/site-packages/flask/cli.py", line 586, in main
return super(FlaskGroup, self).main(*args, **kwargs)
File "/app/.heroku/python/lib/python3.6/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/app/.heroku/python/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/app/.heroku/python/lib/python3.6/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/app/.heroku/python/lib/python3.6/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/app/.heroku/python/lib/python3.6/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/app/.heroku/python/lib/python3.6/site-packages/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/app/.heroku/python/lib/python3.6/site-packages/flask/cli.py", line 425, in decorator
with __ctx.ensure_object(ScriptInfo).load_app().app_context():
File "/app/.heroku/python/lib/python3.6/site-packages/flask/cli.py", line 392, in load_app
app = locate_app(self, import_name, None, raise_if_not_found=False)
File "/app/.heroku/python/lib/python3.6/site-packages/flask/cli.py", line 257, in locate_app
return find_best_app(script_info, module)
File "/app/.heroku/python/lib/python3.6/site-packages/flask/cli.py", line 83, in find_best_app
app = call_factory(script_info, app_factory)
File "/app/.heroku/python/lib/python3.6/site-packages/flask/cli.py", line 117, in call_factory
return app_factory(script_info)
File "/app/app/init.py", line 23, in create_app
app.config.from_object(config_options[config_name])
KeyError: <flask.cli.ScriptInfo object at 0x7f8d6a5d77b8>
here's my create_app
from flask import Flask
from flask_bootstrap import Bootstrap
from config import config_options
from flask_sqlalchemy import SQLAlchemy
from flask_login import LoginManager
from flask_uploads import UploadSet,configure_uploads,IMAGES
from flask_mail import Mail
from flask_simplemde import SimpleMDE
login_manager = LoginManager()
login_manager.session_protection = 'strong'
login_manager.login_view = 'auth.login'
simple = SimpleMDE()
bootstrap = Bootstrap()
db = SQLAlchemy()
mail = Mail()
photos = UploadSet('photos',IMAGES)
def create_app(config_name):
app = Flask(name)
app.config.from_object(config_options[config_name])
config_options[config_name].init_app(app)

bootstrap.init_app(app)
db.init_app(app)
login_manager.init_app(app)
mail.init_app(app)
simple.init_app(app)

from .main import main as main_blueprint
app.register_blueprint(main_blueprint)

from .auth import auth as auth_blueprint
app.register_blueprint(auth_blueprint,url_prefix = '/auth')

configure_uploads(app,photos)

return app

will appreciate any help

will appreciate any help

To use flask db migrate ensure that you have installed flask migrate and initialized it in your application.Then, in your create_app factory function, add the argument db to migrate.init_app.
From:
# Your previous code

def create_app(config_name):
	app = Flask(name)

	app.config.from_object(config_options[config_name])
	config_options[config_name].init_app(app)

	bootstrap.init_app(app)
	db.init_app(app)
	login_manager.init_app(app)
	mail.init_app(app)
	simple.init_app(app)

	from .main import main as main_blueprint
	app.register_blueprint(main_blueprint)

	from .auth import auth as auth_blueprint
	app.register_blueprint(auth_blueprint,url_prefix = '/auth')

	configure_uploads(app,photos)

return app

To:
# Your previous imports
from flask_migrate import Migrate # <-------------------------------------------------------------- new

migrate = Migrate() # <-------------------------------------------------------------- new

def create_app(config_name):
	app = Flask(name)

	app.config.from_object(config_options[config_name])
	config_options[config_name].init_app(app)

	bootstrap.init_app(app)
	db.init_app(app)
	login_manager.init_app(app)
	mail.init_app(app)
	simple.init_app(app)
	migrate.init_app(app, db) # <-------------------------------------------------------------- new

	from .main import main as main_blueprint
	app.register_blueprint(main_blueprint)

	from .auth import auth as auth_blueprint
	app.register_blueprint(auth_blueprint,url_prefix = '/auth')

	configure_uploads(app,photos)

return app

Then, you can run heroku run flask db upgrade


Path aliases not respected

Currently whenever I run ‘pnpm dr:build’ all the imports start with “@“ eg “@/routes” even though I have defined only “~” pointing to “./src” in my tsconfig. Is there a way to instruct package on which alias it should use?

Not right now. We'd have to update the CLI code to handle that. Feel free to PR.


Added Order Placed Button With Truck Animation

This pull request introduces the implementation of the Order Place Button With Truck Animation, a fun and engaging interactive button for users. The purpose of this PR is to add this exciting button to the repository and make it available for users to enjoy.
🚚😊
Fixes: #689

@Spyware007 Could You Check My PR And Merge It !!😊☺

Hi, there seems to be a merge conflict, please resolve it


@auth with list of owners results in "type mismatch error, expected type LIST"

Describe the bug
When creating an item that has auth specified as a list of owners without specifying the owner on creation results in the owner field being automatically assigned a "string" rather than ["string"].
To Reproduce
Create a new Amplify project with amplify init
amplify add api and select GraphQL
Setup API with Cognito User Pool
Create the following schema
type Task
  @model
  @auth(rules: [
    { allow: owner, ownerField: "collaborators" }
  ]) {
  id: ID!
  name: String!
  description: String
  collaborators: [string]
}

amplify push
Go to AppSync console > Query. Login as a user.
Run the following mutation
mutation CreateTask {
    createTask(input: { name: "task 1", description: "task 1 description" }) {
        id
        name
    	description
    	collaborators
    }
}

This is the result
{
  "data": {
    "createTask": {
      "id": "ef3ee7ae-781b-440f-9079-caa4ac88961f",
      "name": "task 1",
      "description": "task 1 description",
      "collaborators": null
    }
  },
  "errors": [
    {
      "path": [
        "createTask",
        "collaborators"
      ],
      "locations": null,
      "message": "Can't resolve value (/createTask/collaborators) : type mismatch error, expected type LIST"
    }
  ]
}

The Task dynamo DB table contains the user ID in the collaborators field as a string:
collaborators: "abcd"
Expected behavior
As per the documentation here
Expected result is collaborators is assigned a list with the ID of the owner.
{
  "data": {
    "createTask": {
      "id": "ef3ee7ae-781b-440f-9079-caa4ac88961f",
      "name": "task 1",
      "description": "task 1 description",
      "collaborators": ["abcd"]
    }
  }
}

Additional context
aws-amplify v1.1.7

Is this your exact schema?
type Task
  @model
  @auth(rules: [
    { allow: owner, ownerField: "collaborators" }
  ]) {
  id: ID!
  name: String!
  description: String
  collaborators: [string]
}

If so you need to capitalize "string" to be "String".
type Task
  @model
  @auth(rules: [
    { allow: owner, ownerField: "collaborators" }
  ]) {
  id: ID!
  name: String!
  description: String
  collaborators: [String]
}

The schema validation should have caught this. I will look into why it didn't error at build time.

@mikeparisstuff sorry I made a mistake, I had tried many different ways before I simplified it down to the schema I provided and manually updated the string field - you are correct, it should be String, the schema validation does flag this as an error.
I have run through the steps on a new project and copied the exact code that was used (updated in the original post. The error still exists.


Include paths contain godot_cpp

It is recommended by the Godot maintainers that GDExtensions are written as if they are part of the engine code. Therefore, any includes should not contain godot_cpp/:
Example:
#include <godot_cpp/godot.hpp>

should be:
#include <godot.hpp>

The SConstruct file needs updating to ensure this behaviour.

Hey @bitbrain! Thanks for the feedback. From which maintainer did you hear this from?
This is very confusing since the official godot-cpp example even does it the this way.
If there were a concise documentation like proposed here: https://github.com/godotengine/godot-docs/issues/5618, it would be easier to check up on the correct way to do this. However they even mentioned in the blog post regarding GDExtension that even the official repo isn't structured in the correct way. This project was mainly introduced so others could have another stepping stone to try this new feature out. If there will be any updates concerning GDExtension from the maintainers, I will be glad to update this repo.

From which maintainer did you hear this from?

I saw it in an issue thread on Github, I cannot remember who said it.
Since there is no official stance on this anywhere documented let me close this again.


Add ability for Dapr CLI to add/get secrets from secret store component

Describe the proposal
Dapr CLI should have a command like dapr secrets to add/put secrets from a secret store component. Something like:
dapr secrets add  --data '{"connectionString":"343ab342c34a2"}'
dapr secrets add  --data-file ./mysecrets.json
dapr secrets get  --key "connectionString"
With https://github.com/dapr/dapr/issues/3301 the set secret command should work with any secret component.
Without a POST API, the CLI set command add and get commands should interact with the local JSON file created by https://github.com/dapr/cli/issues/741
Release Note
RELEASE NOTE: ADD new secrets command to get/set secrets from a local JSON secret store

Pairs with #741 to improve secrets management for Dapr

I like this idea, but I think it should probably be write only. We add the CLI to simplify adding secrets, that can then only be read by the Dapr secrets API.

Depends on https://github.com/dapr/dapr/issues/3301

I like this idea, but I think it should probably be write only. We add the CLI to simplify adding secrets, that can then only be read by the Dapr secrets API.

I actually think adding the ability to get secrets from the CLI would be a useful inner loop feature. Component scoping can allow CLI access to be disabled in production scenarios

Also a dapr secrets list which returns the known keys (which can then be retrieved using dapr secrets get.  It's often useful for developers (and their tools) to be able see values as their application sees them rather than have to dig through the the various backing stores to uncover them.

Mockup for local file

Mockup for any secret store

Based on feedback from this morning it sounds like dapr user-secrets is the preferred command to indicate this is for local development only.


Sub task handling

POC of sub task handling. Result of #11 .
On my test environment I'm not able to see labels. The background color is not propagated even if the color defaults to black :-( So I'm not sure if it's just my issue or it's some bug in display.
I can see labels only if I select them.
Ready for comments and advises as I'm not JavaScript or TypeScript developer.

The labels and label colors are pulled directly from todoist, I'm guessing that if you're adding labels, without adding colors, that will have problems displaying.
(I'm just quickly looking at this on my mobile, haven't had a chance to test it)
But I wonder what would be the best way of displaying subtasks. Maybe we shouldn't be mixing that up with labels, which are a different concept. Maybe an indent would work? Or a smaller font?
I'm also curious as to how this works with sorting options, I don think it's easy to get the subtasks to always show under their respective tasks...
Thanks for your work, I'm looking forward to evolving this into a great new feature.

Hi,
I wrote in the 1st line it's POC or in other words MVP :-)
I just want to have some link between task and the subtask. I was considering to add prefix to the content filed (actually it looks like title of the task). But it brings new problems with the update of the subtask.
Label seems to me minimal change and sorting? Still better than before when subtasks were displayed together with common tasks.

Can you post some screenshots of how it looks?
I'm currently reinstalling my OS and don't have a dev environmet set up

Next week. I'm not close to my HA.


Run rubocop at end of sprint

It's been three weeks, we're about due for a good lintin'.

Fixed in #180


API: consistency with .ix and .loc for getitem operations (GH8613)

closes #8613
In [1]:      df = DataFrame(np.random.randn(5,4), columns=list('ABCD'), index=date_range('20130101',periods=5))

In [2]:      df
Out[2]:
                   A         B         C         D
2013-01-01 -1.380065  1.221596  0.279703 -0.119258
2013-01-02  1.684202 -0.202251 -0.961145 -1.595015
2013-01-03 -0.623634 -2.377577 -3.024554 -0.298809
2013-01-04 -1.251699  0.456356  1.257002  1.465878
2013-01-05  0.687818 -2.125079  1.454911  1.914207

In [5]:      s = Series(range(5),[-2,-1,1,2,3])

In [6]:      s
Out[6]:
-2    0
-1    1
 1    2
 2    3
 3    4
dtype: int64

In [7]:      df.loc['2013-01-02':'2013-01-10']
Out[7]:
                   A         B         C         D
2013-01-02  1.684202 -0.202251 -0.961145 -1.595015
2013-01-03 -0.623634 -2.377577 -3.024554 -0.298809
2013-01-04 -1.251699  0.456356  1.257002  1.465878
2013-01-05  0.687818 -2.125079  1.454911  1.914207

In [9]:      s.loc[-10:3]
Out[9]:
-2    0
-1    1
 1    2
 2    3
 3    4
dtype: int64

# this used to be a KeyError
In [15]: df.loc[2:3]
TypeError: Cannot index datetime64 with <type 'int'> keys

Previously this worked
In [8]: s.loc[-1.0:2]
Out[2]:
 -1    1
  1    2
  2    3
 dtype: int64

# but will now raise
In [8]: s.loc[-1.0:2]
TypeError: Cannot index type <class 'pandas.core.index.Int64Index'> with float keys

cc @immerrr
@jorisvandenbossche
@shoyer
I think this should resolve things and make .loc strict, while matching .ix (notably in that .ix tries for positional access when it can, but .loc will refuse to guess).

First of all, big thanks for working on this, @jreback!
However, I don't think we need to enforce distinctions between float and int for slice bounds, because there is no ambiguity in the ordering between integers and floats (whereas the ordering with respect to strings, for example, is indeed problematic). Users shouldn't have to check the type of their index (if they know it's numeric) before doing indexing.
In particular:

I think it should be OK to index an Int64Index with floats, like .loc[0.5:2.5] or .loc[-1.0:2].
I think it should be OK to index a Float64Index with integers, like .loc[0:10]. (Of course, these are still to be interpreted as labels, not integer positions)

The binary search logic for determining slice bound locations should already work for both of these cases.

To give a more concrete example from my experience: an index corresponding to a variable like longitude might variously consist of integer (e.g., if it's in multiples of whole degrees) or as floating point numbers (e.g., if it's in multiples of half degrees). It's a bad user experience to need to cast their slice bounds to the type of index. Code like s.loc[min_lon:max_lon] should work regardless of dtypes.

@shoyer
I added in these checks, mainly because numpy is deprecating float indexers (IIRC it has a warning now), and we have been warning on it.
Its easy to take out, but I think that indexing with a float on an integer index is really not in the .loc domain. This is moving back to the .ix functionaility (which actually doesn't work in 0.15.2 or atm), though .loc did (which is odd).

@jorisvandenbossche what do you think?

I agree that using float indexers is a based idea for position based indexing (.iloc). Position based indexing should always use integers. We should be consistent with numpy for this.
But indexing with .loc uses labels. Here (e.g., with searchsorted), numpy is more flexible -- the important thing is that there is a well defined sort order.

this should then also work for .ix/[]?

To elaborate:
There is never a good reason to use floats for position based indexing. Numpy warns and/or raises because using a float is an indication that your code probably has a bug.
In contrast, there certainly are good reasons to use floats for label based indexing, because the labels might include floats themselves.

.ix and [] are an unpredictable mess! We hadn't even written down the rules for exactly how fall-back indexing with [] works last time I checked.
If I had my way, we would make some hard choices and deprecate/remove fall-back indexing all together (#9213). In the mean-time, I suspect the best approach is to simply try to break/change as little existing code as possible.

ok, here is the change (plus making some testing more robust)
In [3]:  index = tm.makeIntIndex()
In [4]:  s = Series(np.arange(len(index))+10,index+5)
In [5]: s
Out[5]:
5     10
6     11
7     12
8     13
9     14
10    15
11    16
12    17
13    18
14    19
dtype: int64

In [6]: s.loc[6.0:8.5]
Out[6]:
6    11
7    12
8    13
dtype: int64

In [7]: s.loc[5.5:8.5]
Out[7]:
6    11
7    12
8    13
dtype: int64

In [8]: s.loc[5.5:8.0]
Out[8]:
6    11
7    12
8    13
dtype: int64

I could enable for []/ix, but then would s[5.0:8.0] because fallback, so what would s[5.5,8.5] do (non-fallback I think), so they become very sensitive to the actual values, hence bad

I agree with the logic of allowing float indexers for integer indexes when using loc (so as you now implemented)
Ideally, I think it should also work for ix (to keep ix as much as possible to the logic of try: s.loc[]; except: s.iloc[]. In principle , we say that for an integer index axis, we do never integer location fallback (http://pandas.pydata.org/pandas-docs/stable/indexing.html#different-choices-for-indexing), so following the loc behaviour should not be ambiguous in this case I think.
For [], it is something else. For that, we should first have a better documented overview of it behaviour (I started with that last week, will try to put it up as a PR)

ok, I added the .ix support for float indexers on an Int index. It is trivial to make this work for []. I am thinking that this should have the same semantics as .ix (e.g. label based with an int index). It works the same.

But for [] slicing is 'integer-location' based (an exception on the other types of slicing), so this should follow the semantics of iloc in this case, disallowing floats.

Example:
In [46]: s = Series(range(4), index=[-1,0,1,2])

In [47]: s
Out[47]:
-1    0
 0    1
 1    2
 2    3
dtype: int64

In [48]: s[0:2]    # <--- slicing = integer location based
Out[48]:
-1    0
 0    1
dtype: int64

In [49]: s[0]    # <--- single label = label based
Out[49]: 1

ok, is updated, any more comments
@jorisvandenbossche @shoyer

Generally seems fine by me.

indexing is such a rabbit hole :)
I cleaned up _convert_slice_indexer a bit. I tried to use _maybe_cast_slice_bound directly there (I added the kind argument which is the type of indexing, e.g. ix/loc/iloc/getitem). But so many cases are handled that its really hard to get this right. Because the different kinds are handled in different places
its tricky because you have the kinds of indexers (ix/loc/iloc/getitem) x index_types (Index,Float,Int,Datetime....) x kinds of indexing (slice/scalar)
So its better  (meaning slightly cleaner), with the semantics we discussed above.

Thanks a lot for this!

Indeed, greatly appreciated!

That's great that you guys (and especially @jreback) actually went through this :+1:

trying to make indexing slightly less of a rabbit hole :)


Align dependencies and Golang version with the rest

Well as per title. We're moving to a world, where the K8s dependencies are defined by pkg and they "just" update.
Also bumped the Golang version for good measure to prime new repos correctly. The licenceclassifier seems to have been a thing of the past too.
/assign @mattmoor @vaikas @dprotaso

Thanks!!
/lgtm
/approve

Thanks!!
/lgtm
/approve


404 page error / german GEMA

I really like the 404 page. The panic button is also very funny.
But in germany - there is the GEMA. They block this video:

Maybe we can find something else. Something that doesn't need too much bandwith.
I know everyone can change the button, but something that work for everyone by default would be nice.
Any suggestions?

dam! I suppose that we need to embed another youtube video with copyright 😐

Maybe theres something more lightweight than a youtube video? Like:
https://github.com/notwaldorf/emoji-rain - Demo
or a simple gif:
https://media.giphy.com/media/lKZEeXJGhU1d6/giphy.gif

The problem is that currently we have many JS and CSS around 404. I love emoji rain, but change this part of the project cost the same that make a blog refactor and I feel that I need to support other issues first.
I'm sorry but I have to close this issue to maintain common points of the project. Of course feel free to implement in your personal fork. (and show me when you do it, please)
Why GEMA, WHY.


bump version to 12.12.1

Corrects name for checkboxes documentation (PR #638)

Component guide for this PR:
https://govuk-publishing-compon-pr-639.herokuapp.com/component-guide/

Hey @DilwoarH , just checking for all the dependabot PR's about this release, what does this change for the dependant applications?


[deps] Add support for django-allauth 0.43.0

Closes #200

Coverage remained the same at 98.477% when pulling 2d5c0cadcd8830042f6771967a5da58416ea6e23 on Purhan:issues/200-upgrade-django-allauth into 57bc06842425d95c738ddb9cb2c4eab3b8f745bd on openwisp:master.


"Update" section does not correctly verify solution

I was excited when my solution to Update worked and I checked my solution against the recommended solution and realized I had an error in my code.  I went back and made changes to my code to make it even more wrong and nothing I did would cause the 'verify' solution to fail.
My bad code which should fail:
var url = 'mongodb://localhost:27017/learnyoumongo';

var mongo = require('mongodb').MongoClient;

mongo.connect(url, function(err, db) {
  if(err) {
    console.log("Error Connecting to DB: ", err);
    db.close();
    return 0;
  }

  var collection = db.collection('docs');

    collection.update({
        bob: 'bob'
    },{
        $set: { age: 41 }
    }, function(err, data) {
        if(err) throw err;
        db.close();
    });
});

the 'Remove' section does not correctly verify either.

Confirmed. I will try and get this fixed as soon as I can.

Sorry for the delay. Both of those exercises have been fixed in v2.0.0. I did have to make some breaking changes though, so be sure to read the problem carefully :]


free more memory on destruction

with these most direct leaks reported by asan should be handled, the only thing remaining is the shared/weakptr reports and a few indirect ones inside wlroots/libwayland which im not sure if its just false positives.

uhh clang format

uhh clang format

Im trying to figure out what it needs, effing arch being outdated 🥺

"/__w/Hyprland/Hyprland/src/desktop/Popup.cpp:138:6: error: code should be clang-formatted [-Wclang-format-violations" i mean i didnt even touch that file lol

uhh clang format

yeah i think it comes from this commit https://github.com/hyprwm/Hyprland/commit/155fe6f165f70ea69a44cbd6989fc01a473e6946 heh

just clang-format this and I'll merge it and we'll be done


journal_server: enable journaling for everyone by default

lgtm. assuming this'll go in after the next public build?

In the standup today, everyone decided it should go into the next public build.  So will merge as soon as I fix the tests (need to disable auto journals by default for the tests still).


maximum upload/download speed not reached

When running ollama pull, in some cases the download rate is lower than downloading with wget or the browser

I am also facing this issue

This was fixed in #626


Type system entirely broken in mac-thumb2 platform binaries.

Version and Platform (required):

Binary Ninja Version: 3.6.4654
OS: macOS

Bug Description:
The type system does not work at all in mac-thumb2 platform binaries. Types cannot be changed.
The following error is provided:
[Platform] Failed to parse platform types for file: /Applications/Binary Ninja.app/Contents/MacOS/plugins/../../Resources/types/platform/mac-thumb2.c

Steps To Reproduce:

Open Attached Binary
Try to change any type

identityservicesd-ios10.zip
Additional Information:
Type view edit dialog shows an empty error box when trying to change types. If you missed the error amidst the deluge of thumb2-related MachoView warnings, this may be confusing.

Fixed in dev builds >= 3.6.4751


Get feedback on prototype GBIF service

The prototype GBIF service is currently available on GBIF's test system, GBIF-UAT.org.  Once the minimum metadata is set up (#8) and if there are no significant problems, it can be enabled on GBIF.org.
The initial implementation is for general SQL downloads on any GBIF data, which provides a lot of flexibility.  Careful choice of SQL can produce a valid occurrence cube.
General documentation for making SQL downloads through the GBIF (UAT) API are here: https://techdocs.gbif-uat.org/en/data-use/api-sql-downloads
A guide for a cube is here: https://techdocs.gbif-uat.org/en/data-use/data-cubes
Note the test system only includes a small amount of data, and it will include test/incorrect data, so the resulting cubes are useless.

Hi @MattBlissett.
I am trying to reproduce your very clear documentation about API SQL Downloads.
I get the following error while trying the query validation step:
C:\Users\damiano_oldoni\Documents>curl --include --header "Content-Type: application/json" --data @query.json https://api.gbif-uat.org/v1/occurrence/download/request/validate
HTTP/1.1 404 Not Found
Vary: Origin, Access-Control-Request-Method, Access-Control-Request-Headers
Allow: DELETE, GET
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Pragma: no-cache
Expires: 0
X-Frame-Options: DENY
Content-Length: 0
Date: Tue, 13 Feb 2024 09:56:25 GMT
Cache-Control: no-cache, no-store, must-revalidate
X-Varnish: 203554900
Age: 0
Via: 1.1 varnish (Varnish/6.0)
Connection: keep-alive

where query.json is copied from your example (with my user name and my email):
{
  "creator": "userName",
  "notificationAddresses": [
    "userEmail"
  ],
  "sendNotification": true,
  "format": "SQL_TSV_ZIP",
  "sql": "SELECT datasetKey, countryCode, COUNT(*) FROM occurrence WHERE continent = 'EUROPE' GROUP BY datasetKey, countryCode"
}

Notice also that while trying to download by using the query above I get an error about output format (I used my credentials in curl commando here below):
C:\Users\damiano_oldoni\Documents>curl --include --user userName:PASSWORD--header "Content-Type: application/json" --data @query.json https://api.gbif-uat.org/v1/occurrence/download/request
HTTP/1.1 400 Bad Request
Vary: Origin, Access-Control-Request-Method, Access-Control-Request-Headers
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Pragma: no-cache
Expires: 0
X-Frame-Options: DENY
Content-Type: text/plain
Content-Length: 52
Date: Tue, 13 Feb 2024 10:02:55 GMT
Cache-Control: no-cache, no-store, must-revalidate
X-Varnish: 183776482
Age: 0
Via: 1.1 varnish (Varnish/6.0)
Connection: keep-alive

Cannot parse SQL_TSV_ZIP into a known DownloadFormat

I also tried the query for the species occurrence cube of Slovenia:
{
  "creator": "userName",
  "sendNotification": false,
  "format": "SQL_TSV_ZIP",
  "sql": "SELECT \"year\", gbif_eeargCode(1000, decimallatitude, decimallongitude, COALESCE(coordinateUncertaintyInMeters, 1000)) AS eeaCellCode, speciesKey, COUNT(*) AS \"count\", MIN(COALESCE(coordinateUncertaintyInMeters, 1000)) AS \"minCoordinateUncertaintyInMeters\" FROM occurrence WHERE occurrenceStatus = 'PRESENT' AND speciesKey IS NOT NULL NOT array_contains(issue, 'ZERO_COORDINATE') AND NOT array_contains(issue, 'COORDINATE_OUT_OF_RANGE') AND NOT array_contains(issue, 'COORDINATE_INVALID') AND NOT array_contains(issue, 'COUNTRY_COORDINATE_MISMATCH') AND (identificationVerificationStatus IS NULL OR NOT (LOWER(identificationVerificationStatus) LIKE '%unverified%' OR LOWER(identificationVerificationStatus) LIKE '%unvalidated%' OR LOWER(identificationVerificationStatus) LIKE '%not able to validate%' OR LOWER(identificationVerificationStatus) LIKE '%control could not be conclusive due to insufficient knowledge%' OR LOWER(identificationVerificationStatus) LIKE '%unconfirmed%' OR LOWER(identificationVerificationStatus) LIKE '%unconfirmed - not reviewed%' OR LOWER(identificationVerificationStatus) LIKE '%validation requested%')) AND countryCode = 'SI' AND \"year\" >= 2000 AND hasCoordinate GROUP BY \"year\", eeaCellCode, speciesKey ORDER BY \"year\" DESC, eeaCellCode ASC, speciesKey ASC"
}

I get a HTTP/1.1 400 Bad Request error. Also validation didn't work, same error as above.
Notice that queries for standard downloads work well both on GBIF as on GBIF-UAT. I could for example trigger a download based on the example shown in API Downloads page:
{
  "creator": "userName",
  "notificationAddresses": [
<EMAIL_ADDRESS>  ],
  "sendNotification": true,
  "format": "SIMPLE_CSV",
  "predicate": {
    "type": "and",
    "predicates": [
      {
        "type": "equals",
        "key": "BASIS_OF_RECORD",
        "value": "PRESERVED_SPECIMEN"
      },
      {
        "type": "in",
        "key": "COUNTRY",
        "values": [ "VC", "GD" ]
      }
    ]
  }
}

Any idea? Thanks for the technical support.

Hi,
Thanks for looking — unfortunately, you caught the UAT system just as I was testing a fix to the current production system, which doesn't yet support cubes.
I've now put the test system back to the cubes prototype version, and started a cube download: https://www.gbif-uat.org/occurrence/download/0000000-240213133425371

Thanks @MattBlissett. I timed it very well 😄
I have just repeated all the tests (both validation and download request) on UAT and everything works fine.


clean up build dependencies

I'm working on this package in nixpkgs and noticed that the build dependencies can be streamlined to only depend upon poetry-core. I picked poetry-core version 1.5.2 since that is what poetry 1.4 depends on.

I vaguely remember not having setup tools breaking things in the past but that was a while ago

If CI works it's good by me!


Paint state modifier

Paints never_idle, keep_walking, keep_on_path & keep_crafting green.
When there an uneven amount of those steps. To indicate it is turned on.

I understand what you mean, but I think it will be a very infrequent issue. Most of the time people won't go far back in step list to modify these types of steps. Since they mostly matter for active development.
Anyway, i will try just because i like having more colour in the GUI.

Done


Coverage 6.0 reports less than 100% coverage for Tomli

Describe the bug
Hello, Tomli maintainer here!
Coverage version 6.0 seems to report less than 100% coverage for tomli. Coverage versions 5.x correctly report 100% coverage. Note that version 6.0 is the first one that depends on Tomli. I expect this may be relevant.
I definitely do not expect you to debug this for me :smile: but given how well you know coverage, perhaps you can tell what's wrong or what I'm doing wrong off the top of your head.
To Reproduce

What version of Python are you using?

Python 3.8.10

What version of coverage.py are you using? The output of coverage debug sys is helpful.

coverage-6.0

What versions of what packages do you have installed?   Give us a specific commit of a specific repo that we can check out.

https://github.com/hukkin/tomli/tree/c5045f47782341190517ae62307be2bd14e13d2d

What commands did you run?

foo@bar:~$ cd /tmp/
foo@bar:/tmp$ git clone https://github.com/hukkin/tomli.git
Cloning into 'tomli'...
remote: Enumerating objects: 1885, done.
remote: Counting objects: 100% (510/510), done.
remote: Compressing objects: 100% (260/260), done.
remote: Total 1885 (delta 313), reused 335 (delta 229), pack-reused 1375
Receiving objects: 100% (1885/1885), 310.30 KiB | 1.85 MiB/s, done.
Resolving deltas: 100% (775/775), done.
foo@bar:/tmp$ cd tomli/
foo@bar:/tmp/tomli$ python3 -m venv venv
foo@bar:/tmp/tomli$ . venv/bin/activate
(venv) foo@bar:/tmp/tomli$ pip install . pytest-cov python-dateutil
Processing /tmp/tomli
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting pytest-cov
  Using cached pytest_cov-3.0.0-py3-none-any.whl (20 kB)
Collecting python-dateutil
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting pytest>=4.6
  Using cached pytest-6.2.5-py3-none-any.whl (280 kB)
Collecting coverage[toml]>=5.2.1
  Using cached coverage-6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (255 kB)
Collecting six>=1.5
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting py>=1.8.2
  Using cached py-1.10.0-py2.py3-none-any.whl (97 kB)
Collecting iniconfig
  Using cached iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Collecting attrs>=19.2.0
  Using cached attrs-21.2.0-py2.py3-none-any.whl (53 kB)
Collecting packaging
  Using cached packaging-21.0-py3-none-any.whl (40 kB)
Collecting toml
  Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting pluggy<2.0,>=0.12
  Using cached pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting pyparsing>=2.0.2
  Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Building wheels for collected packages: tomli
  Building wheel for tomli (PEP 517) ... done
  Created wheel for tomli: filename=tomli-1.2.1-py3-none-any.whl size=11869 sha256=9771a834d2d82bfb701afd93a640bbd093c9ac5d7896425e2b03ac0a6d8dcbc6
  Stored in directory: /tmp/pip-ephem-wheel-cache-4_qivjx_/wheels/2a/be/a2/c2e2f410105e5a607018317cab46427d50645c558623d032be
Successfully built tomli
Installing collected packages: py, iniconfig, attrs, pyparsing, packaging, toml, pluggy, pytest, tomli, coverage, pytest-cov, six, python-dateutil
Successfully installed attrs-21.2.0 coverage-6.0 iniconfig-1.1.1 packaging-21.0 pluggy-1.0.0 py-1.10.0 pyparsing-2.4.7 pytest-6.2.5 pytest-cov-3.0.0 python-dateutil-2.8.2 six-1.16.0 toml-0.10.2 tomli-1.2.1
(venv) foo@bar:/tmp/tomli$ python -m pytest --cov --cov-fail-under=100
========================================== test session starts ==========================================
platform linux -- Python 3.8.10, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /tmp/tomli, configfile: pyproject.toml
plugins: cov-3.0.0
collected 466 items

tests/test_error.py ...                                                                           [  0%]
tests/test_extras.py ............................................................................ [ 16%]
................................................................................................. [ 37%]
...............................................................................................   [ 58%]
tests/test_flags.py .                                                                             [ 58%]
tests/test_for_profiler.py .                                                                      [ 58%]
tests/test_misc.py ....                                                                           [ 59%]
tests/test_toml_compliance.py .............................................................x.x... [ 73%]
................................................................................................. [ 94%]
.........................                                                                         [100%]

---------- coverage: platform linux, python 3.8.10-final-0 -----------
Name               Stmts   Miss Branch BrPart  Cover
----------------------------------------------------
tomli/_parser.py     452     60    178      0    90%
tomli/_re.py          33     13      8      0    68%
----------------------------------------------------
TOTAL                485     73    186      0    89%

FAIL Required test coverage of 100% not reached. Total coverage: 88.82%

==================================== 464 passed, 2 xfailed in 1.34s =====================================

Expected behavior
Expected 100% coverage (as reported by coverage==5.*) but got 88%.

Worlds collide!  This is definitely because coverage is using tomli now.  The missing lines are all of the top-level statements in _parser.py and _re.py, presumably because they are imported by coverage.py before coverage measurement has started.  I'll have to think about what we can do about that...

Thanks for the quick response!

I'll have to think about what we can do about that...

Awesome! But don't worry/think too much about it. If this is something that affects exactly one project and one maintainer on the entire planet, then I think it's perfectly reasonable to decide that coverage does not support measuring coverage of its own dependencies.

Well, full disclosure, when I said "what we can do about that", there was a strong possibility that the answer would be something you would do in the tomli repo :)

This is now released as part of coverage 6.0.1.


Server does not start

Describe the bug
daytona server does not start.
 rm -rf ~/.config/daytona
daytona server
FATA[0000] open : no such file or directory

To Reproduce
Steps to reproduce the behavior:

Install v0.12.0
Delete your config
Try to start the server
See error

Expected behavior
The server would start
Desktop (please complete the following information):

OS: Mac 14.4.1
Daytona Version: v0.12.0

@metcalfc the server config is located at ~/.config/daytona only on Linux.
On Mac it's on ~/Library/Application\ Support/daytona/.

The error you have is because of breaking changes introduced in the last 2 versions. You can check how to upgrade in the release notes.

That's great and also surprising. I would expect config there from a dmg installed tool vs a command line install in ~/.config.

We use UserConfigDir() to determine the base path.


example codes are added

example codes are added

Draw_BOX

Draw_LINE  (computationally cheap)

Thanks! Cheaper computation is good. I'll modify my repo.


Autofilter does not working correctly when using a template

Hi ,
also there is an problem when you have multiple sheets . Then it not getting the right reference from localSheetId.
Workaround for me is to find the sheet via name.
Changes for the Workaround  :
zcl_excel_reader_2007:

See Pull request from me

The below Excel file can be used for the tests, by displaying it via ZDEMO_EXCEL37. Weird results.
issue_547.xlsx

@AndreaBorgia-Abo Could you review this PR please? Thanks 👍
Note that it doesn't fix another bug which is referred as #819 (along with a fix), whose consequence is to not show the filter icon in second-sheet:

The fix of #819 will complete this fix of #547, and we'll get the right icon:

Done... on to the last one!


Python exited with an error

All goes well while installing, but right at the end when is should be launching i get hit with this error

The error indicates that the port RVC uses is already in use, which might mean you (or whatever this installer is) has already started an instance.
You can try to navigate to localhost:7865 and see if it returns anything. If it doesn't, restart your machine and open cmd and run python C:\Users\Tiny\Documents\VOCAL AI\ai\Retrieval-based-Voice-Conversion-WebUI\infer-web.py

I had the same problem. It turned out to be "NTKDaemon.exe" running in the background using this port (that's from "Native Instruments"). Just putting it out here in case someone can't figure out what's going on. I guess if you're interested in RVC Project you might also be into music and have some NI app(s) installed.
To solve it I just permanently replaced 7865 with 7866 in configs\config.py. That's it. Works like a charm.


Merge of SP2019 Public Preview changes into v3.0 branch

This PR merges all private changes into the public SharePointDsc repo. This will add support for the SharePoint 2019 Public Preview build.
@NikCharlebois To you the honors of reviewing this PR ;-)

[X] Change details added to Unreleased section of changelog.md?
[X] Added/updated documentation and descriptions in .schema.mof files where appropriate?
[X] Examples updated for both the single server and small farm templates in the examples folder?
[X] New/changed code adheres to Style Guidelines?
[X] Unit and Integration tests created/updated where possible?

This change is

AppVeyor not starting closing and reopening issue

Codecov Report

Merging #847 into v3.0-development will decrease coverage by <1%.
The diff coverage is 69%.

@@               Coverage Diff                @@
##           v3.0-development    #847   +/-   ##
================================================
- Coverage                92%     92%   -1%
================================================
  Files                   111     111
  Lines                 10875   10931   +56
  Branches                  1       1
================================================
+ Hits                  10012   10063   +51
- Misses                  862     867    +5
  Partials                  1       1

@ykuijs There are tests failing
https://ci.appveyor.com/project/PowerShell/sharepointdsc/build/<IP_ADDRESS>0?fullLog=true#L13168
https://ci.appveyor.com/project/PowerShell/sharepointdsc/build/<IP_ADDRESS>0?fullLog=true#L16654

@ykuijs and now they are passing. 😃 Ready for @NikCharlebois to review, if have time. 🙂

Yep, on it

@ykuijs all yours to review mate


[WIP] Fusing the kernels in stddev + mean.

The graph before the changes

After the changes:

No progress in 2 weeks


Treeview checkbox cant be wrapped

I am trying to customise my treeview but when I wrap a frame around the checkbox element the checkboxes stop working.
This works:
<material:TreeView ItemsSource="{Binding Nodes}" IsExpandedPropertyName="IsExtended" >
    <material:TreeView.ItemTemplate>
        <DataTemplate>
              <material:CheckBox Text="{Binding Name}" >
                  <material:CheckBox.Behaviors>
                      <material:TreeViewHierarchicalSelectBehavior />
                  </material:CheckBox.Behaviors>
              </material:CheckBox>
        </DataTemplate>
    </material:TreeView.ItemTemplate>
</material:TreeView>

but this doesn't:
<material:TreeView ItemsSource="{Binding Nodes}" IsExpandedPropertyName="IsExtended" >
    <material:TreeView.ItemTemplate>
        <DataTemplate>
            <Frame>
                <material:CheckBox Text="{Binding Name}" >
                    <material:CheckBox.Behaviors>
                        <material:TreeViewHierarchicalSelectBehavior />
                    </material:CheckBox.Behaviors>
                </material:CheckBox>
            </Frame>
        </DataTemplate>
    </material:TreeView.ItemTemplate>
</material:TreeView>

I am using the example from the docs.

It seems it searches in all children hierarchy in the code
https://github.com/enisn/UraniumUI/blob/da0b199f1024afd221474759960ed2aa24150491/src/UraniumUI.Material/Controls/TreeViewHierarchicalSelectBehavior.cs#L112C22-L112C45
But there can be some logical issues. I'll take a look at it asap. You can copy this behaviour to your project and customize as you wish until the solution is released

@enisn I have started working on a way to drag an drop within the treeview. This meant that I had to wrap the checkbox in lots of stuff making it quite unreadable as you can see hear:

Hi,
This method should make it already
https://github.com/enisn/UraniumUI/blob/da0b199f1024afd221474759960ed2aa24150491/src/UraniumUI/Extensions/ViewExtensions.cs#L30
It checks all the descendants to find a specific type in the hierarchy, maybe there can be a bug in this method, I need to take a look at it


TTNN Op Interface: Op Perf model

What
Provide a TTNN API to return Op perf estimate to the TT-MLIR optimizer. The goal of this issue is to do an end-to-end PoC and conclude what would be the proper solution. Based on the conclusion we will open a new issue to define the whole work.
PoC
Initially, on an example of MNIST NN and all Ops that are parts of MNIST, do the following tasks:

[ ] Implement Op perf API in TTNN (we should be in sync with core TTNN team),
[ ] Consume the API in TT-MLIR Optimizer.
[ ] Conclude on the Op perf API design in TTNN and define the issue to do the whole work on TTNN side.

The above tasks should be done Iteratively until it works well for both TT-MLIR Optimizer and TTNN.

We know it would be a lot of complex work to provide and maintain "Op perf" API with Ops being actively developed and optimized, hence we should also explore potential alternatives in terms of how Optimizer would make performance related decisions. This is to be discussed at least with @s-jovic, @pavlepopovic, @nobradovictt.

For this sprint, we are not implementing proposed POC.
#303 and #304 have been flagged as more urgent, and they carry similar challenges as #305:

API integration, how to avoid bloating op and building all details without compiler being too specific about the op variant and every single parameter.
Maintenance, how to avoid having maintenance hell

On the perf estimate itself, we do want to make a set of experiments to establish a method to measure/calculate perf of op (compute and data movement).

In the current sprint we will be working on the following tasks needed to deliver Perf model API:

add perf model API to tt-metal & infra to test the API against real execution runtime on the CI [owners @s-jovic @skrsticTT ]
model specific ops:

relu [owner @skrsticTT ]
add [owner @skrsticTT ]
softmax  [no owner yet]
reduce [owner @ipotkonjak]

come up with some sort of standardized approach for modeling new ops [owner @skrsticTT ]

Furthermore, we plan to add an API to run an op with specific attributes to get the real execution time instead of the estimate. [#1201 ] [owner @s-jovic ]


Unskip/uncomment tests after #1181

When implementing #1181 we had to skip or comment out several tests, for numerous reasons, to avoid slowing down and blocking the PR from getting merged. All of these have been marked as TODOs. These should be unskipped and added back again ASAP.

Closed via https://github.com/FuelLabs/fuels-ts/pull/1181


Re-implement virtual sites in GROMACS export

[x] Re-implement BondCharge virtual sites
[x] Re-implement planar DivalentLonePair virtual sites
[x] Collate virtual sites with their associated molecules in to_gro
[x] De-duplicate molecules in GROMACS export via Topology.identical_molecule_groups
[x] One-off sanity checks using an example of a ligand in TIP4P water

I almost have an example of a ligand in TIP4P water running. The main issue is that the hydrogens are not getting charges modified by the virtual site handler - though the virtual site particle does end up with the correct charge - so the simulation has an incorrect net charge (-1.02 e per water). This is not an issue in the OpenMM export, so the issue is contained to the GROMACS export. If I fix those lines manually (and constrain bonds) I can run a simulation.
import numpy
from openff.toolkit import ForceField, Molecule, Topology
from openff.toolkit.typing.engines.smirnoff.parameters import (
    LibraryChargeHandler,
    VirtualSiteHandler,
)
from openff.units import unit

from openff.interchange import Interchange

dummy = VirtualSiteHandler.VirtualSiteType(
    type="DivalentLonePair",
    name="EP",
    smirks="[#1:2]-[#8X2H2+0:1]-[#1:3]",
    distance=-0.15 * unit.angstrom,
    charge_increment1=0.0 * unit.elementary_charge,
    charge_increment2=0.52 * unit.elementary_charge,
    charge_increment3=0.52 * unit.elementary_charge,
    sigma=1.0 * unit.angstrom,
    epsilon=0.0 * unit.kilocalories_per_mole,
    outOfPlaneAngle=0 * unit.degree,
    match="once",
)

sage = ForceField("openff-2.0.0.offxml")
handler = VirtualSiteHandler(version=0.3)
handler.add_parameter(parameter=dummy)
sage.register_parameter_handler(handler)

sage["LibraryCharges"]._parameters = list()

sage["LibraryCharges"].add_parameter(
    parameter=LibraryChargeHandler.LibraryChargeType(
        name="TIP4P",
        smirks="[#1:1]-[#8X2H2+0:2]-[#1:3]",
        charge1="0.0 * elementary_charge",
        charge2="0.0 * elementary_charge",
        charge3="0.0 * elementary_charge",
    )
)

ligand = Molecule.from_smiles("CCOC(=O)C1=C(NC(=S)N[C@H]1C2=CC(=CC=C2)O)C")
ligand.generate_conformers(n_conformers=1)
ligand.conformers[0] -= numpy.min(ligand.conformers[0], axis=0)

water = Molecule.from_smiles("O")
water.generate_conformers(n_conformers=1)
water.conformers[0] -= numpy.min(water.conformers[0], axis=0)

topology = Topology.from_molecules([ligand])

for x in range(10):
    for y in range(10):
        for z in range(10):
            x_, y_, z_ = (1 + 0.5 * _ for _ in (x, y, z))
            displacement = unit.Quantity(
                numpy.array(3 * [x_, y_, z_]).reshape((3, 3)), unit.nanometer
            )
            water_ = Molecule(water)
            water_.conformers[0] = water_.conformers[0] + displacement
            topology.add_molecule(water_)

out = Interchange.from_smirnoff(sage, topology)

out.to_openmm(combine_nonbonded_forces=True)
print("writing gro ... ")
out.to_gro("foo.gro")
print("writing top ... ")
out.to_top("foo.top")

and
gmx grompp -f em.mdp -c foo.gro -p foo.top -o em.tpr -maxwarn 1002
GMX_MAXCONSTRWARN=-1 gmx mdrun -v -deffnm em
gmx grompp -f default.mdp -c em.gro -p foo.top -o out.tpr -maxwarn 1002
GMX_MAXCONSTRWARN=-1 gmx mdrun -v -deffnm out
echo 0 | gmx trjconv -f out.gro -o unwrapped.gro -pbc nojump -s out.tpr
echo 0 | gmx trjconv -f out.xtc -o unwrapped.xtc -pbc nojump -s out.tpr
pymol unwrapped.gro unwrapped.xtc

em.mdp.txt
default.mdp.txt
The same should be picked up for bond charge virtual sites but I haven't tested that in a multi-molecule simulation.
I have yet to work on mono- or trivalent lone pair virtual sites or divalent lone pair virtual sites with non-zero outOfPlane angle. Those shouldn't be too much work

Haven't looked at forces or anything, but a nitrogen lone pair on pyridine at least looks sensible enough:


Allow to favour feeds

In NNW4 there was a “Favorites” section, which came in very handy when one subscribed to a large number of feeds, but read only some of them on a daily basis.

+1


Parallel scenario execution

I'm looking into the possibility of running browser-based scenarios concurrently on cloud machines, for fast cross-browser testing.
So I would like to execute scenarios in parallel, but I would also like to execute each scenario multiple times (i.e. run the same scenario against lots of different devices concurrently). Ideally, the whole suite would execute in just over the time of the slowest scenario on the slowest device.
Historically there was some support for this in parallel_test, but this is broken in cucumber 2.0.
My first questions are:

Does the new cucumber API support asynchronous and/or concurrent scenario execution, or is it designed to execute scenarios in serial?
Do I need to fork a process for each scenario?
Applying this to a local web app (let's assume rails) how do I isolate the application state for each scenario run?

If anybody can point me in the right direction with respect to the current cucumber API, I'll give it a crack and see what happens.

Regarding the API, things are still in flux: we've cleaned up the internals a great deal and put many of then into https://github.com/cucumber/cucumber-ruby-core
We're trying to keep this as the 'inner hexagon' that can compile and run test cases, with the cucumber gem then being a client of that.
So you might want to start there and build something from scratch. If you look at the README you'll see an example that lets you build one in only a few lines.
The key abstraction you'll need to leverage is the Filter API. This isn't well documented, but you can think of it a bit like Rack middleware for a cucumber suite. The filters are arranged into a chain with the Gherkin compiler at the input. As test cases are compiled from the gherkin, they're passed into the chain. Filters can choose to either pass the test case through, pass a modified copy, or not pass it on.
I guess you'll need to implement something that supports the Filter protocol but does the distribution of work to various runners. You'll then need something that supports the Report API at the other end to collate the results.
Have a spelunk through the code in the core and see if that makes any sense. If you want to book some pairing time with me to go through this in more depth (assuming the code you're working on is OSS) just shout.

Thanks for your help @mattwynne!
I had a little play with it, got something working with core filters and the parallel gem and ended up with this slightly monstrous hack: https://github.com/featurist/cucumber-parallel
In this example I run cucumber (scenarios * contexts) + 2 times in total, in a single process that forks up to 50 others:

once as a dry run to capture scenario locations
once per scenario per context (a context might be a particular browser, for example)
once at the end with a filter that "replays" the results of the many scenario runs, so that normal cucumber formatters can be used.

My thinking was that the ideal parallel cucumber runner would behave just like cucumber, but faster. So I need to reuse the bits of cucumber-ruby that interpret CLI options, find gherkin files and so on. Without pulling the code apart that means building an alternative Cucumber::Cli::Main that doesn't try to Process.exit all the time.
Because I'm running cucumber so many times, I'm assuming this will repeat various bits of work like reading gherkin files and won't be as fast as if I did it with more filters and fewer passes.
I also think a separate "parallel formatter" API might make more sense, because the above implementation doesn't give feedback until all scenarios have been executed.
Anyway, I just wanted to post progress here to see if anybody had any more feedback about running parallel scenarios.
Does it make sense to have a cucumber_parallel binary in a separate gem, with a similar UI to cucumber? or is there an alternative way of packaging it?
If you run the same scenarios in multiple contexts, what kind of report would you like to see? A normal cucumber report, with aggregated errors, or something else?

@joshski When you talk about "scenario per context", it seems like you are basically talking about "suites" as they are defined in #821. So here we talk about two things, the ability to run the same scenario several times with different context, and the ability to run scenarios in parallel. In the longer run I think it would rather be "pickles" as they are defined in gherkin3 that should be distributed over different contexts and runners. Cucumber-Ruby already have a compile step like the one described in gherkin3 (even though "pickle" is not a term used in Cucumber-Ruby). But there will probably take some time to get there.

I would like it best if we can make the distribution of test cases to runners something that you can hook into and control from cucumber itself, so that you can do it using processes or even across machines or whatever, but all using the regular cucumber UI.
so rather than this being a binary, it might be a plug-in extension.
So I say carry on hacking on it outside for now, but try to help me understand where we need to put in those extension points for you so that it could be a plug-in in the end.
Make sense? I wish I had more time to focus on this Josh but not right now!

I looked into this a little bit and it looks like there are at least two places parallel execution could be started from within cucumber-ruby-core.
For features one possible location where the parallel threads could start is in Cucumber::Core's compile method.
For test cases, Cucumber::Core::Test::LocationsFilter seems to be the right spot to start parallel execution (I've tested this out a little bit and it does seem to work).
My idea is to check the configuration for the '--parallel' flag (probably feature by default and '--parallel scenario' or something of the sort to toggle) and have a parallel runner object take care of the work. This could at some point also take a node location for test distribution -- but I don't want to think too much about that at the moment.
Let me know what you think, if I'm on the right path or maybe should think of more options.

Note that there are some changes coming in the way this API works.
I think I would approach this by replacing the Core::Test::Runner as a DistributedRunner or some such that takes test cases and distributes them across the nodes, collates the results and then fires out events. The slave nodes could use a real Core::Test::Runner.
The challenge will be in serializing test cases that include their step definitions, and re-hydrating those on the slave. They do contain a location for the source of the step definition block, so this is possible, but that's probably the hardest part of this. That and defining the protocol between the master and the slaves.
Does that make any sense?

On second thoughts there's an alternative where the ActivateSteps, AddBeforeHooks etc filters are run on the slaves. That would make the test cases easier to serialise / deserialise.

I agree, conceptually the slaves would be sent a pickle. Event though Cucumber-Ruby, in terms of Ruby classes, does not make any difference between pickles and test cases, they are useful when thinking about the design.

So, if we want to go the Filter route, I think it would be possible to add a PickleDistributor / ParallelFilter somewhere near the Quit filter. It might also be nice to use a config file to set the number of processors to use, location of distributed machines (if distributing as well as running in parallel) etc.
With filters the parallel part seems pretty straight forward to me. I'm just not sure about how to go about distribution -- is this something we would use DRb for? Or are there other/better ways to achieve this that you know of?

it feels like this deserves to be in a repo of its own, built as a plugin. I wonder what extension points we'd need to add so that cucumber was flexible enough for you to be able to plug this in?
I think it would be worth building a spike in another repo (or a gist) to explore this some more. It could be a variation of the example in the readme for the core. WDYT?

I have created a cucumber-threads repo to explore this more. Currently there is a working example using a Queue and some worker Threads.

t-morgan informed me of this thread, as I have been working on a solution for some time.  I thought I would give my insights into some of my findings.
SUMMARY
There are too many global variables to the cucumber_world to make this possible right now (including results, cucumber_runtime instance, etc).  In order to do parallel testing, multiple ruby instances must be created unless those items global to the cucumber_world are abstracted out.
That being said, if we were to abstract all those parts out, we can already execute the test code as part of the cucumber pickle . (This was tested by using a before hook)
Before do |scenario|
  scenario.instance_variable_get(:@test_case).instance_variable_get(:@test_steps)[0].instance_variable_get(:@action).instance_variable_get(:@block).call
end

this could be called in a .each loop on the test_steps to execute all steps within the pickle.  This is possible due to the block being an executable proc
CURRENT SOLUTION:
My current solution is a client/server configuration, that utilizes a sinatra server that feeds test locations to runners.
server:
pass in feature directory through command line, and parse for tests by feature or scenario (by using Dir.glob or Gherkin::Parser on a single file)
Then I start a sinatra server with a single endpoint to feed the tests to the runners
runners:
I mixin a run_without_setup method on the runtime
 def run_without_setup
    self.visitor = report
    @features = nil
    @filespecs = nil
    receiver = Cucumber::Core::Test::Runner.new(report)
    compile features, receiver, filters
  end

Then I manually start a Cucumber::Runtime, and load the language
I then get a test from the sinatra server and run it using cucumber CLI.  For each test after that, i change the path in the runtime options, and run the run_without_setup command on the runtime.
Issues overcome:

Distributed parallel exeuction: All runners are connected to the sinatra server feeding it what test to run.  Each client only asks for 1 test at a time (following the FAIR distribution pattern)

Multiple runtimes:  You cannot have multiple Cucumber::Runtime environments in one ruby process.  This means each individual runner must start it's own process (done by calling the CLI)

Each machine must have access to the code, and an environment to run the code in.  We solved this by having a startup script that provides us what we need, with a future state of having docker containers with rvm already configured, and a shared data container with the must up-to-date code.

Multiple runners per machine: Since each runner is calling CLI, there can be as many running tests on one machine as it can handle.  There more is overhead then ideal, as each runner needs its own ruby env.

There is no communication back to the server to report the test case:  We use a formatter to collect all the data from the different runners and store it in a centralized location.

Drawbacks:

No error handling: Test distribution is fire and forget.  If a runner crashes before the test is complete, it doesn't get counted.  Have though about using a framework like rabbitmq and using manual ack to signal when a test is complete (in place of the sinatra server), but I haven't experimented with that yet.

Wow what great exploration @westlakem!
It seems to me that we need to do more work to make the code in cucumber-ruby more modular before this is going to be easy. The abstractions like the Runner and Filters from the core will make this easier, but as you've seen, there's still some pretty gnarly code in and around the runtime.


Partials and templates with expressions are not CSP compliant in root instance

Description:
Pre-parsed partials and templates  cannot be used on environments that enforce a strict Content Security Policy (CSP).
It turns out that only expressions in partials/templates used in components are successfully pre-converted to functions. In the root ractive instance the the pre-parsed expression functions are ignored.
See #3285 for a similar issue
Versions affected:
1.3.6
Platforms affected:
All pages that enforce a strict CSP policy
Reproduction:
Using a pre-parsed templates/partials with expression functions and csp: true and allowExpressions: false options. See JSFiddle Testcase
The fiddle uses pre-compiled templates/partials (with expressions functions) generated via
const serialize = require('serialize-javascript')
const Ractive = require('ractive')
const output = serialize(Ractive.parse(source), {space: 2})

from the following templates:
template:
ROOT:

        template value:{{test()}}

    {{>partial1}}

COMPONENT:

    <test></test>

template2:

    template value:{{test()}}

{{>partial2}}

partial:

partial value:{{test()}}

Testcode:
var fixture = document.getElementById( 'qunit-fixture' );

test( 'CSP tests', function ( t ) {
	var partial = {
  "v": 4,
  "t": [
    {
      "t": 7,
      "e": "div",
      "m": [
        {
          "t": 13,
          "n": "class",
          "f": "partial",
          "g": 1
        }
      ],
      "f": [
        "partial value:",
        {
          "t": 7,
          "e": "span",
          "m": [
            {
              "t": 13,
              "n": "class",
              "f": "testvalue",
              "g": 1
            }
          ],
          "f": [
            {
              "t": 2,
              "x": {
                "r": [
                  "test"
                ],
                "s": "_0()"
              }
            }
          ]
        }
      ]
    }
  ],
  "e": {
    "_0()": function (_0){return(_0());}
  }
}
	var template = {
  "v": 4,
  "t": [
    "ROOT: ",
    {
      "t": 7,
      "e": "div",
      "m": [
        {
          "n": "id",
          "f": "root",
          "t": 13,
          "g": 1
        }
      ],
      "f": [
        {
          "t": 7,
          "e": "div",
          "m": [
            {
              "t": 13,
              "n": "class",
              "f": "template",
              "g": 1
            }
          ],
          "f": [
            "template value:",
            {
              "t": 7,
              "e": "span",
              "m": [
                {
                  "t": 13,
                  "n": "class",
                  "f": "testvalue",
                  "g": 1
                }
              ],
              "f": [
                {
                  "t": 2,
                  "x": {
                    "r": [
                      "test"
                    ],
                    "s": "_0()"
                  }
                }
              ]
            }
          ]
        },
        " ",
        {
          "t": 8,
          "r": "partial1"
        }
      ]
    },
    " COMPONENT: ",
    {
      "t": 7,
      "e": "div",
      "m": [
        {
          "n": "id",
          "f": "component",
          "t": 13,
          "g": 1
        }
      ],
      "f": [
        {
          "t": 7,
          "e": "test"
        }
      ]
    }
  ],
  "e": {
    "_0()": function (_0){return(_0());}
  }
}
	var template2 = {
  "v": 4,
  "t": [
    {
      "t": 7,
      "e": "div",
      "m": [
        {
          "t": 13,
          "n": "class",
          "f": "template",
          "g": 1
        }
      ],
      "f": [
        "template value:",
        {
          "t": 7,
          "e": "span",
          "m": [
            {
              "t": 13,
              "n": "class",
              "f": "testvalue",
              "g": 1
            }
          ],
          "f": [
            {
              "t": 2,
              "x": {
                "r": [
                  "test"
                ],
                "s": "_0()"
              }
            }
          ]
        }
      ]
    },
    " ",
    {
      "t": 8,
      "r": "partial2"
    }
  ],
  "e": {
    "_0()": function (_0){return(_0());}
  }
}

  var component = Ractive.extend({
    isolated: false,
    template: template2,
    partials: {
      partial2: partial
    }
  })

  Ractive({
    el: fixture,
    csp: true,
    allowExpressions: false,
    template: template,
    components: {
      test: component
    },
    partials: {
      partial1: partial
    },
    data: {
      test: function () {
        return 'testValue'
      }
    }
  })

  t.equal( fixture.querySelector('#component .template .testvalue').innerHTML, 'testValue' )
  t.equal( fixture.querySelector('#component .partial .testvalue').innerHTML, 'testValue' )
  t.equal( fixture.querySelector('#root .template .testvalue').innerHTML, 'testValue' )
  t.equal( fixture.querySelector('#root .partial .testvalue').innerHTML, 'testValue' )
})

I think there are two separate issues here: a bug and a docs bug.
allowExpressions is meant to completely disallow evaluating any expressions, including pre-parsed expressions in a template, in case that you don't trust your templates at all. The docs don't make that particularly clear.
The bug is that partials loaded from a template object don't currently preload their expression payload to make them available without re-parsing.
I have a fix for the bug ready, and I'll get the docs bug in a bit.

The fix is now published on edge, and will be cherry-picked over to 1.3 and 1.2 branches when the next releases are cut.


Articels: Configure Endpoint for external Kafka and external PG

From Console - left menu "Service Integrations", we direct users to different help articles. For some there are no articles or I could at least not find them on how to configure the endpoint. The following two articles seem to be missing.

How to send your Kafka logs to an external Kafka cluster
External Kafka Integration: there is no directs explanation how to setup/configure the endpoint for an external kafka integration. With the aim of then sending Kafka Logs.

Integrate your Aiven services with an external PostgreSQL database
There is not article on how to setup/configure the endpoint for an external PostgreSQL integration.

_note: For Google Logging there is an article, the link to it is just missing in console. A Jira ticket exists to update eh wording on the Service integration page and include the missing links. Please let me know if there are suitable articles to link to or once the articles have been written, so I can request the links in console to be updated/added.
Irene

articles have been written on this


tsc --build tsconfig.json fails

I tried to compile using tsc because the project runs awfully slow on a Raspberry Pi Zero W since the TypeScript merge but it fails with the following error:
pi@raspberrypi:~/HAP-NodeJS $  tsc --build tsconfig.json
src/lib/gen/importAsClasses.ts:182:9 - error TS7053: Element implicitly has an 'any' type because expression of type 'string' can't be used to index type 'typeof Formats'.
  No index signature with a parameter of type 'string' was found on type 'typeof Formats'.

182     if (Formats[key] == format)
            ~~~~~~~~~~~~

src/lib/gen/importAsClasses.ts:191:9 - error TS7053: Element implicitly has an 'any' type because expression of type 'string' can't be used to index type 'typeof Units'.
  No index signature with a parameter of type 'string' was found on type 'typeof Units'.

191     if (Units[key] == units)
            ~~~~~~~~~~

Found 2 errors.

Well it looks like you have some rules in tsconfig.json that cause an error and maybe it is not important for the whole project to work properly?
Can you share what is inside of yours tsconfig.json?

@Shaquu it's the same tsconfig.json one as in this project, running it with ts-node is no problem, it just takes ages to start and parse accessories.
I wanted to compile and try it that way but encountered that error.

The problem is that the both Formats and Units are enums with string values now, and TypeScript complains about accessing the values directly with string keys.
You can change the offending lines to
if (Formats[key as keyof typeof Formats] == format)
and
if (Units[key as keyof typeof Units] == units)
to fix this.
I'm unsure if someone should make a pull request for a fix like this: The offendign file is in the /gen subdirectory, which sounds like it's being generated from something, but I can't find the the script generating importAsClasses.ts. It would make more sense to fix it there, rather than in the generated code.


Character sheets broken (Foundry version 10 support)

I recently updated my Foundry to Version 10 (Build 284) and so far have noticed at least one major bug with pf2e (v. 3.13.6). None of the character sheets load properly (PC or NPC sheets, see attached screenshot). I had found a reddit post about this exact same issue from a year ago and it seems the solution at the time was they had updated Foundry but were not on a pf2e version that supported that Foundry version. Considering Foundry Version 10 is new and I'm on the latest version of pf2e I wanted to make sure you are aware of this issue. I see no errors in my logs when opening the character sheets and I can tell all of the information is there, they just don't render properly.

There's not a system version compatible with Foundry V10 yet. The warning you clicked through would have made that clear.


Support for 3.6.9?

Hello again, I am trying to install this on google colab with poetry but it fails as the project requires 3.8^ . Is there any version difference that makes the package not run on <3.8?

No, you can modify the version and run.

Great, thanks!


Enhencement add property for outfile/export and add test

Proposed changes
Issue Number: close #xxx
Problem summary
This pr does three things:

add delete_existing_files property for outfile/export. If delete_existing_files = true, export/outfile will delete all files under file_path first.
add p2 test for export
modify docs

TODO:

modfiy english docs

Checklist(Required)

[ ] Does it affect the original behavior
[ ] Has unit tests been added
[ ] Has document been added or modified
[ ] Does it need to update dependencies
[ ] Is this PR support rollback (If NO, please explain WHY)

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 34.25 seconds
stream load tsv:          423 seconds loaded<PHONE_NUMBER>9 Bytes, about 168 MB/s
stream load json:         23 seconds loaded<PHONE_NUMBER> Bytes, about 97 MB/s
stream load orc:          58 seconds loaded<PHONE_NUMBER> Bytes, about 18 MB/s
stream load parquet:          30 seconds loaded 861443392 Bytes, about 27 MB/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230426044616_clickbench_pr_135394.html

run buildall

We should add a FE config enable_export_delete_existing_files. And the default value is false.
Because this is a very dangerous operation, that user may specified export dir /, and it will delete the whole dir!
So:

add this config to disable it by default.
add comment and document to notice that it is dangerous config that should only be used in test env.

run buildall


Rework logging dashboards

How to categorize this PR?

/area logging
/kind enhancement
/priority normal
What this PR does / why we need it:
New logging dashboard is added prefixed with Logging .... for each exposed component.
Logging panels from Kubernetes Control Plane Status are removed
Which issue(s) this PR fixes:
Fixes # https://github.com/gardener/logging/issues/72
Special notes for your reviewer:
@wyb1 @vlvasilev @rfranzke
@g-pavlov Can you please take a look on the documentation changes?
Release note:

New logging dashboards are added, prefixed with `Logging` for each exposed component.
Logging panels in `Kubernetes Controlplane Status` are removed

I didn't see any logs in the dashboards, but I also just created a shoot to test this. If this is expected then /lgtm

If the Grafana is opened via the Ingress, it should contains logs.
port-forwarding does not working because Loki expects header with orgID

/ping @rfranzke