Added interegular support

This is basically the same as #715 , with (I think) all requested changes being considered. I also added an Example to the log output that can show in what situation two terminals would collide.
I created a new PR after having a mess after a git rebase. This just seemed easier.
I guess it should be mentioned somewhere in the docs, not sure where.

So many checks, many of which are hard to execute locally tbh:

tox -e mypy works, although tox needs to be installed first (well, as long as the dependencies are added in the correct places)
tox -e test doesn't work, fails with some mysterious git submodule error (didn't look into this) Luckily, one of the 5 other ways to execute tests does work.
tox -e format doesn't work with the same error message. I don't know how to manually execute this, so I have to just push and see what github says.

Sorry, I didn't look into the codecov yet. I'm okay with just disabling it in the meantime.
As for tox, makes sense that mypy would pass but regular tests won't. Mypy doesn't run the tests.

The code looks good, but I'm not crazy about the warning message -

Collision between Terminals 'B' and 'A': Example(prefix='', main_text='', postfix='a')

I'm sure we can do better.

Doesn't seem like prefix and main_text are ever used? At least, I couldn't find an example where they have a value.

The warning should explain the consequences of the collision. e.g. "The lexer will choose between them arbitrarily."

And ofc better not to use the repr version of Example.

Doesn't seem like prefix and main_text are ever used? At least, I couldn't find an example where they have a value.

Uhm... This is then a bug in interegular. OTOH, I could have sworn the eample for A and B put the text into main_text where it belongs. (Edit: aha yeah, I know what the bug is: -0 == 0)
The idea behind the three categories is that main_text is what actually get's captured by the regex and prefix and postfix are what is needed by lookbehind/lookahead. For example, the lark grammar without the change would produce a conflict on Example(prefix='', main_text='?', postfix='_')

The warning should explain the consequences of the collision. e.g. "The lexer will choose between them arbitrarily."

Fair, although I worry that a grammar would many collision would product a lot of warning text.

And ofc better not to use the repr version of Example.

I am unsure how to best format this. What prefix and postfix mean is kinda hard to explain in a short text. I guess I can do it multiline and use ^ to underline the main part, similar to python syntax errors.

Ok, I am now relatively happy with the warning message. It's decently complex code, but the result is IMO very readable. The formatting should now also never break, not even for newlines or weird unicode characters. Especially newlines is the reason I was using repr before.

I can add them here. Where would you want them?

Hmm yeah there's no obvious spot. I'd say -

List of features
Maybe mention it here? https://lark-parser.readthedocs.io/en/latest/how_to_use.html#work-process

I'm open to ideas.
Anyway, once that's in, I wanted to add a strict flag, so Lark(..., strict=True) will throw an exception when there are collisions. (regex collision and also shift-reduce). So then users can also learn about it through the API docs.

how_to_use#debug
is certainly a place it should belong, I wasn't even really aware that that section exists.
I assume strict is a separate PR?

Btw, the default config parser="earley", lexer="dynamic" doesn't benefit from this at all, right? How bad are regex collisions in that context?

I assume strict is a separate PR?

I think so.

the default config parser="earley", lexer="dynamic" doesn't benefit from this at all, right?

Well, colliding regexes are still slower, because Earley has to try all variations. But it won't cause an error. So maybe a milder warning in that case? (assuming it doesn't complicate things too much)

So maybe a milder warning in that case? (assuming it doesn't complicate things too much)

I would have to look through the earley code base to find a good place to put this check. They don't construct a Lexer object after all.

I rephrased the docs a little bit, let me know if you're okay with the changes: https://github.com/lark-parser/lark/pull/1260
Also, I just realized we're only showing shift/reduce warnings when debug=True. Maybe we should use the same convention for interegular?

Yeah, those changes are good. I am not that skilled with words :-)

Maybe we should use the same convention for interegular?

I was thinking about this as well. I decided against it, since regex collisions are from my understanding more severe than shift/reduce collisions. I.e. for shift/reduce the automatic resolving probably will do the correct thing, for regex collision, the automatic resolving will maybe do the correct thing.

I think I agree. But I was also thinking about the possibility of interegular throwing exceptions, which might interfere with normal use. For example certain regex features it doesn't know about, or perhaps some edge case it wasn't expecting. If that's a possibility, then it would be nice to have a way to disable/enable it, that isn't installing/uninstalling it.

interegular should catch every exception it can throw and only produce a warning on it's internal logger. We probably should make the debug flag make these warnings visible. I don't believe that performance could ever be bogged down enough by these checks to be worth having a disable flag for that. If the performance is that important, create a standalone parser.

Can you please show me where this exception gets caught? https://github.com/MegaIng/interegular/blob/master/interegular/fsm.py#L431
Maybe I'm missing something obvious.

https://github.com/MegaIng/interegular/blob/ffde88d87a1eb4209fc3bdad2153643b4944c064/interegular/fsm.py#L889
It's used internally inside of .fsm to correctly crawl the (potentially incomplete) FSM.

Ah, yes. Missed that it's a callback.

I also published a new version of interegular that should double the performance again. :-D


KrakenD endpoints validating

I was faced with the problem of insufficient functionality of the cat build a chain of queries.
What I want:
When accepting a request for a specific endpoint, the service must take information from this request and make a new one in order to check (validate) the received request. After successful verification, the initial request continues.
Example:
[1]Client - - - > [2]<krakend> / api/service/project1/secret-info - - - > [3]<server>/project1/secret-info

1.    Send JWT in auth header (with userID)
2.1. Get userID from JWT & projectID from URL
2.2. Send UID & GID to /api/check-users-project
2.3. Check response (if access denied - abort, if access allowed - continue)
3.    Accepts a normal request

What I did:
1. Tried using a Sequential Proxy, but it's not exactly what I need. This scheme cannot terminate requests if verification fails.
2. Tried to use a Checking requests and responses with CEL in the request chains. The check works, but it still fails to terminate requests.
3. Trying to use Lua scripts. Everything was fine, but I ran into the current problem.

Main question:
I want to use Lua scripts to send https requests using "socket" and "luasec". This would solve my problem, but I don't see a way to use .so files for Lua inside Krakend.
Simple config example:
{
    "endpoint": "/api/secret",
    "method": "GET",
    "extra_config": {
        "github.com/devopsfaith/krakend-lua/request": {
            "sources": [
                "/etc/krakend/lua/validateRequest.lua",        <--- (1)
                "/etc/krakend/lua/socket.lua",                       <--- (2)
                "/etc/krakend/lua/https.lua",                          <--- (3)
                "/etc/krakend/lua/socket/core.so"                 <--- (4)
            ],
            "pre": "validateRequest(ctx.load())",
            "live": true,
            "allow_open_libs": true
        }
    },
    "backend": [
        {
            "extra_config": {
                "github.com/devopsfaith/krakend-cel": [
                    {
                        "check_expr": "'True' in req_headers['Krakend-Access']"
                    }
                ]
            },
            "host": [
                "http://<ip:port>"
            ],
            "url_pattern": "/secret",
            "encoding": "json"
        }
    ]
}

(1) validateRequest.lua:
local socket = require "socket"   <--- OK
...

(2) socket.lua:
local socket = require "https"   <--- OK
...

(3) https.lua:
local socket = require "socket/core"   <--- this is core.so and KrakenD can't load or find this module inside
...

I will be grateful for advice on an alternative solution or how to fix the current algorithm. For everything in general. Thanks

duplicated with https://github.com/devopsfaith/krakend-lua/issues/10
also, you can go with a handler plugin: https://www.krakend.io/docs/extending/writing-plugins/#writing-and-injecting-plugins


Update invocation command to case friendly structure and add a force …

…rescan option
This also will address OS/X workaround of needing to unload/reload plugin.

obsolete


Using hash instead of tuple

New qiskit is deprecating getting the register name from a qubit.  We therefore need a better way of uniquely identifying qubits and their (register, index) combination.   Qiskit Bit's have their own hash defined on the tuple of register name and index, and so will uniquely define each bit.
Though these bits will be harder to debug and read, the hash will be sufficiently unique per qubit to allow matching up qiskit qubits and PL wires.

I have no idea why one completely unrelated test would fail.  Seems quite strange.

And now it's passing? Maybe a bad test?


CAPT-1702 Add slack notification on failure

Context
For permanent environments we want to notify on Slack when a deploy fails.
Changes in this PR
Add an extra step to the deploy_test job to notify on slack if is fails.

Tested with this commit :https://github.com/DFE-Digital/claim-additional-payments-for-teaching/pull/2864/commits/cc4279f6f0dbd4250847880fc254419f368d2166
Posted to Slack 👍


Update

AppImage generation - do not merge yet

Getting
    ERROR in ./node_modules/r2-streamer-js/dist/es6-es2015/src/http/server.js
    Module not found: Error: Can't resolve 'css2json' in '/home/travis/build/edrlab/readium-desktop/node_modules/r2-streamer-js/dist/es6-es2015/src/http'
     @ ./node_modules/r2-streamer-js/dist/es6-es2015/src/http/server.js 10:17-36
     @ ./src/main/streamer.ts
     @ ./src/main/di.ts
     @ ./src/main.ts

What am I doing wrong?
Reference:
https://travis-ci.org/edrlab/readium-desktop/builds/322123618#L2710-L2715

This PR is now very obsolete due to changes in the Electron build settings (package.json). We can add "category": "Utility" in a separate PR.
I am going to look into the TravisCI Linux dist test.


Can't pickle <class 'celery.utils.log.ProcessAwareLogger'>

Hi,
My remote worker seems to fail after a while with this message
ERROR - (pickle.PicklingError) Can't pickle <class 'celery.utils.log.ProcessAwareLogger'>: it's not found as celery.utils.log.ProcessAwareLogger
It seems that it is trying to picke the logger but i have no clue why.
I have a two server test setup where one server runs airflow webserver and scheduler
And the other server is running the worker.
The dags are only located on server the webserver/scheduler and with pickly they should go to the worker.
And i use Celery as the executor.
Now when i added the tutorial (as of the offical docs) to the dags folder, then everything seems to work fine (aka i see about 24  successfull tasks)
But after a little while nothing seem to happen anymore.
If i check Flower then it tells me thats tasks fail with this message:
AirflowException: Celery command failed
And if i then check the workers log i get the Cant picke the logger error.
Hopefully anyone knows what its going wrong
Cheers

We also faced the issue. The error is on the stderr of the scheduler
ERROR:airflow.jobs.SchedulerJob:This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (pickle.PicklingError) Can't pickle <class 'celery.utils.log.ProcessAwareLogger'>: it's not found as celery.utils.log.ProcessAwareLogger [SQL: u'INSERT INTO dag_pickle (pickle, created_dttm, pickle_hash) VALUES (%(pickle)s, now(), %(pickle_hash)s) RETURNING dag_pickle.id'] [parameters: [{'pickle_hash': -1909267389068563253, 'pickle': <DAG: check_users>}]]
Traceback (most recent call last):
  File "/home/ivanov/src/airflow/airflow/jobs.py", line 624, in _execute
    dag = dagbag.get_dag(dag.dag_id)
  File "/home/ivanov/src/airflow/airflow/models.py", line 155, in get_dag
    orm_dag = DagModel.get_current(dag_id)
  File "/home/ivanov/src/airflow/airflow/models.py", line 1901, in get_current
    obj = session.query(cls).filter(cls.dag_id == dag_id).first()
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2634, in first
    ret = list(self[0:1])
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2457, in __getitem__
    return list(res)
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2736, in __iter__
    return self._execute_and_instances(context)
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2749, in _execute_and_instances
    close_with_result=True)
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2740, in _connection_from_session
    **kw)
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 905, in connection
    execution_options=execution_options)испортиться
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 910, in _connection_for_bind
    engine, execution_options)
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 313, in _connection_for_bind
    self._assert_active()
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 214, in _assert_active
    % self._rollback_exception
InvalidRequestError: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (pickle.PicklingError) Can't pickle <class 'celery.utils.log.ProcessAwareLogger'>: it's not found as celery.utils.log.ProcessAwareLogger [SQL: u'INSERT INTO dag_pickle (pickle, created_dttm, pickle_hash) VALUES (%(pickle)s, now(), %(pickle_hash)s) RETURNING dag_pickle.id'] [parameters: [{'pickle_hash': -1909267389068563253, 'pickle': <DAG: check_users>}]]

ProcessAwareLogger defined  in celery.utils.log inside function and can not be created during unpickling. Airflow does not handle this error correctly (rollback should be explicitly called) and scheduler db session becomes unusable.
Disabling pickle just works

can you please create a jira for this?


Can't connect to MS SQL Server using Kerberos

Description
Can't connect to MS SQL Server using Kerberos after upgrading to 22.3.4
Integrated authentication failed. ClientConnectionId:4dc878ab-f699-4e09-9c64-8beb561f9f9e
GSSException: No valid credentials provided (Mechanism level: KDC has no support for encryption type (14))
No valid credentials provided (Mechanism level: KDC has no support for encryption type (14))
KrbException: KDC has no support for encryption type (14)
KDC has no support for encryption type (14)
Identifier doesn't match expected value (906)
Identifier doesn't match expected value (906)
After reverting to 22.3.3 - problem disappear
DBeaver Version
Community Edition 22.3.4
Operating System
Ubuntu 20.04
Database and driver
Microsoft JDBC Driver for SQL Server 9.2.0.jre8
Steps to reproduce
Try to use Kerberos authentication for MS SQL Server.
Additional context
Kerberos ticket acquired using:
/bin/cat ~/.config/pw | /bin/kinit -F -p -r 7d username
and periodically refreshed using:
/bin/kinit -R
klist -V shown "Kerberos 5 version 1.17"

Hello @vpetryaev
Could you please provide the log files? You can find instructions here: https://dbeaver.com/docs/wiki/Log-files/. Thank you.
Also, can you provide steps about Kerberos authentication in DBeaver in your case? How did you use it?

Driver properties (not default only):
authenticationScheme: javaKerberos
integratedSecurity: true
serverSpn<EMAIL_ADDRESS>

klist output:
Ticket cache: FILE:/tmp/krb5cc_1001
Default principal<EMAIL_ADDRESS>Valid starting       Expires              Service principal
07.02.2023 11:07:11  07.02.2023 21:07:11<EMAIL_ADDRESS>renew until 14.02.2023 11:07:10

Please check your connection with the newest SQL Server driver versions.

Some changed, but still can connect using Kerberos authentication in 22.3.3 and found error in 22.3.4
Looks like it can't obtain user name from Kerberos cache.
!ENTRY org.jkiss.dbeaver.model 4 0 2023-02-09 13:12:48.076
!MESSAGE Cannot get any of properties: [user, USER] from con properties not available to garner  authentication information  from the user
!SUBENTRY 1 org.jkiss.dbeaver.model 4 0 2023-02-09 13:12:48.076
!MESSAGE Cannot get any of properties: [user, USER] from con properties not available to garner  authentication information  from the user
!STACK 0
javax.security.auth.login.LoginException: Cannot get any of properties: [user, USER] from con properties not available to garner  authentication information  from the user
at jdk.security.auth/com.sun.security.auth.module.Krb5LoginModule.promptForName(Unknown Source)
at jdk.security.auth/com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Unknown Source)
at jdk.security.auth/com.sun.security.auth.module.Krb5LoginModule.login(Unknown Source)
at java.base/javax.security.auth.login.LoginContext.invoke(Unknown Source)
at java.base/javax.security.auth.login.LoginContext$4.run(Unknown Source)
at java.base/javax.security.auth.login.LoginContext$4.run(Unknown Source)
at java.base/java.security.AccessController.doPrivileged(Unknown Source)
at java.base/javax.security.auth.login.LoginContext.invokePriv(Unknown Source)
at java.base/javax.security.auth.login.LoginContext.login(Unknown Source)
at com.microsoft.sqlserver.jdbc.KerbAuthentication.intAuthInit(KerbAuthentication.java:77)
at com.microsoft.sqlserver.jdbc.KerbAuthentication.generateClientContext(KerbAuthentication.java:207)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.sendLogon(SQLServerConnection.java:6030)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConnection.java:4911)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.access$100(SQLServerConnection.java:90)
at com.microsoft.sqlserver.jdbc.SQLServerConnection$LogonCommand.doExecute(SQLServerConnection.java:4849)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7620)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:3916)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:3358)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:2950)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:2790)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1663)
at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1064)
at org.jkiss.dbeaver.model.impl.jdbc.JDBCDataSource.lambda$0(JDBCDataSource.java:206)
at org.jkiss.dbeaver.model.impl.jdbc.JDBCDataSource.lambda$1(JDBCDataSource.java:225)
at org.jkiss.dbeaver.utils.SecurityManagerUtils.wrapDriverActions(SecurityManagerUtils.java:96)
at org.jkiss.dbeaver.model.impl.jdbc.JDBCDataSource.openConnection(JDBCDataSource.java:223)
at org.jkiss.dbeaver.model.impl.jdbc.JDBCExecutionContext.connect(JDBCExecutionContext.java:103)
at org.jkiss.dbeaver.model.impl.jdbc.JDBCRemoteInstance.initializeMainContext(JDBCRemoteInstance.java:102)
at org.jkiss.dbeaver.model.impl.jdbc.JDBCRemoteInstance.(JDBCRemoteInstance.java:61)
at org.jkiss.dbeaver.model.impl.jdbc.JDBCDataSource.initializeRemoteInstance(JDBCDataSource.java:128)
at org.jkiss.dbeaver.model.impl.jdbc.JDBCDataSource.(JDBCDataSource.java:106)
at org.jkiss.dbeaver.model.impl.jdbc.JDBCDataSource.(JDBCDataSource.java:97)
at org.jkiss.dbeaver.ext.mssql.model.SQLServerDataSource.(SQLServerDataSource.java:80)
at org.jkiss.dbeaver.ext.mssql.SQLServerDataSourceProvider.openDataSource(SQLServerDataSourceProvider.java:137)
at org.jkiss.dbeaver.registry.DataSourceDescriptor.openDataSource(DataSourceDescriptor.java:1180)
at org.jkiss.dbeaver.registry.DataSourceDescriptor.connect(DataSourceDescriptor.java:1049)
at org.jkiss.dbeaver.runtime.jobs.ConnectJob.run(ConnectJob.java:70)
at org.jkiss.dbeaver.runtime.jobs.ConnectionTestJob.run(ConnectionTestJob.java:103)
at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:105)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)

Your issue is likely connected with the newest version of Java that we added in the last release.
Java was repacked in the Early Access version. Can you please check this version? https://dbeaver.io/files/ea/

Can you please check this version? https://dbeaver.io/files/ea/
Still the same issue as in 22.3.4 release

Kerberos
Deprecate 3DES and RC4 in Kerberos
3DES and RC4 Kerberos encryption types have now been disabled by default. Both 3DES and RC4 are weak encryption algorithms that should not be used. The Kerberos 3DES and RC4 encryption types are officially deprecated in RFC 8429.
By default the des3-hmac-sha1 and rc4-hmac encryption types are now disabled, but can be re-enabled, at your own risk, by setting the allow_weak_crypto property to true in the krb5.conf configuration file. However, note that will also re-enable other weak encryption types that are already disabled such as des-cbc-crc and des-cbc-md5. Alternatively, set the default_tkt_enctypes, default_tgs_enctypes, and permitted_enctypes properties to the encryption types that are allowed.
Issue: JDK-8139348
Possible reason

Issue: JDK-8139348

Possible reason

Initially I posted:

klist -V shown "Kerberos 5 version 1.17"

Following your link:

MIT krb5 has done this in 1.14

DBeaver 22.3.5 - still the same issue

Seems to be related.
https://stackoverflow.com/questions/23801169/kdc-has-no-support-for-encryption-type-14
In 22.3.3 we upgraded to Java 17, some encryption settings may change.
We still trying to figure out how Java 17 affects Kerberos in SQL Server.

In the new Java Kerberos authentication is more strict. For instance it validates that realm matches KDC realm exactly (full domain name).
To get more debug info you can add line
-Dsun.security.krb5.debug=true in the end of dbeaver.ini file.
Then restart DBeaver, try to connect and then look at dbeaver-debug.log file.

I'm witth same problem

Can't create driver instance
Error creating driver 'MS SQL Server / SQL Server' instance.
Most likely required jar files are missing.
You should configure jars in driver settings.
Reason: can't load driver class 'com.microsoft.sqlserver.jdbc.SQLServerDriver'
Error creating driver 'MS SQL Server / SQL Server' instance.
Most likely required jar files are missing.
You should configure jars in driver settings.
Reason: can't load driver class 'com.microsoft.sqlserver.jdbc.SQLServerDriver'
com/microsoft/sqlserver/jdbc/SQLServerDriver has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0
com/microsoft/sqlserver/jdbc/SQLServerDriver has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0

@ceresaconsultoria This issue was fixed in the latest Early Access version.

My version is 22.3.5, what is new version?
Linux mint 21.1 reference Ubuntu 22.04
Atenciosamente,
[image: Mailtrack]
https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality11&
Sender
notified by
Mailtrack
https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality11&
01/03/23,
11:21:08
Em qua., 1 de mar. de 2023 às 05:38, Serge Rider @.***>
escreveu:

@ceresaconsultoria https://github.com/ceresaconsultoria This issue was
fixed in the latest Early Access version.
—
Reply to this email directly, view it on GitHub
https://github.com/dbeaver/dbeaver/issues/18978#issuecomment-1449570541,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ANMHJ2HDGXA5U7F6RXZSKBLWZ4DHZANCNFSM6AAAAAAUTR7M44
.
You are receiving this because you were mentioned.Message ID:
@.***>

New version is 23.0.
Early Access package can be downloaded at https://dbeaver.io/files/ea

I installed it and it worked, thanks for the support


update tensorflow mechanics 101 index.md

Update tensorflow mechanics 101 index.md according to the fully_connected_feed.py.
Now we don't need sess.graph_def but only need sess.graph.

Can one of the admins verify this patch?


Add rancher-logging 0.1.5

https://github.com/rancher/rancher/issues/25182
This broke because privileged was removed in https://github.com/rancher/system-charts/pull/89/.
This adds the option to provide global.privileged = true to add privileged: true to the launched pods for Rancher logging. This is a post upgrade action on the App created in the System project. This choice was made versus the choice to make this the default, as it adds privileged to the pods and it's only needed for a small percentage of installations (RedHat/CentOS + SELinux enforcing + Docker selinux enabled). This needs to be added to the docs.
The choice here is to make it work for everyone (by adding privileged by default) or make it an option for the smaller set of usecases (see https://github.com/fluent/fluentd-kubernetes-daemonset#running-on-openshift as an example as well)

In the app readme, add in a change log of what's been updated.


Problem with applying styles in minified browser bundle

Hi, when i try to apply style to a range using minified browser bundle, i've got an exception
Uncaught (in promise) Error: Unknown type: Style
    at ArgHandler.js:79
    at lodash.js:2861
    at lodash.js:4901
    at qe (lodash.js:2860)
    at Function.pe.every (lodash.js:9131)
    at t.value (ArgHandler.js:64)
    at t.value (ArgHandler.js:45)
    at r.value (Range.js:330)

in full bundle everything is OK.
it happens because after minification style class name become 'o', so args handler can't match and process it

Yep. You're spot on. ArgHandler should be using references for classes instead of class names.

It's been a year, any chance this will be fixed?


exploitdb: 2023-06-24 -> 2023-06-27

Diff: https://gitlab.com/exploit-database/exploitdb/-/compare/refs/tags/2023-06-24...2023-06-27
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 240079 run on x86_64-linux 1

1 package built:

exploitdb


Data Import from NVD data Script Added

Summary
This pull request enhances the vulnerability data import by allowing data to be imported from a json data stored in the local directory NVD_Data. It also considers robust error handling to manage duplicate package entries and missing data. Additionally, the code supports storing a simplified "plain" package URL without qualifiers or subpaths.
Related Issues
https://github.com/aboutcode-org/vulnerablecode/issues/1437
Testing Instructions
Run the import process with the following command:
python manage.py import_data
Test importing data from a local folder i.e NVD_Data.
Check that duplicate package entries are gracefully handled and skipped.
Review the logs to ensure they accurately reflect each step of the process, including any errors or skipped files.
Verify that plain package URLs (without qualifiers and subpaths) are being correctly stored in the database.

I have tried to import the data directly from the github api but that was quite a slow process and also it was not importing full data from almost 6400 files it was able to import from 1800 files.Maybe there is rate limit problem of github api. So what would you recommend to do in this situation.

@pombredanne The recent changes are working fine you just have to put your GitHub PAT in the code and you can run python manage.py import_nvd_data the import will start and when I have runned the script to import the data on my local machine then almost cumulatively 25k data entries were created in the Package Related Vulnerabilities , package, vulnerability,vulnerability reference Models.


Incorrect whitespace in status line

:ver
VIM - Vi IMproved 8.1 (2018 May 18, compiled May 18 2018 18:36:07)
MS-Windows 32-bit GUI version with OLE support
Included patches: 1
Compiled by mool@tororo

It seems that statusline doesn't properly work with whitespace.
Here is my .vimrc:
set encoding=utf-8
set guifont=Consolas:h10
colorscheme desert

let separator = '    '
set laststatus=2
set statusline=
set statusline+=%n
set statusline+=%{separator}
set statusline+=%t
set statusline+=%=
set statusline+=%(%l:%c%V%)
set statusline+=%{separator}
set statusline+=%{&expandtab?'spaces':'tab\ size'}:\ %{shiftwidth()}
set statusline+=%{separator}
set statusline+=%{&fileformat}
set statusline+=%{separator}
set statusline+=%{(strlen(&fileencoding)?&fileencoding:&encoding).((exists('+bomb')&&&bomb)?'-with-bom':'')}
set statusline+=%{separator}
set statusline+=%{strlen(&filetype)?&filetype:'text'}

And here is how Vim looks:

As you can see, at some places there are only 3 spaces instead of 4.  and it looks like a bug for me.
Then, if we use hard-coded spaces instead of separator - everything will be correct:
set encoding=utf-8
set guifont=Consolas:h10
colorscheme desert

set laststatus=2
set statusline=
set statusline+=%n\ \ \ \ "
set statusline+=%t
set statusline+=%=
set statusline+=%(%l:%c%V%)\ \ \ \ "
set statusline+=%{&expandtab?'spaces':'tab\ size'}:\ %{shiftwidth()}\ \ \ \ "
set statusline+=%{&fileformat}\ \ \ \ "
set statusline+=%{(strlen(&fileencoding)?&fileencoding:&encoding).((exists('+bomb')&&&bomb)?'-with-bom':'')}\ \ \ \ "
set statusline+=%{strlen(&filetype)?&filetype:'text'}

As your :version heading shows, this Vim executable was compiled on the very first day of the 8.1 release, almost 9 months ago.

Actually, it some kind of fresh version :-) I downloaded it January 30. It seems that Windows binary is a bit outdated.
Best regards you too :-)

I think this is a duplicate of #1431

It seems that statusline doesn't properly work with whitespace.
Here is my .vimrc:
set encoding=utf-8
set guifont=Consolas:h10
colorscheme desert

let separator = '    '
set laststatus=2
set statusline=
set statusline+=%n
set statusline+=%{separator}
set statusline+=%t
set statusline+=%=
set statusline+=%(%l:%c%V%)
set statusline+=%{separator}
set statusline+=%{&expandtab?'spaces':'tab\ size'}:\ %{shiftwidth()}
set statusline+=%{separator}
set statusline+=%{&fileformat}
set statusline+=%{separator}
set statusline+=%{(strlen(&fileencoding)?&fileencoding:&encoding).((exists('+bomb')&&&bomb)?'-with-bom':'')}
set statusline+=%{separator}
set statusline+=%{strlen(&filetype)?&filetype:'text'}

And here is how Vim looks:

As you can see, at some places there are only 3 spaces instead of 4.  and looks like a bug for me.
Then, if we use hard-coded spaces instead of separator - everything will be correct:
set encoding=utf-8
set guifont=Consolas:h10
colorscheme desert

set laststatus=2
set statusline=
set statusline+=%n\ \ \ \ "
set statusline+=%t
set statusline+=%=
set statusline+=%(%l:%c%V%)\ \ \ \ "
set statusline+=%{&expandtab?'spaces':'tab\ size'}:\ %{shiftwidth()}\ \ \ \ "
set statusline+=%{&fileformat}\ \ \ \ "
set statusline+=%{(strlen(&fileencoding)?&fileencoding:&encoding).((exists('+bomb')&&&bomb)?'-with-bom':'')}\ \ \ \ "
set statusline+=%{strlen(&filetype)?&filetype:'text'}

I hesitate to change how 'statusline' deals with whitespace especially
for %{} items.  There are some tricks to drop whitespace, especially if
one of the items is empty.  Existing implementations depend on that.
A better solution would be to explicitly have an item that specifies the
amount of white space, which then would be inserted literally.
--
ARTHUR: Charge!
[They all charge with swords drawn towards the RABBIT.  A tremendous twenty
second fight with Peckinpahish shots and borrowing heavily also on the
Kung Fu and karate-type films ensues, in which some four KNIGHTS are
comprehensively killed.]
ARTHUR: Run away!  Run away!
"Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD
/// Bram Moolenaar --<EMAIL_ADDRESS>-- http://www.Moolenaar.net   \
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \
\\  an exciting new programming language -- http://www.Zimbu.org        ///
\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

@brammool

I hesitate to change how 'statusline' deals with whitespace especially
for %{} items.  There are some tricks to drop whitespace, especially if
one of the items is empty.  Existing implementations depend on that.

In other words, it's some kind of bug, which is considered as feature?

A better solution would be to explicitly have an item that specifies
the amount of white space, which then would be inserted literally.

Interesting. As n00b, I don't know the way how it should be done, but
still interesting.
// There are at least 2 Monty Python fans here!

What if we allow %0{...} to disable the space eating?  Zero pad is not otherwise used for { in the statusline.  Implementation is very simple:
--- a/src/buffer.c
+++ b/src/buffer.c
@@ -4223,7 +4223,7 @@ build_stl_str_hl(
            break;

        case STL_VIM_EXPR: /* '{' */
-           itemisflag = TRUE;
+           itemisflag = zeropad ? FALSE : TRUE;
            t = p;
            while (*s != '}' && *s != NUL && p + 1 < out + outlen)
                *p++ = *s++;

Does it exist a better workaround then using "?
" This line is used as separator:
set statusline+=%\ \ \ \ "

I have the same problem, is there any news? Thanks

I have the same problem, is there any news? Thanks


Error under Separate Containers: /etc/docker-gen/templates/nginx.tmpl is a directory.

I exactly follow the instructions as below:
docker run -d -p 80:80 -p 443:443
--name nginx
-v /etc/nginx/conf.d
-v /etc/nginx/vhost.d
-v /usr/share/nginx/html
-v /path/to/certs:/etc/nginx/certs:ro
nginx
docker run -d
--name nginx-gen
--volumes-from nginx
-v /path/to/nginx.tmpl:/etc/docker-gen/templates/nginx.tmpl:ro
-v /var/run/docker.sock:/tmp/docker.sock:ro
jwilder/docker-gen
-notify-sighup nginx -watch -only-exposed -wait 5s:30s /etc/docker-gen/templates/nginx.tmpl /etc/nginx/conf.d/default.conf
The first one Nginx works normally, but the second one Docker-gen exited in no time. The docker log shows: 2016/03/15 01:24:42 unable to parse template: read /etc/docker-gen/templates/nginx.tmpl: is a directory
Could you kindly tell me what's wrong and how to make it? Thank you very much.

I think your nginx.tmpl is either missing or has a syntax error. could post those 2 commands with the real paths along with the content of your nginx.tmpl?

First, I simply run the two commands exactly the same with that in the instructions:
hyp530@vultr:~$ docker run -d -p 80:80 -p 443:443 \

--name nginx
-v /etc/nginx/conf.d
-v /etc/nginx/vhost.d
-v /usr/share/nginx/html
-v /path/to/certs:/etc/nginx/certs:ro
nginx
Unable to find image 'nginx:latest' locally
latest: Pulling from library/nginx

fdd5d7827f33: Already exists
a3ed95caeb02: Pull complete
716f7a5f3082: Pull complete
7b10f03a0309: Pull complete
Digest: sha256:f6a001272d5d324c4c9f3f183e1b69e9e0ff12debeb7a092730d638c33e0de3e
Status: Downloaded newer image for nginx:latest
cbe27695310d474aad23eeee28387f3aaad3ade2a510b21e84934324ed27903f
hyp530@vultr:/path/to$ docker run -d \

--name nginx-gen
--volumes-from nginx
-v /path/to/nginx.tmpl:/etc/docker-gen/templates/nginx.tmpl:ro
-v /var/run/docker.sock:/tmp/docker.sock:ro
jwilder/docker-gen
-notify-sighup nginx -watch -only-exposed -wait 5s:30s /etc/docker-gen/templates/nginx.tmpl /etc/nginx/conf.d/default.conf
Unable to find image 'jwilder/docker-gen:latest' locally
latest: Pulling from jwilder/docker-gen
4d690fa98655: Pull complete
a3ed95caeb02: Pull complete
15845219ea85: Pull complete
Digest: sha256:48471b4b3317ca38df9e143ab377ec225d4da64a010a63c86c5508314ff831ec
Status: Downloaded newer image for jwilder/docker-gen:latest
4efb0a437bcd2e408b5ec028db2e27c5f97c56e673aa7c9cd5fe1c10c2001e78

**Second, I found Nginx-gen container not running, and I use docker logs to find the error as below: **
hyp530@vultr:/path/to$ docker logs nginx-gen
2016/03/16 02:32:27 unable to parse template: read /etc/docker-gen/templates/nginx.tmpl: is a directory
**Third, I found /path/to/nginx.tmpl automatically created, and it's a folder. Then I remove it and replace with a new file of /path/to/nginx.tmpl with content copied from (https://github.com/jwilder/nginx-proxy/blob/master/nginx.tmpl). **
**Then, I restart Docker-gen container again and encounter a new error: **
hyp530@vultr:/path/to$ docker start nginx-gen
Error response from daemon: Cannot start container 4efb0a437bcd2e408b5ec028db2e27c5f97c56e673aa7c9cd5fe1c10c2001e78: [9] System error: not a directory
Fourth, I check the logs with the following records:
hyp530@vultr:/path/to$ docker logs 4efb0a437bcd
2016/03/16 02:32:27 unable to parse template: read /etc/docker-gen/templates/nginx.tmpl: is a directory
Timestamp: 2016-03-16 02:36:50.210673626 +0000 UTC
Code: System error
Message: not a directory
Frames:
0: setupRootfs
Package: github.com/opencontainers/runc/libcontainer
File: rootfs_linux.go@40
1: Init
Package: github.com/opencontainers/runc/libcontainer.(*linuxStandardInit)
File: standard_init_linux.go@57
2: StartInitialization
Package: github.com/opencontainers/runc/libcontainer.(*LinuxFactory)
File: factory_linux.go@240
3: initializer
Package: github.com/docker/docker/daemon/execdriver/native
File: init.go@35
4: Init
Package: github.com/docker/docker/pkg/reexec
File: reexec.go@26
5: main
Package: main
File: docker.go@18
6: main
Package: runtime
File: proc.go@111
7: goexit
Package: runtime
File: asm_amd64.s@1721
Timestamp: 2016-03-16 02:37:01.470234867 +0000 UTC
Code: System error
Message: not a directory
Frames:
0: setupRootfs
Package: github.com/opencontainers/runc/libcontainer
File: rootfs_linux.go@40
1: Init
Package: github.com/opencontainers/runc/libcontainer.(*linuxStandardInit)
File: standard_init_linux.go@57
2: StartInitialization
Package: github.com/opencontainers/runc/libcontainer.(*LinuxFactory)
File: factory_linux.go@240
3: initializer
Package: github.com/docker/docker/daemon/execdriver/native
File: init.go@35
4: Init
Package: github.com/docker/docker/pkg/reexec
File: reexec.go@26
5: main
Package: main
File: docker.go@18
6: main
Package: runtime
File: proc.go@111
7: goexit
Package: runtime
File: asm_amd64.s@1721

The directory /path/toin the README is an example and must be replace to a path to a directory on the host.
Also this directory must contain (if you follow the commands in the README):

A directory certs
The nginx.tmpl file

Please read the documentation about docker volumes (espacialy the Mount a host directory as a data volume)

If future folks need some clarity, the exact problem is the voluming of the nginx.tmpl file.
This:
-v /path/to/nginx.tmpl:/etc/docker-gen/templates/nginx.tmpl:ro

... should be this:
-v /path/to:/etc/docker-gen/templates


feat: command changes from tutorial feedback

addresses some points from #314, particularly the ones under the Command changes section
Feedback changes:

renames the CLI binary to miden
for miden account show we now show all account info and we remove the flags to show the vault/script/etc.
created a root import command (so now you do miden import (account|note) <args>) and removed the ones under input-notes and account
added defaults for some root commands (for account, input-note, tags and tx we always default to the List subcommand)

TODO

[ ]  rebase against igamigo-next-0.3 once it gets stabilized
[ ] decide if we need to change any clap option into a comand or the opposite

Not for this PR: I'm wondering if we can have just miden import <filepaths> --no-verify and the client will figure out based on the file whether it is importing accounts or notes. One way to do this is to add something like a 4-byte "magic" value in front of each file during serialization. This could then be used to identify the type of object we are dealing with.

for miden account show we now show all account info and we remove the flags to show the vault/script/etc.

Similar thought to the ones discussed elsewhere (and probably not for this PR): for me personally, miden account --show <account_id> would have been a more natural command.

created a root import command (so now you do miden import (account|note) <args>) and removed the ones under input-notes and account

Not for this PR: I'm wondering if we can have just miden import <filepaths> --no-verify and the client will figure out based on the file whether it is importing accounts or notes. One way to do this is to add something like a 4-byte "magic" value in front of each file during serialization. This could then be used to identify the type of object we are dealing with.

If we want to handle both, couldn't we just try deserializing into one of the two structs and if that fails we try with the other one? That should be doable in the current state

If we want to handle both, couldn't we just try deserializing into one of the two structs and if that fails we try with the other one? That should be doable in the current state

Yep - that would work too. With a "magic" value upfront, error messaging could be a bit cleaner - but that's a pretty minor thing.

You also don't risk having a file deserializing as an incorrect struct, which is probably not a likely scenario (especially considering the difference between eg accounts and notes) but that could definitely happen eventually.


Full Access Key Management

Story
As a user, I want to encounter full access keys when they are relevant to me, and learn when and how to add and remove new keys, so I can intentionally manage them.
Acceptance Criteria

[ ] Full Access Keys do not appear in Summary
[ ] Full Access Keys appear in Profile

[ ] Description of full access key purpose (see notes)
[ ] List of active keys with info:

[ ] Key type (see notes)
[ ] Date created

[ ] User can deauthorize key

[ ] User must confirm intent to deauthorize

[ ] Mobile ok
[ ] Tested

Assets

Mockup:

NOTES

Full Access Key description:
Your full access keys are displayed here. These keys can be used to perform any action on your account, including: creating new accounts, transferring all of your tokens, deploying contracts, calling functions on any contract, staking and unstaking NEAR, and creating and deleting limited access keys.

Key types

Web Wallet (created on signup)
Seed Phrase Recovery (created if user opts for seed phrase)
Email Recovery (created if user opts for email recovery)
External (created with Ledger, third party wallet)
NEAR Shell (created through near login in near-shell)

I think we should replace "Full Access Keys" with some more user-focused language. E.g. "Authorized Devices" or smith like that.

User must confirm intent to deauthorize

Also shouldn't be possible to remove last key without completely removing account.

Full Access Keys do not appear in Summary

I think that it's good idea to show device that have access in summary. That way user is more likely to notice that there are some old device still in the list and can clean them up. It's likely best to highlight which devices haven't been used for signing for a while.

I think we should replace "Full Access Keys" with some more user-focused language. E.g. "Authorized Devices" or smith like that.

Agreed. I think we can show both "Recovery Options" (keys associated w/ email, seed phrase, etc) and "Authorized Devices", and avoid using the word "key" entirely.

Agreed. I think we can show both "Recovery Options" (keys associated w/ email, seed phrase, etc) and "Authorized Devices", and avoid using the word "key" entirely.

+1.


[DRAFT] Refactor common code: virt-handler/{isolation,device-manager}

What this PR does / why we need it:
The PR introduces some refactoring aimed to reduce code duplication. It includes changes to the following packages:

virt-handler/isolation: wrap the code which parses /proc/mountinfo into an iterator-like function. The function handles the IO operations, parses each line and invokes the given lambda passing the parsed record as an argument. The itaration process can be stopped before the EOF by returning true from the lambda.
virt-handler/device-manager: use struct composition for device plugins. PCI and mediated device plugins share a lot of common code with generic device. To reduce code duplication the GenericDevicePlugin structure can be embedded into the 'derived' types allowing the existing common methods to be reused.

Special notes for your reviewer:
Regarding the changes to the device-manager, for the moment I just quickly refactored the methods which are fully identical. In theory there is more code which shares the same common bits. But it may slightly differ for various device types. If the approach with composition is okay (and if the PR does not break the tests in CI) I can also try to refactor it further.
Release note:

NONE

Ah, looks like there is an issue with composition. There are "private" methods which are invoked from the "Public" ones and that is not going to work as expected out of the box. Need to decouple those... which makes things more complicated. So will spend some more time on the PR, probably in its current shape it is not correct.

Ah, looks like there is an issue with composition. There are "private" methods which are invoked from the "Public" ones and that is not going to work as expected out of the box. Need to decouple those... which makes things more complicated. So will spend some more time on the PR, probably in its current shape it is not correct.

For now decided to split the PR.

For now decided to split the PR.

/retest

/retest

/test pull-kubevirt-e2e-k8s-1.17-rook-ceph

/test pull-kubevirt-e2e-k8s-1.17-rook-ceph

/lgtm
/approve
/retest
/ok-to-test

/lgtm
/approve
/retest
/ok-to-test


Preparing/testing for v0.2.0

It has been long enough that I feel it's time for a v0.2.0 release.
If anyone has any concerns about the current state of NDB, speak up now!
If there is anyone who would be willing to test the new functionality re: dialogs, please do.
@clisboa if you want to test with KoboMail, artifacts are available from the Actions tab, along with documentation (for the latest run). Any feedback from a potential user would be most appreciated.

Thanks for having a test.
If you end up using the new features in KoboMail, you are going to want to do a version test. You can call the ndbVersion method for that.

For KoboMail I'm thinking of not having the progress bar.
The use case is to have a few ebooks to download and that's quite fast, the progress bar would just flicker :)
But I'm working on something with rclone (similar to your own but without metadata processing) and for that I think the progress bar is going to be a plus.
I'll look for the ndbVersion and check if it's > 0.2

It's finally out! -> https://github.com/shermp/NickelDBus/releases/tag/0.2.0


updating packages

I have resolved some security vulnerabilities by doing the following:

upgraded some packages
installed some peer dev dependancies manually
removed the firebase-tools package as had a vulnerability in it and the package wasn't being used anywhere. Note: If we need it in future add the latest (secure) version.
updated sass-loader to v10 (which is the latest version of sass-loader that is compatible with Vue CLI v4). The application now compiles faster :)

I also resolved some minor compilation warnings coming from the linter
Note: The majority of the vulnarabilities left are moderate and caused by an outdated version of postcss but upgrading this would require many other packages to be upgraded too.
PREVIEW:DEVELOP

@HalcyonJAC  do you have an idea of how many vulnerabilities were fixed here?
This is what it is left.
51 vulnerabilities (5 low, 45 moderate, 1 high)

@HalcyonJAC do you have an idea of how many vulnerabilities were fixed here?

@lloback the parent ticket shows that before these changes where were 229 vulnerabilities (105 low, 93 moderate, 31 high) in 2422 scanned packages

62 vulnerabilities (5 low, 50 moderate, 7 high)

I think we should aim to at least have ZERO high risk vulnerability

npm audit --production now returns found 0 vulnerabilities using node v12.22.1 (npm v6.14.12)

Hey @HalcyonJAC and @lloback,
Ok so this is not an area know much about (shhh don't tell anyone!).
If I run npm audit --production on the main branch (i.e. before this fix) I get the following report:

If I run npm audit --production on this branch (i.e. with this fix) I get the following report:

So that's great we've gone from 5 vulnerabilities down to 0.
If I run npm audit so that we're checking devDependencies too then we still have vulnerabilities:

I understand that vulnerabilities here are less of an issue and almost expected as fixing libraries used by developers is lower priority than fixing libraries used in production.... but what should our tolerance here be? Is it ok to have high priority vulnerabilities in our dev tooling?
Have created the following ticket to answer and document these questions:
https://github.com/jac-uk/documentation/issues/38

I have left a comment on https://github.com/jac-uk/documentation/issues/38. In short, I think the principle is fine, the problem is when you can only resolve the vulnerability by upgrading a dependency a major version which then breaks loads of stuff...

I tried updating additional dependancies to resolve the remaining vulnerabilities but then the app would not compile reliably, so I have left it as is.

@joy-ade and @KateMJAC , you also need to approve the PR so it can be merged. You can't test this, but you need to trust the developers that approved it!


80 column character limit

Linux kernel deprecated the 80 column character limit rule.
Can Ninja do the same?
https://www.phoronix.com/scan.php?page=news_item&px=Linux-Kernel-Deprecates-80-Col

While the old Coding Guidelines explicitly mentioned the 80-column limit, it's not a Ninja-rule, but part of the Google C++ Coding Style: https://google.github.io/styleguide/cppguide.html#Line_Length
I want to keep the additions where we deviate from the Google C++ Coding Style as short as possible.

Why do you want to avoid deviating from the google coding style?

Talk to Google then.
Because it's easier to point to it than to bike-shed over stuff like this.

I don't work for google, and neither do you (since the last time you talked about it, anyway).
sigh fine.  Not the end of the world.


Add unsigned 64 bit int read/write methods to cythonized stream

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

[ ] Make sure the PR title is formatted like:
[BEAM-<Jira issue #>] Description of pull request
[ ] Make sure tests pass via mvn clean verify. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
[ ] Replace <Jira issue #> in the title with the actual Jira issue
number, if there is one.
[ ] If this contribution is large, please file an Apache
Individual Contributor License Agreement.

R: @dhalperi @aaltay was able to verify locally that tests pass on my virtual env with cython enabled.

Retest this please

Coverage decreased (-0.007%) to 69.711% when pulling a08967b32ab7482c70017e2824a226869d6f4866 on vikkyrk:py_coder_cy into bea101a4479645b020198dccefb2c6f51a7cc6f0 on apache:master.

LGTM. It was not obvious to me where this is different than what I tried, but if your tests pass LGTM :)

@dhalperi I think you tried with cpdef write_bigendian_uint64(self, libc.stdint.int64_t v). Note that the function parameter is int64_t instead of uint64_t

Coverage decreased (-0.0005%) to 69.717% when pulling d35aa09cb5a50bf50aa4a456c4d0a0739107ff61 on vikkyrk:py_coder_cy into bea101a4479645b020198dccefb2c6f51a7cc6f0 on apache:master.

LGTM. Thank you.


Updated Outlook.AddressEntry.GetExchangeUser.md

Fixed code formatting in the example.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: FlameHorizon sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

Docs Build status updates of commit 10c5c4e:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

api/Outlook.AddressEntry.GetExchangeUser.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

@FlameHorizon Nice catch! We really appreciate your help keeping the docs nice and tidy for other customers. Thanks much<EMAIL_ADDRESS>


[hotfix][tests] Change some StreamTask tests to create a test task in the task's thread

What is the purpose of the change
The changes introduced in #8692 PR have an invariant, that the StreamTask's mailbox loop is running in the same thread that set in the taskMailboxExecutor. Currently, the thread reference is set in StreamTask instantiation time, this conflicts with some StreamTask tests, as they were creating the task instance in a different (non task's) thread.
Brief change log

StreamTaskTest is rewritten to properly run a task under test, instead of overriding some fields via Whitebox;
the rest of tests in flink-streaming-java are rewritten to delegate task instance creation to the task's main thread.

Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
Running the tests in flink-streaming-java and flink-streaming-scala should be green. Also, running the same tests with changes in #8692 should be green.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

CC @StefanRRichter, @pnowojski.

I tried to drop  the mock(StreamOperator.class) and mock(OperatorChain.class) calls, but it's not that easy :/ It would require more sophisticated lifecycle simulation in StreamTaskTest to make it work, so I'm dropping this issue for now.

CI fails on some other test:
01:36:14.687 [ERROR] testMigrationAndRestore[Migrate Savepoint: 1.2](org.apache.flink.test.state.operator.restore.unkeyed.ChainLengthIncreaseTest)  Time elapsed: 7.853 s  <<< ERROR!
java.util.concurrent.ExecutionException: java.util.concurrent.CompletionException: org.apache.flink.runtime.checkpoint.CheckpointException: Task received cancellation from one of its inputs
Caused by: java.util.concurrent.CompletionException: org.apache.flink.runtime.checkpoint.CheckpointException: Task received cancellation from one of its inputs
Caused by: org.apache.flink.runtime.checkpoint.CheckpointException: Task received cancellation from one of its inputs
Caused by: org.apache.flink.runtime.checkpoint.CheckpointException: Task received cancellation from one of its inputs

Another, unrelated PR with the same failure:
https://travis-ci.org/apache/flink/jobs/551265900

Thanks. Indeed failure seems unrelated. I'm merging it.


⚠ AGE-Vicmap_Position/MS has degraded performance

In 1abdf04, AGE-Vicmap_Position/MS (https://vicmap.land.vic.gov.au/hosting/rest/services/vicmap/Vicmap_Position/MapServer/0/query?where=1%3D1&returnCountOnly=true) experienced degraded performance:

HTTP code: 200
Response time: 2181 ms

Resolved: AGE-Vicmap_Position/MS performance has improved in a347bbf after 4 minutes.


[opencv4] Disable building cpufeatures since it conflict with libwebp

Disable building cpufeatures as it is the same as provided by libwebp.
They are use the same source (in Android NDK),
Upstream issue: https://github.com/opencv/opencv/issues/9155 but it didn't completely fix it.
Fixes https://github.com/microsoft/vcpkg/issues/19106.

cc @cenit.

it is necessary also on opencv 3 i suppose

it is necessary also on opencv 3 i suppose

I'm thinking if the upstream opencv doesn't fix this, should we fix it completely on the vcpkg side: export the target cpufeatures in libwebp and add the related option in opencv to use it.

Actually, this is just correct; we also need to rip cpufeatures out of libwebp, and ship cpufeatures as its own library, imo. Until we do that tho, this is strictly better, imo. Thanks!


Fix fallback API not being triggered

⚠ There may be a safer way to do this ⚠
This PR addresses #63. The problem was that the NetworkAPI.IGNORED_ERRORS variable did not include many errors like server not reachable, 400 etc...
To fix it, I included ALL exceptions in IGNORED_ERRORS, as we want the fallback to be triggered no matter what the error is, except one in the codebase (the latter is yet to be implemented).

Interesting. Not certain how I feel about allowing all exceptions. What exception was  the bad API giving that we weren't retrying over?
Ideally, we would catch cases where an API is removed, changed, or broken. But of course you want it to be stable for users at the same time.

What exception was the bad API giving that we weren't retrying over?

requests.exceptions.HTTPError: 523 Server Error

Ideally, we would catch cases where an API is removed, changed, or broken. But of course you want it to be stable for users at the same time.

Yes, basically any HTTP error + any network error (if the site is blocked or DNS not resolving). I am not very familiar with Exceptions. Would adding all exceptions listed at https://2.python-requests.org/en/master/_modules/requests/exceptions/ work?

Fantastic, thank you!


refactor: constructor of commands and passing around of contextual tu…

…ples in Check & ListObjects

Description

References

Review Checklist

[ ] I have clicked on "allow edits by maintainers".
[ ] I have added documentation for new/changed functionality in this PR or in a PR to openfga.dev [Provide a link to any relevant PRs in the references section above]
[ ] The correct base branch is being used, if not main
[ ] I have added tests to validate that the change in functionality is working as expected

There are a lot more changes in the scope of this PR than just constructor refactoring. These changes are large changes to the Check implementation that had purposeful design behind it, and so at minimum this warrants a large discussion before we open a draft PR.


[DRM] Fix forecefully switching to the preferred mode, use the current mode rather

[ ] Has this change been discussed on the forum or in an issue before?
[x] Does the code follow the SFML Code Style Guide?
[ ] Have you provided some example/test code for your changes?
[ ] If you have additional steps which need to be performed list them as tasks!

Description
In order to prevent an unnecessary modeswitch and use the current TTY resolution, don't use the preferred mode reported by DRM. This is useful for people who set a custom resolution at boot and are forced into the monitor's preferred resolution when a SFML DRM app starts
Tasks

[x] Tested on Linux
[ ] Tested on Windows
[ ] Tested on macOS
[ ] Tested on iOS
[ ] Tested on Android

How to test this PR?
Make sure your TTY is not set at your monitor's preferred resolution and start any app

SFMLBuildMaster: Build this please.

https://github.com/SFML/SFML/issues/2252
I made an issue for discussing moving these DRM files into SFML. Please continue any relevant conversation in there.

SFMLBuildMaster: Build this please.

If pushed two small style fixes

Thanks for this fix! 🙂


Added doc for Go introduction

What
Added documentation for "Intro to Go"
How to review
Check for typos and check if it reads right.
Who can review
Anyone

It is really great to discover some new references, thank you for putting these together it is really helpful


App using plugins, and live reload of plugins

[x] the app can load plugins as well
[x] we can use a user config plugins folder (eg. ~/.config/Headlamp/plugins)
[x] live reload happens when the plugins folder changes

How to use
make backend run-backend
make run-frontend
# copy a plugin into ~/.config/Headlamp/plugins

Testing done

[x] app can load plugins from the user config plugins folder
[x] live reload happens when creating/updating/deleting plugins

Thanks for the review. I pushed some changes encapsulating the plugins functions into plugins.go, and also addressed a couple of your notes.
I'll look into the config folder name, and the header now.

I noticed one bug in the headless app mode not reladed to this PR... When reloading the page it errors and shows as a blank page. The reloading works fine in the app and with other ways of running Headlamp however.

I added the backwards compatible copying of plugins from ./plugins to ./.plugins and use the .plugins folder as a default if it exists. So this way the lokomotive build.sh should still work without changes.

I added the ".plugins" folder to gitignore

In the docker image, it keeps things in "headlamp/plugins" again. I had to update the find command.


Getting rid of covariate argument

This is an idea to fix #83. get_like_lpdf() would be the "official" method used by other classes, and the user can implement either of the overloaded versions of like_lpdf(). Same with other methods. Do you like it?

I think this makes sense!

With like_lpdf(), I had smooth sailing. With marg_lpdf() though, it becomes more complicated, due to the fact that it is a free function. First of all, in functions such as conditional_pred_lpdf(), I tried the usual pattern:
if (is_dependent())
  return marg_lpdf(posterior_hypers, datum, covariate);
else
  return marg_lpdf(posterior_hypers, datum);

with all the needed static casts. However, I get the following error:
/home/bruno/git/bayesmix/src/hierarchies/conjugate_hierarchy.h:63:59: error: no matching function for call to ‘NNIGHierarchy::marg_lpdf(const NNIG::Hyperparams&, const RowVectorXd&, const RowVectorXd&) const’
   63 |       return static_cast<Derived const *>(this)->marg_lpdf(posterior_hypers,
      |              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~
   64 |                                                            datum, covariate);
      |                                                            ~~~~~~~~~~~~~~~~~
/home/bruno/git/bayesmix/src/hierarchies/nnig_hierarchy.cc:18:8: note: candidate: ‘virtual double NNIGHierarchy::marg_lpdf(const NNIG::Hyperparams&, const RowVectorXd&) const’
   18 | double NNIGHierarchy::marg_lpdf(const NNIG::Hyperparams &params,
      |        ^~~~~~~~~~~~~
/home/bruno/git/bayesmix/src/hierarchies/nnig_hierarchy.cc:18:8: note:   candidate expects 2 arguments, 3 provided

This also happens for prior_pred_lpdf() and the other 2 hierarchies involves, for a total of 6 times. Please note that I also added the following methods to the ConjugateHierarchy class:
virtual double marg_lpdf(const Hyperparams &params,
                         const Eigen::RowVectorXd &datum,
                         const Eigen::RowVectorXd &covariate) const {
  throw std::runtime_error("Not implemented");
}

virtual double marg_lpdf(const Hyperparams &params,
                         const Eigen::RowVectorXd &datum) const {
  throw std::runtime_error("Not implemented");
}

These appear to be ignored by the compiler, which only sees the implemented version in the derived class, therefore the error. Am I missing something?
One easy solution I can think of would be to abolish the ConjugateHierarchy class and make conjugacy depend on the appropriate flag, just like dependent hierarchies right now. Which I'm a fan of, by the way, because giving different treatments to conjugacy and dependancy makes no sense and leads to shaky stuff like marg_lpdf() being a free function.

I am not a fan of removing ConjugateHierarchy, since it really cleans up the code for conjugate hierarchies.
Does the same happen if instead of having a member function bool is_dependent() we have a private attribute like
const bool IS_DEPENDENT = true / false?

We cannot use a const bool, because it does not work correctly with inheritance. I believe this was the reason we went with a method instead of a variable member in the first place:
#include <iostream>
class Foo {
public:
    bool getx() { return x; }
protected:
    const bool x = false;
};
class Bar: public Foo {
protected:
    const bool x = true;
};

int main() {
    Foo foo;
    Bar bar;
    std::cout << foo.getx() << " " << bar.getx() << std::endl;  // prints 0 0
}

This won't work because BaseAlgorithm needs it in the abstract class:
[...]/src/algorithms/base_algorithm.cc:79:27: error: ‘using element_type = class AbstractHierarchy’ {aka ‘class AbstractHierarchy’} has no member named ‘IS_DEPENDENT’
   79 |     if (unique_values[0]->IS_DEPENDENT == false) {
      |                           ^~~~~~~~~~~~

We have no access to the template types in the algorithm, which was the whole point of the existence of AbstractHierarchy, as far as I remember.
Besides, using a free bool member would be yet another shaky practice, which I don't particularly endorse.

We know what, we might just bite the bullet and solve the implementation choice that leads to marg_lpdf() being a free function: the use of structs for states and hyperparameters. If we replace them with protos, we would have all sorts of benefits, among which getting rid of the complicated CRTP pattern, and ultimately solving the issue at hand. Was there any reason for which we kept the structs?

You know what, we might as well bite the bullet and change the implementation choice that leads to marg_lpdf() being a free function: the use of structs for states and hyperparameters. If we replace them with protos, we would have all sorts of benefits, among which getting rid of the complicated CRTP pattern, and ultimately solving the issue at hand. Was there any reason for which we kept the structs?

The main reason is efficiency, suppose your state is a large covariance matrix. Then every time you need to do any math on this, you need to instantiate an Eigen::MatrixXd from a Matrix proto, which can be quite expensive.
I don't feel like giving up just yet (also, reverting back from the CRTP can be a huge pain, we should try to stick to it)

This won't work because BaseAlgorithm needs it in the abstract class:
[...]/src/algorithms/base_algorithm.cc:79:27: error: ‘using element_type = class AbstractHierarchy’ {aka ‘class AbstractHierarchy’} has no member named ‘IS_DEPENDENT’
   79 |     if (unique_values[0]->IS_DEPENDENT == false) {
      |                           ^~~~~~~~~~~~

We have no access to the template types in the algorithm, which was the whole point of the existence of AbstractHierarchy, as far as I remember. Besides, using a free bool member would be yet another shaky practice, which I don't particularly endorse.

Ok but this can be solved quite easily. In BaseHierarchy::get_marg_lpdf we do
if (if static_cast<Derived*>(this)->IS_DEPENDENT)
  return marg_lpdf(posterior_hypers, datum, covariate);
else
  return marg_lpdf(posterior_hypers, datum);

we keep the AbstractHierarchy::is_dependent() member, but we override it in the BaseHierarchy as
bool BaseHierarchy<...>::is_dependent() {
  return static_cast<Derived*>(this)->IS_DEPENDENT;
}

It doesn't seem to work either. Can you please check that I implemented correctly what you wanted?

So I'm obviously doing the same for the update_summary_statistics() hierarchy methods, as well as for the Mixing class. Should I touch any other function? For instance, NNW has efficient upgrades for prior_pred_lpdf_grid() and conditional_pred_lpdf_grid(), which expose some Eigen::MatrixXd(0, 0). Should I also deal with those?

I think it would be good to a cleanup for all the "basic" functions, don't worry too much about the prior_pred_lpdf_grid() in the derived classes for the moment! I just wanted to have a cleaner code for new users/developers

BTW, I think we can also try to revert to the old is_dependent() check instead of static casting the stuff, since now all the hierarchies have both the methods!

BTW, I think we can also try to revert to the old is_dependent() check instead of static casting the stuff, since now all the hierarchies have both the methods!

You mean changing ConjugateHierarchy like this?:
-    if (static_cast<Derived const *>(this)->IS_DEPENDENT) {
+    if (is_dependent()) {

That won't work unfortunately.
Anyways, I should be done. One noteworthy thing is that stuff breaks down unless we keep the IS_DEPENDENT bool public, as opposed to what good practice mandates. Does this bother us?

So I was just anticipating the work on the merge, and checking that everything ran fine afterwards. Unfortunately, when trying a test with LinRegUniHierarchy, the following error was raised:
Running run.cc
Initializing... Done
Running Neal3 algorithm with LinRegUni hierarchies, DP mixing...
[libprotobuf FATAL google/protobuf/reflection_ops.cc:74] CHECK failed: (to->GetDescriptor()) == (descriptor): Tried to merge messages of different types (merge bayesmix.LinRegUniLSState to bayesmix.UniLSState)

I know for a fact that it worked before the merge, and this looks pretty much like the error you had fixed in the other PR. Maybe you've forgot about this one? Please tell me if you need any file to test this out.

Sorry - my fault. I also updated the tests with the new notation, they all run fine

Great, thanks! Now we just need to address this:

One noteworthy thing is that stuff breaks down unless we keep the IS_DEPENDENT bool public, as opposed to what good practice mandates. Does this bother us?

Great, thanks! Now we just need to address this:

One noteworthy thing is that stuff breaks down unless we keep the IS_DEPENDENT bool public, as opposed to what good practice mandates. Does this bother us?

I made it work with is_dependent() in the BaseHierarchy class. Now I leave it up to you to decide if keeping IS_DEPENDENT as is it is or revert back to overriding the is_dependent() member in the child classes

I don't get it 😐 But I'm surely going back to the original, simpler structure if it works

I should be finished. Please check that everything is OK before merging :)

Sorry about the long wait. After renaming the functions, I went down a rabbit hole of a general cleanup, that is sorting functions and adding missing documentation. I found a couple of things to correct, but we should really merge this one first...


How do I pass values to the HTML interface, using loadData or postUrl, and how should the HTML interface accept parameters?

I found a solution to ban'f in an example.

@wangyn111 On what event you have called this method?

@wangyn111 thank you for the example syntax, I can't pass the postData as null. Yet, you have to at least send the empty string as @wangyn111 provide above.
@dharmitaepaisa It might very late but you can call this method in onWebViewCreated event.
Hope this will help future users.


Engine crash when changing Radiance Size (Godot 3.5.3)

Godot version
3.5.3 RC
System information
Windows 10, Radeon RX 560 Series
Issue description
When changing the Radiance Size in ProceduralSky, between 128 and 32, the engine freezes and crashes.
(The error only occurs in GLES3 rendering mode)
Steps to reproduce
Place WorldEnvironment in the scene, change the radiance size in the Environment resource of the WorldEnvironment node
Minimal reproduction project
error.zip

Related to https://github.com/godotengine/godot/issues/35914.

This is most likely an AMD graphics driver bug we can't fix.

Related to Editor crashes when setting procedural sky texture size to anything larger than current setting #35914.

This is most likely an AMD graphics driver bug we can't fix. However, the issue should be gone in 4.0 and later thanks to the renderer rewrite.

in 4.2 dev4 this error is not observed


Accessories not loading in HOOBS

I am using hoobs to control all devices in my house. Installed this plugin, added few accessories in config. Hoobs logs shows, that bridge started, that accessories added, but this is only in log. No accessories appear in hoobs dashboard. Other plugin (homebridge_gpio_relay) works perfect.
Versions:
Raspberry PI 3B+
hoobs:~# node -v
v14.17.6
hoobs:~# npm -v
6.14.15
hoobs:~# hoobsd -v
Initializing HAP-NodeJS v0.9.4...
4.0.100
Any ideas ?

Same

Looks like it may be something with the hoobs editor, where the plugin config is not setting right. Instead click on advanced and add the configs there.

Same here, anyone with a solution?


Update AccountVault to account for updated TieredSmt logic.

Upstream logic for the TieredSmt has changed to support removals.  We should review the TODO's in the rust implementation of AccountVault.  We should also modify the implementation of the ToAdviceInputs trait to support bottom leaves.

Closed by #185.

Closed by #185.

I was thinking this issue would have a greater scope than just #185.

closed by #195


Request for ground truth of testing data which used in RGB+TOF Depth Completion @MIPI-challenge

Hi @dwHou,
Thanks for your great work! I'm curious about there is anyway to get ground truth(GT ) of tesing data(not the validation_input)?
The reason I'm pursuing GT is because I want to reproduce the performance you mentioned in your paper. However, the GT depth are not available to participants in the challenge, the official said.

You're welcome. The challenge is formed by two phases: validation and the final test. The GT for validation is here. The GT for the final test is not officially released, so I couldn't get it either. Fortunately, the data sources of these two phases are similar.

Hi @dwHou ,
One more quesiton, I also found the "Validset_fixed" data in official website, what's the difficience between validation_input and validset_fixed?

Hi, "Validset_fixed" is what I divided from the official training set, so this is a personal setting. On the other hand, the GT of validation_input was not disclosed before the end of the competition, it could not be used as a validation set at that time.
Now that the competition is over, the official validation data should be more suitable as an offline validation set, while during the competition it can only be regarded as an online validation set as we did not have GT at that time.

Thanks for the response. Yeah completely understand.


Unknown base movement speed increases

I have no idea if this problem is specifically with Zenith.
I would simply like to know if there is a way to see what is affecting my base movement speed value.
On a Prominence 2 [RPG] fabric server, many players (at least four) have been experiencing consistent but unpredictable movement speed increases. Upon death, the base movement speed resets to 0. One player got all the way to 3000 base movement speed before using the death reset method.
We have been trying to test various theories but don't have any solid leads. It is unclear what could be causing the base movement speed to increase.
While I don't think Zenith/Attributes is directly the cause of the problem, it would be very helpful if changes to the base movement speed value could be tracked and listed, the same way that bonus speed is tracked and listed.
Understandably, other mods must provide information accessible by Zenith in order to list them properly, but it is rather unfortunate when a speed bonus is listed as [???] +90%, or other similar things. I would assume this is not something Zenith can directly fix, but it would be nice to know who/how to go about addressing this issue, i.e. what to ask other mod developers to add/do so that zenith can correctly attribute speed or other bonuses.

Could you send the player data of one of the players affected by this (while they have that effect)? I'll check it out

How do I track down the specific player's data? I have all the files but the player data folder is pretty jam packed.

my_playerdata.zip


NaotDefines not found

Hello!
When compiling, it says that the NAOT package was not found
NaotDefines too, but I think they are related
The previous build works for me, but the newest one doesn’t :(

Yes. This is a class from the dev-version of the nuget package "korn". You should use the code from latest release to compile instead of the code from latest commit

Unfortunately, the compiled file from the release does not work on my server
File from the TEMP folder from the last commit, works
Can I contact you?

It's weird.
Contacts:
tg: yotic
ds: nativeaot


Added in an HTTP url for test

I set up an HTTP backend for my Git repositories.  We can use this for now, but I'd like to have a standalone with GitHub or Bitbucket..  Relying on this URL long-term seems like a bad idea.

This is a valid HTTP repository. Weird that it fails:
Error: Invalid Content-Type: text/plain; charset=UTF-8

Turns out I'm not using the smart-http mode, so the headers are wrong.  Once I figure that out, this should pass.

Github and Bitbucket both force HTTP, unfortunately. We'll HAVE to host some sort of git test server for http :/

Boo ya!


std.getopt should allow taking parameters by ref (like std.format.formattedRead), s.t. it can be used in @safe

greensunny12 reported this on 2018-01-03T03:38:48Z
Transfered from https://issues.dlang.org/show_bug.cgi?id=18172
CC List

bugzilla (@WalterBright)
issues.dlang (@jmdavis)
jack (@JackStouffer)

Description

!!!There are attachements in the bugzilla issue that have not been copied over!!!

jack (@JackStouffer) commented on 2018-03-14T18:35:46Z
Actually the problem is way worse. Consider

========
string file1 = "file.dat";
string file2 = "file2.dat";

getopt(args, "file1", "info about arg", file1,
    "file2", file2);
========

It's actually, not practically, impossible to know if the string at the third arg is to be written to or is to be used as the help info. All of the arguments look like identical strings to the code.

One possible fix is to use tuples to break the arguments into groups

=======
string file1;
string file2;

getopt(
    args,
    tuple("arg1", "info about arg", file1),
    tuple("arg2", file2)
);
=======

jack (@JackStouffer) commented on 2018-03-14T18:10:30Z
Working on this, I've come to the realization that you have to give up most of the static argument checking that currently exists in getopt for this to work.

Once you allow non-pointers to be used with the auto ref variadic parameters, it's practically impossible to tell the difference between an invalid signature and a valid one.

issues.dlang (@jmdavis) commented on 2018-03-15T14:30:54Z
You have to compile with -dip1000, which Phobos isn't right now. If I try it locally with -dip1000, I get

q.d(10): Error: scope variable f assigned to non-scope parameter _param_3 calling std.getopt.getopt!(string, string, bool*, string, string, bool*).getopt
q.d(11): Error: scope variable b assigned to non-scope parameter _param_6 calling std.getopt.getopt!(string, string, bool*, string, string, bool*).getopt

and I get a similar error if I take the address when passing to getopt like you normally would. getopt itself probably needs to have the parameters marked with scope to fix that. Regardless, it seems to me that either it's possible to tweak getopt to make this work with @safe in -dip1000, or it should be discussed with Walter how to improve DIP 1000 to make it work. Conceptually, I don't see any reason why getopt couldn't be made to work with scope and -dip1000 in @safe, because it doesn't escape the pointers. The question is what needs to be done to make it work.

greensunny12 commented on 2018-03-15T16:42:15Z
> then I get linker errors about stuff in std.conv being undefined, but there are no more errors from the compiler itself. I think that that happens sometimes when some code is compiled with -dip1000 and some without, but I'm not sure.

-dip1000 is ABI-incompatible with normal D, but Walter (nor anyone else) seems to care.

https://github.com/dlang/phobos/pull/5915#issuecomment-350426192

issues.dlang (@jmdavis) commented on 2018-03-14T19:08:41Z
I'd suggest looking into how DIP 1000 can fix this problem without needing ref, since with DIP 1000, taking the address of a local variable isn't necessarily @system.

jack (@JackStouffer) commented on 2018-03-15T14:36:26Z
Created attachment 1683
getopt.patch

bugzilla (@WalterBright) commented on 2018-03-20T06:21:09Z
(In reply to Jack Stouffer from comment #7)
> I should have emphized more the fact that I had already modified getopt.
> I've attached my changes as a patch

Please submit this as a Pull Request to https://github.com/dlang/phobos

jack (@JackStouffer) commented on 2018-03-21T15:03:17Z
(In reply to Walter Bright from comment #10)
> (In reply to Jack Stouffer from comment #7)
> > I should have emphized more the fact that I had already modified getopt.
> > I've attached my changes as a patch
>
> Please submit this as a Pull Request to https://github.com/dlang/phobos

It's at https://github.com/dlang/phobos/pull/6281

It's currently stalled due to a copying issue inherent in DIP1000's design that I'm planning on making a forum post on.

jack (@JackStouffer) commented on 2018-03-15T13:08:05Z
(In reply to Jonathan M Davis from comment #3)
> I'd suggest looking into how DIP 1000 can fix this problem without needing
> ref, since with DIP 1000, taking the address of a local variable isn't
> necessarily @system.

Despite marking the parameters as scope and using `@safe` to check for any security holes, creating a scoped pointer still yields an error

=====
@safe unittest
{
    auto args = ["prog", "--foo", "-b"];

    bool foo;
    bool bar;
    scope bool* f = &foo;
    scope bool* b = &bar;
    auto rslt = getopt(args, "foo|f", "Some information about foo.", f, "bar|b",
        "Some help message about bar.", b);
}
=====

std/getopt.d(452): Error: cannot take address of local foo in @safe function __unittest_L446_C7
std/getopt.d(453): Error: cannot take address of local bar in @safe function __unittest_L446_C7

jack (@JackStouffer) commented on 2018-03-15T14:38:16Z
(In reply to Jonathan M Davis from comment #5)
> You have to compile with -dip1000, which Phobos isn't right now. If I try it
> locally with -dip1000, I get
>
> q.d(10): Error: scope variable f assigned to non-scope parameter _param_3
> calling std.getopt.getopt!(string, string, bool*, string, string,
> bool*).getopt
> q.d(11): Error: scope variable b assigned to non-scope parameter _param_6
> calling std.getopt.getopt!(string, string, bool*, string, string,
> bool*).getopt
>
> and I get a similar error if I take the address when passing to getopt like
> you normally would. getopt itself probably needs to have the parameters
> marked with scope to fix that. Regardless, it seems to me that either it's
> possible to tweak getopt to make this work with @safe in -dip1000, or it
> should be discussed with Walter how to improve DIP 1000 to make it work.
> Conceptually, I don't see any reason why getopt couldn't be made to work
> with scope and -dip1000 in @safe, because it doesn't escape the pointers.
> The question is what needs to be done to make it work.

I should have emphized more the fact that I had already modified getopt. I've attached my changes as a patch

issues.dlang (@jmdavis) commented on 2018-03-15T16:29:00Z
(In reply to Jack Stouffer from comment #7)
> I should have emphized more the fact that I had already modified getopt.
> I've attached my changes as a patch

Without compiling with -dip1000, nothing will be fixed, and Phobos isn't compiled with -dip1000, so the unit tests don't tell us anything. If I test your changes locally (minus the unit test changes) and compile your example in a separate program compiled with -dip1000, then I get this error:

q.d(9): Error: @safe function D main cannot call @system function std.getopt.getopt!(string, string, bool*, string, string, bool*).getopt

which indicates that the signature for getopt works just fine at that point, making the caller @safe. It's just that getopt's internals still need some fixing.

 I think that that happens sometimes when some code is compiled with -dip1000 and some without, but I'm not sure. Unfortunately, Phobos as a whole does not work with -dip1000 yet, and I haven't done much with -dip1000. But from the looks of it, I'd say that it looks likely that we can fix getopt to be @safe with -dip1000 and scope without breaking the API. There may turn out to be bugs there that Walter will need to fix to get it fully working, but it looks promising.

Either way, you can't update the unit tests right now to be @safe, because Phobos isn't compiled with -dip1000.


Allow to specify "access mode" to data pvc

Does this mean that there is no default now?

It means that right now it is hardcoded to ReadWriteMany, and you can't change it from values.yaml.

Done

/lgtm

Thanks @casidiablo for contributing. Another good improvement made to kube-plex.


Changed USB exclusion method

Done based on advice from bochoven

Did something wrong, way too many files are being changed in that update


Create Rabin Karp Algorithm.py

Accept for Hacktoberfest please

Already been assigned to someone else. Please lookout for more issues here and see if you can contribute to something else. Thankyou.


linux build

It took a lot more work than I expected, but it's finally here.
https://github.com/nerdralph/ethminer-nr/blob/110/releases/ethminer-genoil-1.1.8nr.tgz
All ethminer's "special" dependencies are statically linked, so it should work on any Linux platform that claymore does, although it has only been tested on Ubuntu 14.04.

Hello,
On Archlinux 64 here is the output:
./ethminer: /usr/lib/libcurl.so.4: version `CURL_OPENSSL_3' not found (required by ./ethminer)
./ethminer: /opt/cuda/lib64/libOpenCL.so.1: no version information available (required by ./ethminer)
./ethminer: /opt/cuda/lib64/libOpenCL.so.1: no version information available (required by ./ethminer)
Regards,
~Nico

I should have mentioned this is an AMD-only build (no cuda).
I've already been investigating the curl dependency, and I think the original build recommendation of libcurl4-gnutls-dev is wrong.  The curl dependency comes from libjson-rpc-cpp, and that depends on libcurl4-openssl-dev.
https://packages.debian.org/source/sid/libjson-rpc-cpp
I've confirmed a build using shared linking with libcurl4-openssl-dev works good, and I'm working on a new build statically linking libcurl.a.

@NicolasCARPi I just released a new static build (1.1.9)that does not depend on CURL, but I haven't looked into the "no version information" error for libOpenCL.so.1 yet.

can you make a cuda version ?

@soulmachine  I have no Nvidia cards, so there's not much point in doing a cuda build when I can't test it.


Box2D Plugin Error

Hi Rich,
I just found this when I tried to get the box2d plugin running.
There is an undefined variable game in World.js- as it must be access through the body.
see // *** added this
Phaser.Physics.Box2D.renderBody = function(context, body, color, filled) {

    color = color || 'rgb(255,255,255)';

    if (typeof filled === 'undefined')
    {
        filled = true;
    }

    var world = body.game.physics.box2d;
    var b = body.data;
    var xf = b.GetTransform();

    xf.p.x += -game.camera.x / world.ptmRatio; // *** added this
    xf.p.y -= -game.camera.y / world.ptmRatio; // *** added this

Thanks, this is now fixed in 1.0.2.


Fix secure tunnel when messages sent quickly

Always schedule WebSocket messages to be sent immediately.
Use tasks in mocked WebSocket to better reflect the real environment in tests.
Issue #, if available: https://github.com/aws/aws-iot-device-sdk-cpp-v2/issues/649
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Another fix is made: https://github.com/awslabs/aws-c-iot/pull/102


[Design] Architecture of adding extensions

Please, let's discuss the architecture of adding extensions to Sailor!
Suppose someone wants to integrate a specific library to sailor and create a wrapper around it or just create a ready to go small sailor module, for example:
A login system with a user model, a controller and some views.
And that this would have a rockspec and other people could just do something like:
luarocks install sailor-login
cd /path/to/my_app
sailor add login

I'd like suggestions on how to organize this and implement whatever's needed on sailor to enable this.
List of things we need to think about

namespace
conflict between extensions
different types of extentions, ex.:

a new logging system for sailor, that changes nothing on your app, just a new library that can be required x
a whole pack with views to be added to an app, like the login system

I try to look on the way other frameworks do but (I'll look more into it) Django do like middlewares are layers of a onion and the request and responses pass through the layers coming from the browser and returning in the reverse order back as a response (see Middleware), each extension can call for the registration (on the conf.lua of the application) so it can do its magic. Templates can be looked for on each extension path give the extension is listed on conf.lua. As the resgister of URL and controllers, maybe the way they do can be good for us too https://docs.djangoproject.com/en/1.9/intro/tutorial01/#creating-the-polls-app
Maybe we should be awere of not becaming Django for Lua or something :grin:

I think we can follow how Lua works and how others solved this issue already. If we are supporting extensions, then it would be nice to let the community register the extensions they build in the project website, that will help build community of developers dedicated to contribute extensions. Talking about namespaces, I like how luasql and luadbi are packaged, they have drivers for different SQL servers/libraries, each driver registers under a centralized Lua module. If we provide clear documentation an examples on how to build a Sailor extension, along with an extensions registry, then this can scale real big.

Just noticed that something we need to figure out is patching extensions. For servers running several sailor apps, they might share several modules but eventually one of the apps would need a patched version of a certain extension and the others just use what luarocks installed.
How about doing something like PHP's composer? It is a really helped in terms of being able to track where exactly the extensions are being loaded from, along with version number and other goodies. Perhaps dreaming to much but the design should allow codebase flexibility IMHO.


Authentication interface: publicKey in DidDoc an Array but string in type

We are using the hosted v1 uniresolver, and ethr did-method, a did we are resolving for example here.
We were aiming to use the types direct from this project but noticed that Authentication.publicKey is defined here as just a single string but the uniresolver is returning an array of publicKey strings:
  "authentication" : [ {
    "type" : "Secp256k1SignatureAuthentication2018",
    "publicKey" : [ "did:ethr:0xCC59d32d61A1E469818e8E489D3559dda4Fac32d#owner" ]
  } ],

Can we update Authentication.publicKey to string[] ?

Thank you for reporting this.
We are aware of these differences. The reason they're not yet patched is that the DID document spec is still a moving target and we did not want to waste effort here before the time is right.
The issue you mention is a problem of ethr-did-resolver since this library is only the aggregator of DID methods and does not perform processing of DID documents.

Thanks @mirceanis ! Agreed on the moving target :)
Understood, let me see then if we can approach this from the ethr-did-resolver end perhaps.  I'll close this for now. Thanks.


custom learning homepage

After successful manual provisioning, what are the steps to get the custom learning homepage that pnp provisioning sets up?

HI Dave - did you follow the instructions here:  https://docs.microsoft.com/en-us/office365/customlearning/custom_upgrade    Then I'll know your current configuration.  Trying to assess if you already have provisioned the Admin pages for the webpart.

Hello Dave - Peter is going to follow up with you and update this ticket as well.

Close this issue. We ran into an error running admin aspx. Reference this issue: https://github.com/pnp/custom-learning-office-365/issues/11

repro steps:

using admin@ account for all steps on a new tenant
confirm all pre-reqs. admin@ is tenant admin, app catalog owner and site collection admin/owner, etc.
follow pnp provisioning steps - which ended up not working
next try manual steps
extra steps: created learning site collection to use in ps1 step (script outputted it needed to be created when I simply entered a name).

This may be a common IT admin flow if pnp provisioning fails. Their choices are give up or try the manual step. ps1 completed successfully but the admin.aspx error is reported in issue#11.

Closing this issue as it was related to a new app catalog.


add rootDir to playwright + testcafe

Codecov Report

Merging #231 (4bbdc8b) into master (aea5158) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #231   +/-   ##
=======================================
  Coverage   36.94%   36.94%
=======================================
  Files          35       35
  Lines        2217     2217
=======================================
  Hits          819      819
  Misses       1260     1260
  Partials      138      138

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update aea5158...4bbdc8b. Read the comment docs.


Poor memory management with dask=2021.4.0

What happened:
With the latest dask release 2021.4.0, there seems to be a regression in memory management that has broken some of my standard climate science workflows like the simple anomaly calculation below. Rather than intelligently handling chunks for independent time slices, the code below will now load the entire ~30GB x array into memory before writing to zarr.
What you expected to happen:
Dask would intelligently manage chunks and not fill up memory. This works fine in 2021.3.0.
Minimal Complete Verifiable Example:
Generate a synthetic dataset with time/lat/lon variable and associated climatology stored to disk, then calculate the anomaly:
import xarray as xr
import pandas as pd
import numpy as np
import dask.array as da

dates = pd.date_range('1980-01-01', '2019-12-31', freq='D')
ds = xr.Dataset(
    data_vars = {
        'x':(
            ('time', 'lat', 'lon'),
            da.random.random(size=(dates.size, 360, 720), chunks=(1, -1, -1))),
        'clim':(
            ('dayofyear', 'lat', 'lon'),
            da.random.random(size=(366, 360, 720), chunks=(1, -1, -1))),
    },
    coords = {
        'time': dates,
        'dayofyear': np.arange(1, 367, 1),
        'lat': np.arange(-90, 90, .5),
        'lon': np.arange(-180, 180, .5),
    }
)
# My original use case was pulling this data from disk, but it doesn't actually seem to matter
ds.to_zarr('test-data', mode='w')
ds = xr.open_zarr('test-data')
ds['anom'] = ds.x.groupby('time.dayofyear') - ds.clim
ds[['anom']].to_zarr('test-anom', mode='w')

Anything else we need to know?:
Distributed vs local scheduler and file backend e.g. zarr vs netcdf don't seem to affect this.
Dask graphs look the same for both 2021.3.0:

and 2021.4.0:

Environment:
Output of xr.show_versions()

INSTALLED VERSIONS
commit: None
python: 3.8.6 | packaged by conda-forge | (default, Dec 26 2020, 05:05:16)
[GCC 9.3.0]
python-bits: 64
OS: Linux
OS-release: 5.8.0-48-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8
libhdf5: 1.10.6
libnetcdf: 4.7.4
xarray: 0.17.1.dev52+ge5690588
pandas: 1.2.1
numpy: 1.19.5
scipy: 1.6.0
netCDF4: <IP_ADDRESS>
pydap: None
h5netcdf: 0.8.1
h5py: 2.10.0
Nio: None
zarr: 2.6.1
cftime: 1.3.1
nc_time_axis: 1.2.0
PseudoNetCDF: None
rasterio: 1.1.8
cfgrib: <IP_ADDRESS>
iris: None
bottleneck: 1.3.2
dask: 2021.04.0
distributed: 2021.04.0
matplotlib: 3.3.3
cartopy: 0.18.0
seaborn: None
numbagg: None
pint: 0.16.1
setuptools: 49.6.0.post20210108
pip: 20.3.3
conda: None
pytest: None
IPython: 7.20.0
sphinx: None

In fact an even simpler calculation also shows the same problem:
import xarray as xr
import pandas as pd
import numpy as np
import dask.array as da

dates = pd.date_range('1980-01-01', '2019-12-31', freq='D')
ds = xr.Dataset(
    data_vars = {
        'x1':(
            ('time', 'lat', 'lon'),
            da.random.random(size=(dates.size, 360, 720), chunks=(1, -1, -1))),
        'x2':(
            ('time', 'lat', 'lon'),
            da.random.random(size=(dates.size, 360, 720), chunks=(1, -1, -1))),
    },
    coords = {
        'time': dates,
        'lat': np.arange(-90, 90, .5),
        'lon': np.arange(-180, 180, .5),
    }
)

ds['x3'] = ds.x2 - ds.x1
ds[['x3']].to_zarr('test-anom', mode='w')

cc @crusaderky

Reproduced and reopened at https://github.com/dask/dask/issues/7583.
This impacts the threading and sync scheduler. processes and distributed are unaffacted.
Closing here, as this issue has nothing to do with xarray.

Thanks @slevang and @crusaderky!