MySql, $query->batch() from more millions of record

I tried run yii\db\Query::batch() for big table and got memory leak. This happens because method  \yii\db\Command::queryInternal() run
$this->pdoStatement->execute();

I changed attribute PDO::MYSQL_ATTR_USE_BUFFERED_QUERY -> false and it helps me.
I think, we should add description about this feature into documentation.

Could you specify where did you set the attribute?
I tried it in config and runtime(Yii::$app->db->pdo->setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY,false)), and got error 2014.

As I understand this problem, you did not use 'fetch' or 'execute' method after creation request.
Also you can try the next solutions before I will see my code in the evening:

$command->reset()
read issues 3695

Nevertheless, you can attach code and error stack to better understand your problem.
Other way, you can use setAttribute(PDO::MYSQL_ATTR_USE_BUFFERED_QUERY,false) for mehod batch() and after that return back attribute to true for other cases

Unfortunately, I cannot find my codes with this attribute, maybe it was a previous project. But you may use advice indicated above or show your code and we will try to  help.

I also have this problem and the above fix didn't work for me. Any other suggestions?
Yii 2.0.6 PHP 5.6.10 MySQL 5.6.22

unable to reproduce the issue, see #11152

https://github.com/yiisoft/yii2/issues/9269#issuecomment-250739252

Any Update on this issue?

I can reproduce the issue, working on a fix.
What is the opinion of the yii team concerning configurability?
One could argue that it never makes sense to use batch in combination with buffered mode and that the default mode should be unbuffered.
Alternatively we could make the default unchanged and just add a parameter that allows explicitly disabling buffered mode (meaning it won't break BC).
Currently testing with unbuffered mode, there are some issues:

We need to guarantee the database schema is known (otherwise the AR layer will try to do a SHOW FULL COLUMNS FROM xx).
Any logic inside the batch processing cannot run queries.

I've just made a proof of concept that circumvents this issue, I'd love some feedback on it:
When running an unbuffered query, clone the database connection and use the cloned connection for the unbuffered query.
The advantage of this approach is that we can guarantee there are no issues with other queries (since the cloned connection is not accessible outside the scope of the batch query object(s)).
The downside is that it costs an extra connection (meaning connection set-up delay). Also it could give strange behavior when combined with transactions.

Also, unbuffered mode will not work with relations that are not loaded through explicit joins since these require separate queries.
We can conclude several things:

This is complex (should this be removed from 2.0.12?)
If we want truly functional stuff we need to use multiple database connections.
The simple approach will work in a number of specific cases only so it should not be the default behavior but instead be explicitly specified.
Before any specific solution can be created the Yii team needs to decide what direction they want to go in since there is no simple "just make it work in all cases" solution.

It doesn't make sense to batch in buffered mode. That is for sure. Issues with unbuffered mode are significant though...

Does it even make sense to have batch() as a function? I always assumed it would actually get the results one batch at a time (reducing delays caused by roundtrips), but this does not seem the case.
Using ->each() is just as efficient at the moment..
We could create a separate unbufferedBatch that comes with appropriate warnings (I am willing to implement it), that just clones the connection and uses that.
Not sure this needs to be in the core though, it could just as well be released as a separate yii2-big-data extension.

Probably not considering the difficulties.

I'd deprecate batch() altogether...

@klimov-paul, @cebe, @SilverFire, @arogachev opinions?

One thing to consider is  batch() in https://github.com/yiisoft/yii2-elasticsearch. @beowulfenator would you please check if batch() is redundant in ES extension or not?

In the ES extension batch() works great, and it is in fact the only way to get large result sets because of the way ES results are fetched. (In ES query API deep pagination is extremely costly and batch() and each() use scan/scroll API).
As for MySQL, I routinely use each() and batch() to fetch huge amounts of data. My typical use case is as follows:
$unbufferedDb = new \yii\db\Connection([
    'dsn' => Yii::$app->db->dsn,
    'username' => Yii::$app->db->username,
    'password' => Yii::$app->db->password,
    'charset' => Yii::$app->db->charset,
]);
$unbufferedDb->open();
$unbufferedDb->pdo->setAttribute(\PDO::MYSQL_ATTR_USE_BUFFERED_QUERY, false);

foreach ($query->batch(1000, $unbufferedDb) as $batch) {
    //...
}

To me batch() and each() are useful and interchangeable in the sense that sometimes I need to iterate over entries, and sometimes over batches of entries. When I iterate over batches, I tend to print a dot every time a batch is processed to give users a sense of progress.
I believe you should not deprecate these methods, just explain in the docs that they will fail if the result set is too large for MySQL's internal buffer.

@beowulfenator If you use unbuffered mode do you manually load the database schema before querying the models?
Otherwise you'll run into problems.
Also the current batch without unbuffered mode basically just eats your servers memory while ignoring the PHP memory limit.
If we choose not to deprecate batch in the RDBMS connections it could make sense to create a better default implementation.
For example:

Create a new connection to do the batch query in unbuffered mode.
Prevent the existing connection from doing transactions (transactions would create different views for the 2 connections).

As said before these could also be implemented in a plugin. In that case the functions in the default implementation could just be left unchanged (and in the future throw an exception pointing to the fact that an extension is needed).
Note that technically the functionality works just fine as it is now, it just offers no real benefits over ->all().

@beowulfenator are there cases when you prefer batch() to each()?

We could support that via a different construct, something like an iterator buffer.
Something like this:

@samdark I use batch to indicate progress:
foreach ($query->batch(1000, $unbufferedDb) as $batch) {
    //...
    foreach ($batch as $model) {
        //...
    }
    echo '.';
}

@SamMousa I've never had any problems with loading the schema because I'm using a separate unbuffered connection for the big query.
Perhaps I misunderstand the problem, but to me it seems that on the PHP side batch() and each() work exactly as intended. We execute a SQL statement, get an iterator that points to the result set, and proceed with reading the data. So if you specify batch size of 1000 and PHP doesn't have enough RAM to load 1000 models, that's not really an implementation issue. This is distinctly different from all() which will load all models into RAM, and not just a specified chunk.
The problem with batch() in MySQL is on the server side, because MySQL tries to load the whole result set into its buffer. It may or may not be true for other DBMS. As far as I know, there is no way to query a very large result set in MySQL with a buffered query. But this is a MySQL-specific caveat that should be described in the docs. There's no point deprecating batch() because of it.

@beowulfenator No, it's a client side buffer. The MySQL driver will load all data locally regardless if your batching (in buffered mode).
The only difference is that the drivers memory use does not count towards the php memory_limit setting. This actually makes the problem bigger since it allows PHP to use memory until the process gets killed by the OS.
So with a batch size of 1 your PHP process still can crash.
My point is, if your data set fits in memory there is no reason for "batching it", as opposed to copy it to PHP directly (using all()). If it doesnt fit in memory then batching doesn't help.
Regarding deprecation, I agree that that is not the right solution since it is part of the contract and extensions implement it as well. There are alternatives though, such as not implementing it. As it stands now, it doesnt actually work well if you do proper memory usage measurements.
Regarding your specific use case, you could just as well do:
foreach($q->all() as $i => $m) {
    ...
    $i % 1000 || echo '.';
}

Of course that's not really relevant, there are plenty of solution to that problem.
In my opinion batch and each are for iterating over large results sets in an efficient manner. The default implementation for RDBMSes doesn't really do that.
What if we had a final scenario where it went like this (not sure about the route to get there):

You use ->batch().
You get an exception telling you to install yii2-big-data
You install it.
You no longer get the exception, batched queries are automatically executed in a different connection, unbuffered. During the batch execution, transactions on the main connections are disabled.

@SamMousa OK, so then there is also a client-side buffer. Because in my case selecting with an buffered query several million entries from a 10 gigabyte table in hopes of iterating them sequentially brings down the MySQL server first! (The server in question has 1 gigabyte of RAM).
Are you absolutely certain that PDO driver really loads the whole dataset into RAM locally if query is buffered?

I would argue against automatically disabling the transactions on the main connection and automatically creating unbuffered connections.
Transactions only create issues when they deal with the same table. This is not always the case. Automatically creating an unbuffered connection for every batch query is wasteful because then this connection can not be reused.
I think the problem can be formalized as follows: batch() and each() make no sense in buffered connections. How about this approach: in the core framework in the BatchQueryResult class we check whether its db property is an unbuffered connection, and if it's not, we throw a warning.
That way the user can either manually provide an unbuffered connection or use an extension.

Transactions only create issues when they deal with the same table. This is not always the case.

It might not always be the case but it can lead to hard to debug problems, think things like foreign key constraints and such, we could make it optional though (as an argument to batch).

Automatically creating an unbuffered connection for every batch query is wasteful because then this connection can not be reused.
When doing large queries where this kind of batching is required connection setup is minimal compared to total time spent, if you iterate over 1 million records the setup time of 1 vs 2 connection is irrelevant.

I think the problem can be formalized as follows: batch() and each() make no sense in buffered connections. How about this approach: in the core framework in the BatchQueryResult class we check whether its db property is an unbuffered connection, and if it's not, we throw a warning.
That could work; throwing a warning I assume means an exception?

Just checked, you're right, in buffered mode batching is useless.

Not just useless, it could even be broken depending on the cache containing schema information.

We need to guarantee the database schema is known (otherwise the AR layer will try to do a SHOW FULL COLUMNS FROM xx).

@samdark I think because of this schema query this is actually a bug, not just a documentation issue.
I've just tried to do a batch query with ActiveRecord from a console command where I've set MYSQL_ATTR_USE_BUFFERED_QUERY, and it breaks with a "Cannot execute queries while other unbuffered queries are active." error.  So basically you can not use large batch queries with AR (unless the schema is accidentally cached - which we can not rely on).

@mikehaertl Just make two connections, and use the one that's unbuffered for batch.

@beowulfenator Thanks, but I think creating an empty AR instance is easier:
$dummy = new SomeModel(['id' => 0]);
SomeModel::getDb()->setAttribute(\PDO:MYSQL_ATTR_USE_BUFFERED_QUERY, false);
foreach (SomeModel::find()->each() as $model) {
...
}

Anyway, both are just workarounds. I think a proper fix should check in batch() and each() whether it's an active query, and trigger a schema query first.
Adding a second DB connection is something that the framework should not decide on it's own. There may be side effects in certain scenarios. As a developer I want to know, how many DB connections my app opens.

@mikehaertl The problem is that there is no reliable way to determine the required schemas. (There is an upper bound defined by any all relations that are selected for greedy loading).
This will get messy very quickly as soon as relations are involved since Yii by default uses a separate query for loading related records.
There is no solution that handles ALL cases adequately.

@SamMousa Hmm, i see. As there's no silver bullet we should at least make this very clear in the documenation. To sum up what's missing there right now:

When using batch() or each() on MySQL Queries, you have to set PDO::MYSQL_ATTR_USE_BUFFERED_QUERY to false
Additionally if you have an ActiveQuery, setting the above is not enough. You have to let Yii load the schemas first, e.g. by calling MyModel::getTableSchema() on all AR classes involved in the query.
Add a note, that during the batch no other queries can be performed on the DB due to the above setting

That only applies if you have a dataset that does not fit in memory. (I personally do all iteration with batch and each whether the dataset is large or small)
It should be possible for for Yii to load all schemas before executing the active query, would need to take a closer look at this. If we refactor that thinks would improve:

A query starts by getting all required schemas, this needs to be done regardless and is generally faster than the actual query.
Any errors reading the schema result in faster dying. Instead of first doing a slow query and then getting the schema, we get the schema first.

That would resolve the "simple" cases where you just set unbuffered queries to true..

That only applies if you have a dataset that does not fit in memory.

I'd argue that this is the whole point of doing a batch() query in the first place. You say you always use it - but for no reason. Or what is the benefit for you?

It expresses more explicitly what I need to do.
I need to iterate over results, not generate an array and iterate over the array.
Also there is the case where PHP has a memory limit, say 128M, but the dataset is larger 150M. Since it is very likely 150M will fit in server memory but within the PHP memory limit, each will work here.
The drivers' memory does not count towards PHPs memory limit.

@mikehaertl Buffered queries also require substantial amount of RAM on the server. The reason I started using unbuffered queries is the need to iterate over the whole 10 Gb table on a server with 1 Gb RAM. SELECT * FROM my_table would simply bring the server down in buffered mode.

I think that the simple and Ugly way to solve this is to add auto generated offset/limit parameters to the query，it will decrease the size of result, which won't cause the memory issue.
every time when batch() is called, the offset/limit parameters  should be adjust to add the size of batch().
the problem is that maybe an  order by primary key or something else   should be done to keep the order of the dataset

@ihipop That's not a bad idea, actually. Not really ugly, and it is essentially what we do in DataProvider with pagination. @samdark, what do you think about using offset/limit to implement batch() and each()?

It is a really bad idea. Test it, the performance will be terrible.
It will also not provide a consistent view of the data unless implemented
with severe locking effectively blocking all writes to the table(s) in
question.
On Apr 21, 2017 8:33 AM, "Konstantin Sirotkin"<EMAIL_ADDRESS>wrote:

@ihipop https://github.com/ihipop That's not a bad idea, actually. Not
really ugly, and it is essentially what we do in DataProvider with
pagination. @samdark https://github.com/samdark, what do you think
about using offset/limit to implement batch() and each()?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/yiisoft/yii2/issues/8420#issuecomment-296091844, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAhYzad6PguOPc2dpXXsix-jRZDSloN7ks5ryE2ogaJpZM4EamFy
.

OK.

@SamMousa the consistent view of the data  is as mentioned

the problem is that maybe an order by primary key or something else should be done to keep the order of the dataset

@ihipop It is not.
Let me explain all my arguments against your method.

It is slow, for every page the whole query needs to be executed only to just take a few records from them.
It is inconsistent, in any scenario where the keys is not an auto incremented integer. If I have keys 1 and 15 what happens if I insert 2 during your iteration, most likely it'll be skipped or cause some other entry to be returned twice.
Sorting by primary key sounds simple, but for every join a sort by primary key of that table will need to be added to guarantee consistent ordering, also it breaks any other order clauses.

An alternative, if you really want one is this, it is still far from optimal and only works for integer keys. We use this method in production for slow running jobs. And it only provides an upper bound for the numbe r of results.

Get min and max of primary key.
Divide the range so that the maximum number of records it can return, including joined records fits in memory.
Iterate over the query using a condition like: where id > 10 and <= 20.

This approach is a lot faster but can still miss records inserted during processing. It will never return records twice.
As I've said before, no solution is valid for all cases and therefore we should not just implement something. Instead implement it yourself as a custom function or behavior in your query class.
@beowulfenator you mention buffered queries require RAM on the server; this is not clear to me. Do you mean the PHP or DB server?

@SamMousa I mean the DB server. Executing a buffered MySQL query with the result set that is too large to fit into MySQL server's RAM brings down the MySQL server. At least that's how it is for me.

Hmm, that seems strange to me, since the buffering is happening in the PDO driver as far as I know

Finally，I write a new static method to get a unBuffered Query PDO $db instance
public static function getUnbufferedMysqlDb($db, $db_identifier = null)
    {
        if (is_string($db)) {
            if (empty($db_identifier)) {
                $db_identifier = $db;
            }
            $db = Yii::$app->get($db);
        }
        if (!($db instanceof \yii\db\Connection) || !strstr($db->getDriverName(), 'mysql')) {
            throw  new InvalidParamException('Not a Mysql Component');
        };
        if (empty($db_identifier)) {
            $db_identifier = md5(sprintf("%s%s%s%s", $db->dsn, $db->username, $db->password,
                var_export($db->attributes, true)));
        }
        $db_identifier = 'unbuffered_' . $db_identifier;
        if (!Yii::$app->has($db_identifier)) {
            $_unbuffered_db = Yii::$app->getComponents()['db'];//clone the configuration
            $_unbuffered_db['attributes'][\PDO::MYSQL_ATTR_USE_BUFFERED_QUERY] = false;
            Yii::$app->setComponents([$db_identifier => $_unbuffered_db]);
        }

        return Yii::$app->get($db_identifier);
    }

When Using ,pass it to the $db parameters of batch()/each()
It Will establish a new Connection to mysql which PDO::MYSQL_ATTR_USE_BUFFERED_QUERY  is false
,and Only Used for  batch()/each(),Other Query could be done as normal During batch()/each()
$unbuffered_db = SomeClass::getUnbufferedMysqlDb($Model::getDb());
foreach($activeQuery->batch($unbuffered_db) as $foobar){
        #foobars
}

Please conform that this problem is specific to

PHP 7
mysqlnd
\PDO::MYSQL_ATTR_USE_BUFFERED_QUERY === true

Nevermind me. That's clearly wrong.
If mode is buffered (\PDO::MYSQL_ATTR_USE_BUFFERED_QUERY === true) then you can run out of memory.
If PHP 7 and mysqlnd then the OOM killer gets you, otherwise PHP runs out of memory and stops the script.

It may be worth keeping this feature but the guide is misleading

When working with large amounts of data, methods such as yii\db\Query::all() are not suitable because they require loading all data into the memory. To keep the memory requirement low, Yii provides the so-called batch query support. A batch query makes use of the data cursor and fetches data in batches.

And then proceeds to demonstrate the methods that fail to do this.
There needs to be a warning about memory exhaustion in PDO's buffered mode and advice how to avoid it.
Some elaboration on "makes use of the data cursor" would also be welcome.

@tom-- You're welcome to submit your PR for the documentation.

I believe this is the first time I have seen "PRs welcome" deployed in the Yii project.
I did not submit my PR because I do not have one. I was unable to author one because I don't understand the mechanisms Yii and its use of PDO well enough to explain the problem and advise on workarounds. I am still struggling to debug my own problem with it and the difficulty comes mostly from this lack of knowledge.
More expert contributors than myself have expressed differing opinion on what to do about this bug  including scrap it, fix it, and keep it. My comment is simply that unless we scrap it pretty soon, the guide needs a fix. I also wanted to revive the discussion. I regret that now.

@tom--  https://github.com/yiisoft/yii2/issues/8420#issuecomment-301423833  just do  as this.
it will solve your problems ,even  not the best one,but it would work

@ihipop I tried your workaround now as my solution (i.e. "hack") did not work with with() queries. Unfortunately I found that your solution suffers from the same problem. Say you have "product has many variations". And your batch query is for products using ->with('variations'). Yii then performs a 2nd query for the related Variation records, using the main buffered db connection and an product_id IN ( ... ) condition with all the product ids.
Unfortunately it seesm like the list of these product ids is not reset for each batch. It gets bigger and bigger so that the same related records are queried again and again - plus the new ids on top.
So I still end up with an out of memory error.

As a 4 year developer of Yii2, I feel surprised that the each() method can exhaust memory until today, that I query on a table with a column of MEDIUMTEXT.
To sum up, basically, there are three methods to handle iterating on each record in a query set problem:

the current one, to leave the dirty work to pdo extension, however, pdo will exhaust memory when there are too much records or there are one column with too much data(MEDIUMTEXT for example)
to iterate on query with offset/limit. Pros is simplicity, cons are i) pagination become slow when offset increases, and DB can exhaust memory, ii) records will be queried twice when record insert during the query.
to iterate on query with primary key range. Pros are i) it is faster than offset/limit and will never exhaust memory, ii) can avoid record queried twice, cons are i) B tree index of integer column is required(hash index doesn't support range query) ii) the result is unexpected in some requirements if we don't need the record inserted after query
to query twice, first query all the primary keys(can be implemented with current batch mechanism, leave it to pdo), then iterate on query with IN condition. Pros are i) the speed is acceptable, considering ActiveQuery::with() is implemented with IN condition, ii) can apply to any table with any kind of index, iii) Record will not repeat. Cons are i) can exhaust memory when records exceed tens of million

One solution is to add more parameters to batch() and each(), the first is mode, indicating the implementation we need to use, another is specific parameters for the implementation, such as the index column.
But these heterogeneous parameters will pollute the batch() and each() function.
Another solution is to add several services for each implementation. The usage is like:
new RangeQueryIterator(User::find()->where(...), $batchSize)->each())
It can decouple ActiveQuery from Iterator, which is preferred.

Thanks for ideas. @terabytesoftw should be interesting for Yii 3.

An example of RangeQueryIterator:
<?php

namespace ZhiShiQ\Yii\Db;

use yii\db\ActiveQuery;
use yii\db\BatchQueryResult;

class RangeQueryIterator implements \Iterator
{
    private $query;
    private $pk;
    /**
     * @var BatchQueryResult
     */
    private BatchQueryResult $batchIterator;

    private $_batch;
    private $_key;
    private $_value;

    public function __construct(ActiveQuery $query, $batchSize = 100)
    {
        $this->query = $query;
        $this->pk = ($query->modelClass)::primaryKey();
        $this->batchIterator = (clone $query)->select($this->pk)->batch($batchSize);
    }

    /**
     * Destructor.
     */
    public function __destruct()
    {
        $this->reset();
    }

    public function reset()
    {
        $this->_batch = null;
        $this->_value = null;
        $this->_key = null;
        $this->batchIterator->reset();
    }

    public function rewind()
    {
        $this->reset();
        $this->next();
    }

    public function next()
    {
        if ($this->_batch === null || next($this->_batch) === false) {
            $this->batchIterator->next();

            $keyCols = array_map(function ($val) {
                $arr = [];
                foreach ($this->pk as $key) {
                    $arr[$key] = $val[$key];
                }
                return $arr;
            }, $this->batchIterator->current());

            $this->_batch = (clone $this->query)->andWhere(['IN', $this->pk, $keyCols])->all();
            reset($this->_batch);
        }

        $this->_value = current($this->_batch);
        if ($this->query->indexBy !== null) {
            $this->_key = key($this->_batch);
        } elseif (key($this->_batch) !== null) {
            $this->_key = $this->_key === null ? 0 : $this->_key + 1;
        } else {
            $this->_key = null;
        }
    }

    public function key()
    {
        return $this->_key;
    }

    public function current()
    {
        return $this->_value;
    }

    public function valid()
    {
        return !empty($this->_batch);
    }
}

Usage
foreach(new RangeQueryIterator(User::find()->where(...), $batchSize)) as $user) {
    //xxx
}

As a 6 year developer of Yii2, I feel surprised that the each() method can exhaust memory until today, that I query on a table with a column of MEDIUMTEXT.
To sum up, basically, there are three methods to handle iterating on each record in a query set problem:

the current one, to leave the dirty work to pdo extension, however, pdo will exhaust memory when there are too much records or there are one column with too much data(MEDIUMTEXT for example)
to iterate on query with offset/limit. Pros is simplicity, cons are i) pagination become slow when offset increases, and DB can exhaust memory, ii) records will be queried twice when record insert during the query, iii) can't apply to query which already has offset/limit
to iterate on query with primary key range. Pros are i) it is faster than offset/limit and will never exhaust memory, ii) can avoid record queried twice, cons are i) B tree index of integer column is required(hash index doesn't support range query) ii) the result is unexpected in some requirements if we don't need the record inserted after query,  iii) can't apply to query which already has offset/limit
to query twice, first query all the primary keys(can be implemented with current batch mechanism, leave it to pdo), then iterate on query with IN condition. Pros are i) the speed is acceptable, considering ActiveQuery::with() is implemented with IN condition, ii) can apply to any table with any kind of index, iii) Record will not repeat,  iii) can apply to query which already has offset/limit. Cons are i) can exhaust memory when records exceed tens of million

One solution is to add more parameters to batch() and each(), the first is mode, indicating the implementation we need to use, another is specific parameters for the implementation, such as the index column.
But these heterogeneous parameters will pollute the batch() and each() function.
Another solution is to add several services for each implementation. The usage is like: new RangeQueryIterator(User::find()->where(...), $batchSize))
It can decouple ActiveQuery from Iterator, which is preferred.

Same surprice for me after 5 years of frekquent usage of each() and batch() to carefully optimize memmory usage, but no optimization happened 😠
It's for me another reason to leave this "old" world of PHP + MySQL in favor of NodeJS + MongoDB.

Can someone lock this topic. Developers moving away from php because they don't understand inner workings of stuff is really not a relevant issue. There's plenty of other languages in which those people can be bad developers. Commenting on a closed issue that's several years old just to complain about taking 6 years to understand something seems counterproductive.

Can someone lock this topic. Developers moving away from php because they don't understand inner workings of stuff is really not a relevant issue. There's plenty of other languages in which those people can be bad developers. Commenting on a closed issue that's several years old just to complain about taking 6 years to understand something seems counterproductive.

I don't want create any flame war here, but I need to react.
As I understand from the comments here the additional explanation about the each() and batch() was added to guide in 2017. I study Yii before, so there was no warning in documentation that  each() and batch() can work different with MySQL and not as expected.
@SamMousa, your comment is simply arrogant (and counterproductive).
Thank you @lubobill1990 for providing solution 👍

Yes, thanks for the input from everyone, let's not vent our frustration like that. Please move this discussion to the forum if necessary.


Adds rows / keys / values editing to KeyValue form component

When we disable key editing using ->disableEditingKeys(), values cannot be edited too.
This PR introduces two new methods to handle editing:
->disableEditingValues() - Disables only values editing.
->disableEditingRows() - Disables both key and value editing.

Hey, surely disableEditingRows() is just disabled()?

Hey, surely disableEditingRows() is just disabled()?

Done

Thanks!


Modify Windows startup command from start to run

Applies enhancement as proposed in BZ-62392 - https://bz.apache.org/bugzilla/show_bug.cgi?id=62392
It is a very simple fix so might be easier to apply directly rather than accepting this patch.
If approved, I can probably apply it directly myself in SVN, which will give me some practice with that.

Closing per comments in the BZ ticket


Better way to install docker in tests

Currently we are installing docker on top of an ubuntu base image to run the unit tests for various chain compatibility tests. It would be ideal if we could avoid that step by having an image with it preinstalled.

turns out docker is already installed on the stock ubuntu image used by GH actions. Closing this issue as not actionable.


Heapster 1.2.0 redirect error

Looks like heapster 1.2.0 does not correctly follow the url if it does end with a  /
Redirect does not work correctly

Hi all,
looks like with Kubernetes 1.4.6 (tested version) the request to the kubelet is broken (at least with TLS/https). Looks like heapster is requesting /stats which is redirected via 503 to /stats/ from kubelet (works verified with curl -L) but heapster does not follow the redirect and fails.
I would need this fixed as we have a dependency to kubernetes 1.4.6 here :-(
Thanks!

Just to add it, the parameters we are using:
        - /heapster
        - --source=kubernetes.summary_api:''?kubeletPort=10250&kubeletHttps=true&insecure=true&auth=true
        - --sink=influxdb:http://monitoring-influxdb:8086
        - --tls_cert=/opt/kube/certs/localworker.pem
        - --tls_key=/opt/kube/certs/localworker-key.pem
        - --tls_client_ca=/opt/kube/certs/ca.pem

@thomasfricke did it work with Heapster 1.1?
@timstclair has anything changed in Kubelet which may have impact to this?

Checking...

There are some changes to Kubelet TLS auth in 1.5, but I can't think of anything that would have affected 1.4

Hi all, as far as I am able to see, it looks working with heapster 1.1.0, but not with 1.2.0.
Any idea?

@jimmidyson @mwringe @burmanm @DirectXMan12 any ideas?

We are using 1.2 in OpenShift (based on Kubernetes 1.4), but with kubernetes source and not the kubernetes.summary_api.
You can see our option here: https://github.com/openshift/origin-metrics/blob/master/deployer/templates/heapster.yaml#L85
Where url we use is "https://kubernetes.default.svc:443"
If you use the kubernetes source, does this work for you?


strange work / etext pattern

I'm not sure the migration of https://www.tbrc.org/xmldoc?rid=W4JW162 is handled properly, it's migrated as an expression of W1KG4884, which is a bit odd... and there seems to be no link to a UT file, nor any link from a UT file to this work... so what's this record?

The case of W4JW162 appears to be the remnants of an experiment to associate a Work record with an Etext work; hence, the parent pointing to W1KG4884 which displays w/ a Search eTexts tab owing to the convention that we replaced the W by UT and check for the existence of the resulting RID in the /db/eTextsChunked/*/ collections.
There are some bits of information in the W and MW records such as the Edition Statement:

computer input under the direction of dpal brtsegs bod yig dpe rnying zhib 'jug khang for the purpose of printing; processed into Unicode Tibetan XML by TBRC

and the license indication (c) and fair use for the eTexts but there's no logic in the tbrc.org system to look for this information. Other information has been duplicated from W1KG4884.
I've collected some info from tbrc.org to help figure out what all we have in this area. First here is a list of all of the 516 distinct Etext works:
ETEXTS-WorkRids.txt
A distinguishing characteristic of W4JW162 is that it contains a <w:info/> element like:
<w:info nodeType="unicodeText"/>

Here's a list of the 201 Works that claim to be "unicodeText"s:
ETEXTS-AllInfoUnicode-WorkRids.txt
86 of these are contained in the list of Etexts:
ETEXTS-nfoUnicode-WorkRids.txt
The remaining 115 are not directly Etexts:
ETEXTS-InfoUnicode-NoEtext-WorkRids.txt
Of these 115 there are 78 that have
<w:info nodeType="unicodeText" parent="Wxxx"/>

where the @parent is an Etext:
ETEXTS-InfoUnicode-NoEtext-WithParent-WorkRids.txt
that leaves 37 Works that are neither directly Etexts nor that have @parents that are Etexts:
ETEXTS-InfoUnicode-NoEtext-WithParentNotEtext-WorkRids.txt
I suppose we now need to mull about what to do. Maybe see what @karmagongde thinks.

Ignoring the potential harvest of information in these records, my first intuition if we follow the current model would be to withdraw the 115 works that are not etexts. what do you think?
Another solution would be to slightly change the bibliographical model (or rather, our use of it), I'll make a proposal in that direction but that's sort of orthogonal...

I might be helpful for @karmagongde to take a look at the 115 works in ETEXTS-InfoUnicode-NoEtext-WorkRids.txt and see if there is information that should be retained or otherwise dealt with.


To numeric enhance

This is a test PR copy of #13425 to test travis - do not merge

[Current coverage][cc-pull] is 84.34%

Merging [#13512][cc-pull] into [master][cc-base-branch] will increase coverage by <.01%

@@             master     #13512   diff @@
==========================================
  Files           138        138
  Lines         51107      51113     +6
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits          43103      43109     +6
  Misses         8004       8004
  Partials          0          0

Powered by Codecov. Last updated by [bf4786a...ac37bd4][cc-compare]
[cc-base-branch]: https://codecov.io/gh/pydata/pandas/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/pydata/pandas/compare/bf4786a15f51d14ccf1318e44ae080868d8595ee...ac37bd45fd35a5c743d9e76fd4e0b446870ce66b
[cc-pull]: https://codecov.io/gh/pydata/pandas/pull/13512?src=pr


python312Packages.soundcard init at 0.4.3

Description of changes
https://github.com/bastibe/SoundCard
A python package to interact with your soundcard.
Things done

Built on platform(s)

[X] x86_64-linux
[X] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[X] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[X] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[X] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

fore-push: update pname to all lowercase + use fetchFromGitHub

The python312Packages name is case-sensitive, so please update it to use lowercase starting p

A final nit would be the ordering of the top attrset:
I'd really recommend putting the python dependencies next to each other and putting lib above everything.

I have just sorted the dependencies.
The way you do it sounds good to me at first. I'm still new to the nix environment and would be grateful for any feedback.
I've learned a lot from your review, thank you very much!

The package itself LGTM

What happens now with the MR? Can you merge it directly? Is there anything else I need to do?

Well, I don't have any commit access to the repo, so you'll probably have to wait for someone who does to comes along.
The waiting time can be improved a bit by submitting this PR into some other place, such as
the "PRs already reviewed" thread on the NixOS discourse (https://discourse.nixos.org/t/prs-already-reviewed/2617/1599)
or maybe the Matrix chat for Review Requests (https://matrix.to/#/#review-requests:nixos.org)
I'll probably post it to "PRs already reviewed" after I do some tests on a Mac emulator later today.

Force push: rebase to latest master and fix merge conflict in maintainers/maintainer-list.nix


Automatically deploy most recent commit on main to development.scrumlr.fra.ics.inovex.io

I think it would be cool if the most recent commit on the main branch would automatically be deployed to https://development.scrumlr.fra.ics.inovex.io/. This would allow us to test the latest commits before, for example, we publish a new release.

related: https://github.com/inovex/scrumlr.io/issues/2882


[e2e]: generic workflow_dispatch branch1 default slsa3

Repo: https://github.com/slsa-framework/example-package/tree/branch1
Run: https://github.com/slsa-framework/example-package/actions/runs/8274821404
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.generic.workflow_dispatch.branch1.default.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.generic.workflow_dispatch.branch1.default.slsa3.yml
Trigger: workflow_dispatch
Branch: branch1
Date: Thu Mar 14 03:06:32 UTC 2024

Repo: https://github.com/slsa-framework/example-package/tree/branch1
Run: https://github.com/slsa-framework/example-package/actions/runs/8290697689
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.generic.workflow_dispatch.branch1.default.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.generic.workflow_dispatch.branch1.default.slsa3.yml
Trigger: workflow_dispatch
Branch: branch1
Date: Fri Mar 15 03:07:37 UTC 2024

Repo: https://github.com/slsa-framework/example-package/tree/branch1
Run: https://github.com/slsa-framework/example-package/actions/runs/8304868790
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.generic.workflow_dispatch.branch1.default.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.generic.workflow_dispatch.branch1.default.slsa3.yml
Trigger: workflow_dispatch
Branch: branch1
Date: Sat Mar 16 03:06:11 UTC 2024

Repo: https://github.com/slsa-framework/example-package/tree/branch1
Run: https://github.com/slsa-framework/example-package/actions/runs/8312517827
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.generic.workflow_dispatch.branch1.default.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.generic.workflow_dispatch.branch1.default.slsa3.yml
Trigger: workflow_dispatch
Branch: branch1
Date: Sun Mar 17 03:06:35 UTC 2024

Repo: https://github.com/slsa-framework/example-package/tree/branch1
Run: https://github.com/slsa-framework/example-package/actions/runs/8320661247
Workflow file: https://github.com/slsa-framework/example-package/tree/main/.github/workflows/e2e.generic.workflow_dispatch.branch1.default.slsa3.yml
Workflow runs: https://github.com/slsa-framework/example-package/actions/workflows/e2e.generic.workflow_dispatch.branch1.default.slsa3.yml
Trigger: workflow_dispatch
Branch: branch1
Date: Mon Mar 18 03:16:52 UTC 2024
Tests are passing now. Closing this issue.


fix: remove PostChargingNotificationResponse in PostChargingNotification

The openapi defines these two return parameters. So these should not be removed.


core: second in flight command fix

In order to prevent too many commands from clogging the queue, we remove them if they look identical.
This changes it so that we only drop a command if it does not come with a callback. If it has a callback associated with it, it's better to call it anyway, otherwise odd races or just Denied results when you don't expect it tend to be the result.
This is the same as I fixed previously, however, this time also for CommandLong which I forgot last time.

then we keep growing the queue?

Yes, but for requests that we do every few seconds, we usually do it without callback. And if we do it with callback then we would realize that it doesn't work or takes a long time, and could fix it.

I'm merging this so that we at least match CommandLong and CommandInt.


Automator: update common-files@master in istio/istio@master

Generated by Automator - 2023-03-27T20:49:15+00:00

@istio-testing: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests-arm64_istio
c4998d4d27f53e21e12fc9b50ebaae48f25ec836
link
true
/test unit-tests-arm64

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test unit-tests-arm64


hebrew text is invalid when exporting to Web

Godot version
4.0.4.stable
System information
Windows 11, Chrome latest
Issue description
Added buttons and option button with some text in Hebrew
Everything works as expected When playing and also when exporting to Windows.
However when exporting to Web, and running in Chrome,  text becomes Gybrish
Windows

Web

Steps to reproduce
Adda button with Hebrew text, export to web project and serve,
Minimal reproduction project
.

Unlike Windows, Web platform can't load system fonts, and default font is limited to Latin, Greek and Cyrillic, you should use your own font.

Is this limitation documented?

It is documented in the SystemFont docs:

Note: This class is implemented on iOS, Linux, macOS and Windows, on other platforms it will fallback to default theme font.

But system font fallback is used automatically and enabled by default, so it probably should be mentioned on some generic GUI page as well (not sure where exactly).

But system font fallback is used automatically and enabled by default, so it probably should be mentioned on some generic GUI page as well (not sure where exactly).

This is documented in the System fonts section of the Using fonts page.

It probably needs a more obvious indication in some way, but I'm not sure what would be the most suited. Could we detect translations present in a project and display a warning in the Export dialog?

Not sure about export dialog, but adding a node configuration warning when it's using system fallback should be possible (in general, it's not a good idea to rely on the fallback on any platform).

Thanks for the quick response!
So i assigned a custom font and it indeed worked, i see the buttons now correctly in Chrome.
However the buttons are ok, but the options not, any advice? I set font in the theme overrides.
See attachements

You need to set the font in the theme for the popup as well, it is under PopupMenu I think, also check the get_popup method on the OptionButton

Thanks! i managed to do it via code (i don't think it's possible using the editor..)
code:
levelButton.get_popup().add_theme_font_override("font", load("res://myfont.ttf"))

You can set it in the theme, which is the general way to set up properly


Implement Categories

Easier than I thought. Thank you @craigh for your work!

Thank you @craigh for your work!

I'll send you a bill. 💵 💵 💵 💵 💵 💵 💵 💵 💵 💵 💵 💵


custom json (de)serialisation

Change Summary
Add json_loads and json_dumps to Config
Related issue number
#714
Checklist

[x] Unit tests for the changes exist
[x] Tests pass on CI and coverage remains at 100%
[ ] Documentation reflects the changes where applicable
[x] changes/<pull request or issue id>-<github username>.rst file added describing change
(see changes/README.md for details)

This is something that we have implemented seperately (adding a to_json function in a base class) so I am glad to see it make its way into the v1 release.
I took a quick look at the code and at a glance I didn't see anything inherently wrong with the implementation.

Yeah this looks great to me!

Good stuff! thanks @samuelcolvin


[Bug] Detect/Split Scanned photos needs scripts under root /scripts

"Detect/Split Scanned photos" searches for the script "split_photos.py" under root /scripts/ and not under the Stirling-PDF install folder ./scripts/. This may work in docker installations but needs to copy or symlink the scripts folder to / in standard installations.

Good point, will resolve next patch

Done as part of v0.8.0 release. which is now live


Empty space where sidebar used to be

There's a big empty space on the left at all non-mobile sizes. The sidebar used to be in that space, but now it looks weird. I think that the space wasn't there after @paulproteus' first change, but then was re-introduced when fixing the top bar genre list (but I'm not sure).

Would you say the best fix is to move "Featured apps" almost all the way to the left? If so, that should be reasonably easy to do.
My goal was to align the "Featured apps" text with left-padding location of the Sandstorm icon in the top-left. I see here I didn't achieve that.

I defer to @neynah on what the final layout should be.

Fixed by #43

Closing per @neynah request!


⚠ Orpheus has degraded performance

In 0a9e0db, Orpheus ($OPS) experienced degraded performance:

HTTP code: 200
Response time: 9645 ms

Resolved: Orpheus performance has improved in 59cc0f8.


Allow for same mlag domain on multiple mlag pairs

Enhancement summary
The schema for node_groups does not allow the same group to be used on multiple MLAG pairs. Either relax this requirement or implement another mlag_domain knob.
Which component of AVD is impacted
eos_designs
Use case example
.
Describe the solution you would like
.
Describe alternatives you have considered
No response
Additional context
No response
Contributing Guide

[X] I agree to follow this project's Code of Conduct

so the current blocking point is that we can't have multiple time the same group name for a given node_type? (we have under 5clos molecule the same group name across multiple node type and it works afaik)
        node_groups:
          type: list
          convert_types:
            - dict
          primary_key: group
          items:
            "$ref": "#/keys/node_type/keys/defaults"
            type: dict
            keys:
              group:
                type: str
                required: true
                description: The Node Group Name is used for MLAG domain.
              nodes:
                "$ref": "#/keys/node_type/keys/nodes"

Could you please share a snippet of the use case here? I feel like having multiple times the same group name under a give type would confuse our logic if some parameters are different across node types and would rather prefer to have an extra knob for mlag_domain that defaults to the group name.

Use case is brownfield where everything is deployed with a single static mlag domain. They have no use of the groupings, but it is the only way we allow to configure mlag.
With the list data models it is possible to define the same group everywhere and eos_designs works as expected. The only issue is the schema warning.


indicate rebase errors inside PR

right now the bot doesn't show when he can't rebase a PR due to conflicts.
Adding a comment to the PR would probably be a good first step.
There's an updated dockerhub image which contains the new feature from this branch:
nicolai86/github-rebase-bot:v0.0.4-2

Now rebased with latest master, available as nicolai86/github-rebase-bot:pr-14

will revisit this ft sometime in the future…


Update Japanese translation

Coverage remained the same at 24.341% when pulling 9ac971e4621862c9afc3d1a44a09de7b3b5f5c85 on MMP0:patch-17 into 2cca143121c19d5b1fe45c690b973b7465ef7aa9 on NyaaPantsu:dev.

Changes Unknown when pulling b2b7d4f1b09668ceb404b13fc9c1442fe9be11f1 on MMP0:patch-17 into ** on NyaaPantsu:dev**.

Coverage remained the same at 24.341% when pulling b2b7d4f1b09668ceb404b13fc9c1442fe9be11f1 on MMP0:patch-17 into e1cf80169c1c06dfc0b78bb0f76f920c59c38b3b on NyaaPantsu:dev.


GH Actions workflow "Nightly Checks and Tasks" doesn't run due to startup failure

See https://github.com/corona-warn-app/cwa-app-ios/actions/workflows/Nightly.yml.
@marcussc is probably the best to investigate this & already aware of the issue.

Seems to run correctly now, is this fixed @marcussc?

Yep, its working again, closing the Issue therefore.


App Dependencies Loaded Validation

Description
This system test validates that the app-dependencies-loaded event is sent when a module in the weblog has been loaded. The scenario has been created by a weblog endpoint that loads a module.
Check list
Your PR is not ready to be reviewed? Please save it as a draft :pray:

[ ] Follows the style guidelines of this project (See how to easily lint the code)
[ ] CI is passing
[ ] There is at least one approval from code owners

Yes to all? Feel free to merge it whenever you want :heart:

There are issue on nodejs : https://github.com/DataDog/system-tests/actions/runs/4083212142/jobs/7038945824#step:13:202


Filter/Standardize rank display

Right now, GoKibitz puts an enormous amount of trust in SGFs themselves to contain the correct type of information. The most obvious place this has gone wrong is in rank displays, where one SGF editor uses the word "Pharisee" instead of "kyu" (I'd love to know the story behind that one).
We should filter the information to only display number plus k or d for kyu or dan. Ideally, we can make it friendly enough to transform things like Kyu or kyu into k, and Dan or dan into d, but if it's unparseable, we shouldn't display it.

Are you already working on this? Do you mind if I give it a shot?
I was thinking that a simple custom filter should do the trick (Pharisee? wot? :D), but there's still the issue of game comments, e.g.:
vegeta [4SuperSayan]: I'm only using 40% of my tsumego knowledge
goku [4SuperSayan]: okie, good game

I guess we cannot really parse that?

Go for it! Definitely don't worry about ranks in game comments... That's not a big deal. The main thing is the rank that displays in our UI.
--
Nate Eagle

On May 17, 2015, at 1:19 AM, sphaso<EMAIL_ADDRESS>wrote:
Are you already working on this? Do you mind if I give it a shot?
I was thinking that a simple custom filter should do the trick (Pharisee? wot? :D), but there's still the issue of game comments, e.g.:
vegeta [4SuperSayan]: I'm only using 40% of my tsumego knowledge
goku [4SuperSayan]: okie, good game
I guess we cannot really parse that?
—
Reply to this email directly or view it on GitHub.

Is this issue still open because you want to use the filter in other places?

Yes, but I guess we could deal with those in separate issues. Thanks for the nudge: closing.


feat(server): add option to disable mid-message EOF detection

Add an option to disable mid-message EOF detection for users who want to avoid the extra syscall. I tested the change by checking that a mid-message EOF is not detected when the new option is set to false, this seemed like the easiest way. It's copy pasted from the http1_allow_half_close test. I am happy to test it differently if you have any suggestions.
Closes #2911

Closed since the change provided insignificant performance gains in benchmarks, and thus is not worth the increased API surface.


[Core-dev, Refactoring] Avoid creating new parameters in ParametricBlock.dagger

Reuse the parameters of the block.

@dominikandreasseitz this can be closed ?

@Roland-djee if we go for a replacement of the current block system, then yes


Is there any documentation?

Is there any documentation?

No, I created #5 as a joke.

@alcohol haven't had your morning coffee yet? :neckbeard:

Sorry, it just seems kind of hard to miss it, as the list of open issues really is not that long yet, and documentation is clearly there :p
I have been slacking. Have not had time to actually document the API yet. Need to add it to the gh-pages branch so it will show up on https://iso3166.thephpleague.com/


Add velocity gauge option

Add the option to perform calculations in velocity gauge. According to the results in the Potvliege paper, it should be easier to converge the calculations in velocity gauge, and it could also be useful to look for exceptional points in velocity gauge.
The input file already allows for this option, so what is required is passing a gauge flag to the Floquet function and setting the interaction matrix based on this.

Commit 6f77ed6 adds a gauge flag to the Floquet function and constructs the interaction matrix based on the flag's value. Doing Block-LU factorization seems to work fine, but there is an issue with the PARDISO option since it assumes that H_FL is complex symmetric which is no longer the case in velocity gauge, because V is no longer complex symmetric.  I should therefore update the PARDISO wrapper to be able to use a more appropriate matrix type for velocity gauge. I have to double-check that Block-LU still produces sensible values because I think that it also assumes the H_FL is complex symmetric.


How to properly test the find generator?

What is the best way to test the find_data generator, which relies on futures?

closed with #7


Fixed typo in DEVELOPER_GUIDE.md.

Trivial typo fix. (Sorry if you don’t want such tiny PRs, I tried looking for other typos in .md files but couldn’t find any.)
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@drunken-monkey Thanks for the contribution!


Overlay. There is always "loading" white page in Overlay.

Seems that after some manipulations with models availability on staging, now it's impossible to use Overlay.
It's Ok in incognito mode as user can login there. But in previously opened Overlay there is no logout option.

Need to investigate the issue. Not sure about steps to reproduce

Verified on staging env: If models are not received and user uses Overlay in the mode without header, login page is shown.


Typo in Readme

Hey!
I've found env variables in the readme.md. There is a typo https://github.com/paritytech/txwrapper-core/blob/main/packages/txwrapper-core/README.md#txwrapper_metadata_cache_max-1
TXWRAPPER_METADATA_CACHE_MAX has to be replaced to TXWRAPPER_METADATA_CACHE_MAX_AGE https://github.com/paritytech/txwrapper-core/blob/86d9620701e5ea17bb80b40a0c2d318877d5b3ea/packages/txwrapper-core/src/core/metadata/createMetadata.ts#L54

Ahh thanks for reporting this, will get this fixed.


Fix return type deprecation warnings

Similar to https://github.com/ramsey/uuid-doctrine/pull/173
Resolves https://github.com/ramsey/uuid-doctrine/issues/171
#173 only fixes https://github.com/ramsey/uuid-doctrine/issues/171#issuecomment-988066981 but it doesn't fix OP's issue, which is my issue as well.

Method Doctrine\DBAL\Types\Type::convertToPHPValue() might add mixed as a native return type declaration in the future. Do the same in child class Ramsey\Uuid\Doctrine\UuidBinaryOrderedTimeType now to avoid errors or add an explicit @return annotation to suppress this message.
Method Doctrine\DBAL\Types\Type::convertToDatabaseValue() might add mixed as a native return type declaration in the future. Do the same in child class Ramsey\Uuid\Doctrine\UuidBinaryOrderedTimeType now to avoid errors or add an explicit @return annotation to suppress this message.

@ramsey is there something that should be fixed/changed within this PR?

Thank you for contributing! 🎉

awesome! thank you!

@ramsey Is it possible to tag a new release?

@tarlepp Thanks for the reminder. I'll merge it in and tag a release soon.

@ramsey Please tag a release for this fix.


Add rafx-dynamic feature/crate for dynamic linking

Dynamic linking enables shorten build times, sometimes can even save you few minutes while building. (for example, used by bevy and veloren)
It may require introducing new crate rafx-dynamic which wraps all internal rafx-* crates and enables:
[lib]
crate-type = ["dylib"]

in Cargo.toml. You could then add it as feature in main crate.
Loading dynamic libs can be implemented as mechanism through libloading (platform independent) crate directly or as example for user's own implementation.
On the web, this can be done through loading WASM instead of .dylib/dll/so files.
Known issues:

Sometimes Windows does not allow loading DLLs which is completely random (seems to work fine with lld linker).

Thanks, will look into profiling and there are cargo plugins which also can show which packages slow downs compilation in rafx and my program itself.


Trying for a 10th time

Trying to do this the 1st and most ideal way for the bonus points.

I've sent a new one


Added the ability to reset validation state.

The functionality was added as an option to the module so that it is the default if the user chooses for it to be. It can also be ran with a new method resetValidation or via a parameter to the clear and reset methods.
This is useful when the same form is being reused without having to refresh the page or re-render the entire view.
The original functionality is retained as to not break backward-compatibility. This feature, in all cases, is purely optional. I updated the changelog portion of the README to reflect a minor update as their should be no breaking changes but it is not simply a bug patch.

hey @kylefarris, just thinking out loud here, and off the cuff.  shouldn't reset also reset the validation state?  otherwise, if you didn't want to reset the validation state, you'd just use clear.  that may not be how it's implemented ATM, but seems possibly more clear?  what do you think?

@cdaringe I would personally prefer that but it could cause some people's apps to act in an unexpected ways if they, for some reason, expect it to work as it currently does. With this PR, they can easily add in the feature by adding and setting the clearValidationOnReset option to true.
If they want to reset an individual field back to original state and clear validation state, they can run fieldVarHere.reset(true). OR to completely clear the field and clear validation state, they can run fieldVarHere.clear(true). Or they can just reset or clear fields as they have been without resetting validation state by calling the methods without the param (unless they have the clearValidationOnReset option set to true).
In other words, if we just made it so that reset did what clearValidation() does, users would lose the ability to decide to clear validation state--even if we both agree it makes much more sense to always do this (at least on reset).
My 2 cents.

gosh, i'm sorry, i'm still having a hard time understanding this.  apologies for the inconvenience, but can you baby step me through this?  I'm pretty sure i'm just a little slow today 🐢

Hmm... No problem man. So, let's assume this single field form (plus submit button)...
var FormView = require('ampersand-form-view');
var InputView = require('ampersand-input-view');
var Users = require('./collections/users');
var users_collection = new Users();

var name_field = new InputView({
    label: 'Name',
    name: 'name',
    required: true,
    tests: [
        function(val) {
            if (val.length <= 1) return "The name must be more than 1 character!"
        }
    ]
});

var form = new FormView({
    el: this.queryByHook('place-on-page'),
    submitCallback: (form_data) => {
        users_collection.create({name: form_data.name}, {wait:true, success: (m,r,o) => {
            form.reset();
        });
    },
    fields: () => {
        return [
            name_field,
            new InputView({
                name: 'submit',
                label: 'Submit',
                type: 'submit'
            })
        ];
    }
});

Now, if someone were to fill out the name field with, say, "K" and then press the Submit button, it would prevent the form from being submitted and place the invalidClass on the field/label (maybe make the border of the field red or something) and place the error message next to the field. This is great. The user corrects it to "Kyle" and, as they type, this plugin changes the class on the field/label to the validClass (yay! green outlines and stuff!). So far so good. This is what the module has been doing forever.
Okay, so now the user presses the submit button again after fixing their mistake and it posts the field to the back-end, or whatever, and it clears the form using ampersand-form-view'sreset method so that it can be used again for something else.
form.reset();

And... here is the problem... the invalidClass and 'required' message are now applied to the name field even though it's empty and the form has been reset. This is because the field is required, was filled out, and is now empty again. If this were to happen in normal circumstances (not re-using the same form again), that would be the correct behavior. But, what if we wanted to basically fully reset the form, including the validation stuff, so that we can use the same form over and over again without re-rendering its parent view or refreshing the page?
That's where the PR comes into play.
Now we have several options on how to handle the scenario where we want our form to be in the exact state it started in...
Option 1:  Completely clear the form and validation classes by passing true to the clear method
This might be the simplest and most effective/useful option IMO. This will completely clear all the fields (regardless of what their initial values were when the form was initially rendered on the page) and the validation classes (and messages) would be removed..
form.clear(true);

Option 2: Do a full reset of the form by passing true to the reset method and clear validation classes
This is very similar to option 1 but will just reset the form back to it's original state. So, if name field had a value of 'Bob' by default, it would now be reset back to 'Bob' (as opposed to cleared entirely) and the validation classes (and messages) would be removed.
form.reset(true);

Option 3: clear/reset specific fields of your choosing
This is nice if you only need to reset 1 or two fields on a form with, say, dozens...
name_field.clear(true);
// ... or ...
name_field.reset(true);

Option 4: Have the clear and reset method also clear validation by default using a module option
If you want this functionality by default for all times when you reset/clear a field (either directly or via the parent form's reset method), you can just set it in the field's options...
var name_field = new InputView({
    label: 'Name',
    name: 'name',
    required: true,
    clearValidationOnReset: true, // <-- THIS
    tests: [
        function(val) {
            if (val.length <= 1) return "The name must be more than 1 character!"
        }
    ]
});

var form = new FormView({
    fields: [name_field]
});
form.reset(); // <-- no need for TRUE to be passed here

Option 5: Just reset the validation (don't clear or reset the fields)
This is useful if you just want to clear the validation classes for whatever reason. Not sure there's too many use-cases for this, but, I'm sure someone will think of one. Doesn't hurt to add it.
name_field.resetValidation()

I hope that makes it a bit clearer.

@kylefarris, amazing writeup.  thanks so much!
with this new understanding, i'm on board.  i'm going to speak through my thought process below.  i'm sorry to keep dragging out the review, but:

are you still convinced reset need the ability?  select-view clears it by default.  however, to counter that, clearing by default doesn't represent a true reset if validation was present on init. so 👍
API.  my personal pref would be for the API to be more akin to field.clear({ validate: false }).  more specifically, an object based argument, to ++communication/readability.  it's definitely not the status quo (shame on me!) but if this will be rippling back up to form and possibly into other fields, it may be a good opportunity to make it great 💪

No problem at all--I totally welcome (and appreciate) a thoughtful discussion on any PR. It's what makes open source software so much better than anything else.
That said, on your first point, I do think reset needs the option for it (even if its rarely used) purely for aesthetics and to meet what the end-user expects in such a scenario.

Contrived User Update Form:
var name_field = new InputView({
    label: 'Name',
    name: 'name',
    value: 'Bob'
    tests: [
        function(val) {
            if (val.length <= 1) return "The name must be more than 1 character!"
        }
    ]
});

var form = new FormView({
    el: this.query('#update_user_form'),
    fields: [name_field]
});

User highlights the default text in the name field containing valid value 'Bob' and continues to type replacement value, 'Robert'.

Form field flickers from red (invalid) to green as the character count climbs above 1.

User decides that they've changed their mind--they like the name 'Bob' and so they hit the 'Reset' button on the form which is tied to an event which triggers it's parent form's reset method.
this.query('#update_user_form').onreset = function(e) {
    form.reset();
});

Field is reverted back to 'Bob' but the form is now green unlike how it was when they initially loaded the page which may lead the end-user to think that something didn't work properly. Or they might just think it's weird/ugly and move on.

It might be a somewhat rare scenario but it would be there should a developer need such functionality.

As for point 2, I completely agree with you. I tend to prefer that style of parameterization in javascript as well. It does, however, make merging with defaults slightly more of a pain, maybe. Maybe not... there's several ways to go about it.
Which do you prefer?
1. Override directly in default object
function foo(params) {
    params = params || {};
    var options = {
        validate: params.validate || true,
        foo: (params.foo && typeof params.foo === 'object' && Object.keys(params.foo).length > 0) ? params.foo : {
            bar: 'baz'
        })
    };
}

2. Merge parameter object with a defaults object
function foo(params) {
    var options = {
        validate: true,
        foo: {
            bar: 'baz'
        }
    };
    if (params && typeof params === 'object' && Object.keys(params).length > 0)
        options = Object.assign({}, options, params);
}

My thoughts on it...
Option 1 would be more direct, faster, would be more-controllable, and would work on all browsers. It might get a little hairy for multi-level deep parameter objects (rare), though.
Option 2 might scale a little better (from a "code-cleanliness" standpoint) for items with a large number of options but would also be a bit slower and wouldn't work with browsers that don't support Object.assign (IE, of course). We could get around that by using lodash.assign but that's just another chunk of code to include (its probably already included through other modules, but, still..). Another potentially bad thing (although minor if things are coded properly) is that additional options could be merged into the default that we don't want or support.

_.merge is a good option to address collisions.  _.defaults could be used as well.  whatever is most fitting for what you observe in the codebase.  _.assign is fine if parameters are flat-ish.  call the ball, based on what you see


feat: Canvas actions uplift

Hey @mfortman11
Could you please improve the title of this PR to something that describes what was changed/implemented?


[Bug] TypeError: "t._sortStrategyMap[e] is undefined"

The app doesn't show anything (even the menu)
To Reproduce
I use the app since some times now.
Maybe some of my data don't update.
I have just been on the app (already 'logged in' with github)
Expected behavior
Display the app
Notes
I use Firefox.
This is the error :
TypeError: "t._sortStrategyMap[e] is undefined"
    getNotesSorter https://takenote.dev/main.47a113d09968dab64621.bundle.js?47a113d09968dab64621:42
    NoteList https://takenote.dev/main.47a113d09968dab64621.bundle.js?47a113d09968dab64621:42
    ca https://takenote.dev/vendors.47a113d09968dab64621.bundle.js?47a113d09968dab64621:17
    Qa https://takenote.dev/vendors.47a113d09968dab64621.bundle.js?47a113d09968dab64621:17
    So https://takenote.dev/vendors.47a113d09968dab64621.bundle.js?47a113d09968dab64621:17
    hu https://takenote.dev/vendors.47a113d09968dab64621.bundle.js?47a113d09968dab64621:17
    pu https://takenote.dev/vendors.47a113d09968dab64621.bundle.js?47a113d09968dab64621:17
    iu https://takenote.dev/vendors.47a113d09968dab64621.bundle.js?47a113d09968dab64621:17
    Kl https://takenote.dev/vendors.47a113d09968dab64621.bundle.js?47a113d09968dab64621:17
    unstable_runWithPriority https://takenote.dev/main.47a113d09968dab64621.bundle.js?47a113d09968dab64621:27
    Hl https://takenote.dev/vendors.47a113d09968dab64621.bundle.js?47a113d09968dab64621:17
    Kl https://takenote.dev/vendors.47a113d09968dab64621.bundle.js?47a113d09968dab64621:17
    ql https://takenote.dev/vendors.47a113d09968dab64621.bundle.js?47a113d09968dab64621:17
    au https://takenote.dev/vendors.47a113d09968dab64621.bundle.js?47a113d09968dab64621:17
    notify https://takenote.dev/main.47a113d09968dab64621.bundle.js?47a113d09968dab64621:1
    notifyNestedSubs https://takenote.dev/main.47a113d09968dab64621.bundle.js?47a113d09968dab64621:1
    handleChangeWrapper https://takenote.dev/main.47a113d09968dab64621.bundle.js?47a113d09968dab64621:1
    v https://takenote.dev/main.47a113d09968dab64621.bundle.js?47a113d09968dab64621:1
    p https://takenote.dev/main.47a113d09968dab64621.bundle.js?47a113d09968dab64621:42
    dispatch https://takenote.dev/main.47a113d09968dab64621.bundle.js?47a113d09968dab64621:1
    O https://takenote.dev/main.47a113d09968dab64621.bundle.js?47a113d09968dab64621:1
    L[l.q]/< https://takenote.dev/main.47a113d09968dab64621.bundle.js?47a113d09968dab64621:42
    d https://takenote.dev/main.47a113d09968dab64621.bundle.js?47a113d09968dab64621:42
    y https://takenote.dev/main.47a113d09968dab64621.bundle.js?47a113d09968dab64621:42
    h https://takenote.dev/main.47a113d09968dab64621.bundle.js?47a113d09968dab64621:42
    l.q https://takenote.dev/main.47a113d09968dab64621.bundle.js?47a113d09968dab64621:42
    u https://takenote.dev/main.47a113d09968dab64621.bundle.js?47a113d09968dab64621:42
    g https://takenote.dev/main.47a113d09968dab64621.bundle.js?47a113d09968dab64621:42
    h https://takenote.dev/main.47a113d09968dab64621.bundle.js?47a113d09968dab64621:42
    s https://takenote.dev/main.47a113d09968dab64621.bundle.js?47a113d09968dab64621:42

Looks like the sorting PR broke something with existing data. I’ll have to reverse the PR, so I’ll do that soon.

Does it work for you now?

Hey !
Thanks for your quick answer :)
Everythings if fine
Thanks again for this app !


Differentiate between user-defined variables and built in variables

Currently both are just @variable

Currently both are just @variable

For example? it is easy to realize, but the buildin variable I already have known is 'true' and 'false'

https://mesonbuild.com/Reference-manual.html#builtin-objects

meson, host_machine, build_machine, target_machine.

https://github.com/nvim-treesitter/nvim-treesitter/pull/3662

Currently both are just @variable

Take a look, if there are still something need to add

Thanks!


Moving CommonMark-py to readthedocs

Recently readthedocs kindly asked me if I'd mind to move recommonmark to their organization and I accepted, I'd move the development of CommonMark-py there as well, you'd be ok with that?

We're hoping to fully support Commonmark & Recommonmark on Read the Docs, and take some responsibility in maintenance.
We would of course give you full credit as the author, and commit to the repos. We're also happy to keep development here; we're just trying to provide a solid commonmark stack on our site, and increase development effort in the dependencies.

@ericholscher since there is no activity I'd move it.


Server doesn't get a connection on a previously paired host

I use Windows 10 as a host system that I pair with my Pi.
I've added hosts MAC as trusted on my Pi etc.
The first time I find my Pi and pair it with Windows 10 (no pin code used) the server in Pi sees the socket connection and works fine.
If I turn Bluetooth off on Windows 10 and turn it back on - the server doesn't see the connection. Even though device in Win 10 displayed as "Connected" after turning Bluetooth back on. Restarting the server in Pi doesn't help.
No combination of restarting Bluetooth/Pi/Windows/your server works - I have to manually remove the Bluetooth device in Win 10 and find it and pair it again.
I don't know if this project is supported still, would appreciate any help debugging this. Not sure where to start.

Just want to point out that I'm seeing the same problem an on Android phone.

I recommend to use Across Center guys. It's commercial, but if you just want to relay mouse, not keyboard (like me), it's free. It's rock solid with the Raspberry 4B BT-HID connection, and works like a charm.


Results w.r.t to the testing.

Hi, @zzy14 , thanks for your implementation first. But with regard to your testing method, you actually give out the number of classes as a prior knowledge in the TopkRankClassifier, this is not strict theoretically. In fact, I test the performance using the following code and it works not perfect.
for i in range(epoch):
                self.model1.train_one_epoch()
                self.model2.train_one_epoch()
                if i % 10 == 0:
                    self.get_embeddings()
                    new_embedding = []
                    keys = list(set(self.vectors.keys()))
                    for i in keys:
                        new_embedding.append(self.vectors[i])
                    #testing.
                    #labels are in the form of multi-hot vector for multi-label classification.
                    X_train,X_test, y_train, y_test = train_test_split(new_embedding, labels, test_size=0.5, random_state=123)
                    clf = OneVsRestClassifier(LogisticRegression(), n_jobs = 2)
                    clf.fit(X_train, y_train)
                    test_labels_pred = clf.predict(X_test)
                    test_scores_pred = clf.predict_proba(X_test)
                    #from sklearn.metrics import classification_report
                    #acc = classification_report(y_test, test_labels_pred)
                    #print(acc)
                    acc = sklearn.metrics.accuracy_score(y_test, test_labels_pred)
                    auc = sklearn.metrics.roc_auc_score(y_test, test_scores_pred)
                    precision = sklearn.metrics.precision_score(y_test, test_labels_pred, average = 'macro')
                    recall = sklearn.metrics.recall_score(y_test, test_labels_pred, average = 'macro')
                    f1 = sklearn.metrics.f1_score(y_test, test_labels_pred, average = 'macro')
                    print(f"Average accuracy: {acc}")
                    print(f"Average auc: {auc}")
                    print(f"macro precision: {precision}")
                    print(f"macro recall: {recall}")
                    print(f"macro F1: {f1}")

Could you please help me with this issue? Thanks.

Could you please describe your input data?

@zzy14 ,Hi, the input is the embedding of all the nodes in order. and the labels are a vector for every node describing the classes the node belong to such as [1,0,0,1,..,1].
Thanks,

According to your code, I think the evaluation process is correct.

@zzy14 ,thanks, it tends out that I do not make the embeddings in order properly. And the testing performance is great though.


USB Audio device on ESP32-S3 : Hitting upper limit of open endpoints when I should not; issue in portable/synopsys/dcd_dwc2.c? (IDFGH-12779)

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

General issue report
I am implementing a headset (speaker+mic) on ESP32-S3. I notice that when I pause the playback or record 3 times, playback or record does not happen any more.
I traced the issue to a couple of TU_ASSERT in dcd_edpt_counter_release(). When I pause playback, the endpoint is released and opened again when I start playing.
// Release an endpoint.
static bool dcd_edpt_counter_release(uint8_t rhport, uint8_t dir)
{
  if (dir) {
    TU_ASSERT(dwc2_ep_counters[rhport].active_in == 0);    // Check if number of opened EPs is not zero
    dwc2_ep_counters[rhport].active_in--;                  // Decrease endpoint counter
  } else {
    TU_ASSERT(dwc2_ep_counters[rhport].active_out == 0);
    dwc2_ep_counters[rhport].active_out--;
  }
  return true;
}

Obviously the ASSERT fails when active_in and active_out are >0 and they are not decremented. Do this 3 times and total number of open endpoints are 6, which the limit. Subsequently, dcd_edpt_counter_available() returns false.
When I changed '==' to '>', the problem went away. It makes sense to me. Also the comment implies that it should be '!=' or '>'.
Anybody else faced this issue? Am I missing something or thinking it wrong?

@giri-a,
Thanks for reporting the issue.
The fix was already merged (https://github.com/espressif/tinyusb/pull/25), we will publish the new version soon.
Sorry for inconvenience.

Hi @giri-a,
There is a new version of espressif/tinyusb is available (v<IP_ADDRESS>).
If there will be anything else or you will run into any other issue, fell free to reopen this ticket or create a new one.
Thanks for your help.

Perfect. That was quick. Thanks.


Implement semantic adversarial attacks

These are adversarial images that semantically represent the same object as the original images. For example, in [2] these are created by perturbing the following components of an image:

Hue
Saturation
Rotation
Brightness
Contrast

Would these be of interest? If yes, I can implement them for ART.
My implementation: https://github.com/sarathknv/adversarial-examples-pytorch/tree/master/semantic_adversarial_examples
Papers:

Semantic Adversarial Examples (CVPR Workshops 2018)
Towards Compositional Adversarial Robustness: Generalizing Adversarial Training to Composite Semantic Perturbations (CVPR 2023)

Hi @sarathknv Sorry for the delayed response. Yes, these attacks would be very interesting for ART and you are very welcome to implement them in ART. Are you already familiar with the architecture of ART?


New version: CSVReader v1.0.2

Registering package: CSVReader
Repository: https://github.com/tk3369/CSVReader.jl
Created by: @tk3369
Git reference: register/CSVReader/v1.0.2
Version: v1.0.2
Commit: 28e4df8375da62b239d6d922dbda97f6e5215ecc
Reviewed by: @tk3369
Reference: https://github.com/tk3369/CSVReader.jl/commit/28e4df8375da62b239d6d922dbda97f6e5215ecc#commitcomment-33521529

Can you re-add
[compat]
julia = "1"

to your project file and retrigger the bot?


Anti-Cheat doesn't work

If a exploiter (This time it was @tubersssbrother94 who was using server sided exploits like making everything blue or green and making himself giant) is in the game and doing stuff like flying or even server sided stuff, he doesn't get kicked/banned and nothing in exploit logs.
That the player gets kick/banned and/or I will get a notification from exploit logs.
The exploiter doesn't get kick/banned and I get nothing in the exploit logs.
I can't defend my game from exploiters.
(My game is not backdoored)
I noticed that all after, the problem #855

Adonis is NOT a anticheat.

But month previous people with exploits got kicked

Also your game is clearly backdoored if they use serversided exploits

I tried it with an alt, to scan for backdoor in the game

Dumbest issue i've ever read apart from the L8X Pull requests; u don't scan for backdoor in game using alt. If u know the game of exploiter, I suggest u ban them (tubersssbrother94). You shouldn't rely on the Adonis Anti"-Cheat" (in fact the Adonis module is named just "Anti"), as the Adonis Anti module is supposed to provide a layer of protection to the Adonis module itself, from foreign third-party scripts messing with the code.

Yeah sorry, I’m not a good at Developing or exploiting. But I can’t ban him bc the damastner doesn’t save it and after a rejoin he unbanned again. So I banned him with a extra script.

Get rid of the serverside backdoors in your game. You can find help on how to do so on the Roblox DevForum if you're confused. One method is to only keep scripts that you trust/understand and delete suspicious scripts or models.
The Adonis datastore issue should be fixed now; tell me if you still can't use ban.
Adonis cannot stop exploiters with serverside access from using cheats in your game.


Fix Eslint Errors

This will make Eslint no longer throw an error in regards to conflicting with Perfetto's Eslint.  This means Eslint should work in Perfetto UI.

Only issue I'm noticing is linebreak-style.  It defaults to Unix line endings, but since we all work on different OS it might be simpler to turn this rule off in Perfetto's lint config.  "linebreak-style": "off"

This looks fine to me.
On the matter of line breaks, isn't that a detail that is better controlled at the git level via the .gitattributes settings for text file line-ending conversions?


fix(ranges): parse ranges for all command

'<,'>s/foo/bar errors:
(script_file [0, 0] - [1, 0]
  (range_statement [0, 0] - [0, 5]
    start: (mark [0, 0] - [0, 2])
    end: (mark [0, 3] - [0, 5]))
  (ERROR [0, 5] - [0, 14]
    (marker_definition [0, 7] - [0, 14])))

after:
(script_file [0, 0] - [1, 0]
  (unknown_builtin_statement [0, 0] - [0, 14]
    start: (mark [0, 0] - [0, 2])
    end: (mark [0, 3] - [0, 5])
    (unknown_command_name [0, 5] - [0, 6])
    (arguments [0, 6] - [0, 14]
      (command_argument [0, 6] - [0, 14]))))

I don't know if parsing ranges before all commands has issues though.

Please commit the generated parser.c (and other files).

I forgot to make them optional ...

I forgot to make them optional ...

I ran into another problem, since the ending ("/") in range marks were optional statements that used an range in pattern ranges would result in errors.
/acall/
(script_file [0, 0] - [1, 0]
  (range_statement [0, 0] - [0, 2]
    start: (pattern [0, 1] - [0, 2]))
  (ERROR [0, 2] - [0, 6]))

I think they were optional because range statements don't need to end, which is why i made a new _complete_range for commands and statements.

I am running make and nothing is changing, and the artifact seems related to matchparen?

Hey @clason i have 2 questions.
Should i rebase this off of fix_ci ?
I want to add a query for ranges do i add it in this pr or do i open another one?


Anti brute force protection

Hello; i looked everywhere but did not find anything in the docs or by searching, so i thought i should ask here.
Does influxdb2 support some form of anti-brute force protection, like Grafana (https://grafana.com/docs/grafana/latest/setup-grafana/configure-grafana/#disable_brute_force_login_protection) ?
If not, how can this be achieved (which logs should contain authentication errors, how should they look like, how to create a fail2ban configuration...) ?

There are no builtin provisions for brute force protection. The unauth'd logs should look like the one below.
2023-01-05T15:19:06.991609Z     info    Unauthorized    {"log_id": "0fCcBaFG000", "error": "token required"}

Which log is that supposed to be ? The influx2 binary (e.g. in journald if run using systemd) ?
Unfortunately, this log does not include any IP address that could be used to ban the offending IP.
Any advice on how to protect an influxdb instance against such attacks ?

Taking the liberty to add the comment from someone in the Influx team i got on the community slack: "with regards to the UI login currently not. You can either disable the UI entirely. Enterprise and Cloud handle these risks with greater consideration. The API is all token based so would be extremely hard to brute force login".
I guess that for the OSS version, the solution is:

disable the web UI when unused (https://docs.influxdata.com/influxdb/v2.0/reference/config-options/#ui-disabled)
monitor/supervise request rate on e.g. nginx frontend


Stepper component manualInputMode with decimalPoint issue

Framework7 version: latest
Platform and Target: all (core issue)
Live Link or JSFiddle: official doc manual input https://framework7.io/docs/stepper

Describe the bug
Manual input does not allow to input some of float values like 1.01 or 1.01
To Reproduce
Steps to reproduce the behavior:

Input 1
Input .
Input 0
See that it is formatted to integer "1"

it's because it interprets the value as string not a number @nolimits4web it was an old issue i think, posted something here:https://forum.framework7.io/t/react-stepper-increments-10-then-1/14719


Add performance test for state and pubsub

In what area(s)?
/area runtime
Describe the feature
Dapr currently only has a performance test for actor and service invocation.
This issue is to add perf test for state. The protocol will be gRPC.

There are some tasks for this issue:

add an in-memory state store
extend fortio to support no-operation (for baseline test of state api)
extend fortio to support dapr state gRPC API
add perf test case for state store

Related issue: https://github.com/dapr/dapr/issues/1787  In this issue, fortio is forked and extended to support dapr gRPC API, and service invocation perf test case is added in this issue.
Release Note

@mukundansundar is this not closed via https://github.com/dapr/dapr/pull/4543?

closing.


Allow to "catch and run" a new process instance spawned by the run itself

Marked as duplicate and closed, because it's old and now better described by request #81.


Custom query/mutation with VTL resolver

It is not clear to me what I have to do to create custom query/mutation and use VTL resolver. Consider the following schema
type Post @model {
  title: String
}

I want to add the following custom mutations (please note that this is a very simple example)
type Mutations {
  promotePost(): Post
}

After manually add Mutation.promotePost.req.vtl and Mutation.promotePost.res.vtl inside resolvers directory (not build/resolvers directory) if I try to invoke the mutation I get the following error

resolve is not a function

I am using transformation v2. Looking at the documentation I understand that I have to create a custom VTL resolver for each custom query/mutation but for me this is an expensive operation (that I have to repeat for each custom query/mutation I want).
Is this the only way? Am I missing something? Thanks in advice

@nikname when you're creating a custom resolver, adding just the request and response template to resolver wont work. Each of the operations (mutation or query) need a data source which says where the is retrieved or stored. In the above example of promotePost the resolver templates wont be able to infer that Post is stored in DynamoDB  PostTable.

I understand that I have to create a custom VTL resolver for each custom query/mutation but for me this is an expensive operation (that I have to repeat for each custom query/mutation I want).

You can create a single stack that has all the custom query and mutation in it and add the resolvers for them there.

Hi @yuth, thanks for the explanation. I have looked to the old documentation and to https://github.com/aws-amplify/amplify-cli/issues/3321 and now everything is more clear.
Let me pose some other questions. You said to create a single stack with all the custom query/mutation like in stacks/CustomResources.json or in a new different template? If I create a new template inside stacks folder it will be automatically created as nested from the root stack? I don't think so. Again looking at the documentation I should create a custom function with amplify add custom and the add to the resources something like the following
{
  "Resources": {
    "GetMyModel": {
      "Type": "AWS::AppSync::FunctionConfiguration",
      "Properties": {
        "Name": "GetMyModel",
        "ApiId": {
          "Ref": "AppSyncApiId"
        },
        "DataSourceName": "MyModelTable",
        "FunctionVersion": "2018-05-29",
        "RequestMappingTemplateS3Location": {
          "Fn::Sub": [
            "s3://${S3DeploymentBucket}/${S3DeploymentRootKey}/resolvers/Function.MyModel.req.vtl",
            {
              "S3DeploymentBucket": {
                "Ref": "S3DeploymentBucket"
              },
              "S3DeploymentRootKey": {
                "Ref": "S3DeploymentRootKey"
              }
            }
          ]
        },
        "ResponseMappingTemplateS3Location": {
          "Fn::Sub": [
            "s3://${S3DeploymentBucket}/${S3DeploymentRootKey}/resolvers/Function.MyModel.res.vtl",
            {
              "S3DeploymentBucket": {
                "Ref": "S3DeploymentBucket"
              },
              "S3DeploymentRootKey": {
                "Ref": "S3DeploymentRootKey"
              }
            }
          ]
        }
      }
    }
  }
}

The problem now is that the template has no S3DeploymentBucket and S3DeploymentRootKey parameters. How do I tell the root template to pass theese parameters?

I would not recommend adding a custom stack using amplify add custom. This should be used only when you're creating a new resource and not augmenting an existing API. As you mentioned in your comment, you will not be able to get S3DeploymentRootKey as its tied to API resource.
GraphQL transformer allows adding a custom stack to add new query/mutation or subscription to cover edge cases like yours. The old way was to create a custom cloudformation stack which is deprecated. The new way to do this is to use override.ts as described. With the override.ts you're creating a custom cloudfomration stack and in this stack you can add resolver configuration for all the custom query, mutation and subscriptions.

Thanks again for your answer. To be honest it is not very clear now. I was able to create a custom VTL resolver as a custom resource using CDK as described here but I don't like this solution becouse I am defining a custom AWS resource while my intent was to extend the API.
As you said I should edit the override.ts file but I don't know how. To which property should I register the custom function/resolver? Can you please share with me an example? It could be very helpful
By overriding the API stack I should be able to test the API locally using amplify mock api? With a custom resource it is not possible

This is what I have tryed to do but I keep getting errors like "Error: Skipping override due to VMError: Cannot find module 'fs'"
import { AmplifyApiGraphQlResourceStackTemplate } from "@aws-amplify/cli-extensibility-helper";
import * as appsync from "@aws-cdk/aws-appsync";

export function override(resources: AmplifyApiGraphQlResourceStackTemplate) {
  resources.models["Post"].resolvers = {
    ...resources.models["Post"].resolvers,
    mutationPromotePostResolver: new appsync.CfnResolver(resources.api.GraphQLAPI.stack, "PromotePostResolver", {
      apiId: resources.api.GraphQLAPI.attrApiId,
      fieldName: "promotePost",
      typeName: "Mutation",
      requestMappingTemplate: appsync.MappingTemplate.fromFile(`./resolvers/Mutation.promotePost.req.vtl`).renderTemplate(),
      responseMappingTemplate: appsync.MappingTemplate.fromFile(`./resolvers/Mutation.promotePost.res.vtl`).renderTemplate(),
      dataSourceName: "PostTable"
    })
  };
}

Nobody can help me?


Convert input data into DTensor

Codecov Report
Attention: 49 lines in your changes are missing coverage. Please review.

Comparison is base (3d3a378) 77.67% compared to head (98d912f) 57.23%.
Report is 9 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff             @@
##           master   #18556       +/-   ##
===========================================
- Coverage   77.67%   57.23%   -20.45%
===========================================
  Files         334      334
  Lines       32302    32415      +113
  Branches     6297     6320       +23
===========================================
- Hits        25092    18553     -6539
- Misses       5631    12440     +6809
+ Partials     1579     1422      -157

Flag
Coverage Δ

keras
57.23% <26.86%> (-20.36%)
:arrow_down:

keras-jax
?

keras-numpy
57.23% <26.86%> (-0.06%)
:arrow_down:

keras-tensorflow
?

keras-torch
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files
Coverage Δ

keras/backend/jax/core.py
19.02% <0.00%> (-70.11%)
:arrow_down:

keras/backend/jax/trainer.py
0.00% <0.00%> (-95.52%)
:arrow_down:

keras/distribution/distribution_lib.py
93.93% <66.66%> (-1.13%)
:arrow_down:

keras/backend/jax/distribution_lib.py
28.12% <37.50%> (-64.74%)
:arrow_down:

keras/backend/tensorflow/core.py
60.41% <58.33%> (-35.83%)
:arrow_down:

keras/backend/tensorflow/distribution_lib.py
29.16% <22.22%> (-6.13%)
:arrow_down:

keras/backend/tensorflow/trainer.py
14.47% <12.12%> (-64.06%)
:arrow_down:

... and 177 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Since Bing move to a new team,  I will take the PR from here.

Context: the PR has failed tests when tests running together, but will pass when running separately. Synced offline with Scott about the issue and the error log, will leave the PR for Scott

Hi @hubingallin Can you please resolve conflicts? Thank you!

I will close this PR for now, since Bing is working on some other projects.