Database queue driver doesn't reconnect

Hi,
The database queue driver doesn't seem to reconnect to the DB after losing connection.
Although I am able to work around this issue it's a big gotcha for other users of the queue daemon with similar settings.
Normal DB operations already have a tryAgainIfCausedByLostConnection method so the same logic could be used for this case: https://github.com/laravel/framework/blob/5.2/src/Illuminate/Database/Connection.php#L691
System:
Laravel: 5.2.45
PHP: 7.0.10
Mac: OS X 10.11.6
Steps to reproduce:

Start queue daemon:

QUEUE_DRIVER=database php artisan queue:work --daemon --quiet --sleep=1

Restart MySQL:

brew services stop mysql  # or "service mysql stop" on Ubuntu
sleep 10
brew services start mysql # or "service mysql start" on Ubuntu

Stack trace
PDOException: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away in /Users/example/vendor/laravel/framework/src/Illuminate/Database/Connection.php:584
Stack trace:
#0 /Users/example/vendor/laravel/framework/src/Illuminate/Database/Connection.php(584): PDO->exec('SAVEPOINT trans...')
#1 /Users/example/vendor/laravel/framework/src/Illuminate/Queue/DatabaseQueue.php(162): Illuminate\Database\Connection->beginTransaction()
#2 /Users/example/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(180): Illuminate\Queue\DatabaseQueue->pop()
#3 /Users/example/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(149): Illuminate\Queue\Worker->getNextJob(Object(Illuminate\Queue\DatabaseQueue), NULL)
#4 /Users/example/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(111): Illuminate\Queue\Worker->pop(NULL, NULL, 0, '1', 0)
#5 /Users/example/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(85): Illuminate\Queue\Worker->runNextJobForDaemon(NULL, NULL, 0, '1', 0)
#6 /Users/example/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(119): Illuminate\Queue\Worker->daemon(NULL, NULL, 0, 128, '1', 0)
#7 /Users/example/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(78): Illuminate\Queue\Console\WorkCommand->runWorker(NULL, NULL, 0, 128, true)
#8 [internal function]: Illuminate\Queue\Console\WorkCommand->fire()
#9 /Users/example/vendor/laravel/framework/src/Illuminate/Container/Container.php(507): call_user_func_array(Array, Array)
#10 /Users/example/vendor/laravel/framework/src/Illuminate/Console/Command.php(169): Illuminate\Container\Container->call(Array)
#11 /Users/example/vendor/symfony/console/Command/Command.php(256): Illuminate\Console\Command->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#12 /Users/example/vendor/laravel/framework/src/Illuminate/Console/Command.php(155): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#13 /Users/example/vendor/symfony/console/Application.php(791): Illuminate\Console\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#14 /Users/example/vendor/symfony/console/Application.php(186): Symfony\Component\Console\Application->doRunCommand(Object(Illuminate\Queue\Console\WorkCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#15 /Users/example/vendor/symfony/console/Application.php(117): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#16 /Users/example/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(107): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#17 /Users/example/artisan(35): Illuminate\Foundation\Console\Kernel->handle(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#18 {main}

We should automatically reconnect for such errors.

Hmm, seems we only do that for query executions, not acquiring transactions. Ping @taylorotwell.

Although I am able to work around this issue
Can you provide a work around? :)

+1 to get notified on this thread. Might be connected on a issue in my package https://github.com/yajra/laravel-oci8/issues/191. Thanks!

Seem to be encountering the same issue on 5.2 on PHP 7
The queues work for a day, then failed and sit in reserved. Seems It's also trying to log into the failed jobs table but that is also failing.
[2016-09-09 14:59:04] local.ERROR: Symfony\Component\Debug\Exception\FatalThrowableError: Call to a member function prepare() on null in /var/www/api/build-369/vendor/laravel/framework/src/Illuminate/Database/Connection.php:453
Stack trace:
#0 /var/www/api/build-369/vendor/laravel/framework/src/Illuminate/Database/Connection.php(718): Illuminate\Database\Connection->Illuminate\Database\{closure}(Object(ThermoView\Database\Connections\MysqlConnection), 'insert into `fa...', Array)
#1 /var/www/api/build-369/vendor/laravel/framework/src/Illuminate/Database/Connection.php(681): Illuminate\Database\Connection->runQueryCallback('insert into `fa...', Array, Object(Closure))
#2 /var/www/api/build-369/vendor/laravel/framework/src/Illuminate/Database/Connection.php(454): Illuminate\Database\Connection->run('insert into `fa...', Array, Object(Closure))
#3 /var/www/api/build-369/vendor/laravel/framework/src/Illuminate/Database/Connection.php(410): Illuminate\Database\Connection->statement('insert into `fa...', Array)
#4 /var/www/api/build-369/vendor/laravel/framework/src/Illuminate/Database/Query/Processors/Processor.php(32): Illuminate\Database\Connection->insert('insert into `fa...', Array)
#5 /var/www/api/build-369/vendor/laravel/framework/src/Illuminate/Database/Query/Builder.php(2055): Illuminate\Database\Query\Processors\Processor->processInsertGetId(Object(Illuminate\Database\Query\Builder), 'insert into `fa...', Array, NULL)
#6 /var/www/api/build-369/vendor/laravel/framework/src/Illuminate/Queue/Failed/DatabaseFailedJobProvider.php(58): Illuminate\Database\Query\Builder->insertGetId(Array)
#7 /var/www/api/build-369/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(313): Illuminate\Queue\Failed\DatabaseFailedJobProvider->log('beanstalkd', 'default', '{"job":"Illumin...')
#8 /var/www/api/build-369/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(204): Illuminate\Queue\Worker->logFailedJob('beanstalkd', Object(Illuminate\Queue\Jobs\BeanstalkdJob))
#9 /var/www/api/build-369/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(156): Illuminate\Queue\Worker->process('beanstalkd', Object(Illuminate\Queue\Jobs\BeanstalkdJob), '2', 0)
#10 /var/www/api/build-369/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(111): Illuminate\Queue\Worker->pop(NULL, NULL, 0, '3', '2')
#11 /var/www/api/build-369/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(85): Illuminate\Queue\Worker->runNextJobForDaemon(NULL, NULL, 0, '3', '2')
#12 /var/www/api/build-369/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(119): Illuminate\Queue\Worker->daemon(NULL, NULL, 0, 128, '3', '2')
#13 /var/www/api/build-369/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(78): Illuminate\Queue\Console\WorkCommand->runWorker(NULL, NULL, 0, 128, true)
#14 [internal function]: Illuminate\Queue\Console\WorkCommand->fire()
#15 /var/www/api/build-369/vendor/laravel/framework/src/Illuminate/Container/Container.php(507): call_user_func_array(Array, Array)
#16 /var/www/api/build-369/vendor/laravel/framework/src/Illuminate/Console/Command.php(169): Illuminate\Container\Container->call(Array)
#17 /var/www/api/build-369/vendor/symfony/console/Command/Command.php(256): Illuminate\Console\Command->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#18 /var/www/api/build-369/vendor/laravel/framework/src/Illuminate/Console/Command.php(155): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#19 /var/www/api/build-369/vendor/symfony/console/Application.php(794): Illuminate\Console\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#20 /var/www/api/build-369/vendor/symfony/console/Application.php(186): Symfony\Component\Console\Application->doRunCommand(Object(Illuminate\Queue\Console\WorkCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#21 /var/www/api/build-369/vendor/symfony/console/Application.php(117): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#22 /var/www/api/build-369/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(107): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#23 /var/www/api/build-369/artisan(35): Illuminate\Foundation\Console\Kernel->handle(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#24 {main}
'

https://github.com/laravel/framework/blob/v5.2.40/src/Illuminate/Database/Connection.php#L453
pdo here returns null

@vaidas-lungis The workaround is probably to not use --daemon

@vaidas-lungis Workarounds are ugly

calling die() when you get a PDOException inside the worker
manually restarting the worker when you receive too many errors on your monitoring tool
Not using --daemon, or restarting the worker every X minutes

Yea I used second option.
listen command will use more CPU as it loops work command?

@vaidas-lungis Probably. My personal opinion is that the daemon mode is an abomination. Php is meant to die. The daemon mode introduces complex scenarios where you need to handle dead database connections, memory limits and garbage collection, and other nasty things regarding long running php scripts. All these need to be handled. Basically, it adds complexity to save some cpu.
The queue documentation has removed any mentions of the daemon workers in 5.3, and the supervisor configuration example no longer contains --daemon. I do not know if this was an oversight, or intentional.

There’s several issues that just make PHP the wrong tool for this. Remember, PHP will die, no matter how hard you try. First and foremost, there’s the issue of memory leaks. PHP never cared to free memory once it’s not used anymore, because everything will be freed at the end – by dying. In a continually-running process, that will slowly keep increasing the allocated memory (which is, in fact, wasted memory), until reaching PHP’s memory_limit value and killing your process without a warning. You did nothing wrong, except expecting the process to live forever. Under load, replace the “slowly” part for “pretty quickly”.

But more importantly, keeping a program running forever was never PHP’s top priority, that’s why problems like the above one weren’t never really solved.

Source: https://software-gunslinger.tumblr.com/post/47131406821/php-is-meant-to-die

Hmm, seems we only do that for query executions, not acquiring transactions.

This might be what's causing my queue problems.

I have supervisor set up to run php artisan queue:work --tries=3 and I'm seeing this error in my laravel.log every 3 seconds (implicit --sleep=3). This is preventing my queued jobs from running.
[2016-09-15 09:30:12] ERROR: exception 'ErrorException' with message 'Error while sending QUERY packet. PID=12902' in /example/vendor/laravel/framework/src/Illuminate/Database/Connection.php:610
Stack trace:
#0 [internal function]: Illuminate\Foundation\Bootstrap\HandleExceptions->handleError(2, 'Error while sen...', '/example/...', 610, Array)
#1 /example/vendor/laravel/framework/src/Illuminate/Database/Connection.php(610): PDO->beginTransaction()
#2 /example/vendor/laravel/framework/src/Illuminate/Queue/DatabaseQueue.php(175): Illuminate\Database\Connection->beginTransaction()
#3 /example/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(184): Illuminate\Queue\DatabaseQueue->pop('default')
#4 /example/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(155): Illuminate\Queue\Worker->getNextJob(Object(Illuminate\Queue\DatabaseQueue), 'default')
...

Is the cause that beginTransaction() is not checking if it needs to reconnect() first?
If it helps, I can run php artisan queue:work --once to manually process a job without any issue. Also, I'm running Laravel 5.3.8 and using MySQL.

I'm running Laravel 5.3.9 with MySQL, and have the same issue as @nhowell . I'm also running several Laravel 5.1 projects, but none of them has such issue. Below is the stack trace.
[2016-09-25 10:56:41] .ERROR: ErrorException: Error while sending QUERY packet. PID=22891 in /example/vendor/laravel/framework/src/Illuminate/Database/Connection.php:608
Stack trace:
#0 [internal function]: Illuminate\Foundation\Bootstrap\HandleExceptions->handleError(2, 'Error while sen...', '/var/www/vhosts...', 608, Array)
#1 /example/vendor/laravel/framework/src/Illuminate/Database/Connection.php(608): PDO->beginTransaction()
#2 /example/vendor/laravel/framework/src/Illuminate/Queue/DatabaseQueue.php(175): Illuminate\Database\Connection->beginTransaction()
#3 /example/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(184): Illuminate\Queue\DatabaseQueue->pop('user')
#4 /example/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(154): Illuminate\Queue\Worker->getNextJob(Object(Illuminate\Queue\DatabaseQueue), 'user')
#5 /example/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(112): Illuminate\Queue\Worker->runNextJob('database', 'user', Object(Illuminate\Queue\WorkerOptions))
#6 /example/vendor/laravel/framework/src/Illuminate/Queue/Worker.php(74): Illuminate\Queue\Worker->runNextJobForDaemon('database', 'user', Object(Illuminate\Queue\WorkerOptions))
#7 /example/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(97): Illuminate\Queue\Worker->daemon('database', 'user', Object(Illuminate\Queue\WorkerOptions))
#8 /example/vendor/laravel/framework/src/Illuminate/Queue/Console/WorkCommand.php(78): Illuminate\Queue\Console\WorkCommand->runWorker('database', 'user')
#9 [internal function]: Illuminate\Queue\Console\WorkCommand->fire()
#10 /example/vendor/laravel/framework/src/Illuminate/Container/Container.php(507): call_user_func_array(Array, Array)
#11 /example/vendor/laravel/framework/src/Illuminate/Console/Command.php(169): Illuminate\Container\Container->call(Array)
#12 /example/vendor/symfony/console/Command/Command.php(256): Illuminate\Console\Command->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#13 /example/vendor/laravel/framework/src/Illuminate/Console/Command.php(155): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#14 /example/vendor/symfony/console/Application.php(818): Illuminate\Console\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#15 /example/vendor/symfony/console/Application.php(186): Symfony\Component\Console\Application->doRunCommand(Object(Illuminate\Queue\Console\WorkCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#16 /example/vendor/symfony/console/Application.php(117): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#17 /example/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(122): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#18 /example/artisan(35): Illuminate\Foundation\Console\Kernel->handle(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#19 {main}

This issue should be fixed by https://github.com/laravel/framework/pull/15511, please ping me if it's not.

@zhouyixiang  I am running L 5.3.10 / MySQL 5.7.16-0ubuntu<IP_ADDRESS>, app is connecting without problems but when I run CLI php artisan queue:work I am getting same error, strange :/ any idea to fix?

@pun-ky Updating to at least Laravel v5.3.19 should fix it.

@themsaid
I still having this issue on Laravel 5.2. I'm running Queue Daemon using DB Queue driver, monitored by Supervisord.
I'm just having 500 error messages like this everyday on my log:
PDOException: SQLSTATE[HY000]: General error: 2006 MySQL server has gone away in /home/cdt/www/[dont-care]/releases/20161207205655/vendor/laravel/framework/src/Illuminate/Database/Connection.php:584 Stack trace: #0 /home/cdt/www/[dont-care]/releases/20161207205655/vendor/laravel/framework/src/Illuminate/Database/Connection.php(584): PDO->exec('SAVEPOINT trans...') #1 /home/cdt/www/[dont-care]/releases/20161207205655/vendor/laravel/framework/src/Illuminate/Queue/DatabaseQueue.php(162): Illuminate\Database\Connection->beginTransaction() #2

I still having this issue on Laravel 5.2

@iget-esoares There's your problem ;)
Laravel 5.2 does not have the #15511 fix.

@nhowell Do you know if this will be fixed on L5.2? If so I can do a PR.

Laravel 5.2 is no longer supported. Can you upgrade to Laravel 5.3?

@sisve, Yes I can, but it will take a few days to migrate and test since this is an old project that doesn't use unit testing. :(

@iget-esoares  I'm afraid you'll need to upgrade :)

@themsaid Is the fix going to be rolled into the LTS version of Laravel 5.1?

@njovin The mentioned pull request is already targetting 5.1.

I am using 5.4 and still getting this issue. Same setup as @iget-esoares . After dropping a dev database and creating a new one, whilst laravel is running.  Is there anything I can add to my superviser conf file to prevent it though?
[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /home/daryl/php_code/allnatives54/artisan queue:work database --sleep=3 --tries=3
autostart=true
autorestart=true
user=daryl
numprocs=3
redirect_stderr=true
stdout_logfile=/var/log/laravel-worker.log

I am runnig Laravl 5.3.31 and I have the same problem with queue DB driver and connection lost. Shoul it by fixed in 5.3.18?

@dusansalaj - same here, you find a solution?

@themsaid The issue still happening on 5.3.31 on some situations.
Yesterday I've finally upgraded from 5.2 to 5.3 just to get this bug solved.
Before this upgrade, running the daemon always causes the "MYSQL has gone away"  error after the connection timeout.
Now, the error don't occour anymore on timeout, BUT, if I restart the mysql server, it cause infinite "MYSQL has gone away" again.

@themsaid @GrahamCampbell Please take a look here
    public function beginTransaction()
    {
        if ($this->transactions == 0) {
            try {
                $this->getPdo()->beginTransaction();
            } catch (Exception $e) {
                if ($this->causedByLostConnection($e)) {
                    $this->reconnect();
                    $this->pdo->beginTransaction();
                } else {
                    throw $e;
                }
            }
        } elseif ($this->transactions >= 1 && $this->queryGrammar->supportsSavepoints()) {
            $this->getPdo()->exec(
                $this->queryGrammar->compileSavepoint('trans'.($this->transactions + 1))
            );
        }

        ++$this->transactions;

        $this->fireConnectionEvent('beganTransaction');
    }

Note that the connection fail will only be caught if transactions count is zero.
My logs point the error to the line 623 (the exec inside the elseif condition).
Since there's not a try-catch block here, the lost connections won't be detected and the Queue Woker will throw infinite errors on the log but will not reconnect.

+1
Same error with 5.4 and PHP 5.6
Reopened #19072

I know that its been a long time that this discussion had its last reply; we currently have the same issue and trying to investigate whats wrong... below i'll explain what we've seen so far.
With the following setup (forge provisioned on DO):
PHP (7.2.19)
MySQL (5.7.27)
laravel/framework (v5.8.4)
doctrine/orm (v2.6.3)
sentry/sentry (1.10.0)
sentry/sentry-laravel (0.8.1)
Only visible while dispatching through the horizon queue worker; like already said by other people it's not seen in horizon as an failed job but that's probably because of the architect of the queuing system itself. When failing (throwing an exception) laravel horizon will try the job again. What will happen on the background is as follows:
Currently a fresh setup; no open connections to the database (no sleeps) except for the "show processlist" command via CLI:
mysql> show processlist; +----+-------+-----------+------+---------+------+----------+------------------+ | Id | User  | Host      | db   | Command | Time | State    | Info             | +----+-------+-----------+------+---------+------+----------+------------------+ | 47 | forge | localhost | NULL | Query   |    0 | starting | show processlist | +----+-------+-----------+------+---------+------+----------+------------------+ 1 row in set (0.00 sec)
also Sentry is empty and no errors are logged yet. Then while running a background tasks via the CLI which simply checks the database for any expired invite tokens to be removed the issue occurs.
Inside Sentry the exception SQLSTATE[HY000]: General error: 2006 MySQL server has gone away has been received and if we check the processlist we see the following:
mysql> show processlist; +-----+-------+-----------------+-------+---------+------+----------+------------------+ | Id  | User  | Host            | db    | Command | Time | State    | Info             | +-----+-------+-----------------+-------+---------+------+----------+------------------+ |  47 | forge | localhost       | NULL  | Query   |    0 | starting | show processlist | | 604 | forge | localhost:40066 | forge | Sleep   |   69 |          | NULL             | +-----+-------+-----------------+-------+---------+------+----------+------------------+ 2 rows in set (0.00 sec)
So a new connection with PID 604 has been made and is idling right now... so when running the same script again we don't get an error in Sentry anymore since the system is using the same PID and refreshed the idling time as followed:
mysql> show processlist; +-----+-------+-----------------+-------+---------+------+----------+------------------+ | Id  | User  | Host            | db    | Command | Time | State    | Info             | +-----+-------+-----------------+-------+---------+------+----------+------------------+ |  47 | forge | localhost       | NULL  | Query   |    0 | starting | show processlist | | 652 | forge | localhost:40426 | forge | Sleep   |   11 |          | NULL             | +-----+-------+-----------------+-------+---------+------+----------+------------------+ 2 rows in set (0.00 sec)
So for what i can see is that when there is no current active or idling thread in mysql (also if you kill it on purpose) the issue occurs. Of course horizon is passing because of the second try, at that moment there is an active connection been made which is idling).
But if you kill it or don't have an connection available it will occur again... I saw some people talking about the framework which should reconnect if that happens but in my opinion we are never getting at that point in the code... or at least it looks like that... the exception has been thrown and is obvious being logged by Sentry and catched by horizon to retry again... so the framework wasn't involved at this moment. Also if this is an wider issue we should see it while connecting to the db in general via the framework but it only happens while queueing jobs and mostly when they are triggered by and CLI command inside Laravel.
Maybe this help a little further in finding a solution for this issue... but it looks like that either the framework/horizon or mysql is remembering somehow the PID number from a last alive connection (which has expired in the meantime) or booting mysql connections from queued workers isn't working correctly...
Either way i'll hope we can solve this issue some day.


Re-evaluate the best strategy for dealing with checks that validade the coherence between METADATA.pb and data on the font binaries

On #2210 a reasonable request was made: that both METADATA.pb and name table values should be validated by the same check. That issue in particular refered to the expected copyright notice strings, but the same could be true about other checks of METADATA.pb values that need to be coherent with the contents of the font binaries.
Our current approach up to now was to have complementary checks that performed the same validation on different places. For instance, one check could enforce the string format on METADATA.pb while another check would simply expect the value to be equal (without explicitly re-validating the strings).
Another approach would be what we just did on #2210.
One problem, though, is that font_metadata is a condition for running those METADATA.pb specific checks and that means most upstream font-projects won't run those checks at all. So I think in that case we should make font_metadata optional, but emit a WARN.
Because of it, this approach could end up complicating the codebase, so we should be careful before going that route.

in https://github.com/googlefonts/gftools/issues/200#issuecomment-631513002 I wrote,

@graphicore and @m4rc1e I just had a video call about this; we resolved that there are dozens of metadata checks in the gf profile that need to be refactored, as there appears to be a lot of redundancy, and Lasse will do this along with refactoring other parts of the gf profile, and the underlying fontbakery core code, related to super/sibling families, in June.

Seems this was a duplicate of this issue, so bumping to P0 priority

yes, indeed! Thanks, Dave

Generally, the metadata files need to be accurate to what is in the font,
so if the fonts have X and the METADATA file has Y, we should prefer X.

Also, METADATA.pb checks are 100% specific to the gfonts profile, while the corresponding checks for font binary data may be generaly useful on other profiles (ideally universal).
So having font binary data as the "source of truth" and keeping separate checks for binaries and metadata, would seem to be the way to go, in my opinion.
Does it change the perception of there being lots of redundancy? Or are there any other aspects to such redundancy you've been discussing in that meeting?

I've just assigned this to @m4rc1e because it seems to be an issue closely coupled to his work on refactoring name table checks, as seen at #3404

I don't think we should do this; we want to be able to check the name entries regardless of the presence or absence of a METADATA.pb file, so the check should be independent. If anything we want to coalesce checks which touch the same file (i.e. do all the METADATA.pb things at once instead of loading/parsing it multiple times) rather than coalesce across files which may or may not be present.


拉取最新代码模块调整pom中模块没有去掉

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
拉取最新代码模块调整pom中模块没有去掉
Expected Behavior

Steps To Reproduce
No response
Environment
ShenYu version(s):

Debug logs
No response
Anything else?
No response

why remove？ pls in english

There is no corresponding module under the parent POM。this is lastest master branch

can you modify this issue title?
more plugin，you can not clone success,  you can see this : https://shenyu.apache.org/community/faq


Custom localization

It is super great that the package includes multiple languages by setting the local but still, I couldn't find the way to configure the package to use the custom language file.
Is there a way to publish the language files and modify them as needed?
Thanks

you can custom lang file with set message function and modify message
setMessages(string $locale, array $messages)


Why are the docs not updated?

This commit fixed a broken link: https://github.com/elm/random/commit/a1eae055f72db1269284337f81d7fdca42133612 . This was back in 2019. It is now 2021 The version of the docs at https://package.elm-lang.org/packages/elm/random/latest/Random is still broken

Probably Evan doesn't have time to handle all PRs. I have noticed that there is another member, who can contribute to Elm project. Maybe @terezka can help with all PRs in elm/random.


python310Packages.pymicrobot: 0.0.1 -> 0.0.3

Description of changes
https://github.com/spycle/pyMicroBot/releases/tag/v0.0.3

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 186712 run on x86_64-linux 1

2 packages built:

python310Packages.pymicrobot
python39Packages.pymicrobot


404 Custom error handler

I would remove it from the archetype definition because this is a per server configuration and not a per project customization: if two people of the same team are working on different projects and they started their work from this archetype, they could think that every one is working on "his own" 404 error handler.
I would propose to create a new, per AEM instance, archetype with all best practices and ideas, like the 404 error handler valid for a whole AEM instance, not for a single project: does it make sense?

@artika4biz, Thank you for bringing up this idea. We will take care of it.
CQ-4250956


Typescript definitions

Hello, Thomas!
Have you considered to provide and maintain definition file for typescript?
Thanks.

Yes, it is something I'm aware would be nice to provide. I don't use TypeScript, and would be happy to help any effort.

These are working pretty well for me, I ended out having to write my own for rxjs-router5 though: https://www.npmjs.com/package/@types/router5

@slively I think those definitions are from v3 (v4 was an internal refactor).

I made https://gist.github.com/Keats/1b8833b581d01e751048e1f51041817f
It's not complete though

@Keats that's really helpful. Would you mind starting a PR?

Yep will do so in a bit

Anyone good at writing definition files or has an example of definition that works for something like https://gist.github.com/Keats/1b8833b581d01e751048e1f51041817f#file-router5-d-ts-L132 ?
I'm pretty sure the declare module should not be there.

Added to v4.6.0


Error: Minified React error

Thanks! This seems to be a duplicate of another issue on this repo. I'll close this issue for now and we'll continue this discussion there.


Clusterupdate : clustering of deleted sequences and conversion to tsv file

Expected Behavior
I want to update my clusters after a database update (in which I add new sequences but also delete sequences compared to the old database).
The clusterupdate command works, but when I try to convert the cluster database to a tsv file, I have an error message related to the index (see below).
I tried the same thing on a new database where I just added sequences and it worked perfectly, so I assume the problem comes from the fact that I remove sequences from the old database?
Current Behavior
Error when trying to generate the tsv file.
In the cluster database obtained after clusterupdate ('CLU_updated') the removed sequences still appear, but they are absent of the updated sequence database ('DB_updated').
Steps to Reproduce (for bugs)

Creation of old DB (oldDB.fa : 17 amino acid sequences)
 mmseqs createdb oldDB.fa DB_old

Clustering of old DB
mmseqs cluster DB_old CLU_old tmp

Creation of new DB (newDB.fa : 13 sequences are identical with the old DB, 4 were removed, 4 were added)
mmseqs createdb newDB.fa DB_new

Cluster update
mmseqs clusterupdate DB_old DB_new CLU_old DB_updated CLU_updated tmp
No error there, but even though sequences of numeric identifiers 12 , 11 , 16 , 15 in the old db have been removed, they appear in the CLU_updated file. They do not appear in the DB_updated files.

Conversion of cluster DB in tsv :
mmseqs createtsv DB_updated DB_updated CLU_updated clusters.tsv
=> Error message, generation of empty files : clusters.tsv.1 ... clusters.tsv.7 and clusters.tsv.index.1 ... clusters.tsv.index.7

MMseqs Output (for bugs)
Program call:
createtsv DB_updated DB_updated CLU_updated clusters.tsv

MMseqs Version:                  	2f66ae897fc813450fa5ef0c78123bd3c41c4717
first sequence as respresentative	false
Target column                    	1
Add Full Header                  	false
Database Output                  	false
Threads                          	8
Compressed                       	0
Verbosity                        	3

Query database: DB_updated
Touch data file DB_updated_h ... Done.
Result database: CLU_updated
Start writing to clusters.tsv
Invalid database read for database data file=DB_updated_h, database index=DB_updated_h.index
getData: local id<PHONE_NUMBER>) >= db size (17)

Context
Providing context helps us come up with a solution and improve our documentation for the future.
Your Environment

Git commit used: 2f66ae897fc813450fa5ef0c78123bd3c41c4717
Which MMseqs version was used: Compilation from source
Cmake versions used: cmake version 3.5.1
Operating system and version: Ubuntu 16.04 LTS

Thank you in advance for your help :)

Sorry for the delay, would you mind uploading the two FASTA files (with the 17 seq each)?
I started refactoring the code and think I know whats wrong.

Here are the fastas .
Thanks for your help
fastas.zip

Thanks, I can reproduce the issue. I'll have to investigate whats going wrong.
Meanwhile, if you want a set of stickers (see https://twitter.com/thesteinegger/status/1201076220957315074), send me your postal address to milot at mirdita de.

I've found out that we are not dealing well with deleted sequences.
Their presence in the clustering is resulting in the error you are seeing. I am refactoring some code, but it turns out to be a bit more work.

@milot-mirdita I'm running into this same issue. Any update on progress?

I appear to be getting a similar error:
$ mmseqs clusterupdate --min-seq-id 0.9 -c 0.8           ../tests/output_n10/genes/cluster/genes_db /ebio/abt3_scratch/nyoungblut/user_genes/genes_db ../tests/output_n10/genes/cluster/clusters_db           /ebio/abt3_scratch/nyoungblut/cluster_updated/genes_db /ebio/abt3_scratch/nyoungblut/cluster_updated/clusters_db.0 /ebio/abt3_scratch/nyoungblut/cluster_update
clusterupdate --min-seq-id 0.9 -c 0.8 ../tests/output_n10/genes/cluster/genes_db /ebio/abt3_scratch/nyoungblut/user_genes/genes_db ../tests/output_n10/genes/cluster/clusters_db /ebio/abt3_scratch/nyoungblut/cluster_updated/genes_db /ebio/abt3_scratch/nyoungblut/cluster_updated/clusters_db.0 /ebio/abt3_scratch/nyoungblut/cluster_update

MMseqs Version:                     	11.e1a1c
Seq. id. threshold                  	0.9
Coverage threshold                  	0.8

===================================================
=== Update the new sequences with the old keys ====
===================================================
===================================================
====== Filter out the new from old sequences ======
===================================================
===================================================
======= Extract representative sequences ==========
===================================================
result2repseq /ebio/abt3_projects/software/dev/Struo2/tests/output_n10/genes/cluster/genes_db /ebio/abt3_projects/software/dev/Struo2/tests/output_n10/genes/cluster/clusters_db /ebio/abt3_scratch/nyoungblut/Struo2_122419461619/cluster_update/7316799743718053916/OLDDB.repSeq

Invalid database read for database data file=/ebio/abt3_projects/software/dev/Struo2/tests/output_n10/genes/cluster/clusters_db, database index=/ebio/abt3_projects/software/dev/Struo2/tests/output_n10/genes/cluster/clusters_db.index
31mInvalid database read for database data file=/ebio/abt3_projects/software/dev/Struo2/tests/output_n10/genes/cluster/clusters_db, database index=/ebio/abt3_projects/software/dev/Struo2/tests/output_n10/genes/cluster/clusters_db.index

[... a lot of output ...]

31mSize of data: 363542
mRequested offset: 412570
Requested offset: 399738
31mRequested offset: 367758
Requested offset: 408364
31mRequested offset: 386682
39mRequested offset: 393723
mRequested offset: 403458
Requested offset: 381782
39mRequested offset: 413970
mRequested offset: 406964
m31Requested offset: 398528
Requested offset: 367053
mRequested offset: 415370
Error: result2repseq died

conda env:
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       0_gnu    conda-forge
bzip2                     1.0.8                h516909a_2    conda-forge
ca-certificates           2020.6.20            hecda079_0    conda-forge
certifi                   2020.6.20        py38h32f6830_0    conda-forge
fasta-splitter            0.2.6                         0    bioconda
gawk                      5.1.0                h516909a_0    conda-forge
gettext                   <IP_ADDRESS>          hc5be6a0_1002    conda-forge
ld_impl_linux-64          2.34                 h53a641e_7    conda-forge
libblas                   3.8.0               17_openblas    conda-forge
libcblas                  3.8.0               17_openblas    conda-forge
libffi                    3.2.1             he1b5a44_1007    conda-forge
libgcc-ng                 9.2.0                h24d8f2e_2    conda-forge
libgfortran-ng            7.5.0                hdf63c60_6    conda-forge
libgomp                   9.2.0                h24d8f2e_2    conda-forge
libidn2                   2.3.0                h516909a_0    conda-forge
liblapack                 3.8.0               17_openblas    conda-forge
libopenblas               0.3.10          pthreads_hb3c22a3_3    conda-forge
libstdcxx-ng              9.2.0                hdf63c60_2    conda-forge
libunistring              0.9.10               h14c3975_0    conda-forge
llvm-openmp               8.0.1                hc9558a2_0    conda-forge
mmseqs2                   11.e1a1c             h2d02072_0    bioconda
ncurses                   6.2                  he1b5a44_1    conda-forge
numpy                     1.19.0           py38h8854b6b_0    conda-forge
openmp                    8.0.1                         0    conda-forge
openssl                   1.1.1g               h516909a_0    conda-forge
perl                      5.26.2            h516909a_1006    conda-forge
perl-constant             1.33                    pl526_1    bioconda
perl-exporter             5.72                    pl526_1    bioconda
perl-file-util            4.161950                pl526_3    bioconda
perl-lib                  0.63                    pl526_1    bioconda
pigz                      2.3.4                hed695b0_1    conda-forge
pip                       20.1.1                     py_1    conda-forge
prodigal                  2.6.3                h516909a_2    bioconda
python                    3.8.4           cpython_h425cb1d_0    conda-forge
python_abi                3.8                      1_cp38    conda-forge
readline                  8.0                  he28a2e2_2    conda-forge
seqkit                    0.13.2                        0    bioconda
setuptools                49.2.0           py38h32f6830_0    conda-forge
sqlite                    3.32.3               hcee41ef_1    conda-forge
tk                        8.6.10               hed695b0_0    conda-forge
vsearch                   2.15.0               h2d02072_0    bioconda
wget                      1.20.1               h22169c7_0    conda-forge
wheel                     0.34.2                     py_1    conda-forge
xz                        5.2.5                h516909a_1    conda-forge
zlib                      1.2.11            h516909a_1006    conda-forge

Dealing with deleted sequences is currently still broken. I had begun working on it, but didn't have time to finish up the work.

Thanks for the quick update! FYI: there doesn't seem to be any documentation about the differences between concatdbs and mergedbs

This took some time, but dealing with deleted sequences should hopefully work correctly now.


Use AbstractGPs

The last bit of this changeover is blocked by https://github.com/JuliaGaussianProcesses/KernelFunctions.jl/pull/208


Standalone NSFS read_rand_seed: error TypeError: fh.read is not a function

Environment info

NooBaa Version: master-20210427
Platform: Standalone

Actual behavior

$ brew reinstall noobaa-core
...
$ noobaa-core nsfs .
Apr-27 10:59:10.985 [Endpoint/86589]    [L0] core.rpc.rpc:: RPC register_n2n_proxy
Apr-27 10:59:11.271 [Endpoint/86589]    [L0] core.endpoint.endpoint:: Configured Virtual Hosts: []
Apr-27 10:59:11.271 [Endpoint/86589]    [L0] core.endpoint.endpoint:: Configured Location Info: { region: '' }
Apr-27 10:59:11.272 [Endpoint/86589]   [LOG] CONSOLE:: read_rand_seed: reading 32 bytes from /dev/random ...
Apr-27 10:59:11.273 [Endpoint/86589]   [LOG] CONSOLE:: read_rand_seed: error TypeError: fh.read is not a function
    at read_rand_seed (/snapshot/noobaa-core-20210427-80602-1didalo/src/util/nb_native.js:86:44)
    at async init_rand_seed (/snapshot/noobaa-core-20210427-80602-1didalo/src/util/nb_native.js:51:18)
Apr-27 10:59:11.273 [Endpoint/86589]   [LOG] CONSOLE:: read_rand_seed: closing fd ...
Apr-27 10:59:11.273 [Endpoint/86589]   [LOG] CONSOLE:: read_rand_seed: closing fd error TypeError: fh.close is not a function
    at clean_fh (/snapshot/noobaa-core-20210427-80602-1didalo/src/util/nb_native.js:69:22)
    at read_rand_seed (/snapshot/noobaa-core-20210427-80602-1didalo/src/util/nb_native.js:91:19)
    at async init_rand_seed (/snapshot/noobaa-core-20210427-80602-1didalo/src/util/nb_native.js:51:18)
Apr-27 10:59:11.273 [Endpoint/86589]   [LOG] CONSOLE:: read_rand_seed: delay before retry

Expected behavior

Should not fail - not sure why this fs/promises open file handle flow is different after using pkg.

Steps to reproduce

see above.

More information - Screenshots / Logs / Other output

https://github.com/vercel/pkg/issues/1162

@jeniawhite Until the issue from pkg is resolved, perhaps we can just identify that await fs.promises.open() returns a number (fd) instead of object and return from the read_rand_seed function? this will avoid making redundant changes to our code and just wait until the issue is resolved by skipping the random see for standalone.
https://github.com/noobaa/noobaa-core/blob/efe45cfffee2cc4b0b24671743013b5ddc5cca43/src/util/nb_native.js#L83


when ... is should be a parse error if the body isn't tabbed.

This is valid currently:
    when color is
    Red -> (value, g, b)
    Green -> (r, value, b)
    Blue -> (r, g, value)

It probably shouldn't be. We probably want to require that the body of the when ... is is tabbed in compared tot the rest of the function.

this is probably a good first issue, but I don't know the parser enough to know for sure.

This may be a deliberate decision by @joshuawarner32, I know he likes the parser to be lenient.

I've been chatting with @rtfeldman recently about how we can make the parser mostly or completely indentation-agnostic  (so long as it's not ambiguous or likely to cause user confusion) - and rely on the auto-formatter to confirm communicate to the user how the compiler is interpreting the syntax. So - yes, that's loosely the direction I've been thinking in.
That said, glancing at the code, I'm a little fuzzy on why this is happening. The code seems to be wanting at least one space there.

Closing for now as this is behaving in the intended way -- it is parsing and formatting as Josh described.
@bhansconnect let me know if you think this is incorrect, and we can re-open.
main =
    color = Red
    msg =
        when color is
        Red -> "(value, g, b)"
        Green -> "(r, value, b)"
        Blue -> "(r, g, value)"

    Stdout.line msg

Formats to...
main =
    color = Red
    msg =
        when color is
            Red -> "(value, g, b)"
            Green -> "(r, value, b)"
            Blue -> "(r, g, value)"

    Stdout.line msg


Adding tls-client support and check logic

Lots of lint cleanup, fix tcp server/client issues and added the ability to specify a custom hash of options for the server.

Thanks for your work, @justicel :beers:
I added some more tests and will merge it via #147.


Deploy CakePHP cache clearing

Changed deployment script to clear CakePHP's cache during application install as the last task and during application update as the first and last task.

Codecov Report

Merging #510 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##             master    #510   +/-   ##
========================================
  Coverage      29.3%   29.3%
  Complexity      912     912
========================================
  Files            84      84
  Lines          3112    3112
========================================
  Hits            912     912
  Misses         2200    2200

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1a15109...5032016. Read the comment docs.


Very slow data loading

Not clear if I can ask a question here.
When the first one was tried, it took quite long for data loading (not for downloading thru internet).

Is this simply because of the file size?
Is there a way of doing it faster?

Thanks for sharing the code. I have been trying to learn pytorch and this site is helping me a lot.

Questions are always welcome!
I believe the issue is due to spaCy being very, very slow. You can change the tokenization for the sentences from using the spacy tokenizer to a very basic custom tokenizer which will be a lot faster, but I believe you will get worse results. Your custom tokenizers have to be functions that take in a string and return a list of strings (the tokenized example).
For example, we can change:
TEXT = data.Field(tokenize='spacy')
LABEL = data.LabelField(dtype=torch.float)

to:
def tokenize(s):
    return s.split(' ')

TEXT = data.Field(tokenize=tokenize)
LABEL = data.LabelField(dtype=torch.float)

This tokenizer simply splits the string on spaces. It won't be as good as the spaCy one, but will be a lot faster!

I see.
So it would be better for me to do offline tokenizing and save the data. Then load the data with the simple split(' ') tokenizer. Because most of the experiments are on developing a neural network, preprocessing the data may be a way of separating the whole process.
I really appreciate your kind answer.

Yep, that'd be a lot faster.
You can run the following code once:
from torchtext import data
from torchtext import datasets
import json

TEXT = data.Field(tokenize='spacy')
LABEL = data.LabelField()
train_data, test_data = datasets.IMDB.splits(TEXT, LABEL)

train_examples = [vars(t) for t in train_data]
test_examples = [vars(t) for t in test_data]

with open('.data/train.json', 'w+') as f:
    for example in train_examples:
        json.dump(example, f)
        f.write('\n')

with open('.data/test.json', 'w+') as f:
    for example in test_examples:
        json.dump(example, f)
        f.write('\n')

This will use spaCy to tokenize all of the examples and then save each in the json lines format. We can then replace loading the IMDB dataset with:
TEXT = data.Field()
LABEL = data.LabelField()

fields = {'text': ('text', TEXT), 'label': ('label', LABEL)}

train_data, test_data = data.TabularDataset.splits(
    path = '.data',
    train = 'train.json',
    test = 'test.json',
    format = 'json',
    fields = fields
)

As the dataset has already been tokenized this should be very quick (~2 seconds on my machine, compared to the original which took ~5 minutes).

wow, this is great. I was simply trying to run spacy separately to save the result, but had no idea of using JSON format thru TabularDataset.
Thanks a lot.

 in
7     start_time = time.time()
8
----> 9     train_loss, train_acc = train(model, train_iterator, optimizer, criterion)
10 #     valid_loss, valid_acc = evaluate(model, valid_iterator, criterion)
11
 in train(model, iterator, optimizer, criterion)
12         predictions = model(batch.text).squeeze(1)
13
---> 14         loss = criterion(predictions, batch.label)
15
16         acc = binary_accuracy(predictions, batch.label)
C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\module.py in call(self, *input, **kwargs)
530             result = self._slow_forward(*input, **kwargs)
531         else:
--> 532             result = self.forward(*input, **kwargs)
533         for hook in self._forward_hooks.values():
534             hook_result = hook(self, input, result)
C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\modules\loss.py in forward(self, input, target)
599                                                   self.weight,
600                                                   pos_weight=self.pos_weight,
--> 601                                                   reduction=self.reduction)
602
603
C:\ProgramData\Anaconda3\lib\site-packages\torch\nn\functional.py in binary_cross_entropy_with_logits(input, target, weight, size_average, reduce, reduction, pos_weight)
2122
2123     if not (target.size() == input.size()):
-> 2124         raise ValueError("Target size ({}) must be the same as input size ({})".format(target.size(), input.size()))
2125
2126     return torch.binary_cross_entropy_with_logits(input, target, weight, pos_weight, reduction_enum)
ValueError: Target size (torch.Size([64])) must be the same as input size (torch.Size([970]))
why do i get this error?


Add 'note' parameter to coupon codes

Tested it and it works fine

The @SerializedName annotation is probably unnecessary

While we appreciate all PRs, we do not guarantee that any specific PR will be merged. With any PR the code has to go through out internal processes, including review, testing and test builds, and as such we are unable to predict timescales for this - likely to be when we are working on other features at the same time.

It's literally 3 lines of code, how much reviewing and testing do you need for 3 lines of code???????????????????

Regardless of the code quantity, we have the same internal process to go through. As per my previous message, we will most likely review any/all outstanding PRs when we are next working on the plugin internally.


Courses: FreeCodeCamp -JavaScript

Description: JavaScript is a high-level programming language that all modern web browsers support. It is also one of the core technologies of the web, along with HTML and CSS that you may have learned previously. This section will cover basic JavaScript programming concepts, which range from variables and arithmetic to objects and loops.
Official link: https://www.freecodecamp.org/learn/javascript-algorithms-and-data-structures/basic-javascript/

yess, you can add it.

Hey @Sidd2024 can you please specify the file structure where I can find the course section

@jose-jimmy you can find it on the website here
and if the HTML file here

@jose-jimmy you also have to update Added_Opportunities.md respectively...


request.defaults({ ... }).delete appears to be missing

I see code trying to pass it through at https://github.com/request/request/blob/2ef974a823eca5601d0e17eece4212c8bc8d01cc/index.js#L106, but locally (with version 2.71.0) I only have request.defaults({ ... }).del.  Is this intentional and the documentation should be updated?  Or is something else happening?

Without any true investigation, I wonder if https://github.com/request/request/blob/2ef974a823eca5601d0e17eece4212c8bc8d01cc/index.js#L73 is the issue?  Is there a difference between .del and [delete]?

Hi @ncalexan can you try with version 2.72? The delete method was added in that version.


Add V2 Storage Accounts to the unsupported limitations

As of now Storage Accounts V2 are not supported by Azure File Share Backup, please go ahead and add them to unsupported scenarios on the limitations section.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: bc12f304-5826-fb32-1728-5e43f1b05d8a
Version Independent ID: 7cc3d5a7-9b0e-dd6b-3b34-2036219f5bd6
Content: Support matrix for replicating VMware VMs and physical servers to Azure with Azure Site Recovery
Content Source: articles/site-recovery/vmware-physical-azure-support-matrix.md
Service: site-recovery
GitHub Login: @rayne-wiselman
Microsoft Alias: raynew

I put in the wrong threat it was meant to be added in https://docs.microsoft.com/en-us/azure/backup/troubleshoot-azure-files#limitations-for-azure-file-share-backup-during-preview

@saulcp95 Thanks for the feedback, I assigned the other thread to the author.


Debian stretch doesn't provide a version number

Its os-release file doesn't have a version number. To aid users, I would actually add it as part of the currently ongoing work on distro inconsistency handling.
WDYT?

:+1:

This can be closed.


docs: Add ApertureDB to vectorstore integrations page

Description: Add ApertureDB to vectorstore integrations page
**Issue:**none
Dependencies: none
Twitter handle: aperturedata

I was unable to lint or test because the development tools don't work and/or the documentation is wrong.
https://github.com/langchain-ai/langchain/discussions/28834
https://github.com/langchain-ai/langchain/discussions/28833

hey there! we're not adding new ones to this page without prior discussion, and we're coming up with a way to show more of the dedicated integration packages that folks are launching currently!
Also, if you're interested in maintaining this integration without us in the loop, we'd love to get an integration package out (this would also make it eligible for showing in any future tables that are powered by packages.yml)! Future PRs against langchain would just be {docs updates, as well as registering your package in libs/packages.yml, deprecating this community integration in favor of your integration package}
Here's the guide, and if you have questions, feel free to leave them in the comments on those pages so others can see them! https://python.langchain.com/docs/contributing/how_to/integrations/

we're not adding new ones to this page without prior discussion

Thanks for the feedback.  Where would this discussion take place?

Also, if you're interested in maintaining this integration without us in the loop, we'd love to get an integration package out (this would also make it eligible for showing in any future tables that are powered by packages.yml)! Future PRs against langchain would just be {docs updates, as well as registering your package in libs/packages.yml, deprecating this community integration in favor of your integration package}

I'll look into converting over to an integration package if that is now the recommended approach.


Add support for textAlign:start/end

Description
React Native is missing support for textAlign:start and textAlign:end on Text and TextInput elements - https://developer.mozilla.org/en-US/docs/Web/CSS/text-align#syntax
These values are important for supporting RTL layouts, and should react to the direction style on ancestral Views on all platforms.
Steps to reproduce

Set textAlign:end
Notice it doesn't work

React Native Version
0.75
Affected Platforms
Runtime - Android, Runtime - iOS
Output of npx react-native info
.

Stacktrace or Logs
.

Reproducer
.
Screenshots and Videos
No response

Hello, Can I try to resolve this issue?

@cortinico I can pick this up, but had a few points.
If we are to match web specs, then the way left and right currently behave don't work according to the specs. It internally checks whether the direction is RTL or LTR, and applies the opposite.
According to the CSS spec

left aligns the text to the left of the bounding box
right aligns the text to the right of the bounding box
start aligns the text to the left if LTR and right if RTL
end aligns the text to the right if LTR and left if RTL

Based on this, I can see two ways to do it:

Either accommodate for this change in the Text component itself
Fix this in the base text shadow node to match the spec

Which one would you recommend?

@cortinico I can pick this up, but had a few points.

those are good questions for @necolas

@necolas whenever you can, if you can validate these points then I will make the fix accordingly.
I have already done the same for Android, but awaiting confirmation will do the for iOS.

then the way left and right currently behave don't work according to the specs. It internally checks whether the direction is RTL or LTR, and applies the opposite

Fixing this might be a breaking change if you're saying left currently becomes right in an RTL context.

should react to the direction style on ancestral Views

I think the fix to the way direction works on Android is behind a feature flag. cc @NickGerleman

The above code mentioned is unfortunately only used in the legacy architecture. The new architecture has a different path for setting text direction, though I think it may also do some swapping (though IIRC maybe only when swapLeftAndRightInRTL is enabled, which I think is default too).

@NickGerleman in that case, can you point me to where this would be?
@necolas are we aligned that the fix for this should be inline with CSS spec? If so then I will start accordingly.

@necolas Hello, any update on this? I can pick this up accordingly.

Yes, inline with CSS spec

Cool, sorry for the delay was unwell. I'll pick this up over this weekend!


Do we want to use serde_json::Value in the public API?

https://github.com/virtee/kbs-types/blob/ec9b1b4c7908e72f6ff30146479f0eb2d7f943cb/src/lib.rs#L49
I guess the idea is to allow arbitrary, untyped json-like data on the extra_params field. I'm not sure that using the type of a serialization library on a public API is a good idea, since this will introduce a transitive dependency on serde_json and consumers might have to deal with version mismatches or having to perform extra conversion when using a different serialization library.
Alternatives would be to provie some sort of flexible tree structure, which doesn't have to be full json, or just type extra_params as Vec<u8>, since processors of extra_params have to perform a parsing step anyway, also with serde_json::Value.

For most of the packages in VirTEE we have been placing any serde related dependencies behind a feature flag. I am definitely open to either of these ideas. Let me look into this a bit deeper. It might make sense to allow various methods of passing the extra_params based upon the use-case.

@mkulke I think it would be fine to have serde_json::Value, but we need to pub use serde_json::Value to reexport the struct in kbs-types lib. Btw, the version of serde_json is marked "1" and no Cargo.lock given, thus the concrete version of serde_json would be determined by concrete project that imports kbs-types.
Another observation is that there are two parts that will consume the messages. The version of serde_json or serde differs between sender and receiver does not matter as the two codes does not belong to the same process/binary. wdyt?

if we use a serialization type as a data-type on the public structure, it will force consumers to pull in serde, serde_json, and serde_derive into their projects, which might be unacceptable in certain situations (restricted environments like fw or a limited set of allowed dependencies).
another nuisance could be version mismatch on a consuming crate if you have to use a pinned version and there are incompatibilities in the patch version.
error: failed to select a version for `serde_json`.
    ... required by package `kbs-types v0.7.0`
    ... which satisfies dependency `kbs-types = "^0.7.0"` of package `bla v0.1.0 (/home/magnuskulke/tmp/bla)`
versions that meet the requirements `^1.0` are: 1.0.45, 1.0.122, ...
the package `kbs-types` depends on `serde_json`, with features: `alloc` but `serde_json` does not have these features.

maybe the ergonomic improvements between raw bytes and untyped json aren't so big either, in both cases you have to serialize/deserialize:
let extra_params_json= json!({
    "extra": "data",
});
request.extra_params = extra_params_json;

...vs

#[repr(C)]
struct ExtraParams {
   "extra": String,
}
let extra_params = ExtraParams { extra: "data".into() };
request.extra_params = bincode::serialize(&extra_params).unwrap();

..and

let extra_params = serde_json::from_value::<ExtraParams>(request.extra_params).unwrap()
...vs
let extra_params: ExtraParams =  bincode::deserialize(&request.extra_params).unwrap();

@mkulke Yea, actually the change does not import any new crates like serde and serde_json as kbs-types already imports them for a long time. See the change
About dependency version conflict, the version is "1.0" in Cargo.toml. Due to the document, if kbs-types and another project both intend to use serde/serde_json, a version 1.x.x will be used. If you get any errors on it, just try cargo update -p serde_json and it will work I think.

I think the problem is putting a serialization type on the public api, not using serde in the project (although it should behind a feature flag, imo). We know how to work around this, the issues I'm referencing do not appear in unrestricted high-level code (like trustee) where you can just bump your dependencies freely, it's more about restricted low-level projects (like coconut svsm). If that's not a problem scope for kbs-types, we can close this issue. cc @larrydewey

Might be off of the topic -- in sigstore-rs, we use a same name "shim type" to "re-export" types from 3rd dep, like https://github.com/sigstore/sigstore-rs/blob/main/src/registry/config.rs#L33-L42


Only transform files that need to be transformed

This fixes a ton of weird bugs we are having, such as __name not defined in our webpack bundle, and made our code run much faster, while still working with ESM modules.

I think this change avoids the bugs, rather than fixing them, by preventing tsx from transforming certain files.
__name not defined is likely related to https://github.com/esbuild-kit/tsx/issues/113 but I don't see the relevance of Webpack. Can you elaborate on your issue there?
I would like to improve performance but I'm not sure if this is a good signal. What if the file has types or next generation syntax but no import/`export?
Also, please test your code before marking a PR as ready.

relevance of Webpack

This is my guess:

We overwrite the node module loader.
Webpack loads modules to put in bundles.
Webpack uses Function.toString() and eval to reduce functions into strings and save as bundles.
ESBuild doesn't support Function.toString() or eval for multiple reasons, such as variable names can be changed or scope lost.

Before this change, webpack-dev takes about 5 minutes to load the bundles. After this change about 30 seconds.

What if the file has types

I agree, we should look at the file extension and always transform non-".js" files.

next generation syntax but no import/export?

I'm never going to find the reference, but I remember reading part of the compromise for ESM to be permitted in files that have the .js extension is that the file must have an import or export statement.

Here's the generated webpack hot reloading code that fails because `__name` is not defined. I formatted it with prettier so it's readable.
var currentModuleData = {};
var installedModules = __webpack_require__.c;
var currentChildModule;
var currentParents = [];
var registeredStatusHandlers = [];
var currentStatus = "idle";
var blockingPromises = 0;
var blockingPromisesWaiting = [];
var currentUpdateApplyHandlers;
var queuedInvalidatedModules;
__webpack_require__.hmrD = currentModuleData;
__webpack_require__.i.push(function (options) {
  var module2 = options.module;
  var require2 = createRequire(options.require, options.id);
  module2.hot = createModuleHotObject(options.id, module2);
  module2.parents = currentParents;
  module2.children = [];
  currentParents = [];
  options.require = require2;
});
__webpack_require__.hmrC = {};
__webpack_require__.hmrI = {};
function createRequire(require2, moduleId) {
  var me = installedModules[moduleId];
  if (!me) return require2;
  var fn = __name(function (request) {
    if (me.hot.active) {
      if (installedModules[request]) {
        var parents = installedModules[request].parents;
        if (parents.indexOf(moduleId) === -1) {
          parents.push(moduleId);
        }
      } else {
        currentParents = [moduleId];
        currentChildModule = request;
      }
      if (me.children.indexOf(request) === -1) {
        me.children.push(request);
      }
    } else {
      console.warn(
        "[HMR] unexpected require(" +
          request +
          ") from disposed module " +
          moduleId
      );
      currentParents = [];
    }
    return require2(request);
  }, "fn");
  var createPropertyDescriptor = __name(function (name2) {
    return {
      configurable: true,
      enumerable: true,
      get: function () {
        return require2[name2];
      },
      set: function (value) {
        require2[name2] = value;
      },
    };
  }, "createPropertyDescriptor");
  for (var name in require2) {
    if (Object.prototype.hasOwnProperty.call(require2, name) && name !== "e") {
      Object.defineProperty(fn, name, createPropertyDescriptor(name));
    }
  }
  fn.e = function (chunkId) {
    return trackBlockingPromise(require2.e(chunkId));
  };
  return fn;
}
__name(createRequire, "createRequire");
function createModuleHotObject(moduleId, me) {
  var _main = currentChildModule !== moduleId;
  var hot = {
    _acceptedDependencies: {},
    _acceptedErrorHandlers: {},
    _declinedDependencies: {},
    _selfAccepted: false,
    _selfDeclined: false,
    _selfInvalidated: false,
    _disposeHandlers: [],
    _main,
    _requireSelf: function () {
      currentParents = me.parents.slice();
      currentChildModule = _main ? void 0 : moduleId;
      __webpack_require__(moduleId);
    },
    active: true,
    accept: function (dep, callback, errorHandler) {
      if (dep === void 0) hot._selfAccepted = true;
      else if (typeof dep === "function") hot._selfAccepted = dep;
      else if (typeof dep === "object" && dep !== null) {
        for (var i = 0; i < dep.length; i++) {
          hot._acceptedDependencies[dep[i]] = callback || function () {};
          hot._acceptedErrorHandlers[dep[i]] = errorHandler;
        }
      } else {
        hot._acceptedDependencies[dep] = callback || function () {};
        hot._acceptedErrorHandlers[dep] = errorHandler;
      }
    },
    decline: function (dep) {
      if (dep === void 0) hot._selfDeclined = true;
      else if (typeof dep === "object" && dep !== null)
        for (var i = 0; i < dep.length; i++)
          hot._declinedDependencies[dep[i]] = true;
      else hot._declinedDependencies[dep] = true;
    },
    dispose: function (callback) {
      hot._disposeHandlers.push(callback);
    },
    addDisposeHandler: function (callback) {
      hot._disposeHandlers.push(callback);
    },
    removeDisposeHandler: function (callback) {
      var idx = hot._disposeHandlers.indexOf(callback);
      if (idx >= 0) hot._disposeHandlers.splice(idx, 1);
    },
    invalidate: function () {
      this._selfInvalidated = true;
      switch (currentStatus) {
        case "idle":
          currentUpdateApplyHandlers = [];
          Object.keys(__webpack_require__.hmrI).forEach(function (key) {
            __webpack_require__.hmrI[key](moduleId, currentUpdateApplyHandlers);
          });
          setStatus("ready");
          break;
        case "ready":
          Object.keys(__webpack_require__.hmrI).forEach(function (key) {
            __webpack_require__.hmrI[key](moduleId, currentUpdateApplyHandlers);
          });
          break;
        case "prepare":
        case "check":
        case "dispose":
        case "apply":
          (queuedInvalidatedModules = queuedInvalidatedModules || []).push(
            moduleId
          );
          break;
        default:
          break;
      }
    },
    check: hotCheck,
    apply: hotApply,
    status: function (l) {
      if (!l) return currentStatus;
      registeredStatusHandlers.push(l);
    },
    addStatusHandler: function (l) {
      registeredStatusHandlers.push(l);
    },
    removeStatusHandler: function (l) {
      var idx = registeredStatusHandlers.indexOf(l);
      if (idx >= 0) registeredStatusHandlers.splice(idx, 1);
    },
    data: currentModuleData[moduleId],
  };
  currentChildModule = void 0;
  return hot;
}
__name(createModuleHotObject, "createModuleHotObject");
function setStatus(newStatus) {
  currentStatus = newStatus;
  var results = [];
  for (var i = 0; i < registeredStatusHandlers.length; i++)
    results[i] = registeredStatusHandlers[i].call(null, newStatus);
  return Promise.all(results);
}
__name(setStatus, "setStatus");
function unblock() {
  if (--blockingPromises === 0) {
    setStatus("ready").then(function () {
      if (blockingPromises === 0) {
        var list = blockingPromisesWaiting;
        blockingPromisesWaiting = [];
        for (var i = 0; i < list.length; i++) {
          list[i]();
        }
      }
    });
  }
}
__name(unblock, "unblock");
function trackBlockingPromise(promise) {
  switch (currentStatus) {
    case "ready":
      setStatus("prepare");
    case "prepare":
      blockingPromises++;
      promise.then(unblock, unblock);
      return promise;
    default:
      return promise;
  }
}
__name(trackBlockingPromise, "trackBlockingPromise");
function waitForBlockingPromises(fn) {
  if (blockingPromises === 0) return fn();
  return new Promise(function (resolve) {
    blockingPromisesWaiting.push(function () {
      resolve(fn());
    });
  });
}
__name(waitForBlockingPromises, "waitForBlockingPromises");
function hotCheck(applyOnUpdate) {
  if (currentStatus !== "idle") {
    throw new Error("check() is only allowed in idle status");
  }
  return setStatus("check")
    .then(__webpack_require__.hmrM)
    .then(function (update) {
      if (!update) {
        return setStatus(applyInvalidatedModules() ? "ready" : "idle").then(
          function () {
            return null;
          }
        );
      }
      return setStatus("prepare").then(function () {
        var updatedModules = [];
        currentUpdateApplyHandlers = [];
        return Promise.all(
          Object.keys(__webpack_require__.hmrC).reduce(function (
            promises,
            key
          ) {
            __webpack_require__.hmrC[key](
              update.c,
              update.r,
              update.m,
              promises,
              currentUpdateApplyHandlers,
              updatedModules
            );
            return promises;
          },
          [])
        ).then(function () {
          return waitForBlockingPromises(function () {
            if (applyOnUpdate) {
              return internalApply(applyOnUpdate);
            } else {
              return setStatus("ready").then(function () {
                return updatedModules;
              });
            }
          });
        });
      });
    });
}
__name(hotCheck, "hotCheck");
function hotApply(options) {
  if (currentStatus !== "ready") {
    return Promise.resolve().then(function () {
      throw new Error(
        "apply() is only allowed in ready status (state: " + currentStatus + ")"
      );
    });
  }
  return internalApply(options);
}
__name(hotApply, "hotApply");
function internalApply(options) {
  options = options || {};
  applyInvalidatedModules();
  var results = currentUpdateApplyHandlers.map(function (handler) {
    return handler(options);
  });
  currentUpdateApplyHandlers = void 0;
  var errors = results
    .map(function (r) {
      return r.error;
    })
    .filter(Boolean);
  if (errors.length > 0) {
    return setStatus("abort").then(function () {
      throw errors[0];
    });
  }
  var disposePromise = setStatus("dispose");
  results.forEach(function (result) {
    if (result.dispose) result.dispose();
  });
  var applyPromise = setStatus("apply");
  var error;
  var reportError = __name(function (err) {
    if (!error) error = err;
  }, "reportError");
  var outdatedModules = [];
  results.forEach(function (result) {
    if (result.apply) {
      var modules = result.apply(reportError);
      if (modules) {
        for (var i = 0; i < modules.length; i++) {
          outdatedModules.push(modules[i]);
        }
      }
    }
  });
  return Promise.all([disposePromise, applyPromise]).then(function () {
    if (error) {
      return setStatus("fail").then(function () {
        throw error;
      });
    }
    if (queuedInvalidatedModules) {
      return internalApply(options).then(function (list) {
        outdatedModules.forEach(function (moduleId) {
          if (list.indexOf(moduleId) < 0) list.push(moduleId);
        });
        return list;
      });
    }
    return setStatus("idle").then(function () {
      return outdatedModules;
    });
  });
}
__name(internalApply, "internalApply");
function applyInvalidatedModules() {
  if (queuedInvalidatedModules) {
    if (!currentUpdateApplyHandlers) currentUpdateApplyHandlers = [];
    Object.keys(__webpack_require__.hmrI).forEach(function (key) {
      queuedInvalidatedModules.forEach(function (moduleId) {
        __webpack_require__.hmrI[key](moduleId, currentUpdateApplyHandlers);
      });
    });
    queuedInvalidatedModules = void 0;
    return true;
  }
}
__name(applyInvalidatedModules, "applyInvalidatedModules");

Here is a repro of the webpack issue:
https://stackblitz.com/edit/nextjs-3iyyq9?file=pages%2Findex.js
To see it, hit control-c, and re-run with npx tsx ./node_modules/.bin/next dev. Then look at the console:

I've updated the PR so all tests pass. Nice testing framework!

RE: Webpack issue

Curious why you need to run next with tsx?
I'm not familiar with using Next, but you can configure Webpack to not use eval via devtool

RE: Skipping files
The skipped files will not receive any of these transformations.

Why do you run Webpack with tsx?

Correct, that is expected, as published modules that don't target ESM are already transpiled or built to target the current version of Node.

Not true at all. Most Node packages aren't built or transpiled. This was one of the motivations behind tsx.
Your filter doesn't target published modules.

Most Node packages aren't built or transpiled.

Maybe we have a difference in terminology here? Can you share some public modules that are not ESM but need to be transpiled to use in node? I'm talking about code in our node_modules directory. I haven't seen this other than maybe when code targeted a newer version of node than we were using, but this has been rare. Thanks for helping me understand this better!

Your filter doesn't target published modules.

It does, that's why it's fixing the problems for us. "Published modules" in my mind are the modules we're installing into node_modules - is that what you mean?

Can you share some public modules that are not ESM but need to be transpiled to use in node?

I don't have a specific example at the moment, but most pre-ESM sindresorhus packages are uncompiled and use new ES features.

I'm talking about code in our node_modules directory.

"Published modules" would typically refer to packages installed from the npm registry, but in that case, uncompiled user-code is very common. Any use-case that symlinks (e.g. 'npm link' or monorepos). Check some tsx issues of devs that import unbuilt TS projects (which can have allowJs enabled).

Maybe the filter should only target node_modules

I think this is what "target node_modules" implies, and also what I was asking about. Currently, it also skips transformations on local modules which is likely uncompiled.

Even if https://github.com/evanw/esbuild/issues/2605 is addressed, there's no guarantee that other tranformations or helper functions like __name couldn't be added in the future that will break otherwise working code because of the .toString().

Why do you need to use tsx to run Webpack?

To move forward with this, I'll need confirmation that esbuild doesn't have JS transformations for Node.js v12.20 and above. IIRC theres a file in esbuilds repo with transformations and the Node.js version ranges.

I don't have a specific example at the moment, but most pre-ESM sindresorhus packages are uncompiled and use new ES features.

His ESM packages are the reason we're using tsx. 😄  We had no trouble using his packages before the ESM migrations. The "new ES features" were always in line with the current LTS version of Node, and he always uses the engines prop in package.json so you get warnings if you try to use a module that's not compatible with your version of node.

Any use-case that symlinks (e.g. 'npm link' or monorepos).

Yes, I agree. We use a monorepo and never "build" anything except the webpack bundle or when publishing a node module for others to use.
I've updated the PR so we always transform code that is not in /node_modules/.

Currently, it also skips transformations on local modules which is likely uncompiled.

Sorry, I'm not sure what you mean by "currently". Before and after the changes from this PR local code is transformed. Otherwise we couldn't use it to run our projects which are in TypeScript.

Why do you need to use tsx to run Webpack?

When developers run yarn start it:

Finds available ports for the backend and frontend.
Spawns the backend graphql server with mock data.
Starts up webpack-dev-server. This is where webpack is loaded and the problem occurs.
Once ready, opens or refreshes a browser tab with the web content.
If the backend is changed, it re-launches the backend, and then reloads the web page.

All of that is done in a project written in TypeScript, so we run it with tsx.

An engineer on my team just discovered tests failing because of esbuild transpiling jsdom.

I dug into this a little more, since workarounds without fully knowing the root cause always makes me a bit nervous. I haven't found the ultimate root cause, but it does look like this error might be related to esbuild mangling names. I've tracked it down to this code in JSDOM - or mainly, that's the code that should be getting called when constructing the event path for the top-level window object. The problem is that this call isn't holding when parent is window:
idlUtils.wrapperForImpl(parent).constructor.name === "Window"
Instead the constructor name is "windowConstructor", which causes this code to be executed instead, and that incorrectly sets the event target to the Window object instead of the actual DOM node we expect. But only when the event is dispatched for listeners directly on the window object.

This PR avoids this issue by not transpiling jsdom.

Please file a separate issue separately with reproduction. We should seek to address the problem than to avoid them without knowing why.
In case you missed it:

To move forward with this, I'll need confirmation that esbuild doesn't have JS transformations for Node.js v12.20 and above. IIRC theres a file in esbuilds repo with transformations and the Node.js version ranges.

Please file a separate issue separately with reproduction. We should seek to address the problem than to avoid them without knowing why.

Both of these are known caveats with esbuild mentioned on https://esbuild.github.io/api/

The jsdom issues is because of the name being dropped (we're on a build before __name was always included because of the webpack issue).
The webpack hmr issue is because Function.toString() it used to generate the code, there's no way ESBuild or Webpack can know the __name function needs to be included in that toString() output.

To move forward with this, I'll need confirmation that esbuild doesn't have JS transformations for Node.js v12.20 and above. IIRC theres a file in esbuilds repo with transformations and the Node.js version ranges.

This would be for those using Node < 12.20? Node 12 was EOL more than 6 months ago, do we still need this to support versions 12 and under?

Hi @privatenumber -
I decided that our goals aren't the same and it would be best to build our own ESM loader using babel-core so you don't need to make changes you don't want and we're not blocked by bugs introduced by nuances in ESBuild.
I truly appreciate your hard work on this project. You helped us finally be able to use third-party ESM modules. I hope we can use your work again in the future.

__name not define  when use @esbuild-kit/cjs-loader !!!


Remove in-place rotation for omni-directional robots

With the new global planner (smac), there is no need for the omni-directional robots to perform an in-place rotation, rather they can just travel in the given robot heading. This is not applicable to differential drive robots, as omni-directional robots comes with 2 laser scanners.
Even for the NavFn it can be accordingly accommodated by calculating the directional derivative between the robot and the look-ahead point.

This feature is currently in the process of development and it is purely experimental at this point.

#15 takes care of it


Crash in djxl from a file with JPEG reconstruction data with XMP in the JFIF file but no xml JXL box

Describe the bug
djxl from a file, that has JPEG reconstruction data with XMP in the JFIF file but no corresponding xml box in the JXL file, to JPEG crashes
To Reproduce
djxl out4.jxl out.jpg
with out4.jxl from out4.jxl.zip
Expected behavior
should not crash
Environment

OS: Linux
Compiler version: g++ 9.4.0
CPU type: x86_64
cjxl/djxl version string: JPEG XL decoder v0.7.0 2f872a1 [AVX2,SSE4,SSSE3,Scalar]

Additional context
$ valgrind  ~/libjxl/build/tools/djxl out4.jxl out.jpg
==1311894== Memcheck, a memory error detector
==1311894== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==1311894== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info
==1311894== Command: /home/even/libjxl/build/tools/djxl out4.jxl out.jpg
==1311894==
JPEG XL decoder v0.7.0 2f872a1 [AVX2,SSE4,SSSE3,Scalar]
Read 757 compressed bytes.
==1311894== Invalid read of size 8
==1311894==    at 0x128D58: jpegxl::tools::DecodeJpegXlToJpeg(jxl::DecompressParams, jpegxl::tools::JpegXlContainer const&, jxl::CodecInOut*, jxl::ThreadPool*) (box.cc:318)
==1311894==    by 0x121FB9: jpegxl::tools::DecompressJxlToJPEG(jpegxl::tools::JpegXlContainer const&, jpegxl::tools::DecompressArgs const&, jxl::ThreadPool*, jxl::PaddedBytes*, jpegxl::tools::SpeedStats*) (djxl.cc:209)
==1311894==    by 0x11D111: jpegxl::tools::(anonymous namespace)::DecompressMain(int, char const**) (djxl_main.cc:116)
==1311894==    by 0x4FFD0B2: (below main) (libc-start.c:308)
==1311894==  Address 0x8 is not stack'd, malloc'd or (recently) free'd

Could you provide the original jpg? With a slight modification, I could reconstruct the attached jpg.

Could you provide the original jpg?

I believe it was

So the pixel content of your reconstruction is OK, but there's a all-null XMP content in it, which is probably acceptable given how I generated the .jxl file (using the encode ABI, passing the full JPEG content to JxlEncoderAddJPEGFrame() , without stripping the XMP segment, but not using  JxlEncoderAddBox() to generate a JXL 'xml ' box)
My main point was that djxl shouldn't crash.

Ok, so if I understand correctly, the .jxl you generated does not even contain the xmp-content anymore, right?
In that case #1404 should be a sufficient fix, at least it doesn't crash anymore.

I'd argue that making an empty xml box after a call to JxlEncoderAddJPEGFrame() is an encoder API bug ;

the .jxl does not even contain the xmp-content anymore, right?

presumably not (assuming that JxlEncoderAddJPEGFrame() ignores the XMP content that is provided through the JPEG code stream)

Presumably the jxl was created with a libjxl that didn't have this encoder fix yet: https://github.com/libjxl/libjxl/pull/1350
As to what to do with jxl files that have a jbrd that expects exif or xmp but the corresponding boxes are missing: I propose we update 18181-2 to make it clear that this is an invalid bitstream. What libjxl has to do with those, we can choose: either refuse to decode, or reconstruct with a missing marker (but in that case I would propose not to write the marker at all instead of writing it and filling it with zeroes). Refusing to decode is my preferred option, to avoid surprises (or rather, if we don't refuse to decode, and accept 'reconstructions' that are not file-identical, we might as well also allow reconstruction of jpegs when jbrd is missing, in cases where that is possible).

To summarizes, the todo-items here are:

update 18181-2 to make clear such a bitstream is invalid.
refuse to decode such files


xcactivitylog is not generated

This is not exactly an issue with the xclogparser, but with xcodebuild. Whenever I build within Xcode I get the xcactivitylog file, however if I delete derived data folder, then build with xcodebuild the log is just not there. (We use a fresh VM for every new CI build) If I rerun xcodebuild (so I rebuild) the xcactivitylog is generated.
Did you encounter this issue before?

I haven't seen it. Is there a LogStoreManifest.plist file generated in the Logs/Build folder? You can run xclogparser manifest --project MyProjectto check it.

Yes the manifest is there, but it does not have any entries. I do set the derivedDataPath.
I had some success by adding -resultBundlePath '/Users/jenkins/derived_data/resultbundle' and this way I get an xcactivitylog file, but that file parsed with xclogparser produces a totally different json output. (there are no module targets with substeps, there are no steps with cCompilation detailStepTypes, etc.)

@aronpammer I got the same issue when running xcodebuild for archive (-exportArchive). For some reason the logs are not generated in that way. Are you by any chance using the same option?

I hit similar problem. 😞 Building directly with Xcode generates .xcactivitylog file with details such us how many seconds was taken to compile some specific module, but using xcodebuild directly (xcodebuild test -workspace <workspace> -scheme <scheme> -destination 'platform=iOS Simulator,name=iPhone 8,OS=13.1' OTHER_SWIFT_FLAGS="-Xfrontend -debug-time-function-bodies") produces different result which is more noisy (for example you can't see how long it took to build some module).
Could you share which options of xcodebuild command do you use in order to get as detailed .xcactivitylog as building directly in Xcode?

@aleksandergrzyb we don't do anything special to generate the logs with xcodebuild. What do you mean you don't get how long it took to build some module? Is the module missing from the log, or the time is not available?

@aleksandergrzyb, @polac24 was able to reproduce your issue with Xcode 11.1. I found a fix to generate a complete log with xcodebuild by providing the option -resultBundlePath
Something like this should work in your case:xcodebuild test -workspace <workspace> -scheme <scheme> -destination 'platform=iOS Simulator,name=iPhone 8,OS=13.1' -resultBundlePath results_bundle OTHER_SWIFT_FLAGS="-Xfrontend -debug-time-function-bodies"
Mind that this will create a folder called results_bundle so probably you will need to put it in a path that can be cleaned once you CI job finishes

Thanks!


Home screen. AccountBalanceWidget displays "0" balance while loading.

Replace incorrect data with loading animation

Fixed in latest revision (symbol-wallet-prod0.4-11-gc3068a7-c3068a7-201126-release.apk). I'm closing.

Fixed in latest revision (symbol-wallet-prod0.4-11-gc3068a7-c3068a7-201126-release.apk). I'm closing.


Maybe pre-instance CONNECTION_RESET_EXCEPTION_READ too?

Hi everybody,
we have been using netty 3.9 in the past and are looking to move to 4.0 now.
Because we are running our servers behind an haproxy, and due to the way haproxy balances connections, there are a lot of connection resets (~10k per minute) on our backend servers. With the NIO transport, we just ignored them. With the EPOLL transport, we are going to do the same, but I noticed that each of them has a stacktrace still, wheres with the NIO transport, they stopped having stacktraces after the first few.
Looking into the epoll code, I stumpled upon the comment
"TODO: We could even go futher and use a pre-instanced IOException for the other error codes, but for all other errors it may be better to just include a stacktrace." (Native.java@L129)
So I thought I'd just give you a heads up describing our case, in which pre-instantiation / an empty stack trace would definitely be preferable. Though, of course, the JIT can probably be expected to already kind of mitigate the impact, as it is. (Because in our case the stack trace will always be the same.)
Anyways, I'm just going to use this opportunity to say that I love this project. Awesome work everyone!
Regards

@besief which version are you using ?

Upgrading to 4.0.25.Final, but from skimming the sources here on github, behaviour doesn't seem to have changed in the other versions. (?)
Upgrading from 3.9.0.Final.

I'm asking because CONNECTION_RESET_EXCEPTION_READ should be used already. Can you show me the stack trace you are seeing ?

java.io.IOException: readAddress() failed: Connection reset by peer
io.netty.channel.epoll.Native.newIOException(Native.java:112):1 in ""
io.netty.channel.epoll.Native.ioResult(Native.java:128):1 in ""
io.netty.channel.epoll.Native.readAddress(Native.java:219):1 in ""
io.netty.channel.epoll.EpollSocketChannel$EpollSocketUnsafe.doReadBytes(EpollSocketChannel.java:675):1 in ""
io.netty.channel.epoll.EpollSocketChannel$EpollSocketUnsafe.epollInReady(EpollSocketChannel.java:714):1 in ""
io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:326):1 in ""
io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:264):1 in ""
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116):1 in ""
io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137):1 in ""
java.lang.Thread.run(Thread.java:745):1 in ""

I'm talking about the following part of Native.java. As far as I understand it, certain exceptions are always reused (they are static final), while other get new instances via calls to newIOException.
    private static IOException newIOException(String method, int err) {
        return new IOException(method + "() failed: " + ERRORS[-err]);
    }

    private static int ioResult(String method, int err, IOException resetCause) throws IOException {
        // network stack saturated... try again later
        if (err == ERRNO_EAGAIN_NEGATIVE || err == ERRNO_EWOULDBLOCK_NEGATIVE) {
            return 0;
        }
        if (err == ERRNO_EPIPE_NEGATIVE) {
            throw resetCause;
        }
        if (err == ERRNO_EBADF_NEGATIVE) {
            throw CLOSED_CHANNEL_EXCEPTION;
        }
        // TODO: We could even go futher and use a pre-instanced IOException for the other error codes, but for
        //       all other errors it may be better to just include a stacktrace.
        throw newIOException(method, err);
    }

Of course, resetCause will always be the same CONNECTION_RESET_EXCEPTION_READ object (that is also static final), but what is given to the user code then will be a newly instanced IOException.

@besief ah I see... it's a bug. Let me fix it.

Cool, thanks.
ps: Sorry for not including all the details in the initial posting, I thought everything might be clear enough by just referring to that comment. :)

@besief no worries... Thanks again to open the bug report!

Fixed...


Fix arrow format patch error on python 3.7

Summary
Disentangle temporary directory management

Fixed to remove and move manually each files instead of whole directory itself.

How to test

Checklist

[ ] I have added unit tests to cover my changes.
[ ] I have added integration tests to cover my changes.
[ ] I have added the description of my changes into CHANGELOG.
[ ] I have updated the documentation accordingly

License

[ ] I submit my code changes under the same MIT License that covers the project.
Feel free to contact the maintainers if that's a concern.
[ ] I have updated the license header for each file (see an example below).

# Copyright (C) 2023 Intel Corporation
#
# SPDX-License-Identifier: MIT

Codecov Report
Patch coverage: 77.77% and project coverage change: -0.03 :warning:

Comparison is base (eaae1e1) 78.65% compared to head (b9db137) 78.62%.

Additional details and impacted files
@@                Coverage Diff                 @@
##           releases/1.2.0     #939      +/-   ##
==================================================
- Coverage           78.65%   78.62%   -0.03%
==================================================
  Files                 228      228
  Lines               26342    26349       +7
  Branches             5249     5252       +3
==================================================
- Hits                20718    20717       -1
- Misses               4406     4409       +3
- Partials             1218     1223       +5

Flag
Coverage Δ

macos-11_Python-3.8
?

ubuntu-20.04_Python-3.8
78.62% <77.77%> (-0.01%)
:arrow_down:

windows-2019_Python-3.8
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

datumaro/plugins/data_formats/arrow/exporter.py
73.58% <77.77%> (-1.54%)
:arrow_down:

... and 2 files with indirect coverage changes
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Add first version of "python packages" talk

Yeah, I wasn't sure on what level to start. But I can add a slide or two on packages in general (and maybe PyPI)

That would be great. I guess, we'll have a well-mixed audience. And skipping a slide is much easier than making it up on the spot...

thanks!


multiview border colors

@Julusian
Hi Julian, this is sort of a follow up request to [Request #269]...In Blackmagic's latest firmware update to Atem Constellation 8K, they added three new features (among many other things) allowing users to control the following multiview settings:
(i) turning borders on/off,
(ii) changing border colors, and
(iii) turning labels on/off.
Can you please incorporate these new control features into Bitfocus Companion so I can adjust these settings using my streamdeck? For what it's worth, if it's too time consuming to program all the various color options, I'm really only interested in using black. Perhaps the first iteration of the release could be black/gray/white? Completely up to you of course. Thank you so much for your time and help, and please let me know if you have any questions.

I don't have access to a model which supports this, so can't implement it currently.
@audioengineer1 could you run https://builds.julusian.dev/atem/ against your atem and send me the resulting file? It takes a copy of the state of the atem, which will let me use a tool to pretend I have that atem. Either attach it here (the 8k doesn't have any streaming api keys to worry about), or email it to me<EMAIL_ADDRESS>

Hi Julian, thank you so much for your response. I have attached the file for your review...I'm not sure if this is what you're looking for. I exported the text out of the terminal app. Please let me know how I can assist moving forward. Thanks again!
Rob
Terminal Saved Output.txt

Unfortonately not, it failed to write the file to disk which looks to be because you didn't copy the executable out of the dmg before running it

Sorry about that...I'm emailing you the file as the this platform doesn't support the file type.

Thanks for the file, I believe I have done the reverse engineering work needed. I haven't started on pulling that across to the module yet


Fixes 1215: Language client does not automatically send file operation notification/request when working on certain path

@DanTup can you please have a look as well.


Backport v0.5.0 fixes to v0.4 branch

All recent SSL context updates will land in the upcoming v0.5.0 release. This is done because they build on #46, which contains a (rather small) BC break.
Given that there are quite a few consumers which rely on version ~0.4.0 of this library, we should consider backporting the SSL context updates to a v0.4.xrelease.
Relevant tickets:

#43
#47
#48
#54
#61

I think it makes sense to encourage updating to the new v0.5.0 release and keep backporting at a minimum.
See #65 which makes the latest v0.4 branch compatible with PHP 5.6+ at least (fixes #61), but does not address any other of the above issues.

Closed via #65.


Switch from id to content-id in URLs

We refer to content items by their content_item_id in most GOV.UK apps so we should be consistent. Here we’ve switched the param in the URL path to be for the content-id instead of id when referring to content items.

Did you consider using to_param in ContentItem?. Would it simplify the PR?