Usage of Validator

@neomerx is it possible to have documentation for validator usage?
as i think i was not very familiar with the part doing conversion, e.g.: stringtoadateline, etc...
having some custom validation logic,  but cant find entry point of gettings values from request and making it usable on validator

also having some problem validating datetime.
e.g.:
self::isDateTime(self::stringToDateTime(DateTimeBaseType::JSON_API_FORMAT));
with ISO8601 dateformat e.g "2016-10-01T00:00:00+0000" fails in test and return
[items:Neomerx\JsonApi\Exceptions\ErrorCollection:private] => Array
    (
        [0] => Neomerx\JsonApi\Document\Error Object
            (
                [idx:Neomerx\JsonApi\Document\Error:private] =>
                [links:Neomerx\JsonApi\Document\Error:private] =>
                [status:Neomerx\JsonApi\Document\Error:private] => 422
                [code:Neomerx\JsonApi\Document\Error:private] =>
                [title:Neomerx\JsonApi\Document\Error:private] => The value is invalid.
                [detail:Neomerx\JsonApi\Document\Error:private] => The value should be a valid date time.
                [source:Neomerx\JsonApi\Document\Error:private] => Array
                    (
                        [pointer] => /data/attributes/dateline
                    )

                [meta:Neomerx\JsonApi\Document\Error:private] =>
            )

    )

as well as self::isInt(self::stringtoInt());

Can you post a few examples you've got difficulties with?
JSON API Validator parses a JSON API document and validates it looks like a valid one. Then it passes actual string values to validatation rules. As a developer you describe your rules in  JsonApiRuleSetInterface which contains rules for such JSON API document parts as id, type, individual attributes and individual relationships.
Now how rules work.

You work with dates
Expected value string
v::stringToDateTime(DateTimeBaseType::JSON_API_FORMAT)
it will check input value as a string and will convert it to a date time with the format given

Rule chaining
Rules support chaining one rule with another. Let's say you can convert a string to int time and then check its value is within a specified range.
v::stringtoInt(v::between(5, 10))
The first rule has a string as an input and the second one will have int.
So when you write self::isInt(self::stringtoInt()) you firstly check string is int (error).

Can you post a few examples so I can help you with?

for belongs to relationship
i saw
return self::required(self::toOneRelationship(BoardScheme::TYPE, BoardRules::isBoardId()));
is the self::required compulsory

some updates:
it was abit rare that, i re-run the test.
with:
/**
 * @return RuleInterface
 */
public static function dateline(): RuleInterface
{
    return self::stringToDateTime(DateTimeBaseType::JSON_API_FORMAT);
}

it throws:
ErrorException: strlen() expects parameter 1 to be string, object given

for your input
    {
        "data" : {
            "type"  : "submissions",
            "id"    : null,
            "attributes": {
                "dateline": "2017-01-01T00:00:00+0000",
                "present": "true"
            }
        }
    }

rules should be
$typeRule = v::equals('submissions');
$idRule = v::equals(null);
$attrRules = [
  'dateline' => v::stringToDateTime(DateTimeBaseType::JSON_API_FORMAT),
  'present' => v::stringToBool(),
];

adding v::requied makes input field mandatory (will produce an error if not given)
If you post an example for relationships I can help you with it as well

that means if stringToXX present. isInt is not required to be chained right?

thanks @neomerx, the boolean and it part were solved.
but the datetime part wasn't solved yet.
it keep throwing "ErrorException: strlen() expects parameter 1 to be string, object given"

isXXX almost never needed for JSON API inputs. It's for other validators. In JSON API you've got strings as inputs so after stringToXXX you always have the correct type.
As for stringToDateTime I don't see it uses strlen. Can you please provide more details in input data and the rule itself?

models
 Type::INTEGER,
            self::FIELD_ID_USER                 => Type::INTEGER,
            self::FIELD_DATELINE                => JsonApiDateTimeType::NAME,
            self::FIELD_PRESENT                 => Type::BOOLEAN,
            self::FIELD_DESCRIPTION             => Type::TEXT,
            self::FIELD_DISPLAY_ORDER           => Type::INTEGER,
            self::FIELD_CREATED_AT              => JsonApiDateTimeType::NAME,
            self::FIELD_UPDATED_AT              => JsonApiDateTimeType::NAME,
            self::FIELD_DELETED_AT              => JsonApiDateTimeType::NAME,
        ];
    }

    /**
     * @inheritDoc
     */
    public static function getAttributeLengths(): array
    {
        return [];
    }

    /**
     * @inheritDoc
     */
    public static function getRelationships(): array
    {
        return [
            RelationshipTypes::BELONGS_TO => [
                self::REL_USER                 => [User::class, self::FIELD_ID_USER, User::REL_SUBMISSIONS],
            ],
        ];
    }
}

`<?php namespace App\Data\Models;
use Doctrine\DBAL\Types\Type;
use Limoncello\Contracts\Application\ModelInterface;
use Limoncello\Contracts\Data\RelationshipTypes;
use Limoncello\Flute\Types\JsonApiDateTimeType;
/**

Class Submission

@package App\Data\Models
/
class Submission implements ModelInterface, CommonFields
{
/*

Table name
/
const TABLE_NAME = 'submissions';
/*
Primary key
*/
const FIELD_ID = 'id_submission';

/**

Foreign key
*/
const FIELD_ID_USER = User::FIELD_ID;

/**

Field name
/
const FIELD_DATELINE = 'dateline';
/*
Field name
/
const FIELD_PRESENT = 'present';
/*
Field name
*/
const FIELD_DESCRIPTION = 'description';

/**

Field name
*/
const FIELD_DISPLAY_ORDER = 'display_order';

/**

Relationship name
*/
const REL_USER = 'user';

/**

@inheritDoc
*/
public static function getTableName(): string
{
return static::TABLE_NAME;
}

/**

@inheritDoc
*/
public static function getPrimaryKeyName(): string
{
return static::FIELD_ID;
}

/**

@inheritDoc
*/
public static function getAttributeTypes(): array
{
return [
self::FIELD_ID                      => Type::INTEGER,
self::FIELD_ID_USER                 => Type::INTEGER,
self::FIELD_DATELINE                => JsonApiDateTimeType::NAME,
self::FIELD_PRESENT                 => Type::BOOLEAN,
self::FIELD_DESCRIPTION             => Type::TEXT,
self::FIELD_DISPLAY_ORDER           => Type::INTEGER,
self::FIELD_CREATED_AT              => JsonApiDateTimeType::NAME,
self::FIELD_UPDATED_AT              => JsonApiDateTimeType::NAME,
self::FIELD_DELETED_AT              => JsonApiDateTimeType::NAME,
];
}

/**

@inheritDoc
*/
public static function getAttributeLengths(): array
{
return [];
}

/**

@inheritDoc
*/
public static function getRelationships(): array
{
return [
RelationshipTypes::BELONGS_TO => [
self::REL_USER                 => [User::class, self::FIELD_ID_USER, User::REL_SUBMISSIONS],
],
];
}
}
`

`<?php namespace App\Json\Schemes;
use App\Data\Models\Submission as Model;
/**

Class SubmissionScheme

@package App\Json\Schemes
/
class SubmissionScheme extends BaseScheme
{
/*

Attribute type
/
const TYPE = 'submissions';
/*
Model class name
*/
const MODEL = Model::class;

/**

Attribute name
/
const ATTR_DATELINE = Model::FIELD_DATELINE;
/*
Attribute name
/
const ATTR_PRESENT = Model::FIELD_PRESENT;
/*
Attribute name
*/
const ATTR_DESCRIPTION = Model::FIELD_DESCRIPTION;

/**

Attribute name
*/
const ATTR_DISPLAY_ORDER = 'display-order';

/**

Relationship name
*/
const REL_USER = Model::REL_USER;

/**

@inheritDoc
*/
public static function getMappings(): array
{
return [
self::SCHEMA_ATTRIBUTES    => [
self::ATTR_DATELINE      => Model::FIELD_DATELINE,
self::ATTR_PRESENT       => Model::FIELD_PRESENT,
self::ATTR_DESCRIPTION   => Model::FIELD_DESCRIPTION,
self::ATTR_DISPLAY_ORDER => Model::FIELD_DISPLAY_ORDER,
self::ATTR_CREATED_AT    => Model::FIELD_CREATED_AT,
self::ATTR_UPDATED_AT    => Model::FIELD_UPDATED_AT,
self::ATTR_DELETED_AT    => Model::FIELD_DELETED_AT,
],
self::SCHEMA_RELATIONSHIPS => [
self::REL_USER                 => Model::REL_USER,
],
];
}

/**

@inheritDoc
*/
protected function getExcludesFromDefaultShowSelfLinkInRelationships(): array
{
return [
self::REL_USER => true,
];
}

/**

@inheritDoc
*/
protected function getExcludesFromDefaultShowRelatedLinkInRelationships(): array
{
return [
self::REL_USER => true,
];
}

}`

And rules?

<?php namespace App\Authorization;

use App\Data\Models\Submission as Model;
use App\Json\Api\SubmissionsApi as Api;
use App\Json\Schemes\SubmissionScheme as Scheme;
use Limoncello\Application\Contracts\Authorization\ResourceAuthorizationRulesInterface;
use Limoncello\Auth\Contracts\Authorization\PolicyInformation\ContextInterface;
use Limoncello\Flute\Contracts\FactoryInterface;
use Settings\Passport;

/**
 * Class SubmissionRules
 *
 * @package App\Authorization
 */
class SubmissionRules implements ResourceAuthorizationRulesInterface
{
    use RulesTrait;

    /**
     * Action name
     */
    const ACTION_VIEW_SUBMISSIONS = 'canViewSubmissions';

    /**
     * Action name
     */
    const ACTION_CREATE_SUBMISSIONS = 'canCreateSubmissions';

    /**
     * Action name
     */
    const ACTION_EDIT_SUBMISSIONS = 'canEditSubmissions';

    /**
     * @inheritdoc
     */
    public static function getResourcesType(): string
    {
        return Scheme::TYPE;
    }

    /**
     * @param ContextInterface $context
     *
     * @return bool
     */
    public static function canViewSubmissions(ContextInterface $context): bool
    {
        return self::hasScope($context, Passport::SCOPE_ADMIN_SUBMISSIONS) || self::hasScope($context, Passport::SCOPE_VIEW_SUBMISSIONS);
    }

    /**
     * @param ContextInterface $context
     *
     * @return bool
     */
    public static function canCreateSubmissions(ContextInterface $context): bool
    {
        return self::hasScope($context, Passport::SCOPE_ADMIN_SUBMISSIONS);
    }

    /**
     * @param ContextInterface $context
     *
     * @return bool
     */
    public static function canEditSubmissions(ContextInterface $context): bool
    {
        return self::hasScope($context, Passport::SCOPE_ADMIN_SUBMISSIONS);
    }

    /**
     * @param ContextInterface $context
     *
     * @return bool
     */
    private static function isSubmissionOwner(ContextInterface $context): bool
    {
        $isSubmissionOwner = false;

        if (($userId = self::getCurrentUserIdentity($context)) !== null) {
            $identity = self::reqGetResourceIdentity($context);

            $container = self::ctxGetContainer($context);
            $factory = $container->get(FactoryInterface::class);
            $api = $factory->createApi(Api::class);
            $submission = $api->readResource($identity);
            $isSubmissionOwner = $submission !== null && $submission->{Model::FIELD_ID_USER} === $userId;
        }

        return $isSubmissionOwner;
    }
}

<?php

namespace App\Http\Controllers;

use App\Json\Api\SubmissionsApi as Api;
use App\Json\Schemes\SubmissionScheme as Scheme;
use App\Json\Validators\SubmissionCreate;
use App\Json\Validators\SubmissionUpdate;

/**
 * Class SubmissionsController
 *
 * @package App\Http\Controllers
 */
class SubmissionsController extends BaseController
{
    /**
     * Api class name
     */
    const API_CLASS = Api::class;

    /**
     * Schema class name
     */
    const SCHEMA_CLASS = Scheme::class;

    /**
     * Validator class name
     */
    const ON_CREATE_VALIDATION_RULES_SET_CLASS = SubmissionCreate::class;

    /**
     * Validator class name
     */
    const ON_UPDATE_VALIDATION_RULES_SET_CLASS = SubmissionUpdate::class;
}

<?php

namespace App\Json\Validators\Rules;

use App\Data\Models\Submission as Model;
use App\Json\Schemes\SubmissionScheme as Scheme;
use App\Json\Schemes\UserScheme;
use Limoncello\Flute\Types\DateTimeBaseType;
use Limoncello\Flute\Validation\Rules\ExistInDatabaseTrait;
use Limoncello\Flute\Validation\Rules\RelationshipsTrait;
use Limoncello\Validation\Contracts\Rules\RuleInterface;
use Limoncello\Validation\Rules;
use PHPMD\Rule;

/**
 * Class SubmissionRules
 *
 * @package App\Json\Validators\Rules
 */
final class SubmissionRules extends Rules
{
    use RelationshipsTrait, ExistInDatabaseTrait;

    /**
     * @return RuleInterface
     */
    public static function isSubmissionType(): RuleInterface
    {
        return self::equals(Scheme::TYPE);
    }

    /**
     * @return RuleInterface
     */
    public static function isSubmissionId(): RuleInterface
    {
        return self::stringToInt(self::exists(Model::TABLE_NAME, Model::FIELD_ID));
    }

    /**
     * @return RuleInterface
     */
    public static function isUserRelationship(): RuleInterface
    {
        return self::toOneRelationship(UserScheme::TYPE, UserRules::isUserId());
    }

    /**
     * @return RuleInterface
     */
    public static function dateline(): RuleInterface
    {
        return self::stringToDateTime(DateTimeBaseType::JSON_API_FORMAT);
    }

    public static function present(): RuleInterface
    {
        return self::stringToBool();
    }

    /**
     * @return RuleInterface
     */
    public static function description(): RuleInterface
    {
        return self::isString();
    }

    /**
     * @return RuleInterface
     */
    public static function displayOrder(): RuleInterface
    {
        return self::stringToInt();
    }
}

so far seems fine. What about SubmissionCreate and SubmissionUpdate?

  public function testCreate()
    {
        try {
            $this->setPreventCommits();

            $jsonInput = <<<EOT
        {
            "data" : {
                "type"  : "submissions",
                "id"    : null,
                "attributes": {
                    "dateline": "2017-01-01T00:00:00+0800",
                    "overtime": "-1",
                    "archive": null,
                    "display-order": "AAA"
                },
                "relationships": {
                    "user": {
                        "data": { "type": "users", "id": "55" }
                    }
                }
            }
        }
EOT;

            $headers = ['Authorization' => 'Bearer ' . $this->getAdministratorsOAuthToken()];
            $response = $this->postJsonApi(self::API_URI, $jsonInput, $headers);
            $this->assertEquals(201, $response->getStatusCode());

            $json = json_decode((string)$response->getBody());
            $this->assertObjectHasAttribute('data', $json);
            $submissionId = $json->data->id;

            $this->assertEquals(200, $this->get(self::API_URI . "/$submissionId", [], $headers)->getStatusCode());

            $query = $this->getCapturedConnection()->createQueryBuilder();
            $statement = $query
                ->select('*')
                ->from(Submission::TABLE_NAME)
                ->where(Submission::FIELD_ID . '=' . $query->createPositionalParameter($submissionId))
                ->execute();
            $this->assertNotEmpty($values = $statement->fetch());
            $this->assertNotEmpty($values[ Submission::FIELD_CREATED_AT ]);
        } catch (JsonApiException $e) {
            print_r($e->getErrors());
        }
    }

<?php

namespace App\Json\Validators;

use App\Json\Schemes\SubmissionScheme as Scheme;
use App\Json\Validators\Rules\SubmissionRules as v;
use Limoncello\Flute\Contracts\Validation\JsonApiRuleSetInterface;
use Limoncello\Flute\Types\DateTimeBaseType;
use Limoncello\Validation\Contracts\Rules\RuleInterface;

/**
 * Class SubmissionCreate
 *
 * @package App\Json\Validators
 */
class SubmissionCreate implements JsonApiRuleSetInterface
{
    /**
     * @inheritDoc
     */
    public static function getTypeRule(): RuleInterface
    {
        return v::isSubmissionType();
    }

    /**
     * @inheritDoc
     */
    public static function getIdRule(): RuleInterface
    {
        return v::equals(null);
    }

    /**
     * @inheritDoc
     */
    public static function getAttributeRules(): array
    {
        return [
            Scheme::ATTR_DATELINE      => v::required(v::dateline()),
            Scheme::ATTR_PRESENT       => v::required(v::present()),
            Scheme::ATTR_DESCRIPTION   => v::description(),
            Scheme::ATTR_DISPLAY_ORDER => v::stringToInt(),
        ];
    }

    /**
     * @inheritDoc
     */
    public static function getToOneRelationshipRules(): array
    {
        return [
            Scheme::REL_USER                 => v::required(v::isUserRelationship()),
        ];
    }

    /**
     * @inheritDoc
     */
    public static function getToManyRelationshipRules(): array
    {
        return [];
    }
}

<?php

namespace App\Json\Validators;

use App\Json\Schemes\SubmissionScheme as Scheme;
use App\Json\Validators\Rules\SubmissionRules as v;
use Limoncello\Flute\Contracts\Validation\JsonApiRuleSetInterface;
use Limoncello\Validation\Contracts\Rules\RuleInterface;

/**
 * Class SubmissionUpdate
 *
 * @package App\Json\Validators
 */
class SubmissionUpdate implements JsonApiRuleSetInterface
{
    /**
     * @inheritDoc
     */
    public static function getTypeRule(): RuleInterface
    {
        return v::isSubmissionType();
    }

    /**
     * @inheritDoc
     */
    public static function getIdRule(): RuleInterface
    {
        return v::isSubmissionId();
    }

    /**
     * @inheritDoc
     */
    public static function getAttributeRules(): array
    {
        return [
            Scheme::ATTR_DATELINE      => v::dateline(),
            Scheme::ATTR_PRESENT       => v::present(),
            Scheme::ATTR_DESCRIPTION   => v::description(),
            Scheme::ATTR_DISPLAY_ORDER => v::displayOrder(),
        ];
    }

    /**
     * @inheritDoc
     */
    public static function getToOneRelationshipRules(): array
    {
        return [
            Scheme::REL_USER                 => v::isUserRelationship(),
        ];
    }

    /**
     * @inheritDoc
     */
    public static function getToManyRelationshipRules(): array
    {
        return [];
    }
}

i runned the test again,
            $jsonInput = <<<EOT
        {
            "data" : {
                "type"  : "submissions",
                "id"    : null,
                "attributes": {
                    "dateline": "2017-01-01T00:00:00+0800",
                    "present": "-1",
                    "display-order": "AAA"
                },
                "relationships": {
                    "user": {
                        "data": { "type": "users", "id": "55" }
                    }
                }
            }
        }
EOT;

it was rare again,
the dateline does throw "ErrorException: strlen() expects parameter 1 to be string, object given" now.
but as you see the "display-order" should throw exception but it doesn't

Can you provide stack for 'strlen() expects parameter 1 to be string, object given'?
display-order will not produce any error. For PHP 'AAA' is a valid input to convert it to int. It can be checked for numeric but let's do it later. The 'strlen' error first.

@neomerx let me try reproduce it.

There was 1 error:

1) Tests\SubmissionApiTest::testCreate
ErrorException: strlen() expects parameter 1 to be string, object given

/timable/limoncello-php/vendor/doctrine/dbal/lib/Doctrine/DBAL/Types/ConversionException.php:44
/timable/limoncello-php/vendor/limoncello-php/flute/src/Types/JsonApiDateTimeType.php:53
/timable/limoncello-php/vendor/limoncello-php/flute/src/Adapters/Repository.php:140
/timable/limoncello-php/vendor/limoncello-php/flute/src/Api/Crud.php:320
/timable/limoncello-php/app/Json/Api/SubmissionsApi.php:100
/timable/limoncello-php/vendor/limoncello-php/flute/src/Http/BaseController.php:112
/timable/limoncello-php/vendor/limoncello-php/core/src/Application/Application.php:285
/timable/limoncello-php/vendor/limoncello-php/core/src/Application/Application.php:351
/timable/limoncello-php/app/Json/Exceptions/ApiHandler.php:35
/timable/limoncello-php/vendor/limoncello-php/core/src/Application/Application.php:423
/timable/limoncello-php/vendor/limoncello-php/passport/src/Authentication/PassportMiddleware.php:66
/timable/limoncello-php/vendor/limoncello-php/core/src/Application/Application.php:423
/timable/limoncello-php/vendor/limoncello-php/application/src/Packages/Cors/CorsMiddleware.php:53
/timable/limoncello-php/vendor/limoncello-php/core/src/Application/Application.php:423
/timable/limoncello-php/vendor/limoncello-php/core/src/Application/Application.php:186
/timable/limoncello-php/vendor/limoncello-php/testing/src/ApplicationWrapperTrait.php:160
/timable/limoncello-php/vendor/limoncello-php/core/src/Application/Application.php:146
/timable/limoncello-php/vendor/limoncello-php/testing/src/TestCaseTrait.php:106
/timable/limoncello-php/vendor/limoncello-php/testing/src/JsonApiCallsTrait.php:47
/timable/limoncello-php/tests/SubmissionApiTest.php:115

ERRORS!

Aha, so it passes validation but fails on saving the value to the database...

.                                                                   1 / 1 (100%)string(6038) "#0 [internal function]: Limoncello\Application\Packages\Application\Application->Limoncello\Application\Packages\Application\{closure}(2, 'strlen() expect...', '/timable/limonc...', 44, Array)
#1 /timable/limoncello-php/vendor/doctrine/dbal/lib/Doctrine/DBAL/Types/ConversionException.php(44): strlen(Object(DateTimeImmutable))
#2 /timable/limoncello-php/vendor/limoncello-php/flute/src/Types/JsonApiDateTimeType.php(53): Doctrine\DBAL\Types\ConversionException::conversionFailed(Object(DateTimeImmutable), 'datetime')
#3 /timable/limoncello-php/vendor/limoncello-php/flute/src/Adapters/Repository.php(140): Limoncello\Flute\Types\JsonApiDateTimeType->convertToDatabaseValue(Object(DateTimeImmutable), Object(Doctrine\DBAL\Platforms\MySQL57Platform))
#4 /timable/limoncello-php/vendor/limoncello-php/flute/src/Api/Crud.php(320): Limoncello\Flute\Adapters\Repository->create('App\\Data\\Models...', Array)
#5 /timable/limoncello-php/app/Json/Api/SubmissionsApi.php(100): Limoncello\Flute\Api\Crud->create(NULL, Array, Array)
#6 /timable/limoncello-php/vendor/limoncello-php/flute/src/Http/BaseController.php(112): App\Json\Api\SubmissionsApi->create(NULL, Array, Array)
#7 [internal function]: Limoncello\Flute\Http\BaseController::create(Array, Object(Limoncello\Container\Container), Object(Zend\Diactoros\ServerRequest))
#8 /timable/limoncello-php/vendor/limoncello-php/core/src/Application/Application.php(285): call_user_func(Array, Array, Object(Limoncello\Container\Container), Object(Zend\Diactoros\ServerRequest))
#9 /timable/limoncello-php/vendor/limoncello-php/core/src/Application/Application.php(351): Limoncello\Core\Application\Application->callHandler(Array, Array, Object(Limoncello\Container\Container), Object(Zend\Diactoros\ServerRequest))
#10 /timable/limoncello-php/app/Json/Exceptions/ApiHandler.php(35): Limoncello\Core\Application\Application->Limoncello\Core\Application\{closure}(Object(Zend\Diactoros\ServerRequest))
#11 [internal function]: App\Json\Exceptions\ApiHandler::handle(Object(Zend\Diactoros\ServerRequest), Object(Closure), Object(Limoncello\Container\Container))
#12 /timable/limoncello-php/vendor/limoncello-php/core/src/Application/Application.php(423): call_user_func(Array, Object(Zend\Diactoros\ServerRequest), Object(Closure), Object(Limoncello\Container\Container))
#13 /timable/limoncello-php/vendor/limoncello-php/passport/src/Authentication/PassportMiddleware.php(66): Limoncello\Core\Application\Application->Limoncello\Core\Application\{closure}(Object(Zend\Diactoros\ServerRequest))
#14 [internal function]: Limoncello\Passport\Authentication\PassportMiddleware::handle(Object(Zend\Diactoros\ServerRequest), Object(Closure), Object(Limoncello\Container\Container))
#15 /timable/limoncello-php/vendor/limoncello-php/core/src/Application/Application.php(423): call_user_func(Array, Object(Zend\Diactoros\ServerRequest), Object(Closure), Object(Limoncello\Container\Container))
#16 /timable/limoncello-php/vendor/limoncello-php/application/src/Packages/Cors/CorsMiddleware.php(53): Limoncello\Core\Application\Application->Limoncello\Core\Application\{closure}(Object(Zend\Diactoros\ServerRequest))
#17 [internal function]: Limoncello\Application\Packages\Cors\CorsMiddleware::handle(Object(Zend\Diactoros\ServerRequest), Object(Closure), Object(Limoncello\Container\Container))
#18 /timable/limoncello-php/vendor/limoncello-php/core/src/Application/Application.php(423): call_user_func(Array, Object(Zend\Diactoros\ServerRequest), Object(Closure), Object(Limoncello\Container\Container))
#19 [internal function]: Limoncello\Core\Application\Application->Limoncello\Core\Application\{closure}(Object(Zend\Diactoros\ServerRequest))
#20 /timable/limoncello-php/vendor/limoncello-php/core/src/Application/Application.php(186): call_user_func(Object(Closure), Object(Zend\Diactoros\ServerRequest))
#21 /timable/limoncello-php/vendor/limoncello-php/testing/src/ApplicationWrapperTrait.php(160): Limoncello\Core\Application\Application->handleRequest(Object(Closure), Object(Zend\Diactoros\ServerRequest))
#22 /timable/limoncello-php/vendor/limoncello-php/core/src/Application/Application.php(146): class@anonymous->handleRequest(Object(Closure), Object(Zend\Diactoros\ServerRequest))
#23 /timable/limoncello-php/vendor/limoncello-php/testing/src/TestCaseTrait.php(106): Limoncello\Core\Application\Application->run()
#24 /timable/limoncello-php/vendor/limoncello-php/testing/src/JsonApiCallsTrait.php(47): Tests\TestCase->call('POST', '/api/v1/submiss...', Array, Array, Array, Array, Array, Array, Object(Zend\Diactoros\Stream))
#25 /timable/limoncello-php/tests/SubmissionApiTest.php(116): Tests\SubmissionApiTest->postJsonApi('/api/v1/submiss...', '        {\n     ...', Array)
#26 [internal function]: Tests\SubmissionApiTest->testCreate()
#27 /timable/limoncello-php/vendor/phpunit/phpunit/src/Framework/TestCase.php(1069): ReflectionMethod->invokeArgs(Object(Tests\SubmissionApiTest), Array)
#28 /timable/limoncello-php/vendor/phpunit/phpunit/src/Framework/TestCase.php(928): PHPUnit\Framework\TestCase->runTest()
#29 /timable/limoncello-php/vendor/phpunit/phpunit/src/Framework/TestResult.php(695): PHPUnit\Framework\TestCase->runBare()
#30 /timable/limoncello-php/vendor/phpunit/phpunit/src/Framework/TestCase.php(883): PHPUnit\Framework\TestResult->run(Object(Tests\SubmissionApiTest))
#31 /timable/limoncello-php/vendor/phpunit/phpunit/src/Framework/TestSuite.php(744): PHPUnit\Framework\TestCase->run(Object(PHPUnit\Framework\TestResult))
#32 /timable/limoncello-php/vendor/phpunit/phpunit/src/TextUI/TestRunner.php(537): PHPUnit\Framework\TestSuite->run(Object(PHPUnit\Framework\TestResult))
#33 /timable/limoncello-php/vendor/phpunit/phpunit/src/TextUI/Command.php(212): PHPUnit\TextUI\TestRunner->doRun(Object(PHPUnit\Framework\TestSuite), Array, true)
#34 /timable/limoncello-php/vendor/phpunit/phpunit/src/TextUI/Command.php(141): PHPUnit\TextUI\Command->run(Array, true)
#35 /timable/limoncello-php/vendor/phpunit/phpunit/phpunit(53): PHPUnit\TextUI\Command::main()

I think it should be an error in \Limoncello\Flute\Types\JsonApiDateTimeType::convertToDatabaseValue
Can you try to replace it with the following?
    public function convertToDatabaseValue($value, AbstractPlatform $platform)
    {
        if ($value instanceof DateTimeInterface) {
            $dateTime = $value;
        } elseif ($value instanceof JsonApiDateTime) {
            $dateTime = $value->getValue();
        } elseif (is_string($value) === true) {
            if (($dateTime = DateTime::createFromFormat(DateBaseType::JSON_API_FORMAT, $value)) === false) {
                throw ConversionException::conversionFailed($value, $this->getName());
            }
        } else {
            throw ConversionException::conversionFailed($value, $this->getName());
        }

        return parent::convertToDatabaseValue($dateTime, $platform);
    }

yes. replaced with your snippet and the issue gone

as for display order try to add isNumeric check
    public static function displayOrder(): RuleInterface
    {
        return self::isNumeric(self::stringToInt());
    }

i have two more concerns.
for nullable text validation
i used to do it like this:
    /**
     * @return RuleInterface
     */
    public static function description(): RuleInterface
    {
        return self::nullable(self::isString());
    }

in case of a integer with default value (or anything default numeric with default value)
    /**
     * @return RuleInterface
     */
    public static function description(): RuleInterface
    {
        return self::isNumeric(self::stringToInt());
    }

is it a proper way?
for hasMany relationship
is there any guideline for proper validation?

description looks fine.
If you don't use required then 2 options are fine: valid input and no input. If you use required the value has to be provided and it should be valid. So if no required used some default value in database should be used.
Built-in hasMany expect all values to have the same type. It would be complicated to support many types (and corresponding validators for ids) though possible.
hasMany usage should be simple. If you have any issues with it don't hesitate to ask.

    public static function name(): RuleInterface
    {
        return self::orX(
            self::exists(Model::TABLE_NAME, Model::FIELD_NAME),
            self::stringLengthBetween(Model::MIN_NAME_LENGTH, Model::getAttributeLengths()[ Model::FIELD_NAME ])
        );
    }

and
    /**
     * @return RuleInterface
     */
    public static function uniqueName(): RuleInterface
    {
        return self::andX(
            self::unique(Model::TABLE_NAME, Model::FIELD_NAME),
            self::stringLengthBetween(Model::MIN_NAME_LENGTH, Model::getAttributeLengths()[ Model::FIELD_NAME ])
        );
    }

seems does validate properly
in case of the following input:
            $jsonInput = <<<EOT
        {
            "data" : {
                "type"  : "lessons",
                "id"    : null,
                "attributes" : {
                    "name"           : "Mathematics",
                    "description"    : "some description",
                    "display-order"  : "0"
                }
            }
        }
EOT;

with minimum length of name in model:
    /**
     * Minimum name length
     */
    const MIN_NAME_LENGTH = '3';

and maximum length of name
    /**
     * @inheritDoc
     */
    public static function getAttributeLengths(): array
    {
        return [
            self::FIELD_NAME => 100,
        ];
    }

in Validator "LessonCreate"

    public static function getAttributeRules(): array
    {
        return [
            Scheme::ATTR_NAME          => v::required(v::uniqueName()),
            Scheme::ATTR_DESCRIPTION   => v::description(),
            Scheme::ATTR_DISPLAY_ORDER => v::displayOrder(),
        ];
    }

the test throws correctly
.                                                                   1 / 1 (100%)object(Neomerx\JsonApi\Exceptions\ErrorCollection)#517 (1) {
  ["items":"Neomerx\JsonApi\Exceptions\ErrorCollection":private]=>
  array(1) {
    [0]=>
    object(Neomerx\JsonApi\Document\Error)#518 (8) {
      ["idx":"Neomerx\JsonApi\Document\Error":private]=>
      NULL
      ["links":"Neomerx\JsonApi\Document\Error":private]=>
      NULL
      ["status":"Neomerx\JsonApi\Document\Error":private]=>
      string(3) "422"
      ["code":"Neomerx\JsonApi\Document\Error":private]=>
      NULL
      ["title":"Neomerx\JsonApi\Document\Error":private]=>
      string(21) "The value is invalid."
      ["detail":"Neomerx\JsonApi\Document\Error":private]=>
      string(40) "The value should be a unique identifier."
      ["source":"Neomerx\JsonApi\Document\Error":private]=>
      array(1) {
        ["pointer"]=>
        string(21) "/data/attributes/name"
      }
      ["meta":"Neomerx\JsonApi\Document\Error":private]=>
      NULL
    }
  }
}

as "Mathematics" already exists
but in case of another lesson name which do not exists in the database, say "Geo Science"
it throws:

.                                                                   1 / 1 (100%)object(Neomerx\JsonApi\Exceptions\ErrorCollection)#517 (1) {
  ["items":"Neomerx\JsonApi\Exceptions\ErrorCollection":private]=>
  array(1) {
    [0]=>
    object(Neomerx\JsonApi\Document\Error)#518 (8) {
      ["idx":"Neomerx\JsonApi\Document\Error":private]=>
      NULL
      ["links":"Neomerx\JsonApi\Document\Error":private]=>
      NULL
      ["status":"Neomerx\JsonApi\Document\Error":private]=>
      string(3) "422"
      ["code":"Neomerx\JsonApi\Document\Error":private]=>
      NULL
      ["title":"Neomerx\JsonApi\Document\Error":private]=>
      string(21) "The value is invalid."
      ["detail":"Neomerx\JsonApi\Document\Error":private]=>
      string(49) "The value should be between 3 and 100 characters."
      ["source":"Neomerx\JsonApi\Document\Error":private]=>
      array(1) {
        ["pointer"]=>
        string(21) "/data/attributes/name"
      }
      ["meta":"Neomerx\JsonApi\Document\Error":private]=>
      NULL
    }
  }
}

i think it would be too many questions today 📦

You've found another bug :tada:
In \Limoncello\Validation\Rules\Comparisons\StringLengthBetween::__construct line 39
        parent::__construct($min, $min, ErrorCodes::STRING_LENGTH_BETWEEN, $errorContext);

should be replaced with
        parent::__construct($min, $max, ErrorCodes::STRING_LENGTH_BETWEEN, $errorContext);

all fixes will be published later this week

i have originally a scenario on validator:

validate field with database for duplicate entry (check unique) on CREATE;
validate field with databse for duplicate entry (check unique) on UPDATE, excluding the record itself  which is editing;

for previous version of limoncello-php/app
i used a dirty hack in BaseAppValidator;
snippet:
    private function exists($tableName, $columnName, $value, $primaryKey = null): bool
    {
        /** @var Connection $connection */
        $connection = $this->getContainer()->get(Connection::class);

        if ($primaryKey === null) {
            $query = $connection->createQueryBuilder();
            $query
                ->select($columnName)
                ->from($tableName)
                ->where($columnName . '=' . $query->createPositionalParameter($value))
                ->setMaxResults(1);

            $fetched = $query->execute()->fetch();
            $result = $fetched !== false;
        } else {
            list($primaryKeyName, $primaryKeyValue) = $primaryKey;

            $query = $connection->createQueryBuilder();
            $query
                ->select("`{$primaryKeyName}`, `{$columnName}`")
                ->from($tableName)
                ->where($columnName . '=' . $query->createPositionalParameter($value))
                ->setMaxResults(1);

            $fetched = $query->execute()->fetch();
            $result = $fetched !== false &&
                $fetched[$primaryKeyName] !== $primaryKeyValue;
        }

        return $result;
    }

on 0.7.x forward, i found this hack no longer usable, as i can't find the entry point for capturing the index of record.

so for Create it works fine for you now. No probs here.
As for Update I can see a few possible ways to solve the problem.
a) Do not check uniqueness of the name on update. Intercept the captured data between Validation and API and make additional check (or add the check to API level)
b) Put to app container index of the resource and add custom Validation rule that uses that value (it has an access to app container)
c) Make custom validator which actually consists of 2 validators. The first one parses the input json and the second uses capture from the first one to set up rules for the second validator. This one is the most elegant and I'm working on similar issue currently. Though it's not finished yet so it's not an option at the moment.
Currently, the easiest one would be option b).

Add to container something like CurrentIndexStorage class with methods get/set index.
Add custom rule that takes Connection and  CurrentIndexStorage from container and checks in the database.

i tried to do it in a custom validator like 'isUpdateUniqueRule'
but finally found no where to capture the index :(.. sad
ya, i think option b would be easier for me to do

b) is not a bad option it's just not universal. You can ping me at the end of the week I might have something for c).
I'm working on implementing validation for the publishing process. This involves different rules depending on the input data, validation of combined input data and data stored in the database. I hope something good may come out of this for more generic use.

the procedure would be:

create an configurator inside container folder
put get / set method
make it accessible in controller update
and use it with custom validator

am i correct?
btw, what is the convention of putting my own custom class?

i mean , is there any convention on where to put my own custom class

there are no special requirements for placing custom rules. You can put it near to custom rule example \App\Json\Validators\Rules\IsEmailRule.

the option b works; though it was not really perfect at the moment. but i can really feel the power of container wise capability!!!

@dreamsbond After some working on a bigger project I came to a conclusion that it's better to move some validator files around and structure them a bit different. It will provide a better manageability in the long run.
Here is a new layout/structure.
The key change to make it work in sub-folders is this line in settings that instructs to search in sub folders (note **).
Some new features in Validation: new rule enum, new rule filter (a very very powerful tool that can check emails, URLs, sanitize input, and etc). The filter rule is a wrapper over PHP filter_var and has identical settings. Also, chaining capability was added to all applicable rules.
As for custom validation (option c)) I've got some good news as well. In Controller you can replace default Validator with a custom one which will add extra validation and will be able to use captured data.
    protected static function createOnCreateValidator(ContainerInterface $container): JsonApiValidatorInterface
    {
        $wrapper = new CustomOnCreateValidator (parent::createOnCreateValidator($container));

        return $wrapper;
    }

    protected static function createOnUpdateValidator(ContainerInterface $container): JsonApiValidatorInterface
    {
        $wrapper = new CustomOnUpdateValidator (parent::createOnUpdateValidator($container));

        return $wrapper;
    }

An example of custom validator wrapper could be seen here.


Various error

RuntimeError: Error in __cdecl faiss::FileIOWriter::FileIOWriter(const char *) at D:\a\faiss-wheels\faiss-wheels\faiss\faiss\impl\io.cpp:98: Error: 'f' failed: could not open C:\Users\killg\Downloads\Mangio-RVC-v23.7.0/logs/König/trained_IVF404_Flat_nprobe_1_König_v1.index for writing: No such file or directory
What could I do? I'm newbie at this.
I installed all in the same folder.

I've had the same issue. In my case it was a letter outside of english alphabet in model name that caused it so try naming to model without 'ö'


On the performance of radix sort.

It seems to me that haskell version of radix sort in examples/sort/radix/ can be easily optimised to outperform HVM significantly. With just -fllvm and these three bangs added
- data Arr = Null | Leaf  Word64 | Node  Arr  Arr deriving Show
- data Map = Free | Used         | Both  Map  Map deriving Show
+ data Arr = Null | Leaf !Word64 | Node  Arr  Arr deriving Show
+ data Map = Free | Used         | Both !Map !Map deriving Show

on my Ryzen 3500U (4c8t) I get
GHC-OLD-1 | sort/radix | 20 | 5.794s
GHC-OLD-1 | sort/radix | 21 | 11.456s
GHC-OLD-1 | sort/radix | 22 | 22.594s
GHC-OLD-1 | sort/radix | 23 | 45.181s
GHC-OLD-1 | sort/radix | 24 | 117.001s

GHC-NEW-1 | sort/radix | 20 | 0.713s
GHC-NEW-1 | sort/radix | 21 | 1.451s
GHC-NEW-1 | sort/radix | 22 | 2.916s
GHC-NEW-1 | sort/radix | 23 | 5.835s
GHC-NEW-1 | sort/radix | 24 | 11.682s

HVM-1 | sort/radix | 20 | 6.007s
HVM-1 | sort/radix | 21 | 11.180s
HVM-1 | sort/radix | 22 | 21.369s
HVM-1 | sort/radix | 23 | 40.562s
HVM-1 | sort/radix | 24 | 75.365s

HVM-2 | sort/radix | 20 | 3.945s
HVM-2 | sort/radix | 21 | 6.953s
HVM-2 | sort/radix | 22 | 13.370s
HVM-2 | sort/radix | 23 | 24.608s
HVM-2 | sort/radix | 24 | 42.997s

HVM-4 | sort/radix | 20 | 2.886s
HVM-4 | sort/radix | 21 | 4.953s
HVM-4 | sort/radix | 22 | 9.374s
HVM-4 | sort/radix | 23 | 16.769s
HVM-4 | sort/radix | 24 | 27.646s

HVM-8 | sort/radix | 20 | 2.377s
HVM-8 | sort/radix | 21 | 4.121s
HVM-8 | sort/radix | 22 | 7.444s
HVM-8 | sort/radix | 23 | 14.349s
HVM-8 | sort/radix | 24 | 23.227s

I.e. single-threaded haskell version is 2-3 times faster than hvm with 8 threads. That makes me question the value of the original benchmark.
Now I really wonder what would it take to optimise hvm version to beat haskell again. I find this question really important, as I obviously like the idea of good performance on non-optimised code, but the ability to get high performance after a reasonable amount of manual optimisations is, in my opinion, much more important.

Ah, the life. GHC getting a 10x speedup with a few small changes was unexpected to me. But that's not too surprising either. The assembly generated by GHC will often be vastly superior than HVM's, simply because it has received a few orders of magnitude more investment and development - and I hope I've always been clear about that in my communication. For example, if you write a tail-recursive function, GHC will generate a tight loop, while HVM will do tons of unnecessary allocations and be much slower. That's not a limitation of the tech, but a reflect of the project's maturity. GHC was released 30 years ago and has been optimized and improved by several universities and major companies over 3 decades. Meanwhile, HVM was mostly developed by an unpaid developer, who is barely able to read assembly, over a few months of his spare time. For this case in particular, I suspect there are probably some allocations on the base case that could be skipped and inlined, resulting in the speedup you see.
What I want to show with this project though is that the underlying tech is really solid and, with proper investment, has a lot of potential to become a powerful tool for humanity. We need to compile to LLVM, we need proper inlining, unboxing, proper loops and branching, we need TCO, CSE, a strictness analyzer, hell, 95% of our use of atomics can be avoided. There is just so much we could do to make this astonishingly fast. But, for now, that's what we got :(
I hope people don't get discouraged of the tech if in some cases it won't always beat GHC today - I mean, given the astronomical investment diff, I find it a miracle that it even beats it in some cases - but I can understand and respect if that's the ultimate take here.

Wow! GHC getting a 10x speedup with a few small changes was unexpected to me. But that's not too surprising either. The assembly generated by GHC will sometimes be vastly superior than HVM's, simply because it has received a few orders of magnitude more investment and development - and I hope I've been clear about that in my communication. For example, if you write a tail-recursive function, GHC will generate a tight loop, while HVM will do tons of unnecessary allocations, making it much slower. In this case, there are several allocations on the base case that could be skipped and inlined thanks to the ! annotations, resulting in the speedup you see. In particular, both Leaf n and Both _ _ could be represented by a single word, but HVM doesn't have unboxing yet, so it is allocating >2x more memory than your unboxed GHC version.
Note that's not a limitation of the tech, but a reflect of the project's maturity. GHC was released 30 years ago and has been optimized and improved by major universities and companies over 3 decades. Meanwhile, HVM was mostly developed by a single developer over a few months of his spare time. What I want to show with this project though is that the underlying tech is really solid and, with proper investment, has a lot of potential to become a powerful tool for humanity. Once we compile to LLVM and implement inlining, unboxing, proper loops and branching, TCO, CSE, strictness/atomics analyzer, and so on, HVM will be so much faster. But, for now, that's what we got.
I hope people don't get discouraged of the tech if it doesn't always outperform GHC today - I mean, given the astronomical investment diff, it's a miracle it does so it in so many cases - but I can respect if that's the ultimate take here.

GHC getting a 10x speedup with a few small changes was unexpected to me.

Sadly, it is quite a common thing, especially with strictness-related changes. I am by no means happy about that, but so is the life.

In this case, there are several allocations on the base case that could be skipped and inlined thanks to the ! annotations, resulting in the speedup you see. ..., but HVM doesn't have unboxing yet, so it is allocating >2x more memory than your unboxed GHC version.

I should note that it is not unboxing that made most of the speedup for ghc here, but elimination of a space leak. To wit, instead of making fields strict, I could also make merge use strict application
- merge (Both a b) (Both c d) = Both (merge a c) (merge b d)
+ merge (Both a b) (Both c d) = (Both $! merge a c) $! merge b d

and it would result in following timings
GHC_STRICT_APPLY-1 | sort/radix | 20 | 0.826s
GHC_STRICT_APPLY-1 | sort/radix | 21 | 1.728s
GHC_STRICT_APPLY-1 | sort/radix | 22 | 3.443s
GHC_STRICT_APPLY-1 | sort/radix | 23 | 6.917s
GHC_STRICT_APPLY-1 | sort/radix | 24 | 13.959s

GHC_STRICT_FIELDS-1 | sort/radix | 20 | 0.703s
GHC_STRICT_FIELDS-1 | sort/radix | 21 | 1.444s
GHC_STRICT_FIELDS-1 | sort/radix | 22 | 2.876s
GHC_STRICT_FIELDS-1 | sort/radix | 23 | 5.903s
GHC_STRICT_FIELDS-1 | sort/radix | 24 | 11.660s

So while unboxing Word64 gives a notable speedup, what is really important here is avoiding space leak. As far as I understand, HVM as of today has no analogues to seq, which is super-important for performance in thunk-based lazy evaluated runtimes. In the original post I was mostly asking whether seq of some analogue is appliable to HVM.

..., so it is allocating >2x more memory than your unboxed GHC version.

I don't know how to get allocation stats with HVM, but the difference of max heap residence is indeed huge here: 1030 MB for strict-apply and 991 MB strict-fields vs. 10816 MB for HVM.
By the way, it also clearly shows that the impact of unboxing here is quite low. The real difference here makes strictness - the original, leaky program used to require about 17 GB of memory.

In particular, both Leaf n and Both _ _ could be represented by a single word, ...

I think I am lost here... I can imagine packing 60-bit integer and tag into a single word,  but I don't think I understand how you're going to pack two pointers and tag into a word. FYI, even with Word64 unboxed, Leaf and Both use 2 and 3 words of heap resp. with ghc. No magic here.

Note that's not a limitation of the tech, but a reflect of the project's maturity...

I understand that. I was merely making a point that claims that HVM can sort linked lists 9 times faster than GHC are not exactly true as of today.

I hope people don't get discouraged of the tech if it doesn't always outperform GHC today...

To be honest, as of now it seems to me more like 'HVM quite rarely outperforms GHC today'. HVM clearly wins on code requiring optimal reduction, but on a dozen of others, much more realistic, I dare to say, benchmarks I've tried HVM seems to seriously slower than properly written haskell, often even asymptotically so (likely because of #60; I would probably post some code samples there soon).

Now, to be clear, I'm by no means trying to say that HVM is useless or that you should abandon it. I understand that the project is in early stages of development, has no/little funding, etc. I just find claims like

Despite being relatively new, it already beats mature compilers in many cases, ...

..., allowing it to sort large lists 9x faster than GHC!

quite misleading with today's HVM.

I wrote two identical programs, benchmarked them and reported that HVM performed significantly better, and backed up my claims with replicable code. You then took that code, modified it by adding relevant optimizations, and pointed your changes sped it up considerably. Well, sure, I see how the fact that small changes cause GHC to outperform HVM is relevant, but I wasn't aware of that before I published the results, so it seems dishonest to call me misleading. In fact, I was careful enough to claim that HVM performs better given identical inputs, and that is indeed the case. I also never claimed that HVM outperforms GHC in all cases. Quite the opposite; through all text I write, I'm overly careful with my wording to make it clear that the improvements are limited in scope, and to only promise what I know we can deliver. That sentence you quoted, in particular, should definitely have been worded better, and I've just improved it.
What I do claim, though - and I stand by that claim - is that HVM has the potential to outperform GHC in a significant number of cases (but not all) once sufficiently mature. Issue #60 is indeed a serious theoretical issue that could be problematic, as is the fact HVM doesn't cover all λ-terms. These issues may mean it will never be able to fully replace GHC, and that's fine, since this isn't the goal at all. The goal is to provide a new tool capable of empower programmers and let them do things that weren't possible before, and I'm confident HVM is set to accomplish that goal. This thread, on the other hand, is just pointing the obvious - i.e., that GHC outperforms HVM in some cases today. You don't even need to go that far: just write a simple tail-recursive loop, and GHC will probably run it 10x faster than HVM. I never claimed otherwise and, if you see any other part of the vast written documentation that sounds misleading in that sense (or if you think I could add a remark somewhere), please let me know and I will fix it ASAP.

You then took that code, modified it by adding relevant optimizations that are only implemented on GHC (for now), and pointed your changes sped it up considerably.

And then I came here, asking for whether it is possible to optimise HVM version further. Because in my opinion, a high-performance runtime is that runtime which allows archiving good performance with enough knowledges and reasonable efforts. I definitely lack knowledges about HVM, so I asked for advice. But looks like HVM as of today just isn't a high-performance runtime according to my definition.

In fact, I was careful enough to claim that HVM performs better given identical inputs, and that is still true.

No doubt, I'm by no means saying you made up the results; they are perfectly reproducible. However, I feel like the fact that the program is suffering from severe space leak that is trivial to fix on GHC and impossible to fix on today's HVM is really important here, and deserves at least a small note. That is, I don't mean that these are the benchmark results are misleading, rather I think that the benchmark itself hardly illustrates everything about performance of HVM and GHC.

What I do claim, though - and I stand by that claim - is that HVM has the potential to outperform GHC in a significant number of cases (but not all) once sufficiently mature.

I am certainly not enough competent to judge potential of HVM, I can only hope you're right here.

This current thread, on the other hands, is just pointing the obvious - i.e., that GHC outperforms HVM in some cases today. You don't even need to go that far: just write a simple tail-recursive loop, and GHC will probably run it 10x faster than HVM.

I'm perfectly aware of that (through it seems more like 100x than 10x to me). The thing is that radix sort is mentioned in README as the case when HVM should outperform GHC, but it looks like it is correct only for concrete, inefficient (esp. in terms of memory used), implementation. The fact that GHC outperforms HVM in some (I would rather say 'most', once again) cases is indeed obvious, what I find unobvious here is what are some at least somehow real-world cases does HVM outperform GHC, if by cases I understand algorithms/tasks, not concrete (suboptimal) code fragments?

I meant that Both _ _ can be represented by a single word when both fields are a base constructor; isn't that how GHC unboxes it in that case? If no, I guess I'm mistaken and I apologize; that was just a guess.

Ah, you mean something like replacing Both Free Free with a new nullary constructor Both_Free_Free? No, GHC isn't doing anything like that, for better or worse.

Yes, we have plans to add seq, although it would need to be linear to make sense in the context of HVM.

That is good to know. I hardly can imagine a performant lazy evaluated runtime without optional strictness. Looking forward to rerunning this benchmark with strict applications once HVM would have those.

HVM still have a very naive allocator that reserves ~3/4 of the available system memory when it starts up, so it'd use that amount of memory regardless of what you ran.

I'm pretty sure that this is the real amount of physical memory used, as it clearly changes (through non-linear) with the size of the tree.

n
maximal residence, according to time -v

20
3041 MB

21
4802 MB

22
8320 MB

23
11666 MB

24
11902 MB

In case of GHC it also matches GHC's own rts stats perfectly, through it reserves terabytes of virtual memory on startup.

The benchmarks are already strong, and they're using an old, single-thread, non-compiled version of HVM; this should improve by about 10x soon. So, again, this is an entire proof checker, a complete real-world application, significantly outperforming the alternatives, which were written in Ocaml and Haskell. I can't see how one can see these overwhelming results and still claim with a straight face that HVM has no real-world application.

Sadly I cannot comment much on type checkers benchmarks, as I am completely unaware of implementations of typechecker for dependent types, except that it looks quite impressive indeed. What I find very illustrative, however, is the list-fold benchmark there. Firstly, I cannot reproduce the results with master.
(Fold Nil         c n) = n
(Fold (Cons x xs) c n) = (c x (Fold xs c n))

(Range 0 xs) = xs
(Range n xs) = let m = (- n 1); (Range m (Cons m xs))

(Main n) =
  let size = (* 1000000 n);
  let list = (Range size Nil);
  (Fold list @a@b(+ a b) 0)

import Data.Word
import System.Environment

data List a = Nil | Cons a (List a)

fold :: List a -> (a -> r -> r) -> r -> r
fold Nil         c n = n
fold (Cons x xs) c n = c x (fold xs c n)
-- Otherwise ghc would optimise `fold xs (+)` to `sum xs`, which feels like cheating.
{-# NOINLINE fold #-}

range :: Word64 -> List Word64 -> List Word64
range 0 xs = xs
range n xs = let m = n - 1 in range m (Cons m xs)

main :: IO ()
main = do
  n <- read . head <$> getArgs
  let size = 1000000 * n
  let list = range size Nil
  print $ fold list (+) 0

with HVM 0713836 and GHC 9.2.4 I'm getting following results

n
HVM
GHC
HVM / GHC

1
2.230 s ±  0.063 s
216.1 ms ±  16.1 ms
10.32

2
3.771 s ±  0.098 s
467.8 ms ±  26.9 ms
8.06

4
6.985 s ±  0.165 s
903.2 ms ±  41.5 ms
7.73

8
14.117 s ±  0.978 s
1.869 s ±  0.034 s
7.55

16
28.737 s ±  1.239 s
3.824 s ±  0.048 s
7.52

Which is quite far from 'similar performance on sequential recursive algorithms' if you ask me. I was able to get comparable results by manually optimising fold xs (+) to sum xs
(Sum Nil         a) = a
(Sum (Cons x xs) a) = (Sum xs (+ x a))

(Range 0 xs) = xs
(Range n xs) = let m = (- n 1); (Range m (Cons m xs))

(Main n) =
  let size = (* 1000000 n);
  let list = (Range size Nil);
  (Sum list 0)

import Data.Word
import System.Environment

data List a = Nil | Cons a (List a)

sun :: List Word64 -> Word64 -> Word64
sun Nil a = a
sun (Cons x xs) a = sun xs (x + a)

range :: Word64 -> List Word64 -> List Word64
range 0 xs = xs
range n xs = let m = n - 1 in range m (Cons m xs)

main :: IO ()
main = do
  n <- read . head <$> getArgs
  let size = 1000000 * n
  let list = range size Nil
  print $ sun list 0

n
HVM
GHC
HVM / GHC

1
872.4 ms ±  57.6 ms
132.2 ms ±  10.1 ms
6.60

2
1.041 s ±  0.046 s
313.7 ms ±  16.5 ms
3.32

4
1.248 s ±  0.056 s
610.0 ms ±  15.1 ms
2.05

8
1.731 s ±  0.050 s
1.151 s ±  0.019 s
1.50

16
2.807 s ±  0.101 s
2.464 s ±  0.032 s
1.14

32
4.649 s ±  0.378 s
5.243 s ±  0.044 s
0.88

which on its own looks quite good for HVM: it is even outperforming GHC on 32M. My point is, however, that both programs are utterly silly: they are materialising whole list before starting summation! Why would you do that in lazy language? And once we rewrite the program to generate the list lazily, on-demand, results are quite different.
(Range 0) = Nil
(Range n) = let m = (- n 1); (Cons m (Range m))

(Sum Nil         a) = a
(Sum (Cons x xs) a) = (Sum xs (+ x a))

(Main n) =
  let size = (* 1000000 n);
  let list = (Range size);
  (Sum list 0)

import Data.Word
import System.Environment

data List a = Nil | Cons a (List a)

sun :: List Word64 -> Word64 -> Word64
sun Nil a = a
sun (Cons x xs) a = sun xs (x + a)

range :: Word64 -> List Word64
range 0 = Nil
range n = let m = n - 1 in Cons m (range m)

main :: IO ()
main = do
  n <- read . head <$> getArgs
  let size = 1000000 * n
  let list = range size
  print $ sun list 0

n
HVM
GHC
HVM / GHC

1
890.3 ms ±  72.6 ms
23.9 ms ±   3.2 ms
37.21

2
30.0 ms ±  12.2 ms
1.011 s ±  0.061 s
33.68

4
1.271 s ±  0.050 s
47.3 ms ±  12.0 ms
26.86

8
1.849 s ±  0.226 s
74.5 ms ±  13.4 ms
24.83

16
2.897 s ±  0.062 s
129.2 ms ±  15.4 ms
22.43

32
4.910 s ±  0.200 s
246.6 ms ±  11.8 ms
19.91

64
8.134 s ±  0.754 s
476.6 ms ±  23.1 ms
17.07

128
13.652 s ±  1.212 s
936.2 ms ±  21.2 ms
14.58

Oops...
This is why I'm saying I haven't really seen any properly written programs that HVM is executing faster or at least comparable to GHC. It is definitely possible to construct infinitely similar programs HVM would outperform GHC on; the thing is that, in my opinion, it doesn't matter that much how fast the runtime is on executing deliberately suboptimal programs if it is bad at executing near-optimal ones, which is, as far as I can see, exactly the case for HVM.

Anyway, I think I got the answer I was looking for opening this issue --  seq for HVM is possible, just not implemented yet. I'm closing this, then.

I'm pretty sure that this is the real amount of physical memory used, as it clearly changes (through non-linear) with the size of the tree.

Yes I misunderstood what you was measuring. Point takes.

But looks like HVM as of today just isn't a high-performance runtime according to my definition.

Yes, absolutely. HVM is meant to let us have a sneak peek on what would be possible if this tech was mature enough, and it stands no change of outperforming GHC in all cases, as it doesn't have a fraction of the micro optimizations.
I didn't have the time to investigate your fold implementations and will do that soon (some of these results look like a bug), but one thing that I've noticed, and I hope you take that in consideration, is that most of these optimizations seem to fall in the micro-benchmark trap. As in, seems like GHC is taking radix sort, some of the fold examples, and compiling to tight ASM, in a way that isn't representative of real-world programs, where data comes from the outside, immutable structures and memory allocation dominate, pattern-matching amounts for most computational cost, and garbage-collection becomes an issue.

what I find unobvious here is what are some at least somehow real-world cases does HVM outperform GHC

Ironically, I do feel like GHC has a big edge on these micro-benchmarks, while HVM would perform much better in real world applications, where the parallelization opportunities are vast, the GC-freedom is relevant, and you can't magically turn small algorithms into tight C-like loops or pre-compute a lot of things at compile time. And this can be seen when you acknowledge that Kind performs much better than Agda, a proof checker written in Haskell. Both projects are large enough, and complex enough, that no amount of unboxing, strictness annotations and similar micro optimization wizardry will change their results significantly. Kind is the first real-world application to be developed in HVM, and I believe we'll see that more and more as new applications are developed.
All in all, I get your take and respect it. You're absolutely right that HVM isn't in all cases comparable with GHC today, and I hope it is clear that I never claimed that, but I do believe it already has the edge in important cases, and I believe this will only improve as HVM has a lot of room to grow, while GHC is more or less stable. I'll keep working to push this tech forwards and will keep improving and reporting the results, but I hope you take the arguments I'm giving from the lens of someone who is actually willing to concede, in the same way I'll concede if the ultimate conclusion is that HVM doesn't have a lot of practical value and will always be just a experiment.

Hello again. HVM now has strict arguments. I've also added a bunch of optimizations; see the commit above. You can now emulate strict constructors on HVM by writing an auxiliary function, as follows:
// Strict Constructors
(BOTH !a !b) = (Both a b)
(LEAF !a) = (Leaf a)

These changes nearly doubled the performance of Radix Sort on HVM. On my machine (Apple M1 Max), sorting a list with 2^25 elements on GHC, with your proposed optimizations, takes 12.5 seconds. On HVM, with the new changes, it now takes 5.5 seconds. So, while your changes do cause GHC outperform HVM in a single thread, HVM is faster with 4 or more threads. I hope we'll be able to close that gap as we add more optimizations.


Adds missing import statement

The file uses URL, but it wasn't being imported.

Why is this necessary?

@nixta Good question. It isn't necessary per se, but it is helpful. The import currently isn't needed because UIKit (and therefore Foundation) is implicitly imported by importing SVProgressHUD.h in the bridging header.
Technically we don't need to import UIKit anywhere in the project, but we do because it is the right thing to do. SVProgressHUD could be swiftified some day or go away entirely. Source files may be moved around or copied to other projects. We guard against those changes potentially breaking the build by explicitly importing dependencies in our source files.
Does that make sense?


e2e: use latest tag for e2e test and delete 0.2.0 tag

update image tag
fix kubectl missing kubeconfig

What type of PR is this?

/kind failing-test
What does this PR do?

Which issue(s) does this PR fix?

Fixes #
Special notes for your reviewer:

@duanmengkk
Does this PR introduce a user-facing change?

fix the ci


Calling a deprecated method or property in the Python API gets no warning

According to our @decorators.deprecated implementation, https://github.com/Vector35/binaryninja-api/blob/545fef9216559d08f58f7e20082b36352203a8cf/python/decorators.py#L15-L26, it does not warn the user when the function or property is called.
This might be desired for not spamming the  console, however, since the user would not be able to know he is using a deprecated method, it is not very useful either

There is a https://pypi.org/project/deprecation/ that might help

It actually works fine for both regular functions and properties:
import deprecation

@deprecation.deprecated(details="Use the bar function instead")
def foo():
    """Do some stuff"""
    return 1

class test:
    def __init__(self) -> None:
        pass

    @property
    @deprecation.deprecated(details="Use the def instead")
    def abc(self):
        return 123

foo()

a = test()
print(a.abc)

Fixed in https://github.com/Vector35/binaryninja-api/commit/e4ab8f35ce16baafe11d7056d1d1304f3c057cd5


Use "main" branch for audit_check

@taiki-e It looks like the newer versions of rustsec/audit-check require Cargo.lock to be committed to the repository. I know that Cargo.lock is ignored for a reason. Is it worth it to start committing it?
Alternatively I'm fine with removing this check. It's not like smol-rs crates depend on many other crates that aren't also smol-rs crates.

require Cargo.lock to be committed to the repository.

No. We can run cargo generate-lockfile like https://github.com/smol-rs/fastrand/commit/cfc0ab0ef7ace7e32e4dadee7367ad10062c86a0 instead.

Alternatively I'm fine with removing this check. It's not like smol-rs crates depend on many other crates that aren't also smol-rs crates.

I'm also fine with removing this check.
(Btw, projects under github.com/taiki-e are using cargo-deny+cargo-no-dev-deps which is less false positives way.)

No. We can run cargo generate-lockfile like smol-rs/fastrand@cfc0ab0 instead.

Good call, thanks!


Build SS7MAPer from src，it occurs “fatal error: ei.h: No such file or directory”

I tried to build SS7MAPer from src to set it up . But when I execute the command of “Build the code”，that is “rebar co”，it issues：
Compiling /root/apps/ss7Mapper/ss7MAPer/deps/epcap/c_src/epcap.c
/root/apps/ss7Mapper/ss7MAPer/deps/epcap/c_src/epcap.c:32:10: fatal error: ei.h: No such file or directory
#include <ei.h>
^~~~~~
compilation terminated.
ERROR: compile failed while processing /root/apps/ss7Mapper/ss7MAPer/deps/epcap: rebar_abort
How can i handle this problem？Any suggestion？Thanks.

I tried to build SS7MAPer from src to set it up . But when I execute the command of “Build the code”，that is “rebar co”，it issues：
Compiling /root/apps/ss7Mapper/ss7MAPer/deps/epcap/c_src/epcap.c
/root/apps/ss7Mapper/ss7MAPer/deps/epcap/c_src/epcap.c:32:10: fatal error: ei.h: No such file or directory
#include <ei.h>
^~~~~~
compilation terminated.
ERROR: compile failed while processing /root/apps/ss7Mapper/ss7MAPer/deps/epcap: rebar_abort
How can i handle this problem？Any suggestion？Thanks.

OK，Thanks，I got the answer at #8 error ei.h https://github.com/ernw/ss7MAPer/issues/8


Various BuildTransportPackage.ps1 Refactors and Fixes

Added LastExitCode checks in various parts of the BuildTransportPackage.ps1 script.
Move Microsoft.Telemetry.Inbox.nupkg restore back to yml so that it can work with Nuget Authenticate on the feed.
Rename BuildTransportPackage.ps1 to BuildAll.ps1.


Repurpose tags for Activity Code (CCAs, School Events etc)

JUnit Tests
Rename/Refactor Tags and Related methods
Update Json Files with Tags


Implement Error response object

As oneview-redfish-toolkit developer, I should have an error response structure to map error conditions throughout the code. Current DMTF's Redfish specification (available at: https://www.dmtf.org/sites/default/files/standards/documents/DSP0266_1.3.0.pdf) in the section 6.5.6, there is a well-defined format for error messages composed by an error code, a message and an array with extended info objects. This scope of this issue is to implement this object.

Schemas available at: http://redfish.dmtf.org/schemas/registries/archive/DSP8011_2017.1.zip

Message index available at Base.1.1.0.json at http://redfish.dmtf.org/schemas/registries/archive/DSP8011_2017.1.zip

Message schema: http://redfish.dmtf.org/schemas/v1/Message.v1_0_5.json


Regular app audit

Type of change (place an x in the [ ] that applies)

[ ] New sample
[ ] New feature
[x] Bug fix
[ ] Documentation

Summary
This pull request audits the sample app and makes the following changes:

Update .gitignore to only ignore the local apps
Add a default .slack/apps.json with the JSON content {}
Removes the deno.lock
Disables the deno lock file

Group discussion decided to disable it and allow developers to opt-in to using it

Minor comment tweaks
Fixes a date format issue introduced by the latest Deno runtime (v1.33.x)

Before: new Date().toLocaleDateString("en-CA") returns YYYY-MM-DD
Now: new Date().toLocaleDateString("en-CA") returns mm/dd/YYYY
This caused a runtime validation error when the Workflow was started
Thankfully, the date format seems to work fine for the datastore expressions

Tested functionality:

[x] Linting & Tests pass
[x] Logged a run
[x] Displayed the leaderboard
[x] Scheduled the displaying of the leaderboard

Requirements (place an x in each [ ] that applies)

[x] I’ve checked my submission against the Samples Checklist to ensure it complies with all standards
[x] I have ensured the changes I am contributing align with existing patterns and have tested and linted my code
[x] I've read and agree to the Code of Conduct

Before: new Date().toLocaleDateString("en-CA") returns YYYY-MM-DD
Now: new Date().toLocaleDateString("en-CA") returns mm/dd/YYYY

IMO, we might want to standardize around the (wonderful) ISO 8601 format for dates if we're making these changes. mm/dd/YYYY seems to be in a confusing order?


first travis

travis ci!

Thank you for doing the lesson, will close this out since we want folks to be able to do this on forks.


Add a Gitter chat link to README.mkd

zsh-users/antigen now has a Chat Room on Gitter
@desyncr has just created a chat room. You can visit it here: https://gitter.im/antigen-zsh/Lobby.
This pull-request adds a link to your README.mkd.
If my aim is a little off, please let me know.
Happy chatting.
PS: Click here if you would prefer not to receive automatic pull-requests from Gitter in future.

Addressed in #183


fix go-report and sonar identified issues

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage increased (+1.3%) to 86.034%

Files with Coverage Reduction
New Missed Lines
%

prompt/prompt_autocomplete.go
1
92.06%

Totals

Change from base Build<PHONE_NUMBER>:
1.3%

Covered Lines:
1959

Relevant Lines:
2277

💛  - Coveralls


[GSSOC'23]Adding a footer in hacktoberfest section

Is your feature request related to a problem? Please describe.
Adding a footer in hacktoberfest section
Describe the solution you'd like.
improves ui
Describe alternatives you've considered.

Add any other context or screenshots about the feature request here.
No response

Someone working on the same issue


[BUG] OutOfMemorySparkException only when including a validationIndicatorCol - LightGBMClassifier

SynapseML version
OutOfMemorySparkException only when including a validationIndicatorCol - LightGBMClassifier
System information

Language version ( python 3.10.12, scala 2.12):
Spark Version ( 3.4.1):
Spark Platform (Databricks RunTime 13.3 LTS)

Describe the problem
I have a dataset written to S3 which was created using pyspark.ml's OneHotEncoder and VectorAssembler. So the dataset written to S3 has 3 columns:

Features: a sparse representation of my original features. Features were originally a mix of numerical and categorical, the categoricals got OneHotEncoded
Target column (binary)
EvalFlag (boolean)

When I don't set a validationIndicatorCol in the classifier's constructor, training succeeds. However, as soon as I set validationIndicatorCol='EvalFlag' in the constructor, I'm getting the error #org.apache.spark.sql.execution.OutOfMemorySparkException: Total memory usage during row decode exceeds spark.driver.maxResultSize (60.0 GiB). The average row size was 626.0 B, with 29.0 GiB used for temporary buffers. [shuffleId: None]
Note that I'd already increased spark.driver.MaxResultSize from whatever its default is, to 60gb. Even when it was its default value (which I gather would be much smaller), training without the evaluation flag worked just fine.
So something about including an evaluation set has massively increased the requirements on spark.driver.MaxResultSize
Code to reproduce issue
from pyspark.ml import Pipeline
from pyspark.ml.feature import OneHotEncoder, VectorAssembler
from pyspark.sql import functions as F
from synapse.ml.lightgbm import LightGBMClassifier

df_raw= spark.read.parquet("s3path.pq")

categorical_columns = ["x", "y", "z", ..]
dense_columns = ['a', 'b',  'c', ...]
#df_raw on disk has `categorical_columns`, `dense_columns` and `'target'` as columns

ohe_stage = [OneHotEncoder(inputCol=col, outputCol=f"{col}_ohe") for col in categorical_columns]
ohe_output_columns = [f"{col}_ohe" for col in categorical_columns]
assembler = VectorAssembler(
    inputCols=dense_columns + ohe_output_columns,
    outputCol="features",
    handleInvalid='keep'
)
pipeline = Pipeline(stages=ohe_stage + [assembler])
pipeline_model = pipeline.fit(df_raw)
df_transformed = (pipeline_model.transform(df_raw)
                               .withColumn("EvalFlag", F.rand())
                               .withColumn("EvalFlag", F.col("EvalFlag")<0.1)
)

#optionally either write df_transformed to s3 to materialize the transformation before training, and then read back in, or just pass df_transformed to training, doesn't change the outcome

#df_transformed.write.parquet('s3pathdftransformed.pq')
#df_transformed = spark.read.parquet('s3pathdftransformed.pq')

lightgbm = LightGBMClassifier(
    featuresCol="features",
    labelCol="target",
    categoricalSlotNames=categorical_columns,
    maxDepth=10,
    numLeaves=200,
    learningRate=0.1,
    numIterations=500,
    earlyStoppingRound=10,
    validationIndicatorCol='EvalFlag', #toggle this line on and off
    verbosity=1
)

lightgbm_model = lightgbm.fit(df_transformed)

Other info / logs
No response
What component(s) does this bug affect?

[ ] area/cognitive: Cognitive project
[ ] area/core: Core project
[ ] area/deep-learning: DeepLearning project
[x] area/lightgbm: Lightgbm project
[ ] area/opencv: Opencv project
[ ] area/vw: VW project
[ ] area/website: Website
[ ] area/build: Project build system
[ ] area/notebooks: Samples under notebooks folder
[ ] area/docker: Docker usage
[ ] area/models: models related issue

What language(s) does this bug affect?

[ ] language/scala: Scala source code
[ ] language/python: Pyspark APIs
[ ] language/r: R APIs
[ ] language/csharp: .NET APIs
[ ] language/new: Proposals for new client languages

What integration(s) does this bug affect?

[ ] integrations/synapse: Azure Synapse integrations
[ ] integrations/azureml: Azure ML integrations
[ ] integrations/databricks: Databricks integrations

Facing the same issue

This may be because the validation dataset is loaded in memory on every executor. Therefore, a large validation dataset may cause out of memory errors

yes, I think that's almost certainly right. I would regard this as a bug...or at least a very significant drawback warranting a feature request for it to not work this way (see for example dask xgboost). If you want to reap the benefits of training on large data, you're going to want to have validation sets that scale somewhat.

From the source code perspective, validateData is broadcasted to each executor without any compression processing, which leads to a significant consumption of memory. Therefore, currently, the only way to avoid this problem is to reduce the size of validateData to prevent it from occupying too much memory.

This may be because the LightGBM C++ library does not support streaming validation datasets, but does support streaming training datasets. Therefore, I think the room for improvement in SynapseML is limited. At most, it can change the way the validation data is loaded from a broadcast method to another method that saves more space, and immediately release it after loading. However, it always has to fully load the validation dataset with the LightGBM C++ library, so the optimization space should be limited.


password_reset: Soft reactivate upon password reset

Fixes #22066

The PR is ready for review.

Merged, after some small tweaks to the documentation text, thanks @PIG208!
If you're interested, the most important issue around soft-deactivation is that it can be a bad experience if loading the home view takes a long time due to soft reactivation being slow.
We should discuss the API logistics in more detail in #backend, but I think basically we want loading the app to serve a small page that communicates "It's been a while! We're reactivating your account..." during this loading period, rather than appearing to hang.


Bug: Opening graphs in destroy.vizor.io doesn't seem to work

Replicate:
Open this file and press play:
http://destroy.vizor.io/kschzt/pyoriva-robo/edit
Screenshot from console attached.


[3.8] Remove Facade in Queue Service Provider

Port of #2149 for 3.8

Codecov Report

Merging #2152 (7b1ae26) into master (958c4ed) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##             master    #2152   +/-   ##
=========================================
  Coverage     87.01%   87.01%
  Complexity      664      664
=========================================
  Files            33       33
  Lines          1556     1556
=========================================
  Hits           1354     1354
  Misses          202      202

Impacted Files
Coverage Δ
Complexity Δ

...Jenssegers/Mongodb/MongodbQueueServiceProvider.php
83.33% <100.00%> (ø)
2.00 <0.00> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 958c4ed...7b1ae26. Read the comment docs.


Deadlink causing error when downloading yolov7 release .pt

Before:
https://api.github.com/repos/WongKinYiu/yolov7/releases/latest
which shows:
{
  "message": "Not Found",
  "documentation_url": "https://docs.github.com/rest/releases/releases#get-the-latest-release"
}

This is causing error:
Traceback (most recent call last):
  File "./yolov7/utils/google_utils.py", line 27, in attempt_download
    assets = [x['name'] for x in response['assets']]  # release assets
KeyError: 'assets'

Switch it to /tags/v0.1 to solve it!
https://api.github.com/repos/WongKinYiu/yolov7/releases/tags/v0.1

Hello~
Thanks for your update and contribution!


added headers to program::compile

Coverage increased (+0.02%) to 83.357% when pulling e845a80ab8bc4c25bcf4f086e030d1ee43b96ed0 on Slonegg:compile_program_with_headers into 86211062a771ab0b494116b4223b63a292ad7d11 on boostorg:develop.

@kylelutz do you think we can add support for OpenCL headers to boost::compute? They're convenient if you include them in multiple files. Caching would be nice too...

Caching would be nice too...

I think for caching result of clCompileProgram(..) you would need to store program source hash in boost::compute::program class.

Yes.
But is this an issue? The hash is only 160 bits.
But more difficult seems to figure out what is really included by the compiler itself, which is outside of the Boost.Compute realm...
This will depend on the options passed to the compiler that we need to understand, such as the -I paths and so on...
So the best place to do this kind of caching is inside the OpenCL stack itself, such as done by PoCL or nVidia.

I guess including all headers, compilations options and source of the program in hash should be more or less robust solution. Maybe warn user if he uses -I option, because it may trick caching system.

This will depend on the options passed to the compiler that we need to understand, such as the -I paths and so on...
So the best place to do this kind of caching is inside the OpenCL stack itself, such as done by PoCL or nVidia.

We already do offline-caching of OpenCL programs... so I imagine even right now you can pass -I option to build options and shoot yourself in foot by changing file/s you included. I think he only way to make Boost.Compute caching 100% robust would be to base hash not on program source, but on IL or binary. Unfortunately, binaries or IL are usually much bigger.
Anyway, MR looks ok. I'll merge it. Thanks!


Fixed NoData flicker while loading repo list

Fix for https://harness.atlassian.net/browse/PIPE-23625
Before the fix, a NoData UI appeared when the repo list page was refreshed. This PR fixes the issue.
https://github.com/user-attachments/assets/f22c1e71-e887-46a6-954c-f257379a0e25

@vivek-harness all the lint rules are failing. Please check.


Add missing permissions leading to crash on startup

WAKE_LOCK is required for acquiring wifi lock

https://github.com/alvr-org/ALVR/blob/2a43bb9c3e12845c132ae8f1b2963b8c78c3c330/alvr/client_core/src/platform/android.rs#L186-L198

ACCESS_WIFI_STATE for getting wifi state

https://github.com/alvr-org/ALVR/blob/2a43bb9c3e12845c132ae8f1b2963b8c78c3c330/alvr/client_core/src/platform/android.rs#L149-L159

for.eg.
04:24:28.502  3683-3683  [ALVR NATIVE-RUST]      pid-3683                             E  acquiring wifi lock failed with error Type:JavaException
04:24:28.503  3683-3683  lient.cardboar          pid-3683                             A  thread.cc:2361] No pending exception expected: java.lang.SecurityException: Neither user 10371 nor current process has android.permission.WAKE_LOCK.
	 thread.cc:2361] (Throwable with no stack trace)
	 thread.cc:2361] Caused by: android.os.RemoteException: Remote stack trace:
	 thread.cc:2361] 	at android.app.ContextImpl.enforce(ContextImpl.java:2209)
	 thread.cc:2361] 	at android.app.ContextImpl.enforceCallingOrSelfPermission(ContextImpl.java:2237)
	 thread.cc:2361] 	at android.content.ContextWrapper.enforceCallingOrSelfPermission(ContextWrapper.java:907)
	 thread.cc:2361] 	at com.android.server.wifi.WifiServiceImpl.acquireWifiLock(WifiServiceImpl.java:3920)
	 thread.cc:2361] 	at android.net.wifi.IWifiManager$Stub.onTransact(IWifiManager.java:1044)
	 thread.cc:2361]
	 thread.cc:2361] (Throwable with no stack trace)
	 thread.cc:2361]
04:24:28.508  3683-3707  [ALVR NATIVE-RUST]      pid-3683                             E  What happened:
	 panicked at 'called `Result::unwrap()` on an `Err` value: JavaException', alvr\client_core\src\platform\android.rs:157:10

	 Backtrace:
		0: <unknown>
		1: <unknown>
		2: <unknown>
		3: <unknown>
		4: <unknown>
		5: <unknown>
		6: <unknown>
		7: <unknown>
		8: <unknown>
		9: <unknown>
	   10: <unknown>
	   11: <unknown>
	   12: <unknown>
	   13: <unknown>
	   14: <unknown>
	   15: <unknown>

Thanks!


flutter pub get failing to fetch from this repo

Main Issues

Failing to fetch from this repository when I run flutter pub get  from of the projects that depend
Processing of launching is throwing an exception like so:

Git error. Command: git fetch
stdout:
stderr: fatal: unable to access 'https://github.com/electronic-jedi/dart_utils.git/': Could not resolve host: github.com
exit code: 128
pub get failed (server unavailable) -- attempting retry 1 in 1 second...
Git error. Command: `git rev-list --max-count=1 v1.3.5+18`
stdout:
stderr: fatal: ambiguous argument 'v1.3.5+18': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
exit code: 128
pub get failed (server unavailable) -- attempting retry 2 in 2 seconds...
Git error. Command: `git rev-list --max-count=1 v1.3.5+18`
stdout:
stderr: fatal: ambiguous argument 'v1.3.5+18': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
exit code: 128
pub get failed (server unavailable) -- attempting retry 3 in 4 seconds...
Git error. Command: `git rev-list --max-count=1 v1.3.5+18`
stdout:
stderr: fatal: ambiguous argument 'v1.3.5+18': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
exit code: 128
pub get failed (server unavailable) -- attempting retry 4 in 8 seconds...
Git error. Command: `git rev-list --max-count=1 v1.3.5+18`
stdout:
stderr: fatal: ambiguous argument 'v1.3.5+18': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
exit code: 128
pub get failed (server unavailable) -- attempting retry 5 in 16 seconds...
Git error. Command: `git rev-list --max-count=1 v1.3.5+18`
stdout:
stderr: fatal: ambiguous argument 'v1.3.5+18': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
exit code: 128
pub get failed (server unavailable) -- attempting retry 6 in 32 seconds...
Running "flutter pub get" in flutter_project-1...

It is then going into a loop of retrying  and never gets done. My internet and firewall configs are okay

@chilusoft kindly state how you managed to resolve the issue.


rustc requirement is 1.45 not 1.44

error[E0658]: procedural macros cannot be expanded to statements
  --> /usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/fixed-macro-impl-1.0.0/src/dispatch.rs:8:9
   |
8  | /         paste! {
9  | |             if ($int_bits, $frac_bits, $signed) == ($i, $f, true) {
10 | |                 return fixed::types::[<I $i F $f>]::from_str($s)
11 | |                     .map(|x| x.to_bits()).map(Literal::[<i $w _unsuffixed>])
...  |
15 | |             }
16 | |         }
   | |_________^
...
27 |       fixed_to_literal!(int_bits, frac_bits, signed, s, 8, 8, 0);
   |       ----------------------------------------------------------- in this macro invocation
   |
   = note: see issue #54727 <https://github.com/rust-lang/rust/issues/54727> for more information
   = note: this error originates in a macro (in Nightly builds, run with -Z macro-backtrace for more info)

Ohh, absolutely right, the expanding-to-statements is some fairly new stuff.
I'll change the MSRV to 1.45 then (and maybe add an environment on CI pinned to that as well).


Can't be used any longer

Seems Douyu has an update on it's mechanism and now this plugin can not successfully replace the flash player with html5 player. This happens for many people I know so I guess this should not be my own devices' problem.

Douyu update api.

https://github.com/spacemeowx2/DouyuHTML5Player/blob/master/README.md

0.6.0 should work


GO:0052040 move synonym

modulation by symbiont of host programmed cell death
has narrow synonym
effector-triggered necrosis
this should move down to be exact synonym of
GO:0080185 effector-triggered induction by symbiont of plant hypersensitive response
"A process stimulated by a molecule secreted by a symbiont that activates a host a plant hypersensitive response in order to induce plant cell necrosis. The host is defined as the larger of the organisms involved in a symbiotic interaction.

Also add the synonym
effector-triggered susceptibility (ETS) PMID:22241993
to
GO:0080185
Re @CuzickA #18310
(double check we did not add this to any other term by mistake, it should be OK as we did not create this yet)