Docs mentioning SFC do not demonstrate an SFC

(moved from https://github.com/sveltejs/svelte/issues/1601)
Current documentation
From https://svelte.technology/guide#understanding-svelte-components

A component is a reusable self-contained block of code that encapsulates markup, styles and behaviours that belong together.
Like Ractive and Vue, Svelte promotes the concept of single-file components: a component is just an .html file. Here's a simple example:

Hello {name}!

As only HTML is used in the .html file, this is either not a demonstration of, or a poor demonstration of the concept of including JS, HTML and CSS in a single file.
Additionally, since the file is called .html there is nothing to suggest there should be more inside the file than what is shown, ie, HTML.
Furthermore the component has data that is given to it later, on instatiation.
Suggestion
In the example below, I have avoided the use of obtuse language, tightened the copy and avoided the term 'SFC' as it does not serve a purpose - Svelte components are explained to be in a single file, we do not need a new acronym for this
Understanding Svelte components
Svelte applications are made from one or more components. In Svelte, like other frameworks, a component is a reusable self-contained block of code with HTML, CSS and JS in a single file.

Hello {name}!

Copying a comment mde earlier: while not all the sections are mandatory - we could certainly have an even more minimal example - the question is how well does it illustrate the concept of a single file component? The purpose of documentation is to explain things to users, not to be ultra optimised.

Thanks. I suppose the intention was to indicate that


Nested status fields dirty fix

Hello!
I'm working on provider-jet-yc based on terraform-provider-yandex.
In our case there is fqdn field which is computed and should be Observation filed. Here stores FQDNs of hosts in cluster. We need this field in case building connection string.
fqdn field nested in host subresource and because #27 terrajet loses them.
This PostgresqlClusterHostObservation struct generated with fresh terrajet from main branch.
type PostgresqlClusterHostObservation struct {
	Fqdn *string `json:"fqdn,omitempty" tf:"fqdn,omitempty"`

	ReplicationSource *string `json:"replicationSource,omitempty" tf:"replication_source,omitempty"`

	Role *string `json:"role,omitempty" tf:"role,omitempty"`
}

But struct not present in main Observation:
type PostgresqlClusterObservation struct {
	CreatedAt *string `json:"createdAt,omitempty" tf:"created_at,omitempty"`

	Health *string `json:"health,omitempty" tf:"health,omitempty"`

	Status *string `json:"status,omitempty" tf:"status,omitempty"`
}

Description of your changes
I've made very dirty fix for #27 issue. Fix requires lot of changes in types/builder.go. And I've not found better solution then return multiple types from buildSchema and also wrap type with isObservation flag.
After fix I've generate apis and on postgresqlcluster resource I have:
type PostgresqlClusterObservation struct {
	Config []ConfigObservation `json:"config,omitempty" tf:"config,omitempty"`

	CreatedAt *string `json:"createdAt,omitempty" tf:"created_at,omitempty"`

	Database []PostgresqlClusterDatabaseObservation `json:"database,omitempty" tf:"database,omitempty"`

	Health *string `json:"health,omitempty" tf:"health,omitempty"`

	Host []PostgresqlClusterHostObservation `json:"host,omitempty" tf:"host,omitempty"`

	ID *string `json:"id,omitempty" tf:"id,omitempty"`

	MaintenanceWindow []PostgresqlClusterMaintenanceWindowObservation `json:"maintenanceWindow,omitempty" tf:"maintenance_window,omitempty"`

	Restore []RestoreObservation `json:"restore,omitempty" tf:"restore,omitempty"`

	Status *string `json:"status,omitempty" tf:"status,omitempty"`

	User []PostgresqlClusterUserObservation `json:"user,omitempty" tf:"user,omitempty"`
}

I have:

[ ] Read and followed Crossplane's contribution process.
[ ] Run make reviewable to ensure this PR is ready for review.
[ ] Added backport release-x.y labels to auto-backport this PR if necessary.

How has this code been tested
On my provider-jet-yc.

Thanks @vaspahomov ! I'll review this next week. As a side note, what I had in mind for this issue was that we could possibly have a tool that takes the field path and type of the field, then create it in the status with that path, i.e. how mkdir -p works with directories. But I'm a bit rusty on this part of the codebase now, so I'll think about it more when I do the review. And totally open to other approaches as well.

@vaspahomov The issue has been fixed in https://github.com/crossplane/terrajet/pull/230 together with an extensive refactor. Thanks for your PR!


zmq.error.ZMQError: Operation cannot be accomplished in current state

Thank you for your great work!
Describe the bug
The program can run normally. But when I open the viewer, the program will report the following error. However, no other programs are running in the system. I don't think this is a simple memory overflow problem.
RuntimeError: CUDA out of memory. Tried to allocate 19.04 GiB (GPU 0; 15.73 GiB total capacity; 273.18 MiB already allocated; 12.65 GiB free; 350.00 MiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
zmq.error.ZMQError: Operation cannot be accomplished in current state.

Are you running the latest version of the code? This was an issue like this for a brief period that has since been fixed.
If you are running the latest, which model are you training? What is your training command?

Greetings,
Using the latest version 0.1.12, facing the same error with instant-ngp (even when loading pre-trained models), viewer loads but crashes as soon as one navigates within it.  200W / 270W |   7267MB /  7981MB are the stats on the 3070 desktop I am using. Nerfacto is working fine.

I am using Nerfacto. After I updated the code, reinstalled tinycudann, and restarted the computer, the code worked fine. Thank you!

Having the same issue with latest docker image when running instantngp. (0.1.14)

instant-ngp can use a lot of memory ( it is scene dependent). How much memory does your GPU have?

instant-ngp can use a lot of memory ( it is scene dependent). How much memory does your GPU have?

@tancik For me it is a bit different, it is also a zmq error finally but the complete log is this:
Viewer at: https://viewer.nerf.studio/versions/22-12-16-0/?websocket_url=ws://localhost:7007
Printing profiling stats, from longest to shortest duration in seconds
ViewerState._render_image_in_viewer: 0.0672
Trainer.train_iteration: 0.0386
VanillaPipeline.get_train_loss_dict: 0.0257
Traceback (most recent call last):
  File "/home/user/nerfstudio/nerfstudio/viewer/server/viewer_utils.py", line 814, in _render_image_in_viewer
    render_thread.join()
  File "/home/user/nerfstudio/nerfstudio/viewer/server/viewer_utils.py", line 161, in join
    raise self.exc
  File "/home/user/nerfstudio/nerfstudio/viewer/server/viewer_utils.py", line 147, in run
    outputs = self.graph.get_outputs_for_camera_ray_bundle(self.camera_ray_bundle)
  File "/home/user/.local/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/home/user/nerfstudio/nerfstudio/models/base_model.py", line 176, in get_outputs_for_camera_ray_bundle
    outputs = self.forward(ray_bundle=ray_bundle)
  File "/home/user/nerfstudio/nerfstudio/models/base_model.py", line 138, in forward
    return self.get_outputs(ray_bundle)
  File "/home/user/nerfstudio/nerfstudio/models/instant_ngp.py", line 181, in get_outputs
    field_outputs = self.field(ray_samples)
  File "/home/user/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/user/nerfstudio/nerfstudio/fields/base_field.py", line 111, in forward
    field_outputs = self.get_outputs(ray_samples, density_embedding=density_embedding)
  File "/home/user/nerfstudio/nerfstudio/fields/instant_ngp_field.py", line 174, in get_outputs
    rgb = self.mlp_head(h).view(*ray_samples.frustums.directions.shape[:-1], -1).to(directions)
  File "/home/user/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/user/.local/lib/python3.10/site-packages/tinycudann/modules.py", line 156, in forward
    x_padded.to(torch.float).contiguous(),
RuntimeError: CUDA error: unknown error
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/.local/bin/ns-train", line 8, in <module>
    sys.exit(entrypoint())
  File "/home/user/nerfstudio/scripts/train.py", line 248, in entrypoint
    main(
  File "/home/user/nerfstudio/scripts/train.py", line 234, in main
    launch(
  File "/home/user/nerfstudio/scripts/train.py", line 173, in launch
    main_func(local_rank=0, world_size=world_size, config=config)
  File "/home/user/nerfstudio/scripts/train.py", line 88, in train_loop
    trainer.train()
  File "/home/user/nerfstudio/nerfstudio/engine/trainer.py", line 187, in train
    self._update_viewer_state(step)
  File "/home/user/nerfstudio/nerfstudio/utils/decorators.py", line 58, in wrapper
    ret = func(self, *args, **kwargs)
  File "/home/user/nerfstudio/nerfstudio/engine/trainer.py", line 250, in _update_viewer_state
    self.viewer_state.update_scene(self, step, self.pipeline.model, num_rays_per_batch)
  File "/home/user/nerfstudio/nerfstudio/utils/decorators.py", line 82, in wrapper
    ret = func(*args, **kwargs)
  File "/home/user/nerfstudio/nerfstudio/viewer/server/viewer_utils.py", line 417, in update_scene
    self._render_image_in_viewer(camera_object, graph, is_training)
  File "/home/user/nerfstudio/nerfstudio/utils/decorators.py", line 82, in wrapper
    ret = func(*args, **kwargs)
  File "/home/user/nerfstudio/nerfstudio/utils/profiler.py", line 43, in wrapper
    ret = func(*args, **kwargs)
  File "/home/user/nerfstudio/nerfstudio/viewer/server/viewer_utils.py", line 820, in _render_image_in_viewer
    self.vis["renderingState/log_errors"].write(
  File "/home/user/nerfstudio/nerfstudio/viewer/server/visualizer.py", line 146, in write
    return self.window.send({"type": "write", "path": path, "data": data})
  File "/home/user/nerfstudio/nerfstudio/viewer/server/visualizer.py", line 50, in send
    self.client.send_multipart(
  File "/home/user/.local/lib/python3.10/site-packages/zmq/sugar/socket.py", line 675, in send_multipart
    self.send(msg, zmq.SNDMORE | flags, copy=copy, track=track)
  File "/home/user/.local/lib/python3.10/site-packages/zmq/sugar/socket.py", line 620, in send
    return super().send(data, flags=flags, copy=copy, track=track)
  File "zmq/backend/cython/socket.pyx", line 746, in zmq.backend.cython.socket.Socket.send
  File "zmq/backend/cython/socket.pyx", line 793, in zmq.backend.cython.socket.Socket.send
  File "zmq/backend/cython/socket.pyx", line 255, in zmq.backend.cython.socket._send_copy
  File "zmq/backend/cython/socket.pyx", line 250, in zmq.backend.cython.socket._send_copy
  File "zmq/backend/cython/checkrc.pxd", line 28, in zmq.backend.cython.checkrc._check_rc
zmq.error.ZMQError: Operation cannot be accomplished in current state

so seems like
RuntimeError: CUDA error: unknown error
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

is the real error. I'm using only one 3080. Original instant-ngp implementation does not cause any problem tho.

Yea, it seems like a different issue. You should run with CUDA_LAUNCH_BLOCKING=1 to see where it is actually being raised.

Yea, it seems like a different issue. You should run with CUDA_LAUNCH_BLOCKING=1 to see where it is actually being raised.

Where should I add the flag CUDA_LAUNCH_BLOCKING=1? I tried inserting between ```ns-train instant-ngp --data data`` but it does not work

Yea, it seems like a different issue. You should run with CUDA_LAUNCH_BLOCKING=1 to see where it is actually being raised.

@tancik Sorry I was logging the wrong account: @SizheAn. I found out it might be viewer's problem since the zmq error happens only when I open the viewer. This is the error log:
  File "/home/user/.local/lib/python3.10/site-packages/aioice/turn.py", line 216, in request_with_retry
    response, addr = await self.request(request)
  File "/home/user/.local/lib/python3.10/site-packages/aioice/turn.py", line 203, in request
    return await transaction.run()
  File "/home/user/.local/lib/python3.10/site-packages/aioice/stun.py", line 299, in run
    return await self.__future
aioice.stun.TransactionFailed: STUN transaction failed (403 - Forbidden IP)
Task exception was never retrieved
future: <Task finished name='Task-54' coro=<TurnClientMixin.send_data() done, defined at /home/user/.local/lib/python3.10/site-packages/aioice/turn.py:244> exception=TransactionFailed(Message(message_method=Method.CHANNEL_BIND, message_class=Class.ERROR, transaction_id=b'\xad\xc5\x1e\x12\xeaf<\xfeu0\xe6`'))>
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.10/site-packages/aioice/turn.py", line 263, in send_data
    await self.channel_bind(channel, addr)
  File "/home/user/.local/lib/python3.10/site-packages/aioice/turn.py", line 88, in channel_bind
    await self.request_with_retry(request)
  File "/home/user/.local/lib/python3.10/site-packages/aioice/turn.py", line 216, in request_with_retry
    response, addr = await self.request(request)
  File "/home/user/.local/lib/python3.10/site-packages/aioice/turn.py", line 203, in request
    return await transaction.run()
  File "/home/user/.local/lib/python3.10/site-packages/aioice/stun.py", line 299, in run
    return await self.__future
aioice.stun.TransactionFailed: STUN transaction failed (403 - Forbidden IP)
Task exception was never retrieved
future: <Task finished name='Task-56' coro=<TurnClientMixin.send_data() done, defined at /home/user/.local/lib/python3.10/site-packages/aioice/turn.py:244> exception=TransactionFailed(Message(message_method=Method.CHANNEL_BIND, message_class=Class.ERROR, transaction_id=b'\x10\xd0\xb9\x023\xfbG\x19i\n\xbfW'))>
Traceback (most recent call last):
  File "/home/user/.local/lib/python3.10/site-packages/aioice/turn.py", line 263, in send_d

Also about the viewer I have a question (might relate to the previous bug): seems like if I run the program with docker, it is not able to render anything:
, whereas if I run it without docker everything looks fine. Any clue why it happens?

For the flag, run CUDA_LAUNCH_BLOCKING=1 ns-train ...
For the docker, Im not sure. You can try running with --viewer.skip-openrelay True to see if that helps.
This github issue is turning into a series of issues. The discord may be a better place to work through some of this troubleshooting.


Improve undeployment in parallel deployment scenario

Main reason for this pull request is to improve automatic undeployment of applications when using common version numbers, e.g. application##1.2.3.
Currently, HostConfig#checkUndeploy() uses plain sorting of the context names without any special handling of the version identifier. When having a version 1.0.9 deployed, deploying 1.0.10 will lead to immediate undeployment of 1.0.10 because the ordering is wrong.
This change introduces ContextName#compareTo() which knows how to handle commons version numbers.

I have applied the optimisation commit (thanks) but not the ordering changes.
The patch changes the order for undeploy without changing the order for version mapping. This will lead to unexpected behaviour.
The current simple String based ordering was selected as a trade off between performance and usability. The type of ordering proposed in this patch was considered too expensive to incur on every request when multiple versions are deployed. To consider such a change, we'd need to see evidence of the performance impact (including GC) on the mapping process of switching to this ordering mechanism. Be aware that mapping code is highly optimised for String.
Reviewing the proposed patch I see several things that would need to be fixed in additional to the more architectural issue described above:

imports from sun.* are not allowed
the code is Java 8 specific - any fix needs to be available to back-ported to 7.0.x will has to run on Java 6
the patch does not compile
Pattern instances should be pre-compiled

Finally, the patch only considers purely numeric versions. Many projects include a test string (e.g. RELEASE) in the version number. It would be nice to handle these as well.

@markt-asf @ChristopherSchultz
Thanks for your feedback so far and for the partial merge. To be honest I didn’t think about any performance implications or side effects because I didn’t notice that this could be used in other places, especially during each request.
I’m sorry for the broken second patch and pushed an updated version which fixes the unused imports in HostConfig and broken syntax in ContextName.
I’ll have another look at the regex precompilation and Java 7 compatibility later. After that I want to look at improving performance and backwards-compatibility. Maybe someone can point me at the right code locations where this might be important?

I think there is a lot of merit in Chris's idea. The concept of some sort of internal pre-computed version string that addresses ordering concerns by zero padding numerical components is one that would be easy to implement and have minimal impact on the existing system.

I added another bunch of patches. One of them implementing a different approach to a cached version representation based on @ChristopherSchultz’s suggestion. Instead of creating zero-padded strings I had the idea of using the version components as UTF-16 code points and building a string from those.
I did not yet test all edge cases. So this might not be fail-safe.
I’m still using regular expressions as the overhead of implementing a tokenizer seemed a bit much given the one-time nature of the new approach.
I did not include code for non-numeric version components (especially suffixes, e.g. 1.2.3-SNAPSHOT or 1.2.3.pre5.

I added support for version suffixes and rebased this on trunk. Maybe you want to have another look.

@markt-asf I’m sorry to nag you about this, but I didn’t get any feedback since my improvements on this patch series.

I'm starting to lean toward requiring this new feature to require a configuration option to enable it, and have it default to false. My justification is that it represents a breaking behavioral change that is just slightly different than previous behavior as to be unnoticeable until it starts behaving completely unexpectedly.
Can you please add a configuration option for <Host> to enable or disable this? Simply swap-out the compareTo implementation depending upon the value of that setting.

I agree this should configurable on the Host.
It should be possible to remove the Comparator entirely if the version code is used directly in the Mapper.
I'd like this to be more robust. I'm thinking pad any dot-separated segment to a given length with a suitable character (possible space).


add flake8 in pre-commit

closes #13

linting is now moved to lint.yml, but I don't know why it is not ran by the github actions... I'll try to fix this later

As far as i remember, this is ready to be reviewed / merged

I don't recall exactly but I guess that yes it is because of black.
We can merge #53 first, and I'll create a new branch from master, and add the .yml files. May be the simplest


[AMBARI-25202] recoverHost operation fails if public hostname is diff…

…erent from actual hostname (apappu)
What changes were proposed in this pull request?
have used "content.hostName" instead of "content.hostName" instead of "content.publicHostName" as publichostname calls would fail.
(Please fill in changes proposed in this fix)
How was this patch tested?
Deployed the changes in working ambari setup and tried to recover the host - after changes operation is successful.
(Please explain how this patch was tested. Ex: unit tests, manual tests)
(If this patch involves UI changes, please attach a screen-shot; otherwise, remove this)
Please review Ambari Contributing Guide before opening a pull request.

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/5025/
Test PASSed.

@ishanbha Can you please review the changes and approve ?

Hi @ishanbha ,
Can you please review and merge this pr , its PR in trunk branch  of https://github.com/apache/ambari/pull/2881


Get participatory researchers access to academic resources

Summary
In order to do academic work participatory researchers need to have journal access.

There are lots of places/ways to access papers now:
Unpaywall
Paper panda
open access button
OA.mg
Thanks to @EKaroune for these suggestions!


quick add of event data sets start time incorrectly

[x] I have searched through existing Issues and confirmed that this particular issue has not already been reported.
[x] I have read and performed all the steps listed in the Wiki.

What steps will reproduce the problem?
When adding the following info for quick add
A Taste of Schwab
Thursday, September 27, 2018
6:30 p.m. Registration & Reception
7:00 p.m. Presentation & Dinner
The Huntley Hotel
1111 Second Street
Santa Monica, CA 90403
Complimentary parking provided
it sets everything in the correct spot but sets the start time incorrectly. From the many events i added in the past it appears to believe that the time is in UTC format thus is adjusting it by 7 hours (i live in PDT) and sets that as the start time... .how can we have it just use the time written and not make that adjustment
System Information:

Version:
Language:
Timezone Offset:
User Agent:

https://github.com/manastungare/google-calendar-crx/wiki/Quick-Add-Bugs


Localizing or Translating Slugs?

Hi,
I am using Dimsav\Translatable to localize my application. I would like to have localized slugs, too.
I have installed Sluggable, appended the Model and the database, but get the error "SQLSTATE[42S22]: Column not found: 1054 Unknown column 'slug' in 'field list' (SQL: select slug, id from units where slug LIKE test-unit%) "
Could this be because the Model attempts to save the slug in the "units" table instead of the unit_translations-table?
Disclaimer: I am not very experienced with PHP/Laravel, please excuse me if I am asking the obvious.
Regards
Wolf
The Model:
use \Dimsav\Translatable\Translatable;
use Cviebrock\EloquentSluggable\SluggableInterface;
use Cviebrock\EloquentSluggable\SluggableTrait;

class Unit extends Eloquent implements SluggableInterface {
// All functions related to sluggable
use SluggableTrait;
protected $sluggable = array(
    'build_from' => 'name',
    'save_to'    => 'slug',
);

// All functions related to translation
protected $fillable = ['name', 'description', 'slug'];

public $translatedAttributes = array('name', 'description', 'slug');

}
the Controller:
public function store()
   {
  //Store created unit in database
  $input = Input::all();
  $validation = Validator::make($input, Unit::$rules);
    if ($validation->passes())
        {
        Unit::create($input);
        return Redirect::route('units.index')
        ->with('message', trans('units.success_create'));
        }
    return Redirect::route('units.create')
        ->withInput()
        ->withErrors($validation)
        ->with('message', trans('validation.error'));
   }

the migration for unit_translations:
public function up()
   {
    Schema::create('unit_translations', function(Blueprint $table)
    {
        $table->increments('id');

        // Translatable attributes
        $table->string('name');
        $table->string('description');
        $table->string('slug')->index();

        // Translatable attributes

        $table->integer('unit_id')->unsigned()->index();
        $table->foreign('unit_id')->references('id')->on('units')->onDelete('cascade');
        $table->string('locale')->index();
        $table->unique(['unit_id', 'locale']);
        $table->timestamps();
    });
}

My package works by automatically generating a slug based on the model attributes when you call $model->save().  I'm not sure how (or even if) Dimsav\Translatable automatically creates the translatable versions of model attributes.
I'm not 100% sure if this would work, but looking at his project, I would think you just need to make the translated entity models sluggable as well, e.g.:
// models/Country.php
class Country extends Eloquent implements Cviebrock\EloquentSluggable\SluggableInterface {

    use Cviebrock\EloquentSluggable\SluggableTrait;
    use \Dimsav\Translatable\Translatable;

    public $translatedAttributes = array('name');
    protected $fillable = ['iso', 'name'];
    protected $sluggable = [
        'build_from' => 'name',
        'save_to'    => 'slug',
    ];
}

// models/CountryTranslation.php
class CountryTranslation extends Eloquent implements Cviebrock\EloquentSluggable\SluggableInterface {
    use Cviebrock\EloquentSluggable\SluggableTrait;

    public $timestamps = false;
    protected $fillable = ['name'];
    protected $sluggable = [
        'build_from' => 'name',
        'save_to'    => 'slug',
    ];

}

As long as the main model and the Translation model both implement sluggable, and both have the same sluggable configuration, I think that will do what you are expecting.

I had the same question today and the "name" attribute only exists in the CountryTranslation model so you don't need to implement the sluggable functionality in Country, just in the translation. So it would end like this:
// models/Country.php
class Country extends Eloquent {

    use \Dimsav\Translatable\Translatable;

    public $translatedAttributes = ['name', 'slug'];
    protected $fillable = ['iso', 'name'];
}

// models/CountryTranslation.php
class CountryTranslation extends Eloquent implements Cviebrock\EloquentSluggable\SluggableInterface {
    use Cviebrock\EloquentSluggable\SluggableTrait;

    public $timestamps = false;
    protected $fillable = ['name'];
    protected $sluggable = [
        'build_from' => 'name',
        'save_to'    => 'slug',
    ];

}

That makes even more sense.  Closing this now, but feel free to open again if you are still having issues @broszies

Hello,
I'm having the same 'issue', but the method explained above by @mauro-moreno doesn't seem to do it for me.
Here's my 'base' entity:
<?php namespace Modules\Product\Domain\Entities;

use Dimsav\Translatable\Translatable;
use Illuminate\Database\Eloquent\Model;

class Category extends Model
{
    use Translatable;

    protected $table = 'product__categories';
    public $translatedAttributes = ['name', 'slug', 'description'];
    protected $fillable = ['parent_id', 'name', 'slug', 'description'];
}

And my translatable model:
<?php namespace Modules\Product\Domain\Entities;

use Cviebrock\EloquentSluggable\SluggableInterface;
use Cviebrock\EloquentSluggable\SluggableTrait;
use Illuminate\Database\Eloquent\Model;

class CategoryTranslation extends Model implements SluggableInterface
{
    use SluggableTrait;

    public $timestamps = false;
    protected $fillable = ['name', 'slug', 'description'];
    protected $table = 'product__category_translations';
    protected $sluggable = array(
        'build_from' => 'name',
        'save_to' => 'slug',
    );
}

When saving my Category, the slug column in the product__category_translations doesn't get set.
Did I miss something ?
Thanks,

Are you using a "non standard" relationship Id? Translatable has a "translationForeignKey" attribute that you should use in the Category model if so.

Hm I use the standard foreign key.

I think that the models are OK so I would check the queries, maybe your problem is in your database definition.

Hm possibly, but doubtful since I have the usual migration from the Translatable package.
Here's the db definition (product__ is the prefix)
CREATE TABLE `product__categories` (
  `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `parent_id` int(11) unsigned DEFAULT NULL,
  `order` int(11) unsigned DEFAULT NULL,
  `created_at` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
  `updated_at` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=12 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;

CREATE TABLE `product__category_translations` (
  `id` int(10) unsigned NOT NULL AUTO_INCREMENT,
  `name` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
  `slug` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
  `description` text COLLATE utf8_unicode_ci NOT NULL,
  `category_id` int(10) unsigned NOT NULL,
  `locale` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `product__category_translations_category_id_locale_unique` (`category_id`,`locale`),
  KEY `product__category_translations_locale_index` (`locale`),
  CONSTRAINT `product__category_translations_category_id_foreign` FOREIGN KEY (`category_id`) REFERENCES `product__categories` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=21 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;

try this:
public function sluggable () {

            return [
                'tag_slug' => [
                    'source' => 'tag'
                ]
            ];

        }

public function sluggable () {

            return [
                'name_of_your_custom_column' => [
                    'source' => 'name_column_source'
                ]
            ];

        }


Albania telegram channel is broken

it is a dead link
nothing is listed on https://wiki.openstreetmap.org/wiki/List_of_OSM_centric_Telegram_accounts

@bhousel WikiProject lists https://t.me/joinchat/C1Bx60HYZ4ByK-Kll0ww5g though..

@danieldegroot2 I guess that new issue for that would be a good idea if that channel is active

Index: Added 2018 here
Wiki: Added 2020 here (working link with several online members already gives it away somewhat)
old link looks like a valid account on first sight -it is not a group, true-, but may no longer be active.


Docs failed to build for 0.16

Version
0.16
Operating system & version
N/A
What you did
Went to https://docs.rs/leafwing-input-manager/0.16
What you expected to happen
Documentation
What actually happened
Got docs.rs error page for doc build of crate

Do the docs build locally for you? I might need to poke the docs.rs team if so.

Sorry about the slow reponse, I meant to reply sooner. I'm having a bit of a hard time running the exact command they use on docs.rs, but a plain old cargo doc runs just fine locally at least.

Yeah, that matches what I experienced too. Let me open an issue.


cmd/go: creat() is typo?

What version of Go are you using (go version)?

go version go1.20.3 linux/amd64

Does this issue reproduce with the latest release?
yes
What operating system and processor architecture are you using (go env)?
go env Output
❯ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/orangekame3/.cache/go-build"
GOENV="/home/orangekame3/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/orangekame3/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/orangekame3/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.20.3"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build683663421=/tmp/go-build -gno-record-gcc-switches"

What did you do?
I think ⤵　creat() in cmd/cgo/util.go is typo
https://github.com/golang/go/blob/49d42128fd8594c172162961ead19ac95e247d24/src/cmd/cgo/util.go#L107-L114
It should be ↓
func create(name string) *os.File {
	f, err := os.Create(name)
	if err != nil {
		fatalf("%s", err)
	}
	return f
}

it's referenced only ⤵

What did you expect to see?
none
What did you see instead?
none

Or like that ???
https://en.wikiquote.org/wiki/Ken_Thompson#Quotes:~:text=Ken Thompson was once asked what he would do differently if he were redesigning the UNIX system. His reply%3A "I'd spell creat with an e."

Ken Thompson was once asked what he would do differently if he were redesigning the UNIX system. His reply: "I'd spell creat with an e."


Older version of botbuilder-teams installed

This generator installs botbuilder-teams v0.1.7.
On my system, bot won't return anything until I update to v0.2.1

Thanks @coatsy - I recently noticed this as well and only fixed it for the upcoming release. Will make an intermediate release for this combined with a few other minor fixes.

@coatsy - can you please test with the new preview

npm install generator-teams@preview --global

Resolved in latest preview


Improve performance of ProvideId method

The improvement consists in optimizing ScopeTypeFromTimerInfo function which is twofold:

The nested branching is replaced by a table and arithmetic operations
The function itself is turned into a lambda to facilitate inlining.

The speed-up is measured at the level of 15%.
Test: Unit, manual
Bug: http://b/264439413

I have added a comment. Also, I've fixed a bug, so the table has grown.
Regarding the old implementation which I've commented out. If I was writing this from scratch, I'd probably write some pseudocode to document the function, so that the reader has an idea of what we're trying to achieve here. But hey -- we've already got plain C++ code doing just that.

Converting to draft. I've got an idea how to make it even faster. I'll test the idea on Monday.

Careful analysis reveled no statistically significant performance improvement. Closing.

As pointer out by @beckerhe the performance issue we observe is probably due to cache incoherence.


No module named _winreg

Hi.
My name is Nelson Yalta from Waseda University, and also working with second gradient optimizations.
I was trying to run your library but i got a problem of =
File "test.py", line 493, in
for a in sys.argv[2:]])
File "test.py", line 23, in test_xor
ff = HessianFF([2, 5, 1], debug=True, use_GPU=False)
File "/home/nelson-lab/hessianfree-master/hessianfree/hessianff.py", line 65, in init
layers = [copy.deepcopy(layers) for _ in range(self.n_layers)]
File "/usr/lib/python2.7/copy.py", line 190, in deepcopy
y = _reconstruct(x, rv, 1, memo)
File "/usr/lib/python2.7/copy.py", line 334, in _reconstruct
state = deepcopy(state, memo)
File "/usr/lib/python2.7/copy.py", line 163, in deepcopy
y = copier(x, memo)
File "/usr/lib/python2.7/copy.py", line 257, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/usr/lib/python2.7/copy.py", line 178, in deepcopy
rv = reductor(x)
File "/usr/lib/python2.7/dist-packages/numpy/core/init.py", line 61, in _ufunc_reduce
return _ufunc_reconstruct, (whichmodule(func, name), name)
File "/usr/lib/python2.7/pickle.py", line 817, in whichmodule
if name != 'main' and getattr(module, funcname, None) is func:
File "/usr/lib/python2.7/dist-packages/scipy/lib/six.py", line 116, in getattr
_module = self._resolve()
File "/usr/lib/python2.7/dist-packages/scipy/lib/six.py", line 105, in _resolve
return _import_module(self.mod)
File "/usr/lib/python2.7/dist-packages/scipy/lib/six.py", line 76, in _import_module
import(name)
ImportError: No module named _winreg,
so I was wondering if you made this for windows or did you use in linux. and also if i am lacking of a library.
Best Regards

Sorry, does not matter. Was a problem with scipy. needed to update to the latest version. Take care (btw. I try to use you code to implement something quite similar to caffe.)

Glad you got it working, let me know if you have any questions about working with the code.


Index signature for type 'string' is missing in type when extending Record<string, unknown>

🔎 Search Terms
"Index signature for type string is missing in type", "extending record"
🕗 Version & Regression Information

This is the behavior in every version I tried (4.9.5, 5.0.4, 5.4.5, 5.5.3, v5.6.0-dev.20240709), and I reviewed the FAQ for entries "Common bugs that aren't bugs"

⏯ Playground Link
https://www.typescriptlang.org/play/?ts=5.5.3#code/C4TwDgpgBACgTgewGYEsA2EDCCB2qDmUAvFAEoQDGCcAJgDwDOwcKO+ANFAK44DWOCAO44AfAChQkKNjwp8AWQCGYYmUrV6TFm07xk6LLgLiJ4aDIKqLcpSoBkUAN5iorqMAhMAYjgBcUAAoASmIRbj4BYTEAXzExKhwmKDQERRprQhJg-wzQpxc3BKSEpFVnNwqoRQAjCn9HRX8ARmj2AsqoGggkesaoACZW9qhYiqLgd09gH1Vs8P4hHDyAIhqKZeG4CGAuOCXyjqgAOhOStsPJ7xxzt1jooA
💻 Code
type ProfileConfig = Record<string, unknown>
type ConfigMap = Record<string, ProfileConfig>

type Config = ConfigMap & {
    testFn: () => unknown
}

const loadConfig = (): Config => {
    const conf = {
        abc: {a: 1},
        def: {a: 2},
    }
    const testFn = (): unknown => "abc"
    return {
        ...conf,
        testFn,
    }
}

🙁 Actual behavior
I'm getting an error on the line 14 (return of the loadConfig function):
Type '{ testFn: () => unknown; abc: { a: number; }; def: { a: number; }; }' is not assignable to type 'Config'.
  Type '{ testFn: () => unknown; abc: { a: number; }; def: { a: number; }; }' is not assignable to type 'ConfigMap'.
    Property 'testFn' is incompatible with index signature.
      Type '() => unknown' is not assignable to type 'ProfileConfig'.
        Index signature for type 'string' is missing in type '() => unknown'.(2322)

🙂 Expected behavior
I was expecting the & { testFn: ... } part to override the Record<string, unknown> from the ConfigMap type so I can add custom functions to the object.
I was also testing type Config = Omit<ConfigMap, 'testFn'> & { testFn: ... }, but with no success.
Additional information about the issue
No response

I was expecting the & { testFn: ... } part to override the Record<string, unknown> from the ConfigMap type so I can add custom functions to the object.

This is how it works. You can expect this to be valid and type-checked:
loadConfig().testFn()

The problem is that your return type annotation also expect this to be allowed:
loadConfig().whatever

But the object that you return doesn't have an index signature so it doesn't allow for arbitrary property access. This is what causes the error you are facing.
Depending on your needs this might work for you:
type ProfileConfig = Record<string, unknown>;
type ConfigMap = Record<string, ProfileConfig>;

type Config = ConfigMap & {
  testFn: () => unknown;
};

const loadConfig = () => {
  const conf = {
    abc: { a: 1 },
    def: { a: 2 },
  } satisfies ConfigMap;
  const testFn = (): unknown => "abc";
  return {
    ...conf,
    testFn,
  };
};

If you need to preserve the index signature though... they are lost when spreading, see: https://github.com/microsoft/TypeScript/issues/27273
You might do things like this though:
type ProfileConfig = Record<string, unknown>;
type ConfigMap = Record<string, ProfileConfig>;

type Config = ConfigMap & {
  testFn: () => unknown;
};

function toConfigMap<T extends {}>(obj: T): T & ConfigMap {
  return obj;
}

const loadConfig = () => {
  const conf = {
    abc: { a: 1 },
    def: { a: 2 },
  } satisfies ConfigMap;
  const testFn = (): unknown => "abc";
  return toConfigMap({
    ...conf,
    testFn,
  });
};

I see. Yes, the toConfigMap function works.
I will find out whether it is super important for us that the configMap is spread there, because I think the easier solution would be to do this:
type Config = {
  configMap: ConfigMap,
  testFn: () => unknown,
}

I'd rather not use any "workarounds" for such a small obstacle, to keep the code a bit cleaner.
So I'm assuming this isn't really a bug, since the index signature needs to stay as it is. We can probably close this issue, as it is already reported in the one you mentioned.
And thank you for a quick reply! It was very helpful!

You can expect this to be valid and type-checked:

But be aware of the unsoundness with such types. It's possible to assign whatever value to testFn using the index signature and a string typed key.


Task/dr 472 service search cicd

Description
Create CI/CD pipeline for the Service Search domain, dummy S3 bucket has been created.
Note
Left application and unit tests code commented out for future use when we bring in new application code in the following file. workflows/pipeline-deploy-app-develop.yaml
Context
DR-472 - Required to automatically spin up infrastructure by a CI/CD pipeline.
Type of changes

[ ] Refactoring (non-breaking change)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would change existing functionality)
[ ] Bug fix (non-breaking change which fixes an issue)

Checklist

[x] I am familiar with the contributing guidelines
[x] I have followed the code style of the project
[x] I have added tests to cover my changes
[ ] I have updated the documentation accordingly
[x] This PR is a result of pair or mob programming

Sensitive Information Declaration
To ensure the utmost confidentiality and protect your and others privacy, we kindly ask you to NOT including PII (Personal Identifiable Information) / PID (Personal Identifiable Data) or any other sensitive data in this PR (Pull Request) and the codebase changes. We will remove any PR that do contain any sensitive information. We really appreciate your cooperation in this matter.

[x] I confirm that neither PII/PID nor sensitive data are included in this PR and the codebase changes.

branch closed due to original commit not gpg signed and therefore unable to merge into main. This branch was replaced by a new branch task/DR-472_Service_search_pl  which was successfully merged into main.


chore: auto-approve.yml

[x ] Please merge this PR for me once it is approved.
@averikitsch FYI
@sofisl FYI
@kurtisvg FYI
@eaball35 FYI

Actually - hold off on merging until Sofi reviews.

PTAL

@sofisl The hack of adding just you doesn't' actually work.  I'll see how ridged it is.

@kurtisvg PTAL


Enables communication between the webview and RN

Android devs have been relying for ages on intercepting custom HTTP
calls made by the webview, to map them to native actions,
like here.
Say that we have a "close" button in our web interface. We first detect
if we're running on web or within a webview -- if on web, we simply call window.close().
If we're running within the app things get a bit more complicated, but we can
simply call app://exit and the app will be able to intercept this call and
close the webview for us.
Now, this PR adds the ability of doing it on RN. You basically specify a
protocol for messages that should be forwarded to RN (ie. rn) and then,
when your webview loads a resource from rn://xyz the onMessage callback
will be called. You can access the URL etc etc through event.nativeEvent.url.
Example:
let onMessage = (event) => {
  let {url} = event.nativeEvent;

  if (url.endsWith('exit')) {
    BackAndroid.exitApp(); // just sayin'...
  }
}

return <Webview
  source={...}
  messageProtocol={'myApp'}
  onMessage={onMessage}
>

Or something of that sort. Communicating between the webview and RN / the ability
to intercept resources loaded within the webview has been already discussed but I
feel we need a very simple implementation rather than creating a full-blown
messaging pattern which will just hurt (maintainability, complexity) in the long run.
Also, my Java is pretty rusty ;-)

My biggest doubt here is that this guy is android-specific. I think we already have something working on IOS so I would prefer to see both working in the same way. Anyhow, let me know what you guys think.

Saw @astreet comments on https://github.com/facebook/react-native/pull/6478#issuecomment-208988259 and I agree that regex patterns might be a good, viable solution. Though, what if we need to run cb when a pattern is matched?

In the example, you have: messageProtocol={'myApp'}. Does the WebView load a myApp:// URL? Can you provide a full example please?

Saw @astreet comments on #6478 (comment) and I agree that regex patterns might be a good, viable solution. Though, what if we need to run cb when a pattern is matched?

Can you elaborate please? What do you mean by "what if we need to run cb when a pattern is matched?"?

This is what I meant -- we could pass some object like this:
let routesToIntercept = {
  'myApp://some-action': () => console.log(123),
  'myApp://*': () => console.log(456)
}

return <Webview
  routesToIntercept={routes}
/>

The problem I see with my PR is that IOS / android implementations will vary quite a bit, which sucks. So Im open to consider changing the implementation in favor of something that can work longer term. Let me know your thoughts.

@odino https://github.com/alinz/react-native-webview-bridge

@stereodenis that wont work for legacy implementations that are based on calling custom URLs :) even though I think it's a good solution I was aiming at taking care of the practice of using custom URLs since there's a lot of android + webview implementations that use that method.

@mkonicek would you mind taking a look at this pull request? It's been a while since the last commit was reviewed.

@odino will it be replacement for react-native-webview-bridge?

@stereodenis well, I would like to see a default way to implement some sort of communication between webviews and RN :)

https://github.com/alinz/react-native-webview-bridge/issues/109

Sorry for the delay on reviewing this - there's now a related proposal using postMessage: https://github.com/facebook/react-native/pull/9762

Since the postMessage change is accepted with https://github.com/facebook/react-native/pull/9762 I think this pull request has been superceded. I'm going to close this PR but please feel free to reopen if this is hitting a different use case or has an independent rationale.


Add unit tests

Description

Add testify as a dependency using glide
Refactor the ipify package to make it more testable
Add some unit tests to illustrate the concepts

Motivation and Context

Demonstrate the concepts hos unit testinf an application in Go.
Types of changes

New feature (non-breaking change which adds functionality)
Code cleanup / Refactoring
Documentation

Fixes: https://github.com/rgreinho/go-cli-crashcourse/issues/7

@laconc It is still a work in progress, but I would love your input as this is fairly different than unit testing in Python.


[QNN][TFLite] TFLite rounding mode support

Thanks for contributing to TVM!   Please refer to guideline https://docs.tvm.ai/contribute/ for useful information and tips. After the pull request is submitted, please request code reviews from Reviewers by @ them in the pull request thread.
Add tflite rounding support with corresponding test cases. The tflite rounding mode golden results are generated with a testbench using MultiplyByQuantizedMultiplier function here: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/kernels/internal/common.h#L148
@FrozenGene @anijain2305
Might help fix the problem described here:
https://discuss.tvm.ai/t/supporting-bit-exact-tflite-qnn-inference/5528

Thanks @LiangHao151941 ! I haven't reviewed the code, however, I have some high level comments:

if we have TFLITE rounding support, we should make TFLite frontend using TFLITE rounding and we should get the bit exact result as TFLite.

you should also modify the test_forward.py for tflite, like test_qnn* related test cases, we shouldn't need atol=1 any more ideally.

you could add q_conv2d unit testing case, we could get the same result compared with TFLite. we lack of this unit testing.

Makes sense! I'll follow up on those. @FrozenGene

Thanks for this PR. It will help reduce/remove those 1-off deviations.
One request that I have is - While setting the rounding mode to TFLite in TFLite parser, it might be better to set it by adding an extra arg in from_tflite API. The main reason is that the lowering here might involve many more number of Relay ops, which might degrade performance. So, to enable TVM user to choose different accuracy-performance tradeoff points, we can add a new argument in the TF parser API, and then pass it along to the QNN ops. The default value of this arg can be tflite rounding. @FrozenGene Are you ok with this?
@u99127 @mbarrett97 will also be interested in this.
Also, I think we can have small PRs merged. And incrementally improve test coverage.

One request that I have is - While setting the rounding mode to TFLite in TFLite parser, it might be better to set it by adding an extra arg in from_tflite API. The main reason is that the lowering here might involve many more number of Relay ops, which might degrade performance. So, to enable TVM user to choose different accuracy-performance tradeoff points, we can add a new argument in the TFLite parser API, and then pass it along to the QNN ops. The default value of this arg can be tflite rounding. @FrozenGene Are you ok with this?

Yes, this is my thought too. I am ok with it.

LGTM. I like the scope of this PR. IMO, we can get this in. And send other PRs for TFLite parser.

IMO, I would prefer adding TFLite parser in this PR too. Because this will be organized one complete solution for TFLite rounding. When others review the history, they don't need to check isolated PRs together to get complete support for TFLite rounding support. Maybe these PRs during time is long. It is right that we could make small prs and merge it in quickly so that we could get functionality supporting quickly and could locate the reason quickly. However, from this pr's perspective, TFLite's parser code will not be complex and could be contained in. It is acceptable in one single PR from my opinion. So I would suggest @LiangHao151941 adding TFLite parser in this PR too. If we think it is import, we even could port it back to release 0.6, one PR will make us do it easily too.

Some update on further experiments @FrozenGene @anijain2305

if we have TFLITE rounding support, we should make TFLite frontend using TFLITE rounding and we should get the bit exact result as TFLite.

This is added in tflite relay frontend.

you should also modify the test_forward.py for tflite, like test_qnn* related test cases, we shouldn't need atol=1 any more ideally.

For the current label comparison, the new rounding mode works fine. Unfortunately, the new rounding mode does not bring bit-exact execution. In fact, for the 3 qnn models intest_forward.py,  with L1 norm of prediction error as the metric, mobilenet_v1andmobilenet_v2 do drop with "TFLITE" rounding, but inception_v1 increases the metric (means more error).
Also note that performance will degrade, but not quantitatively measured for now.

you could add q_conv2d unit testing case, we could get the same result compared with TFLite. we lack of this unit testing.

This would be essential to further investigate what are the root causes of bit mismatch. Will follow up.

I thought little more about the bit exact problem. One source of discrepancy for certain is QNN add, and QNN concatenate ops. These call Requantize internally, and they will have default rounding in C++ (UPWARD). For testing, @LiangHao151941 , you can set the C++ default to TFLIte to see if it helps. Meanwhile, we can also think how to can make that C++ rounding visible at python level.

If we think it is import, we even could port this back to release 0.6, one PR will make us do it easily too.

That is ok. Lets try to see if we can get it in single PR.

I thought little more about the bit exact problem. One source of discrepancy for certain is QNN add, and QNN concatenate ops. These call Requantize internally, and they will have default rounding in C++ (UPWARD). For testing, @LiangHao151941 , you can set the C++ default to TFLIte to see if it helps. Meanwhile, we can also think how to can make that C++ rounding visible at python level.

Maybe we could use @register_func to register one function named as @register_func(qnn.requantize.rounding) in TFLite parser, then we could get the value in RequantizeQnnCanonicalize from C++. However, this will have one problem that if users specify the rounding value, it won't work, because we don't know whether the rounding value is setting by default or by user explicitly. So, if we support it this, we will constraint we will only one rounding setting by TFLite parser.
Another way is we provide rounding args for qnn.add / qnn.mul / qnn.conconcate, because they use requantize in fact too, so they need rounding.

I like this way

Another way is we provide rounding args for qnn.add / qnn.mul / qnn.conconcate, because they use requantize in fact too, so they need rounding.

I like this way

Another way is we provide rounding args for qnn.add / qnn.mul / qnn.conconcate, because they use requantize in fact too, so they need rounding.

cc @LiangHao151941 Could you try this way to do bit-extract testing? For conveniently, you could simply change the C++ default rounding value to "TFLITE" to test.

cc @LiangHao151941 Could you try this way to do bit-extract testing? For conveniently, you could simply change the C++ default rounding value to "TFLITE" to test.

just modified add/mul/concat requantize rounding mode and tested, no luck. will change the default rounding behavior for a later test.

update: I forced FixedPointMultiply(PerChannel) rounding mode to be TFLITE, but still unable to get bit-exact results.

:( This will be a challenging task. I think, instead of trying to fix the full e2e test, we should now focus on unit tests. I can help with improving unit test coverage.

one more thing, setting tflite default rounding mode to TFLITE seems to break GPU test of mobilenet_v2, maybe you guys have any ideas/suggestions?

Do we run TFLIte tests on GPU? Little confused.

just modified add/mul/concat requantize rounding mode and tested, no luck. will change the default rounding behavior for a later test.
update: I forced FixedPointMultiply(PerChannel) rounding mode to be TFLITE, but still unable to get bit-exact results.
one more thing, setting tflite default rounding mode to TFLITE seems to break GPU test of mobilenet_v2, maybe you guys have any ideas/suggestions?
@FrozenGene @anijain2305

Let us break the model into layer by layer and compare with tflite. I want to describe my development way, maybe it could help you. For example, we have mobilenetv2 quantized model, you could get the quantized tensorflow and tflite model. Then you could call tflite_convert (feed it quantized tensorflow model) and set the output layer (for example, just the first convolution layer), then you get one quantized tflite model only contain the first convolution layer of mobilenet v2. After you verify it correctly, you could go on until you finish the whole model e2e correctly.
I think when you verify,  you could run on cpu firstly locally to find issue, then consider gpu ci issue.

:( This will be a challenging task. I think, instead of trying to fix the full e2e test, we should now focus on unit tests. I can help with improving unit test coverage.

That would be appreciated, thanks!

Do we run TFLIte tests on GPU? Little confused.

Yes, according to jenkins report.

Let us break the model into layer by layer and compare with tflite. I want to describe my development way, maybe it could help you. For example, we have mobilenetv2 quantized model, you could get the quantized tensorflow and tflite model. Then you could call tflite_convert (feed it quantized tensorflow model) and set the output layer (for example, just the first convolution layer), then you get one quantized tflite model only contain the first convolution layer of mobilenet v2. After you verify it correctly, you could go on until you finish the whole model e2e correctly. Command example:
tflite_convert --input_format=TENSORFLOW_GRAPHDEF --graph_def_file="xx.pb" --output_file=xx.tflite --output_format=TFLITE --input_arrays=input --input_shapes=1,224,224,3 --std_dev_values=127 --mean_values=127 --inference_type=QUANTIZED_UINT8 --inference_input_type=QUANTIZED_UINT8 --default_ranges_min=0 --default_ranges_max=6 --output_arrays=xx

Thanks for the detailed instruction here. Will follow up.

I think when you verify, you could run on cpu firstly locally to find issue, then consider gpu ci issue.

No problem.

ping @LiangHao151941

ping @LiangHao151941 please followup

There is quite some tedious work to do, but thanks for the reminder, will follow up shortly @FrozenGene @tqchen

The problem is located and solved. The discrepancy comes from average pooling calculation when input is integral type. tflite does a UPWARD rounding after the division in avg_pool but current topi implementation of avg_pool does of FLOOR after the division.
The fix is added and tested locally on cpu. Now the qnn tests in tflite test_forward has rtol and atol set to 0, which should meet the bit-exact exectution requirement.
@FrozenGene @anijain2305 @tqchen @u99127 Please have another look, thanks!

The problem is located and solved. The discrepancy comes from average pooling calculation when input is integral type. tflite does a UPWARD rounding after the division in avg_pool but current topi implementation of avg_pool does of FLOOR after the division.
The fix is added and tested locally on cpu. Now the qnn tests in tflite test_forward has rtol and atol set to 0, which should meet the bit-exact exectution requirement.
@FrozenGene @anijain2305 @tqchen @u99127 Please have another look, thanks!

Thanks @LiangHao151941 !
Maybe we have to handle it a bit more. Please see: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/kernels/internal/optimized/integer_ops/pooling.h#L300
The default behaviour of TFLite is optimize, so TFLite will run optimzed of kernel directory. For average pool, it has two conditions. The type of acc will be uint16 if params.filter_height * params.filter_width <= 16 * 16 , otherwise, the type of acc will be int32 as our implementation. If we want to keep bit-exact result, we should handle it here too. You remind me this situation, I met this bug before if we don't handle it.

Another way is we provide rounding args for qnn.add / qnn.mul / qnn.conconcate, because they use requantize in fact too, so they need rounding.

We should handle these ops too.

Maybe we have to handle it a bit more. Please see: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/kernels/internal/optimized/integer_ops/pooling.h#L300
The default behaviour of TFLite is optimize, so TFLite will run optimzed of kernel directory. For average pool, it has two conditions. The type of acc will be uint16 if params.filter_height * params.filter_width <= 16 * 16 , otherwise, the type of acc will be int32 as our implementation. If we want to keep bit-exact result, we should handle it here too. You remind me this situation, I met this bug before if we don't handle it.

I think acc being uint16 is fine, as in that case largest acc value will be 256*255 < 65535, where there should not be any overflow problems (and of course int32 is fine as well). So that might not be a problem.

Another way is we provide rounding args for qnn.add / qnn.mul / qnn.conconcate, because they use requantize in fact too, so they need rounding.

Maybe we can mark these as todos due to lack of testcases for now, as make it for the next pr?
A new problem is that changing the default pooling behavior fails some pool op tests. I'm going to add an tflite_mode bool flag for relay.nn.avg_pool2d, how do you think? @FrozenGene

Maybe we can mark these as todos due to lack of testcases for now, as make it for the next pr?

We could mark it #TODO in tflite.py to remind us. Personally I wish we could handle it in a short time so that we don't forget it :-)

A new problem is that changing the default pooling behavior fails some pool op tests. I'm going to add an tflite_mode bool flag for relay.nn.avg_pool2d, how do you think? @FrozenGene

Yes. I think we have to do it. But we should keep the default value as our previous implementation.

Good finding @LiangHao151941
I don't think we should add tflite_mode attr to Pool2D op. It seems adhoc to me.
I think we can directly implement the UPWARD rounding in pooling, whenever the datatype is interger. For integer division, it is valid to implement rounding by default.
If the tests fail for integer pool, we can change those tests as well. I added those tests in https://github.com/apache/incubator-tvm/pull/3607. Prior to that integer computation in pooling was wrong.

I don't think we should add tflite_mode attr to Pool2D op. It seems adhoc to me.
I think we can directly implement the UPWARD rounding in pooling, whenever the datatype is interger. For integer division, it is valid to implement rounding by default.
If the tests fail for integer pool, we can change those tests as well. I added those tests in #3607. Prior to that integer computation in pooling was wrong.

I like the idea, default pooling implementation is updated. Please have a look @anijain2305 @FrozenGene
I think one more thing is left that one of the qnn full net will trigger gpu test related segment fault, not quite familiar with gpu part of code, you guys have any ideas on this? I think we have the problem since tflite rounding mode is introduced.

@LiangHao151941 Thanks for your perseverance. This is very tedious. Kudos!
Do you think it will be good if you send a separate PR for pooling op? I think it is more like a bug of pooling op integer implementation. It will also make this PR just take care of TFLite rounding.
Regarding GPU, let me take a look today. I am little confused why that is happening.

@LiangHao151941 I looked at this
https://ci.tvm.ai/blue/organizations/jenkins/tvm/detail/PR-4828/16/pipeline/247
I think you are confusing the GPU container with GPU target. The tests are still running on CPU.

You might have done this already. But have you run mobilenetv2 locally?
If it passes locally, then lets check the LLVM version. And re-run with whatever container is running with.
You can also try change the target from llvm to `llvm -mcpu=core-avx2' in TFLite tests. There is a LLVM bug which might be causing this - https://discuss.tvm.ai/t/segfault-in-llvm/3567

Do you think it will be good if you send a separate PR for pooling op? I think it is more like a bug of pooling op integer implementation. It will also make this PR just take care of TFLite rounding.

I'm fine with both choices. The pooling implementation doesn't affect any other test except for this bit-exact computation. It can be regarded as bug or feature to me : ) (as np.mean choose FLOOR implementation for integer inputs). @FrozenGene @tqchen

I think you are confusing the GPU container with GPU target. The tests are still running on CPU.

You might have done this already. But have you run mobilenetv2 locally?

Yes, and it passed smoothly.

If it passes locally, then lets check the LLVM version. And re-run with whatever container is running with.
You can also try change the target from llvm to `llvm -mcpu=core-avx2' in TFLite tests. There is a LLVM bug which might be causing this - https://discuss.tvm.ai/t/segfault-in-llvm/3567

I have LLVM 9.0.1, compiled from source code. In my local env, _test_forward_elemwise(partial(_test_add, fused_activation_function="RELU6")) is failing while not in ci test. I'm not sure about the llvm version used in the ci.

I'm fine with both choices. The pooling implementation doesn't affect any other test except for this bit-exact computation. It can be regarded as bug or feature to me : ) (as np.mean choose FLOOR implementation for integer inputs). @FrozenGene @tqchen

I prefer adding it in this pr.  I wish we could port it back to 0.6 as @u99127 said before

I think you are confusing the GPU container with GPU target. The tests are still running on CPU.

You might have done this already. But have you run mobilenetv2 locally?

Yes, and it passed smoothly

I doubt it is because of  different LLVM version.
Another thing: "Another way is we provide rounding args for qnn.add / qnn.mul / qnn.conconcate, because they use requantize in fact too, so they need rounding". On mobilenet v2 quantized model, we have quantized add, I am curious why we could get the same result without support of quantized add.

I have LLVM 9.0.1, compiled from source code. In my local env, _test_forward_elemwise(partial(_test_add, fused_activation_function="RELU6")) is failing while not in ci test. I'm not sure about the llvm version used in the ci.

In code tvm.testing.assert_allclose(tflite_output[i], tvm_output[i], atol=1e-5, rtol=1e-5) maybe too strict. You could change it to atol=1e-3, rtol=1e-3 to see it is ok on your machine.

I doubt it is because of different LLVM version.

Do you know the default llvm version in ci test? I can have a try locally.

In code tvm.testing.assert_allclose(tflite_output[i], tvm_output[i], atol=1e-5, rtol=1e-5) maybe too strict. You could change it to atol=1e-3, rtol=1e-3 to see it is ok on your machine.

I don't have a local numerical problem, instead I got segfault for this test locally, so let's make llvm version matched first?

Another thing: "Another way is we provide rounding args for qnn.add / qnn.mul / qnn.conconcate, because they use requantize in fact too, so they need rounding". On mobilenet v2 quantized model, we have quantized add, I am curious why we could get the same result without support of quantized add.

Hmm, that's interesting... I'm sure requantize is invoked at runtime.

I doubt it is because of different LLVM version.

Do you know the default llvm version in ci test? I can have a try locally.

In code tvm.testing.assert_allclose(tflite_output[i], tvm_output[i], atol=1e-5, rtol=1e-5) maybe too strict. You could change it to atol=1e-3, rtol=1e-3 to see it is ok on your machine.

I don't have a local numerical problem, instead I got segfault for this test locally, so let's make llvm version matched first?

Another thing: "Another way is we provide rounding args for qnn.add / qnn.mul / qnn.conconcate, because they use requantize in fact too, so they need rounding". On mobilenet v2 quantized model, we have quantized add, I am curious why we could get the same result without support of quantized add.

Hmm, that's interesting... I'm sure requantize is invoked at runtime.

CI's LLVM is LLVM 4.0. I think we should supply quantized add so that we avoid it is just a lucky.

CI's LLVM is LLVM 4.0. I think we should supply quantized add so that we avoid it is just a lucky.

Just tested, it's a lucky case indeed.
BTW, is it possible to follow tensorflow to have a specific llvm version for each tvm release to avoid
certain problems like this?

CI's LLVM is LLVM 4.0. I think we should supply quantized add so that we avoid it is just a lucky.

Just tested, it's a lucky case indeed.
BTW, is it possible to follow tensorflow to have a specific llvm version for each tvm release to avoid
certain problems like this?

To avoid lucky case, when we support TFLite rounding, we could change the data = get_real_image(224, 224) to
np.random.seed(0)
data = np.random.random_integers(low=0, high=128, size=(1, 224, 224, 3)).astype('uint8')

back.
For LLVM version, I think we can not at least currently. We support different LLVM versions on TVM. We have the check in FindLLVM.cmake to make sure the minimal requirement is 4.0.
Could you dig it more? If we can make sure it is the bug of LLVM, we then can discuss how to deal with this case.

CI's LLVM is LLVM 4.0. I think we should supply quantized add so that we avoid it is just a lucky.

Just tested, it's a lucky case indeed.
BTW, is it possible to follow tensorflow to have a specific llvm version for each tvm release to avoid
certain problems like this?

To avoid lucky case, when we support TFLite rounding, we could change the data = get_real_image(224, 224) to
np.random.seed(0)
data = np.random.random_integers(low=0, high=128, size=(1, 224, 224, 3)).astype('uint8')

back.

Update：I made a mistake for the unit test that I still compare the labels instead of numerical predictions, so rtol and atol set to zero does not make any sense. Now that I changed to compare numerical prediction vector, the results begin to differ for mobilenetV2. And further, changing qnn.add rounding mode to TFLITE does not lead to bit exact execution. I have to reimplement the whole algorithm again in TFLITE add op to see it works. But I'm bothered by the segfault problem to get any reasonable test results.

For LLVM version, I think we can not at least currently. We support different LLVM versions on TVM. We have the check in FindLLVM.cmake to make sure the minimal requirement is 4.0.
Could you dig it more? If we can make sure it is the bug of LLVM, we then can discuss how to deal with this case.

I digged a little bit on this, but it seems to be not related with llvm but rather with tflite conversion from tensorflow graph to tflite (not even related to tvm). I'm using tf 1.15 for my test. And I find it's related to this test _test_forward_elemwise(partial(_test_add, fused_activation_function="RELU6")), specifically RELU6 will cause the error, if I change it to RELU, no segfault will be triggered. @FrozenGene Please help , or maybe I should just for now switch back to an older version of tensorflow.

CI's LLVM is LLVM 4.0. I think we should supply quantized add so that we avoid it is just a lucky.

Just tested, it's a lucky case indeed.
BTW, is it possible to follow tensorflow to have a specific llvm version for each tvm release to avoid
certain problems like this?

To avoid lucky case, when we support TFLite rounding, we could change the data = get_real_image(224, 224) to
np.random.seed(0)
data = np.random.random_integers(low=0, high=128, size=(1, 224, 224, 3)).astype('uint8')

back.

Update：I made a mistake for the unit test that I still compare the labels instead of numerical predictions, so rtol and atol set to zero does not make any sense. Now that I changed to compare numerical prediction vector, the results begin to differ for mobilenetV2. And further, changing qnn.add rounding mode to TFLITE does not lead to bit exact execution. I have to reimplement the whole algorithm again in TFLITE add op to see it works. But I'm bothered by the segfault problem to get any reasonable test results.

For LLVM version, I think we can not at least currently. We support different LLVM versions on TVM. We have the check in FindLLVM.cmake to make sure the minimal requirement is 4.0.
Could you dig it more? If we can make sure it is the bug of LLVM, we then can discuss how to deal with this case.

I digged a little bit on this, but it seems to be not related with llvm but rather with tflite conversion from tensorflow graph to tflite (not even related to tvm). I'm using tf 1.15 for my test. And I find it's related to this test _test_forward_elemwise(partial(_test_add, fused_activation_function="RELU6")), specifically RELU6 will cause the error, if I change it to RELU, no segfault will be triggered. @FrozenGene Please help , or maybe I should just for now switch back to an older version of tensorflow.

don’t worry. Let us break down the problem.
Firstly, CI’s tflite version is 1.13. We could keep your tf version as the same firstly.
Second, as I suggested before, you could use tflite_convert tool to convert quantized model layer by layer
tflite_convert --input_format=TENSORFLOW_GRAPHDEF --graph_def_file="xx.pb" --output_file=xx.tflite --output_format=TFLITE --input_arrays=input --input_shapes=1,224,224,3 --std_dev_values=127 --mean_values=127 --inference_type=QUANTIZED_UINT8 --inference_input_type=QUANTIZED_UINT8 --default_ranges_min=0 --default_ranges_max=6 --output_arrays=xx
replace the output arrays args to your target layer name. Like the first layer name / second layer name until you find the issue layer.

Serveral updates:

tflite conversion segfault is resolved by using the tflite whl provided by @FrozenGene in the tvm tutorial and change tf version to 1.13.
qnn_mobilenet_v2 segfault is resolved by setting opt_level explicitly to 2, specifically disabling the FoldScaleAxis pass (this one is causing the problem)
bit-exact execution remains

But it is strange that when mobilnet_v2 qnn test is passed locally, it still fails ci test. Below is a local screenshot.

I'm listing my local env here to see if anything need to be aligned:
llvm 9.0.1 (this I can downgrade, but maybe there is a specific version used in the ci so I can try?)
gcc 7.4.0
Ubuntu 18.04
cmake 3.15.2
Please have another look, thanks! @FrozenGene @anijain2305 @tqchen

Serveral updates:

tflite conversion segfault is resolved by using the tflite whl provided by @FrozenGene in the tvm tutorial and change tf version to 1.13.
qnn_mobilenet_v2 segfault is resolved by setting opt_level explicitly to 2, specifically disabling the FoldScaleAxis pass (this one is causing the problem)
bit-exact execution remains

But it is strange that when mobilnet_v2 qnn test is passed locally, it still fails ci test. Below is a local screenshot.

I'm listing my local env here to see if anything need to be aligned:
llvm 9.0.1 (this I can downgrade, but maybe there is a specific version used in the ci so I can try?)
gcc 7.4.0
Ubuntu 18.04
cmake 3.15.2
Please have another look, thanks! @FrozenGene @anijain2305 @tqchen

Thanks for the hard work.
For FoldScaleAxis, It will be better we could dig what issue it is (I describe some background of this algorithm here: https://github.com/apache/incubator-tvm/pull/1664/files#diff-88bcab09a087491487d7655fa76ba116R527).
I checked CI's docker instance just now. Our CI's LLVM version is LLVM 9 too. So your LLVM is OK IMO. However, I think you could run make cython3 on your  tvm root  directory (not build directory) if you don't do this. Sometimes, cython will expose problem (CI has it).

Update:

resolved one problem of MSVC buliding failure in CMake
found llvm version in ci to be 8.x, so I downgraded my llvm version to 8.0.1, still having a segfault for qnn mobilenet_v2 in ci test.
rebase to recent master resolves the opt_level problem, so I change test opt_level back to 3, and it can pass my local test.

Trying to pull the ci docker to further check @FrozenGene

@LiangHao151941 Was wondering if you ever got a chance to look at the performance of TFLite rounding? I am looking at the performance of UPWARD rounding and surprisingly Requantize was quite expensive in e2e models like mobilenet. I suspect TFLite rounding will be even slower.
So, wondering if you got a chance to check performance. I worry that so much efforts in this PR might give us perfect functionality but not so good performance. And therefore, in future, we might decide to tradeoff performance with accuracy, taking us back to square 1.

@LiangHao151941 Was wondering if you ever got a chance to look at the performance of TFLite rounding? I am looking at the performance of UPWARD rounding and surprisingly Requantize was quite expensive in e2e models like mobilenet. I suspect TFLite rounding will be even slower.
So, wondering if you got a chance to check performance. I worry that so much efforts in this PR might give us perfect functionality but not so good performance. And therefore, in future, we might decide to tradeoff performance with accuracy, taking us back to square 1.

IMO, I think we should keep functionality first, then we should consider performance. Currently, our way is to lower many ops, I suspect this leads worse performance, because we can not control all computation in registers. I think TFLite rounding will not the bottleneck of performance. Let us put the effort to keep functionality firstly, next, we should reconsider our code and find the performance bottleneck leverage the performance tool (like DS5 Streamline).

Actually, Requantize is the bottleneck in my observation on Rasp Pi 4. For TFLite quantized mobilenetv1, currently with master + auto-tuning, I get 56 ms (TFLite is 35 ms).  Upon deeper analysis, I found requantize to be the bottleneck (even with UPWARD rounding which is the cheapest). The reason is that calculations happen in int64 and either ARM does not have good instructions or LLVM does not pick up the right instructions.
As an experiment, I forced the datatype of Requantize to be int32. This will lead to bad accuracy but it will give us an idea what is the minimum latency of Requantize op. The runtime reduced from earlier 56 ms to 36 ms (almost as good as TFLite). So, requantize is the bottleneck.
Additionally, we should notice that TFLite rounding is not a standard rounding. They have moved backwards from the ARM instructions they want to use. Because of that they have 2 roundings instead of 1 - 1 in SaturatingRoundingDoublingHighMul (non-standard rounding) and other is
RoundingDivideByPOT (standard). Given there are 2 roundings, they have also made an accuracy-performance tradeoff. So, my suggestion is that we should also wisely think if it makes sense to completely follow TFLite rounding as the golden reference. For example, following reference function
// This function implements the same computation as the ARMv7 NEON VQRDMULH
// instruction.

template <>
inline std::int32_t SaturatingRoundingDoublingHighMul(std::int32_t a,
                                                      std::int32_t b) {
  bool overflow = a == b && a == std::numeric_limits<std::int32_t>::min();
  std::int64_t a_64(a);
  std::int64_t b_64(b);
  std::int64_t ab_64 = a_64 * b_64;
  std::int32_t nudge = ab_64 >= 0 ? (1 << 30) : (1 - (1 << 30));
  std::int32_t ab_x2_high32 =
      static_cast<std::int32_t>((ab_64 + nudge) / (1ll << 31));
  return overflow ? std::numeric_limits<std::int32_t>::max() : ab_x2_high32;
}

This function alone needs around 10 Relay ops (if not more), but in TFLite this is just  1 ARM instruction - VQRDMULH. I doubt that LLVM will be able to use that instruction automatically.
In summary, I understand the benefits of exact match between TFLite and TVM. But in contrast to FP32 networks, this is one of the cases, where frameworks have made their choice to tradeoff perf and accuracy for a specific ARM device. Note that with non-TFLite rounding, the application accuracy is still similar, so it might be an acceptable tradeoff. Therefore, I would encourage to think more deeply. I worry that the extensive work going in this PR might not be used because of bad performance.
If we relax the exact tensor match criteria, there might be other ways to get the perfromance like using tensorize.

Actually, Requantize is the bottleneck in my observation on Rasp Pi 4. For TFLite quantized mobilenetv1, currently with master + auto-tuning, I get 56 ms (TFLite is 35 ms). Upon deeper analysis, I found requantize to be the bottleneck (even with UPWARD rounding which is the cheapest). The reason is that calculations happen in int64 and either ARM does not have good instructions or LLVM does not pick up the right instructions.
As an experiment, I forced the datatype of Requantize to be int32. This will lead to bad accuracy but it will give us an idea what is the minimum latency of Requantize op. The runtime reduced from earlier 56 ms to 36 ms (almost as good as TFLite). So, requantize is the bottleneck.
Additionally, we should notice that TFLite rounding is not a standard rounding. They have moved backwards from the ARM instructions they want to use. Because of that they have 2 roundings instead of 1 - 1 in SaturatingRoundingDoublingHighMul (non-standard rounding) and other is
RoundingDivideByPOT (standard). Given there are 2 roundings, they have also made an accuracy-performance tradeoff. So, my suggestion is that we should also wisely think if it makes sense to completely follow TFLite rounding as the golden reference. For example, following reference function
// This function implements the same computation as the ARMv7 NEON VQRDMULH
// instruction.

template <>
inline std::int32_t SaturatingRoundingDoublingHighMul(std::int32_t a,
                                                      std::int32_t b) {
  bool overflow = a == b && a == std::numeric_limits<std::int32_t>::min();
  std::int64_t a_64(a);
  std::int64_t b_64(b);
  std::int64_t ab_64 = a_64 * b_64;
  std::int32_t nudge = ab_64 >= 0 ? (1 << 30) : (1 - (1 << 30));
  std::int32_t ab_x2_high32 =
      static_cast<std::int32_t>((ab_64 + nudge) / (1ll << 31));
  return overflow ? std::numeric_limits<std::int32_t>::max() : ab_x2_high32;
}

This function alone needs around 10 Relay ops (if not more), but in TFLite this is just 1 ARM instruction - VQRDMULH. I doubt that LLVM will be able to use that instruction automatically.

The vqrdmulh instruction is pretty standard on all Arm CPUs that have Advanced SIMD in 32 bit. C compilers will struggle with generating that instruction automagically. It's just too big an idiom to recognize and I don't think llvm has that infrastructure to detect it . Is there anything preventing us from having a specific tvm intrinsic that we can lower to this vqrdmulh instruction ? Ofcourse if this fails we could instead lower to the C implementation ?
The equivalent aarch64 instruction is sqrdmulh IIRC and is default on all devices. There should be an llvm intrinsic equivalent of this.
ramana

Moving to discuss as we are talking a lot :)
https://discuss.tvm.ai/t/tflite-rounding/6287

Actually, Requantize is the bottleneck in my observation on Rasp Pi 4. For TFLite quantized mobilenetv1, currently with master + auto-tuning, I get 56 ms (TFLite is 35 ms).  Upon deeper analysis, I found requantize to be the bottleneck (even with UPWARD rounding which is the cheapest). The reason is that calculations happen in int64 and either ARM does not have good instructions or LLVM does not pick up the right instructions.
As an experiment, I forced the datatype of Requantize to be int32. This will lead to bad accuracy but it will give us an idea what is the minimum latency of Requantize op. The runtime reduced from earlier 56 ms to 36 ms (almost as good as TFLite). So, requantize is the bottleneck.
Additionally, we should notice that TFLite rounding is not a standard rounding. They have moved backwards from the ARM instructions they want to use. Because of that they have 2 roundings instead of 1 - 1 in SaturatingRoundingDoublingHighMul (non-standard rounding) and other is
RoundingDivideByPOT (standard). Given there are 2 roundings, they have also made an accuracy-performance tradeoff. So, my suggestion is that we should also wisely think if it makes sense to completely follow TFLite rounding as the golden reference. For example, following reference function

// This function implements the same computation as the ARMv7 NEON VQRDMULH

// instruction.

template <>

inline std::int32_t SaturatingRoundingDoublingHighMul(std::int32_t a,

                                                      std::int32_t b) {

  bool overflow = a == b && a == std::numeric_limits<std::int32_t>::min();

  std::int64_t a_64(a);

  std::int64_t b_64(b);

  std::int64_t ab_64 = a_64 * b_64;

  std::int32_t nudge = ab_64 >= 0 ? (1 << 30) : (1 - (1 << 30));

  std::int32_t ab_x2_high32 =

      static_cast<std::int32_t>((ab_64 + nudge) / (1ll << 31));

  return overflow ? std::numeric_limits<std::int32_t>::max() : ab_x2_high32;

}

This function alone needs around 10 Relay ops (if not more), but in TFLite this is just  1 ARM instruction - VQRDMULH. I doubt that LLVM will be able to use that instruction automatically.
In summary, I understand the benefits of exact match between TFLite and TVM. But in contrast to FP32 networks, this is one of the cases, where frameworks have made their choice to tradeoff perf and accuracy for a specific ARM device. Note that with non-TFLite rounding, the application accuracy is still similar, so it might be an acceptable tradeoff. Therefore, I would encourage to think more deeply. I worry that the extensive work going in this PR might need major reconsideration when we dont see good performance improvements.
If we relax the exact tensor match criteria, there might be other ways to get the perfromance like using tensorize.
@u99127 @masahi might also be interested in this.

Please note this thread, if we don’t have bit-extract result, we will have many problems, not just trade off. Mobilenet accuracy result is not enough.
You could also one blog how do boost the quant perf. Inside it, we could get the bit-extract result and use the same rouding of TFLite, but we could beat TFLite performance a lot. However, the design has a little bit, we design to implement one q_conv2d / q_requantize to contorl it. For q_conv2d, we tensorize it and do requantize inside it. For depthwise convolution, we even not tensorize and use q_conv2d + q_requantize, we could still beat TFLite and QNNPack a lot. Out current design of one problem is we lower too many ops and can not make sure all computations inside register and NCHW has bad cache like previous blog said.
We should make sure functionality firstly. Then we consider the performance. I think it is not only the arm instruction you mention but also many other things we need to do.

Ok. I forgot the accuracy issue of Object detection models.
I will also go through the blog to see if we can do something in the short term. If possible, we should make a plan for what all things that we need to do to get the performance speedup. I will share the results in a discussion post in few days. Maybe you can share your experience from there.

Ok. I forgot the accuracy issue of Object detection models.
I will also go through the blog to see if we can do something in the short term. If possible, we should make a plan for what all things that we need to do to get the performance speedup. I will share the results in a discussion post in few days. Maybe you can share your experience from there.

Yeah. Because of the different design, my original plan is we could achieve the same result of TFLite, then we start to consider how to achieve better performance leveraging our previous implementation of q_conv2d. :-)

Hi, I am wondering if we can take the pool operator changes from this PR and send a separate PR. I know that the earlier decision on this was to put everything in one PR for easy backporting. But, as this PR might take more time, and master TVM today does not implement any rounding for pooling ops for integer datatypes, I think it is valuable to separate that part out. What do you think? @FrozenGene @LiangHao151941


Update github.com/go-enry/go-enry to v2.9.1

go-enry v2.9.1 includes latest file patterns from Linguist, which can identify more generated file type, eg. pdm.lock.

I would like to see this being backported if the new minor is not coming soon.

I was unable to create a backport for 1.22. @stevapple, please send one manually. :tea:
go run ./contrib/backport 32295
...  // fix git conflicts if any
go run ./contrib/backport --continue


Allow the autogeneration of unique world perm names.

As it stands, all world names, when permissioned are lowercased via the superperms api (PermissibleBase.java#53, PermissibleBase.java#69, PermissibleBase.java#89, PermissibleBase.java#254 and PermissibleBase.java#184).
This means that if you have a world WOrLD, it's permission will always be multiverse.access.world. That's fine and dandy, except when a user were to also have a world named World. This permission would also be multiverse.access.world. Obviously, the easy fix is

Hey, users, don't name worlds the same thing with different case" (something that is not possible on case-insensitive filesystems).

However, I know if I say that, then someone will come up with a reason that they want it.
Since putting in a special case for worldnames seems a bit out of line, I'm proposing the following changes:

Uniquify world permissions by adding a digit after them. ex. multiverse.access.world, multiverse.access.world2
If you have permission to access a world's information via /mv info, then you will be able to see what permission is required to enter said world.

This is related to #421
This could even be a config variable that would be off by default. I think I like that idea the best. Thoughts?

Are you sure? For me I know the reason is that I use a case insensitive filesystem, but I'm pretty sure you're allowed to have 2 folders named the same (different case) and bukkit will be cool with it.
I'm not certain.  But when you specify a world in Bukkit.getWorld(String) the casing does not matter.  But perhaps because I'm on a case insensitive system.

Ok. Regardless. I agree. I wanted to get feedback and I did. Closing this as "not gonna do".


validate uri first

when you don't set  'source'  in , iOS will crash, while
Android is OK.
Android will return immediately if uri is empty, and iOS just do the
same thing

More detail:
It will crash at RCTVideo.m#L307
[[NSURL alloc] initFileURLWithPath:[[NSBundle mainBundle] pathForResource:uri ofType:type]];
The log could be *** - [NSURL initFileURLWithPath:]: nil string parameter

Confirmed to work very nicely


Support for metric dimensions

Comparing the SDK MetricTelemetry.ts and the REST API documentation, it seems there is a mismatch. I can't find any reference to stdDev attribute, and all multi-dimensions feature is missing.
Are there any plan to fix that? Would you accept a PR?

@henryju can you provide more details about which REST API documentation you are referring?

Sorry, I moved away from this topic, and I can't remember exactly what it was about. Feel free to close this issue.


Cancel previous CI workflow if new commit pushed

Search before asking

[X] I had searched in the issues and found no similar issues.

Motivation
So that we don't waste time to verify a previous commit - what we're interested in is the latest view.
Solution
Add the following snippet to kvrocks.yaml:
concurrency:
  group: ${{ github.ref }}-${{ github.workflow }}
  cancel-in-progress: true

See also https://github.com/apache/pulsar/blob/b23689930e3af65cfc913e2890164a071acda440/.github/workflows/pulsar-ci.yaml#L29-L31.
Are you willing to submit a PR?

[ ] I'm willing to submit a PR!

For more information about GitHub Actions settings, see also https://docs.github.com/en/actions/using-jobs/using-concurrency#example-only-cancel-in-progress-jobs-or-runs-for-the-current-workflow

Let me take this issue.
Ideally, this will be my first contribution to Apache project.

@Xuanwo Thanks!
I'll assign to you. Go ahead!

I love this feature!