feat(core): Modify core module to implement style guidelines.

Update the core module to implement the style guidelines.
Reduce size of init.js - moved filter logic out to it's own config.
Rename Menus to menuService
Ref: #657
Ref: #943

@rhutchison Left some inline comments.  Overall love the PR and is almost spot on with where I was going with it.

@rhutchison , @trainerbill  once you guys wrap it up, then @codydaig or @ilanbiala can take a look and comment.

@rhutchison Based on a fork by @trainerbill I moved auth-interceptor.service.js to the users module and changed the responseError function to:
function responseError(rejection) {
      if (!rejection.config.ignoreAuthModule) {
        switch (rejection.status) {
          case 401:
            var auth = $injector.get('Authentication');
            auth.signout();
            $injector.get('$state').transitionTo('authentication.signin');
            break;
          case 403:
            $injector.get('$state').transitionTo('forbidden');
            break;
        }
      }

@zeldar that should probably go in a different PR.  This is just to conform to the style-guide.
I think it is ready to go.

Added a few comments.

@ilanbiala took care of double IIFE - just waiting for successful build.

@codydaig @lirantal :eyes:?

Please re-consider having everything use vm in the html views.
Yes, using vm consistently inside the controller function makes sense.
var vm = this;
Awesome.
However, I do not think it is advisable to have vm be the variable inside all the html views.
<div class="container" ng-controller="HeaderController as vm">
There is an important note in John Papa's guidelines about controllerAs with vm.
https://github.com/johnpapa/angular-styleguide#style-y032

Note: When working with larger codebases,using a more descriptive name can help ease cognitive overhead & searchability [emphasis added]. Avoid overly verbose names that are cumbersome to type.

<!-- avoid -->
<input ng-model="customerProductItemVm.text">

<!-- recommended -->
<input ng-model="productVm.id">

@mg1075 I put forth a commit for the community to review.  If they are good with it, I will squash the commit.
@ilanbiala @lirantal @codydaig

My personal preference would be to stick with 'vm'.
@mleanos

I thought we were trying to get away from using the ng-controller attribute in the views.

I don't know if there is a way around it with HeaderController without implementing nested views.  Any ideas?

The John Papa Styleguide examples aren't very clear on using the descriptive vm naming convention. What's the real benefit of doing this? There's only one controller per view, so it seems that vm would never have conflicts with any other vm.

I believe @mg1075 is saying for searchability in a project.

@rhutchison I haven't tested this, or given it a thorough review. https://github.com/meanjs/mean/pull/1183

@lirantal @codydaig @ilanbiala do either of you have an opinion on https://github.com/meanjs/mean/pull/1202#issuecomment-184494736
https://github.com/meanjs/mean/commit/84daa57dd4e3b33966a3233cf9438da6164b1ca0
See comments above.

When someone defines 2 scopes, both with $scope.vm = this, how would it be clear which vm is being referenced?

I prefer it just as vm.  When using nested views you would have to define multiple variables and remember what to place in each variable.  Using just vm, you can easily just include the view with a different controller and place whatever variables the view needs on vm.  I honestly don't know how that would work with nested views.  I guess normally you would also have a different controller, but there may be times that you would not.

@ilanbiala the way mean is structured, it's 1 view, 1 controller.  We do not really have the complexity of a larger application.
Even in the enterprise application I am working on, we stick to controllerAs: 'vm' and I think it works well - the rule is clear/consistent.
I think descriptive names would be more valuable if we were referencing more than 1 controller scope in a view.

I agree with what @rhutchison is saying. I don't see many use cases where you wouldn't just have a one-to-one relationship between views & controllers.
In the case of using nested-views, I think it's more likely that we would have a view that is used within multiple parent views, that have their own controller. in that case, the controller would be providing smaller sets of data/functionality that the nested-view needs. Correct me if I'm wrong, about that assumption.
As Ryan mentioned earlier, I don't think we can avoid having ng-controller in the Header view.

Yeah, if we ng-include the file then we can't really do anything about it I think, we'd have to declare the header as its own ui-view in the layout file.

Why don´t you update to angular 1.5 and use the new component() method? Then you could add your header for example with

@Meistercoach83 or we could just make it a directive

Is the consensus to revert https://github.com/meanjs/mean/commit/84daa57dd4e3b33966a3233cf9438da6164b1ca0 and squash?  I would like to get this merged.

My vote is Yes; revert the commit. I much prefer using vm everywhere, rather than the descriptive naming.

@rhutchison This looks good for the most part. Left a couple line comments.
Can you rebase? I'd like to pull this down, and test.

@rhutchison I think some of these changes were implemented with fixes, and the Angular 1.5 upgrade. Can you rebase, and verify? Once the remaining issues have been addressed, I think this will be good to go.

@mleanos I'll try to rebase this evening

Coverage remained the same at 70.615% when pulling 4db1e19439f33709eb3cb67de3b643380b7bef1c on Gym:core-styleguide into bc3d536ed25815198c805fa1d5e3f71c61e3fd79 on meanjs:master.

@rhutchison left some comments.

I'd like to use $window and $document everywhere that it can be. Is there any case where we have to use window or document vs. $window and $document?

@ilanbiala @codydaig @mleanos @lirantal this is ready.

Coverage remained the same at 70.615% when pulling 368bc79e8ecfda2bf8c0c86da0cd8bdb63e7d915 on Gym:core-styleguide into 89d302d781c1f2854708437bc9a1232d211d2c9b on meanjs:master.

@rhutchison left an explanation of delete comment, but other than that the only thing I'm thinking of is the $window and $document stuff that @codydaig is also referencing. @rhutchison did you find any sources that say it isn't possible to use them in scenarios like ours?

@ilanbiala within the context of angular, I'm sure it's possible, but not sure how it's done in these cases which are bootstrapping angular.  Please let me know if you can find a way and I'll gladly update.

Coverage remained the same at 70.615% when pulling 847ad7aea1613016f8fcd36346da59535b8755ee on Gym:core-styleguide into 59e6daa63d65798886e446fc7696b6331c9e3aba on meanjs:master.

I was out of town over the weekend, so I apologize for the lack of feedback. I'm planning on pulling this down in the next day or so.

@rhutchison I just pulled this down, and tested. Overall, it looks pretty good.
I think there are a few remaining comments that need to be addressed/finalized. Other than that this looks ready to me.

@mleanos thanks for reviewing.  Please point me to the outstanding items.

@rhutchison Looking at it closer, I see the aspects where you need to use window vs. $window. I'm cool with the way that it is currently. (I don't think I saw window or document being used inside a controller or service).

LGTM, only change is the !== -1. Other than that, everything is fine.

@ilanbiala done

Coverage remained the same at 70.615% when pulling b2462ec86cc30c2f92ff330471fa2d6caed76a6d on Gym:core-styleguide into 59e6daa63d65798886e446fc7696b6331c9e3aba on meanjs:master.

@mleanos LGTM.

LGTM. @mleanos Merge before merge conflicts arise!!!!


Only check proptypes at element creation like React does.

React Implmentation

I have a PR for this incoming.

A+, this is something I just overlooked for sure.  Are you thinking just check them in the constructor?

No I have them checking after the vnode is normalized. This approach's only flaw is that children propType checking does not work. I added a test and am working on it now.

@tkh44 indeed - the props.children normalization happens before calling lifecycle methods. thinking we'll need to clone props and inject children when doing the proptype check.

This is resolved in Preact X. Make sure to use preact/compat instead of preact-compat.


this.id or this.created_at is no longer available inside $beforeInsert?

Please read the instructions linked below before opening an issue.
https://vincit.github.io/objection.js/guide/contributing.html#issues

I am going to modify a field value ( remove all spaces etc ) before inserting it into the DB and I felt beforeInsert should be a good place for it, but i wonder now if I can do that using beforeinsert.
What is the best solution for this in Objection?


[MetaSchedule][M4a] Rewrite-Cooperative-Fetch

It is one of the PostProc for mate-schedule. Please see the tracking issue https://github.com/apache/tvm/issues/8473
Co-authored-by: Junru Shao<EMAIL_ADDRESS>Co-authored-by: Xiyou Zhou<EMAIL_ADDRESS>Co-authored-by: Bohan Hou<EMAIL_ADDRESS>Co-authored-by: Ruihang Lai<EMAIL_ADDRESS>Co-authored-by: Hongyi Jin<EMAIL_ADDRESS>Co-authored-by: Wuwei Lin<EMAIL_ADDRESS>cc @junrushao1994 @comaniac

Hey please rebase to the latest main


Split Avalon 2016-18 and Avalon 2019+

CAN fingerprint from a 2020 Avalon (https://gist.github.com/nelsonjchen/e42999408d40bfe40d21b7910cde0c83 ignore it saying 2016)
Reverted added fw from the PRs listed in the commits, this is assuming that the fw previous to these commits was strictly 2016-2018. Also need to make sure that having fw in the 2019 Avalon that exists in the older Avalon candidate won't cause any issues.

[x] Pick known 2016 vs. 2019+ Avalons and test fingerprint them, making sure the Avalons with a longer EPS_STATUS msg fp as AVALON_2019

Merged the opendbc PR, bump it to opendbc master

and Avalon2019h use different dbc file toyota_nodsu_hybrid_pt_generated

@toyboxZ are you sure? The 2021 Avalon hybrid is still TSSP, like the 2020 Avalon. I'm thinking it's the one signal that's different

Getting a user with a 2019 Avalon to test this!

@toyboxZ are you sure? The 2021 Avalon hybrid is still TSSP, like the 2020 Avalon. I'm thinking it's the one signal (EPS_STATUS) that's different. Are you saying the hybrid has more unmatched signals?

yes，Avalon hybrid can use smartdsu s&g. dbc file toyota_nodsu_hybrid_pt_generated.
also can join the group NO_STOP_TIMER_CAR

The only difference between the new Avalon dbc (from the 17 Avalon dbc) is the addition of the BSM signal (which doesn't even seem to work for the hybrid Avalon), the STEERING_LTA message (which also isn't present in the Hybrid since it's not TSS2), the brake module (scales are different), and the addition of a few sport/econ messages. So I don't think it's critical to use a different DBC for the hybrid.


The irsa s3_object always have to be updated

Is this a BUG REPORT or FEATURE REQUEST?:
Feature request
What happened:
The module.irsa.aws_s3_bucket_object.keys_json always have to be updated, since keys_json is reading from file data source.
Terraform will perform the following actions:

  # module.irsa.data.local_file.keys_json will be read during apply
  # (config refers to values not yet known)
 <= data "local_file" "keys_json"  {
 ...

  # module.irsa.aws_s3_bucket_object.keys_json will be updated in-place
  ~ resource "aws_s3_bucket_object" "keys_json" {
  ...

According to the data source document, it might be caused by data resource won't be checked before apply phase.
https://www.terraform.io/docs/language/data-sources/index.html#data-resource-dependencies
NOTE: In Terraform 0.12 and earlier, due to the data resource behavior of deferring the read until the apply phase when depending on values that are not yet known...

What you expected to happen:
The S3 object only have to be updated when necessary.
How to reproduce it (as minimally and precisely as possible):
Launch example cluster:
https://github.com/getamis/vishwakarma/tree/master/examples/kubernetes-cluster
Then, apply the module.master many times.
terraform apply -target=module.master

Anything else we need to know?:
I've wrote a mitigation PR for this issue: https://github.com/getamis/vishwakarma/pull/110

Thank for your help


Leave better breadcrumb for cookbook generator in non-delivery case.

Do we have the cookbook name in a variable already to reference in our output? It'd be nice to provide the real path instead of new_cookbook.

@charlesjohnson It is using the real path, new_cookbook is only used in the tests.

:+1:

http://24.media.tumblr.com/19d3a6d4e8704e86d8d94105a50e403f/tumblr_mvlyw1kzkv1sodo64o1_400.gif


Unsupported object type => Ignore repository, not all repositories

Once an unsupported object type is contained in one of the synced packages, the complete ZABAPGIT is not functional anymore. Though the report does not terminate technically, the HTML-container appears empty.
Desired behavior: Provide an error message only for the affected package/fork. From this repository, no further action should be possible. Other repositories should be unaffected.

Tried to reproduce the error, in this example there are 2 repositories "IA" and "abapGit". The "IA" is not functional, but the abapGit repository seems to work:

I figured out that this happens at eg. startup of abapgit, and not when adding like in the above example. Issue reproduced.

Hi Oliver,
Can you try again, I think it has been fixed in above commit. It will look something like the following:

should have been fixed, closing issue, feel free to reopen in case of any issues

Confirmed. Couldn't reproduce it. Thx


depth_from_video_in_the_wild: not able to reproduce the result

@gariel-google  Dear author, Thanks for sharing the source code of the paper.
I was trying to reproduce the result of the paper using your code. However, with your default setting (batch size=4, learning_rate=0.0002, etc.) training from scratch, the result I got it's quite far from what you stated in the paper (Abs Rel 0.147 for the best checkpoint within around 370k-th step vs 0.128 in the paper). For your information, I am using the evaluation code from sfmlearner as what struct2depth does.
Therefore, may I know what's setting for obtaining the paper's result? Or is there anything critical part missing in the current released code (maybe pretrained checkpoint for example)?
Thank you in advance.

Hi
I am assuming you're training on KITTI?
Did you create a "possibility mobile" mask for each image? Did you use an
segmentation network to do that? Which one?
On Sat, Aug 17, 2019, 7:11 AM juju<EMAIL_ADDRESS>wrote:

@gariel-google https://github.com/gariel-google Dear author, Thanks for
sharing the source code of the paper.
I was trying to reproduce the result of the paper using your code.
However, with your default setting (batch size=4, learning_rate=0.0002,
etc.) training from scratch, the result I got it's quite far from what you
stated in the paper (Abs Rel 0.147 for the best checkpoint within around
370k-th step vs 0.128 in the paper). For your information, I am using the
evaluation code from sfmlearner
https://github.com/tinghuiz/SfMLearner/tree/master/kitti_eval as what
struct2depth does.
Therefore, may I know what's setting for obtaining the paper's result? Or
is there anything critical part missing in the current released code (maybe
pretrained checkpoint for example)?
Thank you in advance.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google-research/google-research/issues/46?email_source=notifications&email_token=ADXKUNH2XZHXPU5SHQ6YEDDQFABJPA5CNFSM4IMPQMMKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HFZCAQA,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADXKUNAY4ZPYW5AVT3TDSE3QFABJPANCNFSM4IMPQMMA
.

Hi, Sorry for the missing information in my previous comment.
Yes, I am using KITTI, eigen split, using data generation code from vid2depth as what struct2depth does.
Yes, I have created a "possibility mobile" mask for each image. I am using the same masks of struth2depth (each object has different object ID and the objects are being tracked across three consecutive sequences). I am using Mask-RCNN to obtain the mask. Also, I have turned boxify=True, so the masks will become bounding boxes as I understand.
For your information, I also attached the the image in tensorboard of variable seg_stack of line 172, model.py.

Thanks for the information, that helps a lot. On KITTI we trained with a
barch size of 16 and a learning rate of 1e-4. All other parameters are at
their default values. In addition, we initialized from an ImageNet
checkpoint for ResNet 18. The current release does not yet support
initialization form a checkpoint, but it should be easy to set up. Fig. 5
in the paper indicates when you should expect convergence. Lastly, are you
using
https://github.com/google-research/google-research/blob/master/depth_from_video_in_the_wild/model.py#L388
to
infer depth?
We are planning to release pretrained checkpoints, more code and more
dicumentation before ICCV. We will do our best to do it sooner than later.
On Sat, Aug 17, 2019 at 6:34 PM juju<EMAIL_ADDRESS>wrote:

Hi, Sorry for the missing information in my previous comment.
Yes, I am using KITTI, eigen split, using data generation code from
vid2depth
https://github.com/tensorflow/models/tree/master/research/vid2depth/dataset
as what struct2depth does.
Yes, I have created a "possibility mobile" mask for each image. I am using
the same masks of struth2depth (each object has different object ID and the
objects are being tracked across three consecutive sequences). I am using
Mask-RCNN to obtain the mask. Also, I have turned boxify=True, so the
masks will become bounding boxes as I understand.
For your information, I also attached the the image in tensorboard of
variable seg_stack of line 172, model.py
https://github.com/google-research/google-research/blob/master/depth_from_video_in_the_wild/model.py
.
[image: Screenshot 2019-08-18 at 9 24 02 AM]
https://user-images.githubusercontent.com/18667188/63219007-1355ea00-c19b-11e9-88ef-e1da17bf8943.png
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google-research/google-research/issues/46?email_source=notifications&email_token=ADXKUNFFVIYEE5PMU3XG3VLQFCRKLA5CNFSM4IMPQMMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4QWRKA#issuecomment-522283176,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADXKUNBXUHPWYVEWW6EDMXDQFCRKLANCNFSM4IMPQMMA
.

Thanks for the reply.
Yes, for depth inference, I am using the link you mentioned in your previous comment.
In Figure 5 of the paper, for Evaluated on KITTI, the training converges at around 1 million training images. So if we assume that batch_size=4, learning_rate=0.0002 has similar convergence, and as I found that my training has Abs_Rel=0.147 for the best checkpoint within around 370k-th step (370k steps=1.48 million images). Therefore, can I conclude that the pretrained ImageNet checkpoint has huge impact (0.147 vs 0.128) on the result?
Looking forward to your release and thanks for the efforts.

I feel that it's too early to draw a conclusion, we need to investigate
this more. If you're ready to do it together, that would be great. We are
planning to release a pretrained KITTI checkpoint in the next few days, and
a first step can be establishing that we agree on how eval metrics are
calculated. We can both run evals on the same checkpoint and exchange
results. Based on that, we'll see how to proceed. How does that sound?
On Sun, Aug 18, 2019 at 8:12 AM juju<EMAIL_ADDRESS>wrote:

Thanks for the reply.
Yes, for depth inference, I am using the link
https://github.com/google-research/google-research/blob/master/depth_from_video_in_the_wild/model.py#L388
you mentioned in your previous comment.
In Figure 5 of the paper, for Evaluated on KITTI, the training
converges at around 1 million training images. So if we assume that batch_size=4,
learning_rate=0.0002 has similar convergence, and as I found that my
training has Abs_Rel=0.147 for the best checkpoint within around 370k-th
step (370k steps=1.48 million images). Therefore, can I conclude that the
pretrained ImageNet checkpoint has huge impact (0.147 vs 0.128) on the
result?
Looking forward to your release and thanks for the efforts.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google-research/google-research/issues/46?email_source=notifications&email_token=ADXKUNFWASS3J5Q4QMRTSDTQFFRHBA5CNFSM4IMPQMMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4RB75Q#issuecomment-522330102,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADXKUNBF5JPKUAW3SO7EAX3QFFRHBANCNFSM4IMPQMMA
.

Hi, it sounds great to me, Let's do it together.

In the paper,it said that some videos selected from 3079 YouTube8M videos  labled 'Quadcopter', so does the IDs of them will be public soon?
And I realized that it need much time for process so many videos into three-frames-split and generate its masks and alignment，so the pretrained YouTube8M checkpoint will be also released soon?
I also notice that the current release does not yet support initialization form the resnet18 checkpoint pretrained on ImageNet, I'm trying to write codes to implement that since  struct2depth has the similar codes organization...

Yes, we will release the IDs soon, and also pretrained checkpoints.
Regarding the initialization mechanism, we're trying to release it soon,
but it might take a bit longer. We will update as soon as possible.
On Mon, Aug 19, 2019 at 7:37 PM linxin<EMAIL_ADDRESS>wrote:

In the paper,it said that some videos selected from 3079 YouTube8M videos
labled 'Quadcopter', so does the IDs of them will be public soon?
And I realized that it need much time for process so many videos into
three-frames-split and generate its masks and alignment，so the pretrained
YouTube8M checkpoint will be also released soon?
I also notice that the current release does not yet support initialization
form the resnet18 checkpoint pretrained on ImageNet, I'm trying to write
codes to implement that since struct2depth has the similar codes
organization...
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google-research/google-research/issues/46?email_source=notifications&email_token=ADXKUNDZJMA5PUZ3NXGIHHLQFNKE7A5CNFSM4IMPQMMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4U3TRQ#issuecomment-522828230,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADXKUNFFSD6GL4UA4PINNG3QFNKE7ANCNFSM4IMPQMMA
.

Hi @gariel-google, I am also evaluating the egomotion prediction by using inference_egomotion to obtain egomition, and using sfmlearner to compute 5-point/3-point ATE.
Since I am just evaluating on my trained model (the one with Abs_Rel=0.147, trained on eigen split training set) and eigen split training set has overlap frames with odometry sequence 09 and 10, the ATE should be reasonably good. However, the result I got it's quite bad compared to what you stated in the paper. (I apologize if I am evaluating the egomotion prediction wrongly.)

Seq. 09
Seq. 10

5-point
0.0296
0.0245

3-point
0.0212
0.0180

For your information, I also attached the plotted trajectories.

Maybe we could exchange the odometry evaluation result as well?

We will release the KITTI-trained checkpoints, that should be enough for
comparing odometry evals, am I correct?
It will take at least a few days though - please bear with me ;-)
On Mon, Aug 19, 2019 at 9:22 PM juju<EMAIL_ADDRESS>wrote:

Hi @gariel-google https://github.com/gariel-google, I am also
evaluating the egomotion prediction by using inference_egomotion
https://github.com/google-research/google-research/blob/master/depth_from_video_in_the_wild/model.py#L423
to obtain egomition, and using sfmlearner
https://github.com/tinghuiz/SfMLearner/blob/master/kitti_eval/eval_pose.py
to compute 5-point/3-point ATE.
Since I am just evaluating on my trained model (the one with
Abs_Rel=0.147, trained on eigen split training set) and eigen split
training set has overlap frames with odometry sequence 09 and 10, the ATE
should be reasonably good. However, the result I got it's quite bad
compared to what you stated in the paper. (I apologize if I am evaluating
the egomotion prediction wrongly.)

Seq. 09
Seq. 10

5-point
0.0296
0.0245

3-point
0.0212
0.0180

For your information, I also attached the plotted trajectories.
[image: seq09]
https://user-images.githubusercontent.com/18667188/63317302-e0e7f080-c344-11e9-9916-90d3327fd694.png
[image: seq10]
https://user-images.githubusercontent.com/18667188/63317304-e3e2e100-c344-11e9-9780-5357bfd24a6b.png
Maybe we could exchange the odometry evaluation result as well?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google-research/google-research/issues/46?email_source=notifications&email_token=ADXKUNHMCKHUBUG2TGK5WZDQFNWRVA5CNFSM4IMPQMMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4VAFVA#issuecomment-522846932,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADXKUNH5Z6TWRLZYN6HYUGDQFNWRVANCNFSM4IMPQMMA
.

Yes, as we exchange results it should be enough for comparing odometry evals. Thanks again for the efforts.

We just released some checkpoints (links in the README file) with the
respective metrics. Note that there is a slight change in the code (in
depth_prediction_net). Would you be ready to try them and see what metrics
do you obtain? YouTube8M IDs coming soon.
On Mon, Aug 19, 2019 at 10:41 PM juju<EMAIL_ADDRESS>wrote:

Yes, as we exchange results it should be enough for comparing odometry
evals. Thanks again for the efforts.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google-research/google-research/issues/46?email_source=notifications&email_token=ADXKUNABRKRBNTNND4SDVPDQFN7YNA5CNFSM4IMPQMMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4VDZ3Q#issuecomment-522861806,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADXKUNE3WQWT3JJD2M35E2TQFN7YNANCNFSM4IMPQMMA
.

Thanks for the release. Yes, I am ready to try! However, I noticed that only the data files are released, and as I understand that to restore a model in TensorFlow, we need 3 files (correct me if I am wrong)--index, data, meta.  Therefore, could you release the complete checkpoints?

Sorry about that. I replaced the links, they now link to zip files that
contain all the checkpoint components. Could you check them out? Thanks!
On Wed, Aug 21, 2019 at 11:14 PM juju<EMAIL_ADDRESS>wrote:

Thanks for the release. Yes, I am ready to try! However, I noticed that
only the data files are released, and as I understand that to restore a
model in TensorFlow, we need 3 files (correct me if I am wrong)--index,
data, meta. Therefore, could you release the complete checkpoints?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google-research/google-research/issues/46?email_source=notifications&email_token=ADXKUND5Z64FBXF64Z3OLC3QFYVGDA5CNFSM4IMPQMMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD44AFLA#issuecomment-523764396,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADXKUNFFXKE4JEXM3LIAY63QFYVGDANCNFSM4IMPQMMA
.

YouTube8M IDs are out (see the README file).
On Thu, Aug 22, 2019 at 10:34 AM Ariel Gordon<EMAIL_ADDRESS>wrote:

Sorry about that. I replaced the links, they now link to zip files that
contain all the checkpoint components. Could you check them out? Thanks!
On Wed, Aug 21, 2019 at 11:14 PM juju<EMAIL_ADDRESS>wrote:

Thanks for the release. Yes, I am ready to try! However, I noticed that
only the data files are released, and as I understand that to restore a
model in TensorFlow, we need 3 files (correct me if I am wrong)--index,
data, meta. Therefore, could you release the complete checkpoints?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google-research/google-research/issues/46?email_source=notifications&email_token=ADXKUND5Z64FBXF64Z3OLC3QFYVGDA5CNFSM4IMPQMMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD44AFLA#issuecomment-523764396,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADXKUNFFXKE4JEXM3LIAY63QFYVGDANCNFSM4IMPQMMA
.

Hi @gariel-google, thanks for the work. The new zip files work for me. I have tested the checkpoint trained on KITTI. The following is what I have:

depth result is different when inference with different batch_size:

abs_rel
sq_rel
rms
log_rms
a1
a2
a3

batch_size=1
0.1262
0.9462
5.2214
0.2086
0.8470
0.9475
0.9774

batch_size=16
0.1305
1.0186
5.3237
0.2136
0.8389
0.9430
0.9751

When batch_size=1, we have the same result, therefore we should be having the same evaluating metrics. However, depth output doesn't have a consistent result when batch_size changes, is it the same case for you? where does the variation come from?

odometry result (ATE) when inference with batch_size=1:

seq_09
seq_10

5-point
0.0231
0.0195

3-point
0.0170
0.0149

The plotted trajectories:
seq_09

seq_10

The odometry result looks quite bad for me. Do you have the same result? Since eigen split training set has overlap with odometry sequence 09 and 10, the ATE should be better than what you have stated in the paper (0.0231 vs 0.012 and 0.0195 vs 0.010)?

Thanks much @liyingliu for testing the checkpoints so quickly. I am happy
that we are getting the exact same result on depth prediction.
Regarding the batch size, we tested at 1. If batch-normalization is
replaced everywhere by randomized layer normalization, the inference
results do not depend on the batch size, as it should be. Due to an
oversight, when we were obtaining the results for the paper, we left a few
batch normalization layers in place. We fixed that since, but to be
compatible with the checkpoints used for the paper, we needed to leave the
batch-norms there, hence the dependence on the batch size.
Regarding odometry, there may be a few explanations to that:

We used a more mature checkpoint for odometry, which seemed to converge
slower than depth prediction.
We used inference time correction for the intrinsics (even though the
result you're showing seems worse than even our uncorrected one).
There might be a difference in the way we stack the rotations and
translations together to obtain a trajectory - this is a bit tricky.

We can debug odometry together by us releasing the checkpoints used for
odometry and the respective inferred trajectories.
At this time I would like to ask you what you would like to prioritize -
getting the code and checkpoint for imagenet initialization, so that you
can reproduce the training, or getting the odometry evaluation right?
Please let me know which one you prefer and I'll start there. It will take
at least a few days, in any case.
Thank you again for your help in debugging this.
On Thu, Aug 22, 2019 at 7:08 PM juju<EMAIL_ADDRESS>wrote:

Hi @gariel-google https://github.com/gariel-google, thanks for the
work. The new zip files work for me. I have tested the checkpoint trained
on KITTI. The following is what I have:

depth result is different when inference with different batch_size:

abs_rel sq_rel rms log_rms a1 a2 a3
batch_size=1 0.1262 0.9462 5.2214 0.2086 0.8470 0.9475 0.9774
batch_size=16 0.1305 1.0186 5.3237 0.2136 0.8389 0.9430 0.9751
When batch_size=1, we have the same result, therefore we should be having
the same evaluating metrics. However, depth output doesn't have a
consistent result when batch_size changes, is it the same case for you?
where does the variation come from?

odometry result (ATE) when inference with batch_size=1:

seq_09 seq_10
5-point 0.0231 0.0195
3-point 0.0170 0.0149
The plotted trajectories:
seq_09
[image: dfvauthorseq09]
https://user-images.githubusercontent.com/18667188/63561747-a02ee800-c58d-11e9-8462-285ec61e84e7.png
seq_10
[image: dfvauthorseq10]
https://user-images.githubusercontent.com/18667188/63561754-aa50e680-c58d-11e9-9f53-762a795cba98.png
The odometry result looks quite bad for me. Do you have the same result?
Since eigen split training set has overlap with odometry sequence 09 and
10, the ATE should be better than what you have stated in the paper (0.0231
vs 0.012 and 0.0195 vs 0.010)?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google-research/google-research/issues/46?email_source=notifications&email_token=ADXKUNERJ6UPVBOJ67HWQILQF5BBBA5CNFSM4IMPQMMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD46446Y#issuecomment-524144251,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADXKUNHCTPAV6C2AHV2CWCLQF5BBBANCNFSM4IMPQMMA
.

Hi @gariel-google, Sorry for the late reply and thanks for your explanations.
Being able to reproduce the training would be a nice start for me. Thanks again for the work.
Also, Thank you in advance for releasing the odometry checkpoint and the respective inferred trajectories.

Hi @gariel-google , thanks for your checkpoint. But I test intrinsic inference with your kitti checkpoint, the intrinsic matrix is not right.
The input images is one pair kitti images like this:

We use top two images to infer intrinsic matrix:
[[119.80293    0.       702.5139  ]
[  0.        74.126114 -29.449604]
[  0.         0.         1.      ]]
But the ground truth intrinsic matrix:
[[241.67446312   0.<PHONE_NUMBER>1]
[  0.<PHONE_NUMBER>7  59.000832  ]
[  0.           0.           1.        ]]
Why the intrinsic matrix is not right?

@liyingliu Cool, so I am aiming for releasing the code for initializing
from an imagenet checkpoint and the respective checkpoint this week.
@buaafish Let's try to debug it together, let me start by asking you some
questions:

Do we have enough evidence to rule out an incorrect normalization of the
images (0-1 vs 0-255), or some other error in running the inference? For
example, were you able to reproduce the depth metrics? Were you able to
obtain reasonable trajectories on the KITTI odometry set?

How did you calculate the intrinsics at inference time? Did you run the
inference twice, swapping the order of images, and taking the average, like
here
https://github.com/google-research/google-research/blob/master/depth_from_video_in_the_wild/model.py#L258
?

As we write in the paper, there are two settings we used for learning
the intrinsics: One with a constraint that the intrinsics are the same
throughout the dataset (like in EuRoC), and the other is where we predict
the intrinsics independently from each pair of images. In the second case,
which is the one you are referring to, Eq. 3 and Fig. 9 show that the
intrinsics are only correct within the accuracy imposed by rotations, and
when there are no rotations, the error can be large. Have you tried to
create a plot similar to Fig. 9 or the example you're showing is the only
one you ran? Does that example have rotations?

On Mon, Aug 26, 2019 at 12:58 AM buaafish<EMAIL_ADDRESS>wrote:

Hi @gariel-google https://github.com/gariel-google , thanks for your
checkpoint. But I test intrinsic inference with your kitti checkpoint, the
intrinsic matrix is not right.
The input images is one pair kitti images like this:
[image: kitti2]
https://user-images.githubusercontent.com/6870525/63674721-b81a9c00-c819-11e9-90b4-17f32a181d82.png
We use top two images to infer intrinsic matrix:
[[119.80293 0. 702.5139 ]
[ 0. 74.126114 -29.449604]
[ 0. 0. 1. ]]
But the ground truth intrinsic matrix:
[[241.67446312 0.<PHONE_NUMBER>1]
[ 0.<PHONE_NUMBER>7 59.000832 ]
[ 0. 0. 1. ]]
Why the intrinsic matrix is not right?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google-research/google-research/issues/46?email_source=notifications&email_token=ADXKUNBFXD3JOZ3HJO72TFDQGOEKLA5CNFSM4IMPQMMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5DTFVQ#issuecomment-524759766,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADXKUNHNQN4NLZB4R73PVATQGOEKLANCNFSM4IMPQMMA
.

@gariel-google

Indeed I made incorrect normalization of the images(0-255). Then I add imagenet_norm operation to your test code.

The inference code like this :
  def _build_egomotion_test_graph(self):
    """Builds graph for inference of egomotion given two images."""
    with tf.variable_scope(tf.get_variable_scope(), reuse=tf.AUTO_REUSE):
      self._image1 = tf.placeholder(
          tf.float32, [self.batch_size, self.img_height, self.img_width, 3],
          name='image1')
      self._image2 = tf.placeholder(
          tf.float32, [self.batch_size, self.img_height, self.img_width, 3],
          name='image2')
      if self.imagenet_norm:
        self._image1 = (self._image1 - reader.IMAGENET_MEAN) / reader.IMAGENET_SD
        self._image2 = (self._image2 - reader.IMAGENET_MEAN) / reader.IMAGENET_SD

      # The "compute_loss" scope is needed for the checkpoint to load properly.
      with tf.name_scope('compute_loss'):
        rot, trans, _, mat = motion_prediction_net.motion_field_net(
            images=tf.concat([self._image1, self._image2], axis=-1))
        inv_rot, inv_trans, _, inv_mat = (
            motion_prediction_net.motion_field_net(
                images=tf.concat([self._image2, self._image1], axis=-1)))
      intrinsic_mat = 0.5 * (mat + inv_mat)
      rot = transform_utils.matrix_from_angles(rot)
      inv_rot = transform_utils.matrix_from_angles(inv_rot)
      trans = tf.squeeze(trans, axis=(1, 2))
      inv_trans = tf.squeeze(inv_trans, axis=(1, 2))

      # rot and inv_rot should be the inverses on of the other, but in reality
      # they slightly differ. Averaging rot and inv(inv_rot) gives a better
      # estimator for the rotation. Similarly, trans and rot*inv_trans should
      # be the negatives one of the other, so we average rot*inv_trans and trans
      # to get a better estimator. TODO(gariel): Check if there's an estimator
      # with less variance.
      self.rot = 0.5 * (tf.linalg.inv(inv_rot) + rot)
      self.trans = 0.5 * (-tf.squeeze(
          tf.matmul(self.rot, tf.expand_dims(inv_trans, -1)), axis=-1) + trans)
      self.inf_intrinsic_mat = intrinsic_mat

  def inference_egomotion(self, image1, image2, sess):
    return sess.run([self.rot, self.trans, self.inf_intrinsic_mat],
                    feed_dict={
                        self._image1: image1,
                        self._image2: image2
                    })

I modified your code :
      if self.imagenet_norm:
        self._image1 = (self._image1 - reader.IMAGENET_MEAN) / reader.IMAGENET_SD
        self._image2 = (self._image2 - reader.IMAGENET_MEAN) / reader.IMAGENET_SD

     intrinsic_mat = 0.5 * (mat + inv_mat)

Then I read RGB images to feed to image1 and image2.
The intrinsic matrix is not right too.
[[ 375.67065    0.      -579.6008 ]
[   0.        89.65622  -65.86415]
[   0.         0.         1.     ]]

@gariel-google Test code like this.
def main(_):
  seed = FLAGS.seed
  tf.set_random_seed(seed)
  np.random.seed(seed)
  random.seed(seed)

  if not gfile.Exists(FLAGS.checkpoint_dir):
    gfile.MakeDirs(FLAGS.checkpoint_dir)

  test_model = model.Model(
      boxify=FLAGS.boxify,
      data_dir=FLAGS.data_dir,
      file_extension=FLAGS.file_extension,
      is_training=False,
      foreground_dilation=FLAGS.foreground_dilation,
      learn_intrinsics=FLAGS.learn_intrinsics,
      learning_rate=FLAGS.learning_rate,
      reconstr_weight=FLAGS.reconstr_weight,
      smooth_weight=FLAGS.smooth_weight,
      ssim_weight=FLAGS.ssim_weight,
      translation_consistency_weight=FLAGS.translation_consistency_weight,
      rotation_consistency_weight=FLAGS.rotation_consistency_weight,
      batch_size=FLAGS.batch_size,
      img_height=FLAGS.img_height,
      img_width=FLAGS.img_width,
      weight_reg=FLAGS.weight_reg,
      depth_consistency_loss_weight=FLAGS.depth_consistency_loss_weight,
      queue_size=FLAGS.queue_size,
      input_file=FLAGS.input_file)

  _test(test_model, FLAGS.checkpoint_dir)

def readImages(path, subdir, name):
  filename = name+".png"
  filepath = os.path.join(path, subdir, filename)
  im = Image.open(filepath)
  im_array = np.array(im)
  img1 = im_array[:, 0:416, :]
  img2 = im_array[:, 416:832, :]
  return img1[np.newaxis, :, :, :], img2[np.newaxis, :, :, :]

def readMat(path, subdir, name):
  filename = name+"_cam.txt"
  filepath = os.path.join(path, subdir, filename)
  data_temp=[]
  with open(filepath) as fdata:
    line=fdata.readline()
    data_temp.append([float(i) for i in line.split(',')])
  return np.array(data_temp).reshape((3,3))

def readFileList(list_data_dir):
  with gfile.Open(list_data_dir) as f:
    frames = f.readlines()
    frames = [k.rstrip() for k in frames]
  subfolders = [x.split(' ')[0] for x in frames]
  frame_ids = [x.split(' ')[1] for x in frames]
  return subfolders, frame_ids

def _test(test_model, checkpoint_dir):
  checkpointpath = "./pretrained/cityscapes_kitti_learned_intrinsics/"

  saver = tf.train.import_meta_graph(checkpointpath+'model-1000977.meta')
  checkpoint = checkpointpath+"model-1000977"
  with tf.device('/cpu:0'):
    with tf.Session() as sess:
      sess.run(tf.local_variables_initializer())
      sess.run(tf.global_variables_initializer())
      logging.info('Loading checkpoint...')
      saver.restore(sess, checkpoint)
      print(reader.IMAGENET_MEAN)
      print(reader.IMAGENET_SD)
      logging.info('Reading data...')
      path = "./kitti/format_data"
      list_data_dir = "test.txt"
      subfolders, frame_ids = readFileList(list_data_dir)
      for (subdir, name) in zip(subfolders, frame_ids):
        img1, img2 = readImages(path, subdir, name)
        logging.info('Start testing...')
        ret = test_model.inference_egomotion(img1, img2,sess)
        print(ret[2])
        mat = readMat(path, subdir, name)
        print(mat)
        logging.info('End testing...')

if __name__ == '__main__':
  app.run(main)

@liyingliu We just added the code for initialization from Imagenet, as well
as some corrections in the hyperparameters for training. Unfortunately I
was unable to obtain clearance to release the specific ImageNet checkpoint
itself yet - sorry about that, things sometimes get more bureaucratic than
expected.
@buaafish Thanks for sharing your code, it's not easy for me though to spot
a bug if there is one. Is there a chance you have an answer for me whether
you were able to reproduce the depth inference metrics and/or whether the
trajectories look reasonable? The Intrinsic matrix is so much off that I
still suspect there is some sort of crude error somewhere.
My next steps are to release the checkpoints we used for calculating
odometry, with learned and given intrinsics, as well as the respective
odometry trajectories. Then I can try to add a small piece of code for
generating Fig. 9 in the paper for the intrinsics, which should hopefully
resolve the intrinsics issue.
Thank you all for your help debugging this, our goal is that everyone will
be able to reproduce out results.
On Mon, Aug 26, 2019 at 8:31 PM buaafish<EMAIL_ADDRESS>wrote:

@gariel-google https://github.com/gariel-google Test code like this.
def main(_):
seed = FLAGS.seed
tf.set_random_seed(seed)
np.random.seed(seed)
random.seed(seed)
if not gfile.Exists(FLAGS.checkpoint_dir):
gfile.MakeDirs(FLAGS.checkpoint_dir)
test_model = model.Model(
boxify=FLAGS.boxify,
data_dir=FLAGS.data_dir,
file_extension=FLAGS.file_extension,
is_training=False,
foreground_dilation=FLAGS.foreground_dilation,
learn_intrinsics=FLAGS.learn_intrinsics,
learning_rate=FLAGS.learning_rate,
reconstr_weight=FLAGS.reconstr_weight,
smooth_weight=FLAGS.smooth_weight,
ssim_weight=FLAGS.ssim_weight,
translation_consistency_weight=FLAGS.translation_consistency_weight,
rotation_consistency_weight=FLAGS.rotation_consistency_weight,
batch_size=FLAGS.batch_size,
img_height=FLAGS.img_height,
img_width=FLAGS.img_width,
weight_reg=FLAGS.weight_reg,
depth_consistency_loss_weight=FLAGS.depth_consistency_loss_weight,
queue_size=FLAGS.queue_size,
input_file=FLAGS.input_file)
_test(test_model, FLAGS.checkpoint_dir)
def readImages(path, subdir, name):
filename = name+".png"
filepath = os.path.join(path, subdir, filename)
im = Image.open(filepath)
im_array = np.array(im)
img1 = im_array[:, 0:416, :]
img2 = im_array[:, 416:832, :]
return img1[np.newaxis, :, :, :], img2[np.newaxis, :, :, :]
def readMat(path, subdir, name):
filename = name+"_cam.txt"
filepath = os.path.join(path, subdir, filename)
data_temp=[]
with open(filepath) as fdata:
line=fdata.readline()
data_temp.append([float(i) for i in line.split(',')])
return np.array(data_temp).reshape((3,3))
def readFileList(list_data_dir):
with gfile.Open(list_data_dir) as f:
frames = f.readlines()
frames = [k.rstrip() for k in frames]
subfolders = [x.split(' ')[0] for x in frames]
frame_ids = [x.split(' ')[1] for x in frames]
return subfolders, frame_ids
def _test(test_model, checkpoint_dir):
checkpointpath = "./pretrained/cityscapes_kitti_learned_intrinsics/"
saver = tf.train.import_meta_graph(checkpointpath+'model-1000977.meta')
checkpoint = checkpointpath+"model-1000977"
with tf.device('/cpu:0'):
with tf.Session() as sess:
sess.run(tf.local_variables_initializer())
sess.run(tf.global_variables_initializer())
logging.info('Loading checkpoint...')
saver.restore(sess, checkpoint)
print(reader.IMAGENET_MEAN)
print(reader.IMAGENET_SD)
logging.info('Reading data...')
path = "./kitti/format_data"
list_data_dir = "test.txt"
subfolders, frame_ids = readFileList(list_data_dir)
for (subdir, name) in zip(subfolders, frame_ids):
img1, img2 = readImages(path, subdir, name)
logging.info('Start testing...')
ret = test_model.inference_egomotion(img1, img2,sess)
print(ret[2])
mat = readMat(path, subdir, name)
print(mat)
logging.info('End testing...')
if name == 'main':
app.run(main)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google-research/google-research/issues/46?email_source=notifications&email_token=ADXKUNHNC2IDOE7JDYVIPN3QGSN25A5CNFSM4IMPQMMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5GLS5Q#issuecomment-525121910,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADXKUNHON74T2IYYN5MCBTTQGSN25ANCNFSM4IMPQMMA
.

@gariel-google Understand and thanks! Looking forward to exchanging the odometry results.

@liyingliu we just released the odometry results, and code for generating
trajectories form checkpoints.
@buaafish intrinsics is coming next.
On Sat, Aug 31, 2019 at 4:18 AM juju<EMAIL_ADDRESS>wrote:

@gariel-google https://github.com/gariel-google Understand and thanks!
Looking forward to exchanging the odometry results.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google-research/google-research/issues/46?email_source=notifications&email_token=ADXKUNCLGJXYI4KIP2SS55TQHJHRBA5CNFSM4IMPQMMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5TKYBY#issuecomment-526822407,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADXKUNGKRK7EEGWIDAEODYDQHJHRBANCNFSM4IMPQMMA
.

@cognitiveRobot  If your question is to restore and train the checkpoint that the author provided. Then, you could try to add a file named "checkpoint" in your checkpoint folder (the folder contains .index, .meta .data-xxxx). The content in "checkpoint" file can be the following:
model_checkpoint_path:"path_to_kitti_learned_intrinsics/model-248900"

@cognitiveRobot the pretrained checkpoint from resnet18 that we used was
taken from here http://cognitiveRobot. We cannot release it here because
it was taken from somewhere else. Sorry about that...
On Wed, Nov 13, 2019 at 12:40 AM juju<EMAIL_ADDRESS>wrote:

@cognitiveRobot https://github.com/cognitiveRobot If your question is
to restore and train the checkpoint that the author provided. Then, you
could try to add a file named "checkpoint" in your checkpoint folder (the
folder contains .index, .meta .data-xxxx). The content in "checkpoint" file
can be the following:
model_checkpoint_path:"path_to_kitti_learned_intrinsics/model-248900"
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google-research/google-research/issues/46?email_source=notifications&email_token=ADXKUND5PBGYDGCEQCQEC6LQTO4QBA5CNFSM4IMPQMMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOED5KOAQ#issuecomment-553297666,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADXKUNEFXKA322UKAGSW7WDQTO4QBANCNFSM4IMPQMMA
.

Sorry, here's the link to where we took the pretrained checkpoint from:
https://pytorch.org/docs/stable/torchvision/models.html
On Wed, Nov 13, 2019 at 3:00 PM Ariel Gordon<EMAIL_ADDRESS>wrote:

@cognitiveRobot the pretrained checkpoint from resnet18 that we used was
taken from here http://cognitiveRobot. We cannot release it here
because it was taken from somewhere else. Sorry about that...
On Wed, Nov 13, 2019 at 12:40 AM juju<EMAIL_ADDRESS>wrote:

@cognitiveRobot https://github.com/cognitiveRobot If your question is
to restore and train the checkpoint that the author provided. Then, you
could try to add a file named "checkpoint" in your checkpoint folder (the
folder contains .index, .meta .data-xxxx). The content in "checkpoint" file
can be the following:
model_checkpoint_path:"path_to_kitti_learned_intrinsics/model-248900"
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google-research/google-research/issues/46?email_source=notifications&email_token=ADXKUND5PBGYDGCEQCQEC6LQTO4QBA5CNFSM4IMPQMMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOED5KOAQ#issuecomment-553297666,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADXKUNEFXKA322UKAGSW7WDQTO4QBANCNFSM4IMPQMMA
.

Regarding the batch size, we tested at 1. If batch-normalization is replaced everywhere by randomized layer normalization, the inference results do not depend on the batch size, as it should be. Due to an oversight, when we were obtaining the results for the paper, we left a few batch normalization layers in place. We fixed that since, but to be compatible with the checkpoints used for the paper, we needed to leave the batch-norms there, hence the dependence on the batch size.

@gariel-google  Hi, I realize that for batch-normalization, in the code is_train is set to False during inference. Therefore, in this case, the depth result shouldn't depend on the batch size.
Another experiment I did is: I trained a model with all the batch-normalization replaced by randomized layer normalization. However, when I evaluated this checkpoint, the depth result is still inconsistent with different batch size.  It seems randomized layer normalization is the reason causing inconsistent results? Can you help to explain this?  Thanks.

@liyingliu Yes, you're right. When is_train is set to false, the inference
results do not depend on the batch size. In the struct2depth code,
upon which our code is based, is_train is set to True at inference, and
with batch normalization the results will depend on batch size.
I don't know why the results would depend on batch size with randomized
later normalization, however, we did discover that we have a bug in the
code, and not all batch normalization layers were replaced by
randomized layer normalization. For example, here
https://github.com/google-research/google-research/blob/master/depth_from_video_in_the_wild/depth_prediction_net.py#L83
batch-norm
is still there. You made a good argument that batch-norm in inference mode
should also be insensitive to the batch size at inference time, so I"m not
sure I know the answer, but maybe it can give you a lead?
I believe that the reason for what you're seeing is a bug that we have in
the code.
This is true, if is_train is set to False at inference, the result should
not depend on the batch size
On Thu, Nov 21, 2019 at 11:15 AM juju<EMAIL_ADDRESS>wrote:

Regarding the batch size, we tested at 1. If batch-normalization is
replaced everywhere by randomized layer normalization, the inference
results do not depend on the batch size, as it should be. Due to an
oversight, when we were obtaining the results for the paper, we left a few
batch normalization layers in place. We fixed that since, but to be
compatible with the checkpoints used for the paper, we needed to leave the
batch-norms there, hence the dependence on the batch size.
@gariel-google https://github.com/gariel-google Hi, I realize that for
batch-normalization, in the code is_train is set to False during
inference. Therefore, in this case, the depth result shouldn't depend on
the batch size.
Another experiment I did is: I trained a model with all the
batch-normalization replaced by randomized layer normalization. However,
when I evaluated this checkpoint, the depth result is still inconsistent
with different batch size. It seems randomized layer normalization is the
reason causing inconsistent results? Can you help to explain this? Thanks.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google-research/google-research/issues/46?email_source=notifications&email_token=ADXKUNHO2IBVECXQEC4DFXLQU3M4DA5CNFSM4IMPQMMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEE3KYZQ#issuecomment-557231206,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADXKUNHIH53PKKMBKSO4WITQU3M4DANCNFSM4IMPQMMA
.

Any news for model used for the paper that can predict intrinsic parameter correctly?

@hyeokhyen Figure 9 estimates shows the accuracy at which intrinsics are
estimated by our model. The limitation is explained in the paper.
Alternatively, we can impose constancy of the intrinsics over the entire
sequence, as we did in EuRoC, and to get very accurate results. This is as
accurate as we can go.
On Thu, Feb 20, 2020 at 4:51 PM Hyeokhyen Alan Kwon <
<EMAIL_ADDRESS>wrote:

Any news for model used for the paper that can predict intrinsic parameter
correctly?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google-research/google-research/issues/46?email_source=notifications&email_token=ADXKUNBNDSNTCNUXYVKUKILRD4QQRA5CNFSM4IMPQMMKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMRDPSI#issuecomment-589445065,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADXKUNH4THJHO6WZWASC2DTRD4QQRANCNFSM4IMPQMMA
.

@liyingliu
sorry to bother you, but I have some questions I'd like to ask you. I tried to infer the depth map using the existing checkpoint cityscape_kitti, but the depth value I read directly from the '.npy' file was far from real depth, whether it was my own images or images in cityscape. Did I do something wrong? Or further operations are required to obtain true depth values. Thank you very much.
'inference.py' from (https://github.com/tensorflow/models/blob/master/research/struct2depth/inference.py)
img_width and img_height are defaults (416,128)

Hi, there is an unknown scale between the predicted depth value from the network and the real depth value. You need to scale the predicted depth by such a scale factor to have true depth value. You could use the median of your prediction divided by the median of ground truth to be the scale.

The predicted depth is up to an unknown scale factor. When evaluating, this
factor is found by normalizing the medians of the predicted and inferred
depth this is standard in this line of publications). Are you
observing strong discrepancies even beyond that global factor?
On Sun, Apr 12, 2020 at 11:26 PM stephen<EMAIL_ADDRESS>wrote:

@liyingliu https://github.com/liyingliu
sorry to bother you, but I have some questions I'd like to ask you. I
tried to infer the depth map using the existing checkpoint cityscape_kitti,
but the depth value I read directly from the '.npy' file was far from real
depth, whether it was my own images or images in cityscape. Did I do
something wrong? Or further operations are required to obtain true depth
values. Thank you very much.
'inference.py' from (
https://github.com/tensorflow/models/blob/master/research/struct2depth/inference.py
)
img_width and img_height are defaults (416,128)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google-research/google-research/issues/46#issuecomment-612771053,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADXKUNC23ORNPVHH5B4GUFTRMKWBLANCNFSM4IMPQMMA
.

@liyingliu @gariel-google
Thank you very much for your two patient guidance, I will try again based on your suggestions.
If I encounter other problems, I will consult you again.
@gariel-google
By the way, I don't quite understand what you mean by "observing strong discrepancies even beyond that global factor". Do you mean whether I found  the scale factor does not apply to the entire image ?
thank you both again.

@gariel-google
Hello, I'm new to this line.
I tried to run trajectory_inference.py. But it seems that the output is egomotion, and I have no idea about how to generate the trajectory for evaluation.
Will you release some examples about how to generate the trajectory from egomotion? Or maybe I missed some examples in this repo?

@StephenStorm by "observing strong discrepancies even beyond that global factor" I mean: If you multiply the predicted depth by a factor such that its median matches the median groundtruth depth, do you still see significant discrepancies?
@player1321 In the KITTI format, the first 3 columns are the (x, y, z) position of the car if I'm not mistaken. This code  generates the inferred (x, y, z)-s of the trajectory.

@gariel-google Thanks a lot for your patient guidance.
I tested your released model. It seems that the model trained for odometry works well on trajectory inference but not so well on depth estimation, and the model trained on cityscapes&kitti works well on depth estimation but not so well on trajectory inference.
Is this caused by the gap between cityscapes and kitti? Or are there some tricks for improving depth estimation and trajectory inference respectively?

@player1321 I looked up the checkpoint we used for KITTI odometry (with given intrinsics), and its depth prediction metric is 0.1321, which is indeed worse than the KITTI-only depth error that we report in the paper for given intrinsics (0.129). Is that your concern? We did observe that for odometry results tend to improve the longer we train, whereas for depth they tend to become slightly worse and more noise beyond some point. We did not try to evaluate the cityscapes + KITTI checkpoints for odometry, and I don't know how it would perform.
Would you like to share your numbers on both evaluations?

@gariel-google Actually I have trouble generating the numbers. I'm using this evaluation tool, and the visualization looks fine but the numbers are very different from yours.
Here is the visualization result:
Seq-09

Seq-10

And here are the numbers:

KITTI
Sequence: 	 9
Trans. err. (%): 	 6.572
Rot. err. (deg/100m): 	 2.254
ATE (m): 	 40.574
RPE (m): 	 0.068
RPE (deg): 	 0.101

Sequence: 	 10
Trans. err. (%): 	 13.659
Rot. err. (deg/100m): 	 3.904
ATE (m): 	 145.783
RPE (m): 	 0.100
RPE (deg): 	 0.097

KITTI + cityscapes
Sequence: 	 9
Trans. err. (%): 	 7.727
Rot. err. (deg/100m): 	 2.220
ATE (m): 	 45.602
RPE (m): 	 0.088
RPE (deg): 	 0.093

Sequence: 	 10
Trans. err. (%): 	 13.032
Rot. err. (deg/100m): 	 2.529
ATE (m): 	 88.142
RPE (m): 	 0.104
RPE (deg): 	 0.082
It seems that the definition of the ATE is not the same as yours. Would you share some evaluation tools? Or any reference about the definitions can be recommended?

@gariel-google Thanks for sharing the code and helping us reproduce the results. I'm able to reproduce similar figures to the paper using the odometry checkpoints, but the scale seems to be wrong. Is the egomotion network supposed to output positions with the real-world scale immediately, or is it assumed that we're performing a scaling as postprocessing? If yes, which type of scaling is used in the paper?

Also, I realized that the given intrinsics weights link with kitti odometry is wrong: it references references the cityscape model from the depth table right above: https://www.googleapis.com/download/storage/v1/b/gresearch/o/depth_from_video_in_the_wild%2Fcheckpoints%2Fcityscapes_learned_intrinsics.zip?generation=1566493765410932&alt=media

@player1321 The definition of ATE we used follows Zhou et al. Our ATE eval is based on theirs, which is given here: https://github.com/tinghuiz/SfMLearner/tree/master/kitti_eval. The numbers typically are in the 10^-2-s. Your's are in meters, and are large-ish, so indeed it's probably the same definition.
Regarding the translation error, we didn't check if tor the city+KITTI checkpoint, and while your numbers are different, it seems that your numbers are not far from ours, assuming you tested checkpoints with learned intrinsics (right?).
@frobinet The odometry predictions are scale-less, just like the depth predictions. We normalized the entire trajectory by its length. That is, we scaled the predicted trajectory uniformly until its total length was identical to the GT length.

@frobinet I'll have a look at the model links and get back to you, thanks for pointing this out.

@gariel-google Any news about releasing the weights for the given intrinsics odometry model? Thanks for helping with this!

@gariel-google Thanks for your guidance!
You are right, I'm using checkpoints with learned intrinsics.
I have a new question now:

In your paper, you show that the network can carve the silhouette of the people out of a rough mask, but in my test, it only happens in a few cases, and in most of the cases the output is like the region of the car, that is, the network cannot catch the residual translation correctly.
Does this happen in your test?

@gariel-google Thanks for sharing the code and helping us.
Even though I am using your pretrained model(learned intrinsic), I can not reproduce the results of egomotion.
I got 0.0259@seq-09  and  0.0210@seq-10, which are much worse than values on your paper.
Can you provide trajectories with poses and/or code to reproduce the same value?
Shared trajecotries only include XYZ positions. If you can add poses, it is very helpful to reproduce the results.

Sorry for the delayed response.
@NHirose we released the trajectories here - please see the table below the title, under the links "trajectory".
@player1321 We didn't evaluate quantitatively the prediction of residual motion. Qualitatively it looks good in most cases - I know it sounds hand-wavy, but unfortunately there is no number that I can quote to support this quantitatively.

@gariel-google Thank you for your replying. However your released trajectory file only includes XYZ position. I additionally need roll pitch yaw angles to reproduce the values on your paper.
Or can you provide the evaluation file to reproduce the value about ego motion on your paper? That can help me to find differences!!

Thank you for your work and code.
I'm having some problems during inference of KITTI egomotion
I modified strtuct2depth inference to infer 2 images:
and I get this error: ValueError: could not broadcast input array from shape (3,3) into shape (1,3)
est_egomotion = np.squeeze(inference_model.inference_egomotion(input_image_stack[:,:,:,:3],
   input_image_stack[:,:,:,3:], sess))
  File "<__array_function__ internals>", line 6, in squeeze
  File "/usr/local/lib/python3.5/dist-packages/numpy/core/fromnumeric.py", line 1479, in squeeze
    return _wrapit(a, 'squeeze', axis=axis)
  File "/usr/local/lib/python3.5/dist-packages/numpy/core/fromnumeric.py", line 47, in _wrapit
    result = getattr(asarray(obj), method)(*args, **kwds)
  File "/usr/local/lib/python3.5/dist-packages/numpy/core/_asarray.py", line 85, in asarray
    return array(a, dtype, copy=False, order=order)
ValueError: could not broadcast input array from shape (3,3) into shape (1,3)

input_image_stack.shape = [1, 128, 416 ,6]
Please help

Nevermind - the problem was the squeeze() around the call

Thank you for your work and code.
I'm having some problems during inference of KITTI egomotion
I modified strtuct2depth inference to infer 2 images:
and I get this error: ValueError: could not broadcast input array from shape (3,3) into shape (1,3)
est_egomotion = np.squeeze(inference_model.inference_egomotion(input_image_stack[:,:,:,:3],
   input_image_stack[:,:,:,3:], sess))
  File "<__array_function__ internals>", line 6, in squeeze
  File "/usr/local/lib/python3.5/dist-packages/numpy/core/fromnumeric.py", line 1479, in squeeze
    return _wrapit(a, 'squeeze', axis=axis)
  File "/usr/local/lib/python3.5/dist-packages/numpy/core/fromnumeric.py", line 47, in _wrapit
    result = getattr(asarray(obj), method)(*args, **kwds)
  File "/usr/local/lib/python3.5/dist-packages/numpy/core/_asarray.py", line 85, in asarray
    return array(a, dtype, copy=False, order=order)
ValueError: could not broadcast input array from shape (3,3) into shape (1,3)

input_image_stack.shape = [1, 128, 416 ,6]
Please help

Hi,
I'm getting an error when loading EuroC MAV checkpoint for training. [depth_from_video_in_the_wild_euroc_ckpt_MachineHallAll]
Key MotionFieldNet/compute_loss/MotionFieldNet_2/Conv1/Relu/MotionBottleneck/weights not found in checkpoint
[[node save/RestoreV2 (defined at /depth_from_video_in_the_wild/model.py:117) ]]
@gariel-google

Did you run the code as is or with modifications? I am asking because a few
months ago, when I uploaded the checkpoint, I did verify that it loads, so
I am trying to track down the reason for the change in behavior.
On Mon, Aug 3, 2020 at 1:29 AM adizhol<EMAIL_ADDRESS>wrote:

Hi,
I'm getting an error when loading EuroC MAV checkpoint for training.
[depth_from_video_in_the_wild_euroc_ckpt_MachineHallAll]
Key
MotionFieldNet/compute_loss/MotionFieldNet_2/Conv1/Relu/MotionBottleneck/weights
not found in checkpoint
[[node save/RestoreV2 (defined at
/depth_from_video_in_the_wild/model.py:117) ]]
@gariel-google https://github.com/gariel-google
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google-research/google-research/issues/46#issuecomment-667886281,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADXKUNEGVWFILC3KXWQRVF3R6ZYOZANCNFSM4IMPQMMA
.

No modifications.
The cityscape and kitti snapshots load normally.
Thank you
On Mon, Aug 3, 2020, 19:46 Ariel Gordon<EMAIL_ADDRESS>wrote:

Did you run the code as is or with modifications? I am asking because a few
months ago, when I uploaded the checkpoint, I did verify that it loads, so
I am trying to track down the reason for the change in behavior.
On Mon, Aug 3, 2020 at 1:29 AM adizhol<EMAIL_ADDRESS>wrote:

Hi,
I'm getting an error when loading EuroC MAV checkpoint for training.
[depth_from_video_in_the_wild_euroc_ckpt_MachineHallAll]
Key

MotionFieldNet/compute_loss/MotionFieldNet_2/Conv1/Relu/MotionBottleneck/weights

not found in checkpoint
[[node save/RestoreV2 (defined at
/depth_from_video_in_the_wild/model.py:117) ]]
@gariel-google https://github.com/gariel-google
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<
https://github.com/google-research/google-research/issues/46#issuecomment-667886281
,
or unsubscribe
<
https://github.com/notifications/unsubscribe-auth/ADXKUNEGVWFILC3KXWQRVF3R6ZYOZANCNFSM4IMPQMMA
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/google-research/google-research/issues/46#issuecomment-668125620,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADD2556GIDE6PEKMOL3MX2LR63SVBANCNFSM4IMPQMMA
.

@gariel-google @adizhol  I am facing the same issue, i.e., the checkpoints of " cityscape and kitti" works well with the model.py.
But the euro one complains.
sess = tf.Session()
saver = tf.train.Saver()
saver.restore(sess, 'euroc_ckpt_ViconRoom1-01/model-2091000')

NotFoundError (see above for traceback): Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:

Key MotionFieldNet/CameraIntrinsics/foci/biases not found in checkpoint
	 [[node save_3/RestoreV2 (defined at <ipython-input-13-a5df47d040a7>:2) ]]

Are there any updates about this?
Thanks

Thanks for pointing this out. This seems to be a bug on our side, then. I will look into it, but it may take some time till I can get to it and debug. Sorry about that :-)

After training on custom data, I'm getting different depth when training and when doing inference (on the same images).
When training, the depth is smooth and when inferencing it's very rough and inaccurate.
I'm using the inference code from struct2depth inference.py.
Has anyone experienced something like this?

Hey there, I'm still facing this:
``
NotFoundError (see above for traceback): Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:
Key MotionFieldNet/CameraIntrinsics/foci/biases not found in checkpoint
[[node save_3/RestoreV2 (defined at :2) ]]
``
Currently, I'm using the latest code version:
        vars_to_restore = [             v for v in tf.trainable_variables()             if v.op.name.startswith(DEPTH_SCOPE + '/conv')         ]         vars_to_restore = {             v.op.name[len(DEPTH_SCOPE) + 1:]: v for v in vars_to_restore         }
Did somebody face the same issue?

@VladimirYugay @adizhol @mathmax12
Run tf.reset_default_graph() to cope with the error below

NotFoundError (see above for traceback): Restoring from checkpoint failed. This is most likely due to a Variable name or other graph key that is missing from the checkpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:
Key MotionFieldNet/CameraIntrinsics/foci/biases not found in checkpoint
[[node save_3/RestoreV2 (defined at :2) ]]

@jaysonph thanks for your reply. Didn't work for me.
However, this one did:
os.environ['TF_FORCE_GPU_ALLOW_GROWTH'] = 'true'

@jaysonph thanks for the reply. Resetting the graph doesn't really help.

@jaysonph thanks for the reply. Resetting the graph doesn't really help.


Create who command

create the 'who' command in order to display all of the currently logged in characters and where they are located.

The who command now works but there were a few initial issues. The mud.send_message for the menu was mud.sned_message. Also the function was receiving 4 parameters but was only configured to receive 3 and the for loop needed two variables instead of the one. We now just need to further refine the printout of this function to add things to the output as well as not include the current user.


Improvements from my branch

Pursuant to our email exchange a week ago, I've sieved out the important parts for merge here:

The NN model is modular, and a different model can either be selected from a built-in set or loaded from an external file.

Many new models are included, trained in a variety of situations.

Maximum attenuation is parameterizable. How useful this feature is is questionable :)

Sample rate is parameterizable. This only affects the mapping of FFT bins to bands, so it's quite general, though it does not affect the input rate, so samples are still expected 480 at a time.

A bug in the !SMOOTH_BANDS mode is fixed

Not included are my broken "fix" to interpolation, the rename, or my various changes to the auto* scripts.

OK, so I still haven't had time to do a full review of your patches, but I guess I can at least give you some feedback:

Regarding the ability to have multiple models, I think the patches have a few issues. I think it can be a bit confusing for the user to have to pick a model. Also, from previous experience, having a global object like "const struct RNNModel model_orig" exposed in the API itself can cause all kinds of problems. I can't remember the details, only that I've been there before and decided it was best to avoid it. At least for now, my preference would be for a compile-time model selection. I think ultimately, it may be possible to have one model that works for everything. That being said, I like your modularization work as it makes it easier to experiment.

I don't think the sample rate patch works, or at least it's not the right way to do it. The problem is that even if you change the band boundaries to match the new sampling rate, the time difference between windows would still change if you keep the window size the same and change the sampling rate. That would cause a mismatch with the training conditions of the model. The only way to support different rates would be to change the frame size but not the bands. For example, for 32 kHz, the frame size would have to be 320 (640-sample FFT). This would at least make it possible to support the important rates like 8, 16, and 32 kHz. I guess rates like 44.1 kHz could always be achieved by just pretending the audio is 48 kHz since a 10% difference should be small enough.

As far as I can tell, the !SMOOTH_BANDS fix is wrong. Right now, the code won't work for !SMOOTH_BANDS because it would need a model trained for that (with fewer bands), but trying to get the code to run for !SMOOTH_BANDS will just give wrong results.

I'll try to have a closer look at the details of the other patches, but at least you have some comments already.

Apologies for the absurdly slow response here, I have other commitments (as of course do you!)
(1) I think the fundamental problem is that regardless of whether it's possible to create a "perfect" model, it will never be all-covering, because people disagree on what is noise. The reason I found the default one to be unsuitable is that what I was recording was myself playing an instrument in my apartment, and the things I consider to be noise in that environment are very different than the things I'd consider to be noise in a crowded coffeeshop. It's not like every possible element in-between is needed, but it's unlikely that a model could be trained in such a way that it worked for both, not because of any limitations in the technique, but because they are actually different problems, albeit facets of the same one. Perhaps a compromise would be to have only the options of the default, or to load from a file, and to include—or not—other models only as files?
(2) Yeah, with your description, I can see that it's not right. We're too far out of my expertise. I agree that just demanding the right framerate in the first place would be a better option.
(3) The current version doesn't compile :). Literally all this change was was to make it compile. Perhaps it should just be nixed if it's neither working nor going to be fixed.

(1) I agree that we may indeed need multiple models. I still think a default build (./configure; make) should only include one model. From there, there's many possible ways of adding models. It could be a compile option --with-model=XYZ, it could be loading a file, or both.
(2) Yeah, I think attempting to handle other rates is going to be complicated. If it's really a useful feature, then the best would be to just integrate a resampler and so it transparently.
(3) Yeah, the best is to just remove that option. I'm wasn't planning on bringing it back.

In an attempt to keep history clean, I recreated the branch with the irrelevant revisions stripped out, so apologies for the appearance of throwing a bunch of commits on at the same time. They're the same, with a couple bugs fixed, and the following changes:
(1) Sample rate adjustment is nixed.
(2) The "fix" to SMOOTH_BANDS has been replaced with the elimination of that non-feature to clean up the code.
(3) Multiple models are supported exclusively through reading model files.
(4) The builtin model is used only by specifying NULL as the model; the "model_orig" variable is not public. (Note that this option was always available)
Thus, this PR now consists of the following changes:
(1) The RNN model is made modularizable and loadable from a file, with the existing model being loaded if none is otherwise provided.
(2) Warnings are fixed, headers made more modular, and other such minor cleanups.
(3) Maximum attenuation is made parameterizable. I retain that this feature itself is of arguable utility, but it more importantly brings with it a general configuration API that should scale to future changes.

Note that in my case where the noise is very low (just cleaning up a voice over recording in a not so soundproof environment), I had to add a path without RRNoise and automate the mix because RRNoise was far too eager on my "fois" which means "times" in French. Roughly one over two sounded "...ois". It seems that the "fffff" sound is close enough to RRNoise's representation of noise that it gets removed.
So having the possibility to use other models might be useful here.
While doing that, I discovered that the phase behaviour of RRNoise is funky, so it cannot be mixed 50% wet. Using a maximum attenuation parameter that can be changed between every processes would enable having an automatable "dry/wet" control (with suitable smoothing of the control to avoid zipping of course, but that responsibility would rely on the wrapper DAW plugin like https://github.com/lucianodato/speech-denoiser/ that I use).

OK, I reviewed the entire stack and after some rebasing (reordering+fusing some patches), I merged all but three changes. The only remaining issue is related to the max attenuation. I'm OK with the general idea, but have two issues with the implementation:

The code that handles it is really convoluted for apparently no good reason. As far as I can tell, you could get almost the same behaviour by just doing a gain = max(gain, attenuation_floor).
Expressing the attenuation as a gain is really counter-intuitive. I think the attenuation should be specified as a (positive) value in dB. Ideally an int value would probably be best in case there's ever a fixed-point implementation of this lib.

So the three patches I didn't merge are

Added parameterizable maximum attenuation.
Let's scale our maximum attenuation correctly...
Added input of model file to the demo.
(the last one because I can't apply it without the first one)

So, should this be reworked and rebased to only include the remaining fixes or be closed?

So, should this be reworked and rebased to only include the remaining fixes or be closed?


build: switch dev app system config to es5

Currently the system-config.js is loaded in directly and it's written in ES6. This breaks loading the dev app in some older browsers that don't support it. These changes rewrite the file to ES5.

Ah, good catch, I totally missed those. They should be fixed now.


Hackathon-2021大赛相关作品的百度网盘提取码全部错误

Hackathon-2021大赛相关作品的百度网盘提取码全部错误，望能够更改修正，感谢大赛工作人员！

Fixed. Thank you for finding the error, tell me if the problem remained :)

Fixed. Thank you for finding the error, tell me if the problem remained :)

Thank you for your quick response! But all the file in the Baidu NetDisk Link is only the Certificate? What about the other reference data?
I know it's a little bit hard to clear up these old files, thanks in advance again!

Sorry I don't know what you mean "other reference data" includes. Orz
I added the names of each original models' of player's repository in the table, is this a little bit helpful?


🐎 ci(Docker): Dockerization

Provide a dockerfile for the storybook.

Nice! It may be useful to add the npm run storybook for the composed storybook.

They are provided in docker_entry.sh
I used this command in the docker CMD instruction, but it did not work. the app was not accessible via ports 6006 and 6007. So I ran both of them separately in the entry script.

I mean in addition (see comments).

I mean in addition (see comments).

Ah ok, thanks. I got it wrong. I though when you run the compose mode you automatically have the html and react on ports 6006 and 6007.

:tada: This PR is included in version 2.0.5 :tada:
The release is available on:

GitHub release
npm package (@latest dist-tag)

Your semantic-release bot :package::rocket:


performance about nuscenes

hi thanks for your great work. I am wondering about the validation performance of nuscenes dataset. or can your provide me the training logs of nuscenes.

The performance on nuScenes validation set  of Cylinder3D is 76.1 and the performance on testset is shown in the leaderboard


Target Date Field added to withdrawals forms.

-Field added in both createForm.tsx and editForm.tsx. -Added targetDate in gql/withdrawals.d.ts types.
-Created SelectDate.tsx file for the custom element.

Closes #1558 Withdrawal: New field
Motivation and context

Screenshots:

Before
After

Paste screenshot
Paste screenshot

Testing
Steps to test
Affected urls

Environment
New environment variables:

[ ] NEW_ENV_VAR: env var details

New or updated dependencies:

Dependency name
Previous version
Updated version
Details

dependency/name
v1.0.0
v2.0.0

Good update!
Are the changes in .yarn/releases/yarn-3.6.2.cjs necessary?


Add @karoun's Typescript typings!

Hey hey!

This is hopefully a good start toward #24, and implements the typings contributed by @karoun.

@karoun, it would be great to get those JSDoc comments in there when you have time.

Perfect! Will get to this tomorrow evening!


Generate fuzzing harnesses inside impl-blocks

Looks like we need two separate macros for it: one for the impl-block itself and one for the function to be fuzzed.

Basic method processing implemented in 112bd7c2a3f97227d48bc9ab4d51bb394f28a3ea
Now we need:

[ ] Tests
[ ] Process "generator" methods (which does not have self in args)
[ ] Automatically skip private and unsafe methods

Skipping private and unsafe methods will be implemented via proper error handing, so it is moved to https://github.com/rust-fuzz/auto-fuzz-test/issues/20