[Webpack 5] Reference error: webpack_require is not defined

Bug report

What is the current behavior?
I would like to upgrade our repo to webpack5. We have custom requires and custom dependency and templates. In webpack4, the dependency could be added without issues. Now though, the compiled code doesnt inlcude the webpack_require inside the function argument, so the webpack_require which we replaced doesnt work anymore.
/***/ 9126: /***/ ((module) => {
This is how the module is described, this is how it should be:
/***/ 9125: /***/ ((module, __unused_webpack_exports, __webpack_require__) => {
How should we adapt the code of the dependency template so that this webpack_require is added to the module arguments?
If the current behavior is a bug, please provide the steps to reproduce.
This is company code, but maybe one could help that knows how these dependencies work and advice me how to adapt it manually :)

What is the expected behavior?
Modules with custom requires should have the webpack_require working, so that the code can access the webpack_require's like below.
/***/ 9125: /***/ ((module, __unused_webpack_exports, __webpack_require__) => {
`

Other relevant information:
webpack version:  5.3.2
Node.js version: 12.17.10
Operating System: MacOS
Additional tools:

It is breaking change, you need fix compatibility with webpack@5

How should we adapt the code of the dependency template so that this webpack_require is added to the module arguments?

Please provide code how you do it

@evilebottnawi thanks for the swift reply!
It is company code I cannot show it. But could i orientate myself at lib/dependencies/AMDRequireDependency.js at the AMDRequireDependencyTemplate where
if (dep.functionRange && !dep.arrayRange) { 			const startBlock = ``${promise}.then((``; 			const endBlock = ``).bind(exports, __webpack_require__, exports, module)).catch(${RuntimeGlobals.uncaughtErrorHandler})``; 			runtimeRequirements.add(RuntimeGlobals.uncaughtErrorHandler);
this has been added? Here they bind webpack_require to the module? Or is that not the right place to find that? Or is there some simple require dependency that I could orientate myself after?
The template is something like this:
`
apply(dep, source, outputOptions, requestShortener) {
if (!dep.range) {
return
}
    var actualContent
    var moduleDependency = outputOptions.moduleGraph.getModule(dep)
    actualContent = '__webpack_require__(' + JSON.stringify(moduleDependency.id)
    var lineDifference = dep.loc.end.line - dep.loc.start.line
    if (lineDifference > 0) {
        for (var i = 0; i < lineDifference; i++) {
            actualContent += '\n'
        }
    }
    actualContent += ')'
    source.replace(dep.range[0], dep.range[1] - 1, actualContent)
}

`

Can you refactor the issue, it is not readable

It is company code I cannot show it.

Unfortunately, most likely you will unanswered in this case, I do not ask to show all the code, I ask to show an example
You can look at code in https://github.com/webpack/webpack/tree/master/lib/dependencies

@evilebottnawi sorry for the inconveniences. What else do you need to see?
We hook into the custom require call inside the compilation when this call is executed. Then we add the dependency to the parser.state.current.addDependency.
Additionally we hook into the compilation and then execute this here:
compiler.hooks.compilation.tap(
        'CustomPlugin',
        function(compilation, params) {
            var normalModuleFactory = params.normalModuleFactory
            compilation.dependencyFactories.set(CustomRequireItemDependency, normalModuleFactory)
            compilation.dependencyTemplates.set(
                CustomRequireItemDependency,
                new CustomRequireItemDependency.Template()
            )
            compilation.dependencyFactories.set(ConstDependency, new NullFactory())
            compilation.dependencyTemplates.set(ConstDependency, new ConstDependency.Template())
        }
    )

Is this helpful?
The dependency itself is a simple dependency nothing crazy...

You need to declare the correct runtimeRequirements in your custom dependency. Choose one from RuntimeGlobals.

Hi @sokra,
thanks as well for the answer!
So you mean just adding runtimeRequirements from the outputOptions and call on it runtimeReqiurements.add(RuntimeGlobals.require)? I only need the webpack_require...
Or do we have to do more like creating this promise like in the template of lib/dependencies/AMDRequireDependency.js?

runtimeReqiurements.add(RuntimeGlobals.require)

Yes that should be it.

@sokra this seemed to work.
Thanks so much! You saved me hours of work!!! :))
I am now having a problem regarding require.context, it throws errors that it cannot resolve module abc/def/ghiTest.js in blablablatoabc/ even though the path is blablablatoabc/abc/def/ghiTest.js.. do you know how to solve that or should i open up a new ticket? The require context is called with the path-alias,true, /[tT]est$/  that should find the ghiTest.js... When calling then the module not found warnings are thrown but when calling the specific test it seems to work, but the devtools cannot open up the test file... Very odd... In webpack 4 it worked though...

Please provide example

@evilebottnawi
Okay the above description is really all i can give you, this is more central coding. And as of now, it just gives out the warnings but the code actually works... I also don't know what exactly i could show you that could help here... Maybe I'll get back to it later.
What I would like to know though, is that I have a custom HtmlPlugin, and this used to be able to get the CSS as an asset for the html page during the emit phase. Now this moved to the thisCompilation(it adds assets) processAssets phase and there i tap at the PROCESS_ASSETS_STAGE_ADDITIONS phase. There it seems though, that the css is not available as an asset yet though. I am using the style-loader and css-loader for loading assets. Do you know which stage these css assets are available now?  I know the css processing is slightly different know but i don't understand why it is not emitted during this phase.
This is the sequence of the array for use when testing against test: /\.css$/,
use: ["style-loader?esModule=false",
{
        loader: 'css-loader',
        options: {
            modules: true,
            esModule: false,
            url: false
        }
    }]

I am sorry for all these vague questions, but your help is so greatly appreciated, I fought with webpack 4 already so much and this is really saving me...

Because style-loader do not create new assets, only mini-css-extract-plugin

@evilebottnawi you are right im sorry, we used the extracttextplugin and that still seemed to work in v4. I will try to upgrade to the mini-css-extract-plugin, and will come back to you in a bit. Let you know soon.
Thank you for your help!

Hi @evilebottnawi TypeError: The 'compilation' argument must be an instance of Compilation is the error im getting when using<EMAIL_ADDRESS>

I want to close, because original problems was solved

TypeError: The 'compilation' argument must be an instance of Compilation is the error im getting when using<EMAIL_ADDRESS>
Double check what you don't have two webpack versions

Anyway feel free to feedback

@evilebottnawi I figured it out and thanks for the info there!
I have one more thing and it is the ParserHelpers.addParsedVariableToModule replacement.
Here they talk about the replacement for this parser helper function. In the old way, you could set addParsedVariableToModule(parser,name,expression). How can i do that with the new way mentioned above?
I have to give a special expression to the expression argument, which sets a custom method there which extends webpack_require...
Thank you!

@evilebottnawi  what i basically want to translate into webpack5 is this coding:
ParserHelpers.addParsedVariableToModule(
    parser,
    'CustomPluginFor' + Name.replace(/[.]/g, '_'),
    `__webpack_require__.map.set('${Name}', require.resolve(` +
    JSON.stringify(resPath + '.specialending') +
    '))'
)

Seems like the only way for me to do it now is to transform this coding into a custom dependency?


[WIP] [jest-jasmine2] Rewrite TreeProcessor.

Summary
Rewrites TreeProcessor (now treeProcessor) and pulls it out of jasmine.
It's significantly simplified under the assumption that suites and specs are already ordered, which jest seems to do already.

Test plan
Tests pending.

Not sure if it was complete, but that looks good to me, we can add more tests later


Is the design right for microservice transaction?

After some involvement in this spec and more reading, I have some doubts about this spec and its design. The fundamental issue here is the central coordinator. If we build the model around that, our MP programming model is not distributed any more. We are back to 'master central controlling' model. Is this the right approach?
In microservice world, each microservice has its own database and self contained. Suddenly, we introduced a central controller. Are we back to old days again? Would event sourcing a better approach to solve microservice transaction? Thoughts?

The design deliberately does not discuss the LRA coordination service since that is an implementation concern. I know that we do identify the coordinator using a URL but:

in MP HTTP endpoints will be service based and they do not force the centralisation of the service sitting behind it (for example they can be load balanced)
there are github issues to remove the dependance on JAX-RS in the spec

The decoupling of the JAX-RS dependence (issue 18) means that we are left with a set of APIs and annotations that I think could be implemented via an event sourcing solution so the two approaches are not mutually exclusive.
Notice also that the coordinator is not a single point of failure (which would be the case if it were a centralised controller). Any coordinator should be able to complete any LRA no matter which coordination service created it.

my two cents: as Mike stated the specification tries to be agnostic how the coordinator works. The coordinator could be backed by event sourcing engine but differently too. That's only the implementation decision.
If you consider what CAP theorem about distributed environments then there is no way to stay consistent other than use the consensus. That means there is need a coordination running in background in either way.
When you take for example Kafka as event source log solution then you hand over the responsibility of coordination to Kafka. The Kafka will provide that service for you and if it is not available then the saga  can't proceed further, it's coordinated by Kafka.
We've studied some other implementation of saga available in Java (https://jbossts.blogspot.com/2017/12/saga-implementations-comparison.html). Even those implementations are focused on providing saga experience for event sourcing (in particular for CQRS) they all provide a coordination over the saga outcome.

My opinion is that we can look at this in two parts similar to JTA and JTS. The LRA API can be a portable API to code microservices that need transactions, a separate specification (or optional part of the main spec) can deal with interop if that is determined to be advantageous.
When we look at transactions for microservices a Saga like API is a great fit for many reasons such as scalability and decoupling of the services.
Also, you can consider that some people do not need interop, they might choose not to expose the transactional capability outside of the corporate network but still wish to build saga-like microservices.

I've been working on a related implementation of the spec, for making sure that services developed with Apache Camel can interop with microprofile LRA services, and I believe the architectural model proposed here is valid.
First, because the single-point-of-failure problem is not a real issue of the spec, because (as @mmusgrov said) there are no prescriptions on how the coordinator should be implemented. Implementing it using a Kubernetes autoscaling resource backed by a sharded and replicated nosql database is allowed, so I don't see any bottleneck imposed by the spec.
Second, event-sourced sagas are certainly a thing and are a hot topic at the moment. But they have a requirement: you have to develop your business logic with a event-sourcing model in mind, and it's not a free lunch. Most developers prefer to stick with the REST API model when developing their applications and REST is one of the most widely used model for its simplicity.
The spec is about this: adding transaction-like capabilities to REST. Executing consistent synchronous or asynchronous transactions involving multiple REST services.
And this is totally valuable.
Event sourcing is a different thing and it probably requires different APIs than the one described here. But this should be seen as a possible way to improve the spec for next releases: making APIs for event sourcing, so that also events and compensating events can be described by the LRA framework, so that event-driven applications and API driven applications can cooperate consistently.

The spec is changed significantly to allow other type of implementations which do not require a central coordinator. Also the term coordinator is removed, see https://github.com/eclipse/microprofile-lra/issues/139
For example, the Payara POC has no coordinator and the Participants are just 'linked' together though a data-grid.

Yes having two significantly different designs certainly helps validate the specification. @Emily-Jiang is it okay to close this issue as fixed now?


1.3.0 snapshot

This is to prepare the repo for building 1.3.0 snapshots.

This is not what I intended.


Remove entities from "event collections"

Event Chronologies: Events already list entities, might be confusing when all not related Events are listed in entities tab.

Do both, but in data panel filter relatedEvents of an Entity by colection.event-ids + add option to show chronology of all events (no entities)


Trying to use in Classic Mode with IIS 8.5

I am trying to use NWebSec (4.1 or 4.2) with IIS 8.5 running in classic mode. I believe I've followed the installation steps correctly. I have remove the module from the system.webServer section and add it to the system.web\httpModules as per the docs but the code still seems to require integrated pipeline mode. I'm using .NET 4.5
Any advise?
This is the exception is receive:
2015-11-27 12:59:55,063 ERROR [12  ] Web.Global                               Error:  Exception: System.PlatformNotSupportedException
Message: This operation requires IIS integrated pipeline mode.
Source: System.Web
at System.Web.HttpResponse.get_Headers()
at NWebsec.Helpers.HeaderResultHandler.HandleHeaderResult(HttpResponseBase response, HeaderResult result)
at NWebsec.Helpers.ConfigurationHeaderSetter.SetSitewideHeadersFromConfig(HttpContextBase context)
at NWebsec.Modules.HttpHeaderSecurityModule.AppBeginRequest(Object sender, EventArgs e)
at System.Web.HttpApplication.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

Oh, I have to admit that this hasn't been specifically tested for quite some time. This still hangs around in the docs since it's been there since the very start of NWebsec. Any chance you could switch to the integrated pipeline?

Unfortunately I'm working with a large enterprise web application that has many dependencies on Classic mode. Do you think there would be a lot of work to modify the modules to working within classic mode?
Also, is it possible with NWebSec, to define different CSP headers on a per-page basis?

Oh, ok. IIRC I moved off a deprecated method to set the headers some time ago, but I'll have to investigate. I'll have a look when I've gotten the vnext packages out the door.
If you're running MVC, the MVC attributes will let you configure headers per controller/action. If you're using webforms or webpages, the only way to tweak the configuration for different pages/paths would be through either a location element in web.config, or the classic sub-web.config files for particular directories.

Hi again, I dug into this a bit and this became a problem with the updates in NWebsec v3. Breaking changes in ASP.NET forced NWebsec over to the new API for setting http headers, which in turn broke classic mode. This went unnoticed on my part.
Unfortunately, the only option would be to try NWebsec v2 for your "classic" application. I've updated the docs to clarify this.


An option to run all android tests

Hi,
Is there an option to dryrun all android tests?

hi, nope there isn't

Ok, but would it be possible to add it?
The second question is, whether we should or not?
What do you think about it?
On Fri, Nov 3, 2017, 21:07 César Ferreira<EMAIL_ADDRESS>wrote:

hi, nope there isn't
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/cesarferreira/dryrun/issues/101#issuecomment-341812691,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFZnnsMEVskIqMotFxVKMZNm8x-y36Q5ks5sy3JtgaJpZM4QRiUC
.

Yep, it's possible to add, instead of installing the app, run the tests, feel free to open a pull request


Spaces between curly and square brackets in urls should not be ignored

Following code is incorrect and should not be coverted to url, and it is not converted to url by any implementation except showdown because there is a space between ] and (
[http://ya.ru] (http://ya.ru)

https://johnmacfarlane.net/babelmark2/?normalize=1&text=[http%3A%2F%2Fya.ru]+(http%3A%2F%2Fya.ru)
Markdown original spec allows spaces or newlines between brackets.


Peer Review - Milestone 2

Hi Rangers,
I have gone through your wiki. I'm waiting for your Jenkins credentials. Can you please share the details as soon as possible so that I can review and get back to you.
Thanks,

I have send you a mail with credentials, please let me know if anything.

Thank you. Will look into it.

I was able to login to Jenkins successfully. I have gone through the architecture diagram in the wiki and tried to relate the job present in the Jenkins homepage. I'm not able to find Add room vacancy micro service,  Feedback micro service, search room micro service. Does all these services are present in user service and room service jobs?

We have not implemented feedback service. Add and virw listings are in room service

Okay got it. I'm planning to perform a commit and check how the application is being deployed. Other peer reviewers or you guys may be playing around with the Jenkins. Can you please provide me a time slot so that I can do this?

I was able to find the branch and IP in the wiki. Please provide a time slot and port for user interface.

I think you can do the dummy commit and check the CI/CD. 3000 port is used for UI. For accessing UI hit http://<IP_ADDRESS>:3000/login

Okay, thank you.

Notification service is deployed successfully through git commit.

Room service is also deployed successfully through git commit.

UI is also deployed successfully through git commit, though it has taken time. I was able to access the UI from browser and perform login and registration. Will test other functionalities after deploying the next(last) service.

I waiting past 15 minutes for the user service to be deployed. The console output is as follows:
Started by GitHub push by sairohithA007
Obtained Jenkinsfile from git https://github.com/airavata-courses/Rangers.git
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Still waiting to schedule task
‘userservice’ is offline

I waiting past 15 minutes for the user service to be deployed. The console output is as follows:
Started by GitHub push by sairohithA007
Obtained Jenkinsfile from git https://github.com/airavata-courses/Rangers.git
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Still waiting to schedule task
‘userservice’ is offline

Problem is solved and build is successfully deployed. I have rebuild the userservice job, can you please check

Thank you. Will try to do a git commit again.

This time user service is deployed successfully. Now I will go through the UI and check if anything is working fine or not.

I faced two issues out of which one is #32 is already discovered and the other is #33

#33 is fixed.

Summary:
The CI/CD process through works fine.
The issue #33  is also fixed.

Closing this issue as #32 is also fixed.


diktat plugins should not check any code in build dir

target for maven and build for gradle should be automatically excluded

Should be not needed with correctly configured input, must check once more and maybe close this issue.

Should be not needed with correctly configured input, must check once more and maybe close this issue.


Cypress/E2E: Add cy.uiClickPostDropdownMenu command

Summary
Add cy.uiClickPostDropdownMenu command.
Test report: to follow (currently running)

I'm okay with the use of constants, feel free to submit PR once this is merged. I'm a bit off to very targeted commands like cy.uiClickPostMenuMarkAsUnread. It will end up with huge overhead, especially for those new to our codebase. Right now, there are too many Cypress commands and I'd prefer to limit it as much as possible.


Fix issue number in release notes for #29681

Copy & paste error
[ ] Configuration Infrastructure
[X] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
Pull Request Attributes
Please check any characteristics that apply to this pull request.
[X] Does not have any changes that may affect Istio users.

Hi @chlunde. Thanks for your PR.
I'm waiting for a istio member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.
Once the patch is verified, the new status will be reflected by the ok-to-test label.
I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/ok-to-test


NixOS manual list of configuration options might be nice as a tree

It's a massive list and has a natural hierarchical structure to it. Perhaps letting us collapse/expand it by default would make it easier to browse available settings.

I suggest we close this until someone steps up and provides a good minimal working design implementation. @domenkozar

Closing because there is no clear actionable.


Promise Namespacing Issue

There is a Namespacing inconsistency in GraphQL::Batch::Loader between the Promises returned by load and load_many causing them to return different classes.
   def load(key)
      cache[cache_key(key)] ||= begin
        queue << key
        Promise.new.tap { |promise| promise.source = self } ### returns GraphQL::Batch::Promise
      end
    end

    def load_many(keys)
      ::Promise.all(keys.map { |key| load(key) })  ### returns Promise
    end

Is there a particular reason for this? I can't seem to reason one.

Your issue doesn't actually explain a problem.  If you are doing something like .is_a?(GraphQL::Batch::Promise) in your application code, change it to .is_a?(Promise)
The reason for the difference is more of an implementation detail.  The only difference between ::Promise and GraphQL::Batch::Promise is that GraphQL::Batch::Promise overrides defer so that we can use GraphQL::Batch::Executor.current.loading in tests to detect when an unbatched query is performed in a test.

That's exaclty my problem, but it didn't seem resonable to have to validate for both Promise and GraphQL::Batch::Promise since I want the code to work both when i call load and load_many seemlessly.

I don't know what is making it hard to write code that works for both Promise and GraphQL::Batch::Promise objects.  Just don't reference GraphQL::Batch::Promise or do class equality checks.
Doing so will also allow Promise.all(promises) to just work, so you don't need to use the more verbose GraphQL::Batch::Promise.all(promises).
Ideally, I would like to just return Promise objects from both load and load_many in the future, deprecate GraphQL::Batch::Promise and alias it to ::Promise.  However, this hasn't been a high priority, since been causing problems for us that I know of.


Forward button is misaligned

The next track button is lower than the other 2 buttons

P.S
Spotify Version <IP_ADDRESS>2.gcc529952
Spicetify 2.10.2
and 1920x1080 screen (16:9)

#39 seems to address this

Pull request merged

Now you might good to go


Refactorize shaderloader

Shader loader class can be refactorized
The methods that load the shader can be refactorized intro just one.

For better code comprehension I've decided this wont be necesary.


[TACHYON-1126] adding missing metrics and a adopting a consistent naming convention

https://tachyon.atlassian.net/browse/TACHYON-1126
Hi everyone,
this PR is a first step for the issue above.
What I have done is add missing metrics for relevant RPCs not yet covered.
Then I have renamed out of sync metrics (e.g. FilesCheckpointed into FilesPersisted and GetFileStatusOps into GetFileInfosOps).
In order to clear any ambiguity, I decided to put a "path" prefix for any metric related to files and directories (e.g. FilesCreated will be PathsCreated).
Best regards,
Saverio

@saveveltri thank you for this improvement ... I like using "path" to refer to both "files" and "directories"! ... left some comments for you

I left some comments. Thanks for this improvement @saveveltri !

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Tachyon-Pull-Request-Builder/7301/Build result: FAILURE[...truncated 3175 lines...][JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/underfs/s3/target/tachyon-underfs-s3-0.9.0-SNAPSHOT.jar to org.tachyonproject/tachyon-underfs-s3/0.9.0-SNAPSHOT/tachyon-underfs-s3-0.9.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/underfs/s3/target/tachyon-underfs-s3-0.9.0-SNAPSHOT-sources.jar to org.tachyonproject/tachyon-underfs-s3/0.9.0-SNAPSHOT/tachyon-underfs-s3-0.9.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/underfs/s3/target/tachyon-underfs-s3-0.9.0-SNAPSHOT-javadoc.jar to org.tachyonproject/tachyon-underfs-s3/0.9.0-SNAPSHOT/tachyon-underfs-s3-0.9.0-SNAPSHOT-javadoc.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/clients/pom.xml to org.tachyonproject/tachyon-clients/0.9.0-SNAPSHOT/tachyon-clients-0.9.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/clients/unshaded/pom.xml to org.tachyonproject/tachyon-client-unshaded/0.9.0-SNAPSHOT/tachyon-client-unshaded-0.9.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/clients/unshaded/target/tachyon-client-unshaded-0.9.0-SNAPSHOT.jar to org.tachyonproject/tachyon-client-unshaded/0.9.0-SNAPSHOT/tachyon-client-unshaded-0.9.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/clients/unshaded/target/tachyon-client-unshaded-0.9.0-SNAPSHOT-sources.jar to org.tachyonproject/tachyon-client-unshaded/0.9.0-SNAPSHOT/tachyon-client-unshaded-0.9.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/clients/unshaded/target/tachyon-client-unshaded-0.9.0-SNAPSHOT-javadoc.jar to org.tachyonproject/tachyon-client-unshaded/0.9.0-SNAPSHOT/tachyon-client-unshaded-0.9.0-SNAPSHOT-javadoc.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/servers/pom.xml to org.tachyonproject/tachyon-servers/0.9.0-SNAPSHOT/tachyon-servers-0.9.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/underfs/hdfs/pom.xml to org.tachyonproject/tachyon-underfs-hdfs/0.9.0-SNAPSHOT/tachyon-underfs-hdfs-0.9.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/underfs/hdfs/target/tachyon-underfs-hdfs-0.9.0-SNAPSHOT.jar to org.tachyonproject/tachyon-underfs-hdfs/0.9.0-SNAPSHOT/tachyon-underfs-hdfs-0.9.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/underfs/hdfs/target/tachyon-underfs-hdfs-0.9.0-SNAPSHOT-sources.jar to org.tachyonproject/tachyon-underfs-hdfs/0.9.0-SNAPSHOT/tachyon-underfs-hdfs-0.9.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/underfs/hdfs/target/tachyon-underfs-hdfs-0.9.0-SNAPSHOT-javadoc.jar to org.tachyonproject/tachyon-underfs-hdfs/0.9.0-SNAPSHOT/tachyon-underfs-hdfs-0.9.0-SNAPSHOT-javadoc.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/pom.xml to org.tachyonproject/tachyon-parent/0.9.0-SNAPSHOT/tachyon-parent-0.9.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/clients/client/dependency-reduced-pom.xml to org.tachyonproject/tachyon-client/0.9.0-SNAPSHOT/tachyon-client-0.9.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/clients/client/target/tachyon-client-0.9.0-SNAPSHOT.jar to org.tachyonproject/tachyon-client/0.9.0-SNAPSHOT/tachyon-client-0.9.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/clients/client/target/tachyon-client-0.9.0-SNAPSHOT-sources.jar to org.tachyonproject/tachyon-client/0.9.0-SNAPSHOT/tachyon-client-0.9.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/clients/client/target/tachyon-client-0.9.0-SNAPSHOT-javadoc.jar to org.tachyonproject/tachyon-client/0.9.0-SNAPSHOT/tachyon-client-0.9.0-SNAPSHOT-javadoc.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/shell/pom.xml to org.tachyonproject/tachyon-shell/0.9.0-SNAPSHOT/tachyon-shell-0.9.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/underfs/glusterfs/pom.xml to org.tachyonproject/tachyon-underfs-glusterfs/0.9.0-SNAPSHOT/tachyon-underfs-glusterfs-0.9.0-SNAPSHOT.pom[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/underfs/glusterfs/target/tachyon-underfs-glusterfs-0.9.0-SNAPSHOT.jar to org.tachyonproject/tachyon-underfs-glusterfs/0.9.0-SNAPSHOT/tachyon-underfs-glusterfs-0.9.0-SNAPSHOT.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/underfs/glusterfs/target/tachyon-underfs-glusterfs-0.9.0-SNAPSHOT-sources.jar to org.tachyonproject/tachyon-underfs-glusterfs/0.9.0-SNAPSHOT/tachyon-underfs-glusterfs-0.9.0-SNAPSHOT-sources.jar[JENKINS] Archiving /home/jenkins/workspace/Tachyon-Pull-Request-Builder/underfs/glusterfs/target/tachyon-underfs-glusterfs-0.9.0-SNAPSHOT-javadoc.jar to org.tachyonproject/tachyon-underfs-glusterfs/0.9.0-SNAPSHOT/tachyon-underfs-glusterfs-0.9.0-SNAPSHOT-javadoc.jarchannel stopped
Test FAILed.

Thank you guys for your feedbacks and suggestions.
I've pushed a new version including

your suggestions
a little issue found in FileSystemMaster#createInternal fixed

when a path was created, only FileCreatedOps metric was increased.
Now files and directories are treated separately into their related metrics

Metric System Test added

checks the correct behaviour of the metrics counters

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Tachyon-Pull-Request-Builder/7302/
Test PASSed.

@saveveltri thank you for the update and providing a unit test! I left some minor comments on naming and code organization.

I left some minor comments. Thanks for this PR!

Thanks for your feedback.
I have integrated them and code has been rearranged.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Tachyon-Pull-Request-Builder/7353/
Test PASSed.

@saveveltri thanks ... one more comment

After the documentation organization, this looks good! Thanks for the great improvement!

Thanks! I've updated the documentation.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Tachyon-Pull-Request-Builder/7406/
Test PASSed.

LGTM

LGTM. Thanks for this big improvement!


fix(versions): update Activiti/activiti-cloud-runtime-bundle-service versions into develop

UpdateBot pushed maven dependency: org.activiti.dependencies:activiti-dependencies to: 7.1.111

UpdateBot commands:
updatebot push-version --kind maven org.activiti.dependencies:activiti-dependencies 7.1.111 org.activiti.api:activiti-api-dependencies 7.1.54 org.activiti.core.common:activiti-core-common-dependencies 7.1.62 org.activiti.build:activiti-parent 7.1.28


load data dynamically

Amazing plugin!
My data is loading dynamically, after page load. To make it work with TrackPlayback, I put all the code in the function block that triggers after my data has loaded.
Upon user input (filtering) I want to render the filtered data. But the function block where all the declaration was done has run and closed. I can't access it globally.
By declaring trackplayback globally and initiating it locally, I'm still able to do trackplayback.dispose() which gets rid of whatever was there on the map; but does not dispose of the playback control; and I'm not able to re-initate trackplayback again.
There is a cross button on the playback control which is generated by this library, which gets rid of everything. I'd like to be able to execute that then re-declare TrackPlayback all over again. But that cross button is also internally generated and there's a listener listening to it that I can't figure out how to trigger. As a workaround I'd like to trigger this "destroy everything" type action and then run the starting commands all over again with my new data.
Any tips on how to move forward would be most appreciated!

Hi @answerquest - I was testing out this plugin as well and had some issues figuring out the best way to clear/dispose of the playback control after another round of user input.
I ended up simulating a click on the close button on the plugin by doing this:
try {
$('.btn-close')[0].click();
} catch {}

Hi @krusaderjake , that's a great hack!
I recently found another way after reading the source code to see what the button's actions were:
trackplayback.dispose()
and then: remove the playback box:
document.querySelector("#mapid > div.leaflet-control-container > div.leaflet-top.leaflet-right > div")
But your solution is more elegant. :+1:


feat: add "instrument" option as alternative to specifying noop instrumenter

it felt a little weird providing a noop instrumenter directly in __coverage__, this pull introduces the instrument flag instead -- making it possible to setup __coverage__ with the following config:
 "nyc": {
    "include": [
      "index.js"
    ],
    "require": [
      "babel-register"
    ],
    "reporter": [
      "lcov",
      "text"
    ],
    "sourceMap": false,
    "instrument": false
  }

which feels cleaner.

@dtinth added the --instrument flag which, in my opinion, makes configuring __coverage__ feel less weird.


Fix space leak in histogram

We had a leak of Doubles in the Timer, that I traced back to the variance tuple in the Histogram. Making them Double of the tuple strict when calculating fixes that.

Thanks for the PR!

I just released <IP_ADDRESS> to Hackage with this fix. Also, I gave you commit access to the project.

Awesome, thanks!


IONIQ EV 2020 - LFA Warning on the dash when OP is running

Describe the bug
When OpenPilot is running the car shows on the dash a warning to "Check LFA system".

My car only has "LKA", but now thinks it has LFA and the warning appeared.

This warining started to appear now with the C3X, before with C2 didn't appear.
With the help of @sshane we were able to identify that: "When we used to send an LFA message to your car, which normally doesn't have LFA, it put the car into a persistent state where it thinks it has LFA, so our missing message is causing a fault"
Discord Thread: car thinks it has LFA
Which car does this affect?
HYUNDAI IONIQ ELECTRIC 2020
Provide a route where the issue occurs
085e33b2085c44f4|2023-08-22--08-08-05
openpilot version
0.9.5-lfa-msg-test (606cb6547c567582924c08df23bdc26e18a70524)
Additional info
With the help of @sshane we tested a branch (lfa-msg-test (606cb6547c567582924c08df23bdc26e18a70524)) that forces sending de "LFA" message and with that branch all warnings disappeared

@sshane is this still an issue?

Yes, the car (cluster, ABS?) is expecting the LFA message but is not receiving it because we detect the camera not sending it.
@dermida do the warning show up without the comma 3X plugged in? What about if the 3X is in dashcam mode? Can you get a route where it's in dashcam mode?

@dermida do the warning show up without the comma 3X plugged in?
Yes. Even when not plugged in.

What about if the 3X is in dashcam mode?
Never tried. But i can try if you need.

Can you get a route where it's in dashcam mode?
Sure. I will try tomorrow.

Sounds like we put one of your ECUs into a state of expecting the LFA message before we detected it in https://github.com/commaai/openpilot/pull/27323. So, the current openpilot behavior is not a bug, and I don't think there's a solution to this that should be merged. I would recommend trying to visit your dealer to have them check it out and re-flash the ECU responsible.
The best we can do is try to narrow down the message that describes the fault, then we can have a good guess of which ECU we affected in the past. Feel free to reach out on Discord.


Screenshots are randomly sized in the screenshots menu

Screenshots are not of same size

They could be sized the same to fit their window, i think the Slovenian screenshot here looks the best size (border)

Hi, the screenshots are randomly sized because they work using Markup Table, and the header of the table decides how big it's column will be.
This is not a wanted behavior, and I'm planning on converting this mechanism to html based one, which won't do such problems. doing that will fix this and #2


MySQL2 stream option 'Illegal invocation' error

Environment
Knex version: 0.21.18
Database + version: MariaDB 10.2.37
OS: macOS 11.2.2
Bug

Explain what kind of behaviour you are getting and how you think it should do
I want to connect to MySQL via an SSH tunnel. So I've set up an SSH tunnel via ssh2 and used the ssh2.forwardOut to create the tunnel. This works as expected.

I've tried to create a connection with MySQL2 and query raw data. Which works as expected, I can query data via the SSH tunnel. See code below.
Now, I've created a Knex connection, with client mysql2 and the same config as the MySQL2 connection I've used before Knex. I expected this to work properly, however, I get an "TypeError: Illegal invocation" error when querying the database. See code below

Error message

02-03 12:36:13 [index.ts, master (10932)] error: uncaughtException:Illegal invocation TypeError: Illegal invocation
    at Cipheriv.update (internal/crypto/cipher.js:161:29)
    at send_ (./node_modules/ssh2-streams/lib/ssh.js:5047:34)
    at send (./node_modules/ssh2-streams/lib/ssh.js:4961:12)
    at SSH2Stream.channelData (./node_modules/ssh2-streams/lib/ssh.js:969:10)
    at Channel._write (./node_modules/ssh2/lib/Channel.js:347:21)
    at writeOrBuffer (_stream_writable.js:352:12)
    at Channel.Writable.write (_stream_writable.js:303:10)
    at Connection.write (./node_modules/mysql2/lib/connection.js:230:32)
    at Connection.writePacket (./node_modules/mysql2/lib/connection.js:279:12)
    at ClientHandshake.sendCredentials (./node_modules/mysql2/lib/commands/client_handshake.js:64:16)
    at ClientHandshake.handshakeInit (./node_modules/mysql2/lib/commands/client_handshake.js:137:12)
    at ClientHandshake.execute (./node_modules/mysql2/lib/commands/command.js:39:22)
    at Connection.handlePacket (./node_modules/mysql2/lib/connection.js:425:32)
    at PacketParser.onPacket (./node_modules/mysql2/lib/connection.js:75:12)
    at PacketParser.executeStart (./node_modules/mysql2/lib/packet_parser.js:75:16)
    at Channel.<anonymous> (./node_modules/mysql2/lib/connection.js:82:25)

Reduced test code

import { Client, ClientChannel } from 'ssh2';
import mysql2 from 'mysql2';
import Knex from 'knex';

const init = async () => {
  // Connect to SSH
  const ssh = new Client();
  await new Promise(async (resolve, reject) => {
    try {
      ssh.connect({ ... });
    } catch (e) {
      reject(e);
    }

    ssh.on('ready', resolve);
    ssh.on('error', reject);
  });

  // Create SSH tunnel from SSH client port 3306 to local PC port 3306
  let stream: ClientChannel;
  stream = await new Promise((resolve, reject) => {
    ssh.forwardOut('<IP_ADDRESS>', 3306, '<IP_ADDRESS>', 3306, (err: Error | undefined, channel: ClientChannel) => {
      if (err) return reject(err);
      resolve(channel);
    });
  });

  // MySQL2 connection
  // Everything goes fine here
  const mysqlConnection = mysql2.createConnection({
    host: '<IP_ADDRESS>',
    port: 3306,
    stream: stream
  });

  mysqlConnection.query('SELECT 1', (err, result) => {
    if (err) {
      throw err;
    }

    console.log('mysql2 result', result);
  });

  // Knex connection
  const knex = Knex({
    client: 'mysql2',
    connection: {
      host: '<IP_ADDRESS>',
      port: 3306,
      stream: stream
    }
  });

  // This will error with 'TypeError: Illegal invocation'
  console.log('knex result', knex.raw('SELECT 1'));
};

init();

Getting exactly the same problem. @bartversluijs did you manage to solve it somehow?

Hi @israelidanny, unfortunately I could not find a fix for this.
I created a workaround, which was creating a tunnel using tunnel-ssh and tunneled the port 3306 to <IP_ADDRESS>:3306, and connected to that. So instead of using the MySQL2 engine to do that, I created a tunnel for the whole Node.js application.
Before you do this, after a while I got a nasty memory leak. I checked everything, and after removing the SSH tunnel and directly connecting to the database with an open firewall, the memory leak was gone. So I would not advise you to use the workaround I did.
I hoped someone of the Knex maintainers would find a solution for this. It's probably something with a wrong bind or something, but haven't had the time and resources to debug it myself.

@bartversluijs It is unlikely that maintainers will find a solution for this, because unfortunately all of them are very busy and are unlikely to dedicate any time to investigating a problem as niche as this. However, any community contributions here would be dearly appreciated.

@kibertoad I totally get that. As said, if I find some time / resources to debug this problem, I probably will. I totally agree it's a niche problem.
Just hoped that someone would see this and thought: "Oh, that's an easy fix" 🙂

Hi. I ran into this issue earlier today. Just thought I'd share that providing a function that returns the connection config, for the connection property seems to work as expected. So, instead of this:
const knex = Knex({
    client: 'mysql2',
    connection: {
      host: '<IP_ADDRESS>',
      port: 3306,
      stream: stream
    }
  });

try this instead:
const knex = Knex({
    client: 'mysql2',
    connection: () => ({
      host: '<IP_ADDRESS>',
      port: 3306,
      stream: stream
    })
  });

I am wondering which driver should I use when connecting to MariaDB v10...
as the https://knexjs.org/ says,
If you want to use a MariaDB instance, you can use the mysql driver.

So knex don't support mysql2 to connet to MariaDB ?

https://github.com/knex/knex/issues/717


suggestion to have gem without javascript dependency

gem 'turbolinks-rails' could have the pure rails code for users that bring in the JS through yarn/npm with webpacker

gemspec available at https://github.com/koenhandekyn/turbolinks-rails/blob/master/turbolinks-rails.gemspec


Is SAM2 really 6 times faster than SAM for image segmentation?

Hi, I have briefly had a look at the introduction of SAM2. If I get it right, it says SAM2 is 6 times faster than SAM when used for image segmentation. But I did a test of segmenting the same image with the same prompts and found SAM2 is actually slower.
I'm comparing SAM using the "vit_h"  model and SAM2 using the "hiera_large" model. Below is the output (I can post my code if necessary). :
Loading time (SAM): 10.890620231628418 seconds
Execution time (SAM): 0.271709680557251 seconds
Loading time (SAM2): 4.8107569217681885 seconds
Execution time (SAM2): 0.4044489860534668 seconds
In the above, loading time is the time used for loading the model while execution time is the  time for making a prediction. We can see that the time for loading the model has been halved but the time for making a prediction has increased by almost 50%.
Did I misunderstand the introduction about the speed of SAM2 or I make any mistake in coding?
Thank you!

Those numbers seem a bit strange (at the execution times). Assuming you're using cuda, you may need to force a synchronization before recording the times? That can often lead to misleading results otherwise. It's also important to run the model many times, since the first run usually is much slower than repeat runs.
From what I've seen, SAMv2 does seem consistently faster than v1, but I think the 6x faster claim requires very careful cherry-picking of the data.
Specifically, I think they may be comparing either SAMv1-huge (a 2.5GB file) or SAMv1-large (1.2GB) running on float32 vs. SAMv2-large (900MB) running on float16, and the speed up is to do with image encoding only. The prompt + mask decoding is very slightly slower on v2.
Here are the results I get (averaged across 50 runs) for different models/data types:

Model
Data type
Image encoding time (ms)

V1-Huge
float32
430

V1-Large
float32
244

V2-Large
float32
158

V1-Huge
bfloat16
165

V1-Large
bfloat16
100

V2-Large
bfloat16
47

So comparing V2-large on float16 vs. V1-large on float32 gives ~5.2x speed up, while comparing to V1-huge w/ float32 gives a 9x speedup. But in more like-for-like comparisons (e.g. comparing the large models on the same data type) it's more like a 2x speed up (still good, but not the claimed 6x).

Yes, SAM 2 really is 6 times faster than SAM for image segmentation. :smiley:
All fps numbers are measured end-to-end (not just image encoding), using the same precision for all models (bfloat16) in a like-for-like and fair comparison - no cherry picking, @heyoeyo.
The default SAM 2 model (throughout the paper) is the B+ model, while the default SAM 1 model (throughout that paper) is the H-model, the default models are compared in Table 6, shown below

Put another way, if you were to visualize an accuracy vs speed plot, at the same accuracy (actually even better accuracy) as SAM (H), the speed-up is 6x using SAM 2 (B+)
You may be interested in Appendix C.3 where we describe how speed is benchmarked. As @heyoeyo suggested, @squashking you might want to discard a small number of batches to get more reliable numbers. You maybe also be interested in Table 15 which has additional benchmarking.

Thanks for clarifying @chayryali, I was thinking picking different sized models counts as cherry-picking, but comparing by matching accuracy makes sense.
And for sake of verification, here's a continuation of the (image encoding) timing I get for the base/tiny models:

Model
Data type
Image encoding time (ms)

V2-base+
bfloat16
22

V2-tiny
bfloat16
10

Which is consistent with the >6x claim for v2-base+ vs v1-huge.

Similarly sized models from different model families can have quite different speed (e.g. v1-L vs v2-L in your benchmarking above), so such comparison is not super useful. And size is anyway often just a (bad) proxy for speed (what we often actually care about).
Thanks for the verification!

Thank you @chayryali  and @heyoeyo , but I'm still not sure why I got the opposite results. As I said, I was comparing SAM1( "vit_h" model) and SAM2 ( "hiera_large" ) for segmenting exactly the same image, I guess I have to post my code for clarification:
import torch
import numpy as np
import cv2
from PIL import Image
from sam2.build_sam import build_sam2
from sam2.sam2_image_predictor import SAM2ImagePredictor

from segment_anything import sam_model_registry, SamAutomaticMaskGenerator, SamPredictor

import time

import hydra

New_SAM = False

# use bfloat16 for the entire notebook
if New_SAM:
    torch.autocast(device_type="cuda", dtype=torch.bfloat16).__enter__()

# image = Image.open('/home/taohu/Projects/Data/RGB/thumbnail_Picture1.png')
# image = np.array(image.convert("RGB"))

image = cv2.imread('/home/taohu/Projects/Data/RGB/thumbnail_Picture1.png')
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

if New_SAM:
    method = "SAM2"
else:
    method = "SAM1"

start_time1 = time.time()

if New_SAM:
    sam2_checkpoint = "models/sam2_hiera_large.pt"
    model_cfg = "sam2_hiera_l.yaml"

    # hydra is initialized on import of sam2, which sets the search path which can't be modified
    # so we need to clear the hydra instance
    hydra.core.global_hydra.GlobalHydra.instance().clear()
    # reinit hydra with a new search path for configs
    hydra.initialize_config_module('sam2_configs', version_base='1.2')
    sam2_model = build_sam2(model_cfg, sam2_checkpoint, device="cuda")

    predictor = SAM2ImagePredictor(sam2_model)
    predictor.set_image(image)
else:
    model_type = "vit_h"
    sam_checkpoint = "models/sam_vit_h_4b8939.pth"
    sam = sam_model_registry[model_type](checkpoint=sam_checkpoint)
    sam.to("cuda")
    predictor = SamPredictor(sam)
    predictor.set_image(image)

end_time1 = time.time()
load_time = end_time1 - start_time1
print(f"Loading time ({method}): {load_time} seconds")

input_box = np.array([58,107, 213,281])
input_point = np.array([[104, 163]])
input_label = np.array([1])

start_time2 = time.time()

masks, scores, logits = predictor.predict(
    point_coords=input_point,
    point_labels=input_label,
    box=input_box,
    multimask_output=False,
)

end_time2 = time.time()
execution_time = end_time2 - start_time2
print(f"Execution time ({method}): {execution_time} seconds")

mask_array = np.array(masks[0])

if New_SAM:
    mask_array = mask_array.astype(np.uint8)*255 # SAM2 use 0~1 values for the mask
    mask_image = Image.fromarray(mask_array)
    mask_image.save("sam2-bw.jpg")
else:
    mask_image = Image.fromarray(mask_array)
    mask_image.save("sam1-bw.jpg")

So I ran the above code twice, the first time I set "New_SAM" to False (which means using SAM1), and the second time I set it to True (which means using SAM2). And I got the results:
Loading time (SAM1): 10.890620231628418 seconds
Execution time (SAM1): 0.271709680557251 seconds
Loading time (SAM2): 4.8107569217681885 seconds
Execution time (SAM2): 0.4044489860534668 seconds
I was running with CUDA. My GPU was RTX 4060 Ti.
The above results showed the SAM2 is almost 50% slower than SAM1 in making a prediction. Can you please check if  made any mistake in my code? Thank you!

@squashking I think the issue is that the image encoder is running within the 'loading' timed block.
More specifically it's the line:
predictor.set_image(image)

The image encoder is the bulk of the SAM model execution time, so if you move that into the execution time block, I think that should give the expected timing results.
As for why your numbers consistently show SAMv2 being ~50% slower than SAMv1 when timing the predict function. That function runs the prompt + mask decoder, and the mask decoder really is going to be consistently slower on v2, due to having some extra steps compared to v1.

@heyoeyo Thank you for the explanation. I do find that after moving "set_image" to the execution time block the speed of SAM2 is 3 times (a bit less) than SAM1:
Loading time (SAM1): 9.282342672348022 seconds
Execution time (SAM1): 1.4434857368469238 seconds
Loading time (SAM2): 4.7463905811309814 seconds
Execution time (SAM2): 0.4952991008758545 seconds
I think under some circumstances the time for making a prediction (or inference) is more important than the time for image encoder.  For example, suppose I need to repeatedly update the prompts to generate the desired mask, in this case "set_image" just need to run once but "predictor.predict" needs to run multiple times, so in the end SAM2 could actually takes more time than SAM1.

... "predictor.predict" needs to run multiple times, so in the end SAM2 could actually takes more time than SAM1

Yes that's a good point! Though the prompt + mask decoder runs quite fast compared to the image encoder, in most cases I think it would take dozens of re-runs of the decoder before it takes more total time than the one-time image encoding (this can happen extremely quickly using the model interactively to be fair).
Oddly though, your updated numbers suggest the decoder is running very slowly. SAM2 was 0.404s with only the "predictor.predict" and then 0.495s after including "set_image", which suggests that "set image" takes ~100ms and "predict" takes ~400ms. It might be worth running the predictor twice and timing only the second run just to double check this, because it should be much faster, something like:
input_box = np.array([58,107, 213,281])
input_point = np.array([[104, 163]])
input_label = np.array([1])

# Run once as a 'warm-up' just to rule out initial loading effects
masks, scores, logits = predictor.predict(
    point_coords=input_point,
    point_labels=input_label,
    box=input_box,
    multimask_output=False,
)

# Run a second time with timing
start_time2 = time.time()
masks, scores, logits = predictor.predict(
    point_coords=input_point,
    point_labels=input_label,
    box=input_box,
    multimask_output=False,
)
end_time2 = time.time()
execution_time = end_time2 - start_time2
print(f"Execution time ({method}): {execution_time} seconds")

I think you should find the decoder runs well under 400ms (I find it only takes ~20ms on CPU for example)

@heyoeyo  You are right  the decoding time is negligible compared to the encoding time. The two results above were from different runs - you cannot simply do the subtraction, because the counted time fluctuates each time I run the code. Now, I have modified the code a bit by printing the time for the encoder and decoder separately and following your advice to run 'predict' twice. Below are the results:
Loading time (SAM1): 9.20648717880249 seconds
Encoder time (SAM1): 1.1701500415802002 seconds
Prediction time (SAM1): 0.008125543594360352 seconds
Loading time (SAM2): 4.628658056259155 seconds
Encoder time (SAM2): 0.09668612480163574 seconds
Prediction time (SAM2): 0.011764764785766602 seconds
I can see the time for the encoder has reduced by 12 times, while the prediction time goes up by 50%. You are right the prediction time is well below 400 ms.

Hi @squashking,
great results. Can you also please share the code for these results? Thanks a lot!

@squashking Those SAMv2 results look more like what I would expect. The SAMv1 result is surprisingly slow still (at least the image encoder part). In case that matters, I believe part of the reason the bfloat16 code near the top of your script:
if New_SAM:
    torch.autocast(device_type="cuda", dtype=torch.bfloat16).__enter__()

This probably means that SAMv1 is using float32. If v1 is made to also use bfloat16, I think the encoder time would drop by ~50% (e.g. it would be closer to 0.5s rather than >1s), though if I remember correctly, SAMv1 may require some (minor) changes to work properly with float16.

@heyoeyo If I use bfloat16 also for SAM1, I will get error: TypeError: Got unsupported ScalarType BFloat16
Can you tell me how to do minor changes to make it work? Thank you!

Can you tell me how to do minor changes to make it work?

It's been a long time since I looked at it, so I don't remember the exact changes... Though I think it's mainly a matter of either (a) changing to float32 temporarily to do certain calculations that aren't supported when using bfloat16 (SAMv2 has examples of this) or (b) there are parts of v1 that hard-code the dtype of some values to float32 and that causes issues when bfloat16 data is used, so the solution is just to make sure the dtype is set to match the incoming data.
The easiest way to make these changes is to run the model with bfloat16 to get an error message which will generally tell you the script/line number where the hard-coded dtype will need to be fixed. You can make those changes, then re-run the model and fix the next dtype error and so on until it runs without any problems.
Alternatively, I have a repo that supports v1 & v2 models using float32 or bfloat16. The repo includes a benchmarking script (the same thing I used for the numbers I posted earlier) that you can use/modify to check the model speeds (just make sure to change the path to point at the model file you want to test). The one caveat is that the loading times won't be the same (they'll be slower) due to extra loading/automatic config checks that the original code doesn't perform.

@heyoeyo  Thank you! Nice repo. I'll give it a try.


double-beep: Watch voodootemp(?:@outlook.com)?

double-beep requests the watch of the watch_keyword voodootemp(?:@outlook\.com)?. See the MS search here and the Stack Exchange search here.

voodootemp(?:@outlook\.com)? has been seen in 2 true positives, 0 false positives, and 0 NAAs.

Approved by Machavity in Charcoal HQ


fix issues serving#421 Symbol not found __ZN6google8protobuf8internal10LogMessageC1ENS0_8LogLevelEPKci

fix issues serving#421
the problem reason is single_image_random_dot_stereograms.cc<<kernels>> be dependent on TensorShapeProto
it define on protobuf, use class LogMessage. but no deps protobuf on BUILD rule python/ops/_single_image_random_dot_stereograms.so .
so Symbol not found: __ZN6google8protobuf8internal10LogMessageC1ENS0_8LogLevelEPKci(google::protobuf::internal::LogMessage::LogMessage)

Can one of the admins verify this patch?

this fix worked for me, thank you so much for fixing it!

If you can sign the CLA we should be able to accept this, thanks!

Can one of the admins verify this patch?

I signed it!

ok @vrv please accept, thanks!

@tensorflow-jenkins test this please

Would the better thing to have done to modify https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tensorflow.bzl#L957 to add the tf_custom_op_library_additional_deps, instead of adding it to the leaf target here?

Actually I don't know how this did anything,
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tensorflow.bzl#L936 was already adding the protobuf header library to the cc_binary rule, if I am understanding correctly.

Oh, we added protobuf here, not protobuf_headers.  My bad.

I get the same issue. Can some one guide me with the set of steps to replicate to fix this issue. Thanks in advance.


Added support for third party push sdks

I had to do this fix so my app would work ok whilst using a third party push sdk which intercepts the appdelegate device token received method.  The method is not implemented at all on the appdelegate since the third party sdk does this work itself.  All I needed to do was just add the method implementation without needing to swizzle an original method or forward the the call to the original method.

Im so sorry I missed this!
This in excellent, thanks for helping out!


Introduce U8 wrapper encode usable with encoded_as

this is a wip implementation, that can be finished if ppl think it is interesting.
at the end user will be able to use it as such
#[derive(Debug, PartialEq, Encode, Decode)]
struct A {
	#[codec(encoded_as = "U8")]
	pub i: u8,
}

I just thought about another approach to solve this:
First we need a sealed trait and a special internal parity_codec type that implements it:
mod private {
    pub trait Sealed {}
    pub enum OptIn {}
    impl Sealed for OptInt {}
    impl Sealed for () {}
}

Then we extend parity_codec::Encode and parity_codec::Decode traits by another associated type:
trait Encode {
    type OptIn: private::Sealed + 'static = ();
    ...
}

We also implement parity_codec::{Encode, Decode} for u8 just as we did for all other types with the only difference that we set OptIn type to our new private::OptIn type.
Then we can easily apply @thiolliere 's proposal using TypeId without restricting ourselves to 'static bounds for T since <T as Encode>::OptIn is bound to be 'static.
We are also safe that no other implementations can abuse OptIn from the outside since it is sealed and cannot be implemented from external crates.
Have not tested this yet, might eventually work.

closing in favor of https://github.com/paritytech/parity-codec/pull/83
might be reopen if 83 fails to succeed


update snippet generation for java v3

depends on https://github.com/microsoftgraph/msgraph-sdk-java/milestone/4
https://github.com/microsoftgraph/microsoft-graph-docs/compare/snippet-generation/31436?expand=1

The changes of this generation have been run against raptor here.
https://microsoftgraph.visualstudio.com/Graph Developer Experiences/_build/results?buildId=31836&view=results
It's complaining about the todo stuff as I haven't updated the feature/v3 branch from latest metadata in a while.

disregard, this is not ready yet, I was looking at the wrong PR, sorry about that

oops, this was draft :D

oops, this was draft :D

you're on a roll :D

you're on a roll :D


Weird value when using placeholder

Here's my HTML:
<input type="text" ng-model="form.phoneNumber" placeholder="Phone Number" ui-mask="(999) 999-9999" ui-mask-placeholder ui-mask-placeholder-char class="form-control">

Here's before focus:

Here's when focused:

Here's when typing:

What am I doing wrong? The documentation isn't very clear about how to use placeholders, I just copied the code from the demo.

+1 same problem here

Have you tried updating your version of ui-mask?
This might have been an issue related to #25 that was fixed partially by #27. At least the issues with having empty ui-mask-placeholder.

@lukepfeiffer10 I have the latest version as of 6 days ago. The latest release was 11 days ago.

@sean-hill Can you provide a plunkr that recreates and shows the issue?

Same problem.  Specifically, see code below.  When typing in a number into the phone number mask, if you type numbers 0-8 everything is okay.  It messes up when you type a '9'.  It seems to almost be treating it like an incorrect character and it loses it's place in the mask.
<input class="form-control" ng-model="number" ui-mask='999  ui-mask-placeholder=' 999' type="text">
This is the simplest test you can work with I think

Hey @lukepfeiffer10 I just made a plunkr with only angular and the mask libraries included here. It appears that the bug is fixed is this situation. I wonder if it's another library that may be conflicting. I also have ui.bootstrap included in my project. Maybe that might be an issue?

If the placeholder character is a ‘9’ (even in the demo on github) the problem shows up.
From: Sean Hill<EMAIL_ADDRESS>Sent: Sunday, December 06, 2015 6:36 PM
To: angular-ui/ui-mask<EMAIL_ADDRESS>Cc: tgiacchi<EMAIL_ADDRESS>Subject: Re: [ui-mask] Weird value when using placeholder (#75)
Hey @lukepfeiffer10https://github.com/lukepfeiffer10 I just made a plunkr with only angular and the mask libraries included herehttp://plnkr.co/edit/aAe5SHxgBjEuapA0mxNZ?p=preview. It appears that the bug is fixed is this situation. I wonder if it's another library that may be conflicting. I also have ui.bootstrap included in my project. Maybe that might be an issue?
—
Reply to this email directly or view it on GitHubhttps://github.com/angular-ui/ui-mask/issues/75#issuecomment-162369576.

@tgiacchi you are experiencing the same issue as #32. Here is the plunkr from that issue that should solve your problem, link. However, that solution hasn't been incorporated into a release at the moment.
@sean-hill I can't be 100% certain that including ui.bootstrap is causing your problem although I can say that I am successfully using both ui.mask and ui.bootstrap in the same angular app. Do you pre-populate your model value that is binding to the text box you have the mask on (not sure if this could even be an issue but something to look into perhaps)?


Typo fix ... INFO:root:Collecting info from requested paths (Get opertaion)...

just a minor typo fix ....

Hey @slieberth ,
Thanks for catching that.
Best,
Anton


Fork, Commit, Merge - Medium Issue (Bootstrap CSS)

Fork, Commit, Merge - Medium Issue (Bootstrap CSS)
Implement a Responsive Product Landing Page using Bootstrap
Note: You don't have ask permission to start solving the issue or get assigned, since these issues are supposed to be always open for new contributors. The actions-user bot will reset the file back to previous state for the next contributor after your commit is merged. So you can just simply start working with the issue right away!
Navigate to the tasks/bootstrap/medium directory from the root of the project.
Then open the landing-page.html file to start working with the solution.

Description:
The goal of this issue is to create a responsive product landing page using the Bootstrap framework.
You need to add all the missing classes to make the page look similar as in all the screenshots. Some classes are already added, you don't need to modify those. Also, you don't need to modify any of the HTML or example text provided in the page.
Screenshots:

Requirements:

Must utilize Bootstrap's Grid system for layout
Must be fully responsive and look good on all devices (mobile, tablet, desktop)
In mobile view, the Navbar should open up, when clicking the hamburger-icon
Use Bootstrap components like Navbar, Buttons, Cards, Forms, etc.
Stick to the Bootstrap color scheme or choose a harmonious palette

Bonus:

Add smooth scrolling to the page
Include animations or transitions for a polished look

Resources:
Bootstrap Documentation

If you are using VS Code and have for example Live Server extension, you can simply open landing-page.html with the server and check how the card looks. After the card is done, you are ready to make a pull request!
If you do not have Live Server extension, you'll find information about installing and how to use it here.
Check out README.md for more instructions and how to make a pull request.
Feel free to ask any questions here if you have some problems!
Also, kindly give this project a star to enhance its visibility for new developers!

Hi @nikohoffren, Can you assign me to this issue?

Hi @nikohoffren, Can you assign me to this issue?

Hi! As it says on the description, we don't assign anyone since these educational issues are supposed to be always open, so you can start working with the issue right away.


Content

Add the actual content to the side. Mostly copy paste from the old version.

Adding Space Game

Added all the games, only Blockland remains

Done :laughing:


asdf-controlled versions sometimes show extra information

Current behavior
Prompt sometimes contains extra info when using asdf:
10:02:54 in ~ via ruby v2.6.6    (set by /Users/mlm/.tool-versions)

This is hard to reproduce, sometimes it just starts happen while moving around directories. It randomly goes away as well.
Expected behavior
Prompt should look normal:
10:02:54 in ~ via ruby v2.6.6

Environment
Spaceship version: 3.11.2
Zsh version: 5.8 (x86_64-apple-darwin19.3.0)
Zsh framework: None
Zsh plugin manager: None
Terminal emulator: Alacritty
Operating system: macOS

This bug maybe come from asdf-vm. You may have to restart your shell when you reproduce this bug.
Next time when it reproduce, you can past the output of asdf current ruby  here.
Thank you so much for reporting this.

Just had it happen again, here's the output:
16:28:34 in ~ via ruby v2.6.6    (set by /Users/mlm/.tool-versions) took 2s
[I] ➜ asdf current ruby
2.6.6    (set by /Users/mlm/.tool-versions)


fix(accessibility): read menu options with screenreader when navigati…

Alaska Airlines Pull Request
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
Fixes: #97
Summary:
Please summarize the scope of the changes you have submitted, what the intent of the work is and anything that describes the before/after state of the project.
Added a <div> in the auro-select template that is hidden from the user, allowing a screenreader to read menu options as the user is navigating select's dropdown via keyboard.
Type of change:
Please delete options that are not relevant.

[ ] New capability
[x] Revision of an existing capability
[ ] Infrastructure change (automation, etc.)
[ ] Other (please elaborate)

Checklist:

[x] My update follows the CONTRIBUTING guidelines of this project
[x] I have performed a self-review of my own update

By submitting this Pull Request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Pull Requests will be evaluated by their quality of update and whether it is consistent with the goals and values of this project. Any submission is to be considered a conversation between the submitter and the maintainers of this project and may require changes to your submission.
Thank you for your submission!
-- Auro Design System Team

:tada: This PR is included in version 2.5.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Limiting certain stats

Hello,
on Dragoon you run into (cd alignments) issues (along with tp issues) with too high skill speed (704+), so it would be nice if you could set a limit on it via config.yaml / cmd argument.

I'll do it sometimes.


Error: error executing command: git rev-list HEAD --max-count=1 --abbrev-commit, got exit status 128

Problem with auto install
Download
curl -s https://api.github.com/repos/tmiland/Invidious-Updater/releases/latest \ | grep "browser_download_url.*sh" \ | cut -d : -f 2,3 \ | tr -d \" \ | wget -qi -
run with advanced and without advanced cfg
Error with db
Synchronizing state of postgresql.service with SysV service script with /lib/systemd/systemd-sysv-install. Executing: /lib/systemd/systemd-sysv-install enable postgresql ➜ Creating user kemal with password 6pwABZjJAIb0lXkzjkeJ6aTU7tuV60Yg CREATE ROLE ➜ Creating database invidious with owner kemal CREATE DATABASE ➜ Grant all on database invidious to user kemal GRANT ➜ Running /home/invidious/invidious/config/sql/annotations.sql psql: error: FATAL: Peer authentication failed for user "kemal" ➜ Running /home/invidious/invidious/config/sql/channel_videos.sql psql: error: FATAL: Peer authentication failed for user "kemal" ➜ Running /home/invidious/invidious/config/sql/channels.sql psql: error: FATAL: Peer authentication failed for user "kemal" ➜ Running /home/invidious/invidious/config/sql/nonces.sql psql: error: FATAL: Peer authentication failed for user "kemal" ➜ Running /home/invidious/invidious/config/sql/playlist_videos.sql psql: error: FATAL: Peer authentication failed for user "kemal" ➜ Running /home/invidious/invidious/config/sql/playlists.sql psql: error: FATAL: Peer authentication failed for user "kemal" ➜ Running /home/invidious/invidious/config/sql/session_ids.sql psql: error: FATAL: Peer authentication failed for user "kemal" ➜ Running /home/invidious/invidious/config/sql/users.sql psql: error: FATAL: Peer authentication failed for user "kemal" ➜ Running /home/invidious/invidious/config/sql/videos.sql psql: error: FATAL: Peer authentication failed for user "kemal" ✔ Finished Database section ➜ Updating config.yml with new info...
End with error GIT
`Done checking player dependencies, now compiling Invidious...
fatal: detected dubious ownership in repository at '/home/invidious/invidious'
To add an exception for this directory, call:
git config --global --add safe.directory /home/invidious/invidious
fatal: detected dubious ownership in repository at '/home/invidious/invidious'
To add an exception for this directory, call:
git config --global --add safe.directory /home/invidious/invidious
Showing last frame. Use --error-trace for full trace.
In src/invidious.cr:68:25
68 | CURRENT_COMMIT = {{ "#{git rev-list HEAD --max-count=1 --abbrev-commit.strip}" }}
^
Error: error executing command: git rev-list HEAD --max-count=1 --abbrev-commit, got exit status 128
✗ [ERROR] Build Process Failed!
This is most likely an issue with Invidious, not this script!
➜ Report issue: [https://github.com/iv-org/invidious/issues]
(https://github.com/iv-org/invidious/issues`)`
Tested on Clean Ubuntu 20.04LTS and 22 LTS in virtualbox.

Please see https://github.com/tmiland/Invidious-Updater/issues/88#issuecomment-1193922977


Is ESP32 CSI support MIMO? (AEGHB-111)

As far as I know, CSI is a martrix which dimension is rx_antenna_numtx_antenna_numsubcarrier_num。For example when collecting CSI using intel 5300 card which has three antenna and a router which has two antenna, the CSI data per frame would be a 3230 martrix.
Although ESP32 only support one antenna, is it possible for esp32 to get 1nsubcarrier_num CSI when AP has n antenna work at the same time?

I have the same question too. Can anybody answer for us plz.

ESP32 don't support MIMO sending for ESP32 chips have one antenna per chip. But ESP32 do support receiving MIMO signals.

Hello, @UncleBigLu @N16104872
Closing this issue, feel free to reopen with more updates. Thanks.


Format updates1

@aarmey  here are the updates to figure 1 and figure 2:
Figure 1:

adjusted width
% axis labels
moved the r2x plot to the first plot
added 3rd PC
included all of the tick marks x for r2x
show only 6 components
y axis starts at 0 for r2x

Figure 2:

y axis starts at 0
made titles y axis labels
added subplot labels
adjusted width

@aarmey I'm not sure how to do the formatting for figure 1. Should I move the r2x plot to its own figure ?


adding entropic map notebook

Submission of Lucas Saban and Coni Soret for OT-ENSAE 2023

Codecov Report

Merging #362 (5e515b9) into main (795fabe) will decrease coverage by 0.60%.
The diff coverage is n/a.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files

@@            Coverage Diff             @@
##             main     #362      +/-   ##
==========================================
- Coverage   88.20%   87.61%   -0.60%
==========================================
  Files          51       51
  Lines        5530     5530
  Branches      831      831
==========================================
- Hits         4878     4845      -33
- Misses        532      563      +31
- Partials      120      122       +2

see 2 files with indirect coverage changes

Thanks!
can you fix the linter issues?
you can check first the CONTRIBUTING.md file and install pre-commit, and run it on your files.
Thanks!

Codecov Report

Merging #362 (1310513) into main (795fabe) will decrease coverage by 0.01%.
The diff coverage is n/a.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files

@@            Coverage Diff             @@
##             main     #362      +/-   ##
==========================================
- Coverage   88.20%   88.20%   -0.01%
==========================================
  Files          51       51
  Lines        5530     5529       -1
  Branches      831      567     -264
==========================================
- Hits         4878     4877       -1
  Misses        532      532
  Partials      120      120

see 1 file with indirect coverage changes

Thanks! you will see the grade of the final project soon, or contact me directly to get it.


Users want support for regex path (prefix) matching

Envoy supports this; It could be nice that Ambassador expose it.
Working thougth is a regex_prefix element in a Mapping, parallel to prefix.

@radu-c I saw on Gitter that you might be looking into implementing this?

Yes, I have a working version, I think it doesn't break anything else .. but I didn't manage to get the tests running yesterday due to some environment issues. Will try again later today.


Windows: gofmt checker fixes

Signed-off-by: John Howard<EMAIL_ADDRESS>4 tweaks to make.ps1 in the gofmt check.

It was writing out the name of the temporary file used, rather than the actual file it was checking.
Checking C:\Users\ContainerAdministrator\AppData\Local\Temp\tmpC0F7.tmp
Congratulations!  All Go source files are properly formatted.

With this change you get this instead:
INFO: Validating go formatting on changed files...
Checking cmd\docker\docker.go
Congratulations!  All Go source files are properly formatted.

It writes the filenames using Windows path semantics (\ instead of /) :smile:

It writes the filename before running gofmt, not after. Much more useful!

It actually checks the exit code correctly 😇. That was a cut and paste error previously on my part. Now make.ps1 does actually fail if gofmt checks fail.

@johnstep

@johnstep would you be opposed to carry this or merge and send follow-up PR with proposed fixes?
Thanks!

@jhowardmsft do you have time to fix nits from @johnstep ?

@johnstep Updated. PTAL.


Expand image choices

e.g. select specify ID, or collection or user.

Just update the reload process - wait for submission in case of ID.
Possibly allow switching with radio buttons between multiple options like droplist for collection and one for popular and one for ID.


Orthogonal viewer plugin (simultaneous orthogonal views)

I first want to congratulate the napari devs for their great work on this package. It is very impressive!
I work in medical imaging and often collaborate with @jni on scikit-image, but for some reason had not gotten around to trying napari. I tried it out yesterday on some large (~16GB) 3D data and browsing through slices was very smooth! Sliding and zooming the volume worked well. I think the only mouse-control I am used to having in other applications that I didn't see was the ability to adjust contrast and window level by dragging the mouse in combination with one of the mouse buttons (I did see there is a slider for it in the left panel).
As you may be aware, there are quite a few good standalone viewers/tools developed in C++ for medical imaging applications (e.g. freeview, FSLeyes, 3D Slicer, Medinria, etc.), but I am excited to see something in the Python space. It is also great that this viewer can be called directly on general NumPy and Dask arrays as opposed to requiring specific medical imaging file formats.
🚀 Feature

I found the following duplicate issue in an archived repository: https://github.com/napari/napari-core/issues/4,
Do you know if there is anyone working on a plugin that would show simultaneous orthogonal views of 3D data (e.g. similar to this screenshot from 3D slicer, although as a first step it could exclude the 4th panel with the 3D rendering)? Another option sometimes used for a 4th panel is to show the time-course of the selected 3D spatial position when the data is 3D + time.
Perhaps this is already possible today? I have only just started looking at napari. I saw that you can already switch between the orthogonal views in the main window, but it would be nice to be able to see them side by side and have the spatial positions linked across the views.
Motivation
nibabel provides a basic 3D orthogonal viewer built using Matplotlib, but Napari could potentially produce something much nicer.

Pitch
A key feature is that clicking on a specific voxel in one view, updates the other two orthogonal views to show the slice at the selected location. Often there is an (optional) crosshair overlaid on each view to show the current location across views.

Alternatives
I think the main argument against developing this is that there are already good non-Python tools that do this. I think there is still an argument that Napari could provide similar functionality, but in a less domain-specific manner and with the flexibility to combine interactively with other Python image processing tools.

Additional context

I don't forsee having time to work on this personally in the immediate future, but if there is interest from the team here, it could be worth advertising Napari on the Neuroimaging Analysis in Python mailing list to help find new users and potential contributors.

Thanks @grlee77!  Glad you've been finding it useful.  Orthoviews are definitely a very high priority for us (see also #760 and #662) and agree that tons of people could use that functionality.   It probably could be cobbled together with the current codebase, but we have a few deeper architectural things we need to shore up that would make it much more natural.  Thanks for bumping the issue and providing the useful context.

Yes, as @tlambert03 said, it's definitely on our priority list, but we'll probably in a better position after things like #1353 merge too and we clean up our slicing model. I'll be very excited when we having this functionality too.

if there is interest from the team here, it could be worth advertising Napari on the Neuroimaging Analysis in Python mailing list to help find new users and potential contributors.

Thanks, that sounds like a great idea, we're always interested in new user and contributors and there are certainly some neuroimaging specific file IO plugins that community might be interested in working on right now as we prepare for ortho views!!

Hello everyone,
An orthogonal viewer would be very useful for some of my users and I wanted to inquire what the status/timeline is of this feature? Are there still some larger underlying issues to be resolved first or would it be possible to start working on this already? I would also be able to contribute some developer time to get this done if necessary. However if the underlying code is undergoing heavy refactoring I would not start yet.

Hi @tibuch so exciting you're willing to help out - there are some underlying issues that need to be sorted out around our layer/ data model - but we're actually in the process of trying to set up a working group that would focus on this area and that you'd be more than welcome to join!
Not sure if you're in the napari zulip https://napari.zulipchat.com/login/ yet, but maybe you join that and i can send you some details for our developer/ community meetings. We've got a couple, so depending on what timezone you're in you can decide which one you'd like to join and we can pick things up from there


Add sensors to opposite roads

About
The current implementation has sensors on only adjacent roads, and not the ones opposite to them. (See simulation file). Add 2 sensors on each road and add them up to get a net traffic level from opposite sides. Then use this traffic level data to vary the signals. This problem is difficult to explain in an issue, so pls approach the maintainer for a more comprehensive explanation
Bounty 140

need this issue pls

@vicks6969 its been over 30 mins, please open a draft pr,

@vicks6969 deassigning you because of lack of response and updates

This issue is now open for others to work on

I would like to try

@LalithaTanmayee  you have been assigned, please open a pr in 30 mins to show your progress.

not working on this, assign to the next person please

can i get assigned if this is still open?

can i get assigned if its open

@Ritwik-Ag you can start working on this issue, please approach the maintainer or open a draft PR to show progess!

@Ritwik-Ag  you have been un-assigned, please approach the maintainer for any queries!

this issue is open to assignments!

can i get assigned to this??

@KunalMaverick you can start working on this, please do give an update or open a draft PR in 30 mins!

i would like to un-assign myself from this


Number of occurrences for dagvlinders incorrect on GBIF

The source dataset has 761.660 fully georeferenced records: http://data.inbo.be/ipt/resource?r=dagvlinders-inbo-occurrences
The dataset page on GBIF indicates:

761.650 records (10 less)
706,013 georeferenced records (55.637 less)

It is not clear why.

Number of occurrences has now been fixed. Number of georeferenced occurrences will be addressed in a separate issue.


⚠ Bofrost.fr has degraded performance

In 7dc07da, Bofrost.fr (https://www.bofrost.fr) experienced degraded performance:

HTTP code: 200
Response time: 1756 ms

Resolved: Bofrost.fr performance has improved in 2f8249c.


Properly mark proc used in call credentials for garbage collection

Fix a bug reported in the mailing list about long-lived call credentials.

Hey @murgatroid99! I tried these changes and we continue to see the updater_proc object being replaced. Do you want to do a hangout and I can show you our setup?

FWIW, I suspect line 266 is the cause:
plugin.state = (void*)proc;

Are you using any credentials compose functions? I just realized that I didn't fix the garbage collection marking for those.

Yes, we are using compose. Here is our code to create the credentials. The client object is a Google::Auth/Signet object.
creds = GRPC::Core::ChannelCredentials.new.compose \
  GRPC::Core::CallCredentials.new client.updater_proc

OK, I think that last commit should fix it.

Unfortunately it does not fix it. With my change to grpc_rb_call_credentials_callback_rescue attempts to call the credentials print the following debug message:
E0314 14:16:29.451012000<PHONE_NUMBER>92256 credentials.c:1162] Getting metadata from plugin failed with error: undefined method `call' for #<Hash:0x007fce53d99990>
E0314 14:16:29.451023000<PHONE_NUMBER>92256 client_auth_filter.c:93] Client side authentication failure: Credentials failed to get metadata.

I think I see the problem now. The credentials objects are keeping the proc alive, but the call and channel aren't keeping the credentials objects alive.

You sure it isn't because the proc object is dereferenced and the pointer to the proc is passed in on the plugin? When ruby moves the location of the proc object that pointer won't be updated and will point to whatever ruby puts there. That's how it reads to me, but I admit I haven't been able to crack this problem yet. :)

If Ruby ever moved objects around in memory, then there would be no way to ever hold a reference to a Ruby object in C, and rb_gc_mark would be meaningless.
I think that what you're seeing is Ruby objects getting garbage collected, and the pointer pointing to new objects allocated in the same location (i.e. use after free).

So I got the tests to pass with the following:
diff --git a/src/ruby/ext/grpc/rb_call_credentials.c b/src/ruby/ext/grpc/rb_call_credentials.c
index 7c4b69f..0baaf4d 100644
--- a/src/ruby/ext/grpc/rb_call_credentials.c
+++ b/src/ruby/ext/grpc/rb_call_credentials.c
@@ -68,6 +68,10 @@ typedef struct callback_params {
   grpc_credentials_plugin_metadata_cb callback;
 } callback_params;

+typedef struct grpc_rb_proc {
+  VALUE proc;
+} grpc_rb_proc;
+
 static VALUE grpc_rb_call_credentials_callback(VALUE callback_args) {
   VALUE result = rb_hash_new();
   VALUE metadata = rb_funcall(rb_ary_entry(callback_args, 0), rb_intern("call"),
@@ -88,7 +92,8 @@ static VALUE grpc_rb_call_credentials_callback_rescue(VALUE args,
   rb_hash_aset(result, rb_str_new2("status"),
                INT2NUM(GRPC_STATUS_PERMISSION_DENIED));
   rb_hash_aset(result, rb_str_new2("details"),
-               rb_str_new2(rb_obj_classname(exception_object)));
+               //  rb_str_new2(rb_obj_classname(exception_object)));
+               rb_funcall(exception_object, rb_intern("to_s"), 0));
   return result;
 }

@@ -125,7 +130,7 @@ static void grpc_rb_call_credentials_plugin_get_metadata(
     void *state, grpc_auth_metadata_context context,
     grpc_credentials_plugin_metadata_cb cb, void *user_data) {
   callback_params *params = gpr_malloc(sizeof(callback_params));
-  params->get_metadata = (VALUE)state;
+  params->get_metadata = ((grpc_rb_proc*)state)->proc;
   params->context = context;
   params->user_data = user_data;
   params->callback = cb;
@@ -242,9 +247,12 @@ static VALUE grpc_rb_call_credentials_init(VALUE self, VALUE proc) {
   grpc_rb_call_credentials *wrapper = NULL;
   grpc_call_credentials *creds = NULL;
   grpc_metadata_credentials_plugin plugin;
+  grpc_rb_proc *proc_wrapper = gpr_malloc(sizeof(grpc_rb_proc));

   TypedData_Get_Struct(self, grpc_rb_call_credentials,
                        &grpc_rb_call_credentials_data_type, wrapper);
+  proc_wrapper->proc = proc;
+  rb_gc_mark(proc_wrapper->proc);

   plugin.get_metadata = grpc_rb_call_credentials_plugin_get_metadata;
   plugin.destroy = grpc_rb_call_credentials_plugin_destroy;
@@ -252,7 +260,7 @@ static VALUE grpc_rb_call_credentials_init(VALUE self, VALUE proc) {
     rb_raise(rb_eTypeError, "Argument to CallCredentials#new must be a proc");
     return Qnil;
   }
-  plugin.state = (void*)proc;
+  plugin.state = (void*)proc_wrapper;
   plugin.type = "";

   creds = grpc_metadata_credentials_create_from_plugin(plugin, NULL);

Essentially passing the pointer to a struct that holds the proc object, instead of the proc object. Now the proc can be moved in memory and the call credentials can keep a hold of it.

Ruby definitely moves objects around in memory. That's the whole point of a generational GC. In my patch a struct is created to hold the reference to the VALUE, and the location of the struct isn't going to be touched by ruby's GC, so a pointer to it is safe to pass to GRPC.

Oh, my addition of rb_funcall(exception_object, rb_intern("to_s"), 0)); isn't needed for my patch, that just helps me see what method is being called on what object. I do think something like that would be more useful for debug messages, but isn't needed for the patch to work.

@murgatroid99 I sent you a PR (murgatroid99/grpc#2) to add my changes to this PR. I'm running our acceptance tests over and over with it and haven't had a single failure yet.

I don't see how that could be the problem. A VALUE is just a number (a uintptr_t on most systems), and in some cases it is actually cast to a pointer. Casting it to a pointer and then back should not be able to cause the kind of failures we are seeing.

I've got multiple successful test runs that show that my patch this does address the problem. Also, if I don't include the call to rb_gc_mark(proc_wrapper->proc); in my patch then it doesn't work.
I'm looking for a URL that explains how the ruby GC works, but I'm not finding what I'm looking for. Short answer is I don't know what specific mechanism ruby uses to keep the VALUE pointing to the correct location in memory. But all evidence points to this being the cause of the errors and segfaults.

I'm pretty sure that the change that you just made only actually worked because you called rb_gc_mark(proc_wrapper->proc) in the initialization function, removing it from garbage collection forever. The VALUE to struct change was orthogonal to that, and probably didn't make a difference.
I think the change I just made should properly fill out the missing parts of the reference graph needed to keep the credentials references around long enough.

Gotcha. My change to mark the proc object makes sense if the problem is that that it was garbage collected. I've pulled your latest change and our tests are passing! So this does seem to solve our issue. I'll run more tests over night just to make sure.
Thanks!

Great. I'm glad I could help.

This is fantastic! Thank you @murgatroid99 and @blowmage for working through this!!!

We've run several test runs using the code in this PR and had no problems at all. :+1: for days.

Ping

@nicolasnoble I would love to see this accepted and released soon.

@murgatroid99 and @nicolasnoble Please excuse me if this is published elsewhere, but is there a date for the next release that would contain this fix?

We expect the release to be early next week.

@murgatroid99 Fantastic!!! Thank you!

@jtattermusch C# code coverage failed ?

Otherwise, LGTM.

coverage failure is a known flake.

@jtattermusch Thank you! Can we still expect the release to be early this week?
/cc @jgeewax

@quartzmo  it is our plan to release soon (hopefully this week, early next week the latest).
@murgatroid99, can you please backport this change to 0.13 branch?

Great, thank you everyone!

@jtattermusch @murgatroid99 We pinned our build to the 0.13.1.pre1 release and everything works both locally and on our CI. So far we've had no issues with 0.13.1.pre1 and our confidence in it is high. :heart: :heart: :heart: :tada: :fireworks: :boom:
We are right up to the end of Q1, and we'd love to push a release out before then. Is it possible to get 0.13.1 released in the next couple days?

I expect that we will release 0.13.1 very soon.

@murgatroid99 Thank you!

We just merged the gcloud-ruby update to grpc 0.13.1 and our acceptance tests passed, thank you!

Thanks everyone!

@murgatroid99 We released gcloud-ruby v0.7.0 today with runtime dependency grpc ~> 0.13.1.


Wrong total calculation

Hi team,
I noticed that the total net worth calculation on farm.army is wrong. It is adding the total in vaults including the profits, and then adding the profits again. I think the "vaults total" should not include accrued profits.
Please see the following screen from a random 0x :

Thanks in advance for your awesome work :)

Thanks to whoever looked into it, it has been fixed ! :)


Saving at 100% quality produces corrupted colors (in some other apps)

I haven't tested this myself yet, but looks important:
https://blog.getpaint.net/2020/08/25/paint-net-4-2-14-alpha-build-7542/#comment-21610

It is a known bug in with some AVIF decoders, they do not correctly handle images that use the CICP Identity matrix and treat those images if they use YUV.
Unfortunately the OP did not specify what decoder they are using, but both Microsoft's HEIF Image Extensions (release date 1/29/2018) and Firefox (version 80.0) have this issue.
The images are decoded correctly in Chrome (version 85.0).
The CICP Identity matrix is a color encoding that allows RGB colors to be used without conversion to the YUV color space, this allows for fully lossless RGB encoding.
libavif has the same issue which provides more background information, https://github.com/AOMediaCodec/libavif/issues/243.

At least XnView MP has this issue, I can see it in thumbnails too but I'm not sure if XnView is the provider for those or something else. Seems to be using libavif (c) 2019. I guess this just needs to be fixed in those affected decoders then. I assume that because 100% is lossless RGB then the chroma subsampling has no effect, should this be made clear in the UI? Currently it is possible to change the setting at 100% without it having any effect.

I assume that because 100% is lossless RGB then the chroma subsampling has no effect, should this be made clear in the UI? Currently it is possible to change the setting at 100% without it having any effect.

I could make the chroma subsampling setting read only when the quality setting is set to 100, but the chroma subsampling is also ignored for monochrome images and paint.net does not currently allow FileType plugins to configure UI based on the image content.
It would be confusing if the UI had different behavior for those two cases.

The fact that the preview in Paint.NET doesn't show corrupted colors is further proof that the bug actually lies elsewhere -- the preview is powered by saving to a temp file and then loading it back.

I have to admit that I have not looked at the preview even once because it takes so long to load(was at normal/medium by default though), a general problem with AV1/AVIF I know. Would it be too fancy to only make a preview of a smaller portion of the image if the image is over certain size?
Took a very long time to preview a 18 MP image from my camera at 100% slow(I know slow is slow but the time it took is absurd). I've had my pc just melt trying to encode AV1 in general. Not sure what kind of logic Paint.net allows with these.
Actually assuming the preview is full size, couldn't Paint.net just save that preview if you save the image when the preview is done. Right now after I wait for a preview, possibly for a long while, after I save it it'll just take as long to actually save.
Sorry I'm getting slightly off topic. I think what is happening is definitely some decoders(or their specific use) assuming the image is YUV when it in fact is RGB, the resulting colours also look something like that. Like changing your monitor's color space to YUV when the input is actually RGB, looks similar.

I think what is happening is definitely some decoders(or their specific use) assuming the image is YUV when it in fact is RGB, the resulting colours also look something like that.

That is what would be happening when the quality is at 100%.

Actually interestingly enough XnView MP shows properly in the image properties that the images are RGB lossless, however their ExifTool shows YUV 4:4:4.

Closing this issue as it is not a bug in the plugin.


Fixing the problem of converting numbers to Persian and Arabic (64 to ۶۴)

When the Windows region settings are set to Persian or Arabic, the .\mvnw compile jib:build command encounters an error.
The error message is as follows:
[ERROR] : Illegal repetition near index 251
[ERROR] : ^(?:((?:[a-zA-Z\d]|(?:[a-zA-Z\d][a-zA-Z\d-]*[a-zA-Z\d]))(?:\.(?:[a-zA-Z\d]|(?:[a-zA-Z\d][a-zA-Z\d-]*[a-zA-Z\d])))*(?::\d+)?)/)?((?:[a-z\d]+(?:(?:[_.]|__|-+)[a-z\d]+)*/)*[a-z\d]+(?:(?:[_.]|__|-+)[a-z\d]+)*)(?::([\w][\w.-]{0,127}))?(?:@(sha256:[a-f0-9]{۶۴}))?$

Error because number 64 convert to Persian or Arabic number character "۶۴"
My change will fix #3939

Thanks for putting up a fix @sheikhoo! However, I believe the use of %d is intentional here as we are expecting an integer value (HASH_LENGTH). %s on the other hand, expects a pointer to a String and modifying %d to %s can lead to unintended results.

@mpeddada1 What do you think about using the Locale.US parameter in the String.format method? It allows to format an integer value using English characters and also fixes the problem of the regional settings affecting the formatting of numbers.
I have made changes and pushed a new commit to the pull request.