Add the ability for users to pin Calculator on top of other windows

Problem Statement
When attempting to use Calculator with other application, such as a spreadsheet, Calculator will lose focus when you click on the other app (e.g. to scroll/copy). Users have requested the ability to pin Calculator on top of other windows to facilitate ongoing use with other apps.
Evidence or User Insights
This is our second-highest customer feedback request in Feedback Hub with some users reporting their only alternative is to pay for an app which is sticky.
Proposal
Add the ability for users to pin Calculator on top of other windows.
Goals

Users can easily pin/unpin the Calculator window on top
Users can easily continue with other tasks while Calculator is pinned
Users have access to full Calculator functionality, but can always perform at least basic calculations when in the more compact view (smaller than currently allowed minimum window sizing)

Non-Goals

Parity with the full version of the app when in smallest of layout states

Low-Fidelity Concept
Here is a quick concept showing what the smallest of modes might look like. There are still several open questions to address for this layout state as well as determining the best way to enter/exit this mode:

Would this use Compact Overlay mode then?

This is awesome, I was planning to to this for a lot of apps

Yes, compact overlay would be great, I currently have a calculator app that does this but when calculator gets this it will take its place.

I suggest this: Once it's pinned, it could still be controlled by number pad. Even it's not focused.

@steevegldev I don't think that would be good when using it to calculate numbers in say a word document, if the mouse is focused on the document the user will not want numbers being typed in to it.

About that specific example. Why would you use number pad to type numbers in Word. You naturally use the one above the letter. This how you learn typing on keyboard.
But maybe it could be a separate feature. I know keyboard with real calculator embedded exist.

Oh, I see now that you wrote the number pad. I agree this would be a neat trick for pro users.

Can someone please make the num keys has the same colors as the whole other keys? I just cannot tolerate it.

I really hope that "always on top" will be added. I don't need any other improvement than this.

I feel like windows itself needs a "pin on top" option for all windows

Ubuntu OS have such pin on top option. It would be good if windows too got that option by default.

I usually use DeskPins for this, but it doesn't seem to work with this version of Calculator.
I agree with @Kurtoid – this functionality shouldn't be limited to the Calculator application.

Of course it should be on other apps too but this repository is for calculator....

I've a question about CompactOverlay. Do we plan to use it because:

It's the only UWP API available but we would prefer a real way to pin the window on top
By design, we prefer a very tiny window.

In my opinion, the compactOverlay mode is perfect for videos but for the calculator, the window is too small in this mode.
If the answer is 1, I can submit this week a PR adding a toggle to pin/unpin a UWP window without changing it's size and still giving access to all features.

Using CompactOverlay is by design and by choice. Among other reasons, where possible, we want to let the system make windowing decisions for us.

I can submit this week a PR adding a toggle to pin/unpin a UWP window

Most issues that we have been addressing so far are defects (i.e., bugs), but we follow a slightly different process for new features and tend to do a bit more upfront background research/design work before submitting PRs. We'll have more to share on this soon, as there have already been a ton of great ideas submitted, and we would like to start moving forward with some of them!

the solution is hacky but it only uses fully supported APIs.

I am sure there are more creative ways to implement this, though we tend to avoid these whenever possible due to support requirements 🙂

I am sure there are more creative ways to implement this, though we tend to avoid these whenever possible due to support requirements 🙂

To be honest, it works well but is probably a little too creative (and limited to Desktop). If the Calculator app can motivate the platform team to allow developers to set a minimum size for the CompactOverlay mode, it will be a great! I look forward to seeing what you're preparing!

Even better, if the On Top Windows can launch at specific coordinates.

Microsoft can just add the permission like Android "Drawing over other apps" and current Maximum size could solve some malicious issue.
Maximum size must be permit explicitly and carefully etc...maybe only up till 45% of screeen.

Feature request: Launch at specific coordinates
Microsoft might be careful thought up how Drawing over other app is going to have bad malicious effect like: Someone overlay password box on screen at specific coordinates. It can be solve as Microsoft highlight the Compact Overlay App's Windows while it get focus or even typing (But it is UX problem for certain case). Sometimes people can be mind wandering can focus only less than 500px x 500px, and the highlight border might get overlook 1% over the whole time.

I created calculator-specs/alwaysOnTop to track progress. For more information on next steps, check out our spec workflow.


docs: Migrates content from dagger/examples repository with history

This commit migrates the code examples from the https://github.com/dagger/examples repository to the current repository, preserving the Git history of the original repository.

Examples are now in a new examples/ directory in the repository root.
SDK examples are now in examples/sdk/[go|nodejs|python]/*.
CLI examples are now in examples/cli/*.

This commit updates the README file in the examples/ directory with new file paths to the available examples.
This commit deletes the LICENSE file from the migrated https://github.com/dagger/examples repository to avoid having two LICENSE files in the current repository. Note that both the current and migrated repositories use the same license terms ie. Apache License 2.0.
The intent is for new examples to be added to the current repository instead of the https://github.com/dagger/examples, which will shortly be deprecated. Also see https://github.com/dagger/examples/pull/36

@kpenfound @jpadams there are some commits in the original examples repo with missing DCO, which is causing the DCO test to fail. I'm not sure if it's possible to re-sign those or if doing this will rewrite the history of the merged repo. Please let me know how you would like to handle this.
As a separate note, I guess we should NOT "squash and merge" this PR like we normally do, as we want to retain the complete commit history of the merged repo? @gerhard for his thoughts.

@kpenfound @jpadams there are some commits in the original examples repo with missing DCO, which is causing the DCO test to fail. I'm not sure if it's possible to re-sign those or if doing this will rewrite the history of the merged repo. Please let me know how you would like to handle this. Also @samalba for thoughts.
As a separate note, I guess we should NOT "squash and merge" this PR like we normally do, as we want to retain the complete commit history of the merged repo? @gerhard @samalba for thoughts.

For the DCO, we need to either have them sign/amend the commits (I see @jkodroff and @kpenfound with missing signatures) - or remove the commits from the rebase (the changes can be added later I guess).
No problem to squash the history, looks like it would simplify a few things.

@kpenfound @jpadams there are some commits in the original examples repo with missing DCO, which is causing the DCO test to fail. I'm not sure if it's possible to re-sign those or if doing this will rewrite the history of the merged repo. Please let me know how you would like to handle this. Also @samalba for thoughts.
As a separate note, I guess we should NOT "squash and merge" this PR like we normally do, as we want to retain the complete commit history of the merged repo? @gerhard @samalba for thoughts.

For the DCO, we need to either have them sign/amend the commits (I see @jkodroff and @kpenfound with missing signatures) - or remove the commits from the rebase (the changes can be added later I guess).

@jpadams which would you prefer?

No problem to squash the history, looks like it would simplify a few things.

I agree. However my understanding from @jpadams was that we should retain the history for licensing/legal reasons as it includes at least one external contribution.

@kpenfound to move this forward, could you try signing your commits with missing DCO and if that works, then we can let @jkodroff know and he can do the same?

@vikram-dagger I think there's somehow an issue with how the git history was pulled over with DCO, because all but one of those failing commits were properly signed in dagger/examples

@vikram-dagger I think there's somehow an issue with how the git history was pulled over with DCO, because all but one of those failing commits were properly signed in dagger/examples

Hmm, that's odd. This is the process I followed: https://stackoverflow.com/questions/40640733/git-move-repository-to-another-repository-as-a-directory. Could you please have a look and let me know if you can suggest anything to do differently? Also asking @marcosnils @gerhard to add their eyes to it.

@vikram-dagger I think there's somehow an issue with how the git history was pulled over with DCO, because all but one of those failing commits were properly signed in dagger/examples

@kpenfound it looks like there are two unsigned ones: 3c7641f and 33de315. They are unsigned in the dagger/examples repository as well. The other failing commits have a DCO but there seems to be a name mismatch, Kyle Penfound vs kpenfound.
Leaving aside the commits with name mismatch, is there a way for you to sign the unsigned ones and we can then check if that is sufficient for the DCO to pass?

Disabled Require linear history for the purpose of merging this PR via a merge commit so that history is preserved, as @vikram-dagger intended. cc @aluzzardi @samalba


Added "natural behavior" option to turn light on/off if brightness changes

Description:
added option for recently discussed on/off behavior of dimmable mqtt lights. If a brightness level > 0 is received, light will be turned on and turned off if the level is 0.
The new behavior can be turned on by configuration and defaults to off to not break current behavior.
I'll update the documentation after successful merge.
Related issue (if applicable): #
https://github.com/home-assistant/home-assistant/issues/1494
Example entry for configuration.yaml (if applicable):
additional option in mqtt light config:
natural_behavior: True | False (defaults to: False)

Checklist:
can't get tox to run. ERROR: toxini file 'tox.ini' not found.
but changes are small and already tested and running on my system.
If code communicates with devices:

[?] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[n] New dependencies have been added to the REQUIREMENTS variable (example).
[n] New dependencies are only imported inside functions that use them (example).
[n] New dependencies have been added to requirements_all.txt by running script/gen_requirements_all.py.
[n] New files were added to .coveragerc.

If the code does not interact with devices:

[?] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[n] Tests have been added to verify that the new code works.

This PR seems to have gone stale. Closing it. You can reopen it when you're ready to finish it.


PR for Issue #89: "Ensure CUPiD is generalizable and portable to other machines"

This pull request addresses issue #89 and contains changes to the config_f.cam6_3_119.FLTHIST_ne30.r328_gamma0.33_soae.001.yaml file at path "examples/nblibrary/config_f.cam6_3_119.FLTHIST_ne30.r328_gamma0.33_soae.001.yaml".
I've removed hardcoded references to glade/ and added global variables to the top of the file and changed to relative file paths throughout the rest of the file. (Note: users should change the start and end year variables to match their cesm output files).

Hi @shivaniikum , thanks for getting this PR started, and nice work finding all of these glade paths.
One main thing that I'm noticing is that we want to generalize how we read in files while also maintaining the functionality of pointing to the current paths.
So, I think it would be useful to set a root directory in a config.yml file parallel to the examples/coupled_model directory (rather than the one in nblibrary), and use this to determine the full paths in config_f.cam6_3_119.FLTHIST_ne30.r328_gamma0.33_soae.001.yaml For instance, the root_directory variable could be something like ../local_cesm_output/ in your config.yml file, and something like /glade/campaign/cesm/development/cross-wg/diagnostic_framework/ in the examples/coupled_model config file.
@mnlevy1981 do you want to add anything else to this suggestion?

Nothing to add to your specific suggestion, but I also wanted to ask @shivaniikum to merge the latest main into this branch since it was started before #91 was merged it

Actually, one thing to add - we're okay with modifying the directory structure under /glade/campaign/cesm/development/cross-wg/diagnostic_framework/ - so if setting a root directory in examples/coupled_model/config.yml results in slightly different paths being passed to ADF that's fine. At some point in the future we'll decide on a file structure assuming the root directory is a path generated by CESM, and then we'll make sure any existing examples fit, but until then the directory structure we talked about last week is preferable to what currently exists on the NCAR machine.

Two more requests before digging in for an actual review:

It looks like your config.yml file that gets data from examples/local_cesm_output/ hasn't been added to this PR yet (if the decision is to make it examples/local_cesm_output/config.yml, then your root directory could be ./ instead of ../local_cesm_output)
Could you please provide a small README in addition to config.yml describing how to get the necessary data from the web?


Just installed Kit on Windows 11 Arm on a Parallels VM.

Error: spawn C:\Users\jefwes\AppData\Local\Programs\kit\resources\fallbacks\windows\clipboard_x86_64.exe ENOENT
    at n (C:\Users\jefwes\AppData\Local\Programs\kit\resources\app.asar\main.prod.js:2:452217)
    at r (C:\Users\jefwes\AppData\Local\Programs\kit\resources\app.asar\main.prod.js:2:452388)
    at ChildProcess.e.emit (C:\Users\jefwes\AppData\Local\Programs\kit\resources\app.asar\main.prod.js:2:452529)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)

Kit 1.32.8
Electron 19.0.16
win32 10.0.22623
Locale: en-US
Let me know what I can do to help resolve the issue.

I haven't done any testing on Windows arm or VMs. I don't really have the bandwidth to dig into them for awhile unfortunately

I'll do some digging around and see if I can maybe see what's going on. If I come up with anything I will let you know. Thanks!

I'm 90% sure I'd have to publish a Windows arm build for it to even have a chance of working...

@johnlindquist Check electron 22, it might have updated how you publish stuff to ARM, also the build system changed completely.

I have the same error during the install, but on a "normal" x86 Windows Machine:
Unhandled Promise Rejection
Error: spawn C:/Users/Leo/AppData/Local/Programs/kit/resources/fallbacks/windows/clipboard_x86_64.exe ENOENT
    at r (C:/Users/Leo/AppData/Local/Programs/kit/resources/app.asar/main.prod.js:2:543688)
    at n (C:/Users/Leo/AppData/Local/Programs/kit/resources/app.asar/main.prod.js:2:543859)
    at ChildProcess.e.emit (C:/Users/Leo/AppData/Local/Programs/kit/resources/app.asar/main.prod.js:2:544000)
    at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)

After dismissing the error, the installer is stuck at "kenv .env found".

I have the same issue as ecrax

@tbogard It's on Electron 22 on the latest build. There's an arm build here (https://github.com/johnlindquist/kitapp/releases/download/v1.40.62/Kit-Windows-1.40.62-arm64.exe), but I don't have a machine to test it on:
https://github.com/johnlindquist/kitapp/releases/tag/v1.40.62
@ecrax @gabrielcarloto
Can you please try this fix and see if it works for you?
https://github.com/johnlindquist/kit/issues/984#issuecomment-1379993381

@johnlindquist it worked 🎉 thank you

@gabrielcarloto Awesome! I'm curious why it's a different behavior/permissions for different people.
Why the spawn error?
The setup process is trying copy the real error to clipboard, but failing. So it's a secondary issue that masks the first. (This is a Windows only issue that I simply never ran into)
The real error
Script Kit bundles a node version that matches the Electron node version and sets it up under ~/.knode. This is by unzipping node, then renaming the dir. The error is happening here:
https://github.com/johnlindquist/kitapp/blob/main/src/main.dev.ts#L897
I need to find a way to reproduce to confirm a fix. I think I can chmod the dir before trying to rename it, but won't know until I try it.

@tbogard It's on Electron 22 on the latest build. There's an arm build here (https://github.com/johnlindquist/kitapp/releases/download/v1.40.62/Kit-Windows-1.40.62-arm64.exe), but I don't have a machine to test it on:
https://github.com/johnlindquist/kitapp/releases/tag/v1.40.62
@ecrax @gabrielcarloto
Can you please try this fix and see if it works for you?
https://github.com/johnlindquist/kit/issues/984#issuecomment-1379993381

I will try it as soon as I can (propably tomorrow :/)
Thanks for your work :)


Update tests

Update tests for djangocms-picture>=2.0 and djangocms-link>=2.0

Coverage remained the same at 82.486% when pulling 4455e37922b97b139c2611b72af6ae1e2723cd85 on jvamvas:master into ebcd3039f4db2ea79c9a0f714b515dceff9ed6d3 on divio:master.

Thanks! @jvamvas


Update sendMessage

Using embeds = @[embed] to compatible with latest dimscord

Thank you for updating the code, will merge this in


[v1.8.2][two-motor tailsitter] position control failed

Hello everyone:
I tried to start position mode on my vtol for the first time today.
I found that in the position mode, the drone's roll angle keeps increasing, I blocked this error switching the altitude mode.
Can anyone provide some advice? Is it a position-speed loop pid parameter problem,or wrong state estimate (I observed that the drone x direction does not have a 4m movement shown in the log), or something else?
I know this is a version a year ago, but I also hope to get help, thank you. :smiley:
log link: https://review.px4.io/plot_app?log=e51dfa7d-c25d-4b97-ba6c-d67a2a11c23c

EDIT: I can control the drone attitude in altitude mode. Why does the attitude control fail in position mode?

I found that gps raw data drifts heavily on the ground and at rest.
log link: https://review.px4.io/plot_app?log=64bd160a-d832-471d-8afe-a67255057d15

But I don't understand why the attitude control failed, even if the gps data is seriously drifting.

@julianoes   Maybe, px4 is not supporting 1.8.2?  :smiley:


remove erroneously checked in file

Removes a file that was not intended to be checked in

Just merging this since I accidentally checked it in. It was in the testing directory and was not used by any tests. All tests still run.


"provider status" command failing with "Unregistered interface error'"

If I run make provider-status in the examples it invokes make provider status and I get a message about "Unregistered interface error'. I added some debugging code and was able to determine this comes from this code
 if err = cctx.PrintOutputLegacy(result); err != nil {
   return err
  }

It looks like the result is retrieved correctly in this command but the PrintOutputLegacy command doesn't know how to display it?
I added some debugging code with fmt.Printf and it looks like it is valid
result of gclient.Status: &{0xc0000a2060 0xc00102e7e0 0xc00102e7e4}
cluster: &{0 {[] [{units:<val:"100" >  quantity:<val:"16777216" >  quantity:<val:"134217728" > }] [{units:<val:"7806" > attributes:<key:"arch" value:"amd64" >  quantity:<val:"14789169152" >  quantity:<val:"269167448064" > }] <nil>}}
bidengine: &{0}
manifest: &{1}

So the fix for this should just be to find a better way of printing this.

Need to investigate replacing this with PrintOutput if possible.

Example output after converting to JSON
{
  "cluster": {
    "leases": 0,
    "inventory": {
      "active": null,
      "pending": null,
      "available": [
        {
          "cpu": {
            "units": {
              "val": "7808"
            },
            "attributes": [
              {
                "key": "arch",
                "value": "amd64"
              }
            ]
          },
          "memory": {
            "size": {
              "val": "15585153024"
            }
          },
          "storage": {
            "size": {
              "val": "269167448064"
            }
          }
        }
      ],
      "error": null
    }
  },
  "bidengine": {
    "orders": 0
  },
  "manifest": {
    "deployments": 0
  }
}

This is fixed, right?


Gh 76

Docs Build status updates of commit 25d0bc9:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

docs/Xamarin.Forms/GestureElement.xml
:white_check_mark:Succeeded

docs/Xamarin.Forms/SearchBar.xml
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Ignores column value save, if the column name has underscore in it.

This issue is caused by the limitation of Google Sheets API v3. To fix some issues included this one I'll migrate to Google Sheets API v4 in the next release.

@pradeepmantha This issues was fixed at 0.4.0


proposal: Replace Context with goroutine-local storage

This is a bit pie-in-the sky, but I think worth considering even if only to reject it.
One of the original motivations for the Context type was to provide a way to plumb distributed trace IDs (as used by e.g., Dapper: https://research.google.com/pubs/pub36356.html) though the call stack.
The Context type suffices to connect inbound and outbound RPCs by passing a trace ID from the inbound request handler to the outbound call, but it leaves significant gaps. In particular, distibuted profiling tools may need to associate trace IDs with specific goroutines in order to, for example, associate CPU time or memory allocations with specific inbound requests. This requires some form of goroutine tagging; e.g., http://golang.org/cl/43751.
Another aspect of Contexts is that for them to be useful, they must be pervasive. The entire purpose of a Context is to plumb values through layers of the call stack that don't care about them--e.g., to pass a cancellation signal or trace id through unrelated layers of code down to the place where a file is written or an RPC call made. This requires that almost every package become Context-aware. This is not, I think, a significant cost, but it is a cost. It is not difficult to find users complaining about the need to add Context parameters to their functions.
Given that goroutine tagging (a form of goroutine-local storage) may be necessary to support some profilers, should we consider replacing Context entirely with goroutine-local storage? Cancellation becomes a goroutine-local 'done' channel. Trace IDs are a goroutine-local value. Profile tags (if distinct from trace IDs) are just another value. The io.Reader/io.Writer interfaces can trivially support cancelation without modification. Existing packages can be retrofitted to support cancellation/value propagation without API changes.
A significant downside is that this would replace a fairly simple interface built from existing components with a language feature. I'm not convinced that this is an improvement. I think it's worth consideration, however.

We (well, @matloob) have put a fair amount of effort into making profiling tags work well with contexts.
In particular, the current approach provides mechanisms for running a goroutine with a new set of labels (this mechanism is unfortunately not very well documented at present).  This proposal needs to explain how the go statement handles goroutine-local storage.

A bit more concretely:

Each goroutine has, effectively, a local map[interface{}]interface{}.
New goroutines inherit the contents of the parent goroutine's local storage.
There is a mechanism for setting the local storage within a lexical scope. (e.g., local.Set(k, v, func() { /* k = v in this scope */ }.)

To run a goroutine with a specific set of labels rather than inheriting the current set, you would temporarily change the current goroutine's labels.

For clarity: does a new goroutine inherit the map value, or does it inherit a copy of the map?

A new goroutine would inherit a copy of the parent's map, not the actual map. (That wouldn't be very local.)
In simple cases, such as a function which starts some worker goroutines and waits for them to complete before returning, this should propagate cancellation or trace ids correctly with no effort on the user's part. More complex cases, such as long-running worker goroutines which service requests with their own cancellation, are of course still tricky.

Worker pools do need to have some way of associating work with a trace id/etc. With GLS, you would still need some Context-like object which can be passed to workers.
// Start a worker goroutine.
go func() {
  for req := range requestc {
    // Set the trace id, cancellation, etc. for this goroutine while processing this request.
    local.SetMap(req.ctx, func() {
      // Request processing goes here.
    })
  }
}()

This is more code for worker pools (although hopefully not much) in return for less in code which does not share a goroutine across multiple, independent operations.
Note that tracing systems which associate CPU time or allocations to specific requests still require that pool goroutines be temporarily associated with a particular trace tag. (c.f., http://golang.org/cl/43751) Arguably, using GLS makes it clearer to the authors of packages that use worker pools that this association may be necessary.

Merovius has a blog post arguing for GLS: https://blog.merovius.de/2017/08/14/why-context-value-matters-and-how-to-improve-it.html
It seems persuasive to me, but I also know that a lot of people have negative feelings about TLS and would not like Go to adopt a TLS-like mechanism.
I think if we follow Merovius's approach, we should make package-level variables GLS by default and use a keyword to make them non-sync shared mutable, which ISTM would be a rare thing to want.

I don't understand the middleware argument in Merovius's blog post: why can't I use a function closure to pass my values through middleware?
Personally I think that dynamic scoping is a recipe for confusion.  It's better to represent it explicitly rather than implicitly.  Same for goroutine-local storage.  Being explicit makes things very clear.

@ianlancetaylor I'm not sure what you mean with your question about the function closure. If I would, I'd be happy to try to explain what I'd see as a problem :) Say, for example, I want request-specific debug logging, e.g. using a X-Log-Verbosity: Debug header or the like. With context, an easy way to do that is (kind of handwaving an API here, try and focus on the essential ideas)
func SetVerbosity(h http.Handler) http.Handler {
    return http.HandlerFunc(res http.ResponseWriter, req *http.Request) {
        ctx := req.Context()
        ctx = context.WithValue(ctx, ctxVerbosity, parseVerbosity(req.Header.Get("X-Log-Verbosity")))
        req = req.WithContext(ctx)
        h(res, req)
    })
}

func Logf(ctx context.Context, v int, format string, vals ...interface{}) {
    if rv, _ := context.Value(ctxVerbosity).(int); rv >= v {
       log.Printf(format, vals...)
    }
}

func main() {
    h := http.Handler(http.HandlerFunc(serve))
    h = SetVerbosity(h)
    http.ListenAndServe(":0", h)
}

func serve(res http.ResponseWriter, req *http.Request) {
    Logf(req.Context(), Debug, "Doing things!")
}

Now, how would this look without context? (You could argue, that the header-parsing doesn't have to happen in a middleware and could instead happen directly from a passed *http.Request in the handler, to which I'd respond that *http.Request.Header is just a string-based, http-specific version of context)
I don't understand how closures would help - to me, that would at best replace the question "how do I pass an int through an API" with "how do I pass a func() int through an API". I'm very much ready to be convinced otherwise, though.

And FTR, I am also not convinced there actually is much of a problem with context.Context - I tend to agree with Sameer here that most criticism of context seems to be opinion not really experience (that includes my article). I also tend to agree after some convincing, that the explicit nature of context is advantageous. I just wanted to separate out the different criticisms of context, taken at face value. And as much as possible distill its usefulness (though I probably did a bad job with that). Because I think depending on what you decide is bad about it, there might be alternative solutions.

@uluyol

How would this work with worker pools? […] Right now you can bound your thread count by using a persistent worker pool.
With GLS, […] you'd have to spawn a new goroutine per request and maintain some sort of counter to limit the number overall.

In my experience, a semaphore tends produce a much cleaner API than a goroutine pool anyway: it does not require an explicit Close or Stop call to shut down background goroutines, and does not pollute goroutine dumps with idle workers in the event of a crash.
If the structure of context-passing discourages the "goroutine pool" pattern, I'd call that a feature, not a bug.

@Merovius You're right, of course: if you want to pull a value from the HTTP request, and let that value influence the behavior of other functions invoked by the request, then you must pass that value to those functions in some way.  You can pass the value in a Context, or you can pass down a function value (in your example a log function that invokes Logf with the right verbosity), but either way something must be passed explicitly.
I agree that goroutine-local variables avoid the need for explicit argument passing, but I also think they add considerable conceptual complexity.

[F]or [Contexts] to be useful, they must be pervasive.

A somewhat less invasive way to reduce the syntactic overhead of Context would be to make it a built-in type (like error).
That would at least cut the stutter by a third:
-func Foo(ctx context.Context) error {
+func Foo(ctx context) error {

An interesting side-effect of the current (explicit parameter) approach is that it is remarkably easy to leak a Context for a very long time by closing over it in (or passing it as an argument to) a go statement. Because the Context can contain arbitrary keys and values, the closure also references all of those keys and values, even if it only needs a small subset of them.
I think that the existence of context.Context as a map-like entity today may be encouraging us to think in terms of map-like solutions, when we really think about Context keys as something more like parametricity: the set of Context keys that a given function requires is the set of keys that it uses, plus the set of keys required by the things that it calls, minus the set of keys that it itself sets. (If you call a function that wants the "trace ID" variable, then you want the "trace ID" variable too.)
Ideally, those sets should be inferred at compile time. (Inferred, not declared: if there is one lesson we can learn from checked exceptions, it is that enumerating the set of things that you merely propagate is incredibly tedious.) Higher order functions and interfaces make that problem much more difficult, especially in the presence of type-assertions, but if we could crack the parametricity problem for context keys, then we could probably apply the same solution for error spaces to reap ~twice the benefit.

For this proposal, I don't see how it is possible for goroutine local storage to replace context. Perhaps it is at best an attempt to approximate process namespaces. But then those values need to be copied or shared and locked.
It's interesting how we are ok with explicit error propagation but simultaneously don't mind the though of a persistent per-goroutine request-scoped storage area. An observation follows: a go error is the way a callee informs the caller there was a problem, a context instead tells the callee there is a problem (i.e., timeout). In either case, a cancellation or error-propagation is a theme. With errors, error information can be dynamically ascertained because error is an interface. This is similar to context.Value, except that the information wants to flow in the reverse direction. In any case, It is not the process or goroutine that wants to communicate, but the request itself.
The one difference that bothers me is that context.Value is used not to provide contextual communication to the callee about why the request should be terminated (unlike error does for the caller) but mostly to pass extra arguments to the function (which is strange because we already have function parameters to solve the problem of giving functions parameters).

Contexts pass values through intermediate layers of the call stack. Cancellation is one such value (in the form of the Done channel), but is not the only one and is not particularly different from any other value plumbed in this fashion.
The problem contexts aim to solve is not that of passing parameters to a function, but of passing parameters through a function that does not care about them.

The idea of to and through are neither mutually exclusive nor well-defined. Somewhere down the chain a callee will care about it, otherwise there wouldn't be a need to have it to begin with. The idea that control flow should have a state is not new, processes have environment blocks and threads have thread local storage on many systems (objects also have methods and member values). These systems also abstract away (poorly) the complexity of making access to those resources atomic. Go is not the type of language that will do this, since the communication and synchronization primitive is the channel: not a lock, context, parameter or object pointer. I don't see how goroutine local storage solves the problem of request scope, it rather seems like a step in the other direction: implicit state, untyped values, and a concurrency based on worker identity rather than communicating pipelines.

The idea of to and through are neither mutually exclusive nor well-defined.

I tried to give a detailed explanation of the problem in this comment. You might not be happy with the choice of words of "to" vs. "through", but please try to understand the difference they describe and work from there.

Go is not the type of language that will do this, since the communication and synchronization primitive is the channel: not a lock, context, parameter or object pointer.

That isn't really true, in practice, though. The usage of locks for synchronization is pretty extensive. I don't see why, once we would be able to place pointers in GLS, we couldn't just slap a mutex on whatever request-scoped mutable data we need, to make it concurrency-safe.

I don't see how goroutine local storage solves the problem of request scope

Basically, by a) storing the request-scoped data in GLS and b) inheriting the stored values (not the storage location itself, though) through go-statements. I gave a more extensive and detailed description of the idea in this post.
(note: I am not trying to argue in favor or against GLS here, just to keep the discussion problem-oriented and constructive)

The thing that strikes me is not so much that GLS is a good idea as that package-level variables without an automatic synchronization mechanism are a bad idea. With a PLV, you either need to initialize it at start up time and then never change it again, or you need to protect individual variables in an ad hoc manner with locks or a rule like only goroutine X is allowed to use the PLV. Moving to GLS for PLV would keep the initialize-then-don't-change pattern working and allow for a news pattern of initializing then having COW GLS for things like timeouts and request variables.
If removing PLVs as they exist now is too disruptive, perhaps there could be a new keyword instead of var to indicate that a given PLV should be treated as goroutine local. A keyword that many languages use is let, but I don't think it conveys the meaning very clearly. Maybe local (x, y int) for "goroutine local package level variables x and y are ints"?

@as :

a go error is the way a callee informs the caller there was a problem, a context instead tells the callee there is a problem (i.e., timeout).

I can see what you're poking at, but I don't think this is the way devs think. When I see an error, I see Go telling me "OK, you've got an error, and I'm making them explicit because I want you to decide what you're going to do with it. You could log an error, you could crash out, you could make it mean false for some boolean check, knock yourself out."
The problem is that context is "You've got a context, and I'm going to force you to pass it down, because you have no idea what things below you exist and whether they are reliant on something in the context you got." Once contexts are introduced, there's no longer any agency for the developer to do anything but continue plumbing. That's why devs feel like they're working against the Go ideal of reducing code ceremony. However, hiding them altogether is in tension with the Go ideal of being explicit vs implicit, and that's why this problem continues to be difficult to solve.
@carlmjohnson : You'll have to forgive me as I'm a bit of a PL dummy, and I am trying to draw the line between your suggestion and what that does for context.Context. Are you suggesting that the context package be written with these locals and then a function like context.Current() would do The Right Thing for the calling goroutine?

Let's say you want to keep some metrics on a request. You might do:
package metrics

local (
    Start time.Time
    RequestID int
    Canceled chan struct{}
)

func MetricMiddleware(h http.Handler) http.Handler {
    return http.HandlerFunc(func (w http.ResponseWriter, r *http.Request) {
        // Set Start, etc.
        h.ServeHTTP(w, r)
    })
}

Since each request is run in its own goroutine, they each have a consistent view of the value of the locals. Any handler that wants to know what its request ID is can just look up metrics.RequestID, which is guaranteed to be concurrent safe and strongly typed. Handlers that don't know or care about metrics don't have to pass anything along to give to other handlers.
In this scenario, the context package would be pretty much obsolete, but there might be a new package like "timeouts" or something to replace that one aspect of context.

The explicit mechanisms we use now were chosen intentionally and seem to be working.
The proposal of using a map seems potentially expensive--the map contents have to be copied for each new goroutine creation, and there are many goroutines.  Nothing prevents this map from being large.  (The implementation of goroutine labels used for profiling (runtime/pprof.Do) was carefully implemented to avoid this potential problem.)
Being explicit in the code tends to be clearer than implicit.  We aren't going to make this change.  Closing.


HURL plugin incompatible with IntelliJ IDEA 2024.3

Upgraded to IntelliJ IDEA 2024.3 yesterday and Intellij HURL plugin declared itself incompatible with it and was disabled by IntelliJ.

Hi, thanks for reporting. I will upgrade the plugin in the next few days!

Thanks, I tried to make a PR myself, but just lacked time to get dev environment set up poperly...

👏🏻 👏🏻 👏🏻


[Feature] Add FallbackByteBufAllocator

Motivation:
Many users use direct memory allocation in large quantities. However, direct memory is limited. When the direct memory allocation is abnormal, such as insufficient direct memory, an error will be thrown by default. However, the jvm heap memory is often sufficient at this time, and some users may would like to continue to use the heap memory (or just print logs), and fallback to the heap memory. Considering that dealing direct memory problems is often very complicated, in some cases, users need a fallback strategy to solve or temporarily downgrade this problem
Modification:
Add FallbackByteBufAllocator and test case to block pooled and unpooled methods through combination
Result:
In extreme cases, users can continue to allocate memory. Can make the user's application to maintain flexibility and appropriate memory allocation strategy

Failing back to heap memory doesn't sound good because it creates more garbage than direct. Also, the heap is limited too.

@hyperxpro This method does not sound very elegant, at least when there is a memory leak, no matter whether direct memory or heap memory is used, it does not help. This solution is suitable for those who do not have memory leaks and users do not "misallocate" memory as they think

@chrisvest Thanks for the suggestion. There is no abstract method to implement the interface in FallbackByteBufAllocator. Why is the japicmp plug-in error reported? Thanks

@chrisvest Thanks for the suggestion. There is no abstract method to implement the interface in FallbackByteBufAllocator. Why is the japicmp plug-in error reported? Thanks

Good question. I don't know why it's complaining.

@chrisvest Maybe we should only catch OutOfMemoryError? right?

@normanmaurer Can you please check the japicmp plugin error? Thanks


App crashed on some azure linux vw

When i run application with default config it halt after some target with output "Killed"
-debug option doesnt show anything.
Last attacked target could be different on several runs in row.
Problem repeated at several WM without any logic. Just today I stopped one process (which worked all nigth) and got this error after next start.
Example:
./db1000n -c ./config/config.json
...
2022/03/26 10:10:57.250146 rawnet.go:71: Attacking <IP_ADDRESS>:3389
2022/03/26 10:10:57.260621 rawnet.go:71: Attacking <IP_ADDRESS>:2000
Killed
Can we get more information about crash reasons?
Could we get run option -ignore_error?
Specifications

Version: db1000n_0.8.8_linux_amd64.tar.gz (and 0.8.2 before)
Platform: Linux (ubuntu 20.04)
More then 450MB free RAM

Thank you!

Most likely it's being killed due to OOM, you can try disabling Prometheus and encryption to decrease memory usage and leave only essential functionality. There's been an instruction in one of the issues already but tag me if you can't find it - I'll take a look when I have some more time

it seems your advice helpped.
With options
-prometheus_on=false -skip-encrypted=true
app is working!


output_to_input() is difficult to understand

There's an output_to_input() function in step.py that produces a reference referring to a given output, for use in subsequent steps' inputs. The terminology is a bit confusing, and could be improved, e.g. by calling the result a reference, rather than an input.

Also self.step_outputs in Step is really self.output_types


Add datasets

Description
Add minst and cifar datasets. Both are graph classification datasets.

@jiaqima I think we can merge this PR.


Author abandoned the project?

Hello, I don't see any new commits within almost a year. So is author no more supporting his project? If yes, are there any fork of jadx that is fixing reported bugs and improving the development?

同求，有没有api文档之类 或者其他人fork 过后，添加扩展的

Anyone can ensure this information? @skylot

Check out pull request, there are some bug fixes in other repos. But there is no one big repo that contains all of them combined.

Hi @matejdro ,
Could you point in here such repos?

Yeah, it would be nice to clone the repo and add all the PRs one by one to see if they can be merged in a non-official way.

Yeah, it would be nice to clone the repo and add all the PRs one by one to see if they can be merged in a non-official way at least.

Is there anyone want to do this?
I think this project is important.

I created a fork of this (https://github.com/sergey-wowwow/jadx), and will begin support of the project after these holidays. If anybody wants to contribute too, please reply and I will add you

@sergey-wowwow this is great! )
I can add you as collaborator so you can support this project without forking
or even transfer ownership to you if collaborator permissions is not enouth (please check this)
What you think?

Hi, @skylot
I think this is cool, collaborator access would be enough. I had plans to firstly fix all security issues I found (because I will use jadx on server side in the nearest future, so they are the most important)

Looks like promising news

GOOOOOOD!!!!!  @All_Of_Developers Thank you very much.

@skylot cool, wouldn't mind to become one too, jadx is great project!

The most important thing to do at this moment is to review the pull requests and attempt to merge them one by one.


Is there any configuration about concurrent calls(or simultaneous calls)?

Hi, I am glad to found this project and very appreciate for your effort!
I'm wondering the concurrent calls is configurable or not?
or is it doesn't have any limits, I can call(and keep in calling state) as many calls in a period of time as my computer can afford?

@Lizz850517 If you are running on a single node (e.g, using Docker Compose), it can only scale vertically. However, you can run Routr in the cloud with Kubernetes and scale it vertically and horizontally.


Compiling from source no longer possible

As of commit eed098ac8e255f09b3727fa13ee577b7e0efada9, if I attempt to compile from source, I get the following errors.
gcc 6.3.1:
src/asm/asmy.y: warning: 4 shift/reduce conflicts [-Wconflicts-sr]
src/asm/asmy.y: warning: 3 reduce/reduce conflicts [-Wconflicts-rr]
src/link/lexer.c:398:0: error: "yywrap" redefined [-Werror]

src/link/lexer.c:74:0: note: this is the location of the previous definition

cc1: all warnings being treated as errors
make: *** [Makefile:106: src/link/lexer.o] Error 1

clang 3.9.1:
src/asm/asmy.y: warning: 4 shift/reduce conflicts [-Wconflicts-sr]
src/asm/asmy.y: warning: 3 reduce/reduce conflicts [-Wconflicts-rr]
src/link/lexer.c:398:9: error: 'yywrap' macro redefined [-Werror,-Wmacro-redefined]
#define yywrap() (/*CONSTCOND*/1)
        ^
src/link/lexer.c:74:13: note: previous definition is here
    #define yywrap yywrap
            ^
1 error generated.
    make: *** [Makefile:106: src/link/lexer.o] Error 1

I am using archlinux
Versions of applicable stuff
make: GNU make 4.2.1
gcc: 6.3.1/clang: 3.9.1
yacc: GNU bison 3.0.4
flex: 2.6.3
pkg-config: 0.29.1
libpng: 1.6.28

Ok, that's weird, considering that I had that same problem and I solved it with %option noyywrap.
Could you try to compile with make FLAGS=" --nounistd --noyywrap" or just make FLAGS="--noyywrap"?

That just makes make throw out this a lot:
make: Wall: Command not found

It's clearly misunderstanding flags in some way, not sure how though.

This is what I have:
$ gcc --version
gcc (Debian 6.3.0-11) 6.3.0 20170321
$ flex --version
flex 2.6.1
$ yacc --version
bison (GNU Bison) 3.0.4

Sorry, I meant LFLAGS.
make LFLAGS="--nounistd --noyywrap"
make LFLAGS="--noyywrap"?

Same effect.

Weird, that works for me. What versions of flex and yacc do you have?

yacc: GNU bison 3.0.4
flex: 2.6.3

Could you try adding int yywrap (void) { return 1; } to line 95 in src/link/parser.y, and removing line 19 in src/link/lexer.l? The one with %option noyywrap.

OK, not sure what was up with that Wall spam, something I did earlier inadvertily broke anything make did in general (presumably some stray variable breaking stuff) even outside rgbds. Using a fresh instance made those issues go away, so I attempted your suggestions again. They didn't really do anything to solve the original issue though.
However, your last suggestion (manually editing files as you suggested) made make work properly.

This should fix it then: https://github.com/AntonioND/rgbds/commit/b8d2950b8daf5f616efa4f25f1b2c19ebc73f80d

Yeah, it works.


(^18^)++NEW~SEX@XXX@VIDEOS]**Pragya Nagra Leaked video Original Video Link Short Clip Indian Actress Pragya Nagra Original Video Viral On Social Media X TikTok Trending Now

39 seconds ago
L𝚎aked Video Pragya Nagra Original Video Viral Video L𝚎aked on X Twitter Telegram
[-wATCH-]— Pragya Nagra Video Original Video Link  Pragya Nagra Video Viral On Social Media X Trending Now
[-wATCH-]— Pragya Nagra ʟᴇᴀᴋᴇᴅ Video ᴠɪʀᴀʟ On Social Media ˣ ᵀʷⁱᵗᵗᵉʳ
[-wATCH-]— Pragya Nagra ʟᴇᴀᴋᴇᴅ Video ᴠɪʀᴀʟ On Social Media ˣ ᵀʷⁱᵗᵗᵉʳ
[-wATCH-]— Pragya Nagra Video Original Video Link  Pragya Nagra Video Viral On Social Media X Trending Now
Pragya Nagra Original Video video took the internet by storm and amazed viewers on various social media platforms.Pragya Nagra, a young and talented digital creator, recently became famous thanks to this interesting video.
L𝚎aked Video Pragya Nagra Original Video Viral Video L𝚎aked on X Twitter
Pragya Nagra Original Video video oficial twitter
L𝚎aked Video  Pragya Nagra Original Video Viral Video L𝚎aked on X Twitter..yjhgrdafassf

.


Auth sources: add new ratings

i.e.

PRIMARY : declared auth source from system of record
SECONDARY : declared auth source, but not from system of record (a non auth-sourced auth source :) )
DISCOURAGED : auth sources exist for this DT but this flow does not use any of those sources
NO_OPINION : no auth sources declared, so no rating can be determined
BLESSED : individual flow authorisation, aka heritage flows
DISALLOWED : deffo should not use

Todo:

[x] source and target graph
[x] auth source calculator
[x] exporters
[x] boingy graph
[x] legend

[x] source and target
[x] boingy

[x] auth source editor
[x] list table needs to be searchable via rating name
[x] summary charts for app

[x] stats table for app

[x] summary charts for datatype
[x] datatype home page (mini bars)
[x] rated flow scorecard
[x] auth source summary section

Need to verify terminology

Silly sample data:
insert into enum_value (type, key, display_name, description, icon_name, position, icon_color)
values ('AuthoritativenessRating', 'HERITAGE', 'Heritage (point to point) flows', 'blah', 'fw', 10, 'brown'),
       ('AuthoritativenessRating', 'WRONG_SOURCED', 'Deffo bad', 'blah', 'fw', 10, 'red');

WIP - colourful example

Example of updating an authority statement:
![Screenshot 2021-07-14 21:53:36](https://user-images.githubusercontent.com/576829/125691463-a6ece4f5-8fd2-413f-8384-
be5c3c3655c9.png)

Updated data type homepage

Application flow stats

Combo view of app page:

Auth source summary (group) panel

Rated flow scorecard:

Tables include color patch:

Updated audit log to include rating (and also log against the dedicated auth source entity page):

Legend (on source/target graph and on the force directed graph):

Source and target popup, with popover showing rating label:


Review

Please review, @Birk12345

Description of repository
Please add a three-paragraph description
Tests

[x] License exists
[ ] Create a fork and start a pull-request
[ ] Pytests fails as expected
[ ] Ruff linter fails as expected

2nd Homework completed @danielhuppmann


New napari-plugin does not show up on the napari-hub

Description
Hi all,
a new plugin napari-lattice does not show up on the napari-hub, even though the setup.cfg appears correct to us.
Can you give us guidance on how to publish this plugin on the hub?
Thanks!
CC @pr4deepr
See also

https://github.com/BioimageAnalysisCoreWEHI/napari_lattice/issues/1https://github.com/BioimageAnalysisCoreWEHI/napari_lattice/issues/1

Similar issue with this plugin :
https://pypi.org/project/napari-LF/
Cc @gschlafly

@haesleinhuepf Thanks for opening this! There might have been a lag between when the plugin was released and when we retrieve new plugin data (though this should be happening every 5 minutes). Looks like those are showing up now.
Do you remember how much time had passed since release, at the time you weren't seeing the data on napari-hub?

@richaagarwal Our napari-LF plugin did not show up on napari-hub for at least four hours. When I checked 19 hours after the release, it was on napari-hub.

Thank you for the info, @gschlafly!
It looks like there's a related issue https://github.com/chanzuckerberg/napari-hub/issues/200 so I think it's worth more investigation at this point.  I'll chat with the team to see when we might be able to prioritize this investigation.

here's another example: https://github.com/napari/napari/issues/4862
napari-deepfinder

released today on PyPI: https://pypi.org/project/napari-deepfinder/
not found on the napari hub: https://www.napari-hub.org/plugins/napari-deepfinder

given that napari now uses the napari hub's API to get the list of available plugins, this is a problem that propagates to napari as well

oddly, for napari-deepfinder, the API is returning a partial response with npe2-related fields only (h/t @tlambert03):
https://api.napari-hub.org/plugins/napari-deepfinder
{
  "conda": [],
  "display_name": "",
  "plugin_types": [],
  "reader_file_extensions": [],
  "writer_file_extensions": [],
  "writer_save_layers": []
}

otoh, the expected response for a non-existant plugin should be an empty blob:
https://api.napari-hub.org/plugins/foobar
{}

Hmmm I thought the npe2 API was entirely switched off still? We shouldn't be seeing these fields at all, no? Did we only switch off the frontend @codemonkey800 ?

iirc the backend changes were just switching off the lambdas, I don't think we ever reverted the API changes. We should definitely gate it if we don't want to revert - not sure what the timelines are looking like for updating the backend work @richaagarwal

@DragaDoncila That's correct, we never reverted the API changes.  I don't know if there is a strong need to do this as we had discussed that no one else is really consuming the npe2 data yet, but maybe @neuromusic can weigh in here.  But if needed, then I am in support of reverting all npe2-enabled features changes.

Do you think it's preferable to revert rather than just add a "gate" to hold the fields back? I think a lot of the API stuff won't change so it might be more effort to revert then add it back?

why are we seeing these npe2 fields for napari-deepfinder (which is a real plugin but doesn't exist on the hub) and not for foobar (which isn't a real plugin)

@klai95 @potating-potato any idea why we wouldn't be seeing data from https://pypi.org/pypi/napari-deepfinder/json populated yet on the hub side?

why are we seeing these npe2 fields for napari-deepfinder (which is a real plugin but doesn't exist on the hub) and not for foobar (which isn't a real plugin)

@neuromusic probably a quirk of when we're returning information. We decided early on that npe2 metadata would always be "available" in the metadata dictionary, so we return default values when we can't retrieve "real" information. When a plugin doesn't exist at all we do an early return of {}.

When a plugin doesn't exist at all

@DragaDoncila so even though it's not showing up, napari-deepfinder must exist somewhere in the hub backend?

@neuromusic correct - somewhere somehow, the napari hub backend is aware of napari-deepfinder and trying to process it, but not finding the PyPI metadata correctly.

For some reason, the backend data for napari-deepfinder was corrupted, I deleted them and the repopulation of the data is then successful. The reason for the corruption is not yet known. I don't have any additional context but that the corrupted json contains nothing but "conda: []",  this is not expected from the typical indexing, I believe this is probably related to npe2 somehow

@klai95 will be doing some exploration on this to understand what may be causing these intermittent lags reported here as well as in #200.  In the meantime, if anyone notices this with any additional plugins, please let us knos!

Thanks for looking into this.
Just as a follow up from the original post, napari-lattice plugin appears on napari-hub but doesn't update when a new release is available on pypi.
v 0.1.5 in pypi
still v 0.1.4b in napari-hub

I digged around on the napari-lattice and I think I found the gap:
This may be related to https://github.com/chanzuckerberg/napari-hub/pull/605/files
when napari-lattice 0.1.5 published, we correctly fetched it and created a profile for it in our system, however when we populate the metadata, we used the non-versioned pypi API to get the latest, which seems to have not updated to correctly report the metadata for 0.1.5, as a result, the 0.1.5 profile is corrupted with 0.1.4b0 data when it was indexed. I will submit another PR to fix this

hey @potating-potato, let me know if there's anything I can do to facilitate here with npe2api.  napari-lattice is looking good here: https://github.com/napari/npe2api/blob/main/public/manifest/napari-lattice.json

hey @potating-potato, let me know if there's anything I can do to facilitate here with npe2api. napari-lattice is looking good here: https://github.com/napari/npe2api/blob/main/public/manifest/napari-lattice.json

thanks for the help! I think I have found the issue and going to submit a PR to close this.

Just to give a quick update here:
I think we may have pinpointed the cause here.  Our hypothesis is that the lag may be due to PyPI's non-versioned endpoint sometimes taking time to catch up with the latest release data. For instance, https://pypi.org/pypi/mage-ai/json currently does not reflect latest release data available in https://pypi.org/pypi/mage-ai/0.2.1/json.
#608 outlines a potential fix to this.

I am going to close this since we have #608 to track work on a potential fix.  We can re-open this in the future if needed or the situation worsens.


⚠ Jenkins Plugins Artifactory has degraded performance

In 53b1121, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1985 ms

Resolved: Jenkins Plugins Artifactory performance has improved in cd3ccf7.


Error with GitHub logon

HI,
Trying James' instructions (https://github.com/fabric8io/fabric8-platform/blob/master/INSTALL.md) and getting this after reaching the Fabric8 logon page when I select GitHub:
{"error":"invalid_request","error_description":"The request is missing a required parameter, includes an invalid parameter value, includes a parameter more than once, or is otherwise malformed.","state":"gtItBFKsP3VO6Va2JG4mvGFYkVZlQ5dlz34whDGQaiM.fabric8-online-platform"}
If I then retry the original fabric8 URL I see the main page and I appear to be logged on. When I then try to create a Space and end up with an error & success indication?

Seems to be the same issue as: https://github.com/fabric8io/fabric8-platform/issues/295

Late last week we reverted back to using http when running minishift / minikube so we can avoid accepting all the self signed certs initially.  FWIW non local deployments are expected to still use https.
Would you mind having another go now that our docs and release are aligned to only use http please?

James, sorry for the delay in getting back to you. I've ben on holiday and only recently got good broadband.
Yes, I can confirm this now works using the minishift 1.3.0 release


Split Playwright tests into two groups

Why
Running all e2e tests on every commit to a PR takes too much time and resources.
It would be better to split the existing set of tests into two groups:

runs every commit to a PR
runs every night

TODOS

[x] categorize tests into the 2 groups (only functionality tests only on chrome for every commit)
[x] check if playwright provides a native way to group and run tests selectively
[x] update github action to run the task every night
[x] evaluate time speedup (~5 minutes per commit)
[ ] find a way to notify developers upon errors on nightly run (e.g. slack)

https://ray.run/blog/organizing-playwright-tests-using-tags
You can tag specific tests and run only those on CLI command.
I still need to check how I can limit these tests to run only for Chrome.

There is no way to use tags to specify browsers. However, it seems reasonable to remove any browser specifications from the config file and simply specify browsers for the entire test set from CLI command. The downside would be that the nightly tests would take slightly longer (because even tests that were originally intended to be run on only chromium would run on every browser) but taking 20 minutes instead of 15 minutes for each nightly run seems rather trivial

https://github.com/vercel/vercel/issues/10550
Apparently cron github jobs only work on production..

https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#schedule
Github cron jobs run only on the last commit of the default branch (develop on corbado/javascript repo)

Keeping it at in-progress until I confirm that the nightly workflow runs properly on develop branch

https://github.com/marketplace/actions/slack-send

https://github.com/ryanrosello-og/playwright-slack-report


Add CI tests to validate all old provider versions against the current master

[ ] documentation is added or updated
[x] tests are added or updated

I care much less about this than I do about the fips related tests..
If you dont need fips then you should probably just be using the same versions of the providers.

There are additional failures in legacy.

There are a few changes to RC4-HMAC
One of them is this.. https://github.com/openssl/openssl/commit/33219939c782cf363b30e9e899b9997fb1ced440

Most of the issues are fixed.
Refer also: openssl/tools#125 for release procedure update to try to not lose sync.

IMO this is too much. I really do not think we want to test all the provider versions ever released. Let's take an example we make some mistake causing regression in a provider in version 3.0.3. We fix it in 3.0.4 adding a test for it. Do we want that test to be special-cased? This in a few years explodes into unmaintainable mess.
I'd just test validated versions of the fips provider plus the latest one from the 3.0 branch and a few selected versions of the legacy provider that are known to not contain any regressions. So again perhaps 3.0.0 plus the latest from the 3.0 branch.
Anyway I would not recommend anyone to use the legacy provider from 3.0.0 especially given the RC4-MD5 security issue fixed in there.

There was extensive discussion about these very points in the OMC meeting yesterday. The result was this ongoing vote:
https://github.com/openssl/general-policies/issues/27
The argument in favour of doing this is that we have committed to a forward compatible ABI for providers. The only way we know we haven't broken that promise is to test it. Therefore we test it by using our own released providers as a proxy for all 3rd party providers.

Do we want that test to be special-cased?

Yes. We need to do this special casing anyway even if all you are testing is a few selected prior versions. Adding more versions of providers should not mean adding more special casing, it just means you need to be more accurate about which specific version number of a provider contains the fix for a bug that a test is testing. It should not (generally) mean needing to special case more tests that you would otherwise. The only case I can think of where this could occur is where you introduce a new bug and then fix it in between validated versions of the fips provider.

This in a few years explodes into unmaintainable mess.

We need to do this in any case at least for the validated versions of the FIPS provider. If you are already doing it for the validated versions it really doesn't add to the explosion to test all versions.

Anyway I would not recommend anyone to use the legacy provider from 3.0.0 especially given the RC4-MD5 security issue fixed in there.

This is misunderstanding the objective. We are not testing the legacy provider from old versions because we necessarily expect or want anyone to use it. We are testing it because it enables us to check that we have not inadvertently broken the provider ABI. The legacy provider is acting as a proxy for other 3rd party providers.

This is still kind of reliant on having high test coverage to some extent.

I still see no point in running the tests with the intermediate versions of legacy or even of the fips provider. Although you're right that in most cases the version checks in the tests will have to be there anyway, my example with one version regression in a legacy or unvalidated fips provider still holds. And running all these version tests would be just mindless resource consumption. What if there are 20 3.0.x releases and 10 3.1.x releases at some point.

Although you're right that in most cases the version checks in the tests will have to be there anyway, my example with one version regression in a legacy or unvalidated fips provider still holds.

Right, introducing a bug in one release, and fixing it in the next, in between validated versions would introduce a new special case that you wouldn't otherwise have. I don't expect there to be too many of those, so I don't really see this as a problem.

And running all these version tests would be just mindless resource consumption. What if there are 20 3.0.x releases and 10 3.1.x releases at some point.

I expect this to be refined over time. If we get to the point where we have so many releases that resource consumption is a problem then we will modify the policy to test a representative sample. At the moment we have 5 which is really not that many. We have 1 validated version which is not a sufficiently large sample size, and we're not testing the legacy provider at all - which is a missed opportunity to test provider ABI codepaths that may not be exercised by the fips provider.

This also doesnt need to be run very often I would imagine?

This also doesnt need to be run very often I would imagine?

Correct. The in-progress vote only talks about doing it on release. I suspect we may do it a little more frequently than that in practice.

There is one more potential issue with testing the legacy provider that might complicate its testing in these scenarios. The legacy provider calls into libcrypto and is not completely isolated from it like the fips provider. This probably is not so big deal if we are testing legacy provider with newer libcrypto (from the same major version). However it won't work universally if the legacy provider is from newer branch than the libcrypto that is being tested. I understand that we currently do not have this problem but eventually we might.

Correct. The in-progress vote only talks about doing it on release. I suspect we may do it a little more frequently than that in practice.

I'd say we might want to run this test on the latest 3.0.x version released on pull requests. This would enable correctly skipping the newly added bug fix testcases if they break. But definitely not the full set.

Running the full set daily would be more than enough.

The argument in favour of doing this is that we have committed to a forward compatible ABI for providers.

We also committed equally to a backwards compatible ABI for (at least FIPS) providers at the same time.  This has not been mentioned anywhere and no actions have resulted.  Personally, I only remember discussing FIPS providers being subject to such forwards and backward compatibility requirements & it makes sense for them given how (hopefully) infrequently validations will occur.
Moreover, none of the fixes here are ABI changes.  They are bug fixes, the ABI is unchanged throughout.  Our provider ABI seems to be pretty solid, our implementation still has some bugs.

If we get to the point where we have so many releases that resource consumption is a problem then we will modify the policy to test a representative sample.

This should be an OTC decision (as has been mentioned elsewhere, the OTC is responsible for ensuring that the tests are run).  However, it's an OMC vote that mandates all releases be covered.  Modifying the policy becomes an exercise in bureaucracy.

I understand that we currently do not have this problem but eventually we might.

I'm expecting a world more hurt from the test everything always approach.
My intention is to schedule the full suite daily or weekly on both 3.0.x and master.  There is no point wasting so many cycles on commits or on PRs.  I will keep the parent's FIPS 3.0.0 provider on PRs -- that makes sense IMO.  We need to know if validated FIPS providers aren't usable ASAP.

New CIs passing.
Switching to cron based scheduling instead of on PR.

Moreover, none of the fixes here are ABI changes. They are bug fixes, the ABI is unchanged throughout. Our provider ABI seems to be pretty solid, our implementation still has some bugs.

Agreed.  But we still have to test this, and I suspect that where there are known bugs in older versions of a provider, we need to insert code to not run the tests that we know trigger that bug, or otherwise be able to handle buggy responses.  That's probably the scarier part, that we might end up with a growing database of such "exceptions".

I still don't think we need to test everything.  My experience here was that all of the problems were present in the 3.0.0 providers and that zero new problems were introduced later.
Moreover, 3.0.4 was identical to 3.0.5 in terms of problems -- not surprising since 3.0.4 was a broken release and 3.0.5 replaced it very soon after.
This is overkill IMO.  Still, the ongoing OMC vote will decide things once and for all.

Typo on a commit message : lergacy provider

Thanks @FdaSilvaYY, fixed.

Moreover, none of the fixes here are ABI changes. They are bug fixes, the ABI is unchanged throughout. Our provider ABI seems to be pretty solid, our implementation still has some bugs.

Agreed. But we still have to test this, and I suspect that where there are known bugs in older versions of a provider, we need to insert code to not run the tests that we know trigger that bug, or otherwise be able to handle buggy responses. That's probably the scarier part, that we might end up with a growing database of such "exceptions".

@levitte https://github.com/openssl/general-policies/issues/27 is still waiting on your vote

This PR is in a state where it requires action by @openssl/otc but the last update was 30 days ago

This PR is in a state where it requires action by @openssl/otc but the last update was 61 days ago

This PR is in a state where it requires action by @openssl/otc but the last update was 92 days ago

Vote failed, no point leaving this hanging around.


Fix/error handling ux

Hoooi
I made some minor bugfixes and upgrades:

Upgraded vue/test-utils package
Throw error messages when the documentation upload or deletion fails (currently just swallows it)
Clear the file import field as well when the documentation uploading was successful

thanks :tada:


raftstore-v2: add proposal conflict checker

What is changed and how it works?
Issue Number: Close #12842
What's Changed:
The implementation is a bit simplified and adjusted for v2, including:
- Not require term for every methods, because when leader's term is changed, it must become follower. In this case, the checker won't be accessed until it becomes leader again. So pass term in every methods is useless.
- Use the checker to detect if it's splitting and merging.

Check List
Tests

Unit test
Integration test

Release note
None

PTAL

/merge

/merge


[ICE: 24] Change Ownership for router blocker

From p2pvalidator chat:
" we have another blocker issue: we just tried to change ownership via proposeRouterOwner/acceptProposedRouterOwner methods via louper.dev, but it fails on the last step - on the acceptProposedRouterOwner. The estimated gas fee is some huge value of millions dollars. The same time, I can get the correct address of new owner via getProposedRouterOwner, we are sure we are connected with the correct wallet of new owner and we tried this for 3 networks: on Arbitrum, BSC and Gnosis. I think the big estimated gas fee is some smart contract guard from accepting the new router owner from not correct address, but the thing is that we have the correct one wallet which is trying acceptProposedRouterOwner method. So definitely need your help here, there might be smart contract issue, I think we are the first ones who tries to change the router owner"
https://connextscan.io/router/0xC4Ae07F276768A3b74AE8c47bc108a2aF0e40eBa
@oncall

They need to wait 7 days

Nothing to do here, router can accept ownership after 7d elapses.


Register ArgCheck: 1.0.1

Registering: ArgCheck
Repository: https://github.com/jw3126/ArgCheck.jl
Version: v1.0.1
Commit: 92c9d906945779f70f64c3e86c4b4756d4efedc3
Proposed by: @jw3126
Reviewed by: @jw3126
Reference: https://github.com/jw3126/ArgCheck.jl/issues/29

Consider enabling TagBot to make sure the release is persistent in your repo, see https://discourse.julialang.org/t/ann-tagbot-creates-tags-and-releases-for-your-julia-packages-when-theyre-registered/23084


Annoying double scrollbar on hierarchy document list.

Describe the bug
Annoying double scrollbar on hierarchy document list. May be related to issue #20, which I reported separately as I am unsure if they are related.
To Reproduce
Steps:

Install hierarchical-document-list and add it to your structure following installation steps
Run sanity dev (npm run dev) and in your browser, open the new structure page added to Sanity
See error in browser console.

Expected behavior
To only have one scroll bar to navigate the hierarchy document list.
Screenshots

Which versions of Sanity are you using?
@sanity/cli (global)                3.34.0 (up to date)
@sanity/eslint-config-studio         3.0.1 (up to date)
@sanity/hierarchical-document-list   2.0.0 (up to date)
@sanity/icons                       2.11.2 (up to date)
@sanity/ui                          2.0.10 (up to date)
@sanity/vision                      3.34.0 (up to date)
sanity                              3.34.0 (up to date)

What operating system are you using?
Windows 11
Which versions of Node.js / npm are you running?
Node v21.7.0
NPM 9.1.2
Additional context
Display Resolution 3840 x 2160.
Scale is set to 250% in Windows Display settings (the recommended setting).
Tested on Firefox Developer Edition 124.0b9 (64-bit) and Chrome Version 122.0.6261.129 (Official Build) (64-bit).

It appears to be caused by the getTreeHeight function in utils not calculating the height correctly. It assumes the "row height" to be 51px with a static constant, whereas on my computer it is 57px.
I'm not a React dev, however, I'll attempt a pull request once I'm confident with my fix.


WebWorker and dom lib does not support transfers in the type for postMessage

TypeScript Version:  2.0.10
Code
// Create a 32MB "file" and fill it.
var uInt8Array = new Uint8Array(1024 * 1024 * 32); // 32MB
for (var i = 0; i < uInt8Array.length; ++i) {
    uInt8Array[i] = i;
}

self.postMessage(uInt8Array.buffer, [uInt8Array.buffer]);

Expected behavior:
Compiled with tsc --lib "es6, webworker, scripthost" worker.ts a JavaScript file should be generated.
Actual behavior:
[14:04:55] ➜  Desktop tsc --lib "es6, webworker, scripthost" worker.ts
worker.ts(7,1): error TS2346: Supplied parameters do not match any signature of call target.

As documented in the WhatWG Living Standard, postMessage on DedicatedWorkerGlobalScope should accept a list of objects to be transferred rather than cloned, but the current lib.webworker.d.ts defines postMessage with only a single parameter. postMessage on a MessagePort should also accept a list of transfers, which is not the case in either dom.generated.d.ts or in webworker.generated.d.ts. The postMessage in dom on the window object, however, does currently support transfers.

PRs welcomed. You can find more information about contributing lib.d.ts fixes at https://github.com/Microsoft/TypeScript/blob/master/CONTRIBUTING.md#contributing-libdts-fixes.

TL;DR replace self with (self as DedicatedWorkerGlobalScope) in my-worker.ts.

As of Typescript 2.6.2, the lib.webworker.d.ts and the error message changed. The current situation is the following. I think the issue occurs because postMessage is not in the WorkerGlobalScope interface but --lib "webworker" makes self of type WorkerGlobalScope due to this line in lib.webworker.d.ts.
Confirm by compiling the following Typescript file using tsc --lib "es6, webworker" my-worker.ts:
// OK: isSecureContext from the WorkerGlobalScope
console.log(self.isSecureContext);

// Error: postMessage not in WorkerGlobalScope
self.postMessage("lala");

// OK: postMessage from the DedicatedWorkerGlobalScope
(self as DedicatedWorkerGlobalScope).postMessage("lala");

The middle block fails with

my-worker.ts(5,6): error TS2339: Property 'postMessage' does not exist on type 'WorkerGlobalScope'.

I guess there are 3 possible solutions to this:

Have a separate lib.dedicated-webworker.d.ts with self of type DedicatedWorkerGlobalScope
Replace self with (self as DedicatedWorkerGlobalScope)
Use a different casted symbol instead of self, e.g. const mySelf = (self as DedicatedWorkerGlobalScope)


Issue showing the Helium balance on Ledger Nano X

I tried to view my helium balance on the Ledger Nano X by using the latest version of helium-ledger-cli-v2.2.4 for Mac OS. Ledger Life is up to date and I reinstalled the Helium app on my Ledger. I opened the Helium app on Ledger showing : Waiting for commands..... Ledger Life is closed.. I typed ./helium-ledger-cli and then ./helium-ledger-cli balance. Now an error occurs. I get the following message: Carlos-Mac-Book:helium-ledger-cli-v2.2.4-alpha.0-x86-64-macos carlodehler$ ./helium-ledger-cli balance
Communicating with Ledger - follow prompts on screen
error: Error getting version: Data length is wrong with 3 bytes. Your ledger application may require an update.
Carlos-Mac-Book:helium-ledger-cli-v2.2.4-alpha.0-x86-64-macos carlodehler$
Can you solve my problem? You can send me an e-mail to<EMAIL_ADDRESS>

Same issue with Ledger Nano S Plus and Helium App 2.2.4 and cli 2.2.4

I ran into the same issue on my windows Desktop and Mac laptop. Building the tarball in the releases page didn't work either, however cloning and re-building the main/ branch did work.


"Model is not a full model or PEFT model" despite loadable on another machine

Hey there.
I have two machines.
One (machine 1) has Jupyter running to do all the experiments and training of models (with Nvidia T4).
The other (machine 2) I can only access through SSH and loads the model to provide an api.
I have trained multiple models, on various steps of data example volumes (100, 1000, 10K, 20K).
Somehow, the 10K and 20K models can not run on machine 2.
All the other models are in the same folder and they work fine.
They have all been trained using the same parameters, except for the volume of data examples.
The error I get:

Unsloth: models/mistral_test is not a full model or a PEFT model.

I get the same error both with the model stored locally and when trying to load it from my HF.
I can load it both locally and through HF on machine 1.
Machine 2's unsloth is up to date.
Is there anything I can do like provide you with the model so that you can check if it's corrupted ?

@OsaCode Can you screenshot ur folder?

Here

yeah same issue trying to load adapters.
Also related to this I believe: https://github.com/unslothai/unsloth/issues/59

The issue is that I am able to load other adapters trained in the exact same way.

I apparently found the problem.
All of my recently trained models that do not load correctly have:
"layer_replication": null
in their adapter_config.json, and removing it made it correctly load on my other machine.


S3 - Bucket Policy Missing SecureTransport Statement DE pipeline - lookup-tables-preprod

Pull Request Objective
This piece of work is being tracked in
this
GitHub Issue.
Pull Request Objective
This piece of work is being tracked in
this GitHub Issue.
This  work is part of the remediation for the above ticket where the ITHC report identified issues
to implement aws:SecureTransport" = "false" where needed

[x] I have reviewed the style guide
and ensured that my code complies with it
[x] All checks have passed (or override label applied, if I've
used the override-static-analysis label, I've explained why)
[x] I have self-reviewed my code
[x] I have reviewed the checks and can attest they're as expected

Additional Comments
the override-static-analysis label has been used as there are multiple historical issues not touched by this PR

it has been decided we can ignore the following
Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the last "terraform apply" which may have affected this plan:


Improve diagnostics of test_cmp_http.t

This just fixes three tiny issues with test_cmp_http.t:

Extend diagnostics of mock server launch
Silence check for availability of 'kill' and 'lsof' commands
Fix resumption when skipping after mock server launch failed

[x] tests are added or updated

external-tests CI error is unrelated.

24 hours has passed since 'approval: done' was set, but this PR has failing CI tests.  Once the tests pass it will get moved to 'approval: ready to merge' automatically, alternatively please review and set the label manually.

Merged - thanks @paulidale


Add instructions to run tornado backend on vscode IDE to readme.

We may or may not need to land this ever, but it's mostly for me so I don't need to ask for help on the command to run the tornado backend on VS Code :)


RFE: import local docker image

It'd be cool if I could import a local docker image with enroot. Maybe something like this?
docker build -t my-tmp-image .
enroot import docker:///my-tmp-image

Or maybe there's a way to trick enroot into using my local daemon as a registry somehow?

This hack works for now:
docker run -d -p 5000:5000 --name registry registry:2
docker build -t "$img" .
reg_img="localhost:5000/$USER/$img"
enroot_uri="localhost:5000#$USER/$img"
docker tag "$img" "$reg_img"
docker push "$reg_img"
ENROOT_ALLOW_HTTP=yes enroot import "docker://$enroot_uri"

FYI if I don't add the $USER prefix, then I get this error:
+ enroot import --output tmp.sqsh docker://localhost:5000#tmp:2
[INFO] Querying registry for permission grant
[ERROR] URL http://localhost:5000/v2/library/tmp/manifests/2 returned error code: 404

Yeah, I thought about it but it adds a lot complexity in the docker import since the docker daemon API is not exactly the same as Registry V2.
Some alternatives are:

A local registry (like you did)
Use skopeo+umoci, although you'll probably lose some information from the image (e.g. ENV, WORKDIR, CMD, etc)
Use docker export inside ENROOT_DATA_PATH, but it has the same downsides as above

Maybe we could rely on docker save somehow, I forgot how the format looks like exactly.
Another approach would be to expose docker::_configure, so you could do something like docker inspect img | enroot genconf

Closing as there is now a way to do it in master

Confirmed. With the latest code:
$ cat Dockerfile
FROM alpine
$ docker build -t tmp .
$ enroot import dockerd://tmp
$ enroot start tmp.sqsh grep PRETTY /etc/os-release
PRETTY_NAME="Alpine Linux v3.10"


Allow SERVER_ICON to reference a file

Enhancement Type
Improve an existing feature
Describe the enhancement
Right now it can only be a URL

That would be neat! (Although the env. var. is called SERVER and not SERVER_ICON, according to this?)

In the meantime, one could do (and I do) something like the following in one's Dockerfile:
COPY servericon.png /data-init/
COPY start-custom /data-init/
...
ENTRYPOINT ["/data-init/start-custom"]

where start-custom is something like:
#!/bin/bash
cp /data-init/server-icon.png /data/
/start

Correction, it's ICON
https://github.com/itzg/docker-minecraft-server#server-icon


Resource Manager library adjustments for azure-client-sdk-parent parentage

Right now the resource manager libraries are parenting off of azure-resourcemanager-parent. When the updates are made to swap from azure-resourcemanager-parent to azure-client-sdk-parent we're going to hit issues with the differences in the way the following items are compiled or handled/verified by the track 2 pipelines:

module_info
javadoc generation
sources
checkstyle
spotbugs
revapi
enforcer
README/CHANGELOG verifications
The azure-resourcemanager-parent has its own sets of plugin definitions for many of the above and changes will need to be made to get things working with the azure-client-sdk-parent.

[ ] Remove suppression of ServiceClientCheck on entire package. checkstyle-suppressions.xml L484


Version 9 planning

We just released 8 but there are some breaking changes we'd like to make... 😬 If we're sticking to semver-ish then we need to bump to 9.

[ ] (breaking) Move all static Raven access to its own class (partially done in #375)
[ ] Land React Native support branch which includes changes to stack interface #368
[ ] Add way to set release/env/tags/etc for static access (like appenders do)
[ ] Fix static access for logging integrations #365 [0]
[ ] (stretch) Automatic breadcrumbs on Android if they're not a massive project? #347

[0] I'm actually having second thoughts on this, I'm wondering if we should just disallow static use for logging integrations.

Actually, @mitsuhiko think we should rename to Sentry and make some big fixes, I agree.
We can go to 1.0 on Sentry, and move the appender config into what is now Raven so it's not all duplicated and so Android can use it (release, environment, tags, etc).

Well done @bretthoerner !


Script doing incorrect score calculation and calculating scores for unenabled standards

My AWS environment only has one security standard enabled - AWS Foundational Security Best Practices v1.0.0
The image below verifies this.

However, when I run the script, the following is the output I get -

As you can see, I am getting an incorrect score for the standard that I have enabled and then further, I am getting scores for standards I have not even enabled. I know in the build_standards_dict function, you extract the control name using control.split('/')[1] and control = prodField['StandardsArn']. I am unsure how it is extracting standards that aren't even enabled.

For me, it filters the exact enables standards, but it doesn't return the exact score

the score generation is incorrect


Possible data race

After updating to the version  I get:
  ==================
  WARNING: DATA RACE
  Write at 0x00c000474220 by goroutine 33:
    github.com/wagslane/go-rabbitmq.(*Publisher).startNotifyFlowHandler()
        go/pkg/mod/github.com/wagslane/go-rabbitmq@v0.6.2/publish.go:234 +0x70

  Previous read at 0x00c000474220 by goroutine 24:
    github.com/wagslane/go-rabbitmq.NewPublisher()
        go/pkg/mod/github.com/wagslane/go-rabbitmq@v0.6.2/publish.go:167 +0x3b3
    git.vonroll-infratec.com/go/meas/pkg/queue.NewPublisher()
        ws/go/meas/pkg/queue/rab_pub.go:48 +0x604
    git.vonroll-infratec.com/go/meas/pkg/queue.(*RabbitSuite).SetupTest()
        ws/go/meas/pkg/queue/rabbit_test.go:43 +0x119
    github.com/stretchr/testify/suite.Run.func1()
        go/pkg/mod/github.com/rzajac/testify@v1.6.2-0.20200929081604-1a836a85454e/suite/suite.go:148 +0x8b4
    testing.tRunner()
        /usr/local/go/src/testing/testing.go:1193 +0x202

  Goroutine 33 (running) created at:
    github.com/wagslane/go-rabbitmq.(*Publisher).startNotifyHandlers()
        go/pkg/mod/github.com/wagslane/go-rabbitmq@v0.6.2/publish.go:229 +0x164
    github.com/wagslane/go-rabbitmq.NewPublisher.func1()
        go/pkg/mod/github.com/wagslane/go-rabbitmq@v0.6.2/publish.go:160 +0x44

  Goroutine 24 (running) created at:
    testing.(*T).Run()
        /usr/local/go/src/testing/testing.go:1238 +0x5d7
    github.com/stretchr/testify/suite.runTests()
        go/pkg/mod/github.com/rzajac/testify@v1.6.2-0.20200929081604-1a836a85454e/suite/suite.go:203 +0xf7
    github.com/stretchr/testify/suite.Run()
        go/pkg/mod/github.com/rzajac/testify@v1.6.2-0.20200929081604-1a836a85454e/suite/suite.go:176 +0x944
    git.vonroll-infratec.com/go/meas/pkg/queue.TestRabbit()
        ws/go/meas/pkg/queue/rabbit_test.go:19 +0xa4
    testing.tRunner()
        /usr/local/go/src/testing/testing.go:1193 +0x202
  ==================

in my tests.

OK so I run all my tests with race detector. I was able to fix race errors by returning pointer to the Publisher: https://github.com/wagslane/go-rabbitmq/blob/897061950af91da58bb937bff935bd39a699b8de/publish.go#L137

@rzajac Would you mind opening a PR with the fix you described (assuming its the right way to fix this)

@wagslane Well the fix was just a first thought. Not sure if this is the right way to do it. I can do the PR.

Has this been fixed? I am also seeing the data race.

From a quick skim through the code, it looks like you write to the various channel handles inside mutex locks, but read from them without locking in the publishing codepath.

@rzajac Can you try the latest? Should be fixed


Pulse: Add missing synchronizations

Add missing mutex protection to methods that communicate with PulseAudio thread. The methods are

pulse_enumerate_devices
pulse_register_device_collection_changed

The problem reproduced by firing a number of threads that call the methods.

Thanks!


Gas Profiler

Changes
This adds gas profiling to the test suite.
It will detect improvements and regressions.
Reviewers @huangsuyu @HaoyangLiu

This is ready for review @HaoyangLiu

The "timestamp" only functions as salt, so I remove the require statements. They break the profiling because the zeroeness of the timestamp bytes must not change or the measured gas will change. The only available workaround involves starting ganache-cli in the past, and restarting it every hour, which is not a great developer experience. Unfortunately the ganache developers rejected a pull request that would have made it very easy to workaround this.

I've gotten the ganache-cli workaround to work. The only downside is that you have to restart ganache every hour.

So, I can keep the require statement and you can restart ganache before running tests, or we can agree the timestamp is just salt and remove the require statement. Let me know which you prefer @HaoyangLiu

You can revert the latest commit if you agree we can drop the require statement. Otherwise you will have to restart ganache oncer per hour to run tests.

We can't just solidify the timestamp which will result in the following problems.
><EMAIL_ADDRESS>test /Users/liuhaoyang/workspace/solidity/bep3-smartcontracts/ethereum
> truffle test --network test

Using network 'test'.

Compiling your contracts...
===========================
> Everything is up to date, there is nothing to compile.

Error: Cannot find module 'ganache-time-traveler'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:582:15)
    at Function.Module._load (internal/modules/cjs/loader.js:508:25)
    at Module.require (internal/modules/cjs/loader.js:637:17)
    at require (internal/modules/cjs/helpers.js:22:18)
    at Object.<anonymous> (/Users/liuhaoyang/workspace/solidity/bep3-smartcontracts/ethereum/test/ERC20AtomicSwapTest.js:6:22)
    at Module._compile (internal/modules/cjs/loader.js:701:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:712:10)
    at Module.load (internal/modules/cjs/loader.js:600:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:539:12)
    at Function.Module._load (internal/modules/cjs/loader.js:531:3)
    at Module.require (internal/modules/cjs/loader.js:637:17)
    at require (internal/modules/cjs/helpers.js:22:18)
    at /Users/liuhaoyang/workspace/solidity/bep3-smartcontracts/ethereum/node_modules/mocha/lib/mocha.js:250:27
    at Array.forEach (<anonymous>)
    at Mocha.loadFiles (/Users/liuhaoyang/workspace/solidity/bep3-smartcontracts/ethereum/node_modules/mocha/lib/mocha.js:247:14)
    at Mocha.run (/Users/liuhaoyang/workspace/solidity/bep3-smartcontracts/ethereum/node_modules/mocha/lib/mocha.js:576:10)
    at resolve (/Users/liuhaoyang/workspace/solidity/bep3-smartcontracts/ethereum/node_modules/truffle/build/webpack:/packages/truffle-core/lib/test.js:116:1)
    at new Promise (<anonymous>)
    at Object.run (/Users/liuhaoyang/workspace/solidity/bep3-smartcontracts/ethereum/node_modules/truffle/build/webpack:/packages/truffle-core/lib/test.js:115:1)
    at process._tickCallback (internal/process/next_tick.js:68:7)
Truffle v5.0.29 (core: 5.0.29)
Node v10.15.3

We can temporarily comment out the timestamp checking in the contract. And once the gas profile work is done, we can revert it back.

You have to npm install for the new dependency
On Fri, Aug 9, 2019 at 12:43 AM HaoyangLiu<EMAIL_ADDRESS>wrote:

We can't just solidify the timestamp which will result in the following
problems.

<EMAIL_ADDRESS>test /Users/liuhaoyang/workspace/solidity/bep3-smartcontracts/ethereum
truffle test --network test

Using network 'test'.
Compiling your contracts...

Everything is up to date, there is nothing to compile.

Error: Cannot find module 'ganache-time-traveler'
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:582:15)
at Function.Module._load (internal/modules/cjs/loader.js:508:25)
at Module.require (internal/modules/cjs/loader.js:637:17)
at require (internal/modules/cjs/helpers.js:22:18)
at Object. (/Users/liuhaoyang/workspace/solidity/bep3-smartcontracts/ethereum/test/ERC20AtomicSwapTest.js:6:22)
at Module._compile (internal/modules/cjs/loader.js:701:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:712:10)
at Module.load (internal/modules/cjs/loader.js:600:32)
at tryModuleLoad (internal/modules/cjs/loader.js:539:12)
at Function.Module._load (internal/modules/cjs/loader.js:531:3)
at Module.require (internal/modules/cjs/loader.js:637:17)
at require (internal/modules/cjs/helpers.js:22:18)
at /Users/liuhaoyang/workspace/solidity/bep3-smartcontracts/ethereum/node_modules/mocha/lib/mocha.js:250:27
at Array.forEach ()
at Mocha.loadFiles (/Users/liuhaoyang/workspace/solidity/bep3-smartcontracts/ethereum/node_modules/mocha/lib/mocha.js:247:14)
at Mocha.run (/Users/liuhaoyang/workspace/solidity/bep3-smartcontracts/ethereum/node_modules/mocha/lib/mocha.js:576:10)
at resolve (/Users/liuhaoyang/workspace/solidity/bep3-smartcontracts/ethereum/node_modules/truffle/build/webpack:/packages/truffle-core/lib/test.js:116:1)
at new Promise ()
at Object.run (/Users/liuhaoyang/workspace/solidity/bep3-smartcontracts/ethereum/node_modules/truffle/build/webpack:/packages/truffle-core/lib/test.js:115:1)
at process._tickCallback (internal/process/next_tick.js:68:7)
Truffle v5.0.29 (core: 5.0.29)
Node v10.15.3
We can temporarily comment out the timestamp checking in the contract. And
once the gas profile work is done, we can revert it back.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/binance-chain/bep3-smartcontracts/pull/3?email_source=notifications&email_token=AAGDGVJFMGGJKRRTKC2I5WDQDUGZDA5CNFSM4IKPTMGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD35YILA#issuecomment-519799852,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAGDGVONHV6MQ7YQHP447MLQDUGZDANCNFSM4IKPTMGA
.

The ganache-time-traveler was creating issues for @HaoyangLiu so I've reverted that change.

@HaoyangLiu ptal

Could you do rebase with develop branch?

I can do the merge-squash commit myself, if you are not familiar with git. Please stop force pushing the develop branch.

See https://github.com/binance-chain/bep3-smartcontracts/pull/9 for the rebased branch

This is merged via #9