Files are not restored on next startup

          Hi everyone,

I just wanted to let the dev team know that I am also experiencing the issue described in this thread.
Sometimes, after a clean closing of the application (done by clicking the upper-right cross), followed by a shutdown of the computer (typically during the night or week-end), it won't reopen the opened files at its next launch (but it will often [or always?] re-open the project folder, but with no files/tab opened).
I don't do anything differently, and I always close vscode myself (not letting the OS, Windows here, forcing its termination), then I always wait a few seconds before shutting down the system.
After startup, I always re-open vscode by typing "vscode" in the windows search bar, and clicking the App from there, if that matters.
That would be amazing if you could make a patch for this issue, because when one is working on a big project and in the middle of implementing something, having the tabs organized in a specific way is super productive, and re-opening all the files manually is time consuming.
Many thanks for your work on vscode by the way!
Originally posted by @FranckQC in https://github.com/microsoft/vscode/issues/160077#issuecomment-1485207205

I am having the same issue referenced above. This is on a Dell XPS laptop running Windows 11. I do not now how to troubleshoot it or where to find any log files that would tell me why it is happening.

Can you clarify what steps you take and what you see?

Thank you for your reply. In settings I have Restore Windows set to "all". I have also set it to "preserve". The behavior is the same. I start VSCode and it opens with my workspace showing in the explorer pane. I open several files from the workspace and work on them.  The next day when I start the laptop back up and start VSCode, it opens with my workspace showing in the explorer pane but none of the files that I had open the previous day are open. I do not know what other settings I should have or where to look for diagnostic information. VSCode is installed in Program Files.
From: Benjamin Pasero @.>
Sent: Friday, April 21, 2023 12:39 PM
To: microsoft/vscode @.>
Cc: David Brady @.>; Author @.>
Subject: Re: [microsoft/vscode] Hi everyone, (Issue #180512)
Can you clarify what steps you take and what you see?

Reply to this email directly, view it on GitHubhttps://github.com/microsoft/vscode/issues/180512#issuecomment-1518143608, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AUG3Y7GSQWMOI3UH33VYLDLXCLA3ZANCNFSM6AAAAAAXG3XVPU.
You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>

Can you try to reproduce with our nightly insider builds? You can give our preview releases a try from: https://code.visualstudio.com/insiders/

I suggest you file a new issue for that. The insiders version you are on will be promoted to stable next week.

Thank you
From: Benjamin Pasero @.>
Sent: Tuesday, April 25, 2023 1:45 PM
To: microsoft/vscode @.>
Cc: David Brady @.>; Author @.>
Subject: Re: [microsoft/vscode] Files are not restored on next startup (Issue #180512)
I suggest you file a new issue for that. The insiders version you are on will be promoted to stable next week.

Reply to this email directly, view it on GitHubhttps://github.com/microsoft/vscode/issues/180512#issuecomment-1522251489, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AUG3Y7H2MPHF2XILTPICI4LXDALRVANCNFSM6AAAAAAXG3XVPU.
You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>

I spoke too soon. I opened vscode this morning and it did not restore my session from yesterday.
From: Benjamin Pasero @.>
Sent: Tuesday, April 25, 2023 1:45 PM
To: microsoft/vscode @.>
Cc: David Brady @.>; Author @.>
Subject: Re: [microsoft/vscode] Files are not restored on next startup (Issue #180512)
I suggest you file a new issue for that. The insiders version you are on will be promoted to stable next week.

Reply to this email directly, view it on GitHubhttps://github.com/microsoft/vscode/issues/180512#issuecomment-1522251489, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AUG3Y7H2MPHF2XILTPICI4LXDALRVANCNFSM6AAAAAAXG3XVPU.
You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>

Hi,
I'm experiencing exactly the same problem on v1.77.3. As with dbradomaha I just have an explorer panel and no previously opened files available. All settings are as they should be as detailed in earlier posts.
In addition it VS code also looses all breakpoints when it is restarted which is a nightmare.
I'll try an insider build to see what happens.
Good luck!

Hi,
Just installed 1.78 insider and this issue is resolved, but now I can't set any breakpoints. Time to dig out an old version for me.

Hi,
I've narrowed down the behaviour. Now using the 1.77.3, it only affects one of my projects as the rest are behaving as expected. If I open the failing project, open files, open another project, open the failing project, the failing project has its windows and breakpoints as expected. If I then close the application and reopen the failing project then we lose the open windows etc.
My solution was to remove the .code-workspace file, rename the folder, open the folder within vs code, and then "Save Workspace As..." in the renamed folder. Now it works.
I need help on how to fully remove a workspace from all settings etc to find where this going wrong. Obviously references to the old folder are stuffing things.

By any chance are you using Windows Subsystem for Linux?
From: CraigJamesHamilton @.>
Sent: Tuesday, May 2, 2023 8:57 PM
To: microsoft/vscode @.>
Cc: David Brady @.>; Author @.>
Subject: Re: [microsoft/vscode] Files are not restored on next startup (Issue #180512)
Hi,
I've narrowed down the behaviour. Now using the 1.77.3, it only affects one of my projects as the rest are behaving as expected. If I open the failing project, open files, open another project, open the failing project, the failing project has its windows and breakpoints as expected. If I then close the application and reopen the failing project then we lose the open windows etc.
My solution was to remove the .code-workspace file, rename the folder, open the folder within vs code, and then "Save Workspace As..." in the renamed folder. Now it works.
I need help on how to fully remove a workspace from all settings etc to find where this going wrong. Obviously references to the old folder are stuffing things.

Reply to this email directly, view it on GitHubhttps://github.com/microsoft/vscode/issues/180512#issuecomment-1532362480, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AUG3Y7GC4TXXV4XI2UQ63V3XEG3ONANCNFSM6AAAAAAXG3XVPU.
You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>

Hi dbradomaha,
No, I'm using Windows 11.


Pipelines failing because of common dependency.

Describe the bug
Pipelines failing because of common dependency. Cargo chef seems to be to blame.
Steps to reproduce
Make a commit and you will see failure.
Expected behavior
Expect the pipeline to build as we had it before we called cargo.toml commn from git repo.
Environment
Any
Additional context
Probably need to think about splitting out the common lib to another repo so that we can build it without running into issues. Or, pushing the common library to crates.io.

We will be unable to fix this until the open issue https://github.com/LukeMathWalker/cargo-chef/issues/4 is resolved.
We can take a look at different docker approaches that doesn't use cargo chef. As it stands we will have to swap out one line to make it work.
# Comment this next line if running docker build locally
common = {path = "../common"}
# To run locally uncomment this line
# common = { git = "https://github.com/federal-courts-software-factory/open-case-filing-system.git", version = "0.1.0" }


Add range checks via halo2_gadgets

This PR adds various range-checks to the HashChip, using halo2_gadgets.
This requires https://github.com/privacy-scaling-explorations/halo2curves/issues/37 to be resolved, which is the case on the main branch. However, since halo2 uses an older version of halo2gadgets, I opted for forking halo2 for now until the next update. I pushed for a new halo2curves release here: https://github.com/privacy-scaling-explorations/halo2curves/pull/45.
Changes:

We now require our scalar field to implement ff::PrimeFieldBits instead of just ff::PrimeField, which is needed for the range check gadgets of halo2gadgets and supported by the BN256 curve.
Added some docstrings, documenting for each gadget which inputs / outputs are range-checked by the gadget and assumed to be range-checked elsewhere. This contributes also to #4.
Added a new range check gadget, wrapping halo2_gadgets::utilities::lookup_range_check::LookupRangeCheckConfig. It supports checking arbitrary ranged (both by checking for a certain bit length and a less-or-equal check).

With these range checks, the prover should be sound.
Fixes #7

For completeness, this is the result of the benchmarks:
benches/proof_generation
                        time:   [2.8074 s 3.0029 s 3.1960 s]
                        change: [+36.560% +45.996% +56.898%] (p = 0.00 < 0.05)
                        Performance has regressed.
benches/key_generation  time:   [2.4879 s 2.5859 s 2.7216 s]
                        change: [+5.5723% +10.494% +17.559%] (p = 0.00 < 0.05)
                        Performance has regressed.
benches/verification    time:   [8.5819 ms 10.060 ms 11.047 ms]
                        change: [+20.492% +36.726% +54.897%] (p = 0.00 < 0.05)
                        Performance has regressed.


Slow open/render pdf files.

I am using am SM-N910H smartphone. It is rather fast system to operate with even large books in pdf format. But this app opened and rendering every new page really slow.

@mrprole Unfortunately I think this is due to the fact that the PDF renderer is written in JavaScript (it's pdf.js), so there's nothing we can do. It's the trade-off to have a more secure PDF viewer, as there are currently no other PDF libraries written in a memory-safe language.

It's slow because I originally implemented it in the most straightforward way and then added proper hardening, error handling and optimizations over time. I only got around to properly optimizing it for the next release. Doing the rendering in a way that's friendly to the Chromium canvas implementation is a 10-100x increase in performance and I added caching and a basic prerendering heuristic on top of that. The problem wasn't pdf.js but how pdf.js was being used.
This fork isn't representative of the security, performance and robustness of the real thing. It was a fork of a very early version before it became an official CopperheadOS project and was relicensed. Among other things, it needed more sophisticated handling of concurrency to avoid race conditions when queuing up actions. Some cleverness is required to finish the sandboxing implementation by taking away content:// access from the WebView too.
It would be best if the F-Droid package and https://play.google.com/store/apps/details?id=org.ninthfloor.copperpdf didn't refer to CopperheadOS. The code should be attributed to Daniel Micay as I wrote it on my own time and it only became a Copperhead project with the commit changing the license.

I'm pretty sure you didn't upload that Google Play Store app but they stole your app id. They're pretending to be ninthfloor.org and who knows what they've done to it and plan on doing with it (ads? malware?). If you wanted you could report it as trademark infringement: https://support.google.com/legal/troubleshooter/1114905.

@thestinger I understand you want to protect your brand. At this point, given the scarce interest, I'll consider killing this app altogether. I'd like to have an opinion on the two persons who contributed so far: @eighthave and @Lesik.
I didn't upload the app the the Google Play Store.

My concern is that I wanted a libre PDF viewer, and this app seemed like a good tradeoff between additional security through sandboxing and functionality/speed.
If the original app becomes available in F-Droid, and contributions move to that repo, I'm completely fine with that. If you plan to kill off this fork with no alternative available in F-Droid, I'd rather have it stay and cherry-pick important patches which @thestinger allows us to pull despite the license.

@Lesik the original app won't land in fdroid because of its non-free license. You are of course free to compile it yourself or even to download it from copperhead's fdroid repository (I didn't try this one and it is not officially supported, but it should work).

I'm aware. Thus I said, if the situation stays the way it is, I'd rather not kill off this repo, being the only way to keep an official F-Droid source for this PDF viewer.

This fork was deployed commercially by at least one company and at least two people have uploaded it to the Play Store. I'm not going to relicense a subset of our changes. They can't be easily applied without taking everything anyway. The whole point of using that license is preventing that kind of commercial usage without giving back to the people writing it. It demonstrates that the license is the right choice.
As I pointed out, this doesn't have a comparable sandbox to the real thing. It doesn't have comparable hardening in general. It has race conditions in how it renders which is not a security issue but is definitely a major robustness issue. The pdf.js code isn't being updated which may end up creating vulnerabilities within the sandbox. Updating pdf.js requires a patch to pdf.js or it will break stuff, and until they decide they are willing to accept our patch upstream it falls under our license too: https://github.com/mozilla/pdf.js/issues/8066.
The current code in https://github.com/CopperheadOS/platform_packages_apps_PdfViewer proves that pdf.js can be used in a way that's blazing fast. There's literally a 10-100x performance improvement by rendering in a different way. I also added an LRU cache for rendered pages and text selection data and prerendering in addition to that. This fork doesn't support text selection at all since it predates adding support for that. Being able to copy text is a fairly essential feature for a PDF Viewer.

We all agree your updated PdfViewer is superior to this fork, but as you know there is a another key difference between the two: the license. While I'm glad you are happy with your licensing choices, this fork is for people who prefer the Apache license. Take it as a fact, I don't want to start a discussion about licenses here.
If this app will get an update I'll remove the reference to CopperheadOS from the F-Droid description, as you seem to prefer so.
I don't care about people trying to sell it on the play store.

We all agree your updated PdfViewer is superior to this fork, but as you know there is a another key difference between the two: the license. While I'm glad you are happy with your licensing choices, this fork is for people who prefer the Apache license. Take it as a fact, I don't want to start a discussion about licenses here.

They might prefer that license but they probably expect to be getting updates to dependencies like pdf.js at a minimum.

I don't care about people trying to sell it on the play store.

You should probably care about them using their app id. It's presented as being tied to your ninthfloor.org domain. If they bundle malware, ads with tracking, etc. it's going to reflect on you. They don't publish the sources they're using to build.
I definitely care about them claiming it's based on the CopperheadOS PDF Viewer when it's only based on my early proof of concept.
As a bonus, they steal an Adobe Reader icon and it's not like they're publishing the sources despite copy-pasting something saying it's forked to keep it 'free'.

@thestinger I'm sorry for what happened. That terrible licensing choice cost you about 1 year and 4 months of work; if you fork the project right before the license change you'll probably be able to recreate everything in months (the GPL code is not lost). Keep it free and you'll have a community with you: people recognize and respect your work. Set a concrete goal and donations will come too (a "donate" button is not enough).

It didn't cost me anything. I own the code.

I'm not doing any more work for free and I'm not going to beg for money. Either I get properly compensated to do the work along with a proper full-time development team or I'm wrapping it up and moving on to something better. It's far too broad a project with huge amounts of in-depth research / work to do alone. It also requires custom hardware to accomplish basic goals.
 I don't see much difference between his greed and your own. You're still focused on getting more stuff from me for free while I suffer through being overworked, not having time for a life outside of this and all while not earning a decent income from it. No thanks.

I see you're still pushing an unmaintained fork of my early proof of concept code without incorporating the security updates and with a misrepresentation of the origin: https://f-droid.org/en/packages/org.ninthfloor.copperpdf/.
I think that's quite dishonest and disrespectful. It's harming people. I realize you only want to push an ideology and don't care about the well being of other people including either myself or the users harmed by being misled on security.

@thestinger the origin is attributed correctly: this software was already part of COS at the time it was forked. The README explains the status and aim of this repository, especially the fact that I'm not actively maintaining it, there is nothing dishonest, and I don't benefit in any way from it. I would be very happy to hand it over to somebody else. Yes, I push an ideology. We clearly disagree on what this means, and I have no interest in discussing it here.
I'm sorry if my previous message was not sensible. I understand you are burned out, get your time back.
Over and out.

the origin is attributed correctly

No, it isn't. It's wrong and is incorrect attribution which is against the terms of the license. I've explained that already and you lied about what you were going to do about it.

this software was already part of COS at the time it was forked

CopperheadOS chose to include it as it did with apps like Etar and F-Droid. That doesn't mean that it was a Copperhead(OS) project.
I expect the attribution to be fixed.

The README explains the status and aim of this repository, especially the fact that I'm not actively maintaining it, there is nothing dishonest, and I don't benefit in any way from it

It should explain that you aren't doing security updates and should be marked that way on F-Droid too. You should stop misleading them into thinking this was based on the hardened implementation which was finished before the documentation was written and should be giving correct attribution.

and I don't benefit in any way from it

Sure...

Yes, I push an ideology.

Yeah, while harming the people that actually do the work.

We clearly disagree on what this means, and I have no interest in discussing it here.

So don't talk about it here.

the origin is attributed correctly

No, it isn't. It's wrong and is incorrect attribution which is against the terms of the license. I've explained that already and you lied about what you were going to do about it.

this software was already part of COS at the time it was forked

CopperheadOS chose to include it as it did with apps like Etar and F-Droid. That doesn't mean that it was a Copperhead(OS) project.
I expect the attribution to be fixed.

The README explains the status and aim of this repository, especially the fact that I'm not actively maintaining it, there is nothing dishonest, and I don't benefit in any way from it

It should explain that you aren't doing security updates and should be marked that way on F-Droid too. You should stop misleading them into thinking this was based on the hardened implementation which was finished before the documentation was written and should be giving correct attribution.

and I don't benefit in any way from it

Sure...

Yes, I push an ideology.

Yeah, while harming the people that actually do the work.

We clearly disagree on what this means, and I have no interest in discussing it here.

I don't want to talk about your religious views on software licenses. I expect you to follow the terms of the license by giving correct attribution. The commits are from<EMAIL_ADDRESS>on my own time and the attribution should be to me.

the origin is attributed correctly

No, it isn't. It's wrong and is incorrect attribution which is against the terms of the license. I've explained that already and you lied about what you were going to do about it.

this software was already part of COS at the time it was forked

CopperheadOS chose to include it as it did with apps like Etar and F-Droid. That doesn't mean that it was a Copperhead(OS) project.
I expect the attribution to be fixed.

The README explains the status and aim of this repository, especially the fact that I'm not actively maintaining it, there is nothing dishonest, and I don't benefit in any way from it

It should explain that you aren't doing security updates and should be marked that way on F-Droid too. You should stop misleading them into thinking this was based on the hardened implementation which was finished before the documentation was written and should be giving correct attribution.

and I don't benefit in any way from it

Sure...

Yes, I push an ideology.

Yeah, while harming the people that actually do the work.

We clearly disagree on what this means, and I have no interest in discussing it here.

I don't want to talk about your religious views on software licenses. I expect you to follow the terms of the license by giving correct attribution. The commits are from<EMAIL_ADDRESS>on my own time and the attribution should be to me as I explained months ago.

Not to mention that you merged code based on the work that I did after I changed the license.

I wrote "If this app will get an update I'll remove the reference to CopperheadOS from the F-Droid description", but there were no new releases since then, so no lies. If you really care, just submit a GitLab merge request to change
https://gitlab.com/fdroid/fdroiddata/blob/master/metadata/org.ninthfloor.copperpdf.txt
While I don't care about what you expect from me, I do care about proper attribution. I will change the README.md to describe this as "an app written by Daniel Micay and distributed with CopperheadOS". You will agree this is completely factual.
I'm not aware of copyright infringments. If you want to make such a claim please be precise, and please do it in a separate issue.

https://github.com/paride/CopperPDF/commit/7f66c30ead5b3ca3f025fc9573de6cb0efe4f78e
How exactly is this okay?

I wrote 7f66c30 entirely on my own time.  I own the copyright on 7f66c30.  The work I did on the CopperheadOS version was totally separate.  Copyright is not patent, copyright covers the actual writing, not the ideas.

@thestinger you really need to go relax somewhere and stop beating this dead horse.  Free software is a thing, no matter how much you argue against it.

Lying about the origin of the code now? I think you need to go relax somewhere and learn to be honest.

Anyway, I'll be taking legal action against people stealing my work including here.

You wouldn't have even touched it if you hadn't been paid to develop features for my app. Nice try though.

@thestinger since you're throwing around accusations, I'll state clearly where the code came from, and the context.  I did some work on the Copperhead PDF Viewer, paid for directly by a client of Copperhead's, with Copperhead put me in touch with.  So they own the copyright, not Copperhead.  I submitted that in a merge request to Copperhead.  Some time afterwards, I found myself wanting to use the improvements I'd made.  It wasn't feasible to use Copperhead PDF Viewer, so I wrote new patches based on this project (CopperPDF) and submitted them in a merge request.  So some ideas were developed while working on that contract, but this implementation was entirely on my own time, using zero of the Copperhead client's code.

Not a truthful account and yet it demonstrates why this isn't legal. You don't own the copyright and it was derived from my work which is not licensed this way.

said with no sense of irony... your work would be entirely worthless without PDF.js, a vastly larger body of work that is free software and developed by other people.

and for the record, @thestinger only involvement in the project was reviewing the merge request, which he never accepted.

Continuing to present an inaccurate / misleading account of what happened doesn't change the situation. You didn't follow through on what you were supposed to do and then violated the license. I expect the infringing code to be removed here.

And in response to your deleted comment: I'm not infringing on the Apache 2 license and am in the process of landing my changes (one of which is public already) upstream too. And by that I don't mean submitting a drive-by contribution, ignoring code review and then violating the license.
It doesn't make much difference that it's this tiny little bit of code where the issue is occurring. I would care a lot more if it was the Auditor app / AttestationServer or other larger / more relevant projects but this whole situation is still seriously wrong.


Replace path params in duration label

Overview
Apply path param transformation to prometheus duration label.

@dmgaldi In WDK we add paths like this to prometheus but as a post-matching filter; then we get the matched URL pattern from Jersey and use that. It means we're not adding metrics for 404s, and it's a nice way to ask the services which portions of the path are variable. Not exactly sure what this code is doing; like what is getPathParameters() if we haven't matched to a service yet? The WDK version (which, granted, seems to do a lot of work) is here: https://github.com/VEuPathDB/WDK/blob/master/Service/src/main/java/org/gusdb/wdk/service/filter/PrometheusFilter.java
EDIT: Actually, this needs some work to match how we do the logging. This filter only records duration from beginning-of-request to beginning-of-response. But for streaming responses, we probably want to know how long until all the data is out the door. That is a little more complex, but we already figured out how to do it here: https://github.com/VEuPathDB/WDK/blob/master/Service/src/main/java/org/gusdb/wdk/service/filter/RequestLoggingFilter.java
The trick is: if a response has a non-empty body, you need to "complete" the request in the WriterInterceptor, because the ContainerResponseFilter is too soon. But if it's an empty body, you have to use the ContainerResponseFilter, because no WriterIntercepter is applied in that case.

I finally updated this PR addressing these comments :)


TNS-3 Channel selection

1st parameter
Be able to select a channel to bind the service to receive messages. Thus, it will be possible to switch between different note channels. Implementation as an enum list.

EDIT: the implementation is more convenient and more flexible if we give each user the opportunity to choose his own chat from his list of chats

f530f3f


Fix tox config

use version agnostic py
disable dangerous ignore of python version which can cause  false positives while testing (like using unexpected version of
python)

recheck

@ssbarnea given the flurry of activity in the repo, is this still needed?

@cidrblock Yes. In fact there are more changes needed but I let them out to ease the review. I also dropped one where I disagree with @webknjaz for the same reason.

use version agnostic py

I thought you were adding more envs to the default invocation and not just tox -e py. Could you reflect that somehow?


Change Week Day Lable color on Sunday to Red

I am trying to change the color of Week day lable on Sunday to Red

Use lib function setWeekDayFormatter(here like parameters put own DayViewDecorator, example WeekendDecorator)
WeekendDecorator(): DayViewDecorator{
put here your condition fo decorate}


Add controls tuning to named.conf

I'll rebase and add docs once #56 is merged.

@ekohl done. I guess once this is merged I'll rebase #56 and document it.

:+1: once tests are green and docs are added.

This is ready to be merged, per adding the docs to #56.

:+1:

merged, dzięki @jyaworski!


Migrating from Chibicc

The Type, Obj and Node data structures seem to have a pretty different semantics in widcc than in chibicc.
What is the correct way to :

iterate over all parameters of a function,
iterate over all local variables of a function,
iterate over all actual args (Node and expression per actual argument) of a ND_FUNCALL node?

For 1. 2.
static void visit_params(Obj *var) {
  for (; var; var = var->param_next) {
    if (var->name)
      printf("param: %s\n", var->name);
    else
      printf("unnmaed param of type %d\n", var->ty->kind);
  }
}
static void visit_vars(Scope *sc) {
  for (Obj *var = sc->locals; var; var = var->next) {
    if (var->name)
      printf("lvar: %s\n", var->name);
    else
      printf("unnmaed lvar of type %d\n", var->ty->kind);
  }

  for (Scope *sub_sc = sc->children; sub_sc;) {
    visit_vars(sub_sc);
    sub_sc = sub_sc->sibling_next;
  }
}

static void dump_function_vars(Obj *prog) {
  for (Obj *fn = prog; fn; fn = fn->next) {
    if (!fn->is_function || !fn->is_definition)
      continue;
    visit_params(fn->ty->param_list);
    visit_vars(fn->ty->scopes);
  }
}

For 3. I think I can simplify the layout a bit

https://github.com/fuhsnn/widcc/blob/85ec382abf32524a81fae58caa5d43973e1f15f1/parse.c#L3039-L3041
The assignment node store evaluation result into the temporary lvar, then later place_stack_args() place_reg_args() copy the stored results into calling convention specified places.
You can drop the assign node and make dummy Obj*s that don't allocate lvar space if the backend have better solutions.

Great, thank you very much for the information and the new commit. I try to refactor my code generators accordingly.

Meanwhile I was able to integrate the mentioned commit and adapt my code (https://github.com/rochus-keller/EiGen/tree/widcc); I currently test with the nolib test cases and still get a few segfaults and code generator asserts which I have to investigate; so I assume that I haven't yet properly considered all of your design changes.
May I ask whether and where you allocate an extra parameter in case of struct/union function returns? I also haven't understood why you need an extra list for param_promoted and how it relates to the param_next list.

The segfaults seem to be associated with gotos and switch cases. As it seem, I would have to invest quite a lot of time in debugging, which is too much for me at the moment. I will therefore continue with Libfirm for the time being and come back to the widcc branch later.

extra parameter in case of struct/union function returns

The pointer is saved on stack space without making an extra parameter Obj*:
https://github.com/fuhsnn/widcc/blob/a0bb0108ab4f6ed97556400b7bf298ec49bdc820/codegen.c#L1628-L1634
Because return struct pointer being the first parameter is x86 sys-V specific, I believe this makes the frontend more platform-agnostic.

why you need an extra list for param_promoted

It's for old style functions, because their arguments are always default-promoted. The calling convention use default-promoted types; and callee cast received parameter from promoted type back to locally declared types.
For example: https://godbolt.org/z/zTEndbnfb
A cast instruction is emitted because float are always passed as double in old-style.

Libfirm/cparser

I'd love to see it maintained up-to-date, at least not failing on python versions (bit funny for a c99 project).
There was another one long forgotten - Open64, I always feel like getting it working for historical preservation.

Thanks for the information.
Meanwhile I was able to build my Awfy reference project with cparser and run benchmarks with different optimization levels. Here are the detailed results.
The executable generated by cparser 1.22.1 -O2 is 8% slower than the one generated with gcc 4.8 -O2, but the build time with cparser is 4.3 times slower than with gcc. In the unoptimized case the executable generated with cparser is 36% faster (!) than the one generated with gcc, but the build time with cparser is still 3.7 times slower than with gcc.
So cparser produces very fast code, but takes a lot of time for it.
I already run a subset of Awfy with my chibicc version with Eigen backend (which does no optimization at all) and found that the generated executable is 3 to 4 times slower than the unoptimized gcc generated executable. I will try to build the Boem GC with my chibicc version and then run the full Awfy to enable fair comparison.

at least not failing on python versions

I was able to build it by using the last commit of libfirm before they switched to Python 3, just using Python 2 instead. I agree that it is pretty silly to add a Python dependency to the C compiler; there is actually also a Perl dependency to generate parts of the backend code.

Just had a look at Open64 (https://en.wikipedia.org/wiki/Open64); apparently they used GCC as a frontend, so it's not interesting for me (GCC is a monster).

Meanwhile I managed to integrate the Boem gc and run the full awfy. The detail results are in the referenced report. The chibicc/ecc generated code runs 3.5 times slower than the executable generated from the same code using GCC -O2.
Maybe you want to run performance measurements yourself using widcc. Just in case, here is the source code. When you run the resulting executable, a report is printed to the console.

3.5 times slower than the executable generated from the same code using GCC -O2.

It's... not unexpected, there are tons of dirty hacks in chibicc just to do correct things the simplest way, (I follow that direction in widcc, and only patched some out in slimcc), it's kind of a "so I'm doing this in one way, that one way must account for the worst-case, forgo the others" attitude, for example (num += 7) is strictly worse than (num = num + 7) because the former introduced an extra temporary just to account for the case that the calculation result must be put on %rax, introducing push-pop nodes in AST would help, but in that philosophy that's considered unnecessary complexity.
And there's the volatile thing, chibicc omits the volatile qualifier, which means all memory operation in user code must be implicitly volatile to not break stuff in subtle ways (cproc/qbe does this too), therefore... in all honestly, even if you hook LLVM to chibicc the best it can do will still be worse than clang -O1.


Docker setup

Addresses #3 --

Creates Dockerfile to package api code into an image.
Creates a docker-compose.yaml to automatically build and run the container.
Updates repository structure to clean up Python imports and avoid PYTHONPATH errors in the container.

To test locally:
docker compose up
curl localhost:5000

I will create a separate issue to update the default server to use something more production-ready, like uWSGI or nginx before deploying to Azure.

Oh also I don't think Addresses has been working as a closing keyword for Github. You might want
Closes #3


[BUG] do_not_fail_on_forbidden_empty does not work for cat api

What is the bug?
Two issues:

Setting do_not_fail_on_forbidden to true does not seem to have any impact on some APIs like cat/_indices and cat/_aliases - the whole operation is rejected even if indices exist for which the user has access to.

_cat/indices/some-index-* would work, but not _cat/indices because it include internal indices which the user does not permission for.
Adding permissions under index_patterns: '*' can make them work, but I think this defeats the purpose of having the do_not_fail_on_forbidden parameter.

do_not_fail_on_forbidden does seem to work for _search, but the search result includes internal indices (such as .kibana_1), which is another issue

How can one reproduce the bug?
Steps to reproduce the behavior:
roles.yml
my_user:
  cluster_permissions:
    - "cluster_monitor"
    - "cluster_manage_index_templates"
  index_permissions:
  - index_patterns:
    - "some-index-*"
    allowed_actions:
      - "read"
      - "write"
      - "create_index"
      - "manage"

config.yml
config:
  dynamic:
    do_not_fail_on_forbidden: true
    do_not_fail_on_forbidden_empty: true
    ...

Error for GET _cat/indices
        "type" : "security_exception",
        "reason" : "no permissions for [indices:monitor/settings/get] and User [name=my_user, backend_roles=[], requestedTenant=null]"

Error for GET _cat/aliases
        "type" : "security_exception",
        "reason" : "no permissions for [indices:admin/aliases/get] and User [name=my_user, backend_roles=[], requestedTenant=null]"

Both of these permissions should be covered under the manage default_action_group, but the operations only work when targeted specifically to the index, so do_not_fail_on_forbidden does not seem to be working as intended.
Meanwhile, after setting do_not_fail_on_forbidden: true, GET _search works, but includes an index that should not be there because the user does not have access for that
{
    ....
    "hits" : [
      {
        "_index" : ".kibana_1",
        "_type" : "_doc",
        "_id" : "config:1.2.0",
        "_score" : 1.0,
        "_source" : {
          "config" : {
            "buildNum" : 86767
          },
          "type" : "config",
          "references" : [ ],
          "migrationVersion" : {
            "config" : "7.9.0"
          },
          "updated_at" : "2022-03-10T13:39:33.452Z"
        }
      },
      {
        "_index" : "some-index-001",
        "_type" : "_doc",
        "_id" : "3",
        "_score" : 1.0,
        "_source" : {
          "field1" : "value3"
        }
      }
    ]
  }
}

What is the expected behavior?
do_not_fail_on_forbidden should work similarly for all APIs, where results are filtered based only on the indices that the  user has permissions for.
What is your host/environment?

Version OpenSearch 1.2.3
Plugins OpenSearch Security <IP_ADDRESS>

Thanks for filing @jezsy, I am having trouble reproducing the issue with /_search.  As that data might be sensitive, could you email me directly at<EMAIL_ADDRESS> Sending the security-auditlog-* log entries, should give me the data I need to dive into this further.

Hi @peternied. Apologies for the late response. Upon further tests, it seems my claim about /_search was wrong. kibana_1 was appearing in the results because I had my user mapped to the kibana_user role as well (was doing tests on the Dashboards Dev Tools), which had permissions for the kibana indices.
In that case, the main issue now is just extending usage of parameters do_not_fail_on_forbidden and do_not_fail_on_forbidden_empty to other APIs like _cat (e.g. indices, aliases, shards, maybe more)

Similarly, I think the two parameters should also apply GET _alias and GET _aliases. At the moment, the whole operation is rejected instead of just filtering for those indices which the user does have permission for, even if the parameters are configured true

Thanks for updating us on this issue, we have added this issue into items to be looked at so thank you very much for contributing to this project by filing.  Please feel free to open a pull request if you've got recommendations for this or other features.


Visual object detection: support non maximum suppression method

When performing object detection in images, multiple shaped boxes can be found for the same object (overlapping).  This method should take the best matching shapes and return that.
This method is use non maximum suppression method
Sample code:
def nms(boxes, overlapThresh):
    # if there are no boxes, return an empty list
    if len(boxes) == 0:
        return []

    # initialize the list of picked indexes
    pick = []

    # grab the coordinates of the bounding boxes
    x1 = boxes[:,0]
    y1 = boxes[:,1]
    x2 = boxes[:,2]
    y2 = boxes[:,3]
    prob = boxes[:,4]

    # compute the area of the bounding boxes and sort the bounding
    # boxes by the bottom-right y-coordinate of the bounding box
    area = (x2 - x1 + 1) * (y2 - y1 + 1)
    idxs = np.argsort(prob)
    # keep looping while some indexes still remain in the indexes
    # list
    while len(idxs) > 0:
        # grab the last index in the indexes list, add the index
        # value to the list of picked indexes, then initialize
        # the suppression list (i.e. indexes that will be deleted)
        # using the last index
        last = len(idxs) - 1
        i = idxs[last]
        pick.append(i)
        suppress = [last]
        # loop over all indexes in the indexes list
        for pos in range(0, last):
            # grab the current index
            j = idxs[pos]
            # find the largest (x, y) coordinates for the start of
            # the bounding box and the smallest (x, y) coordinates
            # for the end of the bounding box
            xx1 = max(x1[i], x1[j])
            yy1 = max(y1[i], y1[j])
            xx2 = min(x2[i], x2[j])
            yy2 = min(y2[i], y2[j])
            # compute the width and height of the bounding box
            w = max(0, xx2 - xx1 + 1)
            h = max(0, yy2 - yy1 + 1)
            # compute the ratio of overlap between the computed
            # bounding box and the bounding box in the area list
            overlap = float(w * h) / area[j]
            # if there is sufficient overlap, suppress the
            # current bounding box
            if overlap > overlapThresh:
                suppress.append(pos)
        # delete all indexes from the index list that are in the
        # suppression list
        idxs = np.delete(idxs, suppress)
    # return only the bounding boxes that were picked
    return boxes[pick]

An example where this is needed and used for, is object detection. Consider the following image, where you want to detect the location of the Diamond on the playing card. That image can be detected multiple times, but this NMS (Non Maximum Suppression) method will pick the best rectangle that fits the diamond and avoid the overlapping rectangles.

Source & Article


Added command line flags to codebase

Created cmdline.py to handle the logic of command line flags, and all other files that originally used hard coded flags now import arguments from cmdline.py. Note that if no arguments are specified, the default values for those arguments are used, which can be seen by the default parameter for each argument.
To run with specific arguments, simply add a --[argument name] [desired argument value] for each argument after the original command to run. To execute run.py with Coq, simply execute python run.py --language "Coq".

Thanks!


App Crash due to high resolution Ground Overlay image

I'm submitting a ...  (check one with "x")

[x] question
[ ] any problem or bug report

OS:  (check one with "x")

[x] Android
[ ] iOS
[ ] Browser

cordova information:  (run $> cordova plugin list)
com.ionic.keyboard 1.0.4 "Keyboard"
cordova-plugin-console 1.0.2 "Console"
cordova-plugin-device 1.1.1 "Device"
cordova-plugin-googlemaps 2.3.8 "cordova-plugin-googlemaps"
cordova-plugin-whitelist 1.2.2 "Whitelist"

Current behavior:
Ground Overlay image with resolution 1920x1690 and size 887KB causes the app to crash on Moto-E (1GB RAM Android 5.1) and Mi Redmi 4A (2GB RAM Android 7.1)
Hello
Is there a limitation to the maximum size and resolution of the Ground Overlay image we can use in the plugin?
Using a Ground Overlay image with resolution 1920x1690 and size 887KB causes the app to crash on Moto-E (1GB RAM Android 5.1) and Mi Redmi 4A (2GB RAM Android 7.1)
What is the best possible image size and resolution we should use?
Thanks and Regards.

Out of memory. Reduce image size.

1920x1690

1920 x 1690 x 4 (rgba) = 12,979,200 (12.3MB)
In order to resize the image, the plugin needs to almost the same memory.
12.3 * 2 = 24.6MB.
And then, native Google Maps API display it as texture.
That spends many memory.
Android does not allow to use such a much memory space.

ping


Keybase prevents text expansion with TextExpander OS X when not logged in.

I've noticed that when I'm not logged into Keybase it prevents TextExander from expanding text. It is as if the Keybase login windows keeps stating that secure text input is active (and therefore TextExpander prevents expansion, this is a built in security measure. I think it should not do this until I really open the Keybase login screen in the client app.
Thanks for looking into this.
I'm on OS X Sierra 10.12.1 b4
my log id: d1d5a73b354f1c60f8959a1c

It prevents also Open Siri with press and hold options + space.
It prevents also press and hold for accents on docs.
When logged in, everything work as expected.


Update cloudbuild.yaml

Use $REPO_NAME instead of $_REPO_NAME for variable substitution, as $_REPO_NAME is incorrect.

Hi @g-samroberts. _REPO_NAME in this case is the Artifact Registry repository, not the GitHub repository.
This variable could be renamed for clarity, but this would require updating the docs to suit.

b/363222262


Added Getting Started page for Azure, and added appropriate references

Co-Authored-By: Maanav Dalal<EMAIL_ADDRESS>Co-Authored-By: Rebecca McFadden<EMAIL_ADDRESS>

This change is

Thanks for this PR! There were a couple of prior PRs that did the same work:

https://github.com/kubeflow/website/pull/789
https://github.com/kubeflow/website/pull/800

So I'll close this one. :)
/close


Evaluation suite - nan handling

Context
Most of the evaluation metrics fall into non defined ranges (infinite numbers / division by 0) and these cases must be carefully treated when presenting the results and / or aggregating the data
The users need to be aware of nan values (and some nan values are actually valid outputs)
Describe the solution you'd like
The most flexible way to do this would be to be able to call a specific function as option from the user as different nan-handling strategies exist - those will be progressively documented in ([https://docs.google.com/spreadsheets/d/1I2lAlVCjnPtUCYjfz927MAYFLeLswYmf/edit?dls=true])
It may make sense for each metric to output two values - the non-nan handled and the nan - handled value as a user may want to know which are the nan-value cases
Describe alternatives you've considered
Add optional default behaviours for nan functions (for instance epsilon value on both numerator and denominator)
Additional context
Further discussion will be needed to see what is the best approach to this
Discussion with @AReinke @jenspetersen @danieltudosiu

There is specific nan-handling situations that are handled now and multiple ways of aggregating are also proposed.


Add Japanese(ja-jp) Localization

Sorry, Data/ja-jp.json has not been changed.
I am a beginner in C# and VisualStudio.
There is a possibility that I am doing something wrong. If so, I'm sorry.

The form's default font is set to Malgun Gothic, but this font can not display Japanese nicely.
In this screenshot, I use Meirio font for the label with the check.
This font is an easy-to-read font that is standard in Japanese Windows browsers.
It may be necessary to change the font depending on the language.

Thank you for contributing!

I'll add the font support soon.


Including timestamp in metric

Hello,
I'd like to include the timestamp in the metric according to the Prometheus documentation: https://prometheus.io/docs/instrumenting/exposition_formats/
metric_name [
  "{" label_name "=" `"` label_value `"` { "," label_name "=" `"` label_value `"` } [ "," ] "}"
] value [ timestamp ]

I'd like that feature for the following reason: Some exporters take a while to gather certain information. Therefore I would like to run them asynchronously, not only when the endpoint is scraped. To guarantee that the metrics are as accurate as possible, I'd like to include the timestamp with the metric.
Currently, I don't think this is possible.
Thanks

This isn't currently possible, but a patch to add it would be accepted. #176 is related.
Note that using the timestamp is only recommended when pulling data from another monitoring system that has timestamps, and not when caching.

Has there been any change in regards to this issue? I'd also like to parse timestamps.

What's the timeframe for the openmetrics work? I see that on the roadmap it says near future, but can you share more details? Thank you!

It'll likely be quite soon. However based on how you have described your use case, you should not use timestamps.

Note that using the timestamp is only recommended when pulling data from another monitoring system that has timestamps, and not when caching.

Hi i could use this too. my use case is using https://github.com/GoogleCloudPlatform/k8s-stackdriver/tree/master/prometheus-to-sd to push metrics in prometheus format to stackdriver. Strackdriver needs to know what time range the counters are from / to for them to work

I have question regarding the timestamp value type. The prometheus docs say:

The timestamp is an int64 (milliseconds since epoch, i.e. 1970-01-01 00:00:00 UTC, excluding leap seconds), represented as required by Go's ParseInt() function.

Looking at the python parser and its tests floats (I think microseconds) are used for the timestamps.
What is the right format / type? Am I missing something?

It's seconds for openmetrics.

@brian-brazil since what version? in V2.1.0 prometheus parses the date correctly only with (unix timestamp (s) * 1000).

Just convert timestamp value to ms(miliseconds). For example:
timestamp = int(float(datetime.datetime.now().timestamp()) * 1000)


fix misspelled Europe in some azure regions descriptions

Hey guys,
Sorry to be pernickety but I found out that "Europe" was misspelled "Eurpoe" in some azure's region descriptions and so was the region's label in the UI so I wanted to fix that.
Thanks !

@AlexLamande Thanks!


Submitting via enter doesn't reset the unsaved change warning

Description
The warning that shows when you attempt to leave the edit screen with unsaved changes will show up when changes have been saved by pressing enter in one of the fields.
Suspect all that is needed is to move an event listener from the submit button to the form submit event but haven't looked into this

Steps to reproduce

Edit a model which has regular text inputs
Make a change
Press enter while focused on one of the text fields - the form will successfully save
Attempt to leave the page

Expected result

No 'are you sure you want to leave' confirmation
Actual result

The 'are you sure you want to leave' confirmation shows
Versions
Twill version: 2.x-dev (9b276c5)
Laravel version: 8.75.0
PHP version: 8.1.0
Database engine: MySQL 5.7.34

Hi @freshleafmedia, thanks for reporting this.
Unfortunately, I can't replicate the bug following your instructions: pressing enter on an input field and refreshing the page doesn't show the confirmation message.
Although, I've noticed that the following steps do trigger the confirmation message:

Edit a model which has regular text inputs
Click in one of the inputs to bring focus to it
Click out of the input to remove focus from it
Attempt to leave the page (without saving)

Does this make sense? Can you replicate it?


Add rust-glx to homu repos

@bors-servo r+

:pushpin: Commit 0fc5739 has been approved by metajack

:hourglass: Testing commit 0fc5739 with merge 96a2676...

:sunny: Test successful - travis


a 500 response on a request breaks stuff

if I do:
m.request({type: 'GET', url: '/some/url/', contentType: 'application/json'}).then(function (data) {
  console.log('success', data);
}, function (err) {
  console.log('error', err);
})

and the server returns a 500, it'll break on this line in mithril, without ever getting into my reject block above.
var response = unwrap(deserialize(extract(e.target, xhrOptions)), e.target);
Uncaught SyntaxError: Unexpected token <

I can fix this by passing extract into the request:
extract: function (xhr, xhrOptions) {
  if (xhr.status === 500) return xhr.status;
  else return xhr.responseText;
}

Is this expected behavior? Shouldn't the request be able to continue gracefully even without an extract option?

It sounds like the server is returning some HTML content on a 500 error but mithril expects JSON by default (deserialize option). Docs

yeah it is serving raw HTML. My question is whether it should cause a blocking error or proceed gracefully like jQuery / zepto do?

This has been discussed many times in issues and the forums. As it stands, anything returned that is not JSON must be handled by you. I understand your reasoning, but m.request is a complicated mechanism that returns an m.prop wrapped in a promise. It can also trigger a redraw. Handling non-JSON responses gracefully would require reworking several integrated parts of Mithril core. Mithril would have to grow and might suffer speed penalties.
m.request, is a built-in extra that does simple/limited ajax requests. You're expected to use a 3rd party library or write your own code if you want more/different functionality.
This is just a consequence of keeping Mithril small and on target.

fair enough -- wasn't sure if it was intentional or not.

@dannycochran I know this issue is closed but I was reading the documentation of m.request and found that there is a way to deal with such case: https://lhorie.github.io/mithril/mithril.request.html#using-variable-data-formats
You can use the extract option to specify a function that's going to manage the response.

Indeed. I noted as such in my original comment :)


[small, flakes] How to make a flake with a python library, supporting arbitrary Python versions?

For a python application, it's easy to find examples of flakes exporting a package.
But what if I'm packaging a library and I want it to support a range of python versions? What am I supposed to export (a function taking pythonPackages and producing my library) and which flake output should this function be exposed under?

Since this has only been briefly covered in NH29, @t184256 if you end up with a satisfying complete solution, would you care to post it here (or commit it in a 29 folder)?
I imagine it would look something like this:
{

  description = "Minimal example for Python Libraries as overlay";

  outputs = { self, nixpkgs }:
  let
    system = "x86_64-linux";
    myPyLibOverlay = final: prev: {
      myPyLib = final.callPackage ./packages/myPyLib.nix { };
    };
  in
  {

    overlays = {
      customPyLibs = final: prev: {
        # pythonPackagesExtensions (PR #91850) merged Aug 6, 2022
        pythonPackagesExtensions = prev.pythonPackagesExtensions ++ [
          myPyLibOverlay
          #...
        ];
      };
    };

    nixosConfigurations = {
      someMachine = nixpkgs.lib.nixosSystem {
        inherit system;
        modules = [
          { nixpkgs.overlays = [ self.overlays.customPyLibs ]; }
          ./machines/some/configuration.nix
        ];
      };
    };

  };

}

Please correct me if I'm wrong.

Or using it in a flake package/app? Again, not sure if the import nixpkgs { overlays = ...;  } is evil or there's a better way.

I don't think the

import nixpkgs { overlays = ...;  } part is evil or there's a better way.

I was asking that for a library I'm writing in the open, so I can share the resulting commit: https://github.com/t184256/asyncdbview/commit/f0acee7847c1184daf058903d30a3177a65f170b
As I understood it, the minimal example would be something like this:
{
  description = "mylib/flake.nix example providing an overlay";
  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs { inherit system; overlays = [ overlay ]; };
        overlay = final: prev: {
          pythonPackagesExtensions =
            prev.pythonPackagesExtensions ++ [ mylib-pyextension ];
        };
        mylib-pyextension = pyFinal: pyPrev: {
          mylib = pkgs.callPackage mylib-package { python3Packages = pyFinal; };
        };
        mylib-package = {python3Packages}:
          python3Packages.buildPythonPackage {
            pname = "mylib";
            version = "0.0.1";
            src = ./.;
          };
      in
      {
        overlays.default = overlay;
      }
    );
}

{
  description = "myapp/flake.nix example using the overlay";
  inputs.mylib.url = "...";
  outputs = { self, nixpkgs, flake-utils, mylib }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs {
          inherit system;
          overlays = [ mylib.overlays.${system}.default ];
        };
        python3Packages = pkgs.python312Packages;  # <- non-standard
        myapp = python3Packages.buildPythonPackage {
          pname = "myapp";
          version = "0.0.1";
          src = ./.;
          propagatedBuildInputs = with python3Packages; [ mylib ];  # <-
        };
      in
      {
        packages.default = myapp;
      }
    );
}

Okay, so I played around a little bit, and I have a good bunch of questions related to this, that I think would be a good fit for a Nix Hour. @infinisil would you be willing to make a follow up on custom Python libraries and flakes? @t184256 would you participate in that? I could make it to any of the next two Nix Hours on 23rd ant 30th of May.
For a little heads-up, here is my experiment repo.
Since we haven't talked much about flakes on the Nix Hour it could also serve as a nice practical introduction to talk about it. I'd be willing to give a quick walk-through on air.
Concrete things, I have problems with:

How do I write tests for flakes?
How do I build nixosConfigurations (without nixos-rebuild)?
How do I use nixosModules in nixosTest?

Some of these would nicely tie into topics you've covered on the Nix Hour.

Okay, so I played around a little bit, and I have a good bunch of questions related to this, that I think would be a good fit for a Nix Hour. @infinisil would you be willing to make a follow up on custom Python libraries and flakes? @t184256 would you participate in that? I could make it to any of the next two Nix Hours on 23rd ant 30th of May.
For a little heads-up, here is my experiment repo.
Since we haven't talked much about flakes on the Nix Hour it could also serve as a nice practical introduction to talk about it. I'd be willing to give a quick walk-through on air.
Concrete things, I have problems with:

How do I write tests for flakes?
How do I build nixosConfigurations (without nixos-rebuild)?
How do I use nixosModules in nixosTest?

Some of these would nicely tie into topics you've covered on the Nix Hour.

I've since noticed that overlays shouldn't go under flake-utils.lib.eachDefaultSystem. Please refer to my template repository https://github.com/t184256/cookiecutter-python-nix for a hopefully better and more complete example of exporting an overlay with a python library (there's a reference output directory if you want to take a look at an example output).


(v21.x) CacheKV speedups

Tested against mainnet for state compatability. cref #504

Lets just cherry-pick this commit into the main branch? https://github.com/osmosis-labs/cosmos-sdk/pull/504

Which commit @ValarDragon. The PR you linked should be in osmo/v0.47.5 already

Closing in favor of https://github.com/osmosis-labs/cosmos-sdk/pull/525


Is it possible to disallow direct extension, but allow indirect extension?

I wanna start of with by thanking everybody working on this tool for what it makes possible! It's great and helps me and my team to build better software in guidance with our architectural rules.
My question comes from a want to only allow our internal code to make use of our own exception layer.
The setup is roughly as follows:
final class specificException extends CompanyInputException
{}

class CompanyInputException extends CompanyRuntimeException
{}

Abstract class CompanyRuntimeException extends \RuntimeException
{}

final class SomeService
{
    public function foo()
    {
        throw new \RuntimeException(); // This should not be allowed
    }
}

final class AnotherService
{
    public function foo()
    {
        throw new CompanyRuntimeException(); // This should be allowed
    }
}

The config I'm trying to work with is roughly follows
deptrac:
  paths:
    - src

  analyser:
    types:
      - "class"

  layers:
    - name: Base_Php_Exceptions
      collectors:
        - type: className
          value: ^Exception$
        - type: className
          value: ^LogicException$
        - type: className
          value: ^BadFunctionCallException$
        - type: className
          value: ^BadMethodCallException$
        - type: className
          value: ^DomainException$
        - type: className
          value: ^InvalidArgumentException$
        - type: className
          value: ^LengthException$
        - type: className
          value: ^OutOfRangeException$
        - type: className
          value: ^RuntimeException$
        - type: className
          value: ^OutOfBoundsException$
        - type: className
          value: ^OverflowException$
        - type: className
          value: ^RangeException$
        - type: className
          value: ^UnderflowException$
        - type: className
          value: ^UnexpectedValueException$

    - name: Company_Base_Exceptions
      collectors:
        - type: className
          value: ^Company\\Exception

    - name: Company
      collectors:
        - type: bool
          must:
             - type: className
                value: Company\\.*
          must_not:
             - type: className
                value: Company\\Exception\\.*
  ruleset:
    Company_Base_Exceptions:
      - Base_Php_Exceptions

    Company:
      - Company_Base_Exceptions

The issue I am running into, is that something in the Company layer is using the Company_Php_Exceptions layer, which itself relies on the Base_Php_Exceptions, a violation is raised because it checks the whole dependency graph and not just the first layer.
I've tried using the InheritanceLevel collector, but to no avail.
Now I'm starting to wonder if this is possible at all using Deptrac, or if I have to rely on other tooling.
Any thoughts, or guidance is much appreciated!

No need to look for another tool. It is possible. Give me a few hours to finish at work and I will post a solution.

Can you provide me with the text of the Violation as shown in the console? It would help me craft the code for you. Or the repo in question, if it is public?

The violation I'm getting is this one
  Violation   Company\Infrastructure\SomeClass must not depend on RuntimeException (Base_Php_Exceptions)
              Company\Infrastructure\SomeClass::12 ->
              Company\Exception\SystemException::13 ->
              RuntimeException::24
              src/Exception/CompanyRuntimeException.php:24

Unfortunately I cannot share the repository, but if this isn't enough information, I can recreate the issue in a separate repo

The short answer is - you can use a custom Analyser Event subscriber to skip those violations. Look at the docs: https://github.com/qossmic/deptrac/blob/main/docs/extending_deptrac.md
Deptrac even already does this internally, if you look here: https://github.com/qossmic/deptrac/blob/main/internal/deptrac/IgnoreDependenciesOnContract.php and how it is registered here:https://github.com/qossmic/deptrac/blob/a446b66833b6025ce943cff08a8c79c918e399ed/deptrac.config.php#L18 or here: https://github.com/qossmic/deptrac/blob/a446b66833b6025ce943cff08a8c79c918e399ed/deptrac.yaml#L2
You just need to match the violation in question and stop propagation right there.

Small reproducing repo would certainly be helpful.

Here is a small example project with this structure reproducing the problem.
https://github.com/pckuijper/custom-exceptions
I'm trying to figure out how to set up what you describe in the short answer, but unfortunately to no avail just yet.

No promises, but give me 24 hours.

Provided in https://github.com/pckuijper/custom-exceptions/pull/1

Thanks for helping with implementing this! It was a great help.
One final issue I'm running into is quite likely related to the version of deptrac we're using (0.24) due to php + symfony constraints.
It looks like the code should still work in the same way and I'm trying to tag my event listener by using
services:
  - class: Company\Tests\Framework\StaticAnalysis\Deptrac\AllowCustomExceptionsRule
    tags:
      - { name: kernel.event_listener, event: Qossmic\Deptrac\Contract\Analyser\ProcessEvent }

Copied over from https://github.com/qossmic/deptrac/blob/0.24.0/deptrac.yaml
But while it doesn't seem to get executed.
Is there any known issues with that version and extending deptrac with custom services?

For event listener it might not get executed if there was a listener with higher priority that was executed before yours with the default priority that has called stopPropagation on the event. That how Symfony works AFAIK. I think some of the Violation creating events do that.

Hi, so I looked into it, and yes, DependencyType has been introduced in deptrac v1.0.0 (see https://github.com/qossmic/deptrac/commit/dc928f7a2fe2bdb8a7dea723babb1cb4db5b252e) and is not available in v0.2.4. SO that is certainly a problem. I have updated what I could in your repo in a PR: https://github.com/pckuijper/custom-exceptions/pull/2

I have found a "workaround", it's probably not as optimal as it could be but it's good enough for us for now!
I'll leave the repository up in case this is of any help to anybody in the future. The main branch has a working setup with the current latest version of deptrac ^1.0 and there is a branch for version 0.24
Repo: https://github.com/pckuijper/custom-exceptions (example with version ^1)
https://github.com/pckuijper/custom-exceptions/tree/fix-for-version-0.24


[BUG] Worker nodes reconcile when a control plane node is deleted

Rancher Server Setup

Rancher version: 2.6-head commit id: 4206f57
Installation option (Docker install/Helm Chart): Helm

If Helm Chart, Kubernetes Cluster and version (RKE1, RKE2, k3s, EKS, etc): EKS v1.23.7-eks-4721010

Information about the Cluster

Kubernetes version: v1.24.4+k3s1 Downstream - AWS Node Driver cluster - 1 etcd, 2 cp, 3 worker nodes

User Information

What is the role of the user logged in? (Admin/Cluster Owner/Cluster Member/Project Owner/Project Member/Custom): admin

Describe the bug
[BUG] Worker nodes reconcile when a control plane node is deleted
To Reproduce

Deploy a k3s Node Driver cluster
Edit cluster, change the Upgrade Strategy to - Worker Concurrency: 2 and enable Drain Nodes for both Worker and control plane nodes
Save changes made
Cluster goes into full reconcile, an issue logged for it is here
Wait for the cluster to come up Active.
Delete a control plane node --> Select the ellipsis, clicking on Delete
Node is deleted, and another node is provisioned
But the worker nodes also reconcile

Result

Rancher prov logs

[INFO ] draining bootstrap node(s) sowmya-k3s-iam-pool1-595d8854cf-2gq4n: draining node
--
7:09:06 pm | [INFO ] configuring bootstrap node(s) sowmya-k3s-iam-pool1-595d8854cf-2gq4n: waiting for plan to be applied
7:09:28 pm | [INFO ] uncordoning bootstrap node(s) sowmya-k3s-iam-pool1-595d8854cf-2gq4n: error undraining machine sowmya-k3s-iam-pool1-595d8854cf-2gq4n: an error on the server ("unable to create impersonator account: error getting service account token: error getting secret: apiserver not ready") has prevented the request from succeeding (get nodes sowmya-k3s-iam-pool1-d43a7eea-28n59)
7:09:34 pm | [INFO ] configuring control plane node(s) sowmya-k3s-iam-pool2-6b68c77d5c-99m2s,sowmya-k3s-iam-pool2-6b68c77d5c-fhgcl
7:09:40 pm | [INFO ] draining control plane node(s) sowmya-k3s-iam-pool2-6b68c77d5c-99m2s: draining node
7:09:44 pm | [INFO ] configuring control plane node(s) sowmya-k3s-iam-pool2-6b68c77d5c-99m2s,sowmya-k3s-iam-pool2-6b68c77d5c-fhgcl
7:09:54 pm | [INFO ] uncordoning control plane node(s) sowmya-k3s-iam-pool2-6b68c77d5c-99m2s: waiting for uncordon to finish
7:09:56 pm | [INFO ] draining control plane node(s) sowmya-k3s-iam-pool2-6b68c77d5c-fhgcl: draining node
7:10:10 pm | [INFO ] configuring control plane node(s) sowmya-k3s-iam-pool2-6b68c77d5c-99m2s,sowmya-k3s-iam-pool2-6b68c77d5c-fhgcl
7:10:22 pm | [INFO ] uncordoning control plane node(s) sowmya-k3s-iam-pool2-6b68c77d5c-fhgcl: waiting for uncordon to finish
7:10:24 pm | [INFO ] sowmya-k3s-iam-pool2-6b68c77d5c-99m2s,sowmya-k3s-iam-pool2-6b68c77d5c-fhgcl
7:10:26 pm | [INFO ] draining worker node(s) sowmya-k3s-iam-pool3-c6fd664d7-9lz47,sowmya-k3s-iam-pool3-c6fd664d7-blkr4
7:10:28 pm | [INFO ] draining worker node(s) sowmya-k3s-iam-pool3-c6fd664d7-blkr4: draining node
7:10:34 pm | [INFO ] uncordoning worker node(s) sowmya-k3s-iam-pool3-c6fd664d7-9lz47: Node condition Ready is False., waiting for uncordon to finish
7:10:36 pm | [INFO ] draining worker node(s) sowmya-k3s-iam-pool3-c6fd664d7-blkr4,sowmya-k3s-iam-pool3-c6fd664d7-lchhr
7:11:06 pm | [INFO ] draining worker node(s) sowmya-k3s-iam-pool3-c6fd664d7-blkr4: draining node
7:11:10 pm | [INFO ] configuring worker node(s) sowmya-k3s-iam-pool3-c6fd664d7-blkr4: drain completed
7:11:12 pm | [INFO ] configuring worker node(s) sowmya-k3s-iam-pool3-c6fd664d7-blkr4,sowmya-k3s-iam-pool3-c6fd664d7-lchhr
7:11:14 pm | [INFO ] configuring worker node(s) sowmya-k3s-iam-pool3-c6fd664d7-blkr4: waiting for plan to be applied
7:11:16 pm | [INFO ] configuring worker node(s) sowmya-k3s-iam-pool3-c6fd664d7-blkr4: Node condition Ready is False., waiting for plan to be applied
7:11:18 pm | [INFO ] non-ready worker machine(s) sowmya-k3s-iam-pool3-c6fd664d7-blkr4: Node condition Ready is False.

Expected Result

Worker nodes should not reconcile/upgrade/drain
In RKE2 cluster, this doesn't happen. Worker nodes do not reconcile/drain/upgrade

SURE-6248
SURE-5826 (now closed)

/backport v2.6.9

Assigning to @jakefhyde to investigate and possibly proof-of-concept implement it.

@Sahota1225 is there a jira for this issue?

@snasovich You say to look for server arg.
What if it is a LB IP, would you see the same ?

@zaggash , I believe it will still be the actual IP of the primary control plane node as it's assigned by Rancher provisioning code.

@zaggash Loadbalancers are not currently implemented as part of our v2 provisioning framework. However, loadbalancers would be the correct way to solve this issue.

I was going over when certain issues were added to release notes. This issue doesn't appear to be included in v2.6 notes yet, but is listed in v2.7

Consider using the kube-vip component DaemonSet to deploy on the CP/Master node.
Rancher provides a VIP when deploying Downstream RKE2,
so that this VIP can also be provided to the ACE endpoint without the need for an external load balancer.


Add export capabilities for translations catalogs.

Hello,
I'm actually using Poedit to work on WordPress translations and it works really well 👍.
I'm also working on others kind of projects which are not using GetText translation files. Is there a way to export catalogs in another format than Po/Mo ?
For example working with Twig is great but exporting to Symfony translation catalog format can be a great addition. Same for Laravel framework which use PHP and JSON catalogs.
Don't know if it's on the roadmap but I'll be happy to know if I'm not the only one to request that feature 😄.

Why exporting instead of working with those non-PO formats, i.e. editing them directly?
In other words, is there a need to have the same translation strings in PO and in another format at the same time?

E.g. extraction-related perhaps?

It'll be awesome to be able to work with that non-PO file formats ! I don't think there is a need to have the same two strings in two files at the same time.
Maybe the only use case I can think is to export translations to PO for translators then integrate the changes in the application and it's not the job of Poedit for me to handle that...

It'll be awesome to be able to work with that non-PO file formats !

Well, that one is definitely on the roadmap (and for sooner rather than later, i.e. tentatively 2.1).

I'll be glad to help if possible 😄.

+1 for export to json

@filtration Please don't pollute issues with +1 comments that don't add to the discussion. That's what you have the reactions button for.

I'm trying to find a solution for editing i18next json files. Did the idea of editing files other than PO ever eventuate?

(Progress note: current versions of Poedit can edit XLIFF and JSON files.)


Keyboard Interrupt

Hey websocketd!!
I've been developing an app using your tech, and it's been going well, until I hit this speed bump.  I'm having connection issues, across different browsers.  I develop on a linux platform, and have a front end call out from Salesforce that connects to our backend using websockets, and the reconnectingWebSocket js.
About a month ago, while working on front end, the back end started failing.  This started happening after a month of solid connections, without any problems, or any changes whatsover to the backend websocket app.  I haven't been able to get a solid error message, because what is thrown is constantly changing.  There is one constant though, every time there is a keyboardInterrupt.
I'm developing in python, and use a bunch of packages.( requests, pika, logging, threading, urllib, select, Queue)   I tried rebuilding and slowly started adding back packages to see if I could narrow it down to a cross package error.  No Luck.  When rebuilding everything worked, I thought I fixed a bug, then a couple hours later the error returned.
I think something is happening to STDIN/OUT to throw the keyboardInterrupt, or maybe my WebSockets packets are being denied somewhere, causing the connection to fail.
Are you aware of any python packages that might be causing this problem?
Is it possible this is WebSocket Compatibility across browsers?
Do you have any tips to help me debug this?
I'm a noob when it comes to asking questions, so if you need anything from me let me know.  Your help is greatly appreciated.
Thanks for your time.
Heres the server im on:
Linux inboundwebsocket 3.13.0-62-generic #102-Ubuntu SMP Tue Aug 11 14:29:36 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
Im running python2.7 with websocketd 0.2.11 (go1.4.2 linux-amd64) --

Hey, wanted to update you on my progress with this.
The problem was with Firebug extension with Firefox.  While viewing firebug inspector, my websocket breaks everytime.  Disabling it puts me back into development... YAY.
Have no clue what in firebug is doing it, and haven't attempted to find out, but Ill reach out to them after this.


Raise deprecation warning on adapter instead of Base64StringIO.

Currently, I'm getting the warning even when I set up file_name as a proc:
mount_base64_uploader :avatar, UserAvatarUploader, file_name: ->(_) { SecureRandom.hex(8) }

In order to fix the warning at the moment, you have to do this:
mount_base64_uploader :avatar, UserAvatarUploader, file_name: ->(_) { -> { SecureRandom.hex(8) } }

I don't think the latter is intended to do

Previous version was only testing deprecation on the Base64 class which is not part of the public API that's why this case wasn't reported by the test suite.

All rubocop warnings have been fixed.

Included in 2.5.3

Thanks!

@frodsan Thank you!


Fix train fail in tox

The train command fails with tox but not with pytest which is weird. The error is
def test_train_command():
        with tempfile.TemporaryDirectory() as tmp_dir:
            data_path = os.path.join(tmp_dir, "data.jsonl")
            model_path = os.path.join(tmp_dir, "model.pkl")
            label_binarizer_path = os.path.join(tmp_dir, "label_binarizer.pkl")
            train_info_path = os.path.join(tmp_dir, "train_info.json")

            write_jsonl(data_path, DATA)

            result = runner.invoke(app, [
                "train",
                data_path,
                model_path,
                label_binarizer_path,
                "--approach",
                "tfidf-svm",
                "--parameters",
                "{'tfidf__min_df': 1, 'tfidf__stop_words': None}",
            #    "--train-info",
            #    train_info_path
            ])
>           assert result.exit_code == 0
E           AssertionError: assert 1 == 0
E            +  where 1 = <Result FileNotFoundError(2, 'No such file or directory')>.exit_code

For now we mark the test and skip for tox but it would be nice to resolve this

This is fixed as part of #175


python311Packages.google-cloud-dataproc: 5.5.1 -> 5.6.0

Changelog: https://github.com/googleapis/python-dataproc/blob/v5.6.0/CHANGELOG.md
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

It seems that the url will be changed in the next release.

URL updated.


Haxe 4.0 support

In Haxe 4.0+ , columns are 1 based instead of 0 based (before the first char of a line was 0, now it's 1), and positions in the file are utf8-based instead of binary-based (before "é" was counting as 2 chars, it's one now).
This lead to problems with error reporting not being correctly done in HaxeDevelop, including completion if there are utf8 chars in the file. Could you detect if Haxe is 4.0+ and change the behavior accordingly ?

@ncannasse hi, how can i test it?

Any Haxe error using 4.0-RC3 will display wrongly only HaxeDevelop.
And if you have a string containing "é" before that it will produce another issue.
Disable your A/V to install Haxe, it's safe.

Disable your A/V to install Haxe, it's safe.
ok :)


Recieving the following error when running autobloody

[] Connection to Neo4j
[] No proxy detected
Traceback (most recent call last):
File "/usr/local/bin/autobloody", line 8, in
sys.exit(main())
^^^^^^
File "/usr/local/lib/python3.11/dist-packages/autobloody/main.py", line 82, in main
path_dict = pathgen(args)
^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/autobloody/main.py", line 101, in pathgen
db = database.Database(args.dburi, args.dbuser, args.dbpassword)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/autobloody/database.py", line 9, in init
self._prepareDb()
File "/usr/local/lib/python3.11/dist-packages/autobloody/database.py", line 24, in _prepareDb
session.write_transaction(self._createGraph)
File "/usr/lib/python3/dist-packages/neo4j/_meta.py", line 81, in inner
return f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/neo4j/_sync/work/session.py", line 763, in write_transaction
return self._run_transaction(
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/neo4j/_sync/work/session.py", line 491, in _run_transaction
result = transaction_function(tx, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/dist-packages/autobloody/database.py", line 75, in _createGraph
graph_exists = tx.run("RETURN gds.graph.exists('autobloody')").single()[0]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/neo4j/_sync/work/transaction.py", line 152, in run
result._tx_ready_run(query, parameters)
File "/usr/lib/python3/dist-packages/neo4j/_sync/work/result.py", line 115, in _tx_ready_run
self._run(query, parameters, None, None, None, None)
File "/usr/lib/python3/dist-packages/neo4j/_sync/work/result.py", line 161, in _run
self._attach()
File "/usr/lib/python3/dist-packages/neo4j/_sync/work/result.py", line 269, in _attach
self._connection.fetch_message()
File "/usr/lib/python3/dist-packages/neo4j/_sync/io/_common.py", line 180, in inner
func(*args, **kwargs)
File "/usr/lib/python3/dist-packages/neo4j/_sync/io/_bolt.py", line 659, in fetch_message
res = self._process_message(tag, fields)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/neo4j/_sync/io/_bolt4.py", line 326, in _process_message
response.on_failure(summary_metadata or {})
File "/usr/lib/python3/dist-packages/neo4j/_sync/io/_common.py", line 247, in on_failure
raise Neo4jError.hydrate(**metadata)
neo4j.exceptions.CypherSyntaxError: {code: Neo.ClientError.Statement.SyntaxError} {message: Unknown function 'gds.graph.exists' (line 1, column 8 (offset: 7))
"RETURN gds.graph.exists(("autobloody")) AS gds.graph.exists('autobloody')"
^}

same for me.

Hello DrorDvash, can you type this in your neo4j browser RETURN gds.version(); and give me the output?

@DrorDvash I am also having the same error as mentioned above. The output of the command you have asked for is here

Hello DrorDvash, can you type this in your neo4j browser RETURN gds.version(); and give me the output?
Yep, the same for me.

Unknown function 'gds.version' (line 1, column 8 (offset: 7))
"RETURN `gds`.`version`() AS `gds.version()`"
        ^

It's because the GDS library is not installed in your neo4j setup. Here is the guide to install it, it is also mentioned in the dependencies of the README.

@CravateRouge I have already went through that thread and have done it. Please check error correctly. It saying already exists and not that it does not exists.
The current error is:

The error which you are referring to is for it does not exists which is not our case.

As

@CravateRouge I am also having the same error as mentioned above. The output of the command you have asked for is here

As gds.version() does return an error it seems gds is not correctly installed. See https://neo4j.com/docs/graph-data-science/current/installation/#_verifying_the_installation and verify you have correctly made every steps.

so GDS is installed, now i get a different error

≫ python3 autobloody.py -u adfs_svc -p 'XXXX' --host <IP_ADDRESS> -du 'neo4j' -dp 'bloodhound' -ds<EMAIL_ADDRESS>-dt '0X0SECURITY.LOCAL' -d 0x0security.local
[*] Connection to Neo4j
[*] No proxy detected
Traceback (most recent call last):
  File "/root/Tools/autobloody/autobloody.py", line 6, in <module>
    main.main()
  File "/root/Tools/autobloody/autobloody/main.py", line 82, in main
    path_dict = pathgen(args)
                ^^^^^^^^^^^^^
  File "/root/Tools/autobloody/autobloody/main.py", line 103, in pathgen
    path = db.getPrivescPath(args.dbsource, args.dbtarget)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/Tools/autobloody/autobloody/database.py", line 13, in getPrivescPath
    relationships = session.read_transaction(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/neo4j/_meta.py", line 81, in inner
    return f(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/neo4j/_sync/work/session.py", line 664, in read_transaction
    return self._run_transaction(
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/neo4j/_sync/work/session.py", line 491, in _run_transaction
    result = transaction_function(tx, *args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/Tools/autobloody/autobloody/database.py", line 95, in _findShortestPath
    return result.single()[0].relationships
           ~~~~~~~~~~~~~~~^^^
TypeError: 'NoneType' object is not subscriptable

@DrorDvash it seems you have not path exploitable from<EMAIL_ADDRESS>to 0X0SECURITY.LOCAL. Can you show me a screenshot of the path between those two edges if you think it's a path exploitable by autobloody?

so you're saying there is no path exploitable between them -> maybe print a message about it.

So you're saying there is room for improvement -> maybe make a contribution if you want to have a better tool.

I'll check it once again i have more data in my BH.

autobloody is based on data ingested in BH. If you don't have the data in BH, autbloody will have no data to work on.

how the tool auto exploit a path? let's say i have writeDACL or something like that. what is going to happen?

autobloody will perform exploitations described in the "Abuse Info" of each edge documented in the bloodhound documentation. e.g. WriteDACL abuse info


cmd-q warning

I was installing 10.5 leopard, when I was closing apps and accidentally closed utm, this app really needs a cmd-q warning.

Duplicate of #2321


GDScript: Ensure super() is called in constructors

This is needed so the super class is properly constructed, especially when it requires arguments.
Fix #40861

The thing is the constructor has to be called always. The implicit constructor is, and the user-created constructor should be as well, since it is what creates the object.

If for some reason overriding a constructor can lead to the implicit constructor not being called, then this is of course a bug. But I think that it can be fixed in some other way, without resorting to being forced to always call the parent constructor.
In my opinion, this is a valid feature: the user can override the constructor as they see fit, it is the user's responsibility not to violate the expectations of the base class, just like with other methods. If we just want to reduce the chance that the user will accidentally forget to call super(), then there are softer ways (like warnings) and this should be done for all methods, not just the constructor.

just like with other methods

Discussed during the GDScript meeting. We think that the constructor is an entire different beast considering OOP principles. We should enforce the call of the constructor, any other way seems wrong.
See Typescript for example:

One difference from the prior example is that each derived class that contains a constructor function must call super() which will execute the constructor of the base class.  This is an important rule that TypeScript will enforce.
https://www.typescriptlang.org/docs/handbook/classes.html


Only show prescriptions on phamacy for independent pharmacy roles.

 Share POC Roles and Privileges using metadata deploy

Coverage increased (+0.02%) to 63.212% when pulling 7cec2a582e9dcc0df8f85731a9a732e2a60a786a on ynurmahomed:roles into 3d42215fe4557e2b5fa09727b11dd6b07f7d824f on esaude:master.


Implement mechanism to trigger rollouts on Secret/ConfigMap changes

Implement a mechanism, usable by any controller, that causes Deployment rollouts on Secret or ConfigMap changes.
The implementation will be:

Add a label to the Deployment for each Secret/ConfigMap used which has the hash of the Secret/ConfigMap contents.
The label will change when the data contined in the Secret/ConfigMap changes, triggering a rollout.

It will be first implemented/tested for MappingService controller, which uses a Secret as a source for an environment variable.

I think an annotation would be more suited than a label for this kind of implementation, as it's controller-managed automation/information.

I think an annotation would be more suited than a label for this kind of implementation, as it's controller-managed automation/information.

Hey @miguelsorianod do Pod annotations also trigger rollouts on Deployments? We had this doubt the other day when discussing this.

Hey @miguelsorianod do Pod annotations also trigger rollouts on Deployments? We had this doubt the other day when discussing this.

As far as I know anything that changes the PodTemplateSpec triggers a rollout

As far as I know anything that changes the PodTemplateSpec triggers a rollout


[HUDI-6617] Fix HoodieRecordDelegate NotSerializableException

Change Logs
make HoodieRecordDelegate implements Serializable
Impact
NONE
Risk level (write none, low medium or high below)
NONE
Documentation Update
NONE
Contributor's checklist

[ ] Read through contributor's guide
[ ] Change Logs and Impact were stated clearly
[ ] Adequate tests were added if applicable
[ ] CI passed

@danny0405 thank you for your review.
Have a question to discuss with you. I'm not sure.
Is it necessary to implement KryoSerializable for HoodieRecordDelegate?

Does Spark uses Kryo serialization for this objects, it it is, that makes sense to me.


If a VM resource is requested to transition in a state where it's already in, it returns success, instead it should return 'the resource is in the same state'

ISSUE TYPE

Bug Report

COMPONENT NAME

VM handler

CLOUDSTACK VERSION

4.11

SUMMARY

If a VM is already running on HostA and the Admin requests a Start on HostB, it will return a success with the VM being started on HostA. Instead it should return an error to the administrator that the resource is already in running state.
STEPS TO REPRODUCE

Start a VM on HostA
With cloudmonkey, start the same VM on HostB
Observe the result

EXPECTED RESULTS

Response should fail the job and say VM is already in the same state.

ACTUAL RESULTS

Job succeeds with VM not being the requested host

I would argue that the expected behaviour is a migration to host B but agree that the endstate does not qualify as a success. Do you request that this is implemented as failure result, @borisstoyanov ?

Yes @DaanHoogland, I do believe that it should return a failure if cloudstack is not able to execute the request with all required parameters.


read_parquet(..., filesystem="pyarrow") fails

import dask.dataframe as dd
df = dd.read_parquet("s3://coiled-data/uber/") # works
df = dd.read_parquet("s3://coiled-data/uber/", filesystem="pyarrow")

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
File ~/workspace/dask/dask/backends.py:136, in CreationDispatch.register_inplace.<locals>.decorator.<locals>.wrapper(*args, **kwargs)
    135 try:
--> 136     return func(*args, **kwargs)
    137 except Exception as e:

File ~/workspace/dask/dask/dataframe/io/parquet/core.py:538, in read_parquet(path, columns, filters, categories, index, storage_options, engine, use_nullable_dtypes, dtype_backend, calculate_divisions, ignore_metadata_file, metadata_task_size, split_row_groups, blocksize, aggregate_files, parquet_file_extension, filesystem, **kwargs)
    536     blocksize = None
--> 538 read_metadata_result = engine.read_metadata(
    539     fs,
    540     paths,
    541     categories=categories,
    542     index=index,
    543     use_nullable_dtypes=use_nullable_dtypes,
    544     dtype_backend=dtype_backend,
    545     gather_statistics=calculate_divisions,
    546     filters=filters,
    547     split_row_groups=split_row_groups,
    548     blocksize=blocksize,
    549     aggregate_files=aggregate_files,
    550     ignore_metadata_file=ignore_metadata_file,
    551     metadata_task_size=metadata_task_size,
    552     parquet_file_extension=parquet_file_extension,
    553     dataset=dataset_options,
    554     read=read_options,
    555     **other_options,
    556 )
    558 # In the future, we may want to give the engine the
    559 # option to return a dedicated element for `common_kwargs`.
    560 # However, to avoid breaking the API, we just embed this
    561 # data in the first element of `parts` for now.
    562 # The logic below is inteded to handle backward and forward
    563 # compatibility with a user-defined engine.

File ~/workspace/dask/dask/dataframe/io/parquet/arrow.py:532, in ArrowDatasetEngine.read_metadata(cls, fs, paths, categories, index, use_nullable_dtypes, dtype_backend, gather_statistics, filters, split_row_groups, blocksize, aggregate_files, ignore_metadata_file, metadata_task_size, parquet_file_extension, **kwargs)
    531 # Stage 1: Collect general dataset information
--> 532 dataset_info = cls._collect_dataset_info(
    533     paths,
    534     fs,
    535     categories,
    536     index,
    537     gather_statistics,
    538     filters,
    539     split_row_groups,
    540     blocksize,
    541     aggregate_files,
    542     ignore_metadata_file,
    543     metadata_task_size,
    544     parquet_file_extension,
    545     kwargs,
    546 )
    548 # Stage 2: Generate output `meta`

File ~/workspace/dask/dask/dataframe/io/parquet/arrow.py:1047, in ArrowDatasetEngine._collect_dataset_info(cls, paths, fs, categories, index, gather_statistics, filters, split_row_groups, blocksize, aggregate_files, ignore_metadata_file, metadata_task_size, parquet_file_extension, kwargs)
   1046 if ds is None:
-> 1047     ds = pa_ds.dataset(
   1048         paths,
   1049         filesystem=_wrapped_fs(fs),
   1050         **_processed_dataset_kwargs,
   1051     )
   1053 # Get file_frag sample and extract physical_schema

File ~/mambaforge/envs/test-env/lib/python3.11/site-packages/pyarrow/dataset.py:776, in dataset(source, schema, format, filesystem, partitioning, partition_base_dir, exclude_invalid_files, ignore_prefixes)
    775 if all(_is_path_like(elem) for elem in source):
--> 776     return _filesystem_dataset(source, **kwargs)
    777 elif all(isinstance(elem, Dataset) for elem in source):

File ~/mambaforge/envs/test-env/lib/python3.11/site-packages/pyarrow/dataset.py:466, in _filesystem_dataset(source, schema, filesystem, partitioning, format, partition_base_dir, exclude_invalid_files, selector_ignore_prefixes)
    464 factory = FileSystemDatasetFactory(fs, paths_or_selector, format, options)
--> 466 return factory.finish(schema)

File ~/mambaforge/envs/test-env/lib/python3.11/site-packages/pyarrow/_dataset.pyx:2941, in pyarrow._dataset.DatasetFactory.finish()

File ~/mambaforge/envs/test-env/lib/python3.11/site-packages/pyarrow/error.pxi:144, in pyarrow.lib.pyarrow_internal_check_status()

File ~/mambaforge/envs/test-env/lib/python3.11/site-packages/pyarrow/error.pxi:115, in pyarrow.lib.check_status()

OSError: Error creating dataset. Could not read schema from 'coiled-data/uber/'. Is this a 'parquet' file?: Not a regular file: 'coiled-data/uber/'

The above exception was the direct cause of the following exception:

OSError                                   Traceback (most recent call last)
Cell In[3], line 1
----> 1 df = dd.read_parquet("s3://coiled-data/uber/", filesystem="pyarrow")

File ~/workspace/dask/dask/backends.py:138, in CreationDispatch.register_inplace.<locals>.decorator.<locals>.wrapper(*args, **kwargs)
    136     return func(*args, **kwargs)
    137 except Exception as e:
--> 138     raise type(e)(
    139         f"An error occurred while calling the {funcname(func)} "
    140         f"method registered to the {self.backend} backend.\n"
    141         f"Original Message: {e}"
    142     ) from e

OSError: An error occurred while calling the read_parquet method registered to the pandas backend.
Original Message: Error creating dataset. Could not read schema from 'coiled-data/uber/'. Is this a 'parquet' file?: Not a regular file: 'coiled-data/uber/'

cc @rjzamora

Thanks for raising @mrocklin! Can you share the version of pyarrow? I can look into this on Monday, but my current environment with pyarrow-12 seems to work:
In [1]: %time dd.read_parquet("s3://coiled-data/uber/", storage_options={"anonymous": True}, filesystem="arrow").head()
CPU times: user 731 ms, sys: 330 ms, total: 1.06 s
Wall time: 3.7 s
Out[1]:
  hvfhs_license_num dispatching_base_num originating_base_num  ... access_a_ride_flag wav_request_flag wav_match_flag
0            HV0003               B02867               B02867  ...                  N                N            NaN
1            HV0003               B02879               B02879  ...                                    N                N            NaN
3            HV0005               B02510                 <NA>  ...                  N                N            NaN
4            HV0005               B02510                 <NA>  ...                  N                N            NaN

[5 rows x 24 columns]

I'm on 13 normally, but I just re-ran with 12 and it continued to fail.  I just made a fresh environment as follows and it still failed:
 mamba create -n test-arrow pyarrow=12 dask ipython s3fs

@phofl did you have a workaround for this?

You can create your own filesystem directly:
import boto3
session = boto3.session.Session()
credentials = session.get_credentials()

from pyarrow.fs import S3FileSystem

fs = S3FileSystem(
    secret_key=credentials.secret_key,
    access_key=credentials.access_key,
    region='us-east-2',
    session_token=credentials.token)

But you'll have to use
https://github.com/phofl/dask-expr/tree/testing
if you want to use dask-expr

I just reproduced this on my Macbook Pro. It seems like you sometimes need to specify the region when using filesystem="arrow" for some reason...
dd.read_parquet("s3://coiled-data/uber/", storage_options={"anonymous": True, "region": "us-east-2"}, filesystem="arrow")

This reproduces without Dask:
import pyarrow.dataset as ds
from pyarrow.fs import S3FileSystem
fs = S3FileSystem(
    anonymous=True,
    region="us-east-1",  # Error is resolved by `region="us-east-2”`
)
ds.dataset(["coiled-data/uber/"], filesystem=fs, format="parquet")

Note that the error message is much more helpful if you pass in "coiled-data/uber/" instead of ["coiled-data/uber/"]:
OSError: When getting information for key 'uber' in bucket 'coiled-data': AWS Error UNKNOWN (HTTP status 301) during HeadObject operation: No response body. Looks like the configured region is 'us-east-1' while the bucket is located in 'us-east-2'.


[cloud synch] Deleted projects not synching

Repro steps:

project created in Browser 1
project deleted in Browser 2
project edited in Browser 1
both pages reloaded
project shows in Browser 1 but not Browser 2

Still repros, but repro'ing a different symptom:
after deleting the project in browser 2, making an edit in browser 1 now looks like this:

@eanders-MS does this still repro?


Prevent URLs which can only fail from making calls to the service

For: https://github.com/hypothesis/checkmate/issues/27
This introduces a small delay, but will be much faster than making a call which will fail.

There's some oddity in GHA with the coverage report. It looks to be getting archived ok, but it can't pull it back down or something? I just ran it again to see if it'll help.

It looks like we're only getting coverage for the Python 3 run, so the Python 2 branch is blowing up. Annoyingly you get both locally.

This is probably rubbish now. We have much more complicated normalisation than we can replicate in the client without moving all the normalisation code here out of checkmate. This might be a good idea, but a bit of a pain while we are Python 2 compliant