qv2ray: 2.6.3 -> 2.7.0

Motivation for this change
Qv2ray repo was archived, this is the final version.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 136426 run on x86_64-linux 1

1 package built:

qv2ray

https://github.com/NixOS/nixpkgs/pull/136426/checks?check_run_id=3490521941 any ideas with the failure?

@bobby285271 That is a reproducible clang_7 ICE error, clang_8 works fine.
Now I'm working on the last LINK phase issue.

@bobby285271 The build issues fixed.


Use translation memory to localize Gnome 41

Ахҳәаа
Gnome 41 is in the Freeze state right now.
Ауадаҩрақәа
I have 3 days until Saturday 28/08/21 to upload localized po files and I should announce it in the i18n-gnome mailing list.
Аӡбара

Use translation memory from the locale(Firefox, Gnome, LibreOffice, OpenOffice, VK) parallel corpus.
upload those po files and anounce the change in the i18n-gnome mailing list.

Азхьарԥшқәа:

https://wiki.gnome.org/FortyOne
https://docs.translatehouse.org/projects/translate-toolkit/en/1.10.0/commands
https://l10n.gnome.org/teams/ab/

I didn't make it.


Update types for deliverable document upload API and update the service call

projectId is back in the payload, removed from path

The BE changes have already gone in. FE will fail with uploads until this change goes in.


build: latest a6

v11.2.4 has been released and contains a fix for the issue. Feel free to reopen the issue or to submit a new one if you meet any problems.


feat(ingest) : kafka connect source improvements

Resolve source dataset in presence of Regexrouter for JDBC Connector
Report warnings when source dataset can not be determined for JDBC Connector
Generate MCP workunits rather than MCE

Checklist

[x] The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
[ ] Links to related issues (if applicable)
[x] Tests for the changes have been added/updated (if applicable)
[ ] Docs related to the changes have been added/updated (if applicable)

@mayurinehate @shirshanka what do we need to start jvm for? i do not see any usages

@mayurinehate @shirshanka what do we need to start jvm for? i do not see any usages

@Tonkonozhenko We need to start JVM , because we use java specific libraries in the code.
See https://github.com/linkedin/datahub/blob/v0.8.22/metadata-ingestion/src/datahub/ingestion/source/kafka_connect.py#L362-L372

@mayurinehate thanks. we run datahub in airflow and it looks like a huge overhead to bring java into it.
I'll check how we can replace it with python and try to create a PR.


Support React elements as content for tooltip

Right now it's my understanding that you can supply text or html to the tooltip as content. It would be nice to be able to pass an arbitrary React element if the content needs to be more complex.
// maybe there would be a setting on ReactTooltip to activate this, like there is for html
<ReactTooltip element={true}/>

<div data-tip={<MyComponent/>}/>

Ah this is supported by giving children to <ReactTooltip/>. May also be supported by data-tip. I had html={true} set, which made it not work.


monitoring and abservability

discuss monitoring and observability of the components

provide a sample observability solution (prometheus), OTEL: performance and tracing


Errors while loading a rst file

Hi,
I am getting an error while loading the RST file.
I load the file with the code:
from ansys.dpf import post r = post.load_solution("beam.rst")
and I get the following error messages:
PS C:\Users\Konrad\python_basics\pyapdl> python .\pyansys_post.py
Traceback (most recent call last):
File "C:\Users\Konrad\python_basics\pyapdl\pyansys_post.py", line 3, in
r = post.load_solution("beam.rst")
File "C:\Users\Konrad\AppData\Local\Programs\Python\Python39\lib\site-packages\ansys\dpf\post\post_utility.py", line 33, in load_solution
_model = Model(data_sources)
File "C:\Users\Konrad\AppData\Local\Programs\Python\Python39\lib\site-packages\ansys\dpf\core\model.py", line 48, in init
self._metadata = Metadata(data_sources, self._server)
File "C:\Users\Konrad\AppData\Local\Programs\Python\Python39\lib\site-packages\ansys\dpf\core\model.py", line 228, in init
self._set_data_sources(data_sources)
File "C:\Users\Konrad\AppData\Local\Programs\Python\Python39\lib\site-packages\ansys\dpf\core\model.py", line 355, in _set_data_sources
self._cache_streams_provider()
File "C:\Users\Konrad\AppData\Local\Programs\Python\Python39\lib\site-packages\ansys\dpf\core\model.py", line 252, in _cache_streams_provider
self._stream_provider.inputs.connect(self._data_sources)
AttributeError: 'NoneType' object has no attribute 'connect'
The beam.rst file is in the same dir as the python script.
I autumn 2021 I used this code successfully with ANSYS 2021R2.
Like a month ago I updated ANSYS to 2022R1. However, I suppose that this is rather not a reason of the problem.
After I got the error I reinstalled the library with the code pip install ansys-dpf-post but this did not help.
Could you advice what may be wrong?
Best,
Konrad

Hi @konradkam85 ,
it looks like a mismatch in the versions of the packages. please could you : pip uninstall ansys.grpc.dpf and pip uninstall ansys.dpf.core, then pip install ansys.dpf.core

Thank you @rlagha !
Now it works :)


mail issue

There are three questions：

The parameter mail.smtp.ssl.trust is not supported
The mailSender in PasswordAuthentication() and setFrom() may not be the same in actual use, and it's best to set it up separately
Cannot have "-" in email account name

very good! thanks!
and are you interested in submitting a pull request to resolve this issue?

Ok, I'll research the source code and try to improve it

good！
please refer to How to contribute
Look forward to your good news！

Ok，thanks!

Issue repair results：

The parameter mail.smtp.ssl.trust is not supported --Fixed and tested
The mailSender in PasswordAuthentication() and setFrom() may not be the same in actual use, and it's best to set it up separately  --Fixed and tested
Cannot have "-" in email account name  --It has been fixed in 1.1.0

Help grant permissions for push request @dailidong

An improvement to this issue has been submitted to pull request #746


Ability to choose what file format we can add in the file input component

The Ability to choose what file format we can add in the file input component can be more secure to add files to a server

use the standard attribute accept:
Acceptable file types can be specified with the accept attribute, which takes a comma-separated list of allowed file extensions or MIME types. Some examples:

accept="image/png" or accept=".png" — Accepts PNG files.
accept="image/png, image/jpeg" or accept=".png, .jpg, .jpeg" — Accept PNG or JPEG files.
accept="image/" — Accept any file with an image/ MIME type. (Many mobile devices also let the user take a picture with the camera when this is used.)
accept=".doc,.docx,.xml,application/msword,application/vnd.openxmlformats-officedocument.wordprocessingml.document" — accept anything that smells like an MS Word document.

<form method="post" enctype="multipart/form-data">

    Choose file to upload
    <input data-role='file' type="file" id="profile_pic" name="profile_pic"
          accept=".jpg, .jpeg, .png">

    <button>Submit</button>

</form>

It worked thanks


Added Sender Limitation Note

{% info %} You may create up to 100 unique senders.{% endinfo %}

Description of the change:
Reason for the change:
Link to original source:
@ksigler7

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

thanks Ashley!

Hello @ashleyrose1414,Thanks again for the PR! We want to show our appreciation by sending                                              you some swag. Could you please fill out                                              this form so we can send it to you? Thanks!                                             Team SendGrid DX


isobmff lib make file fix added to CMakeLists.txt, updated Readme with cmake build instructions

White spaces are added by VSCode


UploadAPI gov template

Required items, please complete
Change(s):

Updated the public facing endpoint in PROD template_ThreatIntelligenceUploadIndicators.json
Renamed the filename template_ThreatIntelligenceUploadIndicators_ForFairfax.json to template_ThreatIntelligenceUploadIndicators_ForGov.json, and updated the template to include both Fairfax and Mooncake specific details

Reason for Change(s):

Having the Threat Intelligence Upload Indicators connector available in Fairfax and Mooncake
Having the right public facing endpoint in Prod template

Version Updated:

No

Testing Completed:

Need help

Checked that the validations are passing and have addressed any issues that are present:

Need help

Hello @nbatyrbb, Can you please edit and fill above PR template

Hello @nbatyrbb, Can you please edit and fill above PR template


test: Improve upgrade/downgrade test

Warn if deletion of Cilium or CoreDNS fails to assist in debugging

Delete all etcd-operator resources instead of just deleting pods. Deleting
pods is insufficient as they will get re-created. It also leaves behind
resources which may not be used in older versions of Cilium and then cause
issues.

Clean the state using a minimal DaemonSet and then re-deploy Cilium to avoid
cleaning state again if Cilium restarts for some reason.

This change is

test-me-please

test-me-please

Hit known #8775


Make navbar mobile responsive

Have it switch to vertical flex box when on mobile screen?
This is not good:

Use this as guide http://www.w3schools.com/howto/howto_js_topnav.asp


Elevated priviledges leaving Raspiblitz menu.

When exiting to terminal from Raspiblitz menu, the admin user retains elevated sudo priviledges.
Is this intentional design?

The RaspiBlitz is not a consumer node ... its intented that the user is the Admin with sudo rights when SSH in.
For the additional apps we try to run them as seperated users if possible.


amend: Rollback amends that require --merge

Followup to #170.
If an amend requires a --merge (e.g. the post-amend restack requires an merge), you can end up in an un-restacked state. The amend succeeds, and HEAD gets moved to the new commit, but the restack fails, so you end up in a halfway state.
For example:
$ git amend
...
This operation would cause a merge conflict:
• (2 conflicting files) 4f2b8f07 Something with a conflict
To resolve merge conflicts, retry this operation with the --merge option.

$ git sl
◯ 4c8285ee 12h feat(amend): Initial amend command
┣━┓
┃ ✕ 632a87e5 35m (rewritten as 03c75042) In-progress refactors
┃ ┃
┃ ◯ 4f2b8f07 22m Something with a conflict
┃
● 03c75042 6s (feat/amend-command) In-progress refactors

If an amend requires a merge, I'd prefer to stay on the original commit, and have git branchless prompt me to redo the amend with the --merge flag.

@arxanas Since amend performs its actions within the context of an event transaction, is there a simple way to "abort" a transaction? (Or would we have to invoke undo somehow?)

Actually, I saw this as the desired behavior. It's kind of like the jj approach of allowing conflicts to exist and just having to resolve them later. I think we should just update the wording to suggest that you run git restack --merge instead of retrying the operation.
As a general note, there's not a way to abort a transaction; instead, it's best to just avoid committing the transaction until it's confirmed. Sometimes it's a bit difficult to orchestrate in concert with reference moves, see e.g. here:
https://github.com/arxanas/git-branchless/blob/708b76260e19a9dc5ab4fa910a0eb613fc8f077e/src/core/rewrite/execute.rs#L34-L39


[BUG] Sections not aligned in the center

Most of the sections in the website, specifically the About Us, Courses, Events, Mentorship, Podcast, Blogs and Founder section are not center aligned.
It seems there is extra whitespace on the left side of the sections.

Hey @inclinedadarsh , this has been fixed!!


Update dependencies and fix lint issues

Update dependencies to latest version.

Remove white background from launcher icons to fix lint issues:

"The launcher icon shape should use a distinct silhouette.
Launcher icons should not fill every pixel of their square region; see the design guide for details".

Codecov Report

Merging #12 into develop will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             develop      #12   +/-   ##
==========================================
  Coverage      47.08%   47.08%
  Complexity       290      290
==========================================
  Files             80       80
  Lines           1355     1355
  Branches          85       85
==========================================
  Hits             638      638
  Misses           677      677
  Partials          40       40

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6eece43...1a4da0e. Read the comment docs.


Functions in Kotlin: Android Basics in Compose

URL of codelab
https://developer.android.com/codelabs/basic-android-kotlin-compose-functions?continue=https%3A%2F%2Fdeveloper.android.com%2Fcourses%2Fpathways%2Fandroid-basics-compose-unit-1-pathway-1%23codelab-https%3A%2F%2Fdeveloper.android.com%2Fcodelabs%2Fbasic-android-kotlin-compose-functions#6
In which task and step of the codelab can this issue be found?
in 7. Default arguments
Describe the problem
just search for this:

The second call to birthdayGreeting() still uses the Rex value, which you passed in for the name.

The expression "still uses" seems weird in that context, as if it's something that was already done from way before, it doesn't seem to be referring to the function call just written, but of course it can't refer to anything else, so by exclusion, it's eventually understood.
But maybe's just me that understands it this way.
Additional information

@correabuscar
Thank you for reaching out to us and sharing your feedback.
As the content of the codelab doesn't need any fix so the ticket is closed.


NASR data

https://nfdc.faa.gov/xwiki/bin/view/NFDC/56+Day+NASR+Subscription

https://github.com/johncarney/pikelet


How to calculate height of grid items based on viewport height?

I have a full page RGL and I want to set the height (h of layout items) to be as close as possible (without generating scrollbars) to the viewport height.
So, when generating the layout array I set the h as window.innerHeight / rowHeight which should return the number of rows needed to take the full window height. But the rendered grid is a bit taller than expected (generating scrollbars).
How can I achieve this? Is there a specific formula that take margins/paddings into account that I'm not aware of?
Thanks

See https://github.com/STRML/react-grid-layout/issues/233#issuecomment-319995357


XP when inventory is full

Expected Behavior
spinning pokestops when inventory should still give XP
Actual Behavior
in the console it doesn't tell you how much XP was gained only "- Inventory is full!"
Steps to Reproduce
run bot

done


protoc-gen-go: Silently won't generate code if no service is defined in the proto file

What version of protobuf and what language are you using?
Golang 1.16.5
protoc v3.17.3
protoc-gen-go v1.27.1
protoc-gen-go-grpc v1.1
What did you do?
Try to generate code with two different proto files:
test.proto:
syntax = "proto3";

package test;
option go_package = "./pb";

message TestMessage {
    string id = 1;
}

protoc --proto_path=. --go-grpc_out=. test.proto. <-- No files produced
test-with-service.proto
syntax = "proto3";

package test;
option go_package = "./pb";

message TestMessage {
    string id = 1;
}

service Test {
}

protoc --proto_path=. --go-grpc_out=. test-with-service.proto <-- pb/test-with-service_grpc.pb.go generated
What did you expect to see?
Code generated for both files.
What did you see instead?
Code was only generated for the file with a service defined. The command that failed gives no output, it just silently fails.
I am able to generate for both files with an older version of the plugin, but it's such an old version I can't print out the version:
protoc-gen-go --version
protoc-gen-go: unknown argument "--version" (this program should be run by protoc, not directly)
protoc-gen-go -version
protoc-gen-go: unknown argument "-version" (this program should be run by protoc, not directly)

Anything else we should know about your project / environment?
I am trying to get this working with github actions, so protoc is installed with arduino/setup-protoc@v1.  For example:
protoc --go_out=. --go-grpc_out=. test.proto

See:
https://grpc.io/docs/languages/go/basics/#generating-client-and-server-code
(I'm not how clear on installing an older protoc-gen-go would help, since the command you give doesn't run it. If you only provide a --go-grpc_out flag, you only get service definitions.)


Showing help menu / access to the rootCommand object

Really enjoying the library, I started to write some code like it when using System.CommandLine directly. Thanks for releasing it.
I'm trying to match an existing tool that shows help when run with no arguments, as in this issue: https://github.com/dotnet/command-line-api/issues/1449
I found myself a bit blocked as the rootCommand immediately executes so I don't have something to pass the HelpContext. I hacked up an approximate workaround but it behaves a little differently still:
let rec showHelp (helpBuilder:HelpBuilder) =
    let hc = new HelpContext(helpBuilder,fakeRoot,System.Console.Out);
    helpBuilder.Write(hc)
    0

and fakeRoot = command "fakeRoot" {
    description "Deploys stuff"
    inputs (helpBuilder)
    setHandler showHelp
    addCommand realCommand
}

rootCommand args {
    description "Deploys stuff"
    inputs (helpBuilder)
    setHandler showHelp
    addCommand realCommand
}

I saw you discussed this over on https://github.com/JordanMarr/FSharp.SystemCommandLine/issues/1#issuecomment-1067218760 as well. Am I missing a good way to do this?

Colin,
Thanks for the feedback! I'm happy to hear that you are enjoying the library.
We definitely need a way for the rootCommand to not auto-run.
I'm trying to decide which approach would be best:
A) Add an autoRun operation that takes a boolean that defaults to true, but that you can manually set to autoRun false.
B) Create a new variation of the CE, rootCommandDef or similar that returns the actual RootCommand.
I think I am leaning towards option A (or similar).

Great! Any of those options looks like it would let me achieve what we're after.
Part of me also wonders if rootCommand should default cold and need the extra operation to make it auto-run, but I also really like the convenience you've achieved and most people are going to want to use it that way.

Thanks for the super fast release!
That lets me remove the duplication and pass in the real command. In case anyone wants to see the full hack:
let rec showHelp (helpBuilder:HelpBuilder) =
    let hc = new HelpContext(helpBuilder,root.Configuration.RootCommand,System.Console.Out);
    helpBuilder.Write(hc)
    0

and root: CommandLine.Parsing.Parser = rootCommandParser {
    description "Deploys stuff"
    inputs (helpBuilder)
    setHandler showHelp
    addCommand realCommand
    addCommand realCommand2
}

root.Parse(args).Invoke()

I think I have the same problem: it doesn't make sense to run my root command without a subcommand, and when I do I get no user feedback (unless I run with --help).
@cgravill  I'm not entirely clear on what helpBuilder is in the inputs (helpBuilder) line. Would you mind elaborating on how that is declared or obtained?

It appears to be built-in to the framework:
HelpBuilder.cs

@JordanMarr That's the type, but I was referring to the instance of that type. I'm just not sure how to get hold of one.

You can get an instance injected from the Input module. Here's a bit more of the code:
let envInput = Input.OptionRequired<string>(["--env"; "-e"], "Deployment environment (Earth, Moon, Mars)")

//More stuff

let helpBuilderInput = Input.InjectedDependency<HelpBuilder>()

S.CL beta 4 changed the way dependency injection works, so now the only thing you can inject is the BindingContext.
Consequently, Input. InjectedDependency has been replaced with Input.Context.
So you may be able to call ctx.GetService to get an instance of the help builder.

Good to know. We haven’t needed to update that project but will try that when we do

So it turns out that S.CL beta 4 makes removes the need for the recursive binds for this help scenario because everything you need already lives directly on the InvocationContext:
open FSharp.SystemCommandLine
open System.CommandLine.Parsing
open System.CommandLine.Invocation
open System.CommandLine.Help

let appendCmd =
    let handler (words: string array, separator: string option) =
        let separator = separator |> Option.defaultValue ", "
        System.String.Join(separator, words) |> printfn "Result: %s"

    let words = Input.Option(["--word"; "-w"], Array.empty, "A list of words to be appended")
    let separator = Input.OptionMaybe(["--separator"; "-s"], "A character that will separate the joined words.")

    command "append" {
        description "Appends words together"
        inputs (words, separator)
        setHandler handler
    }

[<EntryPoint>]
let main argv =
    let ctx = Input.Context()

    let showHelp (ctx: InvocationContext) =
        let hc = HelpContext(ctx.HelpBuilder, ctx.Parser.Configuration.RootCommand, System.Console.Out)
        ctx.HelpBuilder.Write(hc)

    let root =
        rootCommandParser {
            description "Appends words together"
            inputs (ctx)
            setHandler showHelp
            addCommand appendCmd
        }

    root.Parse(argv).Invoke()


Facing KarateException: -unknown-:8 - javascript evaluation failed: no variable found with name: response

I've a JS which retrieves the current working directory of Karate named 'base_path.js'.
function() {
var System = Java.type('java.lang.System')
var currentPath = System.getProperty("karate.feature.dir")
return currentPath
}
I'm trying to use this path in a feature file where I call this like :
* def basePath = call read("base_path.js")
Both of this feature file and the javascript file resides in the same directory.
On execution of this test in local machine with versions of tools:
Java : 1.8.0._212
Gradle: 5.0
OS : Windows 10  64-bit
It's working fine.
In the CI build system with Jenkins, the same code throws the error message :
com.intuit.karate.exception.KarateException: -unknown-:8 - javascript evaluation failed: read("base_path.js"), no variable found with name: response
On the line mentioned above as in 'def basePath'.
The build system runs with the versions of tools :
Zulu <IP_ADDRESS>-sa-jdk8.0.222-linux_x64
Gradle : 5.0
Any help would be appreciated.

you'll need to help up solve this, most likely the zulu JVM has some compatibility issues, take it up with their support. or follow this process: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue
no one has ever reported this

Hi Peter,
For more explanation, The variable definition was part of "Background".
Feature: Client Tests

  Background:
    Given url baseUrl
    And header Content-type = "application/json"
    And header Authorization = call read("basic_auth_as_admin.js")
    * def basePath = call read("base_path.js")

In 'basePath' variable definition, it was failing.
Kindly let me know if you need more inputs.

@VenkatADev this doesn't help at all. follow the process - or try to debug the karate code on the server - that is all I can offer

I had a comment as the first line of the ".js" file which was causing this issue. I know this would be documented somewhere not to have this first line as comments but I wish the error message was helpful. The following SO page helped me.
https://stackoverflow.com/questions/46918627/in-karate-dsl-calling-a-javascript-file-returns-java-lang-runtimeexception

@arisingh I look forward to your PR

To speed up the PR for the fix, I would appreciate if anyone who has some idea as to which part of the code may need fixing, could help in pointing out the code which may need fixing. The easiest solution might be to fix the error message - adding something to the effect that this error could also be caused by ...

@arisingh here is the developer guide: https://github.com/intuit/karate/wiki/Developer-Guide


update test for healthcareapis

update test and fix locaiton error:
{"error":{"code":"LocationNotAvailableForResourceType","message":"The
        provided location ''westus'' is not available for resource type ''Microsoft.HealthcareApis/services''.
        List of available regions for the resource type is ''ukwest,northcentralus,westus2,australiaeast,southeastasia,uksouth,eastus,westeurope,southcentralus,eastus2,northeurope''."}}

/azp run automation - sdk

Can one of the admins verify this patch?


README: Add link to a client hook written in Go

Coverage remained the same at 92.244% when pulling 945013579f9afe31ad291296157e50f53b9121ad on clients into f2fe21934cbff87cd054601ab724978bcef1be92 on master.


Cake Day Reminder bug

So... I got this popup today:

My cakeday is on June 3rd. I have a reminder set up to remind me 7 days before and on the exact day. So, there are two issues with this popup:

why did it appear 3 weeks before my cakeday?
my last cakeday wasn't 132 days ago but over 300 days ago

Can you open your settings JSON and see what value the registrationDate property has? And also cdr_days.

"registrationDate_sg":<PHONE_NUMBER>
"cdr_days": "7"
localStorage.esgst_cdr_bYear = "2018"
localStorage.esgst_cdr_dYear = "2019"

You got this immediately after updating to the latest version? For some reason, the feature thought your cake day was on January 3, and not June 3.

I'm not sure, because I have automatic updates of a Firefox addon, so I often don't really notice when exactly they happen, but seeing that there were multiple new versions in the past days it's very possible.

Whatever this was, I think it's gone now with #1396, so I'm closing, but let me know if it happens again.


Big update

I don't think this is ready to merge in, but I'd like to get your opnion on whats going on.
Will need to update readme based on whats deciced.
I am continuing to write tests and integrate with another project.
Let me know your thoughts

I am feeling that ApiModel should not get into the business of massaging validation errors and really be a simple/basic as possible
An implementation using api model should create error fields in realm to deal with displaying errors.
maybe...
Working through this on a project.  Will deliver more thoughts as I move through

more updates. including ApiModel property insertOrUpdate, and updated readme to reflect all the updates.

A lot of great stuff, and I love the effort your putting into this.
In response to:

An implementation using api model should create error fields in realm to deal with displaying errors.```

I mostly agree. I think we can always assume a model will have error properties, and ApiModel should standardize the naming of them `.errors` in on the `Api`-object for example. However, what the backend response should respond with, should be configurable, and not hard coded into ApiModel.

Closing because outdated. Feel free to open again if you make the necessary amendments 💃


[deps] Bump ws, 0.7 prevents build failures that could be caused by bina...

[deps] Bump ws, 0.7 prevents build failures that could be caused by binary add-ons.

LGTM

+1

Thanks @3rd-Eden !


updating pull_request_target event

updating pull_request_target event to match with webhook trigger

Why:

What's being changed:

Check off the following:

[ ] All of the tests are passing.
[ ] I have reviewed my changes in staging. (look for the deploy-to-heroku link in your pull request, then click View deployment)
[ ] For content changes, I have reviewed the localization checklist
[ ] For content changes, I have reviewed the Content style guide for GitHub Docs.

@jay2610 Thanks for opening a PR 👍 Please fill out the pull request template in the top comment in this pull request, then we'll get this up for review!

It looks like this is stale, so I'm going to close it out. @jay2610 feel free to reopen and fill out the pull request template if you'd like to pick up this PR again ✨


Proposition to extend framework "request" instance with validate method

It would be great to extend framework "request" instance in validation provider for same possibility as has laravel 5.5 framework
$validatedData = $request->validate([
        'title' => 'required|unique:posts|max:255',
        'body' => 'required'
    ]);

async request.validate({
    title: 'required|unique:posts|max:255',
    body: 'required'
})

If the validation rules pass, your code will keep executing normally; however, if validation fails, an exception will be thrown and the proper error response will automatically be sent back to the user. In the case of a traditional HTTP request, a redirect response will be generated, while a JSON response will be sent for AJAX requests.

it would make validation more simply than:
const { validate } = use('Validator')

class UserController {

  async store ({ request, session, response }) {
    const rules = {
      email: 'required|email|unique:users,email',
      password: 'required'
    }

    const validation = await validate(request.all(), rules)

    if (validation.fails()) {
      session
        .withErrors(validation.messages())
        .flashExcept(['password'])

      return response.redirect('back')
    }

something like:
requestInstance.validation = (rules) => {
  let validation = await validateAll(this.all(), rules)
  if (validation.fails()) {
     throw new ValidationException(validation.messages())
  }
}

i added this to kernel.js and this works! But maybe this would be better to add directly to ValitationProvider
const { validateAll } = use('Validator')
const Request = use('Adonis/Src/Request')
const CE = use('@adonisjs/validator/src/Exceptions')

Request.macro('validate', async function (rules) {
  let validation = await validateAll(this.all(), rules)
  if (validation.fails()) {
    throw CE.ValidationException.validationFailed(validation.messages())
  }
})

use:
    await request.validate({
      name: 'min:5',
      status: 'in:A,D'
    })

I suggest using Route validators for that.

Did you mean route middleware 'validation'? Yes, I used this, but for each action need to create own class file... This method seems to be much easier... especially for Rest API
I made RequestValidationProvider for this, but it will be great to have this logic by core... like has Laravel 5.5, it is very useful

So design goals of Laravel and Adonis will not be same always, I believe adding too many ways to do one thing leads to confused decisions.
Also we as human beings have our own style of doing things ( no one superior over others ).
So I suggest using the macro as you did in your project. Also you can share it with others by wrapping it inside a package and people who prefer your way will surely use it.

https://github.com/yariksav/adonis-request-validation


Apache JMeter

https://dlcdn.apache.org//jmeter/binaries/apache-jmeter-5.6.3.zip

And plugin manager: https://jmeter-plugins.org/get/

And plugins:

And plugins:

Related to:
https://jmeter-plugins.org/?search=jpgc-casutg


Savings-Account: Add a test for zero balance case

Issue
If current_balance is zero, there is a chance for an infinite loop.
Proposal
We're preventing the infinite loop when current_balance is zero by adding a test scenario that returns 0 for such a case.

The PR description states that this test prevents an infinite loop, but it is causing one. The solution should prevent the infinite loop, the test should only verify.
Can you update the solution file?
You can run the tests locally before submitting, by running rake test:savings-account.

@kotp, oups, my bad! Thank you very much for pointing that out! 🙏
I added a fixup!, can you, please, check it out?
Thank you!

What if the current balance is zero but the desired balance is zero?  This should not raise.  The example code disallows this.

@Adiis What is currently the problem with the infinite loop, from a student's point of view? Is it that they might end up with code that implements an infinite loop and not know how to deal with it? Will the test runner on the site fail in a way that is too confusing?

@Adiis What is currently the problem with the infinite loop, from a student's point of view? Is it that they might end up with code that implements an infinite loop and not know how to deal with it? Will the test runner on the site fail in a way that is too confusing?

@kytrinyx, that was precisely my goal as I run into an infinite loop while working on this exercise and I wanted to create something to prevent that in the first place, but I'm having trouble with how to include it in this exercise. This exercise doesn't need to deal with Exceptions but I need something to prevent the infinite loop.

@Adiis Okay, that's really helpful to know!
What happened when you had an infinite loop? Did the test runner crash? Or did it fail with an error message? (Can you recreate the problem, by any chance, and take a screenshot of what you see?)

@Adiis as we haven't heard anything further on this, and don't have enough information to decide how to proceed I'm going to go ahead and close this. If you have time to pick it back up, please holler, and we can reopen it!

@kytrinyx,

@Adiis as we haven't heard anything further on this, and don't have enough information to decide how to proceed I'm going to go ahead and close this. If you have time to pick it back up, please holler, and we can reopen it!

Sure, thank you!
I'm not sure what's the correct way and how to communicate on this issue.
I was stuck when I added a test (from this commit) that made rake test:savings-account run indefinitely.
I will try to figure out how to add a testing scenario and inform you about my ideas.
Thank you! 🙏

Would you be able to recreate the problem and take a screenshot of what the UI showed you?
In order to solve this we don't necessarily need to prevent an infinite loop, we could potentially give a useful error message if one occurs.


Movies delete button working

Movies delete button is working properly now.
#260  can be closed .
Please review the PR and see the working demo.
https://user-images.githubusercontent.com/81703477/170827053-78b569ea-72e5-4ff2-aef5-0ea1dff5db49.mp4

@SaraswatGit  please review this PR


"Discussion" labels confusing for self-posts

Hi there!
I was checking out https://brutalinks.tech and just thought I'd report what I think is a UX issue.
Using this post as an example: https://brutalinks.tech/~HT472/a2473c7b-797e-49d8-b913-630c79d86258
When viewed on the front page, there's two links, both labeled "Discussion", but they lead to two different places.
The first appears after the post title and (I believe) leads to a page with other self-posts.
The second appears on the line with the date posted, username, etc, and leads to the individual discussion.

Thank you for your interest, I agree it's confusing.
I think I iterated through various labels for these and this is the latest option.
Would it be less confusing if "(discussion)" would be "(discussions)" and the one leading to the actual comments would be "comments" ?


LinearRegression Exception: "AttributeError: 'tuple' object has no attribute 'shape'"

What happened:
In order to use dask-ml models to train on a dask DataFrame, the DataFrame must be converted to a dask array.

#df is a dask DataFrame that looks something like this:
#
#                 flower.petal_length flower.petal_width flower.sepal_length
# npartitions=73
#                            float64            float64             float64
#                                ...                ...                 ...
# ...                            ...                ...                 ...
#                                ...                ...                 ...
#                                ...                ...                 ...
# Dask Name: astype, 1185 tasks

train_array = df.to_dask_array(lengths=True)
train_labels_array = df.to_dask_array(lengths=True)

# If I simply do a train_array.compute() or train_labels_array.compute() here & comment out .fit(), there are no problems

dask_model = LinearRegression()
dask_model.fit(train_array, train_labels_array)

When calling .fit on LinearRegression or LogisticRegression, I'm receiving the following output from the dask cluster:
distributed.worker - WARNING -  Compute Failed
Function:  subgraph_callable
args:      (('rechunk-merge-ff32808be3096a08028f7c8aa2a4bae3', 66, 0))
kwargs:    {}
 Exception: AttributeError("'tuple' object has no attribute 'shape'",)

Followed by the following exception being thrown:
   File "my_model.py", line 89, in model
     dask_model.fit(train_array, train_labels_array)
   File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/dask_ml/linear_model/glm.py", line 187, in fit
     self._coef = algorithms._solvers[self.solver](X, y, **solver_kwargs)
   File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/dask_glm/utils.py", line 17, in normalize_inputs
     mean, std = da.compute(X.mean(axis=0), X.std(axis=0))
   File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/dask/base.py", line 567, in compute
     results = schedule(dsk, keys, **kwargs)
   File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/distributed/client.py", line 2676, in get
     results = self.gather(packed, asynchronous=asynchronous, direct=direct)
   File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/distributed/client.py", line 1991, in gather
     asynchronous=asynchronous,
   File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/distributed/client.py", line 832, in sync
     self.loop, func, *args, callback_timeout=callback_timeout, **kwargs
   File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/distributed/utils.py", line 340, in sync
     raise exc.with_traceback(tb)
   File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/distributed/utils.py", line 324, in f
     result[0] = yield future
   File "/opt/rh/rh-python36/root/usr/lib64/python3.6/site-packages/tornado/gen.py", line 762, in run
     value = future.result()
   File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/distributed/client.py", line 1850, in _gather
     raise exception.with_traceback(traceback)
   File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/dask/optimization.py", line 963, in __call__
     return core.get(self.dsk, self.outkey, dict(zip(self.inkeys, args)))
   File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/dask/core.py", line 151, in get
     result = _execute_task(task, cache)
   File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/dask/core.py", line 121, in _execute_task
     return func(*(_execute_task(a, cache) for a in args))
   File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/dask/utils.py", line 30, in apply
     return func(*args, **kwargs)
   File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/dask/array/reductions.py", line 594, in mean_chunk
     n = numel(x, dtype=dtype, **kwargs)
   File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/dask/array/reductions.py", line 558, in numel
     shape = x.shape
 AttributeError: 'tuple' object has no attribute 'shape'

What you expected to happen:
Minimal Complete Verifiable Example:
Unfortunately, I'm unable to come up with a minimal example that replicates the existing behavior. The DataFrame above is going through several steps before arriving at the point of being converted to an array (being published in a cluster, having the index reset, potentially undergoing several transformations via task_graphs and delayed(func) to .compute() calls). I'm including a basic example of what's happening above to demonstrate, but when doing this in a dask LocalCluster separate from the other environment I don't see the same issue.
Environment:

Dask version: 2.10.1
Dask-ml version: 1.7.0
dask-glm version: 0.2.0
sklearn version: 0.23.2
Python version: 3.6.8
Operating System: Centos7
Install method (conda, pip, source): pip

Thanks for the report. Let us know if you can provide a reproducible example.

Thanks for the report. Let us know if you can provide a reproducible example.

Hi @Jreyno40,
This one would be quite tricky to pin down without a reproducible example.
If your data transformation pipeline is non-confidential, perhaps you could try to provide a full, reproducible example using some sort of artificial or pseudo-random data, via e.g. dask_ml.datasets.make_blobs?

Hi @Jreyno40,
I had a similar error with a very basic example: https://examples.dask.org/dataframes/01-data-access.html
Specifically, at the two lines:
df = dd.read_csv('data/2000-*-*.csv')
df.head()

I end up with the a stack trace similar to yours, but the cause is a bit different "AttributeError: 'tuple' object has no attribute 'head'"
Anyway, I solved it by specifying a scheduler option to the compute method and placing it in between the two lines above.
df = dd.read_csv('data/2000-*-*.csv')
df = df.compute(scheduler='threads')
df.head()

I am only starting to learn how to use Dask, but I guess this problem might be similar to yours, and the culprit could be the scheduler back end (threads vs processes) and how the data serialization works under the hood.
If you try the same example in the link I attached above, but only removing the client option altogether, or changing the processes=False to processes=True, then you don't need the "df = df.compute(scheduler='threads')" line at all and the examples works just fine.
Again, maybe I did some naive mistake as a new user of Dask, but I hope it helps you fix the error you encountered.

I am getting the same exact error right now using logistic regression. I don't think I'll be able to provide a reproducible example but if it is helpful I will describe where I am encountering this issue.
I am using dask version 2021.11.2. I am testing my code by pulling my data in batches of 1000 rows. When I pull 2-4 batches of 1000 rows and load them into separate partitions of a dask dataframe (using from_delayed) I am able to fit a model successfully (spread across 5 workers). When I expand this to 5 batches of 1000 rows (still spread across 5 workers) I start getting this error. It's only when I start to distribute the data to more workers that I receive this error not when my partitions increase in size which is very strange.
I have also tried increasing the number of workers to 6 and and still using 5 batches of 1000 rows as well as 6 batches of 1000 rows. Both failed with this same error.
I know this description might not be helpful without a reproducible example but I figured it was worth posting in case that could point you in the right direction somehow.
Thank you for your help!

As an update:
I tried upgrading the versions of dask==2022.2.0 and dask-ml==2022.5.27 on the client and workers and re-running my code. I no longer receive the attribute error mentioned above but now I receive a different error on the fitting of the logistic regression: "ValueError: shapes (0,) and (739,) not aligned: 0 (dim 0) != 739 (dim 0)". I have printed out the shapes of both arrays before fitting and their shapes do align and neither of them have zero dimensions.
It seems like there might be some sort of communication issue with the dask workers or something else that I do not understand. This code also works perfectly fine with less than 5 partitions of data.
I will try and open up a new issue for this error since it sounds like it is different but could still be related.
Here is the stack trace for this error if it is helpful:
  File "xy_querying_data.py", line 346, in run
    lr.fit(X_train_arr, y_train_arr)
  File "/opt/conda/lib/python3.7/site-packages/dask_ml/linear_model/glm.py", line 188, in fit
    self._coef = algorithms._solvers[self.solver](X, y, **solver_kwargs)
  File "/opt/conda/lib/python3.7/site-packages/dask_glm/utils.py", line 26, in normalize_inputs
    out = algo(Xn, y, *args, **kwargs).copy()
  File "/opt/conda/lib/python3.7/site-packages/dask_glm/algorithms.py", line 265, in admm
    new_betas = np.array(da.compute(*new_betas))
  File "/opt/conda/lib/python3.7/site-packages/dask/base.py", line 573, in compute
    results = schedule(dsk, keys, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/distributed/client.py", line 2994, in get
    results = self.gather(packed, asynchronous=asynchronous, direct=direct)
  File "/opt/conda/lib/python3.7/site-packages/distributed/client.py", line 2152, in gather
    asynchronous=asynchronous,
  File "/opt/conda/lib/python3.7/site-packages/distributed/utils.py", line 310, in sync
    self.loop, func, *args, callback_timeout=callback_timeout, **kwargs
  File "/opt/conda/lib/python3.7/site-packages/distributed/utils.py", line 376, in sync
    raise exc.with_traceback(tb)
  File "/opt/conda/lib/python3.7/site-packages/distributed/utils.py", line 349, in f
    result = yield future
  File "/opt/conda/lib/python3.7/site-packages/tornado/gen.py", line 769, in run
    value = future.result()
  File "/opt/conda/lib/python3.7/site-packages/distributed/client.py", line 2009, in _gather
    raise exception.with_traceback(traceback)
  File "/opt/conda/lib/python3.7/site-packages/dask/utils.py", line 39, in apply
    return func(*args, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/dask_glm/algorithms.py", line 300, in local_update
    maxfun=250)
  File "/opt/conda/lib/python3.7/site-packages/scipy/optimize/lbfgsb.py", line 198, in fmin_l_bfgs_b
    **opts)
  File "/opt/conda/lib/python3.7/site-packages/scipy/optimize/lbfgsb.py", line 308, in _minimize_lbfgsb
    finite_diff_rel_step=finite_diff_rel_step)
  File "/opt/conda/lib/python3.7/site-packages/scipy/optimize/optimize.py", line 262, in _prepare_scalar_function
    finite_diff_rel_step, bounds, epsilon=epsilon)
  File "/opt/conda/lib/python3.7/site-packages/scipy/optimize/_differentiable_functions.py", line 140, in __init__
    self._update_fun()
  File "/opt/conda/lib/python3.7/site-packages/scipy/optimize/_differentiable_functions.py", line 233, in _update_fun
    self._update_fun_impl()
  File "/opt/conda/lib/python3.7/site-packages/scipy/optimize/_differentiable_functions.py", line 137, in update_fun
    self.f = fun_wrapped(self.x)
  File "/opt/conda/lib/python3.7/site-packages/scipy/optimize/_differentiable_functions.py", line 134, in fun_wrapped
    return fun(np.copy(x), *args)
  File "/opt/conda/lib/python3.7/site-packages/dask_glm/algorithms.py", line 234, in wrapped
    return func(beta, X, y) + (rho / 2) * np.dot(beta - z + u,
  File "/opt/conda/lib/python3.7/site-packages/dask_glm/families.py", line 31, in pointwise_loss
    return Logistic.loglike(Xbeta, y)
  File "/opt/conda/lib/python3.7/site-packages/dask_glm/families.py", line 24, in loglike
    return (Xbeta + log1p(enXbeta)).sum() - dot(y, Xbeta)
  File "/opt/conda/lib/python3.7/site-packages/multipledispatch/dispatcher.py", line 278, in __call__
    return func(*args, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/dask_glm/utils.py", line 126, in dot
    return np.dot(A, B)
  File "<__array_function__ internals>", line 6, in dot
ValueError: shapes (0,) and (739,) not aligned: 0 (dim 0) != 739 (dim 0)


Depend on attrs explicitly

Currently, the project depends on attrs, but its only mention is in pyproject.lock. This makes dependency explicit in pyproject.toml.

Pull Request Test Coverage Report for Build 4

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 96.97%

Totals

Change from base Build 3:
0.0%

Covered Lines:
27

Relevant Lines:
27

💛  - Coveralls

Thanks!


chore: Improve logging by moving git operations progress to DEBUG level

This also adds an INFO message every time a repository has been successfully processed giving the end user a meaningful default progress output at the INFO level without flooding stdout with the git operation progress output (Which is now still accessible through the DEBUG log level)
Addressing: https://github.com/gruntwork-io/git-xargs/issues/36

No worries. I was never expecting an instant review :) We all have other priorities.
Your suggestion sounds good to me. I moved that specific message to the logger.


Inconsistent Italics when returning JSX

I noticed when returning JSX using the function keyword and using the function expression syntax, the function expression syntax, for some reason, italicizes the text whereas the function syntax does not.
(Using latest version - 2.0.1)
Screenshot

Hey there! Thanks for filing this. This is actually the intended behavior (this theme maximizes small differences in order to provide a lot of contrast. I have a no italics version if it bothers you.


Reduce the backoff time in Clients

We have a use-case where the customer is latency sensitive, has a huge number of topics and ideally, never wants timeouts to occur (2 seconds).
When brokers are restarted it takes around 1 to 4 seconds for bundles to unload especially those bundles which have 800+ number of topics - which leads to timeouts. To make matters during worse our backoff logic works as follows:-

Try number
Backoff Value
Next try

1
100ms
100ms

2
200ms
300ms

3
400ms
700ms

4
800ms
1500ms

5
1600ms
3100ms

6
3200ms
6300ms

As highlighted if the connect at 1.5 seconds fails then the next attempt is made in 3.1 seconds and we lose out on the 0.5 seconds where we could have potentially got the message.
We could make the initial backoff value (100 ms) and multiplier (2) configurable but as pointed out by @rdhabalia - not many clients will be interested in configuring this value, hence we can may be just hard code the multiplier to 1.5 instead of 2.
Difference: As seen below the growth of the backoff time is slower with a multiplier as 1.5 but it soon catches up and reaches the max value as desired.

Failure Number
Current backoff (ms)
New backoff (ms)

Multiplier = 2
Multiplier = 1.5

1
100
100

2
200
150

3
400
225

4
800
338

5
1600
506

6
3200
759

7
6400
1139

8
12800
1709

9
25600
2563

10
51200
3844

11
60000
5767

12
60000
8650

13
60000
12975

14
60000
19462

15
60000
29193

16
60000
43789

17
60000
60000

18
60000
60000

@saandrews @rdhabalia @merlimat  - The code change for this is very small - just wanted consent on the approach.

👍

There is a concern that this approach would increase the chances of cold start scenario since we try to establish the connections more frequently.
In stead as suggested by @joefk we can go with an approach where we squeeze in a retry 100 millis before the timeout hence if the timeout is 2000 ms

Try number
Backoff Value
Next try

1
100ms
100ms

2
200ms
300ms

3
400ms
700ms

4
800ms
1500ms

5
400ms
1900ms

6
3200ms
5100ms

I like the new approach what do you guys think?

Implemented in #747


System.IO.FileNotFoundException V1Interop.ITaskScheduler.Activate

Can you help me, I just got this error on Windows XP Pro (SP3)
System.IO.FileNotFoundException: The system cannot find the file specified. (Exception from HRESULT: 0x80070002)
at Microsoft.Win32.TaskScheduler.V1Interop.ITaskScheduler.Activate(String Name, Guid& riid)
at Microsoft.Win32.TaskScheduler.TaskService.GetTask(ITaskScheduler iSvc, String name)
at Microsoft.Win32.TaskScheduler.TaskService.GetTask(String taskPath)
at SmartShare.schedulemanager.ScheduleSleep(Int32 idleSetting)
at SmartShare.My.MyApplication.MyApplication_Startup(Object sender, StartupEventArgs e)
Originally posted: 2014-12-08T23:00:02

What is the name of your task? Win XP treats tasks as files but it manipulates the filename. This error often occurs when the name of task, once manipulated, gets into a cycle where the manipulation conflicts with the name. It can also occur when the user account used to construct the TaskService instance does not have permissions on the C:\Windows\Tasks folder.
Originally posted: 2014-12-09T08:19:24

Were you able to resolve your issue?
Originally posted: 2015-01-26T09:43:50

Unknown resolution
Originally posted: 2015-02-17T09:46:05


Output to ffmpeg via -O doesn't work?

I used v4l2loopback to create /dev/video1,
then used v4l2rtspserver -W 1920 -H 1080 -O /dev/video1 -S /dev/video0
after that ffmpeg -i /dev/video1 -vcodec h264 -an -f segment -strftime 1 -segment_time 60 -segment_format mp4 " %d-%m_%H-%M-%S.mp4"
But received only no frame! errors from ffmpeg. I think the v4l2rtspserver -O isn't working is it?
Anyway to fix it, or anyway to capture the stream, in the same way ffmpeg can do?

Hi,
The -O feature was for debug, It writes what was read, then if your device never repeat the SPS & PPS, the virtual device will never receive it.
If you are using the official V4L2 for rapicam, you could enable this with :
 v4l2-ctl  -c repeat_sequence_header=1

Best Regards,
Michel.

 v4l2-ctl  -c repeat_sequence_header=1 works
but occasionally, I would get
 [h264 @ 0x2861380] decode_slice_header error
[h264 @ 0x2861380] no frame!
at the start of ffmpeg still.
Restart the ffmpeg process and everything works again.

I am also using -vcodec copy now instead by the way.

Nevermind. Seems to work well. Thank You!! I'll close this issue now.

Occasionally when starting FFmpeg for recording v4l2, this happens:
DTS<PHONE_NUMBER>665288, next:5933274 st:0 invalid dropping
PTS<PHONE_NUMBER>665288, next:5933274 invalid dropping st:0
DTS<PHONE_NUMBER>705617, next:5999940 st:0 invalid dropping
PTS<PHONE_NUMBER>705617, next:5999940 invalid dropping st:0
DTS<PHONE_NUMBER>746298, next:6066606 st:0 invalid dropping
PTS<PHONE_NUMBER>746298, next:6066606 invalid dropping st:0
DTS<PHONE_NUMBER>786121, next:6133272 st:0 invalid dropping
PTS<PHONE_NUMBER>786121, next:6133272 invalid dropping st:0

after Ctrl-c, this message appears
Assertion av_rescale_rnd(start_dts, 1000, track->timescale, AV_ROUND_DOWN) <= 0 failed at libavformat/movenc.c:2869
Why is it so?

Hi,
This seems related to ffmpeg.
Regards,
Michel.


Display CGM Data in mmol vs. mg/dL

Was able to pull some CGM data from Dexcom Share.  As a Canadian user I am displaying data in mmol; however, data is only being displayed in mg/DL in loop.
Could this be a configurable item for the CGM data graph?

Absolutely, though no setting is necessary because the setting exists in HealthKit.
https://github.com/loudnate/LoopKit/issues/27 is a pre-requisite here.
Please don't hesitate to file any additional issues you encounter with units.

Is a code change required to enable this?  My settings in HealthKit are in mmol.  Interestingly, the Glucose value listed in loop (at the bottom of the app) is displayed in mmol.  But the graph view is displaying in mg/DL.

Yes, this is something I plan to fix. If you'd like to just quickly change your code in the meantime, you'll find two "TODO" items in this file.
https://github.com/loudnate/Loop/blob/master/Loop/Managers/StatusChartManager.swift

I just tried modifying the code, but I noticed that the glucose predictions are not being displayed properly when I change the units.  The graph is also showing from 0 to 50 units, which makes the line almost flat regardless. I'm not sure how to change that -- my own lack of knowledge of this software language, I'm sure.  If you have a chance to direct me at which lines I need to modify, in the meantime, that would be great.
Thanks!

I believe that scale is logarithmic, not linear, so even if it goes to 50 it should still show temps.

I changed line 258 to adjust the scale.  The segments where set to 25 units of whatever and a max of 4 of them.  I changed it to 5 units per segment with a max of 5.


First pass and correct T2w segmentations

@Kaonashi22 While waiting for the vertebral labeling #9 I suggest to:

run the pipeline on all subjects
visually QC the segmentation of the T2w
manually correct the segmentation when needed using https://github.com/spinalcordtoolbox/manual-correction. Manual correction should only be done for levels that will be used for morphometric calculations (C2-T5 if I remember correctly)
save the manually corrected segmentations under the derivatives/labels folder inside the source database. Alternatively, we could also upload the manual corrections as part of a release on this repository (to make sure to track the modifications, to make sure it is not lost, and to facilitate collaboration-- eg: for me to try running the pipeline with your manual corrections).

Thanks, I will do you that!

From: Julien Cohen-Adad @.>
Sent: March 14, 2024 16:01
To: sct-pipeline/spine-park @.>
Cc: Lydia Chougar, Dr @.>; Mention @.>
Subject: [sct-pipeline/spine-park] First pass and correct T2w segmentations (Issue #15)
@Kaonashi22https://github.com/Kaonashi22 While waiting for the vertebral labeling #9https://github.com/sct-pipeline/spine-park/issues/9 I suggest to:

run the pipeline on all subjects
visually QC the segmentation of the T2w
manually correct the segmentation when needed using https://github.com/spinalcordtoolbox/manual-correction. Manual correction should only be done for levels that will be used for morphometric calculations (C2-T5 if I remember correctly)
save the manually corrected segmentations under the derivatives/labels folder inside the source database. Alternatively, we could also upload the manual corrections as part of a release on this repository (to make sure to track the modifications, to make sure it is not lost, and to facilitate collaboration-- eg: for me to try running the pipeline with your manual corrections).

—
Reply to this email directly, view it on GitHubhttps://github.com/sct-pipeline/spine-park/issues/15, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BFCFJYTCAMX6KAHIMNECBSLYYH6ZDAVCNFSM6AAAAABEWB5SPCVHI2DSMVQWIX3LMV43ASLTON2WKOZSGE4DMMZSGUZDKMQ.
You are receiving this because you were mentioned.Message ID: @.***>

@Kaonashi22 let me know when you have corrected 2-3 segmentations. At this point, you can send me the derivatives folder, and I can modify the processing script so that it accounts for these manual corrections. That way we can work in parallel (ie: you correcting the seg, and me updating the analysis script)

Thanks, I will send you the derivative folders soon.

From: Julien Cohen-Adad @.>
Sent: March 19, 2024 09:52
To: sct-pipeline/spine-park @.>
Cc: Lydia Chougar, Dr @.>; Mention @.>
Subject: Re: [sct-pipeline/spine-park] First pass and correct T2w segmentations (Issue #15)
@Kaonashi22https://github.com/Kaonashi22 let me know when you have corrected 2-3 segmentations. At this point, you can send me the derivatives folder, and I can modify the processing script so that it accounts for these manual corrections. That way we can work in parallel (ie: you correcting the seg, and me updating the analysis script)
—
Reply to this email directly, view it on GitHubhttps://github.com/sct-pipeline/spine-park/issues/15#issuecomment-2007238183, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BFCFJYQSOF47U7ZNILF4ZO3YZA7JPAVCNFSM6AAAAABEWB5SPCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMBXGIZTQMJYGM.
 Sometimes, two or three voxels are missing on some slices; how precise the segmentation should be? Is it worth correcting manually these masks?
Also, the lowest slice of the T2 volume usually has a poor signal;  would you recommend discarding the segmentation at this level?

I attached the derivatives folder with the masks manually corrected.
Regarding the rest of the pipeline:
-for the remaining subjects, I gor this error:
derivatives.zip

Overall, the segmentations of T2-w images are accurate. Sometimes, two or three voxels are missing on some slices; how precise the segmentation should be? Is it worth correcting manually these masks?

It depends how precise you want the results to be. For example, if a slice is missing 2-3 pixels, and you average the CSA across, let's say, 20 slices, assuming a CSA of 80 mm2 at a resolution of 0.8mm2, 3 pixels correspond to 1.92 mm2, which represents 2.4% of the CSA computed on a single slice, or 0.14% of the CSA computed over 20 slices (so quite negligible, I would say).

Also, the lowest slice of the T2 volume usually has a poor signal; would you recommend discarding the segmentation at this level?

It depends how important the lowest slice is for your analysis (eg: are you considering it? if not, not important)

About the issues:

err.batch_processing_sub-CG176.log: error during registration of T2w to template. Could you send me this subjects?
err.batch_processing_sub-DEV148Sujet01.log: The script is looking for the file warp_template2T1w.nii.gz, but it does not exist. Strangely, the latest version of the script is looking for a different file name (../anat/warp_template2anat.nii.gz):
https://github.com/sct-pipeline/spine-park/blob/8e541e9064c9be264a85f867ed93371df80c506b/batch_processing.sh#L253-L259
What version of the script did you run? If you send me this subject I can also try to reproduce.
err.batch_processing_sub-DEV203Sujet08.log: Error generated during QC report (portalocker.exceptions.LockException: [Errno 5] Input/output error). I am wondering if this is related to the parallel writing (due to multiple jobs) on a file on a mounted disk. Could you send me this image as well so I can try to reproduce? Tagging @joshuacwnewton so he is aware
err.batch_processing_sub-RC194.log: We get an OSError: [Errno 9] Bad file descriptor. I'm not sure how to interpret this. Could you also send me this subjects?

err.batch_processing_sub-DEV203Sujet08.log: Error generated during QC report (portalocker.exceptions.LockException: [Errno 5] Input/output error). I am wondering if this is related to the parallel writing (due to multiple jobs) on a file on a mounted disk. Could you send me this image as well so I can try to reproduce? Tagging @joshuacwnewton so he is aware

Full traceback for context
Traceback (most recent call last):
  File "/export02/data/lydiac/spinalcordtoolbox/spinalcordtoolbox/scripts/sct_deepseg_sc.py", line 211, in <module>
    main(sys.argv[1:])
  File "/export02/data/lydiac/spinalcordtoolbox/spinalcordtoolbox/scripts/sct_deepseg_sc.py", line 204, in main
    generate_qc(fname_image, fname_seg=fname_seg, args=argv, path_qc=os.path.abspath(path_qc),
  File "/export02/data/lydiac/spinalcordtoolbox/spinalcordtoolbox/reports/qc.py", line 715, in generate_qc
    QcImage(
  File "/export02/data/lydiac/spinalcordtoolbox/spinalcordtoolbox/reports/qc.py", line 242, in layout
    self._make_QC_image_for_3d_volumes(img, mask, plane=self.qc_report.plane)
  File "/export02/data/lydiac/spinalcordtoolbox/spinalcordtoolbox/reports/qc.py", line 290, in _make_QC_image_for_3d_volumes
    self.qc_report.update_description_file()
  File "/export02/data/lydiac/spinalcordtoolbox/spinalcordtoolbox/reports/qc.py", line 529, in update_description_file
    portalocker.lock(dest_file, portalocker.LOCK_EX)
  File "/export02/data/lydiac/spinalcordtoolbox/python/envs/venv_sct/lib/python3.9/site-packages/portalocker/portalocker.py", line 111, in lock
    raise exceptions.LockException(exc_value, fh=file_) from exc_value
portalocker.exceptions.LockException: [Errno 5] Input/output error

err.batch_processing_sub-RC194.log: We get an OSError: [Errno 9] Bad file descriptor. I'm not sure how to interpret this. Could you also send me this subjects?

Full traceback for context
Traceback (most recent call last):
  File "/export02/data/lydiac/spinalcordtoolbox/spinalcordtoolbox/scripts/sct_deepseg_sc.py", line 211, in <module>
    main(sys.argv[1:])
  File "/export02/data/lydiac/spinalcordtoolbox/spinalcordtoolbox/scripts/sct_deepseg_sc.py", line 204, in main
    generate_qc(fname_image, fname_seg=fname_seg, args=argv, path_qc=os.path.abspath(path_qc),
  File "/export02/data/lydiac/spinalcordtoolbox/spinalcordtoolbox/reports/qc.py", line 715, in generate_qc
    QcImage(
  File "/export02/data/lydiac/spinalcordtoolbox/spinalcordtoolbox/reports/qc.py", line 242, in layout
    self._make_QC_image_for_3d_volumes(img, mask, plane=self.qc_report.plane)
  File "/export02/data/lydiac/spinalcordtoolbox/spinalcordtoolbox/reports/qc.py", line 290, in _make_QC_image_for_3d_volumes
    self.qc_report.update_description_file()
  File "/export02/data/lydiac/spinalcordtoolbox/spinalcordtoolbox/reports/qc.py", line 564, in update_description_file
    dest_file.truncate()
OSError: [Errno 9] Bad file descriptor

Both tracebacks explicitly mention the dest_file file object (which we lock using portalocker, and which points to index.html of the QC report). Also, both errors ("[Errno 5]", "[Errno 9]") are Linux system errors. So, I definitely think it's a safe conclusion to say that this is related to file locking + parallel processing + the mounted file system.
I believe it's errors such as this that lead us to recommend not performing processing on mounted drives.
See, for example:

https://github.com/spinalcordtoolbox/spinalcordtoolbox/issues/3030#issuecomment-1322484167
https://github.com/spinalcordtoolbox/spinalcordtoolbox/issues/3900#issuecomment-1343250323
https://github.com/neuropoly/intranet.neuro.polymtl.ca/commit/385e114aa2f2f8640b8f27aff75f999f5d65166e

I believe it's errors such as this that lead us to recommend not performing processing on mounted drives.

The problem is that, in many labs, data are located on a mounted drive and researchers are imposed to process their data that way. So I'm wondering if there is a workaround for this. I guess using -j 1 would be a workaround, but it would increase processing time by a lot.

So I'm wondering if there is a workaround for this.

I think it should be possible to set the QC location somewhere off the mounted drive, while letting the rest of processing happen within the mounted drive? (This is probably worth doing anyway as a test to see if we can isolate the issue.)
I think a similar conclusion was reached by @mguaypaq in this comment:

The problems are showing up because the output directory is a mounted shared drive from duke. But it's generally a bad idea to write results directly to duke, and the proper workflow is to simply use a local directory for the output. (The input can still be directly taken from duke, though.)

I think it should be possible to set the QC location somewhere off the mounted drive, while letting the rest of processing happen within the mounted drive?

YES! Brilliant idea! the same way we do it with our temp folders for processing intermediate NIfTI files.

I've summarized our discussion in a new issue on the SCT repo (https://github.com/spinalcordtoolbox/spinalcordtoolbox/issues/4423), just so that we aren't distracting from the spine-park related discussion in this issue. :)

Thanks @jcohenadad, I'll send you the images by email
For sub-DEV148Sujet01, I was indeed using on older version of the script (https://github.com/sct-pipeline/spine-park/commit/1b378e0d9ae743b8142716991956e1598dff5dd6)

I'm running the script from a mounted drive (where the SCT and the current script are saved), but the input and output folders are located on a server. Does it have any influence?

From: Julien Cohen-Adad @.>
Sent: April 4, 2024 22:21
To: sct-pipeline/spine-park @.>
Cc: Lydia Chougar, Dr @.>; Mention @.>
Subject: Re: [sct-pipeline/spine-park] First pass and correct T2w segmentations (Issue #15)
I believe it's errors such as this that lead us to recommend not performing processing on mounted drives.
The problem is that, in many labs, data are located on a mounted drive and researchers are imposed to process their data that way. So I'm wondering if there is a workaround for this. I guess using -j 1 would be a workaround, but it would increase processing time by a lot.
—
Reply to this email directly, view it on GitHubhttps://github.com/sct-pipeline/spine-park/issues/15#issuecomment-2038640090, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BFCFJYUQAHOIGRYOUZC6I5TY3YDDLAVCNFSM6AAAAABEWB5SPCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZYGY2DAMBZGA.
You are receiving this because you were mentioned.Message ID: @.***>

@Kaonashi22 I've ran the script version 78129ca6e3fe90f39b20af565a5e1918d2f6754e on the data you sent me (incl. those that produced the errors https://github.com/sct-pipeline/spine-park/issues/15#issuecomment-2038269172) and I did not observe any error.
Terminal output
julien-macbook:~/temp/Lydia/results_20240405_125118/log $ ls -1
batch_processing_sub-BB277.log
batch_processing_sub-BJ170.log
batch_processing_sub-CG176.log
batch_processing_sub-DEV148Sujet01.log
batch_processing_sub-DEV203Sujet08.log
batch_processing_sub-DEV206Sujet10.log
batch_processing_sub-LC164.log
batch_processing_sub-LM166.log
batch_processing_sub-RC194.log

Therefore, I suspect the issues you've observed were caused by parallel writing on the locked QC file. One way to overcome this is to use the flag -jobs 1. Can you please try to see if it solves the issue?

Sounds good, thank you.
I'm not sure I see the script version https://github.com/sct-pipeline/spine-park/commit/78129ca6e3fe90f39b20af565a5e1918d2f6754e
78129ca.Is it the file posted here: https://github.com/sct-pipeline/spine-park/commit/78129ca6e3fe90f39b20af565a5e1918d2f6754e https://github.com/sct-pipeline/spine-park/pull/8/files?https://github.com/sct-pipeline/spine-park/commit/78129ca6e3fe90f39b20af565a5e1918d2f6754e

From: Julien Cohen-Adad @.>
Sent: April 6, 2024 15:37
To: sct-pipeline/spine-park @.>
Cc: Lydia Chougar, Dr @.>; Mention @.>
Subject: Re: [sct-pipeline/spine-park] First pass and correct T2w segmentations (Issue #15)
@Kaonashi22https://github.com/Kaonashi22 I've ran the script version 78129cahttps://github.com/sct-pipeline/spine-park/commit/78129ca6e3fe90f39b20af565a5e1918d2f6754e on the data you sent me (incl. those that produced the errors #15 (comment)https://github.com/sct-pipeline/spine-park/issues/15#issuecomment-2038269172) and I did not observe any error.
Terminal output
julien-macbook:~/temp/Lydia/results_20240405_125118/log $ ls -1
batch_processing_sub-BB277.log
batch_processing_sub-BJ170.log
batch_processing_sub-CG176.log
batch_processing_sub-DEV148Sujet01.log
batch_processing_sub-DEV203Sujet08.log
batch_processing_sub-DEV206Sujet10.log
batch_processing_sub-LC164.log
batch_processing_sub-LM166.log
batch_processing_sub-RC194.log
Therefore, I suspect the issues you've observed were caused by parallel writing on the locked QC file. One way to overcome this is to use the flag -jobs 1. Can you please try to see if it solves the issue?
—
Reply to this email directly, view it on GitHubhttps://github.com/sct-pipeline/spine-park/issues/15#issuecomment-2041175816, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BFCFJYWFCW4BBLRUUZADSVLY4BFIRAVCNFSM6AAAAABEWB5SPCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBRGE3TKOBRGY.
You are receiving this because you were mentioned.Message ID: @.***>

The number refers to a git SHA that points to a state of the repository. Here is the link to that version of the repository, that includes the file: https://github.com/sct-pipeline/spine-park/tree/78129ca6e3fe90f39b20af565a5e1918d2f6754e
But what you should do instead of manually downloading the file from this repository, is go to your local repository and run:
git pull

Then you can make sure you are running the proper version by running:
git log --pretty=oneline

which indicates the version (top item in the list):
julien-macbook:~/code/spine-park $ git log --pretty=oneline
78129ca6e3fe90f39b20af565a5e1918d2f6754e (HEAD -> main, origin/main, origin/HEAD) Added .gitignore  <--- THIS ONE
9bfb14077c49237b83535a326357f81d7b9a57de Sort DWI chunks from top to bottom (#18)
8e541e9064c9be264a85f867ed93371df80c506b (jca/17-manual-corr) Create analysis script (#8)
ca8104e9f430e1f7e47605aabae21c1c86790308 Update README.md
2c078e6887ad5a3be9eff721e405269e45d728bb Added doc to convert to BIDS
183e7b90ae59f84e715ae159f26c001469adb709 Refactored zip_and_move_file()
eb9998ba7cc1ffb18f2cbfd0c3f65f74ee86657d Cleanup
5d2a8d06b356378de590b20a1d619950d86c0fc9 Convert to function with input arguments
7ab56392eed3d2789e9ab046b301d93381905ebc Added printouts
b395dd28d215490cb26f617cd25591687c16fd24 Cleanup, added docstrings
44936815a372514e5b587070614feaf07d6219b4 Create directory inside the zip_and_move_nifti() function
96bb57438fe0f1519e12c67ace7af74fbccd2aff Put back .gz extension on output file name
be206ec4d5dcbe11949d2414a734b3b34224bb64 Fixed duplicated 'sub-' prefix, removed .gz
a332bb5c40e905de181cd3b3ab256e434f2580c3 Added printout for ignored file
85202a6f64775c18845c8757033f1fc7090b1671 Added docstrings, added output path
ea22e2539ec86bcde6dee6ea18b4214b817861f5 Pushed first prototype that parses subject directory
0d348fc71c7fca496d8d0bd1756bf85bb3f6eee4 Initial commit

Thank you! I'll let you know how it works

From: Julien Cohen-Adad @.>
Sent: April 8, 2024 09:28
To: sct-pipeline/spine-park @.>
Cc: Lydia Chougar, Dr @.>; Mention @.>
Subject: Re: [sct-pipeline/spine-park] First pass and correct T2w segmentations (Issue #15)
The number refers to a git SHA that points to a state of the repository. Here is the link to that version of the repository, that includes the file: https://github.com/sct-pipeline/spine-park/tree/78129ca6e3fe90f39b20af565a5e1918d2f6754e
But what you should do instead of manually downloading the file from this repository, is go to your local repository and run:
git pull
Then you can make sure you are running the proper version by running:
git log --pretty=oneline
which indicates the version (top item in the list):
julien-macbook:~/code/spine-park $ git log --pretty=oneline
78129ca6e3fe90f39b20af565a5e1918d2f6754e (HEAD -> main, origin/main, origin/HEAD) Added .gitignore  <--- THIS ONE
9bfb14077c49237b83535a326357f81d7b9a57de Sort DWI chunks from top to bottom (#18)
8e541e9064c9be264a85f867ed93371df80c506b (jca/17-manual-corr) Create analysis script (#8)
ca8104e9f430e1f7e47605aabae21c1c86790308 Update README.md
2c078e6887ad5a3be9eff721e405269e45d728bb Added doc to convert to BIDS
183e7b90ae59f84e715ae159f26c001469adb709 Refactored zip_and_move_file()
eb9998ba7cc1ffb18f2cbfd0c3f65f74ee86657d Cleanup
5d2a8d06b356378de590b20a1d619950d86c0fc9 Convert to function with input arguments
7ab56392eed3d2789e9ab046b301d93381905ebc Added printouts
b395dd28d215490cb26f617cd25591687c16fd24 Cleanup, added docstrings
44936815a372514e5b587070614feaf07d6219b4 Create directory inside the zip_and_move_nifti() function
96bb57438fe0f1519e12c67ace7af74fbccd2aff Put back .gz extension on output file name
be206ec4d5dcbe11949d2414a734b3b34224bb64 Fixed duplicated 'sub-' prefix, removed .gz
a332bb5c40e905de181cd3b3ab256e434f2580c3 Added printout for ignored file
85202a6f64775c18845c8757033f1fc7090b1671 Added docstrings, added output path
ea22e2539ec86bcde6dee6ea18b4214b817861f5 Pushed first prototype that parses subject directory
0d348fc71c7fca496d8d0bd1756bf85bb3f6eee4 Initial commit
—
Reply to this email directly, view it on GitHubhttps://github.com/sct-pipeline/spine-park/issues/15#issuecomment-2042761098, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BFCFJYUFI7HJXJZC6MSI7D3Y4KLQDAVCNFSM6AAAAABEWB5SPCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBSG43DCMBZHA.
You are receiving this because you were mentioned.Message ID: @.***>

The processing time is quite longer (~2h30 for 15 subjects). I split the subjects into groups of 15 to make it smoother.

Sorry about that. We'll work on a fix which hopefully will solve the underlying issue

When I re-run the analysis, all the segmentation files are overwritten. I'm not sure if the "segment_if_does_not_exist "${file_t2}" "t2"" is working properly

It should not be the case, as per:
https://github.com/sct-pipeline/spine-park/blob/8e541e9064c9be264a85f867ed93371df80c506b/batch_processing.sh#L78-L79
Can you please give an example of the full path where the segmentation that is being overwritten is located originally? I suspect that you did not locate the manual segmentation in the right folder (it works on my end https://github.com/sct-pipeline/spine-park/issues/17)

At the end of the processing, the error.log file indicates that the "_T2_seg.nii.gz" and "mtr.nii.gz" don't exit. I guess the path to the "anat" folder is missing

Good catch! I've opened an issue: https://github.com/sct-pipeline/spine-park/issues/19

The diffusion maps (FA, MD,...) are only generated for one DWI chunk.

Ah! This is because the name for DTI metrics is generic (eg: dti_FA.nii.gz), so it gets overwritten. My bad. I've opened an issue, it is easy to fix that. https://github.com/sct-pipeline/spine-park/issues/20

Regarding vertebral labeling before registration to template, should we call the SPINEPS function from the script?

Yes, we should. Issue opened https://github.com/sct-pipeline/spine-park/issues/21

err.batch_processing_sub-LD214.log
err.batch_processing_sub-GB300.log

Issue opened on SCT: https://github.com/spinalcordtoolbox/spinalcordtoolbox/issues/4431
Can you please share these subjects with me?

err.batch_processing_sub-ER240.log

The log file says "could the file be damaged?". I'm wondering if this isn't a disk issue? You mentioned you had such issues.

err.batch_processing_sub-GE200.log

Possibly caused by https://github.com/sct-pipeline/spine-park/issues/22. I'll fix it.

I answered to the segmentation files being overwritten issue in https://github.com/sct-pipeline/spine-park/issues/17)
Sending you the subject images by email.
Thanks!

err.batch_processing_sub-LD214.log
err.batch_processing_sub-GB300.log

Issue opened on SCT: spinalcordtoolbox/spinalcordtoolbox#4431

Note: These two subjects that @jcohenadad has highlighted have different errors:

LD214: OSError: [Errno 5] Input/output error

GB300: itk::ERROR: [...] failed to read file

export02/data/lydiac/spinalcordtoolbox/bin/isct_ComposeMultiTransform 2 warp2d_00390Warp.nii.gz -R dest_Z0039.nii warp2d_null0Warp.nii.gz warp2d_00390GenericAffine.mat # in /tmp/sct_2024-04-10_08-42-18_register-slicewise_nx0liske
Exception occurred.
 output_image_filename: warp2d_00390Warp.nii.gz
reference_image_filename: dest_Z0039.nii
[0/2]: FIELD: warp2d_null0Warp.nii.gz
[1/2]: AFFINE: warp2d_00390GenericAffine.mat
terminate called after throwing an instance of 'itk::ExceptionObject'
what():  /__w/build_ANTs/build_ANTs/antsbin/ITKv5/Modules/IO/TransformBase/src/itkTransformFileReader.cxx:144:
itk::ERROR: TransformFileReaderTemplate(0x22dfc60): Transform IO: MatlabTransformIOTemplate
failed to read file: warp2d_00390GenericAffine.mat

Given that the GB300 error occurred A) inside isct_ComposeMultiTransform and B) while trying to read the file (rather than write files), I'm wondering if this is indeed another disk issue, as @jcohenadad suspected.

Then, I'll rerun the processing in a folder with more space. Though I had the error "no storage space left" in some other subjects, which was more straightforward.

@Kaonashi22 Do you know if any of the filesystems you're working with (e.g. /export02/data/lydiac/, /dagher/dagher11/lydia11/) are "NFS mounts" specifically? You can check this by running:
mount | grep nfs

If so, this might be the key to the QC locking issues: https://github.com/spinalcordtoolbox/spinalcordtoolbox/issues/4423#issuecomment-2048258460

The command returns this:
nfsd on /proc/fs/nfsd type nfsd (rw,relatime)
tubal:/home on /home/bic type nfs4 (rw,nodev,relatime,vers=4.2,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=<IP_ADDRESS>,local_lock=none,addr=<IP_ADDRESS>)
nfs.isi.bic.mni.mcgill.ca:/dagher12 on /dagher/dagher12 type nfs4 (rw,nodev,relatime,vers=4.0,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=<IP_ADDRESS>,local_lock=none,addr=<IP_ADDRESS>)
nfs.isi.bic.mni.mcgill.ca:/dagher11 on /dagher/dagher11 type nfs4 (rw,nodev,relatime,vers=4.0,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=<IP_ADDRESS>,local_lock=none,addr=<IP_ADDRESS>)

From: Joshua Newton @.>
Sent: April 10, 2024 15:16
To: sct-pipeline/spine-park @.>
Cc: Lydia Chougar, Dr @.>; Mention @.>
Subject: Re: [sct-pipeline/spine-park] First pass and correct T2w segmentations (Issue #15)
@Kaonashi22https://github.com/Kaonashi22 Do you know if any of the filesystems you're working with (e.g. /export02/data/lydiac/, /dagher/dagher11/lydia11/) are "NFS mountshttps://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/storage_administration_guide/ch-nfs" specifically? You can check this by running:
mount | grep nfs
If so, this might be the key to the QC locking issues: spinalcordtoolbox/spinalcordtoolbox#4423 (comment)https://github.com/spinalcordtoolbox/spinalcordtoolbox/issues/4423#issuecomment-2048258460
—
Reply to this email directly, view it on GitHubhttps://github.com/sct-pipeline/spine-park/issues/15#issuecomment-2048271215, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BFCFJYSJ2CAOXHEZ6DFPQHLY4WF2BAVCNFSM6AAAAABEWB5SPCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBYGI3TCMRRGU.
You are receiving this because you were mentioned.Message ID: @.***>

Aha! I think that explains it then! (/dagher/dagher11 -- where the data lives and where we perform locking for the QC index file -- is an NFS mount.)
I'll take a look at trying portalocker's proposed fix for handling NFS mounted drives. :)

Sounds good, thanks!

From: Joshua Newton @.>
Sent: April 10, 2024 19:58
To: sct-pipeline/spine-park @.>
Cc: Lydia Chougar, Dr @.>; Mention @.>
Subject: Re: [sct-pipeline/spine-park] First pass and correct T2w segmentations (Issue #15)
Aha! I think that explains it then! (/dagher/dagher11 -- where the data lives and where we perform locking for the QC index file -- is an NFS mount.)
I'll take a look at trying portalocker's proposed fix for handling NFS mounted drives. :)
—
Reply to this email directly, view it on GitHubhttps://github.com/sct-pipeline/spine-park/issues/15#issuecomment-2048619216, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BFCFJYWXT5GF3LPMRT6ACRTY4XG2DAVCNFSM6AAAAABEWB5SPCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBYGYYTSMRRGY.
You are receiving this because you were mentioned.Message ID: @.***>

All 4 subjects ran without error, so I suspect the issue was related to disk space
Terminal output
Processing 4 subjects in parallel. (Worker processes used: 4).
Started at 11h44m21s: sub-ER240. See log file /Users/julien/temp/Lydia/results_20240411_114414/log/batch_processing_sub-ER240.log
Started at 11h44m21s: sub-GB300. See log file /Users/julien/temp/Lydia/results_20240411_114414/log/batch_processing_sub-GB300.log
Started at 11h44m21s: sub-GE200. See log file /Users/julien/temp/Lydia/results_20240411_114414/log/batch_processing_sub-GE200.log
Started at 11h44m21s: sub-LD214. See log file /Users/julien/temp/Lydia/results_20240411_114414/log/batch_processing_sub-LD214.log

Hooray! your batch completed successfully :-)

Started: 2024-04-11 11h44m21s | Ended: 12h57m38s | Duration: 01h13m16s

Thanks for this trial.
I'll make sure I have enough storage space for the next run.

From: Julien Cohen-Adad @.>
Sent: April 11, 2024 13:41
To: sct-pipeline/spine-park @.>
Cc: Lydia Chougar, Dr @.>; Mention @.>
Subject: Re: [sct-pipeline/spine-park] First pass and correct T2w segmentations (Issue #15)
All 4 subjects ran without error, so I suspect the issue was related to disk space
Terminal output
Processing 4 subjects in parallel. (Worker processes used: 4).
Started at 11h44m21s: sub-ER240. See log file /Users/julien/temp/Lydia/results_20240411_114414/log/batch_processing_sub-ER240.log
Started at 11h44m21s: sub-GB300. See log file /Users/julien/temp/Lydia/results_20240411_114414/log/batch_processing_sub-GB300.log
Started at 11h44m21s: sub-GE200. See log file /Users/julien/temp/Lydia/results_20240411_114414/log/batch_processing_sub-GE200.log
Started at 11h44m21s: sub-LD214. See log file /Users/julien/temp/Lydia/results_20240411_114414/log/batch_processing_sub-LD214.log
Hooray! your batch completed successfully :-)
Started: 2024-04-11 11h44m21s | Ended: 12h57m38s | Duration: 01h13m16s
—
Reply to this email directly, view it on GitHubhttps://github.com/sct-pipeline/spine-park/issues/15#issuecomment-2050190134, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BFCFJYV43FH6NOYRT2GKU3LY43DODAVCNFSM6AAAAABEWB5SPCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJQGE4TAMJTGQ.
You are receiving this because you were mentioned.Message ID: @.***>


Uncaught Exception: Attempting to call a function in a renderer window that has been closed or released. Function provid

An uncaught exception was reported.
Info:
OS: win32
Arch: ia32
GPMDP Version: 3.0.1
Time: Tue Mar 15 2016 09:31:05 GMT 0100 (Paris, Madrid)
Error:
Attempting to call a function in a renderer window that has been closed or released. Function provid
Error: Attempting to call a function in a renderer window that has been closed or released. Function provided here: Object.<anonymous> (C:\Users\***\AppData\Local\GPMDP_3\app-3.0.1\resources\app\build\inject\GPMInject\interface\mouseButtonNavigation.js:8:8.
    at BrowserWindow.callIntoRenderer (C:\Users\***\AppData\Local\GPMDP_3\app-3.0.1\resources\atom.asar\browser\lib\rpc-server.js:201:19)
    at emitTwo (events.js:92:20)
    at BrowserWindow.emit (events.js:172:7)

This issue was created automatically inside the uncaughtException handler

Closing this automatically reported issue as a duplicate of #470


git bash for windows in terminal does not break lines correctly

Environment
Platform ServicePack Version      VersionString
-------- ----------- -------      -------------
 Win32NT             10.0.18363.0 Microsoft Windows NT 10.0.18363.0

Windows Terminal Preview
Version: 1.7.572.0

Setting for git bash:
            {
                "name": "git bash",
                "commandline": "%PROGRAMFILES%/git/usr/bin/bash.exe -i -l",
                "icon": "%PROGRAMFILES%/Git/mingw64/share/git/git-for-windows.ico",
                "startingDirectory" : "%USERPROFILE%"
            },

Steps to reproduce

Open a terminal window with git for windows bash
Do a rebase operation

Expected behavior
Line breaks start at the beginning of the next line.
Actual behavior
git rod is an alias for git rebase --autostash -i origin/develop
$git rod
error: could not apply 6b04f92e... DuplicateInput now supports generating code for arrays for the VfcTypeImplementation
                                                                                Resolve all conflicts manually, mark them as resolved with
                                                                                "git add/rm <conflicted_files>", then run "git rebase --continue".
                                                                                You can instead skip this commit: run "git rebase --skip".
                                           To abort and get back to the state before "git rebase", run "git rebase --abort".
                                                                                Recorded preimage for 'src/App/Changelog.md'
                                                                                Could not apply 6b04f92e... DuplicateInput now supports generating code for arrays for the VfcTypeImplementation
                      Auto-merging src/App/Tests/Silt.Plugin.SimpleTransformersGeneratorTests/GeneratorTests/DuplicateInputTransformerTests.cs
                                                                                Auto-merging src/App/Plugins/Silt.Plugin.SimpleTransformersGenerator/GeneratorStrategy/DuplicateInputTransformer.cs
                                                                                Auto-merging src/App/Changelog.md
                     CONFLICT (content): Merge conflict in src/App/Changelog.md

It seems that only the console without \r is not returning the operator to the beginnig of the line.
Not sure if there is a setting I need to configure for this to work.

@DHowett can you sanity check what's going on here? I forget how mingw works some times. We've had this one and #9312 in the course of a single week, and I don't think we changed anything about our console mode handling.

Forgot to mention I saw this on preview. I have not checked on the "official" version yet. I have added the version number to the post above.

I also see this in the official version
Windows Terminal
Version: 1.6.10571.0

$git rebase -i develop
Successfully rebased and updated refs/heads/testing.
                                                    myPc /c/data/sw testing

Today this led to a buffer overrun exception:

Well that's super weird. I don't think that's necessarily caused by the same thing that's causing the \r to get dropped here. We've seen that before in #7763, but that had more to do with resizing the buffer with DBCS characters in it, and  presumably that's not happening in this case.
If that happens again, can you use task manager to capture a dump file of Terminal when that dialog pops up?

I see this behavior with every git commit I do. Other git subcommands seem to work for me, though I haven't used rebase recently. Never ran into the buffer overrun, mentioned above.
FYI:
Windows Terminal Preview Version: 1.9.1523.0
git version 2.31.1.windows.1

Does stty onlcr help?

Regarding the error message:

The system detected an overrun of a stack-based buffer in the application. This overrun could potentially allow a malicious user to gain control of this application.

As described in STATUS_STACK_BUFFER_OVERRUN doesn’t mean that there was a stack buffer overrun, the same exception code is nowadays used for other purposes too. I don't know whether Windows nowadays bothers to check the fast-fail code before it pops up the message box with this wording.

To add what we know from the other issue #10152, can you check your TERM environment variable when you experience the formatting issue or even the crash? In my situation, it depends on what value it has. Apparently, Git Bash defaults to xterm, and I personally hit these issues with the value msys when using PowerShell.
Terminal definitely behaves different here from other console emulators like the default conhost, mintty or ConEmu.

In my case in Windows Terminal Preview v1.11.2421.0:
$ echo $TERM
xterm-256color

When I run "Git Bash" from the Start Menu (which uses mintty):
$ echo $TERM
xterm

Both are xterm-256color for me.

How about the stty command?
On Linux, Git would output just a LF, the Linux tty driver would convert that to CR LF, and the terminal would move the cursor to the start of the line on CR and then to the next line on LF.  This behaviour of the terminal driver can be enabled or disabled with the ONLCR flag in the tcsetattr function, or with the stty program.
On Windows, the console host does not convert LF to CR LF, so I assume this is instead done by some emulation layer in the Git for Windows package. If the emulated ONLCR flag has become turned off (e.g. because an editor such as vi turned it off and then was killed before it was able to turn the flag back on), that could cause the symptom described here. In this situation, stty onlcr or stty sane would turn the flag back on and make the output look OK again.

FYI, on Windows this behaviour is controlled by the DISABLE_NEWLINE_AUTO_RETURN output mode. So maybe someone is setting that mode and not restoring it on exit. If that's the problem, it could potentially be fixed by #4954.

I was mislead by https://github.com/MicrosoftDocs/Console-Docs/issues/132.
Anyway, if I run stty -onlcr in Git Bash on Windows Terminal Preview 1.11.2421.0, and then run git --help | cat, it produces the ugly output. If I then start cmd in the same pane and run type to display a file that uses only LFs as line terminators, each line starts at the left margin. After that, if I exit from cmd to Git Bash and run git --help | cat again, the output is still ugly. This makes me believe the ONLCR flag is being emulated by MSYS (which is used by stty and cat but not by git) rather than mapped to DISABLE_NEWLINE_AUTO_RETURN.

So maybe someone is setting that mode and not restoring it on exit.

Correct; vim takes a terminal from git with ENABLE_PROCESSED_OUTPUT | ENABLE_WRAP_AT_EOL_OUTPUT adds  DISABLE_NEWLINE_AUTO_RETURN | ENABLE_VIRTUAL_TERMINAL_PROCESSING and returns it dirty (even if it doesn't crash AFAIK).
a Git for Windows snapshot with a workaround to that is available in:
https://github.com/carenas/git/releases/tag/v2.23.0-rc1

FWIW the workaround from git to reset the terminal settings after vi is invoked has been merged and is released as part of the last Git for Windows prerelease:
https://github.com/git-for-windows/git/releases/tag/v2.34.0-rc0.windows.1
This should avoid the most egregious cases until a final solution to keep them consistent is found

Can some help, gitbash terminal break in middle when i clear the old log then the new log start in the bottom.

FYI, I made a fatal mistake on the way this workaround was implemented originally and therefore the code had to be reverted for 2.34.1, but luckily the Git for Windows maintainer picked it up and might still work when using 2.34.1.windows.1 (or later).
If you are having problems, make sure to NOT have any of the environment variables used to override the editor defined in your environment (ex: VISUAL, EDITOR, GIT_EDITOR, GIT_SEQUENCE_EDITOR) AND NOT have a core.editor setting, OR that they are set exactly to 'vi' or 'vim' ('vim.exe' or '/usr/bin/vi' will still fail to activate the workaround).

I am experiencing something similar with WSL 2.0 and Ubuntu in the Windows Terminal, with git version 2.25.1 - yet that shouldn't be git for windows; but just plain old git:

It happens after having run an interactive rebase (which I assume is also when vim is opened.). I've set the editor to vim, (as per @carenas suggestion) rather than the full vim-path, yet it didn't do anything better or worse.
Is this related to the problem? and do you brilliant people know of any quick fix?

Is this related to the problem? and do you brilliant people know of any quick fix?

the mentioned workaround only applies to git for windows (requires code in git that is not part of the regular codebase yet).
it definitely looks like the same problem, and as documented in #10152 another workaround that doesn't require code changes is to set TERM to something else than xterm*; ex:
> export TERM=vt100

I never tested vt100, @carenas - i did now.

vt100 didn't do much more than removing my beautiful colors. Reset is currently my preferred trick. but might just postfix the reset command to my alias.

@carenas You know if this was ever fixed upstream? Or, if it wasn't, is there an upstream issue we could point this thread at?

You know if this was ever fixed upstream?

AFAIK the "fix" implemented in git for windows[1] is only a workaround to the effects of this bug.
[1] https://github.com/git-for-windows/git/commit/db157fb594b23192d438b0aa1002bf81388c643e

I'm experiencing a similar situation where Git Bash for Windows messed up line wrapping (and consequently doesn't break lines properly). To replicate:

Open Git Bash session in Windows Terminal
Just press up-arrow to load a command from history
Move the cursor to the start of the line of the command that appears, and that becomes rather wonky (especially for long commands that should be line broken).

See this image to illustrate:

Hi all! There's been a lot of changes in this space recently, and we're hoping it gets better with 1.18 which is slated to come out in a few months.

FWIW, in my case what triggers the issue is this line...
export MSYS=winsymlinks:nativestrict # Allow symlink creation on Windows perpetually
...that I added at some point to my .bashrc file. IOW, I've just discovered that if I comment that line and relaunch the terminal I get the expected output:

Whereas as soon I uncomment it and relaunch the terminal, I start getting this again:

Curious thing is for some reason the behavior only changes after the second interaction once activated/deactivated the culprit line (no matter if I relaunch or reload), as can be seen in the first capture. Well, I hope it somehow helps, or give some clue at least, on figuring out what could be going on cause it's certainly annoying... and what if (as happen to be the case) you need such or other similar settings to be active? 🙄 Cheers.

git version 2.45.1.windows.1
GNU bash, version 5.2.26(1)-release (x86_64-pc-msys)

If toggling an msys feature on and off changes the behaviour of msys, it may be worth filing a bug on msys 🙂


client: Fix incorrect handling of context errors

If a client request fails due to a context error (either a timeout or
cancellation), the client code is supposed to return that error
directly.  However, before this patch, Client was returning
ErrClusterUnavailable due to a big detecting context errors.  This
patch fixes the bug so that the appropriate context error is returned
instead.

What is Client was returning ErrClusterUnavailable due to a big detecting context errors?
Can you make your code pass the test?

Hmm actually I'm not sure whether or not this is the right solution, I had somewhat misunderstood the problem I was having.
The issue is if you import "golang.org/x/net/context" instead of importing "github.com/coreos/etcd/Godeps/_workspace/src/golang.org/x/net/context" then the check on line 279 of client.go fails because they're comparing two (technically) different context packages.  This is somewhat surprising to users, but it's not clear to me what the right solution is.
At any rate, I just changed the include in my code and it's working now, so I'll drop this.

@ejj Check  the doc here https://github.com/coreos/etcd/blob/master/client/README.md
We use godep... This is not ideal.