BAMParser Extensibility Enhancements

Created a generic version of the BAMParser class that will allow for
easier extension of the class by developers using the .NET Bio library.
Also, to maintain compatibility with existing code, there is also a
non-generic version of the BAMParser class that inherits from the
generic version of the class with the template parameter set to
SAMAlignedSequence.
If this change is except, I would also like to suggest that we allow the BAMParser class accept a factory method as a constructor parameter. Such a feature would further improve the extensibility of the class by inverting the control of object creation.

Hi KRBeut,
Thanks for submitting a pull request! We'd love to take a look and incorporate any changes.  Just a few quick comments from my end.
Have you seen the BAMParserExtensions in Shared\Extensions\BAMParserExtensions?  These contain a range of utilities to Parse BAM files and I am not sure if this would play well with these changes.  For example, the ParseRange method makes a call to only parse the reads from a Start/End on a given chromosome. However I believe if a user were to create a generic version of BAMParser it would not benefit from these extensions.   I see a problem in that the pull request makes the Parse method virtual, but does not alter any or overwrite any of the extension methods which would allow users to query a subset of all the BAM files.  I think whatever changes we implement should allow all generic classes to use the extension methods (and we also need to implement these so that they return IEnumerable's rather than SequenceAlignmentMaps).
The second issue is it is not clear to me how making this generic helps the end user create special derived classes.  It seems like the greatest asset would be to allow the users to decode the BAM entries in a special way so that they can store certain information in defined fields instead of in dictionaries.  However, this would require overriding the GetAlignedSequence method, which is still kept private here.  Can I ask how this helped you create a special class derived from SAMAlignedSequence? (Hoping to understand the rational a bit more).
Cheers,
N

I have not looked at the BAMParserExtensions yet. I will look at them tomorrow and comment more specifically or just submit additional changes to the pull request as you may already be suggesting.
I think the generic class will help users (at very least me!) create special derived classes. Aside from the pool pattern mentioned above, I originally wanted to show an example of how these changes will improve extensability by re-implementing the PacBioCCSBamReader as a class that inherits from the generic version of BAMParser. However, it appears that the PacBioCCSRead class does not inherit from SAMAlignedRead (even though it is an SAMAlignedRead and probably should inherit from SAMAlignedRead) and the PacBioCCSBamReader class does not inherit from BAMParser (even though it is a BAMParser and probably should inherit from BAMParser).
I would have thought that PacBioCCSBamReader would inherit from BAMParser. However, achieving the same api and functionality with the non-generic BAMReader class may not have been possible or easy. With the generic BAMParser class, this would be very simple - if PacBioCCSRead inherited from SAMAlignedRead as it probably should.

On memory management, I would say be very cautious about trying to outsmart the garbage collector unless you have strong reasons and profiling data to show you can do better.  I have learned this the hard way a few times.
In particular, regarding whether or not it is a good idea to repeatedly create a new SAMAlignedSequence or re-use the same memory, I would keep the following in mind:
1 - GC's are great for short-lived objects - If the object is quickly created and consumed, then it is a short lived object for which garbage collectors can kick-butt performance wise.  Although it depends on the heap manager, memory allocations in C through malloc can be much more expensive than allocations with a garbage collector.  A malloc call can involve a context switch, interaction with a heap manager and all kinds of other stuff.  In C#, memory allocations are typically just a fast pointer increment, and the code to clear the memory is likely as fast or faster than what your C# would be doing.
2 - Memory locality - If you keep alive a single sequence object for parsing, it will eventually get promoted by the garbage collector to an older generation.  This means it will be further away in memory from all the new objects you are creating in generation 0, and that references from generation 0 will be written to the older generations, which can slow down garbage collection.  If everything is in generation 0 and there are no live references to any of those objects, then it is super-fast to reclaim that memory.
Anyway, in general (and particularly on Linux) avoiding memory allocations is good, but I'd encourage you to run some profiling results to verify how much extra performance you are getting for the hassle.  If you are using Visual Studio, the profiling tools there are awesome if you felt like sharing.
Regarding PacBioCCSRead - I actually wanted this to implement ISequence and not inherit from SAMAlignedSequence for this.  Although both are stored in BAM files, CCS reads in BAM files don't have any alignment information associated with them, so it didn't make sense to think of them as "aligned sequences."  Also I didn't want a bloated SAMAlignedSequence object since I would be holding a lot in memory. The PacBioCCSRead class has very few reference types, so takes up less memory and requires less time per class for the GC to track down its referenced objects than if it were derived from SAMAlignedSequence.  Granted, although the parser is inefficient because it wraps the BAM parser, the real problem with this is all the tag data goes from binary to string and then back to binary (and some of the decoded information is just discarded, as many fields are always */0).  However, to properly replace the decoding of the binary information I would have needed to change the decoding function within BAMParser, which given that it was fast enough and easy to write I didn't think was worth the effort.  Though I still think it can be a big performance win for known filetypes, but that isn't being discussed here.
Anyway, all of which is to say I am not opposed to the changes and we're a community project, so whatever helps someone and doesn't hurt anyone else is good to go in my book!  But I just wanted to suggest you look at the extension methods to see if you can incorporate those into the generics, as you say these files are gigantic and I think it would be nice to preserve the random access query abilities across all generic types of BAMParser.
Cheers,
Nigel

Hi evolvedmicrobe,
The memory management guidance from here:
https://msdn.microsoft.com/en-us/library/ms973837.aspx
is the initial reason I wanted to use pools and similar patterns. Specifically, the first piece of advice in the "Too Many Allocations" section.
Here is the to do list for this pull request based on our discussion so far:
1.) work with the BAMParserExtensions to see how they integrate with the suggestions in the current pull request.
2.) Add performance profiling for different implementations of GetNewAlignedSequence
3.) Implement an example of extending the BAMParser class by creating a
public class IonTorrentBAMParser : BAMParser where T : IonTorrentRead, new()
this class will return strongly typed IonTorrentRead objects that have the flow space values parsed out of the tags and ready for strongly-typed consumption.
Does this cover the list of feedback so far? Does anyone else have input they'd like to share?
Can you advise me on how to proceed with this pull request? Basically, I need to know how to add the things listed above to the pull request. Is there a way to directly add more changes to the pull request, or is there some other mechanism I need to use? I guess another option is to accept the current pull request, then submit a second pull request with these changes. Please advise.
-KRBeut

Hi KRBeut,
Apologies for the slow response, we're (like everyone) very busy, but I am really excited about your work to extend the BAM parser to handle new data types.
In any event, all three of those would be great.  Though if the performance profiling is not easy, please do not worry about it at this stage*.  Getting Git and Profiling and Meaningfully-useful-code all right on the first pull request would be difficult, and the third item matters far more than the first two.
Adapting the BAM parser extensions would be good, and having the actual implementation of your specific and novel parser would be really fantastic!  A change that enables future useful work is fine, but a change that not only enables but comes with useful additions is far superior (such as a parser for a new format!). Perhaps maybe at a comment to the source as well to indicate why BAMParser allows you to overide the method and call the generic type?
Regarding git... We have never been version control fascists, but generally the things people aim for are a "clean" history.  This means optimizing for fewer total commits and fewer merge events, with good documentation associated with each "git event".  To learn more about git version control generally, I would recommend reading a quick website or book.  For example, with an email address you can get this nice "Git Succinctly" book that I can verify can be read while riding the bus to work (https://www.syncfusion.com/resources/techportal/ebooks/git).  This should cover most typical conventions.  The central things to know for pull requests are how to branch and rebase.  Borrowing some instructions from a typical workflow -  you should likely branch (rather than use "master") for a new feature and then "squash" repeated commits so that your pull would be one commit on one new branch that was rebased to the current HEAD of the master.  The commands do to this would look a bit like the following:
Add your GitHub fork as a remote:

  git remote add my-remote git@github.com:pb-myusername/foo.git

You can work on a feature branch:

  git checkout -b my-feature origin/master
  git fetch origin
  git rebase origin/master
  ... (Edit files.)
  git fetch origin
  git rebase origin/master
  git push my-remote my-feature

However, as I said, we can clean up the git history before merging, for now only worry about producing useful code, and then just commit on top of your branch and repush to github.  You should not need to generate a new pull request after committing any changes and pushing to the server.
Sorry again for the delay.
Cheers,
Nigel

I also just compared the runtime of your code and the original and found only a 7 second difference out of a total ~ 2 minutes runtime to parse a 1.71 GB BAM file on MacOSX (2m10s vs 2m3s), this is only a ~5% speed difference and so is not worth worrying about.

@evolvedmicrobe
I think it's a bit confusing. Normally, that is reversed; e.g. you have a base class with the name and then have the generic type derive from it. See types like IEnumerable and IEnumerable<T> as an example.
I'd suggest we change the base type here to be something other than BAMParser, although I'm not a huge fan of Base suffixes either, it seems like it might be ok here.. thoughts?

@KRBeut not sure where we left off on this, are you still interested in merging this?

Closing due to inactivity.


LSP Server doesn't play well with polylux with regard to memory usage

Describe the bug
This is a continuation of https://github.com/Myriad-Dreamin/tinymist/issues/419
I'm attaching a document that, when previewing, quickly scales tinymist memory usage from 500MB up to, sometimes, 5GB.
The issue is very easy to reproduce for me: just work a bit with the document. But, anyway, I'm providing precise reproduction steps for one instance:

Configure tinymist to refresh on save (not necessary, but allows for a more controlled procedure)
Create a folder and expand the attached zip inside that folder
Open VSCode on that folder
Open slides.typ
Open the previewer
Memory usage should be around 400-500MB
Uncomment the line // #slide[Foo][Bar] and save
Memory usage ramps up to around 1GB
If you keep commenting/uncommenting that line (and saving) memory keeps increasing, although it's not very linear AFAICS

There is nothing too specific to this line, many other changes produce the same or worse effects. Also, refreshing on type accelerates the ramp up.
I've provided more information in #419 from this comment downward.
Archive.zip
Package/Software version:
VSCode version(Help -> About):
Version: 1.92.0 (Universal)
Commit: b1c0a14de1414fcdaa400695b4db1c0799bc3124
Date: 2024-07-31T23:26:45.634Z (1 wk ago)
Electron: 30.1.2
ElectronBuildId: 9870757
Chromium: 124.0.6367.243
Node.js: 20.14.0
V8: <IP_ADDRESS>-electron.0
OS: Darwin arm64 23.4.0

tinymist extension version: v0.11.0. Get it by tinymist --version in terminal.
tinymist
Build Timestamp:     2024-08-04T19:30:39.635429000Z
Build Git Describe:  v0.11.18
Commit SHA:          fb0a300bc2ef8cfb74a62100441fa4a3f648f4b2
Commit Date:         None
Commit Branch:       None
Cargo Target Triple: aarch64-apple-darwin
Typst Version:       0.11.1

Logs:
tinymist server log(Output Panel -> tinymist):
[2024-08-08T16:14:53Z INFO  tinymist] starting LSP server: LspArgs {
        mirror: MirrorArgs {
            mirror: "",
            replay: "",
        },
        font: CompileFontArgs {
            font_paths: [],
            ignore_system_fonts: false,
        },
    }
[2024-08-08T16:14:53Z INFO  sync_lsp] handling initialize - (0) at Instant { tv_sec: 1694186, tv_nsec: 498865000 }
[2024-08-08T16:14:53Z INFO  tinymist::init] preferred theme: Some("light") {"compileStatus": String("enable"), "exportPdf": String("never"), "formatterMode": String("disable"), "formatterPrintWidth": Number(120), "onEnterEvent": Bool(true), "outputPath": String(""), "preferredTheme": String("light"), "preview": Object {"cursorIndicator": Bool(false), "fontPaths": Array [], "invertColors": Null, "partialRendering": Bool(true), "pinPreviewFile": Bool(false), "refresh": String("onSave"), "scrollSync": String("onSelectionChangeByMouse"), "showInActivityBar": Bool(false), "sysInputs": Object {}, "systemFonts": Bool(true)}, "previewFeature": String("enable"), "semanticTokens": String("enable"), "systemFonts": Bool(true), "trace": Object {"server": String("off")}, "typstExtraArgs": Array []}
[2024-08-08T16:14:53Z INFO  tinymist::server] LanguageState: initialized with config Config { compile: CompileConfig { roots: ["/Users/carlos/Documents/Repos/ucema-pood"], output_path: PathPattern(""), export_pdf: Never, root_path: None, font_opts: CompileFontArgs { font_paths: [], ignore_system_fonts: false }, system_fonts: Some(true), font_paths: [], fonts: OnceCell(Uninit), notify_status: true, periscope_args: None, typst_extra_args: Some(CompileExtraOpts { root_dir: None, entry: None, inputs: {}, font: CompileFontArgs { font_paths: [], ignore_system_fonts: false }, creation_timestamp: None }), preferred_theme: Some("light"), has_default_entry_path: false, lsp_inputs: {"x-preview": "{\"version\":1,\"theme\":\"light\"}"} }, semantic_tokens: Enable, formatter: Disable, formatter_print_width: 120 }
[2024-08-08T16:14:53Z INFO  tinymist::server] LanguageState: initialized with const_config ConstConfig { position_encoding: Utf16, cfg_change_registration: true, tokens_dynamic_registration: true, tokens_overlapping_token_support: false, tokens_multiline_token_support: false, doc_line_folding_only: true, doc_fmt_dynamic_registration: true }
[2024-08-08T16:14:53Z INFO  tinymist::actor] TypstActor: creating server for primary, entry: Workspace { root: "/Users/carlos/Documents/Repos/ucema-pood", main: None }, inputs: {"x-preview": "{\"version\":1,\"theme\":\"light\"}"}
[2024-08-08T16:14:53Z INFO  tinymist::init] creating SharedFontResolver with CompileFontArgs { font_paths: [], ignore_system_fonts: false }
[2024-08-08T16:14:53Z INFO  sync_lsp] handled  initialize - (0) in 45.11ms
[2024-08-08T16:14:53Z INFO  sync_lsp] notifying initialized - at Instant { tv_sec: 1694186, tv_nsec: 545306666 }
[2024-08-08T16:14:53Z INFO  tinymist::server] server initialized
[2024-08-08T16:14:53Z INFO  sync_lsp] notifing initialized succeeded in 3.65ms
[2024-08-08T16:14:53Z INFO  sync_lsp] handling workspace/executeCommand - (1) at Instant { tv_sec: 1694186, tv_nsec: 548954083 }
[2024-08-08T16:14:53Z INFO  sync_lsp] handled  workspace/executeCommand - (1) in 17.08ms
[2024-08-08T16:14:53Z INFO  sync_lsp] handling workspace/executeCommand - (2) at Instant { tv_sec: 1694186, tv_nsec: 566056083 }
[2024-08-08T16:14:53Z INFO  tinymist::actor::typ_client] the entry file of TypstActor(primary) is changing to Workspace { root: "/Users/carlos/Documents/Repos/ucema-pood", main: Some(/Clase 2 - Intro Proba/slides.typ) }
[2024-08-08T16:14:53Z INFO  tinymist::tool::preview] PreviewTask(4ismcx): arguments: PreviewCliArgs {
        preview: PreviewArgs {
            data_plane_host: "<IP_ADDRESS>:0",
            control_plane_host: "",
            enable_partial_rendering: true,
            invert_colors: "never",
            task_id: "4ismcx",
            refresh_style: OnSave,
        },
        compile: CompileOnceArgs {
            input: Some(
                "/Users/carlos/Documents/Repos/ucema-pood/Clase 2 - Intro Proba/slides.typ",
            ),
            root: None,
            inputs: [],
            font: CompileFontArgs {
                font_paths: [],
                ignore_system_fonts: false,
            },
            creation_timestamp: None,
        },
        preview_mode: Document,
        static_file_host: "<IP_ADDRESS>:0",
        not_as_primary: false,
        dont_open_in_browser: false,
    }
[2024-08-08T16:14:53Z INFO  sync_lsp] notifying textDocument/didOpen - at Instant { tv_sec: 1694186, tv_nsec: 567480541 }
[2024-08-08T16:14:53Z INFO  tinymist::server] did open Url { scheme: "file", cannot_be_a_base: false, username: "", password: None, host: None, port: None, path: "/Users/carlos/Documents/Repos/ucema-pood/Clase%202%20-%20Intro%20Proba/slides.typ", query: None, fragment: None }
[2024-08-08T16:14:53Z INFO  typst_preview] Previewer: typst actor spawned
[2024-08-08T16:14:53Z INFO  typst_preview] Data plane server listening on: <IP_ADDRESS>:54008
[2024-08-08T16:14:53Z DEBUG typst_preview::actor::typst] TypstActor: waiting for message
[2024-08-08T16:14:54Z INFO  tinymist::tool::preview] PreviewTask(4ismcx): static file server listening on: <IP_ADDRESS>:54009
[2024-08-08T16:14:54Z INFO  sync_lsp] handled  workspace/executeCommand - (2) in 51.74ms
[2024-08-08T16:14:54Z INFO  tinymist::server] create source: "/Users/carlos/Documents/Repos/ucema-pood/Clase 2 - Intro Proba/slides.typ"
[2024-08-08T16:14:54Z INFO  sync_lsp] notifing textDocument/didOpen succeeded in 50.53ms
[2024-08-08T16:14:54Z INFO  sync_lsp] handling workspace/executeCommand - (3) at Instant { tv_sec: 1694186, tv_nsec: 618019583 }
[2024-08-08T16:14:54Z INFO  tinymist::cmd] first manual focusing is coming
[2024-08-08T16:14:54Z INFO  sync_lsp] handled  workspace/executeCommand - (3) in 99.31ms
[2024-08-08T16:14:54Z INFO  sync_lsp] handling workspace/executeCommand - (4) at Instant { tv_sec: 1694186, tv_nsec: 717348250 }
[2024-08-08T16:14:54Z INFO  sync_lsp] handled  workspace/executeCommand - (4) in 2.27ms
[2024-08-08T16:14:54Z INFO  sync_lsp] handling textDocument/codeAction - (5) at Instant { tv_sec: 1694186, tv_nsec: 719883208 }
[2024-08-08T16:14:54Z INFO  sync_lsp] handling textDocument/documentColor - (6) at Instant { tv_sec: 1694186, tv_nsec: 724743333 }
[2024-08-08T16:14:54Z INFO  sync_lsp] notifying $/cancelRequest - at Instant { tv_sec: 1694186, tv_nsec: 839363291 }
[2024-08-08T16:14:54Z WARN  sync_lsp] unhandled notification: $/cancelRequest
[2024-08-08T16:14:54Z INFO  sync_lsp] handling textDocument/codeAction - (7) at Instant { tv_sec: 1694186, tv_nsec: 841114333 }
[2024-08-08T16:14:54Z INFO  sync_lsp] handling textDocument/foldingRange - (8) at Instant { tv_sec: 1694186, tv_nsec: 867296750 }
[2024-08-08T16:14:54Z INFO  tinymist_query::syntax::lexical_hierarchy] lexical hierarchy analysis took 22.474375ms
[2024-08-08T16:14:54Z INFO  sync_lsp] handled  textDocument/foldingRange - (8) in 93.04ms
[2024-08-08T16:14:54Z INFO  sync_lsp] notifying $/setTrace - at Instant { tv_sec: 1694187, tv_nsec: 402875125 }
[2024-08-08T16:14:54Z WARN  sync_lsp] unhandled notification: $/setTrace
[2024-08-08T16:14:54Z INFO  sync_lsp] notifying workspace/didChangeConfiguration - at Instant { tv_sec: 1694187, tv_nsec: 403457833 }
[2024-08-08T16:14:54Z INFO  sync_lsp] notifing workspace/didChangeConfiguration succeeded in 23.17ms
[2024-08-08T16:14:54Z INFO  sync_lsp] notifying $/setTrace - at Instant { tv_sec: 1694187, tv_nsec: 426658666 }
[2024-08-08T16:14:54Z WARN  sync_lsp] unhandled notification: $/setTrace
[2024-08-08T16:14:54Z INFO  sync_lsp] notifying workspace/didChangeConfiguration - at Instant { tv_sec: 1694187, tv_nsec: 430168125 }
[2024-08-08T16:14:54Z INFO  sync_lsp] notifing workspace/didChangeConfiguration succeeded in 65.54µs
[2024-08-08T16:14:54Z INFO  sync_lsp] notifying $/cancelRequest - at Instant { tv_sec: 1694187, tv_nsec: 430241708 }
[2024-08-08T16:14:54Z WARN  sync_lsp] unhandled notification: $/cancelRequest
[2024-08-08T16:14:54Z INFO  sync_lsp] handling textDocument/codeAction - (9) at Instant { tv_sec: 1694187, tv_nsec: 433354500 }
[2024-08-08T16:14:54Z INFO  sync_lsp] notifying $/cancelRequest - at Instant { tv_sec: 1694187, tv_nsec: 434215083 }
[2024-08-08T16:14:54Z WARN  sync_lsp] unhandled notification: $/cancelRequest
[2024-08-08T16:14:54Z INFO  sync_lsp] handling textDocument/codeAction - (10) at Instant { tv_sec: 1694187, tv_nsec: 445348000 }
[2024-08-08T16:14:54Z INFO  sync_lsp] notifying $/setTrace - at Instant { tv_sec: 1694187, tv_nsec: 446161916 }
[2024-08-08T16:14:54Z WARN  sync_lsp] unhandled notification: $/setTrace
[2024-08-08T16:14:54Z INFO  sync_lsp] notifying workspace/didChangeConfiguration - at Instant { tv_sec: 1694187, tv_nsec: 446269166 }
[2024-08-08T16:14:54Z INFO  sync_lsp] notifing workspace/didChangeConfiguration succeeded in 145.04µs
[2024-08-08T16:14:54Z INFO  sync_lsp] notifying $/setTrace - at Instant { tv_sec: 1694187, tv_nsec: 446424375 }
[2024-08-08T16:14:54Z WARN  sync_lsp] unhandled notification: $/setTrace
[2024-08-08T16:14:54Z INFO  sync_lsp] notifying workspace/didChangeConfiguration - at Instant { tv_sec: 1694187, tv_nsec: 446533041 }
[2024-08-08T16:14:54Z INFO  sync_lsp] notifing workspace/didChangeConfiguration succeeded in 23.54µs
[2024-08-08T16:14:54Z INFO  sync_lsp] handling textDocument/semanticTokens/full - (11) at Instant { tv_sec: 1694187, tv_nsec: 446561500 }
[2024-08-08T16:14:54Z INFO  typst_preview] Peer address: <IP_ADDRESS>:54014
[2024-08-08T16:14:54Z INFO  typst_preview] New WebSocket connection: <IP_ADDRESS>:54014
[2024-08-08T16:14:54Z INFO  sync_lsp] handled  textDocument/semanticTokens/full - (11) in 27.62ms
[2024-08-08T16:14:54Z INFO  tinymist::init] preferred theme: Some("light") {"compileStatus": String("enable"), "exportPdf": String("never"), "fontPaths": Null, "formatterMode": String("disable"), "formatterPrintWidth": Number(120), "hoverPeriscope": Null, "outputPath": String(""), "preferredTheme": String("light"), "rootPath": Null, "semanticTokens": String("enable"), "systemFonts": Bool(true), "typstExtraArgs": Array []}
[2024-08-08T16:14:54Z INFO  tinymist::server] new settings applied
[2024-08-08T16:14:54Z INFO  tinymist::init] preferred theme: Some("light") {"compileStatus": String("enable"), "exportPdf": String("never"), "fontPaths": Null, "formatterMode": String("disable"), "formatterPrintWidth": Number(120), "hoverPeriscope": Null, "outputPath": String(""), "preferredTheme": String("light"), "rootPath": Null, "semanticTokens": String("enable"), "systemFonts": Bool(true), "typstExtraArgs": Array []}
[2024-08-08T16:14:54Z INFO  tinymist::server] new settings applied
[2024-08-08T16:14:54Z INFO  tinymist::init] preferred theme: Some("light") {"compileStatus": String("enable"), "exportPdf": String("never"), "fontPaths": Null, "formatterMode": String("disable"), "formatterPrintWidth": Number(120), "hoverPeriscope": Null, "outputPath": String(""), "preferredTheme": String("light"), "rootPath": Null, "semanticTokens": String("enable"), "systemFonts": Bool(true), "typstExtraArgs": Array []}
[2024-08-08T16:14:54Z INFO  tinymist::server] new settings applied
[2024-08-08T16:14:54Z INFO  tinymist::init] preferred theme: Some("light") {"compileStatus": String("enable"), "exportPdf": String("never"), "fontPaths": Null, "formatterMode": String("disable"), "formatterPrintWidth": Number(120), "hoverPeriscope": Null, "outputPath": String(""), "preferredTheme": String("light"), "rootPath": Null, "semanticTokens": String("enable"), "systemFonts": Bool(true), "typstExtraArgs": Array []}
[2024-08-08T16:14:54Z INFO  tinymist::server] new settings applied
[2024-08-08T16:14:54Z DEBUG typst_preview::actor::render] OutlineRenderActor: waiting for message
[2024-08-08T16:14:54Z DEBUG typst_preview::actor::render] RenderActor: waiting for message
[2024-08-08T16:14:54Z DEBUG typst_preview::actor::render] RenderActor: has_full_render: true
[2024-08-08T16:14:54Z INFO  typst_preview::actor::render] RenderActor: document is not ready
[2024-08-08T16:14:54Z DEBUG typst_preview::actor::render] RenderActor: waiting for message
[2024-08-08T16:14:54Z DEBUG typst_preview::actor::render] OutlineRenderActor: received message: RenderFullLatest
[2024-08-08T16:14:54Z INFO  typst_preview::actor::render] OutlineRenderActor: document is not ready
[2024-08-08T16:14:54Z DEBUG typst_preview::actor::render] OutlineRenderActor: waiting for message
[2024-08-08T16:14:54Z INFO  sync_lsp] handling textDocument/codeLens - (12) at Instant { tv_sec: 1694187, tv_nsec: 491468666 }
[2024-08-08T16:14:55Z INFO  sync_lsp] notifying $/setTrace - at Instant { tv_sec: 1694188, tv_nsec: 27277083 }
[2024-08-08T16:14:55Z WARN  sync_lsp] unhandled notification: $/setTrace
[2024-08-08T16:14:55Z INFO  sync_lsp] notifying workspace/didChangeConfiguration - at Instant { tv_sec: 1694188, tv_nsec: 27308083 }
[2024-08-08T16:14:55Z INFO  sync_lsp] notifing workspace/didChangeConfiguration succeeded in 63.25µs
[2024-08-08T16:14:55Z INFO  tinymist::init] preferred theme: Some("light") {"compileStatus": String("enable"), "exportPdf": String("never"), "fontPaths": Null, "formatterMode": String("disable"), "formatterPrintWidth": Number(120), "hoverPeriscope": Null, "outputPath": String(""), "preferredTheme": String("light"), "rootPath": Null, "semanticTokens": String("enable"), "systemFonts": Bool(true), "typstExtraArgs": Array []}
[2024-08-08T16:14:55Z INFO  tinymist::server] new settings applied
[2024-08-08T16:14:56Z INFO  tinymist::actor::editor] received status request(primary) Compiling
[2024-08-08T16:14:56Z INFO  sync_lsp] handled  textDocument/documentColor - (6) in 1.96s
[2024-08-08T16:14:56Z INFO  sync_lsp] handled  textDocument/codeAction - (7) in 1.85s
[2024-08-08T16:14:56Z INFO  sync_lsp] handled  textDocument/codeAction - (10) in 1.24s
[2024-08-08T16:14:56Z INFO  sync_lsp] handled  textDocument/codeAction - (5) in 1.98s
[2024-08-08T16:14:56Z INFO  sync_lsp] handled  textDocument/codeAction - (9) in 1.27s
[2024-08-08T16:14:56Z INFO  sync_lsp] handled  textDocument/codeLens - (12) in 1.23s
[2024-08-08T16:14:56Z INFO  sync_lsp] handling textDocument/codeAction - (13) at Instant { tv_sec: 1694188, tv_nsec: 833884583 }
[2024-08-08T16:14:56Z INFO  sync_lsp] handled  textDocument/codeAction - (13) in 752.75µs
[2024-08-08T16:14:56Z INFO  typst_preview::actor::webview] WebviewActor: received non-text message from websocket: Close(Some(CloseFrame { code: Away, reason: "" }))
[2024-08-08T16:14:56Z INFO  typst_preview::actor::webview] WebviewActor: exiting
[2024-08-08T16:14:56Z INFO  sync_lsp] handling workspace/executeCommand - (14) at Instant { tv_sec: 1694189, tv_nsec: 184149791 }
[2024-08-08T16:14:56Z INFO  tinymist::cmd] file pinned: None
[2024-08-08T16:14:56Z INFO  sync_lsp] handled  workspace/executeCommand - (14) in 23.13µs
[2024-08-08T16:14:56Z INFO  sync_lsp] handling workspace/executeCommand - (15) at Instant { tv_sec: 1694189, tv_nsec: 184180833 }
[2024-08-08T16:14:56Z INFO  tinymist::actor::preview] PreviewTask(4ismcx): killing
[2024-08-08T16:14:56Z INFO  tinymist::actor::preview] PreviewTask(4ismcx): wait for previewer to stop
[2024-08-08T16:14:56Z INFO  tinymist::tool::preview] Static file server joined
[2024-08-08T16:14:56Z INFO  typst_preview] Data plane server shutdown
[2024-08-08T16:14:56Z INFO  typst_preview::actor::editor] EditorActor: received exit message
[2024-08-08T16:14:56Z INFO  typst_preview::actor::editor] EditorActor: editor disconnected
[2024-08-08T16:14:56Z INFO  typst_preview] Control plane client shutdown
[2024-08-08T16:14:56Z INFO  tinymist::tool::preview] PreviewTask(4ismcx): response channel closed
[2024-08-08T16:14:56Z INFO  tinymist::actor::preview] PreviewTask(4ismcx): wait for static server to stop
[2024-08-08T16:14:56Z INFO  tinymist::actor::preview] PreviewTask(4ismcx): killed
[2024-08-08T16:14:56Z INFO  sync_lsp] handled  workspace/executeCommand - (15) in 1.19ms
[2024-08-08T16:14:56Z INFO  sync_lsp] handling textDocument/codeAction - (16) at Instant { tv_sec: 1694189, tv_nsec: 321557291 }
[2024-08-08T16:14:56Z INFO  sync_lsp] handling textDocument/documentSymbol - (17) at Instant { tv_sec: 1694189, tv_nsec: 321637791 }
[2024-08-08T16:14:56Z INFO  sync_lsp] handled  textDocument/codeAction - (16) in 7.98ms
[2024-08-08T16:14:56Z INFO  tinymist_query::syntax::lexical_hierarchy] lexical hierarchy analysis took 9.881667ms
[2024-08-08T16:14:56Z INFO  sync_lsp] handled  textDocument/documentSymbol - (17) in 25.33ms
[2024-08-08T16:14:56Z INFO  sync_lsp] notifying $/setTrace - at Instant { tv_sec: 1694189, tv_nsec: 346983875 }
[2024-08-08T16:14:56Z WARN  sync_lsp] unhandled notification: $/setTrace
[2024-08-08T16:14:56Z INFO  sync_lsp] notifying workspace/didChangeConfiguration - at Instant { tv_sec: 1694189, tv_nsec: 361129000 }
[2024-08-08T16:14:56Z INFO  sync_lsp] notifing workspace/didChangeConfiguration succeeded in 29.38µs
[2024-08-08T16:14:56Z INFO  tinymist::init] preferred theme: Some("light") {"compileStatus": String("enable"), "exportPdf": String("never"), "fontPaths": Null, "formatterMode": String("disable"), "formatterPrintWidth": Number(120), "hoverPeriscope": Null, "outputPath": String(""), "preferredTheme": String("light"), "rootPath": Null, "semanticTokens": String("enable"), "systemFonts": Bool(true), "typstExtraArgs": Array []}
[2024-08-08T16:14:56Z INFO  tinymist::server] new settings applied
[2024-08-08T16:14:58Z INFO  sync_lsp] handling workspace/executeCommand - (18) at Instant { tv_sec: 1694190, tv_nsec: 633002375 }
[2024-08-08T16:14:58Z INFO  tinymist::tool::preview] PreviewTask(lm8a1f): arguments: PreviewCliArgs {
        preview: PreviewArgs {
            data_plane_host: "<IP_ADDRESS>:0",
            control_plane_host: "",
            enable_partial_rendering: true,
            invert_colors: "never",
            task_id: "lm8a1f",
            refresh_style: OnSave,
        },
        compile: CompileOnceArgs {
            input: Some(
                "/Users/carlos/Documents/Repos/ucema-pood/Clase 2 - Intro Proba/slides.typ",
            ),
            root: None,
            inputs: [],
            font: CompileFontArgs {
                font_paths: [],
                ignore_system_fonts: false,
            },
            creation_timestamp: None,
        },
        preview_mode: Document,
        static_file_host: "<IP_ADDRESS>:0",
        not_as_primary: false,
        dont_open_in_browser: false,
    }
[2024-08-08T16:14:58Z INFO  typst_preview] Previewer: typst actor spawned
[2024-08-08T16:14:58Z DEBUG typst_preview::actor::typst] TypstActor: waiting for message
[2024-08-08T16:14:58Z INFO  typst_preview] Data plane server listening on: <IP_ADDRESS>:54044
[2024-08-08T16:14:58Z INFO  tinymist::tool::preview] PreviewTask(lm8a1f): static file server listening on: <IP_ADDRESS>:54045
[2024-08-08T16:14:58Z INFO  sync_lsp] handled  workspace/executeCommand - (18) in 3.12ms
[2024-08-08T16:14:58Z INFO  sync_lsp] handling workspace/executeCommand - (19) at Instant { tv_sec: 1694190, tv_nsec: 688287500 }
[2024-08-08T16:14:58Z INFO  sync_lsp] handled  workspace/executeCommand - (19) in 34.33µs
[2024-08-08T16:14:58Z INFO  typst_preview] Peer address: <IP_ADDRESS>:54046
[2024-08-08T16:14:58Z INFO  typst_preview] New WebSocket connection: <IP_ADDRESS>:54046
[2024-08-08T16:14:58Z DEBUG typst_preview::actor::render] OutlineRenderActor: waiting for message
[2024-08-08T16:14:58Z DEBUG typst_preview::actor::render] RenderActor: waiting for message
[2024-08-08T16:14:58Z DEBUG typst_preview::actor::render] RenderActor: has_full_render: true
[2024-08-08T16:14:58Z INFO  typst_preview::actor::render] RenderActor: document is not ready
[2024-08-08T16:14:58Z DEBUG typst_preview::actor::render] RenderActor: waiting for message
[2024-08-08T16:14:58Z DEBUG typst_preview::actor::render] OutlineRenderActor: received message: RenderFullLatest
[2024-08-08T16:14:58Z INFO  typst_preview::actor::render] OutlineRenderActor: document is not ready
[2024-08-08T16:14:58Z DEBUG typst_preview::actor::render] OutlineRenderActor: waiting for message
[2024-08-08T16:14:59Z INFO  typst_ts_compiler::diag::console] Some(/Clase 2 - Intro Proba/slides.typ): compilation succeeded in 2.989453s
[2024-08-08T16:14:59Z INFO  tinymist::actor::editor] received diagnostics from primary:5: diag(Some(0))
[2024-08-08T16:14:59Z INFO  tinymist::actor::editor] received status request(primary) CompileSuccess
[2024-08-08T16:14:59Z DEBUG typst_preview::actor::render] RenderActor: has_full_render: false
[2024-08-08T16:14:59Z DEBUG typst_preview::actor::render] OutlineRenderActor: received message: RenderIncremental
[2024-08-08T16:14:59Z DEBUG typst_preview::actor::render] OutlineRenderActor: sending outline
[2024-08-08T16:14:59Z DEBUG typst_preview::actor::render] OutlineRenderActor: waiting for message
[2024-08-08T16:14:59Z INFO  tinymist::actor::editor] received status request(primary) Compiling
[2024-08-08T16:14:59Z INFO  tinymist::task::cache] CacheEvictTask: evict cache in 171.972ms
[2024-08-08T16:14:59Z INFO  typst_ts_compiler::diag::console] Some(/Clase 2 - Intro Proba/slides.typ): compilation succeeded in 251.414ms
[2024-08-08T16:14:59Z INFO  tinymist::actor::editor] received diagnostics from primary:6: diag(Some(0))
[2024-08-08T16:14:59Z INFO  tinymist::actor::editor] received status request(primary) CompileSuccess
[2024-08-08T16:14:59Z DEBUG typst_preview::actor::render] OutlineRenderActor: received message: RenderIncremental
[2024-08-08T16:14:59Z DEBUG typst_preview::actor::render] OutlineRenderActor: sending outline
[2024-08-08T16:14:59Z DEBUG typst_preview::actor::render] OutlineRenderActor: waiting for message
[2024-08-08T16:14:59Z DEBUG typst_preview::actor::render] RenderActor: waiting for message
[2024-08-08T16:14:59Z DEBUG typst_preview::actor::render] RenderActor: has_full_render: false
[2024-08-08T16:14:59Z DEBUG typst_preview::actor::render] RenderActor: waiting for message
[2024-08-08T16:14:59Z INFO  tinymist::actor::editor] received status request(primary) Compiling
[2024-08-08T16:14:59Z INFO  tinymist::task::cache] CacheEvictTask: evict cache in 5.644708ms
[2024-08-08T16:14:59Z INFO  typst_ts_compiler::diag::console] Some(/Clase 2 - Intro Proba/slides.typ): compilation succeeded in 221.859ms
[2024-08-08T16:14:59Z INFO  tinymist::actor::editor] received diagnostics from primary:7: diag(Some(0))
[2024-08-08T16:14:59Z INFO  tinymist::actor::editor] received status request(primary) CompileSuccess
[2024-08-08T16:14:59Z DEBUG typst_preview::actor::render] OutlineRenderActor: received message: RenderIncremental
[2024-08-08T16:14:59Z DEBUG typst_preview::actor::render] RenderActor: has_full_render: false
[2024-08-08T16:14:59Z DEBUG typst_preview::actor::render] OutlineRenderActor: sending outline
[2024-08-08T16:14:59Z DEBUG typst_preview::actor::render] OutlineRenderActor: waiting for message
[2024-08-08T16:14:59Z DEBUG typst_preview::actor::render] RenderActor: waiting for message
[2024-08-08T16:14:59Z INFO  tinymist::task::cache] CacheEvictTask: evict cache in 6.830584ms
[2024-08-08T16:15:00Z INFO  sync_lsp] handling workspace/executeCommand - (20) at Instant { tv_sec: 1694193, tv_nsec: 439417625 }
[2024-08-08T16:15:00Z INFO  sync_lsp] handled  workspace/executeCommand - (20) in 144.96µs
[2024-08-08T16:15:00Z INFO  sync_lsp] handling workspace/executeCommand - (21) at Instant { tv_sec: 1694193, tv_nsec: 462308166 }
[2024-08-08T16:15:00Z INFO  sync_lsp] handled  workspace/executeCommand - (21) in 969.67µs
[2024-08-08T16:15:00Z DEBUG typst_preview::actor::editor] EditorActor: received message from editor: SrcToDocJumpRequest { filepath: "/Users/carlos/Documents/Repos/ucema-pood/Clase 2 - Intro Proba/slides.typ", line: 126, character: 16 }
[2024-08-08T16:15:00Z DEBUG typst_preview::actor::typst] TypstActor: processing src2doc: SrcToDocJumpRequest { filepath: "/Users/carlos/Documents/Repos/ucema-pood/Clase 2 - Intro Proba/slides.typ", line: 126, character: 16 }
[2024-08-08T16:15:00Z INFO  tinymist::actor::editor] received status request(primary) Compiling
[2024-08-08T16:15:01Z INFO  sync_lsp] handling textDocument/codeAction - (22) at Instant { tv_sec: 1694193, tv_nsec: 741220125 }
[2024-08-08T16:15:01Z INFO  sync_lsp] handled  textDocument/codeAction - (22) in 380.04µs
[2024-08-08T16:15:01Z INFO  typst_ts_compiler::diag::console] Some(/Clase 2 - Intro Proba/slides.typ): compilation succeeded in 310.612ms
[2024-08-08T16:15:01Z INFO  tinymist::actor::typ_client] TypstActor: already notified for revision 7 <= 7
[2024-08-08T16:15:02Z INFO  sync_lsp] handling textDocument/codeLens - (23) at Instant { tv_sec: 1694194, tv_nsec: 704961125 }
[2024-08-08T16:15:02Z INFO  sync_lsp] handled  textDocument/codeLens - (23) in 281.92µs
[2024-08-08T16:15:02Z INFO  sync_lsp] handling workspace/executeCommand - (24) at Instant { tv_sec: 1694194, tv_nsec: 882566875 }
[2024-08-08T16:15:02Z INFO  sync_lsp] handled  workspace/executeCommand - (24) in 33.92µs

tinymist client log(Help -> Toggle Developer Tools -> Console):
workbench.desktop.main.js:sourcemap:660  INFO Started local extension host with pid 36181.
2workbench.desktop.main.js:sourcemap:97 [Extension Host] determineVscodeTheme 1
workbench.desktop.main.js:sourcemap:97 [Extension Host] vscodeVariables test: Object
workbench.desktop.main.js:sourcemap:97 [Extension Host] validate /Users/carlos/.vscode/extensions/myriad-dreamin.tinymist-0.11.18-darwin-arm64/out/tinymist args Array(1)
workbench.desktop.main.js:sourcemap:97 [Extension Host] deserializeWebviewPanel Object
workbench.desktop.main.js:sourcemap:97 [Extension Host] Preview Command /Users/carlos/Documents/Repos/ucema-pood/Clase 2 - Intro Proba/slides.typ
workbench.desktop.main.js:sourcemap:97 [Extension Host] Launched preview, data plane port:54008, static server port:54009
workbench.desktop.main.js:sourcemap:1509 An iframe which has both allow-scripts and allow-same-origin for its sandbox attribute can escape its sandboxing.
mountTo @ workbench.desktop.main.js:sourcemap:1509
8workbench.desktop.main.js:sourcemap:97 [Extension Host] determineVscodeTheme 1
workbench.desktop.main.js:sourcemap:660  WARN Semantic token with invalid length detected at lineNumber 2, column 1
workbench.desktop.main.js:sourcemap:660  WARN Overlapping semantic tokens detected at lineNumber 107, column 51
2workbench.desktop.main.js:sourcemap:97 [Extension Host] determineVscodeTheme 1
workbench.desktop.main.js:sourcemap:97 [Extension Host] killed preview services
workbench.desktop.main.js:sourcemap:97 [Extension Host] No dispose function found for task 4ismcx
y @ workbench.desktop.main.js:sourcemap:97
2workbench.desktop.main.js:sourcemap:97 [Extension Host] determineVscodeTheme 1
workbench.desktop.main.js:sourcemap:660  INFO [perf] Render performance baseline is 18ms
workbench.desktop.main.js:sourcemap:97 [Extension Host] Preview Command /Users/carlos/Documents/Repos/ucema-pood/Clase 2 - Intro Proba/slides.typ
workbench.desktop.main.js:sourcemap:97 [Extension Host] Launched preview, data plane port:54044, static server port:54045
workbench.desktop.main.js:sourcemap:1509 An iframe which has both allow-scripts and allow-same-origin for its sandbox attribute can escape its sandboxing.
mountTo @ workbench.desktop.main.js:sourcemap:1509
VM12:12 state eyJtb2RlIjoiZG9jIiwiYXNQcmltYXJ5IjpmYWxzZSwidXJpIjoiZmlsZTovLy9Vc2Vycy9jYXJsb3MvRG9jdW1lbnRzL1JlcG9zL3VjZW1hLXBvb2QvQ2xhc2UlMjAyJTIwLSUyMEludHJvJTIwUHJvYmEvc2xpZGVzLnR5cCJ9
VM12:16 vscode_state Object
VM13:5842 plugin initialized, build info: Object
VM13:5728 WebSocket connection opened WebSocket
VM13:5763 batch 2 messages
VM13:5784 recv partial-rendering 22
VM13:5821 Experimental feature: partial rendering enabled
VM13:5784 recv invert-colors 19
VM13:5827 Experimental feature: invert colors strategy taken: never
VM13:5763 batch 2 messages
VM13:5784 recv diff-v1 1181356
VM13:5784 recv diff-v1 1180
VM13:1387 render_in_window with partial rendering enabled window 1.3134025776070106 0 34.148467017782274<PHONE_NUMBER>684869<PHONE_NUMBER>783688 , patch scale 174644
VM13:368 parse 10.10 ms, rerender 17.00 ms, total 27.10 ms
VM13:1830 updateCanvas start
VM13:1862 updateCanvas one miss undefined cMphHzh72sHE7Cticoyk8u0
VM13:1862 updateCanvas one miss undefined c5Ccgscw4bkoJFHfAKCenIY
VM13:1862 updateCanvas one miss undefined cWj8PnyKDbssBCwa+FnNpgw
VM13:1862 updateCanvas one miss undefined c5JO94mfeNU8OZzvQd6Rcks
VM13:1862 updateCanvas one miss undefined cK5B8RVf7NNoMvsGW+A/YjE
VM13:1862 updateCanvas one miss undefined cQTXreka19zImCaygRHeBNY
VM13:1862 updateCanvas one miss undefined cgwyv++uFnJQLkhTLzIBZ7M
VM13:1862 updateCanvas one miss undefined cINgxPy05utEMli1sXfuZ2Q
VM13:1862 updateCanvas one miss undefined cGQ9nbssnHncOOoyyTHftEk
VM13:1862 updateCanvas one miss undefined cHMG1E1wjl+UBABZTtv4vw8
VM13:5763 batch 1 messages
VM13:5784 recv diff-v1 1180
VM13:1387 render_in_window with partial rendering enabled window 1.313572542901716 0 34.152886115444616<PHONE_NUMBER>570983<PHONE_NUMBER>915756 , patch scale 8719
VM13:1666 cancel canvas rendering
VM13:368 parse 0.20 ms, rerender 4.70 ms, total 4.90 ms
VM13:1845 updateCanvas cancelled<PHONE_NUMBER>4768372
VM13:1830 updateCanvas start
VM13:1862 updateCanvas one miss undefined cakK2tU0Y4gUVj14f9HgnkQ
VM13:1862 updateCanvas one miss undefined cCmcIZzNAYAcG8eU+6NDknE
VM13:1862 updateCanvas one miss undefined c/3KGXfp80X8SkWtGHo6y/I
VM13:1862 updateCanvas one miss undefined ci34qkdegmYkZHfmCpdX1NA
VM13:1862 updateCanvas one miss undefined cyyR1WZSG2lozh9C7QfAczo
VM13:1862 updateCanvas one miss undefined c53YEjl7oeTsFTIkCZS/BlM
VM13:1862 updateCanvas one miss undefined clPdoKK7AGDc3pFZVkqFhMQ
VM13:1862 updateCanvas one miss undefined cbRBl2JUOvM4bVnwoIuxcBI
VM13:1862 updateCanvas one miss undefined cbLo3rztPQTAl3fJgtclVsU
VM13:1862 updateCanvas one miss undefined cciA4CTVWVqEZDViwaluCZ8
VM13:1862 updateCanvas one miss undefined ce2hf7D/NOVcmeOqHLr9ANY
VM13:1862 updateCanvas one miss undefined c7b+Pj7dl24k93nP8HTVK9I
VM13:1862 updateCanvas one miss undefined c1oYHrVmDZIwUXAdGxUBoaw
VM13:1862 updateCanvas one miss undefined cpmsXLXUgM3U5jpsX7S8TCA
VM13:1862 updateCanvas one miss undefined ctoQFUcTrJ2IlrK8OVr3GQ8
VM13:1862 updateCanvas one miss undefined cldDfC2jEXzwokDfDUBWGF8
VM13:1862 updateCanvas one miss undefined cK5XIMHJQ50QRVpNz+09cwE
VM13:1862 updateCanvas one miss undefined cHtcpZkMGU3801YaVzOdp84
VM13:1862 updateCanvas one miss undefined cy9DawkfqvugFlY5d6h32Bg
VM13:1862 updateCanvas one miss undefined c6v+zkY7WXyMQ2RuF5K//k0
VM13:1862 updateCanvas one miss undefined cwI78lyWmmoMXFs4J700TmM
VM13:1862 updateCanvas one miss undefined cVe65U7XTi0AGM7r6nzqQz0
VM13:1862 updateCanvas one miss undefined cf8g+eUeSynczjkuvnYIPn4
VM13:1862 updateCanvas one miss undefined chdDbde7oZEQSNAWEAmhq7M
VM13:1862 updateCanvas one miss undefined ctCxzZGl1euk/0RgwQtY3b0
VM13:1862 updateCanvas one miss undefined cTLfWrISPaGwIzC9mjlV8rE
VM13:1862 updateCanvas one miss undefined ceAxNCH8gNMUruPkGC51eIs
VM13:1862 updateCanvas one miss undefined cauW4NEZ7Jbg1wYUWUezYqY
VM13:1862 updateCanvas one miss undefined cuXmrEQ5JXnc1CTkQQ9XvFc
VM13:1862 updateCanvas one miss undefined cBbGQGDdYhvkSfD8pDl/jXc
VM13:1870 updateCanvas done<PHONE_NUMBER>046326
2workbench.desktop.main.js:sourcemap:97 [Extension Host] skip extension-output file extension-output-myriad-dreamin.tinymist-#1-Tinymist Typst Language Server
VM13:1387 render_in_window with partial rendering enabled window 2.319559228650138 0 60.308539944903586<PHONE_NUMBER>713499 1048.4407713498624 , patch scale 132317
VM13:368 parse 0.10 ms, rerender 19.20 ms, total 19.30 ms
VM13:1830 updateCanvas start

I've been investigating this. I'm quite sure the problem is in Polylux, specifically in #let polylux-slide(max-repetitions: 10, body) in logic.typ. Just change the number of iterations first to 1 and then to 100 and you will see the opposite reactions in terms of memory usage. Perhaps it's memoizing a lot of stuff because of the frequent state updates (and maybe you need some cache eviction policy or something like that). And this while I'm not using any dynamic feature at all! Also, from a cursory look, logical-slide.step() seems to be ever increasing.

It may be that when adding/removing one slide, specially one near the top, the logical-slide count increases/decreases for all the following ones (since there is one more/less state update above them), so a large part of the document gets invalidated, and each one of the slides there has to do the ten iterations dance again. This is arguably not desirable, but if possible tinymist should deal with the situation in a more robust fashion (without scaling memory usage to quite a few GB).

Thanks for investigation. Both can do better from the result.
Alternative solution: From view of performance, touying should do much better, though I haven't use it. I only maintained some deadly-simple polylux slides so I didn't go in depth of optimizing lsp for polylux.

Ok, thanks, I'm already looking into touying or just a "hand-made" slideshow template.  So, to recap, the most reliable way to reproduce this is to change max-repetitions to 100 in logic.typ, restart the extension host and boom. You don't even need to open the previewer. I guess having that multiplier there is equivalent to having a large document with page numbers in the footer, the would require almost full renumbering if you added or removed a line from a page near the beginning. One can't blame polylux on keeping the logical-slide count, although the way it then iterates over each slide certainly doesn't seem to help a memoization strategy. Even worse, a long-running, incremental one, that may be keeping lots of stale data from one moment to the next. Moreover, the fact that each slide is wrapped inside a polylux function that is wrapped inside a theme function that is wrapped inside an end-user function, may be hard on the caching strategy if it works at the function call level, but I'm just guessing.

Also be aware that in the development branch they removed the max-iterations logic and are now iterating as required for each slide https://github.com/andreasKroepelin/polylux/blob/main/logic.typ. Although according to the commit history the motivation for the change was not optimization but getting rid of an artificial limit. So please use the same polylux version than me in order to reproduce the issue, which is the current stable one.

It would be great to track this down to a minimal document without external dependencies. I didn't have the time yet but the example of polylux with regard to the number of iterations may be a good lead.

Another take is that we may want to know who's fault it is in this case. If it already takes a lot of RAM when using typst watch, tinymist cannot do much to prevent this.

Still working on a reliable way to reproduce this - on my laptop which has the exact same software config as my desktop, I can't reliably get a Touying presentation to trigger the RAM usage issue. However, I was able to use another document (~17 pages) to use ~4 GB of RAM using Tinymist, whereas typst watch uses only ~108 MB.
Using /usr/bin/time -v typst watch <document>.typ to report max resident RAM, I get:
        Command being timed: "typst watch <document>.typ"
        User time (seconds): 0.86
        System time (seconds): 0.22
        Percent of CPU this job got: 2%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:42.94
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 108156
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 26842
        Voluntary context switches: 254
        Involuntary context switches: 19
        Swaps: 0
        File system inputs: 0
        File system outputs: 3736
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

So typst watch uses ~108 MB of RAM, whereas Tinymist uses ~4 GB.

yup, I'm here with 10.8 GB RAM usage of tinymist..

I have checked it again and find nothing new. I performed following steps:

clear caches, memory goes to 400MB ~ 500MB.
negate some editions repeatedly, memory keeps in 600MB ~ 1000MB.
clear caches, memory goes to 400MB ~ 500MB.

https://github.com/user-attachments/assets/783943a3-5e78-4982-9acb-2fbb4c165dff
The only difference between typst-cli and tinymist I can remeber soonly is this arguments, the max age of cache. This number won't be exported as setting, to prevent bad settings. But If somebody have interest, they could build tinymist from source and turns down the number for experiment. For performance concerning tinymist definitely needs higher max age, but it is arguable. Btw, tinymist was petty laggy when I followed the setting of typst-cli.
https://github.com/Myriad-Dreamin/tinymist/blob/main/crates/tinymist/src/task/cache.rs#L14

I still have building a minimal example in my TODO list, but have you tried increasing the number of repetitions of polylux as described in https://github.com/Myriad-Dreamin/tinymist/issues/513#issuecomment-2279680018? That's a reliable way of moving into the zone of a handful of GB.

but have you tried increasing the number of repetitions of polylux as described in https://github.com/Myriad-Dreamin/tinymist/issues/513#issuecomment-2279680018?

I have increased it, and this is the result.
DHAT Viewer - dhat-heap.json - Total (blocks).zip

20% of memory are used by allocating std::Vec<T>. When tinymist uses most memory, 43% of memory (1.5GB) are held by std::Vec<T>.
71% of memory are directly allocated by typst::compile and about 6% of memory is directly allocated by tinymist_query.

For anybody who us wondering about what max-repitions is about:
You may want to create slides in which the content gets revealed one by one. Because polylux bases on typst which produces PDF results, this cannot be animated, instead, there need to be a separated PDF page per "step", so called subslides.
The maximum amount of these subslides can be configured by max-repitions, the default is 10. This setting tells the compiler, how many times it should repeat the compilation for each slide.
Source: Polylux Docs: Dynamic Slides - Internals
TL;DR: max-repitions sets the maximum amount of so called subslides

Since the issue is not concrete and hasn't contributed to ideas for a long time, I'll close it.
I have created a more concrete issue, #918, for the following comment here:

yup, I'm sitting here with 10.8 GB RAM usage of tinymist, only change I did was configuring "onSave" trigger instead of "onType"..


test.aria.widgets.form.multiautocomplete.testHighlightMethods.MultiAutoHighlightNavigation fails for Chrome

I have just checked: this test does not fail on Chrome anymore.


renaming release

@l3r8yJ this is your task now, please go ahead. Deadline (when this ticket should be closed) is 2024-01-09T00:39:20.726674.
Estimation here is 60 minutes, that's how much you will be paid.
Remember, you don't have to solve everything in this ticket - you can solve it partially and leave todo markers in the code, which will become future tasks.
If you have any questions don't ask me, I'm not a technical person. Open new tickets instead.
If you don't have time or simply don't want to work on this, you can always resign.

@l3r8yJ please keep in mind that manual assignment of tickets is a bad practice and it is discouraged. Next time, please let me elect the assignee.

@rultor release, tag is 0.1.5

@rultor release, tag is 0.1.5

@l3r8yj Invalid release tag 0.1.5 specified. There is already a release 0.1.5 newer than the given release in this repository.

@rultor release, tag is 0.1.5

@l3r8yj I'm sorry, I don't understand you :( Check this page and try again please

@rultor release, tag is 0.1.6

@l3r8yj OK, I will release it now. Please check the progress here

@rultor release, tag is 0.1.6

@l3r8yj Done! FYI, the full log is here (took me 4min)

@l3r8yJ thank you for resolving this ticket. I've just added it to your active invoice. You can always check all your invoices and more on the Contributor Dashboard.


Use cmake_dependent_option for options that require IREE_BUILD_COMPILER.

Docs:

https://cmake.org/cmake/help/latest/module/CMakeDependentOption.html
https://cmake.org/cmake/help/latest/command/option.html

These options will now all automatically be forced to OFF when IREE_BUILD_COMPILER is OFF, rather than use previously cached/configured values.

That is such a terrible syntax, but also really useful functionality - classic cmake :P

Nice!

Nice! This has been around a long time - how did we miss it? A lot of things in llvm could be made less fragile with this too

Nice! This has been around a long time - how did we miss it? A lot of things in llvm could be made less fragile with this too

I started looking at our root CMake file differently after seeing Tint's and adding it as a dep. Some of these things are hard to see when working as an upstream maintainer and not [also] as a downstream user.
Many parts of our CMake configuration have also grown organically. There's still more low hanging fruit that we can clean up and establish better conventions for.


Docs: Misc Spelling and Grammar Fixes

Description
Minor docs changes to correct for spelling and capitalization.
Type of PR

[x] Documentation improvements

Testing
N/A
Documentation
N/A

PR withdrawn as bots are disallowed from signing the CLA under Mycroft contribution rules.

Understand the withdrawal - though also wanted to clarify - we are open to bot contributions. Unfortunately there needs to be a legal entity granting us the rights to the contributions we receive. While the legal world continues to debate the intellectual property rights of non-human entities we are restricted to human signed CLA's.


Open windows don't show in the window menu

OS: Mac OS X
OS version: 10.11.1
Atom version: 1.2.3
The window menu in the menu bar doesn't show any open Atom windows. Tested in safe mode as well.

This has already been fixed in the 1.3 betas.


Unknown attributes temperatureHvBatteryMin_K, temperatureHvBatteryMax_K

Describe your car model and the situation in which the error was shown
ID.3
Updates in volkswagen_we_connect_id, which is using v0.58.0 reporting warnings of two new attributes.
Paste the error without confidential information such as the VIN
/vehicles/WVXXXX/domains/measurements/temperatureBatteryStatus: Unknown attribute temperatureHvBatteryMin_K with value 289.65
/vehicles/WVXXXX/domains/measurements/temperatureBatteryStatus: Unknown attribute temperatureHvBatteryMax_K with value 290.15

Additional context
N/A

My bad, the component's version is pinned to 0.55.1, but I believe the attributes are still unknown in the latest version.

That is nice! What version is the ID.3 running? I cannot see that value with a 3.0 car.

Released in v0.59.2

That is nice! What version is the ID.3 running? I cannot see that value with a 3.0 car.

It's running 3.2


Uppercase extension .EVTX passed on the commandline was ignored. Fixed.

Looks good. Thank you for contributing to the project!


edgetpu python api deprecated

seems like birdfeeder.py needs to be re-written using PyCoral API
I'm a very beginning programmer but would appreciate any help?
getting the error
rom edgetpu.classification.engine import ClassificationEngine
ImportError: No module named 'edgetpu'

@dcohen-hub can you try solution provided in below reference:
https://github.com/robmarkcole/coral-pi-rest-server/issues/56#issuecomment-632125175


Clean-up Python-related packages.

python{2.6, 3.3} and python{2.6, 3.3}-dev were not required; tox.ini was not using either. Removing these packages then made the deadsnakes ubuntu PPA unnecessary.
I found that python2.6-dev seemingly was needed to build the wheels from the cryptography package in swift, but adding the OpenSSL development package (libssl-dev) fixed this.

One thing I forgot to mention: tox.ini shows testenv:py3pep8 is using python3, but that should be satisfied by Python 3.4.3, which is installed in this Ubuntu Trusty 14.04 environment.

i guess i used to think I cared about py26 and precise - but I don't really anymore...


Using Material 3 on Android, the Status Bar color changes when using Flutter Engine Cache

Steps to reproduce
We use FlutterEngineGroup and FlutterEngineCache in our application.
This is because we are incorporating Flutter screens into an existing native application.
The steps to reproduce the issue are as follows:

Launch the application.
Perform initialization processing in the Android app.
Call up the Flutter screen using FlutterEngineGroup and FlutterEngineCache.
Exit the Flutter screen with the back key. The cache is not discarded in this case.
Launch the Flutter screen again using the cached FlutterEngine.

When you perform the above operations, the color of the Status Bar will change.
Expected results
The status bar color should remain the same.
Actual results
If you discard the cache and start a new engine this problem doesn't seem to occur.
Code sample
We will prepare a minimal sample that will allow you to confirm the phenomenon.
Code sample in Flutter
import 'package:flutter/material.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        colorScheme: ColorScheme.fromSeed(seedColor: Colors.orange),
        useMaterial3: true,
      ),
      home: const MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  const MyHomePage({super.key, required this.title});

  final String title;

  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  int _counter = 0;

  void _incrementCounter() {
    setState(() {
      _counter++;
    });
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text(widget.title),
      ),
      body: Center(
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            const Text(
              'You have pushed the button this many times:',
            ),
            Text(
              '$_counter',
              style: Theme.of(context).textTheme.headlineMedium,
            ),
          ],
        ),
      ),
      floatingActionButton: FloatingActionButton(
        onPressed: _incrementCounter,
        tooltip: 'Increment',
        child: const Icon(Icons.add),
      ),
    );
  }
}

Code sample in Android

application class

package com.example.status_bar_test;

import android.app.Application;

import io.flutter.embedding.engine.FlutterEngine;
import io.flutter.embedding.engine.FlutterEngineCache;
import io.flutter.embedding.engine.FlutterEngineGroup;

public final class SampleApplication extends Application {
    public FlutterEngineGroup engineGroup;

    @Override
    public void onCreate() {
        super.onCreate();
        engineGroup =  new FlutterEngineGroup(this);
        final FlutterEngine mainFlutterEngine = engineGroup.createAndRunDefaultEngine(this);
        FlutterEngineCache.getInstance().put("main", mainFlutterEngine);
    }
}

demo first activity

package com.example.status_bar_test;

import android.app.Activity;

import io.flutter.embedding.android.FlutterActivity;
import io.flutter.embedding.engine.FlutterEngine;
import io.flutter.embedding.engine.FlutterEngineCache;

public class InitializeActivity extends Activity {

    @Override
    protected void onResume() {
        super.onResume();
        // ..run initialization process
        final String enginId = "main";
        if (!FlutterEngineCache.getInstance().contains(enginId)) {
            final FlutterEngine mainFlutterEngine = ((SampleApplication)getApplication()).engineGroup.createAndRunDefaultEngine(this);
            FlutterEngineCache.getInstance().put("main", mainFlutterEngine);
        }
        startActivity(new FlutterActivity.CachedEngineIntentBuilder(MainActivity.class, enginId).build(this));
        finish();
    }
}

main activity

package com.example.status_bar_test;

import io.flutter.embedding.android.FlutterActivity;
import io.flutter.embedding.engine.FlutterEngineCache;

public class MainActivity extends FlutterActivity {

    @Override
    protected void onDestroy() {
        // FlutterEngineCache.getInstance().remove("main");
        super.onDestroy();
    }
}

AndroidManifest

<manifest xmlns:android="http://schemas.android.com/apk/res/android">

    <application
        android:name=".SampleApplication"
        android:icon="@mipmap/ic_launcher"
        android:label="status_bar_test">
        <activity
            android:name=".MainActivity"
            android:configChanges="orientation|keyboardHidden|keyboard|screenSize|smallestScreenSize|locale|layoutDirection|fontScale|screenLayout|density|uiMode"
            android:exported="true"
            android:hardwareAccelerated="true"
            android:launchMode="singleTop"
            android:theme="@style/LaunchTheme"
            android:windowSoftInputMode="adjustResize">
            <meta-data
                android:name="io.flutter.embedding.android.NormalTheme"
                android:resource="@style/NormalTheme" />
        </activity>
        <activity
            android:name=".InitializeActivity"
            android:launchMode="singleTop"
            android:theme="@style/LaunchTheme"
            android:exported="true">
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />
                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
        <!-- Don't delete the meta-data below.
             This is used by the Flutter tool to generate GeneratedPluginRegistrant.java -->
        <meta-data
            android:name="flutterEmbedding"
            android:value="2" />
    </application>
    <!-- Required to query activities that can process text, see:
         https://developer.android.com/training/package-visibility?hl=en and
         https://developer.android.com/reference/android/content/Intent#ACTION_PROCESS_TEXT.

         In particular, this is used by the Flutter engine in io.flutter.plugin.text.ProcessTextPlugin. -->
    <queries>
        <intent>
            <action android:name="android.intent.action.PROCESS_TEXT" />
            <data android:mimeType="text/plain" />
        </intent>
    </queries>
</manifest>

Screenshots or Video

Screenshots / Video demonstration
https://github.com/flutter/flutter/assets/133312849/38b157da-b6cc-4890-ba5c-9c0abcb844d9

Logs
Logs
[Paste your logs here]

Flutter Doctor output
Doctor output
[✓] Flutter (Channel stable, 3.19.2, on macOS 14.4.1 23E224 darwin-arm64, locale ja-JP)
    • Flutter version 3.19.2 on channel stable at /Users/fvm/versions/3.19.2
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision<PHONE_NUMBER> (4 months ago), 2024-02-27 16:51:22 -0500
    • Engine revision 04817c99c9
    • Dart version 3.3.0
    • DevTools version 2.31.1

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/Library/Android/sdk
    • Platform android-34, build-tools 34.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b829.9-10027231)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 15.4)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 15F31d
    • CocoaPods version 1.15.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2022.3)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b829.9-10027231)

[✓] VS Code (version 1.77.1)
    • VS Code at /Users/Downloads/Visual Studio Code.app/Contents
    • Flutter extension can be installed from:
      🔨 https://marketplace.visualstudio.com/items?itemName=Dart-Code.flutter

[✓] Connected device (3 available)
    • sdk gphone64 arm64 (mobile) • emulator-5554 • android-arm64  • Android 12 (API 32) (emulator)
    • macOS (desktop)             • macos         • darwin-arm64   • macOS 14.4.1 23E224 darwin-arm64
    • Chrome (web)                • chrome        • web-javascript • Google Chrome 126.0.6478.61

[✓] Network resources
    • All expected network resources are available.

• No issues found!

Thanks for the detailed report. I was able to replicate the reported behavior with provided details in latest versions.

 stable, master flutter doctor -v
[!] Flutter (Channel stable, 3.22.2, on macOS 12.2.1 21D62 darwin-x64, locale
    en-GB)
    • Flutter version 3.22.2 on channel stable at
      /Users/dhs/documents/fluttersdk/flutter
    ! Warning: `flutter` on your path resolves to
      /Users/dhs/Documents/Fluttersdk/flutter/bin/flutter, which is not inside
      your current Flutter SDK checkout at
      /Users/dhs/documents/fluttersdk/flutter. Consider adding
      /Users/dhs/documents/fluttersdk/flutter/bin to the front of your path.
    ! Warning: `dart` on your path resolves to
      /Users/dhs/Documents/Fluttersdk/flutter/bin/dart, which is not inside your
      current Flutter SDK checkout at /Users/dhs/documents/fluttersdk/flutter.
      Consider adding /Users/dhs/documents/fluttersdk/flutter/bin to the front
      of your path.
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 761747bfc5 (33 hours ago), 2024-06-05 22:15:13 +0200
    • Engine revision edd8546116
    • Dart version 3.4.3
    • DevTools version 2.34.3
    • If those were intentional, you can disregard the above warnings; however
      it is recommended to use "git" directly to perform update checks and
      upgrades.

[!] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    ! Flutter recommends a minimum Xcode version of 13.
      Download the latest version or update via the Mac App Store.
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] VS Code (version 1.62.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (5 available)
    • SM G975F (mobile)       • RZ8M802WY0X • android-arm64   • Android 11 (API 30)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 98.0.4758.80

[✓] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.

[!] Flutter (Channel master, 3.23.0-13.0.pre.262, on macOS 12.2.1 21D62
    darwin-x64, locale en-GB)
    • Flutter version 3.23.0-13.0.pre.262 on channel master at
      /Users/dhs/documents/fluttersdk/flutter
    ! Warning: `flutter` on your path resolves to
      /Users/dhs/Documents/Fluttersdk/flutter/bin/flutter, which is not inside
      your current Flutter SDK checkout at
      /Users/dhs/documents/fluttersdk/flutter. Consider adding
      /Users/dhs/documents/fluttersdk/flutter/bin to the front of your path.
    ! Warning: `dart` on your path resolves to
      /Users/dhs/Documents/Fluttersdk/flutter/bin/dart, which is not inside your
      current Flutter SDK checkout at /Users/dhs/documents/fluttersdk/flutter.
      Consider adding /Users/dhs/documents/fluttersdk/flutter/bin to the front
      of your path.
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 6a8f70f11e (2 hours ago), 2024-06-18 00:12:10 -0400
    • Engine revision 78fdd06af5
    • Dart version 3.5.0 (build 3.5.0-272.0.dev)
    • DevTools version 2.36.0
    • If those were intentional, you can disregard the above warnings; however
      it is recommended to use "git" directly to perform update checks and
      upgrades.

[!] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at /Users/dhs/Library/Android/sdk
    ✗ cmdline-tools component is missing
      Run `path/to/sdkmanager --install "cmdline-tools;latest"`
      See https://developer.android.com/studio/command-line for more details.
    ✗ Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/macos#android-setup for
      more details.

[✓] Xcode - develop for iOS and macOS (Xcode 13.2.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 13C100
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] IntelliJ IDEA Ultimate Edition (version 2021.3.2)
    • IntelliJ at /Applications/IntelliJ IDEA.app
    • Flutter plugin version 65.1.4
    • Dart plugin version 213.7228

[✓] VS Code (version 1.62.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.29.0

[✓] Connected device (3 available)
    • Darshan's iphone (mobile) • 21150b119064aecc249dfcfe05e259197461ce23 • ios
      • iOS 15.3.1 19D52
    • macOS (desktop)           • macos                                    •
      darwin-x64     • macOS 12.2.1 21D62 darwin-x64
    • Chrome (web)              • chrome                                   •
      web-javascript • Google Chrome 109.0.5414.119

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 1 category.

[!] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    ! Flutter recommends a minimum Xcode version of 13.
      Download the latest version or update via the Mac App Store.
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] VS Code (version 1.62.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (5 available)
    • SM G975F (mobile)       • RZ8M802WY0X • android-arm64   • Android 11 (API 30)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 98.0.4758.80

[✓] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.

From Android triage: FYI this appears to be the difference between immersive sticky and edge to edge.

Additional information:
We have confirmed that the same issue occurs during transition animations on Android OS14 and later when calling up the Android screen from the Flutter screen and then returning to the Flutter screen.


Request for documention on how to use poetry with multiple python versions

[X] I have searched the issues of this repo and believe that this is not a duplicate.

Issue

In order to frame my request, let me describe how I anticipate some users come to use poetry (this at least was the case for me):

A developer realizes they need a more integrated dependency management solution than just setuptools, and discovers pipenv because it is the top hit on google.
They do a bit of reading, and decide that pipenv must be good because it is For Humans™ and it is (or at least was) The Officially Recommended Python Packaging Tool from Python.org. They try it out.
The developer inevitably attempts to do something outside the scope of what pipenv is trying to solve and then discover a) that pipenv is really only intended for application developers, and if one is a library developer you still need to use setup.py (making them feel like a second-class citizen), and/or b) the responses to issues filed for pipenv are usually either rude or dismissive.
They look for an alternative, and find poetry. There is no rudeness in the issues! The tool actually tries to help all developers! Huzzah!

For people like myself that came to poetry by way of pipenv, the approach to managing virtual environments taken by poetry tripped me up for quite some time because it differs significantly from pipenv.

In pipenv, one specifies the python version to use for virtual environment creation, and it searches the PATH and creates it for you if a match is found.
In poetry, it creates the virtual environment from the python against which poetry was installed, or uses a pre-made .venv.

This is a fundamental difference in philosophy between the tools which leads to a fundamental difference in usage on this specific aspect of project/dependency management. Because I had been first exposed to the pipenv way, I (wrongly) assumed that poetry also worked this way and struggled to get poetry to use a python 3.5 virtual environment because I had installed poetry against python 3.6; this was especially confusing because I had set the python version in the pyproject.toml to 3.5. I did notice the Poetry and virtualenvs section, but for someone who has not yet grokked the design philosophy of poetry there is not enough information for someone used to what pipenv does to adapt to poetry's approach.
REQUEST (finally)
I think the documentation for poetry would be improved if one (or both) of the following were added to documentation:

A section on the correct way to use poetry with multiple python versions (along with a discussion on the philosophy behind this design decision). Perhaps formalizing the discussion that occurred in https://github.com/sdispater/poetry/pull/82? It would be useful if it address both using pyenv and also using a pre-existing .venv.  This could just mean expanding the Poetry and virtualenvs section.
A section on how to migrate to poetry from pipenv. Here, the differences in philosophy could be specifically addressed and spelled out so that new users do not get frustrated at the first step.

It would also be helpful if this information were in the README itself, or at least linked to in the README.

This issue is very relevant.
Pipenv is making a lot of fuzz at the moment, but pipenv project management is just (imho) atrocious. As a pipenv refugee myself, i found the way that poetry controls virtualenvs very frustrating and un-intuitive, and there was not a lot of info regarding it.
I would really love for it to be clarified. And also honestly it would be amazing if you could set a python version of virtualenv with poetry instead of some roundabout way.

After explanation in #82 i started use pyenv but at the end i would prefere choosing python version at poetry install

After spending some time reading other issues where its mentioned (#9 #35 #57 #77 #82), here's a summary:  A lot of people get confused about what poetry does and what it doesnt do in regard to virtualenv.
It actually is kind of an edge case where it always tries to create virtualenv and work inside of it, but it doesnt provide any support for different virtualenvs and such.
Here are my thoughts on this. Poetry can either:

Extend support of virtualenvs to the point that user could use poetry as an abstraction above virtualenvs (which i would love);
Drop support of virtualenvs altogether in favor of user using external tools such as pew/pyenv/virtualenv to manage virtualenvs by himself;
Clarify documentation in regards to how poetry manages virtualenvs.

Regardles of whatever gets chosen as a solution, we should really update the documentation as it is a common point of confusion among new users (myself included).

Can't we just make venv to poetry what node_modules is to npm?

I was very careful to ensure that the scope of this issue was focused on documentation only when writing it to avoid a prolonged argument on how poetry should behave. I feel as though @sdispater was pretty clear in #82 how they intended poetry to be used, so it would sadden me to see this issue become an argument on that topic rather than a constructive discussion on how to communicate the proper usage to new users.
Having said that, I am not saying that if someone has a request to change behavior they should not pursue it; rather, those requests probably belong in a different issue focused on behavior changes to poetry. If one decides to go down that route, I would suggest not only requesting that the behavior be changed, but also accompany the request with concrete examples of why that behavior would be beneficial and why the current behavior does not fit your needs.

@SethMMorton agreed, adearing to .venv for that use case is probably best from here on out

I agree that the documentation should be clearer as to how to use Poetry to manage multiple python versions.
The idea when I started Poetry that it would work like Bundler in Ruby or NPM in JS. Basically, you would use a specific tool to manage Python versions, like pyenv, and install Poetry for each needed Python version.
Basically:

Install pyenv
Install a specific Python version: pyenv install 3.7.0
Install Poetry for this Python version: pyenv global 3.7.0 && curl -sSL https://raw.githubusercontent.com/sdispater/poetry/master/get-poetry.py | python

This is a process similar to what exists in other languages, like those mentionned above.
The use of virtualenvs is just a side effect of the Python ecosystem but, to me, it should be completely transparent and not something the end user should have to worry about or even know is there.
It's unlikely that Poetry will integrate a Python versions management tool. This is not the purpose of a package/dependency manager and their are other tools that do it well, pyenv particularly.
That being said, the documentation should be updated with complete step by step instructions.

Sorry for diverging this issue from its original direction. The above comment explains the intended workflow in a clear way. I will create a draft for documentation update in a couple of hours with a pull request unless someone wants to do it.

@festinuz That would be great!  I was planning on doing the same once I finished another project, but who knows when that will be done.

As a new pipenv refugee myself, I recently tried to use poetry for a project, installing it on Debian (stable or testing) in a new dedicated virtualenv created/managed by pew through get-poetry.py script, and haven't understood why /usr/local/virtualenvs/poetry/bin/poetry installfails with obscure message [FileNotFoundError] [Errno 2] No such file or directory: 'pip'.
Now, at least, I know from comments on #82 that I have to (probably) use pyenv (sigh! one more tool to understand and grok :-( )...

I am cross-posting this comment from the #332 PR because it seems relevant here as well.
I like pyenv (I use it at home), but there may be an issue with recommending it as the only way to manage multiple python versions because it does not support windows (pyenv/pyenv#62) and it does not look like that will happen in the future either. Is there a cross-platform or windows-specific alternative that can be demonstrated/recommended as well?

@SethMMorton I agree that there is no alternative to pyenv on Windows at the moment which is an issue to have any Python version installed on a Windows system. However, this is beyond the scope of Poetry and you will need to manually install the needed Python versions.
This is a limitation of the Python ecosystem at large.
Note, however, that the new WIP installer (https://github.com/sdispater/poetry/pull/378) will simplify things by requiring only one installation of Poetry. The poetry tool will then pick the currently activated Python executable to create the virtualenvs.
This will not solve the problem of the absence of a good tool on Windows to manage Python versions but it should help not having to install it multiple times.

I just want to clarify that I did not bring up pyenv on Windows in the context of installing different Python versions - that is clearly outside of the scope of poetry.
My understanding is that pyenv is being suggested as a means to switch between already installed environments (or virtual environments), and the concern I was expressing is for that case Windows users might get the impression that poetry is not intended for them since it is suggested to use a tool that does not work on Windows.

No, pyenv is only suggested for managing your main python versions. See https://github.com/sdispater/poetry/issues/305#issuecomment-407463426 for the workflow I have in mind.
Basically, the automatic creation and detection of the virtualenvs is what poetry handles itself.

Basically, the automatic creation and detection of the virtualenvs is what poetry handles itself.

I guess my confusion comes from the fact that poetry can only automatically create a virtualenv from the python version it was installed into (at least as I understand it). It seems that in #378 users will be encouraged to have only a single poetry installation, so within my understanding poetry would not be able to automatically create a virtualenv of any desired python environment. Is my understanding incorrect?
The heart of the reason I filed this issue was that it was unclear to handle this situation, and I was hoping to rectify that by improving the documentation.  I hope you do not feel like I am badgering you with all these questions, I just want to help improve the tool!

I have multiple python versions already installed in the system. I was surprised, that poetry could not pick it up automatically:
> poetry install

[RuntimeError]
The current Python version (3.7.2) is not supported by the project (~3.6)
Please activate a compatible Python version.

Is there a way to specify it manually?
I have never used pyenv before and I never needed it. I just do python<version> -m venv env.

@sirex I think if you create a virtualenv (pythonXY -mvenv env) and then activate it (source env/bin/activate), then poetry install will work. Poetry picks up the currently active virtualenv.

@garyo thanks for the workaround suggestion. Still it would be nice if poetry could pick latest required version automatically without any workarounds. Of course if there is no required python version available, then it could fail.

I'm not sure how you'd want that to work -- should poetry install without a current virtual env install into one of the systemwide pythons? How would it find them? It could try some hack like see if there's something called "pythonXY" on $PATH, but (a) that's fragile and (b) I'm not sure poetry should ever install deps into a systemwide location, at least not without user permission.
Seems to me you should normally always use poetry install in a virtualenv, unless you're doing poetry install --no-dev for production use.
All that said, I think the new poetry env subcommand will address some of your concerns because it'll create a virtualenv for you.

Currently poetry install if not running under virtualenv, creates a new virtualenv automatically in ~/.cache/pypoetry/virtualenvs/<projectname>-py<vresion>.
And it creates this virtualenv using a system python. So instead of blindly taking python it could be more specific with version taking pythonX.Y according to what is specified in pyproject.toml.

In poetry, it creates the virtual environment from the python against which poetry was installed, or uses a pre-made .venv.

This EXACTLY is what is ruining my poetry adoptation at the moment. I have multiple python projects with different python versions.
I have currently in the pyproject.toml
[tool.poetry.dependencies]
python = "^3.6"

But the poetry install command will create virtualenv using Python 3.7. In what sane world is it okay to make virtualenv using completely different python version than what project declares as being dependant of. The project in question does NOT work with Python 3.7.
This is not merely a documentation issue, this is a clear violation of POLA. Either create virtualenv using correct version of python or abort (fail fast, fail hard).
Now, does multiple poetry installations for each python version sound good? Not to me .. this needs to be addressed.

@paranoidi with python = "^3.6" Poetry thinks that it can use any version of Python 3.6-4.0. You want to use a tilde requirementpython = "~3.6". See the README for caret / tilde
(py372) C:\Users\king.kyle\Developer\Packages\common_prStub>poetry shell

[RuntimeError]
The current Python version (3.7.2) is not supported by the project (~3.6)
Please activate a compatible Python version.

(py372) C:\Users\king.kyle\Developer\Packages\common_prStub>

Also, Poetry uses whichever Python version is currently active if you've installed poetry globally using the get-poetry script. Typically the workflow is to create a virtual environment with the desired version, then run poetry install. So you could: conda create -n py368 python=3.6.8 conda activate py368 then poetry install would create a Python 3.6.8 environment for your project

I think Poetry should just use whatever environment it's in, and not creating a virtualenv.

See the README for caret / tilde

Not sure if it is just me, but the links don't seem to work.

Given that #731 is implemented, merged, and documented, this solves the spirit of this issue. I will close the issue now.

In poetry, it creates the virtual environment from the python against which poetry was installed, or uses a pre-made .venv.

This EXACTLY is what is ruining my poetry adoptation at the moment. I have multiple python projects with different python versions.
I have currently in the pyproject.toml
[tool.poetry.dependencies]
python = "^3.6"

But the poetry install command will create virtualenv using Python 3.7. In what sane world is it okay to make virtualenv using completely different python version than what project declares as being dependant of. The project in question does NOT work with Python 3.7.
This is not merely a documentation issue, this is a clear violation of POLA. Either create virtualenv using correct version of python or abort (fail fast, fail hard).
Now, does multiple poetry installations for each python version sound good to anyone? Not at least to me .. this needs to be handled better.

For me this is even worse, my system python is on 3.10 but in a current project I need to use tensorflow which only supports up to 3.9. So I have
python = ">=3.8,<3.10"

Which causes all sorts of problems. My only way of fixing it seems to be to install pyenv and use that as poetry cannot downgrade automatically it seems, at least poetry install fails.