Backport Surfman GL Backend to Hal 0.5

This backports #3151 to 0.5. Only the changes to gfx-backend-gl have been backported. No changes have been made to the examples or to CI or the Makefile. The examples still work using the glutin backend like they are already doing. :tada:
PR checklist:

[x] make succeeds (on *nix)
[ ] make reftests succeeds ( can't test because I don't have vulkan )
[x] tested examples with the following backends: gl
[x] rustfmt run on changed code

Oops, I just realized that I copied the whole tree over from 0.5 and therefore am including some commits from 0.6 in my name. One second while I fix that.

NVM, actually I don't think there is any way to do that properly in Git without getting all the other changes to other files from 0.6. I guess its stuck that way.

Typically I just do it by cherry-picking the commits on top of the release branch:
git fetch upstream hal-0.5
git checkout hal-0.5
git log master
git checkout -b 0.5-your-work-name
git cherry-pick <hash1>
git cherry-pick <hash1>
...
<edit Cargo.toml to bump the patch version>
<edit CHANGELOG.md>
make
git push origin HEAD

Is there a reason you couldn't do that for this PR?

It's because those commits have changes to the examples in them, too, that go along with the changes to the backend. I could do an interactive rebase afterwards to get rid of the changes to the examples, though. :thinking: I'll try that.

Do you know a way to grab the contents of a file at a given commit in git history?

Well I did it, but don't  merge it yet because it is panicking on wgpu. I'm not sure if there was some other changes to the gl backend that needed to be in there that I missed from hal 0.6. That was more trouble that I thought it would be.
I'll try to finish it off tomorrow.

Ah, bummer, I just found out that my attempt at fixing the context issues where the correct context was not being set current was flawed somehow and now the WGPU examples will panic with an invalid command error. That means that the master version will need to be updated too.
I think I might have to figure out why the screen turns black when I set the context current on dereference. Either way, somehow I need to figure out a solid pattern on when to set which contexts to the current context. This could be a bit more work, but oh well, still more to learn! :smiley:

@zicklag how is it going? Anything we could help with?

I've been really busy last few weeks and had a couple other weekend projects I wanted to touch on, but I might be able to take a new look at this today or tomorrow.
I think what needs to happen is we need to set the context in the main GlContainer to the current context on dereference, but there is some point in backend where that should not happen and the context needs to be accessed directly to avoid making it current. I'm assuming that is the reason that the screen renders black when setting the context current on dereference.
I don't really know of a good way to figure out why the screen is rendering black or where to avoid making the context the current context.
My only thought is to set a breakpoint on the make_current function and check the backtrace each time it gets called to try and find a place where it shouldn't be called.
I could probably use some help figuring out where it is going wrong, but I might need to get the code back to where it is making the context current on dereference first.

When you had problems with the blit function, and I was looking into this, precisely this situation was a problem: main GL context used to set itself up as active on dereference, and the blit was broken because of it. Maybe it's just the same problem here? Should be fairly trivial to address: just in a few places (during the blit) we'd need to go directly into the context without making it active. I can help investigating if you are stuck.

Yeah, it is actually exactly the same problem as the black screen with the blit we had earlier. The solution I used at the time was just to not set the context current on dereference.
The issue with not setting it current on dereference is that we were running into segfaults if you did things in the wrong order because we weren't setting the context to current when we needed to.
So I think we need to bring back setting it current on derefernce, but find out where we shouldn't be doing it. Like you showed when we looked at this, we need to not do it during the blit, but there must be somewhere else as well that we need to not make it current so that the screen doesn't render black.
It would be great to get some help on this. :smiley:

I'll work right not on getting this PR to the point where I think it should be, other than the black screen which needs to be investigated.

OK, I just pushed a commit to get the PR where I want it ( other than the black screen ).
Because this is the backport branch the examples are still using the glutin backend, so you will have to grab the quad example from master ( which uses surfman ) and update the gl features in the Cargo.toml:
git checkout master examples/quad

[features]
# Change the gl features
gl = ["gfx-backend-gl", "gfx-backend-gl/surfman", "gfx-backend-gl/surfman-x11"]

Then you can run the quad example and see the black screen.

Looking at this code now. Have a few questions.
Since we have a separate GL context per surface, and we need to blit from one to another, we have to have a place somewhere at the surface creation where we say "please use shared resources from that other context", where the other context belongs to the instance. I'm not seeing where this happens, so maybe you don't have the contexts sharing anything in this PR?
The docs on surfman 0.2 appear to only describe a scenario where you'd want to get the results of a surface context into another one. Our scenario is the opposite though, it's sad that it's not covered in any way.
I do recall though that this missing piece did present in the code you merged to master. Could it have been lost somehow?

I tried to cleanup and re-implement the present_by_copy logic, and I found out that this line was particularly suspicious: https://github.com/gfx-rs/gfx/pull/3216/files#diff-11d4b77124a4a2d39bbc81d17b96e104R278
            gl.surfman_device
                .read()
                .present_surface(&gl.surfman_context.read(), &mut surface)
                .expect("TODO");

You are presenting surface with the context of the instance? Documentation clearly states that this has to be the same context that was used for surface creation, which is swapchain.context.read()
Interestingly, it doesn't work in my own try here - https://github.com/kvark/gfx/commit/21edb4f13775838f6b258e6d490f744fbd67c347#diff-11d4b77124a4a2d39bbc81d17b96e104R288
The present crashes with:

thread 'main' panicked at 'invalid context', /Users/dmalyshau/.cargo/registry/src/github.com-1ecc6299db9ec823/io-surface-0.12.1/src/lib.rs:141:17

I don't have an explanation for this. So that's the second mystery.

Thanks for taking a look at this.

Since we have a separate GL context per surface, and we need to blit from one to another, we have to have a place somewhere at the surface creation where we say "please use shared resources from that other context", where the other context belongs to the instance. I'm not seeing where this happens, so maybe you don't have the contexts sharing anything in this PR?

I think that the resources created by the same device are shared or something like that, maybe.
The surfman README says:

OpenGL's share context or share lists feature allows you to share textures across contexts. However, this often exposes driver bugs, and, even if it works, it causes most operations to take mutex locks. Efficient texture sharing requires the use of platform-specific APIs, which surfman abstracts over.

So it looks like it only allows you to share textures across contexts? I'm assuming that contexts created with the same surfman::Device are automatically configured with the share lists or whatever is necessary for resource sharing, but again, maybe that is only textures.

I do recall though that this missing piece did present in the code you merged to master. Could it have been lost somehow?

I don't think I lost anything. We'd have to double check. We could look back at the PR to master to see if anything is in there, but I don't think there is anything extra on the master one.

You are presenting surface with the context of the instance? Documentation clearly states that this has to be the same context that was used for surface creation, which is swapchain.context.read()

Yeah, that looks wrong, it should probably just be:
        // Present the surfman surface
        #[cfg(surfman)]
        {
            let mut surface = gl
                .surfman_device
                .read()
                .unbind_surface_from_context(&mut swapchain.context.write())
                .expect("TODO")
                .expect("TODO");
            gl.surfman_device
                .read()
                // Present the swapchain context instead, which is the one that
                // is created when `create_surface` is called.
                .present_surface(&swapchain.context.read(), &mut surface)
                .expect("TODO");
            gl.surfman_device
                .read()
                .bind_surface_to_context(&mut swapchain.context.write(), surface)
                .expect("TODO")
        }

When I do that, though, I still get a black screen.

Not discounting other potential problems, but I think that the we can essentially prove that the current issue with the black screen has solely to do with calling make_context_current with the proper timing.
In the current state of the PR, if you simply comment out this line:
https://github.com/gfx-rs/gfx/pull/3216/files#diff-b42bd9be6c2e5cf60482e1230ad3007cR152
impl Deref for GlContainer {
    type Target = GlContext;
    fn deref(&self) -> &GlContext {
        #[cfg(not(wasm))]
        //self.make_current();
        &self.context
    }
}

And add a line here: https://github.com/gfx-rs/gfx/pull/3216/files#diff-3b1612b9f8249907b53c59e1c50b70bdR305
    fn supports_queue_family(&self, _: &QueueFamily) -> bool {
        self.device.write().make_context_current(&self.context.read()).expect("TODO");
        true
    }

That will work regardless of the order of these two lines in the quad demo:
       let adapters = instance.enumerate_adapters();
       let surface = unsafe {
            instance
            .create_surface(&window)
            .expect("Failed to create a surface!")
        };

Without that line in the supports_queue_family function, though, the quad demo break if you re-order those lines and enumerate the adapters after creating a surface: 'Framebuffer does not have any image attached'.
Even with that line in the supports_queue_family function, though, the WGPU cube demo crashes with a segfault. The only way to fix the segfault in WGPU so far is to re-enable the make_current on GlContainer deref, which presents us with the black screen again.

Hmm, this all sounds like magic to me. I want to see where we are actually telling the contexts that they need to share resources.

We'll probably have to look at the surfman source code, because I don't think the surfman API even has anything to do explicit sharing. It is all automatic as far as I know.

Hmm. In glutin, if you want to make the context shared, you have to specify, which context you are sharing with - https://docs.rs/glutin/0.23.0/glutin/struct.ContextBuilder.html#method.with_shared_lists
Seems strange that surfman could imply that somehow.

Well you have to create an manage all contexts with a Device, so could it make sure that all contexts created by the device are shared with some internal context inside of device maybe?

@zicklag I played with it a bit more and made some progress, but haven't resolved the problem yet. Want to share it to see if you have any ideas.
The code is in https://github.com/kvark/gfx/tree/surfman-try
It does 3 interesting things:

it refactors the lifecycle of the surfman surface. I don't think it fixes anything, but it's closer to what we want in the end. The surface lives on our Surface, and the procedure of binding it to surfman context, blitting into it, and then unbinding it, is totally local. So anywhere outside of the present_surface_by_copy the surface is not bound.
when it blits the frame into the surface, it also clears a small rectangle with green color. This is to test if the presentation of the target framebuffer works. And it does! I think this is good news. Now we need to figure out why the blit doesn't result in anything.
right before presenting, we are also clearing the swapchain's FBO with a small red rectangle. This is to make sure that the source of the blit contains something. It doesn't show up in the blit though!

My take from these experiments is that the contexts don't properly share resources, and thus the swapchain.fbos[0] changed in one context doesn't show the contents in another. Please have a look at try to make sense of it, maybe you'll be able to advance from there.

That sounds like great progress! I'll mess with it when I get the chance, hopefully over the weekend.

right before presenting, we are also clearing the swapchain's FBO with a small red rectangle. This is to make sure that the source of the blit contains something. It doesn't show up in the blit though!

Hmm, I wonder if what was actually happening this whole time when it was properly displaying was that the surface's context was the active context and all draw's were actually happening directly to the surface's context, leaving the blit to never actually do anything because what we were rendering was already being written to the surface.
Then when we actually fixed the blit and therefore covered up anything that we were rending directly to the surface and ended up with a blank screen. ( I'm not actually sure if that's how that works, just a thought. )

I wonder if by sharing textures like we thought about in the first place to avoid the blit would actually fix this issue...:thinking:
Anyway I'll check it out, thanks!

Alright, shortly after writing the findings I figured that the lack of sharing is the only explanation. And this is something I asked about earlier in https://github.com/gfx-rs/gfx/pull/3216#issuecomment-623174080 . Inspecting surfman code showed that it doesn't even try to share anything, at least on macos/CGL. I prototyped the sharing in https://github.com/kvark/surfman/tree/share-surface and got it working:

I don't know if this is in scope of surfman, so I filed https://github.com/pcwalton/surfman/issues/65 . Hoping to hear back from Patrick soon.

Wow, that's sweet! I wonder if it is the same on X11. Or maybe your branch will work on X11. I'll try to test it tonight if I can.

@zicklag fwiw, surfman now contains the context sharing that I merged. We can make it work.
I think we should do so on master first though.

I think we should do so on master first though.

I agree. It'll probably be rather easy to update with the sharing because it is pretty much in place already, so I'll probably just do that sometime.


Please invite me to the community

Name
techguru0
Additional Context
I wanna contribute to your org with my free time work

Invitation sent to join the GitHub Organization. Welcome to Infraform 🎉Please check your mail to accept the invitation sent by GitHub! 🤖About usInfraform is an open source solution for serverless platforms, it is designed for developing solutions and enhancing creators to create their infrastructure 💻 Great to have you in our community ✨ Looking forward to your valuable contributions! 🤓 You can also check our community ✨ Github ✨ LinkedIn 💜 You can also share our community with other folks who want to learn about development, DevOps, tech, and contribute together and with us 👉💜👈


[#679] Allow specifing URL for NCT collector to download from

[#679] Allow specifing URL for NCT collector to download from
This allow us to download NCT's XML dump in another process, save to S3 and just
use that on this collector.
opentrials/opentrials#679

About accepting dates or URLs, I agree it can be annoying not to be able to simply run make start nct, but I'm not sure the correct option is keeping accepting dates. Maybe, if called without params, we could download from 2001-01-01 until the current day (i.e. full dump). Or maybe we would end up only using the S3 dumps anyway, so it's not an issue. Happy to deal with it in another issue.


Add .travis.yml for Travis CI

Here is a passing Travis CI build for my cloned repo:
https://travis-ci.org/msabramo/yolk/builds/4750850
To set up Travis CI for your own repo, follow the instructions at http://about.travis-ci.org/docs/user/getting-started/ -- it takes about 2 minutes.

Ping

Ping


Failure to generate consensus

Hi,
I am trying to apply the sicelore pipeline to a matched Illumina/Nanopore data set and am running into some issues generating consensus sequences.
This is start of the error file that I'm seeing:

ComputeConsensus INPUT=p1_ont_got_10xAttributes_umifound_.bam OUTPUT=p1_ont_got_consensus NTHREADS=10 RACONPATH=/nfs/sw/racon/racon-1.3.3/bin/racon POAPATH=/nfs/sw/poav2/poav2-1.0/poa MINIMAP2PATH=/gpfs/commons/home/gmullokandov/software/minimap2-2.17_x64-linux/minimap2 TMPDIR=/tmp/    CELLTAG=BC UMITAG=U8 GENETAG=IG TSOENDTAG=TE UMIENDTAG=UE POLYAENDTAG=PE USTAG=US MAXCLIP=150 VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json USE_JDK_DEFLATER=false USE_JDK_INFLATER=false

[Mon Nov 25 16:02:06 EST 2019] Executing as --------- on Linux 3.10.0-957.el7.x86_64 amd64; Java HotSpot(TM) 64-Bit Server VM 1.8.0_45-b14; Deflater: Intel; Inflater: Intel; Provider GCS is not available; Picard version: null
INFO    2019-11-25 16:02:06     LongreadParser          start...
INFO    2019-11-25 16:02:32     LongreadParser          Processed              500,000 Records.  Elapsed time: 00:00:25s.  Time for last 500,000:   25s.  Last read position: chr2:197,401,753
INFO    2019-11-25 16:02:40     LongreadParser          end...
INFO    2019-11-25 16:02:40     LongreadParser          Total SAMrecords        [703741]
INFO    2019-11-25 16:02:40     LongreadParser          SAMrecords valid        [605709]
INFO    2019-11-25 16:02:40     LongreadParser          SAMrecords unvalid      [98032]
INFO    2019-11-25 16:02:40     LongreadParser          SAMrecords mapqv=0      [3717]
INFO    2019-11-25 16:02:40     LongreadParser          SAMrecords no gene      [0]
INFO    2019-11-25 16:02:40     LongreadParser          SAMrecords no UMI       [0]
INFO    2019-11-25 16:02:40     LongreadParser          SAMrecords chimeria     [94303]
INFO    2019-11-25 16:02:40     LongreadParser          SAMrecords reversed     [12]
INFO    2019-11-25 16:02:40     LongreadParser          Total reads             [605709]
INFO    2019-11-25 16:02:40     LongreadParser          Total reads multiSAM    [0]
INFO    2019-11-25 16:02:41     MoleculeDataset         Total Molecules         [13034]
INFO    2019-11-25 16:02:41     MoleculeDataset         Total Molecules multiIG [2]
INFO    2019-11-25 16:02:41     MoleculeDataset         Calling consensus       start with [10] threads
java.io.FileNotFoundException: /tmp/GAATCGTGTTAGGACGGGGTACACTCAC.pir (No such file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.(FileInputStream.java:138)
at java.io.FileInputStream.(FileInputStream.java:93)
at java.io.FileReader.(FileReader.java:58)
at org.ipmc.sicelore.utils.Molecule.call(Molecule.java:240)
at org.ipmc.sicelore.utils.Molecule.call(Molecule.java:16)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
java.io.FileNotFoundException: /tmp/TATCAGGAGCACACAGCCACTGTTTACT.pir (No such file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.(FileInputStream.java:138)
at java.io.FileInputStream.(FileInputStream.java:93)
at java.io.FileReader.(FileReader.java:58)

Here is an example of the output:

CCGGTAGGTACGCTAT-CCGGAAAAATTT-122

CCTCACATCTCACTCG-CCTCTTATCAAC-1
GCCTTTTTAAAACAAAAAACCACAAAAATCCCAAACCATACTTGCTGTACCCTGGGTAAGTATGTGCTTCTTCTGTGGGGTGGATCGAAGGGGGCTGGGTTGGCTGGATATTCTTAGATGTGGACCACCAAGGGGTTGTTGAAAACCACCGTATTAAATGTCTTTGATAAGCCTTCTGCTCACATTTTGTGAATGTCTGAAGCATAGTTTGTGTATATTGACAGAGCTCTAAATTTGTTTGTACATAAAGGCTTCTCTGTTTCTCTCTCCCCTTGTTAGGGTCAAGTGGTGTCCTCCACCTTAAACAGCTGCTGACAGACACATTAGGAGCAGCTGTTTGTTAGCTGATGACAAATATT

It looks like there is an error for all UMIs >2.
Thanks for your help!

It seems that the poa command is not working, the .pir file is not generated and the consensus sequence can not be retrieved. Could you check that blosum80.mat file is in the same directory as poa (/nfs/sw/poav2/poav2-1.0/). Do you see the .fa files created in the /tmp/ directory ? You can try to run poa command in the terminal to check everything working, for instance for molecule CCGGTAGGTACGCTAT-CCGGAAAAATTT-122, with reads .fa file still in /tmp/ this should be:
/nfs/sw/poav2/poav2-1.0/poa -read_fasta /tmp/CCGGTAGGTACGCTATCCGGAAAAATTT_reads.fa -pir /tmp/CCGGTAGGTACGCTATCCGGAAAAATTT.pir /nfs/sw/poav2/poav2-1.0/blosum80.mat -hb -best

The original directory /nfs/sw/poav2/poav2-1.0/poa did not have the blosum80.mat file so I downloaded and compiled a fresh install of poav2 that did have blosum80.mat. Using the new directory the command you suggested worked and the CCGGTAGGTACGCTATCCGGAAAAATTT.pir file was created.
However, when re-running the script with the new directory I am now seeing the consensus output like this:

GTACAGTCAAGAGCTG-AATTTCATCACA-1983
null
TGTCCCATCGACATAC-CGAGCTGTTCAT-1
TATTAAGATTGTACAACAGATGGCTGTTCTTATGGGCTGTGCCATCTTGCCACATCTTAGAAGTTGGTTGATCATTGAACATGAGTCTTGTGGATAAGGCAGCGTGAGTTCGGATATCATTGCTTTATGCGAAATTATGCATCGGCTGAAGCAGCAACTCATATGCGTATCGAATCTTTTTTTTGATTCTTGTGTTAGCTGCTTTTATGGAAGGGAGTATCTTCCAACAGAGGAAAAGGGTTTGGCTGCTTTCTTGAGAAGGCTTCATTGGAGTATATTACATTCACTCTTATGGATTGAAATACTAAGCAACTACTATACTTTAGTGATGTTAATCCTTTGACTGAGGAATTTCCAGTCTCCTGATGGGG
GTATTGGTCCGTCACT-TTTAGATAATTC-1
CAAAAGCAAGAAGTCCTGGCAAGCGAGACACACTGGTATTAGATTGCTACAACAGATAGCTATTCTTATGGGCTGTGCCATCTTGCCACATCTTAGAAGTTTAGTTGAAATCATTGAACATGGTCTTGTGGATGAGCAGCAGGAAGTTCGGACCATCAGTGCTTGGCCATTGCTGCCTTGGCTGAAGCTTAGCAACTCCTTATGGTATCGAATCTTTTGATTCTGTGTTAAGCAGCACATGGAAGGGTATCCGCCAACACAGAGGAAAGGGTTTGGCTCACTTTCTTGAAGGCTATTGGGTATCTTATTCCTCTTATGGATGCAGAATATGCCAACTACTATACTAGAGAAGTGATGTTAATCCTTATTCGAGGAATTCCAGTCTCTGATGAGGAAATGAAAAAAGAATTGTGCTGAAGGTGGTAAAACAGTGTTGTGGGACAGATGGTGTAGAAGCAAACTACATTAAAACAGGGAGATTCTTCCTCCCGCTTTTTAAACACACTTCTGGCAGCACAGGATGGCTTTGGATGAGAAACTACCGACAGTTAGTTGATACTACTGTGGAGTTATGAAAACAAAGTAGGTGCAGCAGATTATATCCAGGATTGGCCGGATGATCTGAAGGAAGCCGAGGTACAGTACAGAAAAAATGGTGATGGAGACAATTGAGAAAATTATGGGCAATTTAGGGCAGCAGATATTGATCATAAACTTGAAGAACAACTGATTGATGGTATTCTTTATGCTTTCAAGAACAGACTACAGAGAGGACTCAGTAATGTTGAACGGCTTTAGCACAAGTGATTATGCTCTTGGCAAACAGTCAAACCATACTTGCCTCAGATCTGTGGTACAGTTTTGTAGTTTAAATAACAAATCTGCTAAAGTTAGGCAACAGGCAGCTGACTTGATTTCTCGAACTGCTGTTGTCATGAAGACTTGGC
AGCGCCACATCTTCGC-CAATCCATAACG-1531
null

In the tmp directory there are now many files with this structure 'core.XXXX' and no more .fa files.
When trying to debug those .core files I see this:

#0 0x000000000042cd12 in thread_pool::ThreadPool::ThreadPool(unsigned int) ()
#1 0x000000000042d1d4 in thread_pool::createThreadPool(unsigned int) ()
#2 0x0000000000412265 in racon::Polisher::Polisher(std::unique_ptr<bioparser::Parserracon::Sequence, std::default_delete<bioparser::Parserracon::Sequence > >, std::unique_ptr<bioparser::Parserracon::Overlap, std::default_delete<bioparser::Parserracon::Overlap > >, std::unique_ptr<bioparser::Parserracon::Sequence, std::default_delete<bioparser::Parserracon::Sequence > >, racon::PolisherType, unsigned int, double, double, signed char, signed char, signed char, unsigned int) ()
#3 0x0000000000412bd1 in racon::createPolisher(std::string const&, std::string const&, std::string const&, racon::PolisherType, unsigned int, double, double, signed char, signed char, signed char, unsigned int) ()
#4 0x000000000040e60b in main ()

I am running this with 200gb of RAM on 20 threads for reference.

I retried using 400gb of RAM with 20 threads and that seems to have solved the problem! Thanks for your help.

You seems to have quite a lot of reads per UMI (>1500). It looks like a targeted sequencing experiment. Apparently, Racon requires quite a lot of RAM with such huge sequencing depths. Whole transcriptome single cell sequencing typically yields mean UMI sequencing depth of < 20 where Racon has a reasonable memory requirement.
Concerning the error during SNP calling: This exception is thrown by samReader.query(chromosome, start, end, ...) when you query a chromosome name that is not in the dictionary of the bam file. E.g. if you mapped against a reference where chromosome 3 is named “chr3” and you query “3”.  Make sure you name the chromosomes in the csv file exactly as in the reference used for mapping. You can check the naming of the reference chromosomes in the bam file header.
Note: For SNP calling we usually only use UMIs with sufficiently high read numbers. You can filter out the low covered molecules (e.g. < 3 reads/UMI) before remapping and snp calling.

Thanks for your help, you are correct - the reference file was not consistent with bam file, once that was fixed it worked very well.

I'm having another issue with one of my files - with an error at at the parsing of the illumina file.

Exception in thread "main" java.lang.NullPointerException
at com.rw.bamparsermaven.Parser.parse(Parser.java:119)
at com.rw.bamparsermaven.Main.main(Main.java:85)

Other Illumina files were processed identically with cellranger and had no issues so it is likely a problem with this file - any insight into error would also be great. Thank you

The IlluminaParser.jar  used the first SAM record to set the cell barcode length. Apparently in your BAM file, the first SAM record does not have a cell barcode.
This caused the null pointer exception. We fixed this error.  Just download the IlluminaParser-1.0.jar against the new version on Github (IlluminaParser-1.0.jar). No need to replace the  directory.

Hi there, I'm also working on this project and we want to do a genotyping comparison between our Illumina and Nanopore reads at a UMI / UMI level instead of a cell-cell level and so I'm wondering if there is a way to get the actual UMI's associated with the reads that were accounted for in the SNPmatrix. If we have the UMI's, we can then pull the actual reads with/without the SNP from the consensus file.

The IlluminaParser.jar used the first SAM record to set the cell barcode length. Apparently in your BAM file, the first SAM record does not have a cell barcode.
This caused the null pointer exception. We fixed this error. Just download the IlluminaParser-1.0.jar against the new version on Github (IlluminaParser-1.0.jar). No need to replace the directory.

I did as you suggested and am getting another error, but pointing to different line

Exception in thread "main" java.lang.NullPointerException
at com.rw.bamparsermaven.Parser.parse(Parser.java:122)
at com.rw.bamparsermaven.Main.main(Main.java:88)

Try it again with todays build.
If your bam file still triggers the exception we would need just the first SAM entry of your BAM file for verification.

It's working without issue. Thank you!


New Feature: Multi-arch Zones

Description
This PR introduces the multi-arch zones, allowing users to select the VM arch upon deployment.
Multi-arch zone support in CloudStack can allow admins to mix x86_64 & arm64 hosts within the same zone with the following changes proposed:

All hosts in a clusters need to be homogenous, wrt host CPU type (amd64 vs arm64) and hypevisor
Arch-aware templates & ISOs:

Add support for a new arch field (default set of: amd64 and arm64), when unspecified defaults to amd64 and for existing templates & iso
Allow admins to edit the arch type of the registered template & iso

Arch-aware clusters and host:

Add new attribute field for cluster and hosts (kvm host agents can automatically report this, arch of the first host of the cluster is cluster's architecture), defaults to amd64 when not specified
Allow admins to edit the arch of an existing cluster

VM deployment form (UI):

In a multi-arch zone/env, the VM deployment form can allow some kind of template/iso filtration in the UI
Users should be able to select arch: amd64 & arm64; but this is shown only in a multi-arch zone (env)

VM orchestration and lifecycle operations:

Use of VM/template's arch to correctly decide where to provision the VM (on the correct strictly arch-matching host/clusters) & other lifecycle operations (such as migration from/to arch-matching hosts)

Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[x] New feature (non-breaking change which adds functionality)
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)
[ ] build/CI
[ ] test (unit or integration test code)

Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale

[ ] Major
[ ] Minor

Bug Severity

[ ] BLOCKER
[ ] Critical
[ ] Major
[ ] Minor
[ ] Trivial

Screenshots (if appropriate):

How Has This Been Tested?

How did you try to break this feature and the system with this change?

@blueorangutan package

@nvazquez a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✔ el8 ✔ el9 ✔ debian ✔ suse15. SL-JID 10893

@blueorangutan ui

@nvazquez a Jenkins job has been kicked to build UI QA env. I'll keep you posted as I make progress.

UI build: :heavy_check_mark:
Live QA URL: https://qa.cloudstack.cloud/simulator/pr/9619 (QA-JID-436)

@blueorangutan test

@nvazquez a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

[SF] Trillian test result (tid-11274)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 49550 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9619-t11274-kvm-ol8.zip
Smoke tests completed. 139 look OK, 0 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test
Result
Time (s)
Test File

@nvazquez do we also need changes to deployment planner (orchestration), so VM deployment plan/planning considers arch of the iso/template against that of the cluster & hosts?

@blueorangutan package

@nvazquez a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✖ el8 ✖ el9 ✖ debian ✖ suse15. SL-JID 10939

@blueorangutan package

@nvazquez a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✖ el8 ✖ el9 ✖ debian ✖ suse15. SL-JID 10940

@nvazquez can you merge latest main and address conflicts? Thanks.

@blueorangutan package

@nvazquez a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✔ el8 ✔ el9 ✔ debian ✔ suse15. SL-JID 10956

@blueorangutan test

@nvazquez a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@blueorangutan package

@rohityadavcloud a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✖ el8 ✖ el9 ✖ debian ✖ suse15. SL-JID 10977

[SF] Trillian test result (tid-11346)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 53001 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9619-t11346-kvm-ol8.zip
Smoke tests completed. 138 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test
Result
Time (s)
Test File

test_02_unsecure_vm_migration
Error
447.72
test_vm_life_cycle.py

test_04_nonsecured_to_secured_vm_migration
Error
448.07
test_vm_life_cycle.py

@nvazquez can you check/fix the lint issue & kick packaging. After that this should mergeable, pending no other issues.

Packaging result [SF]: ✔ el8 ✔ el9 ✔ debian ✔ suse15. SL-JID 10990

Thanks @rohityadavcloud - fixed
@blueorangutan package

@nvazquez a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✔ el8 ✔ el9 ✔ debian ✔ suse15. SL-JID 10997

@blueorangutan package

@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✔ el8 ✔ el9 ✔ debian ✔ suse15. SL-JID 11012

Minor conflict (no need for repackaging though), pl address @nvazquez
@blueorangutan test

cc @nvazquez hope I haven't screwed up 🤞 the merge conflict
@blueorangutan package

@rohityadavcloud a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

[SF] Trillian Build Failed (tid-11382)

@blueorangutan test

@rohityadavcloud a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests

@blueorangutan package

@rohityadavcloud a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✔ el8 ✔ el9 ✔ debian ✔ suse15. SL-JID 11024

[SF] Trillian test result (tid-11386)
Environment: kvm-ol8 (x2), Advanced Networking with Mgmt server ol8
Total time taken: 55615 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr9619-t11386-kvm-ol8.zip
Smoke tests completed. 137 look OK, 1 have errors, 2 did not run
Only failed and skipped tests results shown below:

Test
Result
Time (s)
Test File

test_mount_shared_fs
Failure
93.24
test_sharedfs_lifecycle.py

test_resize_shared_fs
Error
0.23
test_sharedfs_lifecycle.py

all_test_network
Skipped
---
test_network.py

all_test_nic_adapter_type
Skipped
---
test_nic_adapter_type.py


Page Goes Blank when Deleting Corner Radius

Hi,
I've noticed if you clear the 'Corner Radius' field - then the page just displays white and needs a refresh, loosing your changes.
Cheers
Leacho

Yes that is annoying, I will get that sorted.

This is solved by df194d804398b4e817684122ca085122dccdd198.


SNS AutoConfiguration fail on non web application

SNS AutoConfiguration require "WebMvcConfigurer" class on classpath.
It could be great that SNS can be autoconfigured on spring-batch context.
It can be possible to split in two class, the SNS Configuration and the WebMvc argumentResolvers injection.
Thanks.
Julien.

Thanks @VanRoy for reporting. You should be now able to use the pre-configured AmazonSNS client to publish messages.

Thanks a lot, this is exactly what I needed.


Adding single cell header for table

I am trying to add a row of a single cell for the title of the table in python-pptx. Here is the example that I am using from pptx documentation:
from pptx import Presentation
from pptx.util import Inches

prs = Presentation()
title_only_slide_layout = prs.slide_layouts[5]
slide = prs.slides.add_slide(title_only_slide_layout)
shapes = slide.shapes

shapes.title.text = 'Adding a Table'

rows = cols = 2
left = top = Inches(2.0)
width = Inches(6.0)
height = Inches(0.8)

table = shapes.add_table(rows, cols, left, top, width, height).table

# set column widths
table.columns[0].width = Inches(2.0)
table.columns[1].width = Inches(4.0)

# write column headings
table.cell(0, 0).text = 'Foo'
table.cell(0, 1).text = 'Bar'

# write body cells
table.cell(1, 0).text = 'Baz'
table.cell(1, 1).text = 'Qux'

prs.save('test.pptx')

here is what I want:

--------------------------------------
                title
--------------------------------------
     foo          |       bar
--------------------------------------
     Baz          |       Qux
--------------------------------------

How to add the above title row to the table?

This requires cell merging which is not yet supported.

@scanny any pointers on where should i start if i wanted to work on this feature

_Cell.merge() appears in release v0.6.14 to be released tomorrow, Sep 24, 2018.


Update version and changelog

Codecov Report

Merging #22 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff           @@
##           master     #22   +/-   ##
======================================
  Coverage    77.2%   77.2%
======================================
  Files           7       7
  Lines         443     443
  Branches       14      14
======================================
  Hits          342     342
  Misses         98      98
  Partials        3       3

Impacted Files
Coverage Δ

sdc/rabbit/__init__.py
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update abbf25b...5f30134. Read the comment docs.


Texture/item icon issues identified so far

Duplicate icons of every item show up in NEI
Pink/Black Box icon items: wooden bucket, wooden bucket (water), wooden bucket (milk), Ornate Bow, Ornate Arrow, Ornate arrow (bodkin head), Ornate arrown (broad head), ornate bodkin head, ornate broad head, tin nugget, enderforge broad head
Mithium heavy pick says "heavy Ignotumite pick"
Mattock names do not display correctly for all tiers
Black and Red Steel heavy pick both say "heavy Blue Steel pick"
Gold nugget has incorrect name display.

unfinished stuff


overriding upper limits on memory and cpus

I am trying to use AGC in conjunction with the nf-core/rnaseq workflow, specifically in the challenging case of the axolotl (Ambystoma mexicanum) genome, which is roughly ten times larger than human.  I have not been able to override the memory limits for either the star or salmon steps of the process.
All salmon quant and star alignment steps are failing with an exit code of 137, which indicates inadequate memory
Among the steps I have tried to fix this issue are:
prebuilding the salmon index and the star index (built independently on an r5.12xlarge instance
changing the max-cpus and max-memory options handed directly to the workflow call (using inputs.json), and also providing a custom nextflow.config file that included either this:
process {
withName: 'NFCORE_RNASEQ:RNASEQ:ALIGN_STAR:STAR_ALIGN' {
memory = 360.GB
}
withName: 'NFCORE_RNASEQ:RNASEQ:QUANTIFY_SALMON:SALMON_QUANT' {
memory = 220.GB
}
}
and also
withLabel:process_high {
cpus   = { check_max( 24    * task.attempt, 'cpus'    ) }
memory = { check_max( 320.GB * task.attempt, 'memory'  ) }
time   = { check_max( 16.h  * task.attempt, 'time'    )
}
With some feedback after the last failures, I was directed to the AWS Batch console, and found that in all cases, the requested memory was the standard 72GB (for the star)
I have tested an exact case of this with the same r5.12xlarge machine and found that with 40 CPU threads, the alignment took about 47 minutes and consumed a peak of about 305GB of RAM.
I really just need to know how to override the memory request for both the star alignment, and also the salmon quantification steps.
The logs have already been shared with some of the agc team.
Expected Behavior
A
//: #  (A clear and concise description of what you expected to happen.)
Actual Behavior
Screenshots
Additional Context
Operating System:
AGC Version:
Was AGC setup with a custom bucket:
Was AGC setup with a custom VPC:

The list of EC2 types deployed to the default compute environments may not be sufficiently sized to fulfill your requested CPU/ memory. You can specify instance types used for a context as described here
If this doesn't resolve the issue please feel free to re-open this issue.


The 'Quit' button seems to not properly close grabber

Bug description
When you use the 'Quit' button which can be found under File -> Quit it will not properly close grabber. When you use this, the next time you open grabber you will get the message that grabber was not properly closed last time.
Expected behavior
Close it properly so it will not give a warning the next time you open grabber
System information
Windows 10 with the latest nightly version

Indeed, it would seem that the behavior of this button is equivalent to force-quitting (kill) Grabber with the Windows Task Manager 🤔


Vegan: updating test files

Hi @remimarenco,
If you could merge this PR into yours I think we can eliminate https://github.com/galaxyproject/tools-iuc/pull/640 and https://github.com/galaxyproject/tools-iuc/pull/707 once and for all.
all best,
Youri

Sure, thanks Youri! Merge done ;)

Hmm.. the .tt_blacklist has to be fixed too to resolve a merge conflict. Let's see if I can find some time for this tonight, otherwise remind me tomorrow.


Allow excluding certain types of things from history

I'm using exwm and consult-buffer. I have set up exwm to set the buffer titles to the actual X11 window titles. The web browser sets the window title to the title of the visited website. Because of this there are many new unique entries selected every day, and these keep filling up the prescient history with useless entries that will never be used again. How can I exclude exwm entries from being saved in prescient?

If the remembered candidates are not reused, then they are eventually forgotten. Do you have a way to identify programmatically which buffers are the Exwm buffers? There is the command prescient-forget, to which you can pass the candidate you would like to forget.
I am not familiar with Exwm, but I would guess that there is a hook in which you could run prescient-forget on the buffer's name after switching to the buffer.

Also, Prescient only affects sorting candidates during completion. It does not affect history variables like consult--buffer-history. If you want to remove those candidates from the command's completion history (accessed via M-n, M-p, etc.), that is a question for Consult.

If the remembered candidates are not reused, then they are eventually forgotten.

Yeah, this is true. But there are just so many of them that they fill up 95% of the prescient history, and this seems to push out legitimate history entries. Things that I use quite frequently keep disappearing from the top of the completion candidates, and I suspect this is because the illegitimate history entries fill up the entire history.

I am not familiar with Exwm, but I would guess that there is a hook in which you could run prescient-forget on the buffer's name after switching to the buffer.

Well, this could be a way to work around the problem at the very least, but I would like to prevent them from being saved in the first place. This workaround would still cause some legitimate history entries to be forced off the list, no? Wouldn't one history entry fall off the list each time an exwm buffer is added to the list assuming the history list is full? Switching to and from exwm windows is something I do probably hundreds of times a day, so I assume this still would lead to a significant loss of legitimate history.

Also, Prescient only affects sorting candidates during completion. It does not affect history variables like consult--buffer-history.

But it does save the selected candidate, which I would like to avoid for exwm buffers and some other types of stuff.
I didn't mention it in the original post, but I have some other completion stuff that also ends up being saved in the prescient history. I have a custom thingy that I use to open urls in qutebrowser, with completion from my browsing history. All selected candidates from this also ends up being saved, which I also would like to avoid, since qutebrowser already saves the browsing history and I have even disabled sorting for this command, since that is also handled by the browsing history.
This last thing here has its own command,  does that make it easier to exclude it from the prescient history?
I did a little digging in the source code and found that there is a vertico-prescient--exit-commands, which specifies what commands should have their histories saved by prescient. However, this list contains exit-minibuffer, so I guess this will include all commands that use minibuffer completion anyways.
Maybe I could just put some advice around vertico-prescient--remember-minibuffer-contents that checks if minibuffer-contents-no-properties is a valid exwm buffer. I don't quite see how I can do this in a clean way the way the code is currently structured though, since I would really want to force the variable txt to be nil, but this is fetched directly from the minibuffer contents. Any ideas?

Okay, I had a small brainfart when talking about the advice around the remember function. I figured out how to solve the exwm part of my problem like this:
 (defun dont-remember-qutebrowser-buffers (orig-fun &rest args)
   "Exclude qutebrowser buffers from prescient history."
   (if-let* ((selected-candidate (substring (minibuffer-contents-no-properties) 0 -1))
             (selected-buffer (get-buffer selected-candidate))
             (selected-buffer-x11-class (buffer-local-value 'exwm-class-name
                                                            selected-buffer))
             (is-qb (string= "qutebrowser" selected-buffer-x11-class)))
       (message "Selected buffer is QB, not saving")
     (funcall orig-fun)))
 (advice-add 'vertico-prescient--remember-minibuffer-contents :around #'dont-remember-qutebrowser-buffers)

My second problem (detecting that the command we just ran was my qutebrowser launcher) is still a bit tricky though, as I don't know how to check what the command was that has just exited. But this you might be able to help me with hopefully?

My second problem (detecting that the command we just ran was my qutebrowser launcher) is still a bit tricky though, as I don't know how to check what the command was that has just exited. But this you might be able to help me with hopefully?

In general, there's the variable last-command. For detecting consult-buffer inside vertico-prescient--remember-minibuffer-contents, the variable current-minibuffer-command seems to work. It looks like current-minibuffer-command was added in Emacs 28.

Btw, there is a weird non-letter character (a square box with numbers in it), at the end of every history entry. Do you know what this is and why it is there?

I believe Consult adds those characters to distinguish candidates with the same text that would otherwise be equivalent. In older versions of Emacs, workarounds were needed to know which candidate was selected, such as when a text candidate was used a key for key-value pairs. I don't know if those workarounds are still needed for more recent versions of Emacs.
See the function consult--tofu-append in consult.el.

When I'm thinking a bit more about this, maybe all of consult-buffer should be excluded from the history? Since these entries are only valid while those buffers are open, so the saved history would have a high likelyhood of containing a lot of invalid entries when the user closes buffers.

Maybe advising the function add-to-history would work for you. There's also the user option history-length, which you could increase.


Usefulness on ultra wide displays?

I really like the intent of this plugin, however after trying it I'm not sure if either I have it configured incorrectly or I have the wrong expectations.
What I'm looking to do is achieve a view where I can auto-collapse all splits except for the currently active one.
I work on an ultra monitor, so space isn't an issue and I don't want to resize to the golden ratio (seems this might be more useful on smaller screen?), what I do want to see is an ability to automatically expand the current split and collapse it's sibling splits to a pre-defined, relative size.
It seems that there is :FocusMaximize, but I don't see how to enable this as the default "autoresize" option (seems that by default everything is autoresized using the GR?).
Additionally, I don't see how to define a maxwidth or maxheight (I guess these aren't implemented?), nor do I see a way to define the units as relative for any of the sizing?
So, please correct my assumptions if they are wrong. But in any case, nice work here, I like what you're trying to do, even if it turns out that it's not right for me!

Correct me if I'm wrong, but it seems like the FR here is to add an option for the default resize mode?

@willothy Hi! Yes, I think a default resize mode would be useful.
However, are the widths that "maximise" minimises to configurable, if not, then should they use the same values, or would each resize mode need it's own config?
Additionally, I think that the ability to set widths in proportional widths (I'm not quite sure what the current width/height values are, are they rows/cols in the terminal?), ideally I'd like to say "min width of 5 cols, max width of 20%".
To provide some more context... I use the Amethyst window management tool on Mac OS, it is kind of a port of xmonad for MacOS.
I have my layout in a 3 column split arrangement, where in the centre of my ultra wide I have my current working window, this is full height. Then to the left and right I have "wings" where other secondary apps are stacked in columns. The tool automatically puts new windows in the stacks in these wings when I open windows.
It looks like this:
|               |                      |                  |
|               |                      |                  |
|               |                      |                  |

@willothy Hi! Yes, I think a default resize mode would be useful.
However, are the widths that "maximise" minimises to configurable, if not, then should they use the same values, or would each resize mode need it's own config?
Additionally, I think that the ability to set widths in proportional widths (I'm not quite sure what the current width/height values are, are they rows/cols in the terminal?), ideally I'd like to say "min width of 5 cols, max width of 20%".
To provide some more context... I use the Amethyst window management tool on Mac OS, it is kind of a port of xmonad for MacOS.
I have my layout in a 3 column split arrangement, where in the centre of my ultra wide I have my current working window, this is full height. Then to the left and right I have "wings" where other secondary apps are stacked in columns. The tool automatically puts new windows in the stacks in these wings when I open windows.
It looks like this:
|               |                      |       4      |
|       2       |                      |--------------|
|---------------|           1          |       5      |
|       3       |                      |--------------|
|               |                      |       6      |

I have a key combo to switch the currently focused window to the centre, this can toggle/swap from "central" to "in the wings" (to be clear any window may be focussed, not just the central one).
I then have another shortcut for growing / shrinking the central window (and the wings resize respectively).
This is the kind of split management that I'm trying to simulate. So that my windows are tiled and my vim splits are also tiled - and following the same paradigm - within nvim. You could describe it as fractal in the sense that the window contents is able to tile in the same way as many windows.... (and maybe one day I'll get a bunch of robotic monitor arms and more monitors and do the same  with them 🤡 😉 ).
So yeah, it's a bit more complex and something that I'd love to help out with, if only I had more time (young children, house to renovate etc 😴).

However, are the widths that "maximise" minimises to configurable, if not, then should they use the same values, or would each resize mode need it's own config?

Maximize widths are currently not configurable other than the minwidth for unfocused windows, but that could be changed easily. I don't think giving each mode it's own full config would be necessary, but separating maxwidth when it's implemented would be a good idea I think

ideally I'd like to say "min width of 5 cols, max width of 20%".

Do you mean 20% of the (window manger, not nvim) window size, or 20% of the available space in the nvim window layout? The former is quite doable I think, but like I mentioned earlier determining available space would be very difficult.

So yeah, it's a bit more complex and something that I'd love to help out with

As long as resizing is still based on the total vim screen size this shouldn't be too difficult, I'll try to work on this sometime soon. If you end up having time to work on it feel free to submit a PR though :)

Maximize widths are currently not configurable other than the minwidth for unfocused windows, but that could be changed easily.

Yeah, being able to set a max width (and use proportional values) I think would be necessary.

I don't think giving each mode it's own full config would be necessary, but separating maxwidth when it's implemented would be a good idea I think

In this case would the default also obey the maxWidth value, or ignore? If it ignores it then there needs to be documentation that this value is only for the "maximise" mode. If it were possible to configure each mode independently then this makes the config more self documenting. It could also be possible to support both configs by allowing something like the following as an alternate config:
autoresize = {
        enable = true, -- Enable or disable auto-resizing of splits
        width = 0, -- Force width for the focused window
        height = 0, -- Force height for the focused window
        minwidth = 0, -- Force minimum width for the unfocused window
        minheight = 0, -- Force minimum height for the unfocused window
        height_quickfix = 10, -- Set the height of quickfix panel

        -- Allow configuring per mode, this would override any defaults above (and not be a breaking change)
        modes = {
          golden_ratio = {
            min_width = 0 -- override the min width for this mode
          }
        }
    },

But... (and I know you already said you don't see a need for custom config, but maybe the above changed your mind 😄)... if per-mode config of sizing were allowed, why not allow per-mode config of everything instead, maybe I want to see relative line numbers in the golden_ratio mode, but not in the maximise mode... etc, so maybe the config to allow per-mode config (or overrides might look like:
    enable = true, -- Enable module
    commands = true, -- Create Focus commands
    autoresize = {
        default_mode = 'maximise',
        enable = true, -- Enable or disable auto-resizing of splits
        width = 0, -- Force width for the focused window
        height = 0, -- Force height for the focused window
        minwidth = 0, -- Force minimum width for the unfocused window
        minheight = 0, -- Force minimum height for the unfocused window
        height_quickfix = 10, -- Set the height of quickfix panel
    },
    split = {
        bufnew = false, -- Create blank buffer for new split windows
        tmux = false, -- Create tmux splits instead of neovim splits
    },
    ui = {
        number = false, -- Display line numbers in the focussed window only
        relativenumber = false, -- Display relative line numbers in the focussed window only
        hybridnumber = false, -- Display hybrid line numbers in the focussed window only
        absolutenumber_unfocussed = false, -- Preserve absolute numbers in the unfocussed windows

        cursorline = true, -- Display a cursorline in the focussed window only
        cursorcolumn = false, -- Display cursorcolumn in the focussed window only
        colorcolumn = {
            enable = false, -- Display colorcolumn in the foccused window only
            list = '+1', -- Set the comma-saperated list for the colorcolumn
        },
        signcolumn = true, -- Display signcolumn in the focussed window only
        winhighlight = false, -- Auto highlighting for focussed/unfocussed windows
    }
    modes = {
      maximise = {
        autoresize = {
          minwidth = 30,
          maxwidth = 80,
        }
        split = {
          tmux = true -- allow creating tmux splits when in maximise mode
        }
      }
    }

It's not ideal, it would require some logic to block overriding autoresize.enable & autoresize.default_mode in a mode config (but this could also be solved by separating by moving these to top-level config (i.e auto_resize_enable = true auto_resize_default_mode = "maximise") and then only allow overriding top-level objects & their children).
This could also open the door to allowing users to define their own "modes", so maybe I have a mode in a given vim tab which I use for tmux stuff, and I want it to behave in one way, maybe I have another mode for when I'm related on groups of related files, and I want it to adopt the "maximise" semantics, and maybe I have a general mode that I use for working in a group of un-related files, and this uses the default config. I can invoke each mode in a given vim tab have everything work as expected.... I know I'm going way out of scope with all of this, but it's an attempt to describe why considering per-mode config might open the door to some quite interesting use cases in the future...

Do you mean 20% of the (window manger, not nvim) window size, or 20% of the available space in the nvim window layout? The former is quite doable I think, but like I mentioned earlier determining available space would be very difficult.

all proportional values (20% etc) would be relative to the nvim total space available in nvim, so 20% would take up 20% width in nvim, not 20% of the remaining space after pushing other to their max width. Telescope seems to handle this by using fractional numbers as percentage (e.g 0.5 = 50%) - however, I have an open issue about and I wonder if instead a new optional min_width_unit = 'percentage' or min_width_unit = 'absolute' would be a cleaner way to handle this and avoid the edge cases issue I identified in telescope.

As long as resizing is still based on the total vim screen size this shouldn't be too difficult, I'll try to work on this sometime soon. If you end up having time to work on it feel free to submit a PR though :)

👍


Not able to get wifi networks to show up

I got the berry OS loaded on my tdeck with working colors etc. However wifi networks aren't showing up. I have both 2.4 and 5ghz wifi in range. Have them pop up visibly once when I was troubleshooting inverted colors but never again. I've reinstalled it a couple times now with no luck.
Hoping for any help

It is not so obvious but the wifi is turned off by default. You have to enable it by clicking or touching the WiFi logo (pointed to by the cursor in picture below) Then you will see the wifi networks available

@jeeab As @AdaSzi mentioned, you need to turn on the WiFi icon first. Have you tried it?


Admin campaign index page throws exception when dates for campaign is not set

Login using<EMAIL_ADDRESS>With default data, some of campaigns do not have start and end dates.
While displaying campaigns list, it throws exception as in screenshot.

Can't reproduce with git hash a598be2.  Can you explain more about how to reproduce this or what git version you're using?

I am not able to reproduce this issue either.
The AllReady.dbo.Campaign table defines StartDateTime and EndDateTime as not null.  It looks like commit fb6f00c2 updated the AllReady.DataAccess.SampleDataGenerator.InsertTestData() method to make sure test Campaigns are being initialized with valid StartDateTime and EndDateTime values.

The UI also fails validation if Start or End Date are not provided.  Looks like this has already been solved in several places.

yep thx @zimplicity and @kgipson marking it closed


Enhancement: Modal should close when clicking anywhere outside of the modal

… and maybe also on esc :)

Haha yeah so this should already be happening, but a bug in on-load is
messing this up - should dig deeper into why that's happening
On Thu, Dec 15, 2016, 05:51 Kristina Schneider<EMAIL_ADDRESS>wrote:

Assigned #116 https://github.com/datproject/dat-desktop/issues/116 to
@yoshuawuyts https://github.com/yoshuawuyts.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/datproject/dat-desktop/issues/116#event-894711883,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACWlejDoSQKcAyaiBKO6LL6BSoefSdvHks5rIMdmgaJpZM4LNty6
.

Done


Fixed Popup Issue and Added Favicon to Entry

Fixes the following issue:
https://github.com/Team-Delta-KA/KA-Contest-Judging-System/issues/23
Instead of immediately authenticating, just authenticate when the user tries to submit scores. Then, they can log in and if they're not a valid judge, we tell them through alert() instead of awkwardly redirecting to permissionDenied.html because the re-direction doesn't give them a chance to login again if they used the wrong account.
Also, I added the favicon to entry.html.

Thanks! Will merge after I fix the diff bug.


Update to v2.4.4 and remove static library

Checklist

[x] Used a personal fork of the feedstock to propose changes
[ ] Bumped the build number (if the version is unchanged)
[x] Reset the build number to 0 (if the version changed)
[x] Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
[x] Ensured the license file is being packaged.

Updates to 2.4.4 (closes https://github.com/conda-forge/seccomp-split-feedstock/pull/6) and removes the static library in accordance to https://github.com/conda-forge/cfep/blob/master/cfep-18.md#implementation.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.


⚠ Bofrost.at has degraded performance

In c02774c, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 2298 ms

Resolved: Bofrost.at performance has improved in bb938ed after 6 minutes.


Using PathShape::line to draw an arc results in jagged edges at larger radii

Describe the bug
This could also serve as a feature request, but currently I have a usecase where I need to draw arcs of circles. I have an implementation that works for smaller radii (sub 25 pixels), but it rapidly gets more jaggy and aliased as the radius increase.
To Reproduce
This is the code I'm using, you get the same results if you just iterate over a range of integer degrees.
  let circle_radius = 100.0;
  let mut circle_points = Vec::new();
  let visual_angle = 180.0 + correcting_angle;
  let angle_range = 180.0_f32.max(visual_angle)..180.0_f32.min(visual_angle);
  for i in 1..=512 {
      let (sin, cos) = (i as f32).map_range(1.0_f32..512.0_f32, angle_range.clone()).to_radians().sin_cos();

      circle_points.push(painter.clip_rect().center() + Vec2::new(circle_radius * sin, circle_radius * cos));
  }
  painter.add(PathShape::line(circle_points, Stroke::new(2.0, Color32::GREEN)));

Screenshots

A side by side example of arcs at 2 different radii
Desktop (please complete the following information):

OS: Linux
Browser Baseview backend
Version 0.26

The typical way to render circular arcs is to approximate them using cubic Bezier curves of up to π/2 radians, using multiple curves to construct larger arcs. The computation for a single curve looks like
    // Center of the circle
    let xc = center.x;
    let yc = center.y;

    // First control point
    let p1 = center + radius * Vec2::new(start_angle.cos(), -start_angle.sin());

    // Last control point
    let p4 = center + radius * Vec2::new(end_angle.cos(), -end_angle.sin());

    let a = p1 - center;
    let b = p4 - center;
    let q1 = a.length_sq();
    let q2 = q1 + a.dot(b);
    let k2 = (4.0 / 3.0) * ((2.0 * q1 * q2).sqrt() - q2) / (a.x * b.y - a.y * b.x);

    let p2 = Pos2::new(xc + a.x - k2 * a.y, yc + a.y + k2 * a.x);
    let p3 = Pos2::new(xc + b.x + k2 * b.y, yc + b.y - k2 * b.x);

(see this paper by Aleksas Riškus)
Here's a dial drawn using these approximate arcs:

Would definitely be nice to have these in egui proper -- if it's a desired feature I'd be happy to write a PR!

please do! it'd make my life a lot easier

I'm fairly sure this bug isn't actually a real issue, the root cause on my end was accidentally drawing the same path 1000 times.


[WIP] New Travis build

We should not allow HHVM to fail.

I will change it

:+1:

Do not forget to add
after_success:
    - codecov # If coverage


Fail to list permission encounter 400 invalidRequest

Category

[x] Question
[ ] Documentation issue
[ ] Bug

I request permission for some file but receive error:
{"error":{"code":"invalidRequest","message":"Filter query is_overseas' or mailNickname eq 'S_SpecialAssistant' or mailNicknameq 'S_sales_B' or mailNickname eq 'S_sales_C' or mailNickname eq 'S_mmanager' or mailNickname eq 'S_sales_passistant' or mailNickname eq 'it' or mailNickname eq 'itmanager' or mailNickname eq 'S_Qualityt_manage' or mailNickname eq 'S_BD_Manager' or mailNickname eq 'S_Ament_manage' or mailNickname eq 'S_marketing_Department_manage' or "}}

I have no idea about the resp, I only use simple query without options
GET /_api/v2.0/drives/driveid/items/itemid/permissions
And the resp header also attached
Cache-Control: no-cache
Content-Length: 940
Content-Security-Policy: frame-ancestors 'self' teams.microsoft.com *.teams.microsoft.com *.skype.com *.teams.microsoft.us local.teams.office.com *.powerapps.com *.yammer.com *.officeapps.live.com *.office.com *.stream.azure-test.net *.microsoftstream.com;
Content-Type: application/json
Date: Thu, 17 Jun 2021 10:01:16 GMT
Expires: -1
HTTP/1.1 400 Bad Request
MS-CV: n9K5fF8gAACfWtuc+4CzJw.0
MicrosoftSharePointTeamServices: <IP_ADDRESS>02
NEL: {"report_to":"network-errors","max_age":86400,"success_fraction":0.001,"failure_fraction":1.0}
P3P: CP="ALL IND DSP COR ADM CONo CUR CUSo IVAo IVDo PSA PSD TAI TELo OUR SAMo CNT COM INT NAV ONL PHY PRE PUR UNI"
Pragma: no-cache
Report-To: {"group":"network-errors","max_age":86400,"endpoints":[{"url":"https://spo.nel.measure.office.net/api/report?tenantId=0a2ee0d2-0be9-4a05-82fb-a6a8506ce7ee&destinationEndpoint=Edge-Prod-TPE30r4&frontEnd=AFD"}]}
SPIisLatency: 1
SPRequestDuration: 427
SPRequestGuid: 7cb9d29f-205f-0000-9f5a-db9cfb80b327
Strict-Transport-Security: max-age=31536000
X-Cache: CONFIG_NOCACHE
X-Content-Type-Options: nosniff
X-FRAME-OPTIONS: SAMEORIGIN
X-MS-InvokeApp: 1; RequireReadOnly
X-MSEdge-Ref: Ref A: 8202F8D72B7F4FF5ACE00E74A185D7C5 Ref B: TPE30EDGE0211 Ref C: 2021-06-17T10:01:15Z
X-Powered-By: ASP.NET
X-SharePointHealthScore: 2
X-VroomVersion: 2.0
request-id: 7cb9d29f-205f-0000-9f5a-db9cfb80b327

Thanks for help!

I encountered same issue. Is there any update, please?

This is a bug in the service and a fix is being tracked. Unfortunately there isn't a work around you can use in the meantime.

As part of a repository clean up effort we are closing older issues. If this issue remains, please: open a new issue, reference this issue, and provide any additional details that may help in resolution. Thank you for your understanding as we work to improve our responsiveness.


Fix release links in readme

Thanks @abhisekp for reporting this

Thanks for fixing the second link!
But the first link must be left intact please
https://github.com/stampit-org/stampit/releases/tag/2.0
Those breaking changes links. This is important.

Interesting, I don't know why I thought that both links are broken :) Good catch, it's reverted


Better import progress

Currently locks up UI, need to spin off the processing in a background thread and prevent selection of new files/clicking the import button again
Ideally, provide the user with import progress in some sort of display, showing which file is being processed and which record it is on out of the total
Add a cancel button, which propagates the CancellationRequested

Moving this to 1.1.0, but for 1.0.0, we need to have a spinner here with #42

For whatever pattern we set up here for the improved loading progress, we should set up in the other places where we save to the db

https://learn.microsoft.com/en-us/dotnet/core/extensions/channels


Custom Enchants disappear off of armour

Whenever I put on armour with custom enchants, it removes the custom enchants and leaves it just a vanilla armour piece.

Strange, it might maybe from another plugin. I have heard from another user this happening and it turned out to be another plugin. Though they didn't tell me what plugin it was.

@kicjow , this issue needs to be closed as I have provided the plugin that was causing the issue.


Walkthrough of GC Cloud service order process with Shared Services Canada (SSC) and AWS federal team

Describe the issue
Having recently gained access to the Gov't of Canada Cloud Brokering Service CRM system, we've taken a first look at the process of entering a service order - it appears that there are mandatory fields to be entered that are specific to federal government departments, and we need to consult SSC on what to do with these fields for our service orders.
Which Sprint Priority is this issue related to?
The 'Milestone' should clearly list Sprint Priorities - which one is this issue related to?
Additional context
Add any other context, attachments or screenshots
Definition of done
First try an email inquiry to Pat at SSC, to see if there's an easy solution for our service orders
If not, schedule a meeting to walk through the process and obtain SSC (and possibly AWS) guidance on what needs to be entered into our service orders

2020-10-08 Shared screenshots with SSC, and all good there - may still want to walkthrough with AWS

Discussed with David Lurie (AWS) - believe we're all set to make it work so that we transfer our existing account in under the GC Brokering Service and thereby avoid extra work in porting our Landing Zone to a new account.


modify loginfo when visiting namesrv

What is the purpose of the change
work for this issue, the loginfo is ambiguity when visiting namesrv, so we modify the loginfo to make it clear.
Brief changelog
use the variable channel to replace addr in the log statement.

Coverage increased (+0.01%) to 49.521% when pulling b4ac26385e692fabd8f32bfc5ff5b836888e3baf on keranbingaa:loginfo_modify into 61f4f99b884d6b1d716414b14ed2e494d201bebe on apache:develop.

@keranbingaa address seems more clear than the channel, IMHO, print all channel info isn't a good choice in here.

@duhenglucky I modify what print out， and now it just print the address, and when the variable address is null, it will also print the remote address.

@duhenglucky  I modify what print out， and now it just print the address, and when the variable address is null, it will also print the remote address.

I think printing address is more clear too.And just printing channel.remoteAddress() will be ok.


Collection Types

Fixed missing title

I'd review all of your collection topics to make sure the generic parameter is showing up correctly

So I'm not following - given a link label like [List<T>], how would I format this so that the <T> shows up?

Like this List<T>

with backslash

That isn't working for me (using VS Code). If I enter [List\<T\>] it renders as [List\]'

@cartermp do you have any suggestions on how to fix this?

Maybe we shouldn't be using brackets at all. The .NET conceptual docs on MSDN use parentheses instead of brackets, even though reference docs use the brackets. Does anyone know the history behind this?

Unsure how to handle this w.r.t how DocFX will render it.

The conceptual docs will use parentheses if you have VB selected. If you change your page to C#, it will show the brackets. Since we only have C# now, we should show brackets until they create a way for us to use a tag to reference to APIs by using their AssetID or something.

@chenkennt do you know how we can render brackets with docfx?

Okay, that's really strange behavior. Unless the topic includes a code block there is no way to change the language setting, so the page rendering depends on whatever language you were last viewing. I guess that really only matters for those of us that use both languages, but I could see where it might confuse customers as well.

Yep, I noticed that too.

Canceled; fixed in new branch


Remove extra space when no icon is available

This fixes #34 (presumably).
There're three conditions under which the devicon cannot be shown for a buffer:

nvim-web-devicons is not installed.
The file have no corresponding devicon.
M.options.show_devicons is set to false.

This PR ensures Buffer.icon is nil under all these circumstances, then tests for it when rendering.
I'm willing to refactor the whole Buffer:get_props function, but that might be too much to review.

Thanks for all your PRs to this project and for your enthusiasm here. I’d be willing to review a separate PR but I don’t know necessarily when I’ll have time to do so. I’m willing to merge this as it is now.


Unable to attach cognitive service using python SDK

Package Name:  Azure Cognitive Search
Package Version:  11.2.1
Operating System: Windows
Python Version: 3.7

Describe the bug
When creating an azure search skill set through the Python SDK, I am unable to attach an azure cognitive service. This causes entities populated by the skill set to not be populated. The CognitiveServiesAccount model that is to be used by the searchIndexerSkillset constructor does not seem to exist when I try to import it.
The documentation here says the cognitive services account object is a required parameter, but I am unable to add it and no errors are thrown when I create the skillset without it.
To Reproduce
When creating a skillset, the CognitiveServicesAccount parameter is not required and a skillset is successfully created without it. This directly contradicts the online documentation for the searchIndexerSkillset constructor which states the CognitiveServicesAccount parameter is required.

I am also unable to import the CognitiveServicesAccount object from the Python SDK, so it seems it is not possible to pass it in and specify what account I would like to attach to.

When I create the skillset without it, there are no errors and it appears as if the skillset is successfully created.

However, when I inspect the skillset in the portal, the cognitive services account is null so the skillset will not work. This is causing issues with programmatically deploying a working solution using Python.

Expected behavior
When creating a skillset, I expect to be able to create a CognitiveServicesAccount object and pass it into the searchIndexerSkillset constructor. I also expect this parameter to be required and an error will be thrown if I do not pass in the CognitiveServicesAccount object. When I inspect the skillset in the portal I expect the correct cognitive service to be attached to the skillset.
Screenshots

Additional context
To work around the issue stated above, the REST API can be used to update the skillset created through the Python SDK and attach a cognitive service account to it. The developer can also attach one through the portal.
The Python SDK documentation should be updated to state this issue and any workarounds. The issue should also be logged and eventually fixed so users can deploy a working solution using Python.

Label prediction was below confidence level 0.6 for Model:ServiceLabels: 'Cognitive Services:0.49177337,Search:0.13391021,Docs:0.08864225'

Hi @akazlow , thanks for the feedback. We'll take a look at this and get back to you asap.

@akazlow You can use azure.search.documents.indexes.models.CognitiveServicesAccountKey to set the argument cognitive_services_account of SearchIndexerSkillset to attach the cognitive search.