Download File from URL Requiring Authentication

Hi, I'm trying to download a large file from the Atlassian Jira API, which requires an authentication key to be passed through the headers. The DownloadLargeFilesByURL method will fail, because it is only taking the URL and getting denied at the server. Normally I would use UrlFetchApp.fetch(url, params) to get the file. Is there a way to pass parameters/headers through the method call?
I'm currently using the sample script to attempt this:
function main() {
var url = ""; // Please set URL you want to download. This URL is required to be a direct link of the file.
var exportFolderId = "*****"; // Please set folder ID for saving the downloaded file.
var email = ""; // Please set email address. This is used for sending email when the download is completed and an error occurs.
var functionName = "main"; // If you want to modify the function name, please also modify this.
try {
clearTrigger(functionName);
var prop = PropertiesService.getScriptProperties();
var tempFolderId = prop.getProperty("tempFolderId");
var resource = {
url: url,
exportFolderId: exportFolderId,
tempFolderId: tempFolderId || "",
};
var r = DownloadLargeFilesByUrl.download(resource); // Here, this library is used.
if (!resource.tempFolderId) {
prop.setProperty("tempFolderId", r.tempFolderId);
}
var t = new Date();
t.setMinutes(t.getMinutes() + 3); // This means that the script run again after 3 minutes later.
ScriptApp.newTrigger(functionName).timeBased().at(t).create();
if (r.nextDownload == -1 && r.nextJoin == -1) {
clearEnv(prop, functionName);
MailApp.sendEmail({
to: email,
subject: "Download was completed.",
body: JSON.stringify(r),
});
}
} catch (e) {
clearEnv(prop, functionName);
MailApp.sendEmail({
to: email,
subject: "Error occurred.",
body: JSON.stringify(e),
});
}
}

+1 for this. I have exactly the same use case - I need to pass an oauth bearer token through headers with a download request, and would normally use UrlFetchApp.fetch(url,options) where options is an object containing a stringified object for headers, and also request type.


Is it supported for angular 2?

Is it supported for angular 2? please let me know @akfish

It should. It supports browser and node environment. The web framework shouldn't matter.


checkstyle - intial version of ruleset added

Issue #538
So I added the initial version of the checkstyle ruleset. I run in on the code, and it had many warnings, then I run in on the example code on the guildelines page and it was all good.
A few more things to mention:

it will check only Java files for now
read more on the checks here
I set the level to warning, but later it can be changed to error if you want
each warning will mention the name of the check that causes the violation so you can read about it
I went though the codebase and I haven't found any real javadoc, but it could be also configured later

To be able to run a check, you can add these lines to the main pom:
<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-checkstyle-plugin</artifactId>
            <version>3.0.0</version>
            <dependencies>
                <dependency>
                    <groupId>com.puppycrawl.tools</groupId>
                    <artifactId>checkstyle</artifactId>
                    <version>8.13</version>
                </dependency>
            </dependencies>
            <configuration>
                <configLocation>strongbox-checks.xml</configLocation>
                <consoleOutput>true</consoleOutput>
            </configuration>
        </plugin>
    </plugins>
</build>

And run checkstyle:check. You will see the list of warnings in the console. :)

@leonorader,
I've had a look at the pull and it looks good (although, I haven't yet had a chance to try out myself)! One small request -- would you mind moving the maven-checkstyle-plugin into a Maven <profile> called checkstyle? We wouldn't want to run this upon every build. It would have to run only on our CI server, as it would otherwise slow things down significantly.

sure! would you like to have this profile in the parent pom?

Yeah, I think that would be the best place! Thanks! :)

Hi,
Could you please sign our ICLA?

Of course! :)
Anyway could you please help why Jenkins has test failures? I really don't know what I should do exactly.
2 tests failed in this file: org.carlspring.strongbox.providers.repository.MavenProxyRepositoryProviderTestIT.whenDownloadingArtifactMetadataFileShouldBeMergedWhenExist

Please, disregard these two failures.


Correct the links in docs referring to python tutorials

Jupyter notebooks were moved to h3-py-notebooks repository.

Coverage remained the same at 99.13% when pulling daab8392afdad7269f328f2eb0097d925cae7a6b on rpitonak:docs/python-tutorials-links into f621d07cbf15c3b78243b429f24eca009bdb1f13 on uber:master.

Thanks for fixing this! I'll redeploy the website with the fix now.


merging sign and monotonicity columns into "DCP properties"

Description
This PR is a follow-up to #2563 in hopes of addressing the suggestions in #2559.
The changes are made for two tables in the "atomic functions" section of the docs.
@rileyjmurray @SteveDiamond what do you think of the combined DCP properties column?
I made some slight adjustments, adding "sign" and "monotonicity" when it was otherwise ambiguous.
Also for previously None instances, I just deleted those from the table.
Link for preview (everything fits without scrolling for the atoms table!).
This PR also fixes the solver support table (see below).

Type of change

[ ] New feature (backwards compatible)
[ ] New feature (breaking API changes)
[ ] Bug fix
[x] Other (Documentation, CI, ...)

Contribution checklist

[x] Add our license to new files.
[x] Check that your code adheres to our coding style.
[x] Write unittests.
[x] Run the unittests and check that they’re passing.
[x] Run the benchmarks to make sure your change doesn’t introduce a regression.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.William Zijie Zhang seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

Should the readthedocs be there?

Should the readthedocs be there?

That needs to change before merge

Should the readthedocs be there?

That needs to change before merge

done. thanks for the review!


MD029: Ordered list with Note block

I encountered an issue with Markdown rendering where I have an ordered list in my Markdown file, separated by a note block. Despite the correct content, it generates the MD029 warning.
Original:
1. Fist item:

> [!NOTE]
> Some note

2. second item

It is rendered as

Fist item:

[!NOTE]
Some note

second item

However, this setup triggers the MD029 warning. According to MD029, a solution is proposed to indent the code block so it becomes part of the preceding list item as intended:
1. Fist item:

    > [!NOTE]
    > Some note

2. second item

However, after applying this solution, the note is rendered incorrectly:

Fist item:

[!NOTE]
Some note

second item

You'll probably want to suppress that warning in this case. It seems that GitHub only recognizes notes when they are fully left-aligned. But when that's done, it creates two separate lists and the second list begins with 2 instead of 1, so the markdownlint warning is correct. GitHub allows you to start a list with a different number than 1 which is why this looks OK rendered.


refactor test_read.py

Pull Request
Description
Refactor test_read.py - https://github.com/openclimatefix/pv-site-datamodel/blob/main/tests/test_read.py
as the file is a it long. This PR makes it more readable, and easy to run a smaller amount of tests.
Fixes #132
Checklist:

[Yes] My code follows OCF's coding style guidelines
[Yes ] I have performed a self-review of my own code
[x] I have made corresponding changes to the documentation
[x] I have added tests that prove my fix is effective or that my feature works
[Yes] I have checked my code and corrected any misspellings

Looks like there are some linting errors, see CI, can you format using make format?


Added Release.Namespace to charts

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: dan-jackson-github sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.


Fermat Method for Primality Test in Python

💥 Proposal
(A clear and concise description of what the proposal is.)
Have you read the Contributing Guidelines on Pull Requests?
(Write your answer here.)

Pls assign this to me @sujal111


Adjust the Garden Linux package repo implementation

What would you like to be added:
The current Garden Linux package repo is maintained by following Gitlab repo:

https://gitlab.com/gardenlinux/gardenlinux-package-build

and its corresponding package repos:

https://gitlab.com/gardenlinux?filter=package

The current implementation has some limitations that should be addressed before preparing Garden Linux for the next release.
Why is this needed:
Prepare the package repository for the upcoming Garden Linux release.

Quick update to this issue. The current repo implementation of Garden Linux has been adjusted and merged. It now supports the following features:

Instead of using multiple config files for managing what packages should be mirrored and included in the Garden Linux repo, there is only one configuration left, which contains all the configuration parametes.
The new implementation allows to mirror packages from multiple releases (e.g. one could mirror a single packages from debian/sid whereas all other packages are still mirrored from debian/testing
The published repo of already released Garden Linux versions (dists) can now be updated afterwards. The mirrored packages remain the same but new packages could be added for example.

There is still one PR left but after all, this issue has been completed:

https://gitlab.com/gardenlinux/gardenlinux-package-build/-/merge_requests/117


switch to Smith-Waterman for adapter matching

The current adapter matching routine is very simple. It looks for a match (within a Hamming distance) on either side of the junction adapter. If a match to either 19bp side of the adapter is found, the entire 38bp junction adapter is clipped appropriately  (basically a crude form of seed and extend).  This can miss cases were both sides of the junction adapter have an indel error.
Smith-Waterman alignment will improve this.

@mmokrejs the Smith-Waterman alignment appears to be working well if you are still interested. It can be turned on with the -w flag.
Usage example:
## SW for adapter matching
../nxtrim -s .7 -w -1 EcMG1_ATGTCA_L001_R1_001.fastq.gz -2 EcMG1_ATGTCA_L001_R2_001.fastq.gz --stdout-mp | bwa mem EcMG.fna -p - | gzip -1 > mp.sw.bam
 ../nxtrim -s .7 -w -1 EcMG1_ATGTCA_L001_R1_001.fastq.gz -2 EcMG1_ATGTCA_L001_R2_001.fastq.gz --stdout-un | bwa mem EcMG.fna -p - | gzip -1 > un.sw.bam

I used nxtrim -s .7 -w --separate --preserve-mp --rf for the testing of version v0.4.1-4965b00.
I still see plenty linkers unremoved with minimum similarity 86.11%.  Anyway, not a single-one should slip out.
HFYJ5AFXX.1_5kb_R1.sw_missed_linker.fastq.txt
HFYJ5AFXX.1_5kb_R2.sw_missed_linker.fastq.txt

HFYJ5AFXX.1_5kb.trimmomatic.nxtrim_R2.pe.missed_linkers.blastn.txt
HFYJ5AFXX.1_5kb.trimmomatic.nxtrim_R2.mp.missed_linkers.blastn.txt
HFYJ5AFXX.1_5kb.trimmomatic.nxtrim_R1.unknown.missed_linkers.blastn.txt
HFYJ5AFXX.1_5kb.trimmomatic.nxtrim_R1.pe.missed_linkers.blastn.txt
HFYJ5AFXX.1_5kb.trimmomatic.nxtrim_R1.mp.missed_linkers.blastn.txt
HFYJ5AFXX.1_5kb.trimmomatic.nxtrim_R2.unknown.missed_linkers.blastn.txt

In terms of practical performance, I am pretty happy at this stage and probably won't be working on this for the time being.  The assemblies that come out of this are very good for short read sequence data.
I can align the NxTrim output back to the reference and check the proportion of RF against FR:
joconnell@ubuntu:~/workspace/NxTrim/test$ /usr/bin/python alignment_summary.py mp.sw.bam
Orientation  Frequency   Median insert size
FR              392764                 3384
RF                1706              1242515
FF                2150              1009331
RR                2072               942504

RF/(FR+RF) = 0.004325
joconnell@ubuntu:~/workspace/NxTrim/test$ /usr/bin/python alignment_summary.py un.sw.bam
Orientation  Frequency   Median insert size
FR              110925                 3193
RF                 710               712807
FF<PHONE_NUMBER>
RR<PHONE_NUMBER>

RF/(FR+RF) = 0.006360

The RF here are nominally incorrectly orientated pairs. For confirmed mate-pairs, the RF proportion is less than both RR and FF suggesting very clean data. For unknown, it is slightly elevated, but this is a known limitation of pairs with undetected adapters.
The blast evaluation only shows sensitivity. You need to balance this against how many of those blast hits are not actual adapters. These adapters share substrings with actual genomic DNA once you get down to <=10bp.
You also need to check how many of those hits are from read pairs with additional adapter copies, these are hard to handle.

The blast evaluation only shows sensitivity. You need to balance this against how many of those blast hits are not actual adapters. These adapters share substrings with actual genomic DNA once you get down to <=10bp.

No, the matches are much longer, and second, I do not mind discarding valid DNA if it is similar tot he linker. That DNA will come into the assembly from a whole genome shotgun library, not from MatePaired library. That is the key point. Trying to "protect" possibly valid matches is just wrong apporach. MatePair library data are just not proper source of sequence if some locus has similar sequence.
>NB501598:62:HFYJ5AFXX:1:11205:23960:15051 1:N:0:GCCAAT
          Length = 26

 Score = 52.0 bits (26), Expect = 5e-06
 Identities = 26/26 (100%)
 Strand = Plus / Plus

Query: 12 acacatctagatgtgtataagagaca 37
          ||||||||||||||||||||||||||
Sbjct: 1  acacatctagatgtgtataagagaca 26

and the worst from HFYJ5AFXX.1_5kb.trimmomatic.nxtrim_R1.mp.missed_linkers.blastn.txt file:
>NB501598:62:HFYJ5AFXX:1:11101:4458:1807 1:N:0:GCCAAT
          Length = 155

 Score = 32.2 bits (16), Expect = 4.4
 Identities = 16/16 (100%)
 Strand = Plus / Plus

Query: 8   ttatacacatctagat 23
           ||||||||||||||||
Sbjct: 139 ttatacacatctagat 154

Anyway, in both cases the match is clearly at the end/beginning of the raw read, so you are missing incomplete matches.

You also need to check how many of those hits are from read pairs with additional adapter copies, these are hard to handle.

Didn't you say elsewhere nxtrim discards read mate if after trimming a match remains? I wouldn't do anything different to it. Run blast, find matching readnames, and discard them, before feeding assembler.

Sorry, I think at this point you need to try another tool. Probably the trimming can be improved (albeit marginally) but the current implementation already produces very good assemblies.

No, the matches are much longer, and second, I do not mind discarding valid DNA if it is similar tot he >linker. That DNA will come into the assembly from a whole genome shotgun library, not from >MatePaired library. That is the key point. Trying to "protect" possibly valid matches is just wrong >apporach. MatePair library data are just not proper source of sequence if some locus has similar >sequence.

NxTrim is targeted at assembly using just a single Nextera MP library, not scaffolding. Maybe check out NextClip for scaffolding.
Best of luck!

nxtrim is clearly missing partial while exact matches on edges, the alignments show the Smith-Waterman somehow misses even mismatches, and you say from nucmbers of contigs/scaffodls that the assembly is good? And you did not answer why the above examples were missed. Did you try the tescases I prepared for you?
Thank you for pointing me to yet another partially complete tool. It would have been much more helpful if nxtrim supported a simple CSV input file format telling it where too look for linkers. It can do its own alignments but before starting splitting the reads it could "inject" the coordinates into its decision. That way one could provide a path list with coordinates one wants to use for splitting. I already proposed this before.

Sorry, I could not actually reproduce your first test case:
grep -A3 "NB501598:62:HFYJ5AFXX:1:11205:23960:15051" ~/Downloads/HFYJ5AFXX.1_5kb_R1.sw_missed_linker.fastq.txt > r1.fq
grep -A3 "NB501598:62:HFYJ5AFXX:1:11205:23960:15051" ~/Downloads/HFYJ5AFXX.1_5kb_R2.sw_missed_linker.fastq.txt > r2.fq

./nxtrim -1 r1.fq -2 r2.fq -O test

$ zcat test.pe.fastq.gz
@NB501598:62:HFYJ5AFXX:1:11205:23960:15051 1:N:0:GCCAAT
GTTCTTGTCTCCTATACACTGCTTGTTCACTGTGTGTTTGCTAGTCGTTGTTTTATGTCACCTTCTTGTTTAGCCTGCGTTTCCTCGTGTTTTGCTTGCTTTGTTTTATTGTTTTGTACTTTGTTAAT
+
EEEEEEEEEAEEEEEEEAEEEEEEEEEEEEEEEAEEEEEEAEE<E/EEEEEE/AAEEEEAEA<E<EEEEEEEEEEEE<EEEEEAEAEEEE/AEE<EE<AE/AAEEEA//EEEEEAEE/6/AAAEEEAE
@NB501598:62:HFYJ5AFXX:1:11205:23960:15051 2:N:0:GCCAAT
AGCATTAAAAGCATTTTCAATTTAGTGTTAAATATTGGATATATAGCCATTTGCTTTTAGTTTACATAAAAAAATCATTTAAATGAAGAATAAAAAAATCAGGTGTTTATAATTCAAAATTTTTATACATTGGGCAAATTTTTCAACACACAGAC
+
AAAAAEEEEEEEEEEEEE666/EAE6E6AEEEAEEE/E/AAEEE/E<//6E/EEEEEEE6EEA/E/EE//AAAEEE<EEEAEEEEEEEE6/EE//</EE//EEEEE<E<<E//EEEE///EEEA<<A//EEEAA<AE<E<EEA<//E///</E<<

#the string you find with blast
$ zcat test.pe.fastq.gz  | grep ACACATCTAGATGTGTATAAGAGACA | wc -l
0

Maybe stick to --separate --preserve-mp --rf?
$ grep NB501598:62:HFYJ5AFXX:1:11205:23960:15051 HFYJ5AFXX.1_5kb.trimmomatic.nxtrim_*.fastq
HFYJ5AFXX.1_5kb.trimmomatic.nxtrim_R1.mp.fastq:@NB501598:62:HFYJ5AFXX:1:11205:23960:15051 1:N:0:GCCAAT
HFYJ5AFXX.1_5kb.trimmomatic.nxtrim_R2.mp.fastq:@NB501598:62:HFYJ5AFXX:1:11205:23960:15051 2:N:0:GCCAAT
$

You have in assigned under PE instead of MP.

I still get a PE with those commands. As you say, the adapter is at the start of the read, so it has to be a PE (unless it misses the adapter).
$ grep -A3 "NB501598:62:HFYJ5AFXX:1:11205:23960:15051" ~/Downloads/HFYJ5AFXX.1_5kb_R1.sw_missed_linker.fastq.txt > r1.fq
$ grep -A3 "NB501598:62:HFYJ5AFXX:1:11205:23960:15051" ~/Downloads/HFYJ5AFXX.1_5kb_R2.sw_missed_linker.fastq.txt > r2.fq

joconnell@ubuntu:~/workspace/NxTrim$ ./nxtrim  -1 r1.fq -2 r2.fq -O test --separate --preserve-mp --rf
Output: test.*.fastq.gz
Trimming:
R1:	r1.fq
R2:	r2.fq
--preserve-mp is on: will favour MPs over PEs

Trimming summary:
1 / 1	( 100.00% )	reads passed chastity/purity filters.
0 / 1	( 0.00% )	reads had TWO copies of adapter (filtered).
0 / 1	( 0.00% )	read pairs were ignored because template length appeared less than read length
1 remaining reads were trimmed

0 / 1	( 0.00% )	read pairs had MP orientation
1 / 1	( 100.00% )	read pairs had PE orientation
0 / 1	( 0.00% )	read pairs had unknown orientation
0 / 1	( 0.00% )	were single end reads

0 / 1	( 0.00% )	extra single end reads were generated from overhangs
joconnell@ubuntu:~/workspace/NxTrim$
joconnell@ubuntu:~/workspace/NxTrim$
joconnell@ubuntu:~/workspace/NxTrim$ zcat test_R1.pe.fastq.gz
@NB501598:62:HFYJ5AFXX:1:11205:23960:15051 1:N:0:GCCAAT
ATTAACAAAGTACAAAACAATAAAACAAAGCAAGCAAAACACGAGGAAACGCAGGCTAAACAAGAAGGTGACATAAAACAACGACTAGCAAACACACAGTGAACAAGCAGTGTATAGGAGACAAGAAC
+
EAEEEAAA/6/EEAEEEEE//AEEEAA/EA<EE<EEA/EEEEAEAEEEEE<EEEEEEEEEEEE<E<AEAEEEEAA/EEEEEE/E<EEAEEEEEEAEEEEEEEEEEEEEEEAEEEEEEEAEEEEEEEEE
joconnell@ubuntu:~/workspace/NxTrim$ zcat test_R2.pe.fastq.gz
@NB501598:62:HFYJ5AFXX:1:11205:23960:15051 2:N:0:GCCAAT
GTCTGTGTGTTGAAAAATTTGCCCAATGTATAAAAATTTTGAATTATAAACACCTGATTTTTTTATTCTTCATTTAAATGATTTTTTTATGTAAACTAAAAGCAAATGGCTATATATCCAATATTTAACACTAAATTGAAAATGCTTTTAATGCT
+
<<E/<///E//<AEE<E<EA<AAEEE//A<<AEEE///EEEE//E<<E<EEEEE//EE/<//EE/6EEEEEEEEAEEE<EEEAAA//EE/E/AEE6EEEEEEE/E6//<E/EEEAA/E/EEEAEEEA6E6EAE/666EEEEEEEEEEEEEAAAAA
joconnell@ubuntu:~/workspace/NxTrim$ zcat test_R1.pe.fastq.gz | grep ACACATCTAGATGTGTATAAGAGACA | wc -l
0
joconnell@ubuntu:~/workspace/NxTrim$ zcat test_R2.pe.fastq.gz | grep ACACATCTAGATGTGTATAAGAGACA | wc -l
0

Here is what I fed into nxtrim.
$ grep -A 3 NB501598:62:HFYJ5AFXX:1:11205:23960:15051 HFYJ5AFXX.1_5kb_R?.trimmomatic.?E.fastq
HFYJ5AFXX.1_5kb_R1.trimmomatic.PE.fastq:@NB501598:62:HFYJ5AFXX:1:11205:23960:15051 1:N:0:GCCAAT
HFYJ5AFXX.1_5kb_R1.trimmomatic.PE.fastq-ACACATCTAGATGTGTATAAGAGACATGTTCTTGTCTCCTATACACTGCTTGTTCACTGTGTGTTTGCTAGTCGTTGTTTTATGTCACCTTCTTGTTTAGCCTGCGTTTCCTCGTGTTTTGCTTGCTTTGTTTTATTGTTTTGTACTTTGTTAAT
HFYJ5AFXX.1_5kb_R1.trimmomatic.PE.fastq-+
HFYJ5AFXX.1_5kb_R1.trimmomatic.PE.fastq-AAAA6EEAEA/AEEEEAEEEEEEEEEEEEEEEEEEEAEEEEEEEAEEEEEEEEEEEEEEEAEEEEEEAEE<E/EEEEEE/AAEEEEAEA<E<EEEEEEEEEEEE<EEEEEAEAEEEE/AEE<EE<AE/AAEEEA//EEEEEAEE/6/AAAEEEAE
--
HFYJ5AFXX.1_5kb_R2.trimmomatic.PE.fastq:@NB501598:62:HFYJ5AFXX:1:11205:23960:15051 2:N:0:GCCAAT
HFYJ5AFXX.1_5kb_R2.trimmomatic.PE.fastq-AGCATTAAAAGCATTTTCAATTTAGTGTTAAATATTGGATATATAGCCATTTGCTTTTAGTTTACATAAAAAAATCATTTAAATGAAGAATAAAAAAATCAGGTGTTTATAATTCAAAATTTTTATACATTGGGCAAATTTTTCAACACACAGAC
HFYJ5AFXX.1_5kb_R2.trimmomatic.PE.fastq-+
HFYJ5AFXX.1_5kb_R2.trimmomatic.PE.fastq-AAAAAEEEEEEEEEEEEE666/EAE6E6AEEEAEEE/E/AAEEE/E<//6E/EEEEEEE6EEA/E/EE//AAAEEE<EEEAEEEEEEEE6/EE//</EE//EEEEE<E<<E//EEEE///EEEA<<A//EEEAA<AE<E<EEA<//E///</E<<
$

But you have the sequences reverse-complemented.

I don't know what is happening, but I am finding the adapter in that example and the reads look the same:
joconnell@ubuntu:~/workspace/NxTrim$ cat r1.fq
@NB501598:62:HFYJ5AFXX:1:11205:23960:15051 1:N:0:GCCAAT
ACACATCTAGATGTGTATAAGAGACATGTTCTTGTCTCCTATACACTGCTTGTTCACTGTGTGTTTGCTAGTCGTTGTTTTATGTCACCTTCTTGTTTAGCCTGCGTTTCCTCGTGTTTTGCTTGCTTTGTTTTATTGTTTTGTACTTTGTTAAT
+
AAAA6EEAEA/AEEEEAEEEEEEEEEEEEEEEEEEEAEEEEEEEAEEEEEEEEEEEEEEEAEEEEEEAEE<E/EEEEEE/AAEEEEAEA<E<EEEEEEEEEEEE<EEEEEAEAEEEE/AEE<EE<AE/AAEEEA//EEEEEAEE/6/AAAEEEAE

joconnell@ubuntu:~/workspace/NxTrim$ cat r2.fq
@NB501598:62:HFYJ5AFXX:1:11205:23960:15051 2:N:0:GCCAAT
AGCATTAAAAGCATTTTCAATTTAGTGTTAAATATTGGATATATAGCCATTTGCTTTTAGTTTACATAAAAAAATCATTTAAATGAAGAATAAAAAAATCAGGTGTTTATAATTCAAAATTTTTATACATTGGGCAAATTTTTCAACACACAGAC
+
AAAAAEEEEEEEEEEEEE666/EAE6E6AEEEAEEE/E/AAEEE/E<//6E/EEEEEEE6EEA/E/EE//AAAEEE<EEEAEEEEEEEE6/EE//</EE//EEEEE<E<<E//EEEE///EEEA<<A//EEEAA<AE<E<EEA<//E///</E<<

I suspect your cases where you can blast out some adapters are of the "multiple adapter" sort. Such as this one (from my E.coli):
raw:
@M903:194:000000000-A5FBD:1:1112:6063:10169 1:N:0:ATGTCA
GCTGGAACACAGCCACTGGCCGCGATTTCTGAGGCTTCTTTACCGCTAATCTCTTGCCGCATCCATTCCCGCGCAGCTTCTGGCGACAGTACCAGTCTGTCTCTTATACACATCTAGATGTGTATAAGAGACAGAGATGTGTATAAGATAC
+
AAA@AFFF1CAF1FGF11FGCEGEAEFGEGHGCFGCFGFHH2GE?EAECHHHHHHHHGGGEEGGHFHHFHGGCEEGEFFDGHFFEGGGGHFEFHHHGFHHHHHHFHHHEHFDCHHGFHHGHDFFGHHHHHHFHHFHHHGHEFHFHGHHGHB
joconnell@ubuntu:~/workspace/NxTrim/test$ cat r2.fq
@M903:194:000000000-A5FBD:1:1112:6063:10169 2:N:0:ATGTCA
TATCGTAGATTCAGTATTAGATGTGACTTTATTTAATTGAACCGGCTGAGTTTTTAACGAAAGATCATTTAATGGAAACATTTACACTCCCTGTATCTTATACACATCTCTGTCTCTTATACACATCTAGATGTGTATAAGAGACAGACTG
+
AA1AAF>A>DFFDE33BE333AFDFF3AGFFHBEFHCFGH11G0EEEGCFDGBGECGHGGGAEGGFHHFFFHHHBB1GBGHHBGHHGHFHHHHGFFFGHHHFHFHFHEHHHEGHFHFGGHGBGHFHHFHHHHG2BBGHHHHHHG00B<GFG

where are the adapters?
R1:
                                                                                perfect match                          partial overlap
                                                                                CTGTCTCTTATACACATCTAGATGTGTATAAGAGACAG AGATGTGTATAAGAGACAG
GCTGGAACACAGCCACTGGCCGCGATTTCTGAGGCTTCTTTACCGCTAATCTCTTGCCGCATCCATTCCCGCGCAGCTTCTGGCGACAGTACCAGT|CTGTCTCTTATACACATCTAGATGTGTATAAGAGACAG|AGATGTGTATAAGATAC

                                                                                imperfect match     perfect match
R2:                                                                                CTGTCTCTTATACACATCT CTGTCTCTTATACACATCTAGATGTGTATAAGAGACAG
TATCGTAGATTCAGTATTAGATGTGACTTTATTTAATTGAACCGGCTGAGTTTTTAACGAAAGATCATTTAATGGAAACATTTACACTCC|CTGTATCTTATACACATCT|CTGTCTCTTATACACATCTAGATGTGTATAAGAGACAG|ACTG

The code only catches these weird (unexpected) second adapter copies when there is a perfect string match, so in cases like above, they are missed. Yes, I could blast or sw-align, but they are so rare I just used a simple string find to catch a few of them. This could obviously be improved.

Yes, I could blast or sw-align, but they are so rare I just used a simple string find to catch a few of them. This could obviously be improved.

You probably think of me that I am nit picking but isn't this Issue about switch to Smith-Waterman matching? I thought everything in the code uses SW now. That is expected. But I am happy to help your finiding these places, one by one. ;)
I did not read the code but I fear you are getting just a single match from SW, you really need to ask for all matches (you need the guarantee that you get them all). If not you are out of luck. I spent several years on developing a tool to clean Roche/454 data and in principle I was hitting same issues.
Practically, in this case, you are either thorough and look for the full-length junction adapter, then look for those halves, merge the regions eventually into a single range (allowing say 3nt in between if the alignment was narrower for some reason). Then you continue with usual splitting as you do in your code, which notably evaluates whether any sequence was left leftwards and rightwards from the splitting region. Alternatively you can throw away this particular mate, optionally whole pair (some user might prefer that for a good reason).
I wouldn't be so picky but if the linkers remain in the reads then you have mismerged contigs. It depends on the coverage and aligner, bt basically if you start assembly with these 'chimeric' reads with linker unremoved, you won't get anything good out. Assemblers can overcome some unremoved sequencing adapters provided they appear at the end of reads. This is not the case with linkers. For me 118k unremoved linkers is too much. It does not matter out of how many input reads. Assemblers will respect this as true sample sequence. They are more likely to normalize the "clean" reads and enrich these reads with unremoved linkers.

Nitpicking is fine, but it looks like we have been talking about two different things. The tool does find adapters on the edges, but it doesn't find all the weird cases of extra unexpected adapters. I use SW (or Hamming match) to find the single best match to the adapter. This is trimmed appropriately. Then, as a way to catch the edge case of an extra unexpected adapter (as in my above example), the code looks for a perfect match (regardless of whether SW/hamming is used - this has not changed).
This small bit of code to catch this is here:
https://github.com/sequencing/NxTrim/blob/master/matepair.cpp#L545-L550
So yes, this will only catch perfect match adapters after the primary adapter has been removed. I guess your remaining adapters are mostly these multi-adapter cases with some amount of mismatch in the second copy of the adapter. I would advocate just throwing the entire pair out, they just seem untrustworthy.
I'm happy to take a pull request if you want to try some solutions. I don't have a lot of time to spend on this at the moment.
FYI. I don't know what you are assembling, but I currently get the best results for bacteria using SPAdes and --justmp. The pe/se libraries don't seem to help SPAdes.

I would advocate just throwing the entire pair out, they just seem untrustworthy.

In principle I am fine even with this approach, as a quick-and-dirty way to get data clean. Which is what we need, no matter some tiny data is lost.
Would it be relatively quick to implement (as those matches were just adjacent to the main match), I would prefer the range-based approach as some more real solution. This is not a corner-case IMHO. That works well. If you have two isolated ranges from each other then definitely the read/pair should be discarded. But in those adjacent cases you can save some sequence and in turn, good pairs.
This merely means you are not looping to get all matches out of the aligner. That is bad.
If you could replace the string exact matching approach it would be really helpful. I would say it is the priority number one now. Why cannot you just call the aligner on the upstream and downstream sequence in about the lines L545-L550? And if that finds a match, you return(1) as for the exact match? ;-)
Unfortunately I do not know C/C++ so I cannot help you here. I am certainly willing to test and help with other ideas. And I do not see almost any comments in the code, I mean useful comments. ;-)
I would say, this thread is also a proper place to document where elseare the exact string matches hiding.

BTW, I have a polyploid genome, fish. Previous, possibly velvet-based assembly is crap, full of gaps which are not supported in my eyes by reads (I have lots of readsspanning the gap proving there is not gap at all in the sequence). I assume something was wrong with scaffolding, but actually that must have been an issue during contigging because the contigs just should have been wider (the RapidRun reads just bear the whole chunk of sequence). Something was terribly wrong during data preparationIMHO, or wrong cmdline arguments for assembler, who knows. I did not receive any documentation what the previous person did.

@mmokrejs try this branch
git pull
git checkout iss37
make clean
make

The code now just runs the adapter detection routine twice. After the first run, the adapter is N masked, if another adapter copy is found, the read is discarded. This probably doubles the run-time at the moment.

$ gdb NxTrim/nxtrim core.31390
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-83.el6)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /scratch/work/project/bio/open-9-41/tt_16D1C3L12/mate_pairs_origo/NxTrim/nxtrim...done.
[New Thread 31390]

warning: Ignoring non-absolute filename: <linux-vdso.so.1>
Missing separate debuginfo for linux-vdso.so.1
Try: yum --enablerepo='*-debug*' install /usr/lib/debug/.build-id/79/86395b139f22da92f9154f30c5951e68b4b7f6
Missing separate debuginfo for /apps/gentoo/lib64/ld-linux-x86-64.so.2
Try: yum --enablerepo='*-debug*' install /usr/lib/debug/.build-id/79/86395b139f22da92f9154f30c5951e68b4b7f6
Reading symbols from /apps/gentoo/usr/lib64/libz.so.1...(no debugging symbols found)...done.
Loaded symbols for /apps/gentoo/usr/lib64/libz.so.1
Reading symbols from /apps/gentoo/usr/lib/gcc/x86_64-pc-linux-gnu/4.9.4/libstdc++.so.6...done.
Loaded symbols for /apps/gentoo/usr/lib/gcc/x86_64-pc-linux-gnu/4.9.4/libstdc++.so.6
Reading symbols from /apps/gentoo/lib64/libm.so.6...done.
Loaded symbols for /apps/gentoo/lib64/libm.so.6
Reading symbols from /apps/gentoo/usr/lib/gcc/x86_64-pc-linux-gnu/4.9.4/libgcc_s.so.1...done.
Loaded symbols for /apps/gentoo/usr/lib/gcc/x86_64-pc-linux-gnu/4.9.4/libgcc_s.so.1
Reading symbols from /apps/gentoo/lib64/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /apps/gentoo/lib64/libc.so.6
Reading symbols from /apps/gentoo/lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /apps/gentoo/lib64/ld-linux-x86-64.so.2
Core was generated by `/scratch/work/project/bio/open-9-41/tt_16D1C3L12/mate_pairs_origo/NxTrim/nxtrim'.
Program terminated with signal 6, Aborted.
#0  0x00007fffce2a6044 in raise () from /apps/gentoo/lib64/libc.so.6
(gdb) where
#0  0x00007fffce2a6044 in raise () from /apps/gentoo/lib64/libc.so.6
#1  0x00007fffce2a74aa in abort () from /apps/gentoo/lib64/libc.so.6
#2  0x00007fffce29edcd in __assert_fail_base () from /apps/gentoo/lib64/libc.so.6
#3  0x00007fffce29ee82 in __assert_fail () from /apps/gentoo/lib64/libc.so.6
#4  0x0000000000407abe in sw_match(unsigned char*, int, unsigned char*, int, int, float, signed char*) () at matepair.cpp:105
#5  0x0000000000407df6 in matePair::findAdapter(std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, int, float, bool) () at matepair.cpp:187
#6  0x0000000000409b55 in matePair::build(readPair&, int, float, int, bool, bool, bool, bool) () at matepair.cpp:578
#7  0x000000000040312f in main () at nxtrim.cpp:176
(gdb) bt full
#0  0x00007fffce2a6044 in raise () from /apps/gentoo/lib64/libc.so.6
No symbol table info available.
#1  0x00007fffce2a74aa in abort () from /apps/gentoo/lib64/libc.so.6
No symbol table info available.
#2  0x00007fffce29edcd in __assert_fail_base () from /apps/gentoo/lib64/libc.so.6
No symbol table info available.
#3  0x00007fffce29ee82 in __assert_fail () from /apps/gentoo/lib64/libc.so.6
No symbol table info available.
#4  0x0000000000407abe in sw_match(unsigned char*, int, unsigned char*, int, int, float, signed char*) () at matepair.cpp:105
        std::__ioinit = {static _S_refcount = 5, static _S_synced_with_stdio = true}
        adapter1 = {static npos =<PHONE_NUMBER>3709551615, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x614028 "CTGTCTCTTATACACATCT"}}
        adapter2 = {static npos =<PHONE_NUMBER>3709551615, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x614068 "AGATGTGTATAAGAGACAG"}}
        r2_external_adapter = {static npos =<PHONE_NUMBER>3709551615, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x614148 "GATCGGAAGAGCGTCGTGTAGGGAAAGAGTGT"}}
        seq_nt4_table = '\004' <repeats 65 times>, "\000\004\001\004\004\004\002", '\004' <repeats 12 times>, "\003", '\004' <repeats 12 times>, "\000\004\001\004\004\004\002", '\004' <repeats 12 times>, "\003", '\004' <repeats 139 times>
        r1_external_adapter = {static npos =<PHONE_NUMBER>3709551615, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x6140f8 "GATCGGAAGAGCACACGTCTGAACTCCAGTCAC"}}
        adapterj = {static npos =<PHONE_NUMBER>3709551615, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x6140a8 "CTGTCTCTTATACACATCTAGATGTGTATAAGAGACAG"}}
#5  0x0000000000407df6 in matePair::findAdapter(std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, int, float, bool) () at matepair.cpp:187
        std::__ioinit = {static _S_refcount = 5, static _S_synced_with_stdio = true}
        adapter1 = {static npos =<PHONE_NUMBER>3709551615, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x614028 "CTGTCTCTTATACACATCT"}}
        adapter2 = {static npos =<PHONE_NUMBER>3709551615, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x614068 "AGATGTGTATAAGAGACAG"}}
        r2_external_adapter = {static npos =<PHONE_NUMBER>3709551615, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x614148 "GATCGGAAGAGCGTCGTGTAGGGAAAGAGTGT"}}
        seq_nt4_table = '\004' <repeats 65 times>, "\000\004\001\004\004\004\002", '\004' <repeats 12 times>, "\003", '\004' <repeats 12 times>, "\000\004\001\004\004\004\002", '\004' <repeats 12 times>, "\003", '\004' <repeats 139 times>
        r1_external_adapter = {static npos =<PHONE_NUMBER>3709551615, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x6140f8 "GATCGGAAGAGCACACGTCTGAACTCCAGTCAC"}}
        adapterj = {static npos =<PHONE_NUMBER>3709551615, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x6140a8 "CTGTCTCTTATACACATCTAGATGTGTATAAGAGACAG"}}
#6  0x0000000000409b55 in matePair::build(readPair&, int, float, int, bool, bool, bool, bool) () at matepair.cpp:578
        std::__ioinit = {static _S_refcount = 5, static _S_synced_with_stdio = true}
        adapter1 = {static npos =<PHONE_NUMBER>3709551615, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x614028 "CTGTCTCTTATACACATCT"}}
        adapter2 = {static npos =<PHONE_NUMBER>3709551615, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x614068 "AGATGTGTATAAGAGACAG"}}
        r2_external_adapter = {static npos =<PHONE_NUMBER>3709551615, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x614148 "GATCGGAAGAGCGTCGTGTAGGGAAAGAGTGT"}}
        seq_nt4_table = '\004' <repeats 65 times>, "\000\004\001\004\004\004\002", '\004' <repeats 12 times>, "\003", '\004' <repeats 12 times>, "\000\004\001\004\004\004\002", '\004' <repeats 12 times>, "\003", '\004' <repeats 139 times>
        r1_external_adapter = {static npos =<PHONE_NUMBER>3709551615, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x6140f8 "GATCGGAAGAGCACACGTCTGAACTCCAGTCAC"}}
        adapterj = {static npos =<PHONE_NUMBER>3709551615, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x6140a8 "CTGTCTCTTATACACATCTAGATGTGTATAAGAGACAG"}}
#7  0x000000000040312f in main () at nxtrim.cpp:176
        std::__ioinit = {static _S_refcount = 5, static _S_synced_with_stdio = true}
(gdb)

OK, here is a better stacktrace from CXXFLAGS="-ggdb -O0" binary:
(gdb) bt full
#0  0x00007fffce385625 in raise () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007fffce386e05 in abort () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007fffce37e74e in __assert_fail_base () from /lib64/libc.so.6
No symbol table info available.
#3  0x00007fffce37e810 in __assert_fail () from /lib64/libc.so.6
No symbol table info available.
#4  0x0000000000407ba4 in sw_match (target=0x80b740 '\004' <repeats 22 times>, tlen=22, query=0x6235f0 "\001\003\002\003\001\003\001\003\003", qlen=38, minoverlap=12, min_similarity=0.699999988, mat=0x7fffffffda40 "\001\376\376\376") at matepair.cpp:105
        sa = 1
        go = 1
        ge = 3
        substring_length = -1
        sim = -0
        __PRETTY_FUNCTION__ = "int sw_match(uint8_t*, int, uint8_t*, int, int, float, int8_t*)"
        sb = 2
        r = {score = 0, te = -1, qe = 0, score2 = -1, te2 = -1, tb = 0, qb = 0}
#5  0x0000000000407ed8 in matePair::findAdapter (this=0x7fffffffd8e0, s='N' <repeats 22 times>, minoverlap=12, similarity=0.699999988, use_hamming=false) at matepair.cpp:187
        L2 = 19
        perfect =<PHONE_NUMBER>
        s_tmp = 0x80b740 '\004' <repeats 22 times>
        a = 4209342
        L1 = 22
#6  0x0000000000409e85 in matePair::build (this=0x7fffffffd8e0, readpair=..., minovl=12, sim=0.699999988, ml=21, jr=false, uh=false, pmp=true, jmp=false) at matepair.cpp:586
        tmp = {l = 22, filtered = false, description = true, h = "NB501598:62:HFYJ5AFXX:1:11107:2087:3600 2:N:0:CTTGTA", s = 'N' <repeats 22 times>, l3 = "+", q = '#' <repeats 22 times>}
        L2 = 22
        a1 = 38
        b2 = 30
        rc2 = {l = 155, filtered = false, description = false, h = "", s = "", l3 = "", q = "CAGCATACTTCCCAGCATGCCAACTATCCCCCTCCCTCCTGTCTCTTATACACAACTAGATGTGTATAAGAGACAGGTCTGTGGGGTGCCCCCAAACTTTACATGAGCTTTTGCTGCAGGACAGATTTTTAG"}
        minoverlap2 = -9296
        L1 = 132
        a2 = -8
        b1 = 76
        rc1 = {l = 90, filtered = 255, description = 127, h = "", s = "", l3 = "", q = ""}
#7  0x00000000004032e4 in main (argc=12, argv=0x7fffffffde38) at nxtrim.cpp:176
        weird = false
        m = {joinreads = false, use_hamming = false, preserve_mp = true, _justmp = false, se = {l = 0, filtered = false, description = true, h = "", s = "", l3 = "", q = ""}, mp = {r1 = {l = 0, filtered = false, description = true, h = "", s = "", l3 = "", q = ""}, r2 = {l = 0, filtered = false, description = true,
              h = "", s = "", l3 = "", q = ""}, l = 0, filtered = false}, pe = {r1 = {l = 0, filtered = false, description = true, h = "", s = "", l3 = "", q = ""}, r2 = {l = 0, filtered = false, description = true, h = "", s = "", l3 = "", q = ""}, l = 0, filtered = false}, unknown = {r1 = {l = 0, filtered = false,
              description = true, h = "", s = "", l3 = "", q = ""}, r2 = {l = 0, filtered = false, description = true, h = "", s = "", l3 = "", q = ""}, l = 1, filtered = false}, minoverlap = 12, minlen = 21, similarity = 0.699999988, adapter1_sw = 0x6235b0 "\001\003\002\003\001\003\001\003\003",
          adapter2_sw = 0x6235d0 "", adapterj_sw = 0x6235f0 "\001\003\002\003\001\003\001\003\003", sw_mat = "\001\376\376\376\000\376\001\376\376\000\376\376\001\376\000\376\376\376\001\000\000\000\000\000"}
        npass = 2740
        trim_warn = false
        se_only = 470
        pos = {first = 0, second = 0}
        nweird = 5
        nread = 2740
        out = {weird = -9488, n_mp = 1507, n_unk = 76, n_se = 878, n_pe = 341, _justmp = false, _write_mp = true, _write_se = true, _write_pe = true, _write_un = true, print_to_stdout = 240, mp_out = {separate = true, outfile = {fp = 0x0, _stdout = false}, outfile1 = {fp = 0x623700, _stdout = false}, outfile2 = {
              fp = 0x6691e0, _stdout = false}}, pe_out = {separate = true, outfile = {fp = 0x0, _stdout = 208}, outfile1 = {fp = 0x73a210, _stdout = false}, outfile2 = {fp = 0x77fcf0, _stdout = false}}, unknown_out = {separate = true, outfile = {fp = 0x0, _stdout = false}, outfile1 = {fp = 0x6aec80,
              _stdout = false}, outfile2 = {fp = 0x6f4720, _stdout = false}}, se_out = {fp = 0x7c57d0, _stdout = false}}
        minlen = 21
        write_stdout_un = false
        nodata = 341
        p = {r1 = {l = 132, filtered = false, description = true, h = "NB501598:62:HFYJ5AFXX:1:11101:23244:5416 1:N:0:CTTGTA", s = "CAGCATACTTCCCAGCATGCCAACTATCCCCCTCCCTCCTGTCTCTTATACACAACTAGATGTGTATAAGAGACAGGTCTGTGGGGTGCCCCCAAACTTTACATGAGCTTTTGCTGCAGGACAGATTTTTAG", l3 = "+", q =
    "AAAA6E6EAEE/EEE/EEEAEEEAEEAE</E/AEEEEE<AAAEEEEEEEAAEEE///EEA/E/EA/E/A///E//E</A<E/////A<</E/E/A//EEA//E<///<<EEE////E<//E/</EEEE/<<<"}, r2 = {l = 22, filtered = false, description = true, h = "NB501598:62:HFYJ5AFXX:1:11107:2087:3600 2:N:0:CTTGTA", s = "TATACACATCTAGATGTGTATA", l3 = "+", q =
    "6AA6AEEAE/AEE6EEEEE6E/"}, l = 112, filtered = false}
        similarity = 0.699999988
        prefix = "HFYJ5AFXX.1_8kb.trimmomatic.PE.nxtrim"
        ignorePF = false
        write_stdout_mp = false
        loptions = {{name = 0x410620 "r1", has_arg = 1, flag = 0x0, val = 49}, {name = 0x410623 "r2", has_arg = 1, flag = 0x0, val = 50}, {name = 0x410626 "output-prefix", has_arg = 1, flag = 0x0, val = 79}, {name = 0x410634 "stdout", has_arg = 0, flag = 0x0, val = 0}, {name = 0x41063b "stdout-mp", has_arg = 0,
            flag = 0x0, val = 9}, {name = 0x410645 "stdout-un", has_arg = 0, flag = 0x0, val = 10}, {name = 0x41064f "justmp", has_arg = 0, flag = 0x0, val = 1}, {name = 0x410656 "joinreads", has_arg = 0, flag = 0x0, val = 2}, {name = 0x410660 "rf", has_arg = 0, flag = 0x0, val = 3}, {name = 0x410663 "preserve-mp",
            has_arg = 0, flag = 0x0, val = 4}, {name = 0x41066f "ignorePF", has_arg = 0, flag = 0x0, val = 5}, {name = 0x410678 "mp", has_arg = 0, flag = 0x0, val = 6}, {name = 0x41067b "unknown", has_arg = 0, flag = 0x0, val = 7}, {name = 0x410683 "separate", has_arg = 0, flag = 0x0, val = 8}, {
            name = 0x41068c "smith-waterman", has_arg = 0, flag = 0x0, val = 119}, {name = 0x41069b "similarity", has_arg = 1, flag = 0x0, val = 115}, {name = 0x4106a6 "minoverlap", has_arg = 1, flag = 0x0, val = 118}, {name = 0x4106b1 "minlength", has_arg = 1, flag = 0x0, val = 108}, {name = 0x0, has_arg = 0,
            flag = 0x0, val = 0}}
        c = -1
        justmp = false
        write_stdout = false
        infile = {f1 = 0x614270, f2 = 0x61bc10}
        joinreads = false
        r1 = 0x7fffffffe1e9 "HFYJ5AFXX.1_8kb_R1.trimmomatic.SE.fastq"
        rc = true
        hamming = false
        preserve_mp = true
        minoverlap = 12
        r2 = 0x7fffffffe214 "HFYJ5AFXX.1_8kb_R2.trimmomatic.SE.fastq"
        separate = true
(gdb)

Try again.

Works now ... I will keep the debug binary running on all data. ETA probably ~4-6hrs.

Okay. There may be no need for debug binary here. That above error was an assert failure due to a problematic SW alignment, it would have been caught in the release build.
I have already put the debug binary through valgrind without error.

OK, recompiked&restarted with CXXFLAGS="-ggdb -march=native -O2" make
Did you use for development testing the FASTQ files I uploaded above? Just to know what I shall anticipate. ;-)

joconnell@ubuntu:~/workspace/NxTrim$ ./nxtrim -1 /home/joconnell/Downloads/HFYJ5AFXX.1_5kb_R1.sw_missed_linker.fastq.txt -2 /home/joconnell/Downloads/HFYJ5AFXX.1_5kb_R2.sw_missed_linker.fastq.txt --separate --preserve-mp --rf -O test -w -s 0.7
Output: test.*.fastq.gz
Trimming:
R1:	/home/joconnell/Downloads/HFYJ5AFXX.1_5kb_R1.sw_missed_linker.fastq.txt
R2:	/home/joconnell/Downloads/HFYJ5AFXX.1_5kb_R2.sw_missed_linker.fastq.txt
--preserve-mp is on: will favour MPs over PEs

Trimming summary:
7185 / 7185	( 100.00% )	reads passed chastity/purity filters.
173 / 7185	( 2.41% )	reads had TWO copies of adapter (filtered).
8 / 7012	( 0.11% )	read pairs were ignored because template length appeared less than read length
7004 remaining reads were trimmed

4168 / 7004	( 59.51% )	read pairs had MP orientation
570 / 7004	( 8.14% )	read pairs had PE orientation
1880 / 7004	( 26.84% )	read pairs had unknown orientation
386 / 7004	( 5.51% )	were single end reads

1194 / 7004	( 17.05% )	extra single end reads were generated from overhangs
joconnell@ubuntu:~/workspace/NxTrim$ for i in test*gz;do echo $i;zcat $i | agrep -1 CTGTCTCTTATACACATCT | wc -l;done
test_R1.mp.fastq.gz
0
test_R1.pe.fastq.gz
0
test_R1.unknown.fastq.gz
0
test_R2.mp.fastq.gz
0
test_R2.pe.fastq.gz
0
test_R2.unknown.fastq.gz
0
test.se.fastq.gz
0
joconnell@ubuntu:~/workspace/NxTrim$ for i in test*gz;do echo $i;zcat $i | agrep -1 AGATGTGTATAAGAGACAG | wc -l;done
test_R1.mp.fastq.gz
0
test_R1.pe.fastq.gz
0
test_R1.unknown.fastq.gz
0
test_R2.mp.fastq.gz
0
test_R2.pe.fastq.gz
0
test_R2.unknown.fastq.gz
0
test.se.fastq.gz
0

Seems too few were found. Try this.
# find the missed matches using blastn
for f in *.mp.fastq.gz *.pe.fastq.gz *.unknown.fastq.gz *.se.fastq.gz; do p=`basename $f .fastq.gz`; echo "$f"; gzip -dc $f > "$p".fastq; fq2fa "$p".fastq "$p".fasta; formatdb -p F -i "$p".fasta; blastn -num_threads 18 -task blastn-short -db "$p".fasta -query Nextera_MatePair_hairpin.fasta -word_size 4 -dust no -reward 1 -max_target_seqs 99999 -max_hsps_per_subject 20 -perc_identity 75 -strand plus > "$p".missed_linkers.blastn.txt; done

for f in *.mp.fastq.gz *.pe.fastq.gz *.unknown.fastq.gz *.se.fastq.gz; do p=`basename $f .fastq.gz`; echo "$f"; gzip -dc $f > "$p".fastq; fq2fa "$p".fastq "$p".fasta; formatdb -p F -i "$p".fasta; blastn -num_threads 18 -task blastn-short -db "$p".fasta -query Nextera_MatePair_hairpin.fasta -word_size 4 -dust no -reward 1 -max_target_seqs 99999 -max_hsps_per_subject 20 -perc_identity 75 -strand plus -outfmt 6 | awk '{print $2}' > "$p".missed_linkers.txt; done

reads had TWO copies of adapter (filtered).

How about:
reads had TWO or more copies of adapter (discarding whole pair).
if that is true.

$ grep 'reads had TWO copies of' *nxtrim__preserve_mp_from_original.log | sort
HFYJ5AFXX.1_5kb.trimmomatic.PE.nxtrim__preserve_mp_from_original.log:29721 / 16495106   ( 0.18% )       reads had TWO copies of adapter (filtered).
HFYJ5AFXX.1_5kb.trimmomatic.SE.nxtrim__preserve_mp_from_original.log:39 / 20662 ( 0.19% )       reads had TWO copies of adapter (filtered).
HFYJ5AFXX.1_8kb.trimmomatic.PE.nxtrim__preserve_mp_from_original.log:29331 / 16300973   ( 0.18% )       reads had TWO copies of adapter (filtered).
HFYJ5AFXX.1_8kb.trimmomatic.SE.nxtrim__preserve_mp_from_original.log:40 / 21681 ( 0.18% )       reads had TWO copies of adapter (filtered).
HFYJ5AFXX.2_5kb.trimmomatic.PE.nxtrim__preserve_mp_from_original.log:29124 / 16253919   ( 0.18% )       reads had TWO copies of adapter (filtered).
HFYJ5AFXX.2_5kb.trimmomatic.SE.nxtrim__preserve_mp_from_original.log:14 / 11399 ( 0.12% )       reads had TWO copies of adapter (filtered).
HFYJ5AFXX.2_8kb.trimmomatic.PE.nxtrim__preserve_mp_from_original.log:29476 / 16044969   ( 0.18% )       reads had TWO copies of adapter (filtered).
HFYJ5AFXX.2_8kb.trimmomatic.SE.nxtrim__preserve_mp_from_original.log:40 / 12909 ( 0.31% )       reads had TWO copies of adapter (filtered).
HFYJ5AFXX.3_5kb.trimmomatic.PE.nxtrim__preserve_mp_from_original.log:28822 / 15867509   ( 0.18% )       reads had TWO copies of adapter (filtered).
HFYJ5AFXX.3_5kb.trimmomatic.SE.nxtrim__preserve_mp_from_original.log:45 / 18413 ( 0.24% )       reads had TWO copies of adapter (filtered).
HFYJ5AFXX.3_8kb.trimmomatic.PE.nxtrim__preserve_mp_from_original.log:28387 / 15716944   ( 0.18% )       reads had TWO copies of adapter (filtered).
HFYJ5AFXX.3_8kb.trimmomatic.SE.nxtrim__preserve_mp_from_original.log:33 / 19198 ( 0.17% )       reads had TWO copies of adapter (filtered).
HFYJ5AFXX.4_5kb.trimmomatic.PE.nxtrim__preserve_mp_from_original.log:26571 / 14582359   ( 0.18% )       reads had TWO copies of adapter (filtered).
HFYJ5AFXX.4_5kb.trimmomatic.SE.nxtrim__preserve_mp_from_original.log:29 / 12569 ( 0.23% )       reads had TWO copies of adapter (filtered).
HFYJ5AFXX.4_8kb.trimmomatic.PE.nxtrim__preserve_mp_from_original.log:26923 / 14459869   ( 0.19% )       reads had TWO copies of adapter (filtered).
HFYJ5AFXX.4_8kb.trimmomatic.SE.nxtrim__preserve_mp_from_original.log:29 / 14084 ( 0.21% )       reads had TWO copies of adapter (filtered).

The discarded total is maybe a bit higher than those 180k from previous tests but it is OK. Will now run the blastn searches ...

These still remain in MP data, for example. The scoring must be wrong somewhere.

> NB501598:62:HFYJ5AFXX:1:21310:6408:6560 1:N:0:GCCAAT
Length=151

 Score = 38.2 bits (19),  Expect = 0.072
 Identities = 25/27 (93%), Gaps = 0/27 (0%)
 Strand=Plus/Plus

Query  8   TTATACACATCTAGATGTGTATAAGAG  34
           |||||||||| ||| ||||||||||||
Sbjct  26  TTATACACATATAGTTGTGTATAAGAG  52

> NB501598:62:HFYJ5AFXX:1:21310:1598:1343 1:N:0:GCCAAT
Length=155

 Score = 36.2 bits (18),  Expect = 0.28
 Identities = 24/26 (92%), Gaps = 0/26 (0%)
 Strand=Plus/Plus

Query  8   TTATACACATCTAGATGTGTATAAGA  33
           |||||||||| || ||||||||||||
Sbjct  41  TTATACACATATATATGTGTATAAGA  66

> NB501598:62:HFYJ5AFXX:1:21301:22800:8378 1:N:0:GCCAAT
Length=155

 Score = 36.2 bits (18),  Expect = 0.28
 Identities = 30/34 (88%), Gaps = 0/34 (0%)
 Strand=Plus/Plus

Query  2   TGTCTCTTATACACATCTAGATGTGTATAAGAGA  35
           ||||||||||||  |||||||| | |||||||||
Sbjct  17  TGTCTCTTATACCGATCTAGATTTCTATAAGAGA  50

> NB501598:62:HFYJ5AFXX:1:21205:14665:11772 1:N:0:GCCAAT
Length=155

 Score = 36.2 bits (18),  Expect = 0.28
 Identities = 30/34 (88%), Gaps = 0/34 (0%)
 Strand=Plus/Plus

Query  4   TCTCTTATACACATCTAGATGTGTATAAGAGACA  37
           |||| |||| ||||||||||  ||||||||||||
Sbjct  31  TCTCATATAGACATCTAGATCAGTATAAGAGACA  64

> NB501598:62:HFYJ5AFXX:1:21203:19031:19916 1:N:0:GCCAAT
Length=155

 Score = 36.2 bits (18),  Expect = 0.28
 Identities = 30/34 (88%), Gaps = 0/34 (0%)
 Strand=Plus/Plus

Query  3   GTCTCTTATACACATCTAGATGTGTATAAGAGAC  36
           |||||||||| | |||||||||| ||||| ||||
Sbjct  1   GTCTCTTATAAAAATCTAGATGTTTATAATAGAC  34

If you consider your first example, sure there is ~26bp string match, but the flanking bases are very divergent:
                                                                                CTGTCTCTTATACACATCTAGATGTGTATAAGAGACAG
TGCAAAAGTTCGGTAATGTATTATGTGTAATGTATTTACTTTTACTTTTACAAGCTTGACAAAGTGAAACAACTCATAGTGAAACATCCTAATACCTTACTCTTATACACAACTATATGTGTATAATAAACAGGTATAATTGTGTGTGATT

If you drop to -s 0.6 this will be caught, but I don't know how safe that is in general.
I won't be fixing cases like this sorry.

No, they are just true linkers.
> NB501598:62:HFYJ5AFXX:1:21310:6408:6560 1:N:0:GCCAAT
Length=151

 Score = 44.6 bits (48),  Expect = 4e-04
 Identities = 30/34 (88%), Gaps = 0/34 (0%)
 Strand=Plus/Plus

Query  1   CTGTCTCTTATACACATCTAGATGTGTATAAGAG  34
           |||| | |||||||||| ||| ||||||||||||
Sbjct  19  CTGTTTATTATACACATATAGTTGTGTATAAGAG  52

> NB501598:62:HFYJ5AFXX:1:21301:22800:8378 1:N:0:GCCAAT
Length=155

 Score = 44.6 bits (48),  Expect = 4e-04
 Identities = 32/37 (86%), Gaps = 0/37 (0%)
 Strand=Plus/Plus

Query  2   TGTCTCTTATACACATCTAGATGTGTATAAGAGACAG  38
           ||||||||||||  |||||||| | ||||||||| ||
Sbjct  17  TGTCTCTTATACCGATCTAGATTTCTATAAGAGATAG  53

> NB501598:62:HFYJ5AFXX:1:21205:14665:11772 1:N:0:GCCAAT
Length=155

 Score = 44.6 bits (48),  Expect = 4e-04
 Identities = 32/37 (86%), Gaps = 0/37 (0%)
 Strand=Plus/Plus

Query  1   CTGTCTCTTATACACATCTAGATGTGTATAAGAGACA  37
           || |||| |||| ||||||||||  ||||||||||||
Sbjct  28  CTCTCTCATATAGACATCTAGATCAGTATAAGAGACA  64

> NB501598:62:HFYJ5AFXX:1:21203:19031:19916 1:N:0:GCCAAT
Length=155

 Score = 44.6 bits (48),  Expect = 4e-04
 Identities = 30/34 (88%), Gaps = 0/34 (0%)
 Strand=Plus/Plus

Query  3   GTCTCTTATACACATCTAGATGTGTATAAGAGAC  36
           |||||||||| | |||||||||| ||||| ||||
Sbjct  1   GTCTCTTATAAAAATCTAGATGTTTATAATAGAC  34

Here is the better commandline with increased reward for a match.
# find the missed matches using blastn
for f in *.mp.fastq.gz *.pe.fastq.gz *.unknown.fastq.gz *.se.fastq.gz; do p=`basename $f .fastq.gz`; echo "$f"; gzip -dc $f > "$p".fastq; fq2fa "$p".fastq "$p".fasta; formatdb -p F -i "$p".fasta; blastn -num_threads 18 -task blastn-short -db "$p".fasta -query Nextera_MatePair_hairpin.fasta -word_size 4 -dust no -reward 2 -max_hsps 20 -num_descriptions 99999 -num_alignments 99999 -perc_identity 75 -strand plus > "$p".missed_linkers.blastn.txt; done

for f in *.mp.fastq.gz *.pe.fastq.gz *.unknown.fastq.gz *.se.fastq.gz; do p=`basename $f .fastq.gz`; echo "$f"; gzip -dc $f > "$p".fastq; fq2fa "$p".fastq "$p".fasta; formatdb -p F -i "$p".fasta; blastn -num_threads 18 -task blastn-short -db "$p".fasta -query Nextera_MatePair_hairpin.fasta -word_size 4 -dust no -reward 2 -max_target_seqs 99999 -max_hsps_per_subject 20 -perc_identity 75 -strand plus -outfmt 6 | awk '{print $2}' > "$p".missed_linkers.txt; done

The scores you calculate are unusually low. Would it be possible possible to stick to the scoring which e.g. blastn uses? I know there is some web page describing why blastn scores are different from EMBOSS water, but they are roughly same. Your is a third approach. -s 0.6 sounds weird but these cases woudl really fit into 0.7 easily. See https://www.ncbi.nlm.nih.gov/books/NBK279678/  But I do not see how to figure out what blastn, actually -task blastn-short enforces. Ahh, luckily it is printed at the bottom of plaintext out. So here is what I propose (and what the above blastn really used):
Matrix: blastn matrix 2 -3
Gap Penalties: Existence: 5, Extension: 2

I am not debating that they are true junction adapters, but they are low quality reads and we have to accept a lot of mismatches to get them.  You are welcome to play with the scoring.
Gap-open(go)/gap-extend(ge):
https://github.com/sequencing/NxTrim/blob/sw_full_match/matepair.cpp#L88
Match(sa)/mismatch(sb):
https://github.com/sequencing/NxTrim/blob/sw_full_match/matepair.cpp#L320

You mean this?
$ git diff
diff --git a/matepair.cpp b/matepair.cpp
index f0ab2cf..22615ed 100755
--- a/matepair.cpp
+++ b/matepair.cpp
@@ -91,7 +91,7 @@ int sw_match(uint8_t *target,int tlen,uint8_t *query,int qlen,int minoverlap,flo
     }
     int sa=1;
     int sb=2;
-    int go = 1, ge = 3;
+    int go = 5, ge = 2; // gap open, gap extension
 //    ta_opt_set_mat(sa, sb, mat);
     kswr_t r = ksw_align(qlen, query, tlen, target, 5, mat, go, ge, KSW_XBYTE|KSW_XSTART|(8*sa), 0);
     int substring_length = r.qe - r.qb < r.te - r.tb? r.qe - r.qb : r.te - r.tb;
@@ -347,7 +347,7 @@ int matePair::resolve_overhang(fqread & r1, fqread & r2,int a,int b) {

 matePair::matePair()
 {
-    ta_opt_set_mat(1,2,sw_mat);
+    ta_opt_set_mat(2,-3,sw_mat); //match and mismatch
     adapter1_sw=(uint8_t *)malloc(adapter1.length()+1);
     for(size_t i=0;i<adapter1.length();i++)
     {

But how about the actual calculation, will that work? I found these but they are not the page I was looking for.
https://www.ncbi.nlm.nih.gov/BLAST/tutorial/Altschul-1.html
http://embossgui.sourceforge.net/demo/manual/water.html
https://en.wikipedia.org/wiki/Smith–Waterman_algorithm
Anyway, what do you say?

I made it clearer by putting all the penalty terms at the top of the code:
https://github.com/sequencing/NxTrim/blob/iss37/matepair.cpp#L13-L16
I don't know much about BLAST. The current SW similarity measure is calculated as the alignment score divided by the maximum possible score for the alignment. The maximum possible score is SW_MATCH multiplied by the alignment length.

I used -s .7 -w --separate --preserve-mp but everything matched after this change.
$ git diff
diff --git a/matepair.cpp b/matepair.cpp
index a170c69..9518d38 100755
--- a/matepair.cpp
+++ b/matepair.cpp
@@ -10,10 +10,10 @@ string adapterj = adapter1+adapter2;
 string r1_external_adapter = "GATCGGAAGAGCACACGTCTGAACTCCAGTCAC";
 string r2_external_adapter = "GATCGGAAGAGCGTCGTGTAGGGAAAGAGTGT";

-#define SW_GAP_EXTENSION 3
-#define SW_GAP_OPEN 1
-#define SW_MATCH 1
-#define SW_MISMATCH 2
+#define SW_GAP_EXTENSION 2
+#define SW_GAP_OPEN 5
+#define SW_MATCH 2
+#define SW_MISMATCH -3

 #define DEBUG 0

Trimming summary:
20662 / 20662   ( 100.00% )     reads passed chastity/purity filters.
19351 / 20662   ( 93.66% )      reads had TWO copies of adapter (filtered).
1311 / 1311     ( 100.00% )     read pairs were ignored because template length appeared less than read length
0 remaining reads were trimmed

0 / 0   ( -nan% )       read pairs had MP orientation
0 / 0   ( -nan% )       read pairs had PE orientation
0 / 0   ( -nan% )       read pairs had unknown orientation
0 / 0   ( -nan% )       were single end reads

Sounds good.  Do you want to check your full dataset? I will run some tests next week and merge to master.

It's a bit confusing - you need #define SW_MISMATCH 3 not -3

Good? No, to loose. Currently I am testing on the full dataset:
--- a/matepair.cpp
+++ b/matepair.cpp
@@ -10,9 +10,9 @@ string adapterj = adapter1+adapter2;
 string r1_external_adapter = "GATCGGAAGAGCACACGTCTGAACTCCAGTCAC";
 string r2_external_adapter = "GATCGGAAGAGCGTCGTGTAGGGAAAGAGTGT";

-#define SW_GAP_EXTENSION 3
-#define SW_GAP_OPEN 1
-#define SW_MATCH 1
+#define SW_GAP_EXTENSION 1
+#define SW_GAP_OPEN 2
+#define SW_MATCH 2
 #define SW_MISMATCH 2

 #define DEBUG 0

The read NB501598:62:HFYJ5AFXX:1:21310:6408:6560 is a good example I hope.
What I need is a much better log output into individual files (mp, pe, se, unknown).
$readname  $linker_start  $linker_end $matched_sequence

The second mate is surprisingly shortened and I do not understand why. Probably nxtrim thought it is followed by the very same linker?

No, I have a different theory. It looks the sequence downstream of the linker in R1 read was rever-complemented and aligned to R2 read to find where the mates overlap. But the overlap=alignment should well cover the linker and even its sequence on its left.

You can turn on debugging output by setting #DEFINE DEBUG 2. The output is very verbose so only run it on a very small dataset (ie one read-pair).  You then see things like this:
read L1 = 155
read L2 = 155
adapter locations (first pass): 155 193 42 80
adapter locations (second pass): 155 193 42 80
adapter locations (third pass): 155 193 42 80

it is telling you where the adapters were detected. So here (42,80) means that was the detected adapter in read 2. When the adapter start is == to read length, no adapter is found.

@mmokrejs FYI I just pushed a minor bugfix that was causing minoverlap to not be respected in the SW code. I don't think this substantially changes results though (except that some 12bp adapter matches were missed).

Thanks, will try but probably after I figure out how to get the SW-matching code to work. I with below changes I ended up with some 242k linkers unremoved, about 60k more then with original. :(
--- a/matepair.cpp
+++ b/matepair.cpp
@@ -12,8 +12,8 @@ string r2_external_adapter = "GATCGGAAGAGCGTCGTGTAGGGAAAGAGTGT";

 #define SW_GAP_EXTENSION 3
 #define SW_GAP_OPEN 1
-#define SW_MATCH 1
-#define SW_MISMATCH 2
+#define SW_MATCH 2
+#define SW_MISMATCH 3

 #define DEBUG 0

Some difficult cases (I lazily assume they are in the FASTQ file I already provided):
> NB501598:62:HFYJ5AFXX:1:21312:10138:16626 1:N:0:GCCAAT
Length=105

 Score = 42.8 bits (46),  Expect = 0.001
 Identities = 32/38 (84%), Gaps = 0/38 (0%)
 Strand=Plus/Plus

Query  1   CTGTCTCTTATACACATCTAGATGTGTATAAGAGACAG  38
           |||||||||||| | || ||||||| |||||  |||||
Sbjct  22  CTGTCTCTTATAAAAATATAGATGTTTATAATTGACAG  59

> NB501598:62:HFYJ5AFXX:1:21312:5193:12408 1:N:0:GCCAAT
Length=155

 Score = 42.8 bits (46),  Expect = 0.001
 Identities = 32/38 (84%), Gaps = 0/38 (0%)
 Strand=Plus/Plus

Query  1   CTGTCTCTTATACACATCTAGATGTGTATAAGAGACAG  38
           |||||||||||| | || ||||||| ||||| || |||
Sbjct  23  CTGTCTCTTATAAAAATATAGATGTTTATAATAGTCAG  60

> NB501598:62:HFYJ5AFXX:1:21311:21303:18350 1:N:0:GCCAAT
Length=155

 Score = 42.8 bits (46),  Expect = 0.001
 Identities = 32/38 (84%), Gaps = 0/38 (0%)
 Strand=Plus/Plus

Query  1   CTGTCTCTTATACACATCTAGATGTGTATAAGAGACAG  38
           || ||||| ||| |||||||||| | ||||||| ||||
Sbjct  32  CTCTCTCTAATAAACATCTAGATCTCTATAAGATACAG  69

> NB501598:62:HFYJ5AFXX:1:21310:13226:18075 1:N:0:GCCAAT
Length=155

 Score = 42.8 bits (46),  Expect = 0.001
 Identities = 32/38 (84%), Gaps = 0/38 (0%)
 Strand=Plus/Plus

Query  1   CTGTCTCTTATACACATCTAGATGTGTATAAGAGACAG  38
           ||| || |||||||| ||||||| |||||||| | |||
Sbjct  48  CTGGCTGTTATACACTTCTAGATTTGTATAAGTGTCAG  85

You could also use https://ncbi.github.io/cxx-toolkit/pages/ch_blast#ch_blast.CLocalBlast provided you index both forward and reverse FASTA files separately (after FASTQ to FASTA conversion). But I still think it is faster to run external blastn in parallel (use can provide it before hand, and notably edit it easily -- e.g. remove false positives or add some missed cases), parse it's CSV output, and import into nxtrim. You cannot beat its performance doing SW-alignment one by one read pair IMHO, not even speaking of paralellization being missing. Issue #46 is the way to go IMHO. I will just work and we can focus on the remaining code. I still foresee some problem in the case of NB501598:62:HFYJ5AFXX:1:21310:6408:6560.

I just pushed a minor bugfix that was causing minoverlap to not be respected in the SW code. I don't think this substantially changes results though (except that some minoverlap adapter matches were missed).

Aha, so now the branch is gone and the region changed in the patch below was dropped as well. Was that intentional?
--- a/matepair.cpp
+++ b/matepair.cpp
@@ -12,8 +12,8 @@ string r2_external_adapter = "GATCGGAAGAGCGTCGTGTAGGGAAAGAGTGT";

 #define SW_GAP_EXTENSION 3
 #define SW_GAP_OPEN 1
-#define SW_MATCH 1
-#define SW_MISMATCH 2
+#define SW_MATCH 2
+#define SW_MISMATCH 3

Yes that looks right.
The branch is now merged to master with the bugfix and scoring tweaks.

Closing this - it turns out the SW is not ideal for this problem.  Semi-global alignment is more appropriate, the cutadapt algorithm looks appropriate (chapter 2):
https://eldorado.tu-dortmund.de/bitstream/2003/31824/1/Dissertation.pdf
This is worth revisiting but I won't get time in the short term.

(smith-waterman remains on the sw branch but this is unlikely to be merged)


Depend on create-redux-constants

Zero deps is really nice, but @tbranyen's lib for creating action types is incredibly useful. I should consider adding this as a dep

Maybe just in dev? Wrap in a process.env.NODE_ENV !== 'production' so consumers don't deal with it in prod.

are you thinking of a require() going in that if statement? or somethin else?

Closing in favor of https://github.com/jmeas/resourceful-redux/issues/9


Improve logging around env var loading / dotenv in CLI

From a user logfile:
Running Prisma CLI:
yarn prisma migrate reset --force --schema "/home/.../Projects/.../api/db/schema.prisma"
2021-06-01T17:56:00.450Z prisma:loadEnv project root found at /home/tharshan/Projects/uservitals/api/package.json
2021-06-01T17:56:00.462Z prisma:tryLoadEnv Environment variables not found at null
2021-06-01T17:56:00.463Z prisma:tryLoadEnv Environment variables not found at undefined
2021-06-01T17:56:00.463Z prisma:tryLoadEnv No Environment variables loaded

Context: https://github.com/prisma/prisma/blob/caca0e71233c895ef60aa9ce1bc7598a36537f87/src/packages/sdk/src/utils/tryLoadEnvs.ts#L31
Internal Discussion: https://prisma-company.slack.com/archives/CSP5ZBE0Y/p1622570299011000

Hi there,
As part of our effort to clean up our GitHub repo, we’re closing this issue to help focus on features and improvements that align with our focus areas and those of our community.
This issue was originally created for internal tracking purposes, but if you feel this is important to you, feel free to reopen the issue and get like-minded users to upvote as we will prioritize fixing issues based on community interest.
For more insight into our priorities and how we’re thinking about the future of Prisma, please see our blog post: https://www.prisma.io/blog/prisma-orm-manifesto.
Thank you for your understanding and support!


Get current iPhone total storage?

Hello, I've noticed it's missing the phone total storage, or free space/used space.
Doing a quick research I've crossed with this question on SO:
https://stackoverflow.com/questions/5310445/check-enough-space-on-iphone-device-before-downloading-files
what do you guys think? I know nothing about obj-c so I'm just making a suggestion :D
Would be hard to do it?

Check out https://github.com/itinance/react-native-fs#getfsinfo-promisefsinforesult
You would have to add another library, but this does what you are looking for.

Thanks @nolan-m ! I'm going to check out, great tip! =]


Fix invalid filename bug and add a option

Fix invalid filename when export in windows.
Add option of exportSameNameMesh.

Thank you for fix bug, but I don't know what "exportSameNameMesh" is for

I suppose he means that models (.FBX) came out off UnityStudio with the current date and time name however it has its own name in the asset.
For ex.: I open UnityStudio and find model called 'mesh_BigThoughArmored_V2' and when I export it the Studio suggest name for .FBX like this: '2017-07-15-11-03-28.FBX'.
So it would be very usefull to use original model name for .FBX and produce file 'mesh_BigThoughArmored_V2.fbx'.

Yes, @Tosyk is right.
The original model name plus with time stamp will generate many same models with different names.
This often happens when a model used many times in on scene or in many scenes.
And the option name should be more clearly. "exportSameNameMesh" is not good enough.
May be exportMeshOnce? exportMeshWithoutTimeStamp?
I'm not good at naming...

@xykong But your code will appear if different object have the same name, will export only one file. Before export we don't know which files are useful, so I don't think this option makes any sense


feat: Class-based controllers and view-models

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 39 (100.0%)  changed or added relevant lines in 3 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.3%) to 87.307%

Totals

Change from base Build<PHONE_NUMBER>:
0.3%

Covered Lines:
763

Relevant Lines:
844

💛  - Coveralls


Get the Support Role Arn from the Cluster response body for hypershift clusters

Hypershift cluster have the support role ARN directly in the Cluster response body and don't have an account claim. This PR returns the support role arn directly if the cluster is hypershift.

/lgtm


docs(search-quality): add button and explanation for the Search Quality tab

Added a explanation log for search quality output, including information on precision and tuning parameters.
Improved the display of index quality checks by using a button when precision is not available, and adjusted the layout for better alignment.
layout changes

The text in the explanation box needs to be made more visible.


Unable to activate chef-provisioning-docker-0.6

Running into the following issue.
Unable to activate chef-provisioning-docker-0.6, because chef-provisioning-1.1.1 conflicts with chef-provisioning (~> 0.9)

@jkeiser it looks like the dependency in the gemspec file needs to be updated right?
https://github.com/chef/chef-provisioning-docker/blob/master/chef-provisioning-docker.gemspec#L16

duplicate of #45

1.0 had some breaking API changes so we need to test chef-provisioning-docker with it before we get out a release.  In the mean time, you could use a local Gemfile to pin to the versions you want and use bundle exec to run your commands.

What version dependency changes should I make to make this work?
I removed ~> 0.9 changing dependency to
s.add_dependency 'chef-provisioning'

bundle install shows
[vagrant@localhost chef-provisioning-docker]$ bundle install
Resolving dependencies...
Using rake 10.3.2
...(lines deleted)
Using winrm 1.2.0
Using chef-provisioning 0.15.2 from https://github.com/opscode/chef-provisioning.git (at master)
Using excon 0.45.3
...(lines deleted)
Using chef-provisioning-docker 0.6 from source at .
...(lines deleted)
Using bundler 1.8.4
Bundle complete! 4 Gemfile dependencies, 61 gems now installed.
Use `bundle show [gemname]` to see where a bundled gem is installed.
[vagrant@localhost chef-provisioning-docker]$

But when I run the command in README.md it shows an error I don't understand

ERROR: undefined method `cheffish' for nil:NilClass

Complete output in console is
[vagrant@localhost chef-provisioning-docker]$ CHEF_DRIVER=docker bundle exec chef-client -z docker_ubuntu_image.rb
/home/vagrant/.rvm/gems/ruby-2.2.1/gems/chef-12.0.3/lib/chef/data_bag_item.rb:161: warning: circular argument reference - data_bag
[2015-05-28T07:40:37+00:00] INFO: Started chef-zero at http://localhost:8889 with repository at /vagrant
  One version per cookbook

[2015-05-28T07:40:37+00:00] INFO: Forking chef instance to converge...
Starting Chef Client, version 12.0.3
[2015-05-28T07:40:37+00:00] INFO: *** Chef 12.0.3 ***
[2015-05-28T07:40:37+00:00] INFO: Chef-client pid: 29883
[2015-05-28T07:40:41+00:00] INFO: Run List is []
[2015-05-28T07:40:41+00:00] INFO: Run List expands to []
[2015-05-28T07:40:41+00:00] INFO: Starting Chef Run for ostraaten

Running handlers:
[2015-05-28T07:40:41+00:00] ERROR: Running exception handlers
Running handlers complete
[2015-05-28T07:40:41+00:00] ERROR: Exception handlers complete
[2015-05-28T07:40:41+00:00] FATAL: Stacktrace dumped to /vagrant/.chef/local-mode-cache/cache/chef-stacktrace.out
Chef Client failed. 0 resources updated in 3.850152733 seconds
[2015-05-28T07:40:41+00:00] ERROR: undefined method `cheffish' for nil:NilClass
[2015-05-28T07:40:41+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)
[vagrant@localhost chef-provisioning-docker]$

I was watching for 1.0 compatibility issue for quite some time. At the moment following line in chef-provisioning breaks image creation, since destroying intermediate machine also invoke destroying underlying image we're trying to create (see and this issue). Should it be fixed in chef-provisioning or chef-provisioning-docker?
I think, we need to add flag into machine resource to prevent/enforce image deletion along with container.
At ChefConf 2015 @jkeiser also was talking about significant improvements in image and container procedure for 1.0. Currently it is inefficient in terms of performance (you probably noticed that images are getting created really slow) and images size (see).
Looking forward for chef-provisioning-docker 1.0 .

Made certain adjustments to make it work with chef-provisioning >=1.0 .

:+1: This driver is unusable with the last few DKs without forking with this revision

Fixed by PR #47. Needs a 0.7 release and to then get that sorted in DK.


fix a single account security model bug

Why didn't our ledgerjs tests catch this?

Why didn't our ledgerjs tests catch this?

Because it's on the positive side, not the negative, and I only made rejection tests, no Byron-Shelley positive test was created.


Support Library Fix

Switching the reference to the support library to use the framework tag and removing the support library jar from the libs folder.
This should prevent the multiple dex issues that appear when you use this plugin with others that also depend on the support libraries.

@keab42 Thanks, I'll try it out later today.

@keab42 How did you get this to compile? I get gradle issues where it conflicts with the min SDK version declared by Cordova Android?

Ah I had a manual edit locally in the Manifest.xml to bump that min-version up. Would you like me to add that to the PR?

@keab42 I had to fix an issue in phonegap-start and cordova-android as the min sdk version should be 14 not 10 (or 7 as it was in one case). I'm going to hold off on pulling this request in until cordova-android 4.0.3 is released. That way people won't have to edit the AndroidManifest.xml file in the platforms/android folder.
Sound reasonable?

Sounds good.
Cordova for Android is driving me nuts at the moment, hopefully 4.0.3 will be more stable and work better with Gradle.

@keab42 Yeah, Joe and I were talking about it. Hoping to get a release out soonish.

I have FacebookSDK (which also includes support-v13) installed and I tried your fix, but when I remove the support lib, I'll get errors.
/Users/Hirbod/Desktop/appdev/Oktoberfest/platforms/android/src/com/adobe/phonegap/push/GCMIntentService.java:139: error: cannot find symbol
            mBuilder.setColor(iconColor);
                    ^
  symbol:   method setColor(int)
  location: variable mBuilder of type Builder

If I leave it like it is, I get:
	UNEXPECTED TOP-LEVEL EXCEPTION:
  	com.android.dex.DexException: Multiple dex files define Landroid/support/v4/accessibilityservice/AccessibilityServiceInfoCompatIcs;
  		at com.android.dx.merge.DexMerger.readSortableTypes(DexMerger.java:596)
  		at com.android.dx.merge.DexMerger.getSortedTypes(DexMerger.java:554)
  		at com.android.dx.merge.DexMerger.mergeClassDefs(DexMerger.java:535)
  		at com.android.dx.merge.DexMerger.mergeDexes(DexMerger.java:171)
  		at com.android.dx.merge.DexMerger.merge(DexMerger.java:189)
  		at com.android.dx.command.dexer.Main.mergeLibraryDexBuffers(Main.java:454)
  		at com.android.dx.command.dexer.Main.runMonoDex(Main.java:303)
  		at com.android.dx.command.dexer.Main.run(Main.java:246)
  		at com.android.dx.command.dexer.Main.main(Main.java:215)
  		at com.android.dx.command.Main.main(Main.java:106)

What can I do to get this run?

Any news on this? I stuck. Can't compile Android since I've installed this plugin. Everything works well on iOS, I hoped for some answers, as my app needs to go online soon.

After installing this plugin, I also got this:
UNEXPECTED TOP-LEVEL EXCEPTION:
com.android.dex.DexException: Multiple dex files define Landroid/support/annotation/AnimRes;
	at com.android.dx.merge.DexMerger.readSortableTypes(DexMerger.java:596)
	at com.android.dx.merge.DexMerger.getSortedTypes(DexMerger.java:554)
	at com.android.dx.merge.DexMerger.mergeClassDefs(DexMerger.java:535)
	at com.android.dx.merge.DexMerger.mergeDexes(DexMerger.java:171)
	at com.android.dx.merge.DexMerger.merge(DexMerger.java:189)
	at com.android.dx.command.dexer.Main.mergeLibraryDexBuffers(Main.java:454)
	at com.android.dx.command.dexer.Main.runMonoDex(Main.java:303)
	at com.android.dx.command.dexer.Main.run(Main.java:246)
	at com.android.dx.command.dexer.Main.main(Main.java:215)
	at com.android.dx.command.Main.main(Main.java:106)

[tip] This can be closed now, @macdonst

I'm closing this one as the same change has been merged in master.


catalog: CreateTable

TableHandle::CreateTable doesn't do everything that we need.
The above, creates a storage table and inserts a row into pg_class. (It returns a raw pointer, which is probably not ok).
We need, at a higher level, for creating tables to:

from input of schema and table name, to create a storage table
insert the name and ptr into pg_class
insert the schema (i.e. column information) into pg_attribute, with the correct references to the pg_type for the column types
insert any column default values into pg_attrdef

I don't think creation of the underlying storage should be the catalog's responsibility, but rather the execution engine's.
Specifically, I'm thinking that if we have a transaction doing a large DDL transaction it would be better to stage all of the catalog writes before doing any object creations.  This way if there is an error in the catalog phase (such as a namespace conflict) or competing DDL transactions, we abort before memory intensive operations like initializing SqlTables.
@mbutrovich, any thoughts?

There are several issues here:

The issue talks about TableHandle. This is a view. My recollection is we were ok with views being read-only. If so, a CreateTable method doesn't really belong there.  The partial support for table creation that I describe is a problem.

We have CreateUserTable in catalog class, that does a reasonably complete creation, populating catalog information. This was to exercise the machinery, not expected to be the perfect final API. Your comment about separating storage creation is applicable here.


Order test

This was already merged - but into the wrong branch 😅
See: https://github.com/get10101/10101/pull/150
@bonomat do we have to manually configure to automatically change the base if the underlying PR merges? We used to have that :D

bors r+

bors retry


support for PNG files

Opening a ZIP file with bunch of PNGs doesn't work but JPGs do

Do you have an example ZIP that I could use to reproduce this?

Here(NSFW)
Sorry, didn't have any better examples that I had on me

That particular ZIP file had something abnormal with its headers and was failing because I had the ZIP opening code performing stricter checks.  I turned off the strict checks, and the ZIP file now loads as expected.


Benerate LocalService for Rust

Valuable once https://github.com/smithy-lang/smithy-dafny/issues/374 is completed

Skipping right to codegen at this point


Update IDs to classes in index.html and /assets/main/style.scss

html

ids to classes

scss

ids to classes

6f72328


Add jgallimore's JMS Tests and fix TomEE-2506 autoconnection tracker warning

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

There's a few conflict on here, but it looks good. I'll get it merged.


Angular-Meteor & ionic whatsapp tutorial. Step 4

Hi,
I see there is an update to the WhatsApp tutorial at https://github.com/Urigo/angular-meteor/pull/1042
Is this live at angular-meteor.com?
There is a problem in step 4.
When you click at "Step 4 - Authentication" only the header change. Not the body.
The content of the body stays the same of your previous selection.
Tried on Chrome and Firefox at Ubuntu
The Chrome dev tools show this error when you click at step 4
Exception from Tracker recompute function:
535736076ede66060da906e2482a4345d1ad0cb1.js:3 TypeError: Cannot read property 'files' of undefined
    at null.<anonymous> (535736076ede66060da906e2482a4345d1ad0cb1.js:155)
    at 535736076ede66060da906e2482a4345d1ad0cb1.js:62
    at Function.e._withTemplateInstanceFunc (535736076ede66060da906e2482a4345d1ad0cb1.js:62)
    at n (535736076ede66060da906e2482a4345d1ad0cb1.js:62)
    at null.<anonymous> (535736076ede66060da906e2482a4345d1ad0cb1.js:62)
    at n (535736076ede66060da906e2482a4345d1ad0cb1.js:61)
    at Object.t.nonreactive (535736076ede66060da906e2482a4345d1ad0cb1.js:23)
    at 535736076ede66060da906e2482a4345d1ad0cb1.js:61
    at Object.c._withCurrentView (535736076ede66060da906e2482a4345d1ad0cb1.js:61)
    at Object.c._fireCallbacks (535736076ede66060da906e2482a4345d1ad0cb1.js:61)

Can test with the new 1.3.3 version?

The problem is on the site
http://www.angular-meteor.com/tutorials/whatsapp/ionic/methods
When you click at "Step 4 - Authentication" on sidebar only the title of the tutorial changes. Not the content.

ohhh you are right!
@idanwe


My Requests

Hello, this plugin is very nice. it works great for me. But I would have a few requests. Thanks

Choosing different fonts for full-screen features such as stopwatch, pomodoro
Change font size

I don't take feature requests anymore.


Too long search

Thank you author for your working, great library. But.. it so long.
Simple search request executes about 3 seconds, while other libraries (for example, PyGoogler) takes about 1 seconds.

Hi!
The delay is simply a pause between HTTP requests to make sure Google
doesn't block your IP - try reducing it for faster searches. Note however
that this library is not using any official Google API to work, so you may
experience problems if you try to make too many HTTP requests (in
particular the Captcha will pop up and of course the library won't be able
to solve it).
Cheers,
/Mario
On Tue, Dec 8, 2015 at 5:53 PM, Pavel Shorokhov<EMAIL_ADDRESS>wrote:

Thank you author for your working, great library. But.. it so long.
Simple search request executes about 3 seconds, while other libraries (for
example, PyGoogler) takes about 1 seconds.
—
Reply to this email directly or view it on GitHub
https://github.com/MarioVilas/google/issues/16.

--
“There's a reason we separate military and the police: one fights the enemy
of the state, the other serves and protects the people. When the military
becomes both, then the enemies of the state tend to become the people.”


Testing with Jasmine and karma

Is it possible to get the content that is loaded into tinymce when using jasmine and karma? tinymce.activeEditor.getContent() is null since it acts as if the tinymce object doesn't exist.

Hey, @saboehnke how do you solved this?


Great Britain Flags

[x] England - gb-eng - svg
[x] Northern Ireland - gb-nir
[x] Scotland - gb-sct
[x] Wales - gb-wls -  svg
[x] Add new source files for new flags

https://en.wikipedia.org/wiki/ISO_3166-2:GB
Fixes #112

What codes should we use for all these flags?

Do you guys agree with the codes?

Maybe country name?
flag-icon-scotland
flag-icon-england
flag-icon-wales
flag-icon-northern-ireland

gb-eng, gb-nir, gb-sct, gb-wls are good. They all are from ISO 3166-2

@simonwielens, @Jokero are you guys fine with these flags?

gb-eng, gb-sct, gb-wls are ok, but I'm not sure about Northern Ireland flag.
From Northern Ireland wiki:

The official flag is that of the state having sovereignty over the territory, i.e. the Union Flag.
...
The Ulster flag and the Cross of St Patrick have no official status and, under the Flags Regulations, are not permitted to be flown from Government Buildings

But from flag of Northern Ireland wiki:

The Saint Patrick's Saltire is sometimes used by the government to represent Northern Ireland alongside the other countries of the United Kingdom, and is the centrepiece of the badge of the Police Service of Northern Ireland.
The Ulster Banner remains in use by unionists, a number of sporting organisations in Northern Ireland and some local government authorities under Unionist control.

I do not know which one to use. Maybe we add both flags?

that's one of the reasons that I didn't want to include extra flags for so long.. :/

We can leave only the flags of Scotland, England and Wales. Because they are all countries and have official flag

And GB-NIR is province (https://en.wikipedia.org/wiki/ISO_3166-2:GB)

Sounds like a good idea..

Should be fine now.. :)

Cool!) Thanks :+1:  Publish please new version

Maybe I'll merge the #125.. the most demanded flag so far :)

It would be great)

Published :) should appear soon in your nearest shop..

Great! stuff. Thx. But the GB-NIR is not showing up on your css.
NIR Flag
https://en.wikipedia.org/wiki/File:Flag_of_Northern_Ireland.svg
This NIR flag has been used on football and other events.
http://www.irishfa.com/
http://www.uefa.com/uefaeuro/season=2016/teams/team=63/index.html

@carlosdv4 I tried to investigate this early https://github.com/lipis/flag-icon-css/pull/120#issuecomment-167011317. NIR is not a country and it does not have own official flag

:D You can run into a lot of troubles by just mention "NIR is not a country and it does not have own official flag"
May not be totally official but to every NIR that's their flag.
Could you add that flag, please to your files. I'm sure you won't get that many request for NIR.
Regards.

:D You can run into a lot of troubles by just mention "NIR is not a country and it does not have own official flag"

What kind of troubles?! It's a free world man.. we can say and do whatever we want.. (at least in this part of the world..)

Could you add that flag to your files, please. I'm sure you won't get that many request for NIR.

Nope.. sorry.. maybe if I'll get too many requests I'll reconsider.. :)

@carlosdv4 I don't know the real situation around Northern Ireland, I just use public resources to get information :) So you can take a look at this page https://www.iso.org/obp/ui/#iso:code:3166:GB and find "Northern Ireland". It's marked as "province". In context of ISO 3166-2 my phrase is correct, but there is another question: does this standard contain correct information? :smile: Anyway this question is not related to this repository

I will say that I'm using the flags in a sports-related setting and for the
purpose of soccer, NIR is a "country" that competes with other countries.
Regardless of that though you probably need to pick a standard and stick
with it as there are dozens of such gray areas all over the world.  Kosovo,
South Sudan, Catalonia, Gibraltar, Jersey etc.  I'd definitely look at the
standards out there as they've been dealing with these issues for a while
now.
BTW, great project. really love it.
sean

Sean Forman
Sports Reference LLC, President
http://www.sports-reference.com/
On Wed, May 11, 2016 at 5:54 AM, Lipis<EMAIL_ADDRESS>wrote:

:D You can run into a lot of troubles by just mention "NIR is not a
country and it does not have own official flag"
What kind of troubles?! It's a free world man.. we can say and do whatever
we want.. (at least in this part of the world..)
Could you add that flag to your files, please. I'm sure you won't get that
many request for NIR.
Nope.. sorry.. maybe if I'll get too many requests I'll reconsider.. :)
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/lipis/flag-icon-css/pull/120#issuecomment-218414151

Anyway if you don't want to do it, here is the solution for NIR flag your lazy...
https://github.com/carlosdv4/norther_ireland_flag_svg
Took me 5 min to do it.

Internet is a fun place..! Not sure why people are getting mad.. or think that I'm lazy or care so much for something that they didn't do or wanted to do anyway.. like I mentioned before @carlosdv4 we live in a free world and nobody is paying me for doing that.. so how can you even demand or assume that I'm lazy?! do you think that this project just appeared here?! wtf?
@seanfoo Yes.. that was the whole point.. that I sticked only to the official CIA listed recognised countries  but eventually due to the large demand I added a few more in the #120.. and now the chaos!! I enjoy the ride anyway.. it's fun! Thanks for the support..

@lipis First of all, thanks for your work!
I am writing a small app about the UEFA EURO2016, for which Northern Ireland qualified. So the flag is widley spread on every match schedule everywhere. No offense, just saiing :)
I am using @carlosdv4 flags for now.

@jakwo  That's a good reason.. can you point me to one or two such websites..?!

Well, the official UEFA website is probably a good starting point: http://www.uefa.com/uefaeuro/season=2016/teams/team=63/index.html

@jakwo Sold! Will do :)

@lipis Thank you :)

There :)

That was fast. Appreciate it!

If you publish your app.. feel free to come back to us to show off ;)


ReferenceError: crypto is not defined

I have setup @passwordless-id/webauthn and want to use the server as described in the docs to verify the authentication with server.verifyAuthentication(...). Sadly, I get the following error:
ReferenceError: crypto is not defined
    at null.parseCryptoKey (/Users/rogerkreienbuehl/Projekte/apmatic-next/node_modules/@passwordless-id/webauthn/dist/esm/server.js:93:5)
    at null.verifySignature (/Users/rogerkreienbuehl/Projekte/apmatic-next/node_modules/@passwordless-id/webauthn/dist/esm/server.js:108:27)
    at Object.verifyAuthentication (/Users/rogerkreienbuehl/Projekte/apmatic-next/node_modules/@passwordless-id/webauthn/dist/esm/server.js:30:36)
    at null.<anonymous> (/Users/rogerkreienbuehl/Projekte/apmatic-next/apps/apmatic-next-oauth/src/routes/webauthn.ts:58:53)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at null.noCache (/Users/rogerkreienbuehl/Projekte/apmatic-next/apps/apmatic-next-oauth/src/middleware/no-cache.middleware.ts:6:3)
    at null.contextEnsureOidc (/Users/rogerkreienbuehl/Projekte/apmatic-next/apps/apmatic-next-oauth/src/routes/helpers/ensureOidc.ts:4:5)
    at async serve (/Users/rogerkreienbuehl/Projekte/apmatic-next/node_modules/koa-static/index.js:53:9)
    at null.<anonymous> (/Users/rogerkreienbuehl/Projekte/apmatic-next/node_modules/oidc-provider/lib/helpers/initialize_app.js:222:5)
    at null.errorHandler (/Users/rogerkreienbuehl/Projekte/apmatic-next/node_modules/oidc-provider/lib/shared/error_handler.js:26:7)

Is there anything I missed?
I use node version v20.10.0 and @passwordless-id/webauthn version v1.6.1.

Damn... You are right ...The mistake is on my side. I assumed Node's "crypto" module would implement the same stuff as the browser "crypto" module, but they are just similar, not the same. In other words, it would require a separate build to import some lib. Dammit.

no ...wait ...there is something more fishy ....your whole crypto modules is not defined ...that strange ...what does it do if you start node in the terminal and you simply enter "crypto" in the prompt? On mine, I get the Crypto object back, what about yours?

Perhaps it has also something to do with the bundling of your app... crypto should be a global.

Thanks for your response. You are right, thats strange. As I read, since node 19 crypto should be globally available.
When I start node and enter crypto I get Crypto {} and when I type typeof crypto I get 'undefined'.
I got it working by using globalThis.crypto ??= require("node:crypto").webcrypto as a workaround, maybe this is a problem with tsx which I am using for running in dev environment.

weird ...but I'm afraid I don't really know what's going on. Thanks for the workaround.

crypto in node in available in globals, used the module crypto.
import { webcrypto } from 'crypto';

@xkid1 I beg to differ: https://nodejs.org/docs/latest-v22.x/api/globals.html#crypto_1
Well, it's weird. Good thing @rkreienbuehl found a workaround.
globalThis.crypto ??= require("node:crypto").webcrypto


Request Failed with status 403 - Forbidden Error Message

I have the Node-Red version 2.2.5 installed with Home Assistant. It works great on the “Get Account Vehicles” call but all the others I am getting the error message below.
payload: "Error: Request Failed with status 403 - Forbidden"
_msgid: "_________"
Any help would be greatly appreciated!!

To confirm, here is what I am getting for the "Get Account Vehicles.
object
payload: array[1]
0: object
make: "****"
model: "*******"
vin: "********************"
year: "****"
supportedDiagnostics: array[15]
supportedCommands: array[17]
_msgid: "864da6c006e8ab68"

Absolutely ...

Thanks! Yes, I have deployed all and added debug nodes to each one. Here is the screenshot with those results.

No worries, I appreciate the help. Yes and yes. I've tested the app a few times to confirm as well. Now, for more info that may or may not help ... I have a version running on Homebridge currently. I have tried that UUID as well as a new UUID with same results for both. The Homebridge version is working. I am just trying to move all to Home Assistant.

Here is the code and I will work on the other add-on tonight.
object
payload: array[1]
0: object
make: "GMC"
model: "********"
vin: "***********"
year: "*******"
supportedDiagnostics: array[15]
[0 … 9]
0: "ENGINE COOLANT TEMP"
1: "ENGINE RPM"
2: "LAST TRIP FUEL ECONOMY"
3: "TIRE PRESSURE"
4: "AMBIENT AIR TEMPERATURE"
5: "LAST TRIP DISTANCE"
6: "INTERM VOLT BATT VOLT"
7: "OIL LIFE"
8: "LIFETIME FUEL ECON"
9: "FUEL TANK INFO"
[10 … 14]
10: "HOTSPOT CONFIG"
11: "LIFETIME FUEL USED"
12: "ODOMETER"
13: "HOTSPOT STATUS"
14: "VEHICLE RANGE"
supportedCommands: array[17]
[0 … 9]
0: object
name: "cancelAlert"
description: "Cancel a vehicle alert (honk horns/flash lights)."
url: "https://api.gm.com/api/v1/account/vehicles/***********/commands/cancelAlert"
isPrivSessionRequired: "false"
1: object
name: "getHotspotInfo"
description: "Retrives the WiFi Hotspot info"
url: "https://api.gm.com/api/v1/account/vehicles/***********/hotspot/commands/getInfo"
isPrivSessionRequired: "false"
2: object
name: "lockDoor"
description: "Locks the doors."
url: "https://api.gm.com/api/v1/account/vehicles/***********/commands/lockDoor"
isPrivSessionRequired: "false"
3: object
name: "unlockDoor"
description: "Unlocks the doors."
url: "https://api.gm.com/api/v1/account/vehicles/***********/commands/unlockDoor"
isPrivSessionRequired: "true"
4: object
name: "alert"
description: "Triggers a vehicle alert (honk horns/flash lights)."
url: "https://api.gm.com/api/v1/account/vehicles/***********/commands/alert"
isPrivSessionRequired: "true"
5: object
name: "start"
description: "Remotely starts the vehicle."
url: "https://api.gm.com/api/v1/account/vehicles/***********/commands/start"
isPrivSessionRequired: "true"
6: object
name: "cancelStart"
description: "Cancels previous remote start command."
url: "https://api.gm.com/api/v1/account/vehicles/***********/commands/cancelStart"
isPrivSessionRequired: "false"
7: object
name: "diagnostics"
description: "Retrieves diagnostic vehicle data."
url: "https://api.gm.com/api/v1/account/vehicles/***********/commands/diagnostics"
isPrivSessionRequired: "false"
commandData: object
supportedDiagnostics: object
8: object
name: "sendTBTRoute"
description: "Calculates route and initiate download of turn by turn directions to the vehicle."
url: "https://api.gm.com/api/v1/account/vehicles/***********/commands/sendTBTRoute"
isPrivSessionRequired: "false"
9: object
name: "location"
description: "Retrieves the vehicle's current location."
url: "https://api.gm.com/api/v1/account/vehicles/***********/commands/location"
isPrivSessionRequired: "true"
[10 … 16]
10: object
name: "connect"
description: "Initiates a connection to the vehicle"
url: "https://api.gm.com/api/v1/account/vehicles/***********/commands/connect"
isPrivSessionRequired: "false"
11: object
name: "getHotspotStatus"
description: "Retrive WiFi status"
url: "https://api.gm.com/api/v1/account/vehicles/***********/hotspot/commands/getStatus"
isPrivSessionRequired: "false"
12: object
name: "setHotspotInfo"
description: "update the WiFi SSID and passPhrase"
url: "https://api.gm.com/api/v1/account/vehicles/***********/hotspot/commands/setInfo"
isPrivSessionRequired: "false"
13: object
name: "disableHotspot"
description: "Disable WiFi Hotspot"
url: "https://api.gm.com/api/v1/account/vehicles/***********/hotspot/commands/disable"
isPrivSessionRequired: "false"
14: object
name: "enableHotspot"
description: "Enable WiFi Hotspot"
url: "https://api.gm.com/api/v1/account/vehicles/***********/hotspot/commands/enable"
isPrivSessionRequired: "false"
15: object
name: "startTrailerLightSeq"
description: "Trailer Light Start"
url: "https://api.gm.com/api/v1/account/vehicles/***********/commands/startTrailerLightSeq"
isPrivSessionRequired: "true"
16: object
name: "stopTrailerLightSeq"
description: "Trailer Light Stop"
url: "https://api.gm.com/api/v1/account/vehicles/***********/commands/stopTrailerLightSeq"
isPrivSessionRequired: "true"
_msgid: "7a60794b245c3f5f"

Same error with it.
object
payload: "Error: Request Failed with status 403 - Forbidden"
_msgid: "89c3a4961284a76d"

All is good now, thank you so much!!
MQTT threw an error too and said it was on the vin number so I went back and checked. It was supposed to be a 0 (zero) and not an O. Thank you for your patience!

Absolutely, same here! Thanks again!!