TF2 porting: ImportError: cannot import name 'flatten'

Describe the bug
@w4nderlust
I'm using this issue to report a problem I encountered, outside of the context of a PR.  Before  starting my next task, I ran the unit test tests/integration_tests/test_simple_features.py  This resulted in this error:
_____________________________________ ERROR collecting tests/integration_tests/test_simple_features.py _____________________________________
ImportError while importing test module '/opt/project/tests/integration_tests/test_simple_features.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/integration_tests/test_simple_features.py:24: in <module>
    from ludwig.experiment import full_experiment
ludwig/experiment.py:32: in <module>
    from ludwig.data.postprocessing import postprocess
ludwig/data/postprocessing.py:20: in <module>
    from ludwig.features.feature_registries import output_type_registry
ludwig/features/feature_registries.py:39: in <module>
    from ludwig.features.image_feature import ImageBaseFeature
ludwig/features/image_feature.py:30: in <module>
    from ludwig.models.modules.image_encoders import ResNetEncoder
ludwig/models/modules/image_encoders.py:20: in <module>
    from ludwig.models.modules.convolutional_modules import flatten, ConvStack2D, \
E   ImportError: cannot import name 'flatten'

To Reproduce
Steps to reproduce the behavior:

Position at this commit https://github.com/uber/ludwig/pull/682/commits/d76ec6c5fa7fc839882e5d8c5edf2a54650516b1
pytest tests/integration_tests/test_simple_feature.py

Expected behavior
Successfully execution of the unit test
Screenshots
N/A
Environment (please complete the following information):

OS: [e.g. iOS]
Version [e.g. 22]
Python version
Ludwig version in-progress version for conversion to TF2

Additional context
This commit https://github.com/uber/ludwig/pull/682/commits/9342276c9504bd211b2c441b18650050857aab83 removed the flatten() function.  I could not locate in the code base the new location for flatten() (and the other functions).

I must have removed even if it was used buring cleanup. we can use tf.keras.layers.Flatten instead.


feat:H1005-L2 - Payment capture and void for D-local

Type of Change

[ ] Bugfix
[ ] New feature
[x] Enhancement
[ ] Refactoring
[ ] Dependency updates

Description
Added Capture and void methods
Additional Changes

[ ] This PR modifies the database schema
[ ] This PR modifies application configuration/environment variables

Motivation and Context

How did you test it?

Checklist

[ ] I formatted the code cargo +nightly fmt --all
[ ] I addressed lints thrown by cargo clippy
[ ] I reviewed submitted code
[ ] I added unit tests for my changes where possible
[ ] I added a CHANGELOG entry if applicable

Duplicate PR


Improve test coverage again

This patch adds tests for radio buttons, the date picker, as well as a
number of other classes.

@Hixie

lgtm


☣ Refactor transformMdast and postProcessMdast to a single pipeline

This spike investigates fusing our single and multi-document pipelines. The motivation for this follows from previous conversations at SciPy 2024 about making more responsive MyST start builds, and capturing transform dependencies.
We originally discussed building MyST transforms as a directional graph. This PR is much less ambitious; it's not clear that a directional graph is necessarily desirable; MyST would become much harder to reason about if we can't treat it as a linear series of transforms.
There were other reasons not do go as far as a graph, but I can't remember them at this very moment.

Support fine-grained before and after constraints on plugins
Reason about MyST "pipeline" in one place
Prepare for customisable pipelines e.g. fast and slow passes

We did not have to fuse the pre and post phases; that was a choice to put logic in one place. The rationale is that we can treat the need perform referencing across pages as a "sync" point for builds within a single-document pipeline.

I'm breaking for Christmas, but I will take notes here of what is left to do!

[ ] Simplify set-shared-state & reference-resolution -- this obfuscates parsing of shared states and then generation of MultiPageReferenceResolver. This can be done in a single blocking promise, and should be written somewhere for index generation to use.
[ ] Move body of indexingPromise.then() to transformMdast, and introduce a blocking sync point after?
[ ] Maybe add implementation of barrier makeBarrier(name: string, impl: () => void) to allow barrier body (e.g. cross-reference generation) to be defined in transformMdast?
[ ] Change transformMdast to build a pipeline, and have the caller invoke it.
[ ] Change finalizeMdast to build a pipeline, and have the caller invoke it.


bhy2_get_and_process_fifo does not work

bhy2_get_and_process_fifo does not fire callbacks that were added with bhy2_register_fifo_parse_callback
The reason for this is that the callback is not properly retrieved in the get_callback_info function. There are bugs on lines 11381338 and 11391339 of bhy2.c. Instead of:
info->callback = NULL;
info->callback_ref = NULL;

those lines should be:
info->callback = dev->table[sensor_id].callback;
info->callback_ref = dev->table[sensor_id].callback_ref;

When this change is made, then callbacks that were registered with bhy2_register_fifo_parse_callback are called when the appropriate data is in the FIFO, when bhy2_get_and_process_fifo is called.

Not the owner of this repository, but I was looking at open issues and found yours, The version of the library checked in this repository on May 13 2020, has the following starting at line 1147. It looks as if you are referring to a different version of the library, given the different coding conventions and line numbering
I recently ported the library to a Nordic microprocessor, and callbacks work as expected
int8_t bhy2_register_fifo_parse_callback(uint8_t sensor_id,
                                         bhy2_fifo_parse_callback_t callback,
                                         void *callback_ref,
                                         struct bhy2_dev *dev)
{
    int8_t rslt = BHY2_OK;

    if ((dev == NULL) || (callback == NULL))
    {
        rslt = BHY2_E_NULL_PTR;
    }
    else
    {
        dev->table[sensor_id].callback = callback;
        dev->table[sensor_id].callback_ref = callback_ref;
    }

    return rslt;
}

Ah, looks like I made a mistake in writing the line numbers in my original comment. I meant to write "lines 1338 and lines 1339" instead of "1138 and 1139". I've updated it now.
Thanks for catching that.

But... those lines are for the get_callback_info(), which is not supposed to set any callback.
Also, the current version of the library uses
        dev->table[sensor_id].callback = callback;
        dev->table[sensor_id].callback_ref = callback_ref;

and not
info->callback = dev->table[sensor_id].callback;
info->callback_ref = dev->table[sensor_id].callback_ref;

you have info-> and the current code has dev->
Let me ask again: you really using the library in this repository or a different version?
Mine works because the code is written as you suggest should be written :) but it has been that way for almost two years (in this repository)

get_callback_info() is called by parse_fifo() in the same file. parse_fifo() is supposed to get the registered callbacks via that function, and then invokes them. Perhaps we're using the library in a different way?
I had to make this change to get the example program in euler.c, version 5b26dc4.

I'm not making myself clear, sorry.
Try downloading this repository and compare it with your version of the same library. The code you refer to, doesn't exist in this repository. I believe you are using a version you downloaded from somewhere else.
I understand how the code works. I ported it to a Nordic processor (Arm M4 core), and the callbacks work as expected (from bhy2_get_and_process_fifo() which in turns calls parse_fifo(). The code in this repository correctly registers all callbacks

My initial complaint was that the callbacks that were registered with bhy2_register_fifo_parse_callback were not subsequently invoked. Not that they were not correctly registered - because they are indeed correctly registered. The problem that I have seen is that once those callbacks are registered, they are not correctly retrieved.
I cloned this repository directly from GitHub using git, and the code that I mentioned exists in this repository as I've specified:
https://github.com/BoschSensortec/BHY2-Sensor-API/blob/5b26dc4802f500f75e1086c2a156e3ecf800560c/bhy2.c#L1338
https://github.com/BoschSensortec/BHY2-Sensor-API/blob/5b26dc4802f500f75e1086c2a156e3ecf800560c/bhy2.c#L1339

As I said, it's working for me, for all the virtual sensors I register and initialize
My reading of the get_callback_info() function, is that it "nulls" the callback pointers only if one of these two conditions is met
if ((sensor_id >= BHY2_SPECIAL_SENSOR_ID_OFFSET) && (info->event_size == 0))
or
if ((sensor_id == 0) && (info->event_size == 0))
In both those cases, you want the callbacks to be NULL (BHY2_SPECIAL_SENSOR_ID_OFFSET is 245, so higher than any standard virtual sensor
in all the other cases, the callback pointers are retained from the previous registration, and invoked by line 1481 in parse_fifo()
 info.callback(&data_info, info.callback_ref);
Your proposed fix will not help in the normal code flow, and actually risks causing problems to the proper code execution (once again, I'm saying this as a semi-clueless user of this library and in the spirit of trying to help)

I appreciate the discussion on this. I'm also semi-clueless on this library. I was just posting the change that I made in order to make to get the example program in euler.c to work. Perhaps there is something else wrong with that example.

Unrelated comment:
@robcazzaro I need also to implement this in a nordic micro. Could you share with me what you have done so far?

@dariosortino I never took the time to clean things up and post it. But I should :)
To give you at least a starting point, I created a placeholder project and added an issue where you can see what I did and ask questions. I don't want to sidetrack this issue with unrelated content. Please feel free to add comments/questions to  https://github.com/robcazzaro/nRF52_BHY2_API/issues/1
For what is worth, it's working really well and even if the lack of documentation is annoying, the Bosch BHY2 library is well written and functional. I have not found any bug yet


Trust4 report less clonotype while sequence depth increase

Hello,
I was working on a single cell RNA-seq library which contains whole genome reads, not target-enrichment reads for immune profiling. I downsampled the library to 100k, 500k and 2million reads per cell. And found that for the same cell with more reads, the chance that a clonotype was found is actually decreased. For example, a cell could had clonotype identified when it only have 100k reads, but no clonotype identified if analyze the same cell with 500k and 2M reads. I couldn't understand how could this happen. Is this related to trust4's algorithm for consensus generation?
Thanks!

Is your data smart-seq scRNA-seq data? Could you please show me the commands you used? Thanks.

@mourisl yes, it is smart-seq scRNA-seq data. I was using the default commands
./run_trust4 -1 read1.fastq -2 read2.fastq -f  hg38_bcrtcr.fa --ref human_IMGT+C.fa
Is there specific parameters best suitable for smart-seq scRNA-seq data? Why the default one give less clonotype with increasing sequence depth? Thanks for your prompt reply!

Which version of TRUST4 did you use? In the older versions, there was bug with abundance estimation after scaffolding with mate-pair information, which could affect SMART-seq.
We provide a script "trust-smartseq.pl" in the repo to process SMART-seq data(https://github.com/liulab-dfci/TRUST4#smart-seq-data), which may save you some time.
One strange thing is that even there is a bug, it should still identify CDR3 sequences. Can you show me a few lines in the *_annot.fa file and *_cdr3.out file? Thanks.

Trust4 v1.0.4. The test samples are PBMCs, but based on gene expression, we've identified a group of T/B cells. I will pick up a few that should be T/B cells, but don't have CDR3/clonotype identified. Let me get back to you in ~1hr with the *_annot.fa file and *_cdr3.out file

@mourisl I've added the *_anno.fa and *_cdr3.out for the same cell but 100k and 500k depth to my data folder please check. Thanks in advance for your help!

Thanks for providing the files. From those files, from both depths, I can see the clonotype "TGTGCGAGAGATGTGGGGACCGGCCTGACTACACTTTACTACTTTGACTACTGG" for IGH and the corresponding assembly is also full-length. This clonotype should show up in the final report file as well. Could you please check whether the cdr3.out file matched the report.tsv file? You can upload the report.tsv file so I can check it too.

@mourisl Thank you! The tsv files are added.

I can see the clonotype "TGTGCGAGAGATGTGGGGACCGGCCTGACTACACTTTACTACTTTGACTACTGG" in both report files. Have you post-processed the report files? Or it is another cell have the issue? Thank you.

I've dropped records who are out-of-frame and/or partial, and dropped records whose count equal to 1 (use 2 as cutoff)

I see. Just want to confirm are the lost clonotypes the low abundant ones, which are not likely to be the clonotype for the gene? Those low abundant clonotypes are usually from sequencing errors, which could be fixed by contig scaffolding or other steps. From what I observe, the real clonotype for the cell is always assembled.
In SMART-seq, the filter should pick the most abundant clonotype for each chain instead of removing singleton CDR3s. The trust-smartseq.pl script will pick the representative clonotypes for you.

Thanks! I will use the trust-smartseq.pl script and follow up with you soon. Apperciate your help!

We computed the frequency for chains sperately. So for IG, there the IGH sums up to 1, the IGK/IGL(light chain) sums to 1. The TR entry probably from noise, and TRB and TRA should sum to 1 respectively.
For the trust-smartseq.pl: yes, each row is for the cell's fastq file.

@mourisl I used the --skipMateExtension, and analyzed with the fastq file I provided. The read count decrease. I don't understand why. I though skipMateExtension would not require pair-read overlap, thus have more read identifying clonotypes. Could you please take a look? Here are the report file cdr3.out annot.fa

The mate extension means scaffolding the contigs with mate-pair information. This will change the results in "trust_final.out", which could lead to different alignment results in the following steps including abundance estimation. The change is very small, so it is fine.
We can skip the scaffolding in SMART-seq is because the dominant clonotype is the only real one for a cell and it is usually already nicely assembled, there is no need to extend/fix those low abundant assemblies with mate-pair information to save running time.

@mourisl Thank you! If I understand correct, --skipMateExtension would save processing time, but won't contribute to detection accuracy? If we use bulk samples, not single cells, what parameter setting you would suggest to use?

Right, for the single-cell data, you can skip the scaffolding step.
The default parameter setting is for bulk samples, so it has already been optimized and no need to change.

@mourisl Thanks very much for your help so far. I tested Trust4 on a bulk library with 3.5million reads, the analysis seems to take a long time, I have to kill it. Do you know what's the possible reason for such long processing time?
log info below
process Sample1-1_S2_L001
[Fri Sep 17 11:35:40 2021] TRUST4 begins.
[Fri Sep 17 11:35:40 2021] SYSTEM CALL: /tools/TRUST4/fastq-extractor -1 Sample1-1_S2_L001_R1_001.fastq.gz -2 Sample1-1_S2_L001_R2_001.fastq.gz -t 1 -f /tools/TRUST4/hg38_bcrtcr.fa -o Sample1-1_S2_L001_toassemble
[Fri Sep 17 11:35:40 2021] Start to extract candidate reads from read files.
[Fri Sep 17 11:42:48 2021] Finish extracting reads.
[Fri Sep 17 11:42:48 2021] SYSTEM CALL: /tools/TRUST4/trust4  -o Sample1-1_S2_L001 -f /tools/TRUST4/hg38_bcrtcr.fa -1 Sample1-1_S2_L001_toassemble_1.fq -2 Sample1-1_S2_L001_toassemble_2.fq
[Fri Sep 17 11:42:51 2021] Read in and count kmers for 100000 reads.
[Fri Sep 17 11:42:55 2021] Read in and count kmers for 200000 reads.
[Fri Sep 17 11:42:58 2021] Read in and count kmers for 300000 reads.
[Fri Sep 17 11:43:02 2021] Read in and count kmers for 400000 reads.
[Fri Sep 17 11:43:06 2021] Read in and count kmers for 500000 reads.
[Fri Sep 17 11:43:10 2021] Read in and count kmers for 600000 reads.
[Fri Sep 17 11:43:14 2021] Read in and count kmers for 700000 reads.
[Fri Sep 17 11:43:18 2021] Read in and count kmers for 800000 reads.
[Fri Sep 17 11:43:22 2021] Read in and count kmers for 900000 reads.
[Fri Sep 17 11:43:26 2021] Read in and count kmers for 1000000 reads.
[Fri Sep 17 11:43:30 2021] Read in and count kmers for 1100000 reads.
[Fri Sep 17 11:43:35 2021] Read in and count kmers for 1200000 reads.
[Fri Sep 17 11:43:39 2021] Read in and count kmers for 1300000 reads.
[Fri Sep 17 11:43:44 2021] Read in and count kmers for 1400000 reads.
[Fri Sep 17 11:43:48 2021] Read in and count kmers for 1500000 reads.
[Fri Sep 17 11:43:53 2021] Read in and count kmers for 1600000 reads.
[Fri Sep 17 11:43:57 2021] Read in and count kmers for 1700000 reads.
[Fri Sep 17 11:44:02 2021] Read in and count kmers for 1800000 reads.
[Fri Sep 17 11:44:07 2021] Read in and count kmers for 1900000 reads.
[Fri Sep 17 11:44:12 2021] Read in and count kmers for 2000000 reads.
[Fri Sep 17 11:44:16 2021] Read in and count kmers for 2100000 reads.
[Fri Sep 17 11:44:22 2021] Read in and count kmers for 2200000 reads.
[Fri Sep 17 11:44:27 2021] Read in and count kmers for 2300000 reads.
[Fri Sep 17 11:44:31 2021] Read in and count kmers for 2400000 reads.
[Fri Sep 17 11:44:36 2021] Read in and count kmers for 2500000 reads.
[Fri Sep 17 11:44:41 2021] Read in and count kmers for 2600000 reads.
[Fri Sep 17 11:44:46 2021] Read in and count kmers for 2700000 reads.
[Fri Sep 17 11:44:51 2021] Read in and count kmers for 2800000 reads.
[Fri Sep 17 11:44:56 2021] Read in and count kmers for 2900000 reads.
[Fri Sep 17 11:45:01 2021] Read in and count kmers for 3000000 reads.
[Fri Sep 17 11:45:07 2021] Read in and count kmers for 3100000 reads.
[Fri Sep 17 11:45:13 2021] Read in and count kmers for 3200000 reads.
[Fri Sep 17 11:45:19 2021] Read in and count kmers for 3300000 reads.
[Fri Sep 17 11:45:25 2021] Read in and count kmers for 3400000 reads.
[Fri Sep 17 11:45:30 2021] Read in and count kmers for 3500000 reads.
[Fri Sep 17 11:45:36 2021] Read in and count kmers for 3600000 reads.
[Fri Sep 17 11:45:42 2021] Read in and count kmers for 3700000 reads.
[Fri Sep 17 11:45:48 2021] Read in and count kmers for 3800000 reads.
[Fri Sep 17 11:45:53 2021] Read in and count kmers for 3900000 reads.
[Fri Sep 17 11:45:59 2021] Read in and count kmers for 4000000 reads.
[Fri Sep 17 11:46:05 2021] Read in and count kmers for 4100000 reads.
[Fri Sep 17 11:46:11 2021] Read in and count kmers for 4200000 reads.
[Fri Sep 17 11:46:18 2021] Read in and count kmers for 4300000 reads.
[Fri Sep 17 11:46:23 2021] Read in and count kmers for 4400000 reads.
[Fri Sep 17 11:46:29 2021] Read in and count kmers for 4500000 reads.
[Fri Sep 17 11:46:35 2021] Read in and count kmers for 4600000 reads.
[Fri Sep 17 11:46:41 2021] Read in and count kmers for 4700000 reads.
[Fri Sep 17 11:46:47 2021] Read in and count kmers for 4800000 reads.
[Fri Sep 17 11:46:53 2021] Read in and count kmers for 4900000 reads.
[Fri Sep 17 11:46:59 2021] Read in and count kmers for 5000000 reads.
[Fri Sep 17 11:47:05 2021] Read in and count kmers for 5100000 reads.
[Fri Sep 17 11:47:11 2021] Read in and count kmers for 5200000 reads.
[Fri Sep 17 11:47:17 2021] Read in and count kmers for 5300000 reads.
[Fri Sep 17 11:47:23 2021] Read in and count kmers for 5400000 reads.
[Fri Sep 17 11:47:30 2021] Read in and count kmers for 5500000 reads.
[Fri Sep 17 11:47:36 2021] Read in and count kmers for 5600000 reads.
[Fri Sep 17 11:47:42 2021] Read in and count kmers for 5700000 reads.
[Fri Sep 17 11:47:48 2021] Read in and count kmers for 5800000 reads.
[Fri Sep 17 11:47:54 2021] Read in and count kmers for 5900000 reads.
[Fri Sep 17 11:48:01 2021] Read in and count kmers for 6000000 reads.
[Fri Sep 17 11:48:07 2021] Read in and count kmers for 6100000 reads.
[Fri Sep 17 11:48:13 2021] Read in and count kmers for 6200000 reads.
[Fri Sep 17 11:48:19 2021] Read in and count kmers for 6300000 reads.
[Fri Sep 17 11:48:26 2021] Read in and count kmers for 6400000 reads.
[Fri Sep 17 11:48:32 2021] Read in and count kmers for 6500000 reads.
[Fri Sep 17 11:48:38 2021] Read in and count kmers for 6600000 reads.
[Fri Sep 17 11:48:44 2021] Read in and count kmers for 6700000 reads.
[Fri Sep 17 11:48:51 2021] Read in and count kmers for 6800000 reads.
[Fri Sep 17 11:48:57 2021] Read in and count kmers for 6900000 reads.
[Fri Sep 17 11:49:03 2021] Read in and count kmers for 7000000 reads.
[Fri Sep 17 11:49:09 2021] Read in and count kmers for 7100000 reads.
[Fri Sep 17 11:55:12 2021] Found 7058535 reads.
[Fri Sep 17 11:55:30 2021] Finish sorting the reads.
[Fri Sep 17 12:37:03 2021] Finish rough annotations.
[Fri Sep 17 12:37:11 2021] Processed 100000 reads (99978 are used for assembly).
[Fri Sep 17 12:37:15 2021] Processed 200000 reads (199950 are used for assembly).
[Fri Sep 17 12:37:21 2021] Processed 300000 reads (299837 are used for assembly).
[Fri Sep 17 12:37:30 2021] Processed 400000 reads (399469 are used for assembly).
[Fri Sep 17 12:37:38 2021] Processed 500000 reads (499413 are used for assembly).
[Fri Sep 17 12:37:46 2021] Processed 600000 reads (599381 are used for assembly).
[Fri Sep 17 12:37:54 2021] Processed 700000 reads (699356 are used for assembly).
[Fri Sep 17 12:38:04 2021] Processed 800000 reads (799338 are used for assembly).
[Fri Sep 17 12:38:15 2021] Processed 900000 reads (899293 are used for assembly).
[Fri Sep 17 12:38:27 2021] Processed 1000000 reads (999270 are used for assembly).
[Fri Sep 17 12:38:40 2021] Processed 1100000 reads (1099247 are used for assembly).
[Fri Sep 17 12:38:54 2021] Processed 1200000 reads (1199032 are used for assembly).
[Fri Sep 17 12:39:10 2021] Processed 1300000 reads (1298983 are used for assembly).
[Fri Sep 17 12:39:27 2021] Processed 1400000 reads (1398901 are used for assembly).
[Fri Sep 17 12:39:45 2021] Processed 1500000 reads (1498828 are used for assembly).
[Fri Sep 17 12:40:08 2021] Processed 1600000 reads (1598777 are used for assembly).
[Fri Sep 17 12:40:28 2021] Processed 1700000 reads (1698693 are used for assembly).
[Fri Sep 17 12:40:50 2021] Processed 1800000 reads (1798597 are used for assembly).
[Fri Sep 17 12:41:12 2021] Processed 1900000 reads (1898490 are used for assembly).
[Fri Sep 17 12:41:35 2021] Processed 2000000 reads (1998075 are used for assembly).
[Fri Sep 17 12:42:01 2021] Processed 2100000 reads (2097801 are used for assembly).
[Fri Sep 17 12:42:18 2021] Processed 2200000 reads (2197482 are used for assembly).
[Fri Sep 17 12:42:35 2021] Processed 2300000 reads (2297076 are used for assembly).
[Fri Sep 17 12:42:54 2021] Processed 2400000 reads (2396635 are used for assembly).
[Fri Sep 17 12:43:17 2021] Processed 2500000 reads (2496112 are used for assembly).
[Fri Sep 17 12:43:42 2021] Processed 2600000 reads (2595318 are used for assembly).
[Fri Sep 17 12:44:08 2021] Processed 2700000 reads (2694554 are used for assembly).
[Fri Sep 17 12:44:35 2021] Processed 2800000 reads (2793944 are used for assembly).
[Fri Sep 17 12:45:09 2021] Processed 2900000 reads (2893255 are used for assembly).
[Fri Sep 17 12:45:42 2021] Processed 3000000 reads (2992666 are used for assembly).
[Fri Sep 17 12:46:18 2021] Processed 3100000 reads (3091965 are used for assembly).
[Fri Sep 17 12:46:57 2021] Processed 3200000 reads (3191364 are used for assembly).
[Fri Sep 17 12:47:42 2021] Processed 3300000 reads (3290672 are used for assembly).
[Fri Sep 17 12:48:25 2021] Processed 3400000 reads (3390001 are used for assembly).
[Fri Sep 17 12:49:19 2021] Processed 3500000 reads (3489281 are used for assembly).
[Fri Sep 17 12:50:22 2021] Processed 3600000 reads (3588479 are used for assembly).
[Fri Sep 17 12:51:26 2021] Processed 3700000 reads (3687697 are used for assembly).
[Fri Sep 17 12:52:29 2021] Processed 3800000 reads (3786843 are used for assembly).
[Fri Sep 17 12:53:42 2021] Processed 3900000 reads (3885978 are used for assembly).
[Fri Sep 17 12:54:58 2021] Processed 4000000 reads (3985108 are used for assembly).
[Fri Sep 17 12:56:20 2021] Processed 4100000 reads (4084226 are used for assembly).
[Fri Sep 17 12:57:47 2021] Processed 4200000 reads (4183187 are used for assembly).
[Fri Sep 17 12:59:16 2021] Processed 4300000 reads (4282254 are used for assembly).
[Fri Sep 17 13:00:57 2021] Processed 4400000 reads (4381491 are used for assembly).
[Fri Sep 17 13:02:49 2021] Processed 4500000 reads (4480652 are used for assembly).
[Fri Sep 17 13:04:37 2021] Processed 4600000 reads (4579897 are used for assembly).
[Fri Sep 17 13:06:50 2021] Processed 4700000 reads (4679044 are used for assembly).
[Fri Sep 17 13:08:56 2021] Processed 4800000 reads (4778068 are used for assembly).
[Fri Sep 17 13:10:52 2021] Processed 4900000 reads (4877253 are used for assembly).
[Fri Sep 17 13:12:40 2021] Processed 5000000 reads (4976344 are used for assembly).
[Fri Sep 17 13:14:24 2021] Processed 5100000 reads (5075380 are used for assembly).
[Fri Sep 17 13:16:01 2021] Processed 5200000 reads (5174691 are used for assembly).
[Fri Sep 17 13:17:41 2021] Processed 5300000 reads (5273659 are used for assembly).
[Fri Sep 17 13:19:35 2021] Processed 5400000 reads (5372736 are used for assembly).
[Fri Sep 17 13:21:56 2021] Processed 5500000 reads (5471632 are used for assembly).
[Fri Sep 17 13:24:37 2021] Processed 5600000 reads (5570450 are used for assembly).
[Fri Sep 17 13:28:17 2021] Processed 5700000 reads (5669612 are used for assembly).
[Fri Sep 17 13:34:16 2021] Processed 5800000 reads (5768472 are used for assembly).
[Fri Sep 17 13:43:48 2021] Processed 5900000 reads (5867499 are used for assembly).
[Fri Sep 17 13:57:07 2021] Processed 6000000 reads (5966298 are used for assembly).
[Fri Sep 17 14:15:02 2021] Processed 6100000 reads (6065356 are used for assembly).
[Fri Sep 17 14:36:52 2021] Processed 6200000 reads (6164045 are used for assembly).
[Fri Sep 17 15:07:54 2021] Processed 6300000 reads (6262443 are used for assembly).
[Fri Sep 17 15:46:38 2021] Processed 6400000 reads (6362442 are used for assembly).
[Fri Sep 17 17:26:50 2021] Processed 6500000 reads (6459956 are used for assembly).
[Fri Sep 17 19:44:11 2021] Processed 6600000 reads (6559954 are used for assembly).
[Fri Sep 17 23:01:21 2021] Processed 6700000 reads (6659949 are used for assembly).
[Sat Sep 18 04:03:11 2021] Processed 6800000 reads (6759446 are used for assembly).
[Sat Sep 18 08:56:36 2021] Processed 6900000 reads (6857285 are used for assembly).
[Sat Sep 18 11:09:36 2021] Processed 7000000 reads (6955708 are used for assembly).
[Sat Sep 18 11:24:07 2021] Assembled 7001153 reads.
[Sat Sep 18 11:24:07 2021] Try to rescue 0 reads for assembly.
[Sat Sep 18 11:24:08 2021] Rescued 0 reads.
[Sat Sep 18 12:38:45 2021] Processed 100000 reads for extension.
[Sat Sep 18 14:01:46 2021] Processed 200000 reads for extension.
[Sat Sep 18 15:29:48 2021] Processed 300000 reads for extension.
[Sat Sep 18 16:56:27 2021] Processed 400000 reads for extension.
[Sat Sep 18 18:20:46 2021] Processed 500000 reads for extension.
[Sat Sep 18 19:51:23 2021] Processed 600000 reads for extension.
[Sat Sep 18 21:16:02 2021] Processed 700000 reads for extension.
[Sat Sep 18 22:41:34 2021] Processed 800000 reads for extension.
[Sun Sep 19 00:09:26 2021] Processed 900000 reads for extension.
[Sun Sep 19 01:36:09 2021] Processed 1000000 reads for extension.
[Sun Sep 19 03:03:17 2021] Processed 1100000 reads for extension.
[Sun Sep 19 04:27:38 2021] Processed 1200000 reads for extension.
[Sun Sep 19 06:02:43 2021] Processed 1300000 reads for extension.
[Sun Sep 19 07:31:22 2021] Processed 1400000 reads for extension.
[Sun Sep 19 08:58:56 2021] Processed 1500000 reads for extension.
[Sun Sep 19 10:35:15 2021] Processed 1600000 reads for extension.
[Sun Sep 19 12:05:39 2021] Processed 1700000 reads for extension.
[Sun Sep 19 13:37:01 2021] Processed 1800000 reads for extension.
[Sun Sep 19 15:19:22 2021] Processed 1900000 reads for extension.
[Sun Sep 19 16:52:24 2021] Processed 2000000 reads for extension.
[Sun Sep 19 18:28:40 2021] Processed 2100000 reads for extension.
[Sun Sep 19 19:59:32 2021] Processed 2200000 reads for extension.
[Sun Sep 19 21:28:09 2021] Processed 2300000 reads for extension.
[Sun Sep 19 22:59:41 2021] Processed 2400000 reads for extension.
[Mon Sep 20 00:23:33 2021] Processed 2500000 reads for extension.
[Mon Sep 20 01:42:56 2021] Processed 2600000 reads for extension.
[Mon Sep 20 03:02:33 2021] Processed 2700000 reads for extension.
[Mon Sep 20 04:20:41 2021] Processed 2800000 reads for extension.
[Mon Sep 20 05:46:09 2021] Processed 2900000 reads for extension.
[Mon Sep 20 07:06:43 2021] Processed 3000000 reads for extension.
[Mon Sep 20 08:25:10 2021] Processed 3100000 reads for extension.
[Mon Sep 20 09:46:26 2021] Processed 3200000 reads for extension.
[Mon Sep 20 11:10:19 2021] Processed 3300000 reads for extension.
[Mon Sep 20 12:32:42 2021] Processed 3400000 reads for extension.
[Mon Sep 20 13:50:44 2021] Processed 3500000 reads for extension.
[Mon Sep 20 15:12:39 2021] Processed 3600000 reads for extension.
[Mon Sep 20 16:30:15 2021] Processed 3700000 reads for extension.
[Mon Sep 20 17:43:52 2021] Processed 3800000 reads for extension.
[Mon Sep 20 19:03:47 2021] Processed 3900000 reads for extension.
[Mon Sep 20 20:19:06 2021] Processed 4000000 reads for extension.
[Mon Sep 20 21:35:07 2021] Processed 4100000 reads for extension.
[Mon Sep 20 22:54:59 2021] Processed 4200000 reads for extension.
[Tue Sep 21 00:07:14 2021] Processed 4300000 reads for extension.
[Tue Sep 21 01:27:10 2021] Processed 4400000 reads for extension.
[Tue Sep 21 02:45:08 2021] Processed 4500000 reads for extension.
[Tue Sep 21 03:59:03 2021] Processed 4600000 reads for extension.
[Tue Sep 21 05:19:32 2021] Processed 4700000 reads for extension.
[Tue Sep 21 06:42:05 2021] Processed 4800000 reads for extension.
[Tue Sep 21 08:06:19 2021] Processed 4900000 reads for extension.
[Tue Sep 21 09:31:01 2021] Processed 5000000 reads for extension.
[Tue Sep 21 11:02:20 2021] Processed 5100000 reads for extension.
[Tue Sep 21 12:28:48 2021] Processed 5200000 reads for extension.
[Tue Sep 21 13:56:51 2021] Processed 5300000 reads for extension.
[Tue Sep 21 15:38:38 2021] Processed 5400000 reads for extension.
[Tue Sep 21 17:30:27 2021] Processed 5500000 reads for extension.
[Tue Sep 21 19:41:21 2021] Processed 5600000 reads for extension.
[Tue Sep 21 22:36:37 2021] Processed 5700000 reads for extension.
[Wed Sep 22 03:01:35 2021] Processed 5800000 reads for extension.
[Wed Sep 22 07:29:30 2021] Processed 5900000 reads for extension.

Is this TCR-seq or BCR-seq data? For these TCR/BCR amplified bulk data set, you may want to try the option --repseq for faster processing. It looks like you are running TRUST4 with a single thread, you can try multi-threading (-t N) to further increase the speed.

@mourisl It's BCR-seq. I will try both --repseq and -t. Thanks!

@mourisl should the command be ./run_trust4 -1 read1.fastq -2 read2.fastq -f hg38_bcrtcr.fa --ref human_IMGT+C.fa --repseq BCR-seq -t 30
It returned error saying "Unknown parameter BCR-seq". I'm using the latest verstion TRUST4-<IP_ADDRESS>

You can run it with ./run_trust4 -1 read1.fastq -2 read2.fastq -f hg38_bcrtcr.fa --ref human_IMGT+C.fa --repseq -t 30 . (--repseq is a flag, it does not need parameter).

@mourisl Thank you! I found that the latest version "Fix a serious bug in the run-trust4 wrapper in the original v1.0.5 release for single-cell data." Is this bug exist in v1.0.4? I had completed quite some analysis with v1.0.4, just wondering if there is need to rerun. Thanks!

--repseq will trim the read ends when TRUST4 found sequences might be outside V, J area. Furthermore, internally, it will only consider overlaps with compatible V genes and J genes. Therefore, with "repseq" option, the read-contig overlaps are much less, with a tradeoff of slightly less sensitivity.
The bug is not in v1.0.4. The bug was that TRUST4 did not generate the cdr3 file and report files given the --barcode option.

@mourisl Thanks again for the clarification!

@mourisl I'm trying to visualize the Trust4 alignment and assembling results by matching with the germline sequences. But couldn't figure out a way to use sequences & scores in trust_cdr3.out & trust_annot.fa for visualization purpose. Do you have suggestion for tools I could use? Thanks again!

You can use IMGT website or IgBLAST to align the sequences from trust_annot.fa to the germline database.
The "./annotator" program in TRUST4 also have the option "--geneAlignment". When using this option, in the sequence header of the trust_annot.fa it will also output 3 fields for the alignment on V, J, C regions. For each part, "=" represent match, capital nucleotide indicate the nucleotide in the assembly is different than the germline. "" indicate deletion on the assembly and lower-case indicates the inserted nucleotides in the assembly. The alignment are between the ranges shown in the headers of trust_annot.fa.


ARTEMIS-441 correct time-unit for large msg poll

I ran the PR build locally and it was successful.  I'm not sure what's going on with the automated one.


deep nested delete returns an error

Bug description
Having a schema like :   Invoice => InvoiceClient => InvoiceShippingAddress
I need to delete a shipping address based on the invoice update, I try :
prisma.invoice.update({
      where: { id: invoiceId },
      data: {
        invoiceClient: {
          update: {
            invoiceShippingAddress: { delete: true },
          },
        },
      },
    });

but I get the following error : Expected a valid parent ID to be present for nested update to-one case.
I have no clue why, because the update works using :
prisma.invoice.update({
      where: { id: invoice.id },
      data: {
        invoiceClient: {
          update: {
            invoiceShippingAddress: {
              upsert: {
                create: shippingAddressData,
                update: shippingAddressData,
              },
            },
          },
        },
      },
    });

How to reproduce

generate my schema
play the delete code

Expected behavior
should just delete as well
Prisma information
model Invoice {
  id        String @id(map: "invoice_id") @db.VarChar

  invoiceClientId String        @unique @db.VarChar
  invoiceClient   InvoiceClient @relation(fields: [invoiceClientId], references: [id], onDelete: Cascade)

  @@map("invoice")
}

model InvoiceClient {
  id String @id(map: "invoice_client_id") @db.VarChar

  invoiceId String   @db.VarChar
  invoice   Invoice?

  invoiceBillingAddressId String?               @unique @db.VarChar
  invoiceBillingAddress   InvoiceClientAddress? @relation("InvoiceBillingAddress", fields: [invoiceBillingAddressId], references: [id], onDelete: Cascade)

  invoiceShippingAddressId String?               @unique @db.VarChar
  invoiceShippingAddress   InvoiceClientAddress? @relation("InvoiceShippingAddress", fields: [invoiceShippingAddressId], references: [id], onDelete: Cascade)

  @@map("invoice_client")
}

model InvoiceClientAddress {
  id        String @id(map: "invoice_client_address_id") @db.VarChar
  companyId String @db.VarChar

  billingAddressClient  InvoiceClient? @relation("InvoiceBillingAddress")
  shippingAddressClient InvoiceClient? @relation("InvoiceShippingAddress")

  streetAddress     String? @db.VarChar
  additionalAddress String? @db.VarChar
  postCode          String? @db.VarChar
  city              String? @db.VarChar
  country           String? @db.VarChar

  @@index([companyId], map: "invoice_client_address_companyId")
  @@map("invoice_client_address")
}

Environment & setup

OS: MacOS
Database: PostgreSQL
Node.js version: v16.13.1

Prisma Version
prisma                  : 3.14.0
@prisma/client          : 3.14.0
Current platform        : darwin-arm64
Query Engine (Node-API) : libquery-engine 2b0c12756921c891fec4f68d9444e18c7d5d4a6a (at<EMAIL_ADDRESS>Migration Engine        : migration-engine-cli 2b0c12756921c891fec4f68d9444e18c7d5d4a6a (at node_modules/@prisma/engines/migration-engine-darwin-arm64)
Introspection Engine    : introspection-core 2b0c12756921c891fec4f68d9444e18c7d5d4a6a (at node_modules/@prisma/engines/introspection-engine-darwin-arm64)
Format Binary           : prisma-fmt 2b0c12756921c891fec4f68d9444e18c7d5d4a6a (at node_modules/@prisma/engines/prisma-fmt-darwin-arm64)
Default Engines Hash    : 2b0c12756921c891fec4f68d9444e18c7d5d4a6a
Studio                  : 0.460.0

I could not reproduce this issue with either latest dev or 3.14.0.  Here is the code I am trying:
  await prisma.invoice.create({
    data: {
      id: '1',
      invoiceClient: {
        create: {
          id: '2',
          invoiceShippingAddress: {
            create: {
              id: '3',
            },
          },
        },
      },
    },
  })

  await prisma.invoice.update({
    where: { id: '1' },
    data: {
      invoiceClient: {
        update: {
          invoiceShippingAddress: { delete: true },
        },
      },
    },
  })

@jbdemonte would it be possible to show us what data existis in DB before update call and how it's created? Thank you

@SevInf As you can imagine, since Jun, we had tons of release, so, I'm sorry but I'm not able to get such an old database backup.
we're up to date with the latest prisma.
I just retry and I no longer had this issue, I'll update the workaround we put 😅
btw, thanks for your help


Allow early suppressing of file not found warnings in Zend_Application

Needed to prevent warnings when the (default) Bootstrap object is set, which will trigger a load of ZendX_Application_Resource_FrontController, as described in https://github.com/zendframework/zf1/issues/564

Please add an unit test. (It's very simple!)
Thanks.

Ok test added, this only tests whether the constructor respects the boolean flag.
It does not test the failed loading attempt of ZendX_Application_Resource_FrontController (not sure if this is a bug..., and if so, how to test it, since the non-optimized files DO use the require_once statements, include-paths, etc).
PS, already signed a CLA, while the GitHub bot says I should sign one...

Ok test added

:+1:

already signed a CLA, while the GitHub bot says I should sign one...

We know!

@holtkamp
Thanks!


Extending ERExpressionStringBase is not possible since PU26 or PU27

I'm trying to extend the class ERExpressionStringBase but now, visual studio doesn't allow it since the class has become internal and the methods are not hookable. Thus I get the following error message when compiling : Class 'ERExpressionStringBase' is internal and is not accessible from the current module '[Our Module]'.
Could you please share how to extend the ER functions after the new updates ? Nothing is mentioned on this page.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 7cb9b46a-66ea-0e0c-c7e1-381da4af0a1f
Version Independent ID: 75c59226-4213-3b3f-a8c5-7c77a36aab6a
Content: Extend the list of Electronic reporting (ER) functions - Finance & Operations | Dynamics 365 | #MSDyn365FO
Content Source: articles/dev-itpro/analytics/general-electronic-reporting-formulas-list-extension.md
Service: dynamics-ax-platform
GitHub Login: @NickSelin
Microsoft Alias: nselin

@Apsia-asarazin, thanks for posting this.
@NickSelin, please review and update the content as needed.

As described in the https://docs.microsoft.com/en-us/dynamics365/unified-operations/dev-itpro/analytics/er-apis-app73 page, starting from the 7.3 version of Dynamics 365 for Finance and Operations certain classes were marked as internal. Therefore, for the 7.3 version or higher you can’t extend the list of available ER functions any longer.
Please, consider the following options instead of introduction of custom ER functions to transform texts:

Use the built-in REPLACE function defining necessary regular expressions in arguments to support complex text transformations;
Implement necessary text transformation logic in methods of a new public class. Then, refer to this class from a new ER datasource of the Class type and call those methods with parameters from ER format or mapping;
Register a new request for a new ER extensibility point to obtain a new built-in function.


typo in HTTPClient.gen.go

Just found there is typo of TLS (not TSL) in HTTPClient.gen.go.

Thank you @reidlai!


Set default sprite width

Hi,
is it possible to set default width for the generated sprite.svg?
My concern is when sprite dimensions change when we add new svg file and that file width is bigger then the current sprite then sprite gets new width. We use sprite as background image. And if sprite width gets changed then background-size wouldn't be correct anymore. I hope I'm making any sense.
Thank you for any help.

Hey @marjanbajc,
no, there's no way to limit or control the width of the sprite. And in fact I cannot imagine the circumstances under which this would be necessary. I'd even say that if you're really depending on the sprite width, you're doing something wrong.
It is perfectly possible to use a sprite as background image in a way that is not depending on the sprite width. Unfortunately I'm not able to elaborate deeply on this, but just to give you a hint: You will have to choose an orthogonal sprite layout ("vertical" or "horizontal", depending on your situation), make sure that the shapes all have the same (in your case) width (you could use the maxWidth option for this) and set the background image to background-size: cover or background-size: 100% auto.
Good luck, cheers, Joschi

Hmmm, I think I know what you mean. I'll give it a try. Thank you for your help.


chat(hash) to also return raw response + header object

Currently chat(messages) returns a hash {role: "assistant", content: "Welcome"} and there is no way to access the raw response object which contains usage data:
"usage":{"prompt_tokens":163,"prompt_time":0.059,"completion_tokens":481,"completion_time":1.591,"total_tokens":644,"total_time":1.65}

Nor does it return the HTTP header information that contains rate limit stats:
x-ratelimit-limit-requests: "14400"
x-ratelimit-limit-tokens: "7000"
x-ratelimit-remaining-requests: "14399"
x-ratelimit-remaining-tokens: "6818"
x-ratelimit-reset-requests: "6s"
x-ratelimit-reset-tokens: "1.559999999s"
x-request-id: "req_01hwe17kc9efg93xtwk30d5tx4"

Can we change chat() to return multiple response objects?
message, response, headers = @client.chat("Hello?")
# if you just want one
message, _ = @client.chat("Hello?)

Perhaps this mode is enabled via config @client.chat("Hello?", metadata: true)

I'd love this too!  Currently struggling to stay inside their rate limits & it's hard without better ability to show their headers.  I think it should be enabled by default, or just auto-log with metadata: true., but better to return it as the other result imo.


[AIRFLOW-1052] Fix dags/test_dag.py to be importable

Dear Airflow maintainers,
Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
JIRA

[x] My PR addresses the following Airflow JIRA issues and references them in the PR title. For example, "[AIRFLOW-XXX] My Airflow PR"

https://issues.apache.org/jira/browse/AIRFLOW-1052

Description

[x] Here are some details about my PR, including screenshots of any UI changes:

AIRFLOW-863 broke dags/test_dag.py and it can not be imported now.
This PR fixes it.
Tests

[x] My PR does not need testing for this extremely good reason:

This fix is really trivial and I confirmed that it worked locally.
Commits

[x] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":

Subject is separated from body by a blank line
Subject is limited to 50 characters
Subject does not end with a period
Subject uses the imperative mood ("add", not "adding")
Body wraps at 72 characters
Body explains "what" and "why", not "how"

Codecov Report

Merging #2200 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master   #2200   +/-   ##
======================================
  Coverage    67.5%   67.5%
======================================
  Files         142     142
  Lines       10845   10845
======================================
  Hits         7321    7321
  Misses       3524    3524

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b55f41f...091bf76. Read the comment docs.

Closing this since it's duplicated with #2201.

@criccomini why was this closed?

... because it was merged :)
The only way tickets get closed is if they get merged. We (committers) don't have access to Github directly to close tickets.


Could not parse output script

Background
recovering old wallet, during the rescan periodically get :
2021-04-17 10:48:18.063 [INF] LNWL: Fetching block height=557404 hash=0000000000000000001f22120bf9fc524b5f39d9c05b7d71f0615da2c9b49c54
2021-04-17 10:48:18.437 [WRN] LNWL: Could not parse output script in f43ea9a2850482c5692b16a253cd9bf796832a30a05a63070c9b81acdac2b86b:2: opcode OP_DATA_75 requires 76 bytes, but script only has 39 remaining

rescan is still in progress, so don't know if all coins will be recovered in the end.
Your environment

lnd version 0.12.99-beta commit=v0.12.0-beta-373-ga36c95f7
Linux lnd0 5.4.0-51-generic #56-Ubuntu SMP Mon Oct 5 14:28:49 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
neutrino
fresh install

@Overtorment Did the rescan finish? Have all the coins recovered?

not yet,  ETA ~5 days (it was a phat production node, 150k addresses)

IIRC, that's just a benign message that pops up when someone creates an unspendable output: we'll fail to be able to parse the script, but it doesn't matter since the outputs the wallet uses are always standard.

Just a quick note about that this comes up a lot when rescanning on testnet:
[INF] LNWL: Scanning 2000 blocks for recoverable addresses
[WRN] LNWL: Could not parse output script in faf942db857b00137a8da9a73ee66cbd14b6d66c1d7f6f18edca7ad4a4460b1d:0: opcode OP_DATA_37 requires 38 bytes, but script only has 20 remaining
[WRN] LNWL: Could not parse output script in 1754f671d0e63daef88abc579eb2949018c936f3666a11277e7ac8fb6530a96a:0: opcode OP_DATA_57 requires 58 bytes, but script only has 9 remaining
[WRN] LNWL: Could not parse output script in f16107f7528fd68456d54290b2209e030cfb2bddd09bb9195a59bc87f9efca66:0: opcode OP_DATA_57 requires 58 bytes, but script only has 9 remaining
[WRN] LNWL: Could not parse output script in e7613d37bd8c0bf5ad778576389f92a12bfbec7e6b3e321fd8f41b4987527b24:0: opcode OP_DATA_32 requires 33 bytes, but script only has 21 remaining
[WRN] LNWL: Could not parse output script in bd2a59c59fba3b737e539723afb7f3d155f4730347094c8100085baff10d72ac:0: opcode OP_DATA_32 requires 33 bytes, but script only has 21 remaining
[WRN] LNWL: Could not parse output script in e817578e60fb20b06986a01b30eece3f756c75de4dbaf78258213090e830f86c:2: opcode OP_DATA_75 requires 76 bytes, but script only has 39 remaining
[WRN] LNWL: Could not parse output script in d822c89b6b20539bb1ccffa1a3d9c4fbf60d6a65c836eb9c25fc7428fe47ee50:0: opcode OP_DATA_1 requires 2 bytes, but script only has 1 remaining
[INF] LNWL: Recovered addresses from blocks 2130208-2132207


Tolerances should have helpeful names

These can be gotten using the SetNameGet function of APT

SetNameGet has been implemented


MIT license

Hi,
I would like to use your awesome icons on some MIT licensed projects.
It seems that your license is MIT only for CSS, SCSS & LESS files, but not for the webfont.
I think that means that we can use the webfont on commercial templates on template selling sites but can't use it on open source MIT licensed projects.
Any plan on adding the MIT license to the webfont too?
Regards.

Could you please clarify the license on the fonts?
Thank you,


does not work on honor/huawei

config:
new ConfettiManager(this, this, source, container)                 .setEmissionDuration(ConfettiManager.INFINITE_DURATION)                 .setTTL(CONFETTI_LIFE_DURATION_MS)                 .setVelocityX(0, velocitySlow)                 .setVelocityY(velocityNormal, velocitySlow)                 .setRotationalVelocity(ROTATIONAL_VELOCITY, ROTATIONAL_DEVIATION)                 .setEmissionRate(CONFETTI_PER_SECOND);
works on samsung/asus/etc. but shows nothing on honor 8x (android 9, JSN-L22) and huawei p20 pro (android 9, CLT-L29).
I've got similar issue with Huawei devices - there is not working rotation animation.
For this case I have to use such workaround:
DeviceHelper.isHuawei() ?                         ObjectAnimator.ofFloat(guestAvatarImage, View.ALPHA, 1, 0) :                         ObjectAnimator.ofFloat(guestAvatarImage, View.ROTATION_Y, 0, 90);
public static boolean isHuawei() {         return "HUAWEI".equals(Build.MANUFACTURER.toUpperCase());     }

I was having the same problem, even the CommonConfetti wasn’t  working.
Try using a Handler().postDelayed() with 500/1000 ms, it helped me.


this should be the admin user

when editing users in my app, i get an error:
A NoMethodError occurred in users#update:

undefined method `is_not_root?' for #<User:0x007fd1fc70ec68>
app/middlewares/cookie_domain.rb:11:in `call'

i think that the admin_user instance should be checked, not the user model that will be updated.

thx


Build Compatbility

XAML Builds have been re-enabled in 2018 Update 2, see here: https://docs.microsoft.com/en-us/visualstudio/releasenotes/tfs2018-update2#build-and-release

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e6b0a421-bc31-fec9-fab1-aea1a7d6cebf
Version Independent ID: 584d306a-6f5c-d197-ed32-c246463bdb14
Content: Changes to required for TFS 2018 - TFS 2018
Content Source: docs/tfs-server/requirement-changes-tfs16.md
Service: unspecified
Product: devops-server
GitHub Login: @elbatk
Microsoft Alias: elbatk

Ah, yes, you're right @xdaDaveShaw , I'll make this apparent in the docs, thank you!


Use consistent versions for Commons Collections 4 and Google Protocol Buffers

Rationale
We don't want conflicting versions of libraries across modules.
https://teamcity.labkey.org/buildConfiguration/LabKey_Trunk_Premium_VerifyDependencies/1145949?
Related Pull Requests

https://github.com/LabKey/DiscvrLabKeyModules/pull/65

Changes

Consolidate on Commons Collections 4.2 and Google Protocol Buffers 3.12.2

I closed the PR and deleted the branch from DiscvrLabKeyModules since it doesn't seem to be needed with this change. Let me know if I misinterpreted your suggestion.


Upgrading the Weblet Peertube to version 6.1 as recommended by the user

Which package/s are you suggesting this feature for?
Dashboard
Is your feature request related to a problem? Please describe
The client has deployed Peertube and is seeking to upgrade it to the latest version, specifically version 6.1.

Describe the solution you'd like
Upgrade to the latest version.

Some info:
peertube weblet versuib: https://github.com/threefoldtech/tfgrid-sdk-ts/blob/bdcdc46698d8ebfd6d267190265dc9983781b77e/packages/playground/src/weblets/tf_peertube.vue#L121
tf-images peertube version:
https://github.com/threefoldtech/tf-images/blob/d6f269a5f7bb810a514c3ea7a8fe898714ea1637/tfgrid3/peertube/Dockerfile#L9
on freeflow/peertube, we can see that peertube runs on 3.1:
https://github.com/freeflowuniverse/tf-peertube/blob/db04823c0f8c58e0e0491facaa8fe4a08f353d13/package.json#L4
last person to update was chocobozz:
https://github.com/freeflowuniverse/tf-peertube/commit/29553847dc0aec499cd790c90f5f35d0036dd2a1
Might simply need to put 6.1. Might surely need to update other dependencies.


Refactor SelectInstrument component with carousel

TODO:

fix display of "Previous" button
Get the "logo" to show up
remove outlines

Hey, I'm getting unexpected token on line 162 in Room when webpacking. Can't figure out the issue.

Yea, me too. Forgot about that. Will fix
Rafael Espinoza
www.rafaelespinoza.com http://www.rafaelespinoza.com
On Wed, Aug 17, 2016 at 9:25 PM, Frances Yang<EMAIL_ADDRESS>wrote:

Hey, I'm getting unexpected token on line 162 in Room when webpacking.
Can't figure out the issue.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/ColossalBubble/tbd/pull/60#issuecomment-240620703,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AE-GIEZZyKf9McxbKbksqN96HEJC0HMaks5qg961gaJpZM4JnBbk
.

In general, it looks ok. Can you add a prev button? What's the issue with the start button?

Using #64 instead


Proto encoding for Type, FieldType, Schema

A lot of proto decoding functionality already exists, but most encoding functionality is missing. We're going to need to be able to encode lots of things to proto in order to serialise PolicyConstraints properly. This is the first step in adding that proto encoding, starting at the bottom of the stack (types) and working upwards.

/gcbrun

GCB passing now, PTAL


[Timeline] Introduce new component

Documentation page: https://deploy-preview-21331--material-ui.netlify.app/components/timeline/

Great component. Can you make a file upload component like React-Filepond ?

A new thought, should we make the component names denser, turning
<Timeline>
  <TimelineItem>
    <TimelineItemTail />
    <TimelineItemDot />
    <TimelineItemContent>Code</TimelineItemContent>
  </TimelineItem>
</Timeline>

into?
<Timeline>
  <TimelineItem>
    <TimelineTail />
    <TimelineDot />
    <TimelineContent>Code</TimelineContent>
  </TimelineItem>
</Timeline>

@oliviertassinari here is how the API looks at the moment - I have denser component names, and additional component around the Dot and the Connector (without this the overrides for aligning the gap between the dot and the connector is becoming overwhelming..):
<TimelineItem>
  <TimelineSeparator>
    <TimelineDot />
    <TimelineConnector />
  </TimelineSeparator>
  <TimelineContent>Code</TimelineContent>
</TimelineItem>

My only issue is that all components have a component prop. Since we don't have any documented use case I'd remove it for now. Especially since the default has semantics (e.g. ul in Timeline). It's always easier to add a prop later or maybe find a better implementation for a specific use case than removing a prop because it's used wrong.

@eps1lon thanks for the review, removed component props.

Noticed this on a smaller screen:

Looks like it happens due to the limited space available with all 3 next to each other - demo displays weirdly on mobile too.

Noticed this on a smaller screen
Looks like it happens due to the limited space available with all 3 next to each other - demo displays weirdly on mobile too.

@joshwooding I updated that example to only contain the alternating Timeline, checked on mobile, looks good now..

@mnajdova I have rebased on the master branch to benefit from #21374 (fix the unit test).

@mnajdova Great job 🙏

@mnajdova Great work 🎉

wow, that is cool, it's just what I needed to my webapp, but had to develop it  by myself see it's use case: HERE

Nice. Gives me a few potential enhancement ideas...

A size prop.
Make the separator gap optional.
Make the dot elevation optional (off by default?). It looks okay(ish) on the colored dots, but less so on the grey, and is inconsistent with the separator.


how to write a wrapper for TopLevelVariableDeclaration?

for example, globalInt is a TopLevel const. how to write a wrapper for it?
globalInt is define in dart. how to use it in eval?
const int globalInt = 111;

Currently you can only do it with addSource, like here.
If you need to specify the value externally via the bridge, you could use a bridged top-level function:
class MyPlugin implements EvalPlugin {
  // ...

  @override
  void configureForCompile(BridgeDeclarationRegistry registry) {
    registry.defineBridgeTopLevelFunction(BridgeFunctionDeclaration(
      'package:myapp/functions.dart',
      'getObject',
      BridgeFunctionDef(
          returns: BridgeTypeAnnotation(BridgeTypeRef(CoreTypes.object)), params: [])
    ));

    registry.addSource(DartSource('package:myapp/functions.dart', '''
      const globalObject = getObject();
    '''));
  }

  @override
  void configureForRuntime(Runtime runtime) {
    runtime.registerBridgeFunc('package:myapp/functions.dart', 'getObject',
        (runtime, target, args) => $Object(getObject()));
  }
}

Currently you can only do it with addSource, like here.
If you need to specify the value externally via the bridge, you could use a bridged top-level function:
class MyPlugin implements EvalPlugin {
  // ...

  @override
  void configureForCompile(BridgeDeclarationRegistry registry) {
    registry.defineBridgeTopLevelFunction(BridgeFunctionDeclaration(
      'package:myapp/functions.dart',
      'getObject',
      BridgeFunctionDef(
          returns: BridgeTypeAnnotation(BridgeTypeRef(CoreTypes.object)), params: [])
    ));

    registry.addSource(DartSource('package:myapp/functions.dart', '''
      const globalObject = getObject();
    '''));
  }

  @override
  void configureForRuntime(Runtime runtime) {
    runtime.registerBridgeFunc('package:myapp/functions.dart', 'getObject',
        (runtime, target, args) => $Object(getObject()));
  }
}

thank you so much!!!


Can this be used for time parameterization?

I love the work you've done with both this project and Kinematic_Constrained_Trajectory. I was wondering if any of your work can solve the time parametrization problem? I ran your Matlab code and read through some of it, and it seems you always specify the time interval of your waypoints or max time allowed. Is this a constraint that must be pre-calculated? I'm trying to find a solution for this problem.
Thanks!

Thanks! Your research looks pretty neat as well. Will you be at ICRA 2016?
I'm not sure that I totally understand your question, but I will do my best. Let me know if I've answered it, or if you were intending something else.
The Kinematic_Constrained_Trajectory project solves a very specific problem, and the time way-points must be specified ahead of time. This is done so that the resulting optimization problem is a quadratic program, which can practically be solved in real time.
This project (TrajOpt), handles a general single-phase continuous-time trajectory optimization problem.   The initial and final time are both decision variables. I constrain both of them in most of my example problems, but this need not be the case. For example, you could set the initial time to be zero, and then allow the final time to be any time after the initial time. The optimization will scale the intermediate points accordingly. As of now TrajOpt requires a uniform mesh for it's direct collocation and shooting methods, but it would be a relatively easy change to allow the user to define an arbitrary grid between the initial and final times.
In theory, it is possible to let the time for every grid point be a decision variable. Unfortunately this will result in a badly behaved non-linear program. This I learned from personal experience, before I read it in Bett's book.
They way to get better locations for grid-points is by iteratively solving the trajectory optimization: start with a simple mesh (perhaps uniform, like I use in TrajOpt), and then do an error analysis on the solution to determine where to add grid points, and then solve again on a new grid. This is done automatically in GPOPS II and PSOPT. I would love to implement such a feature in TrajOpt, and have started laying some of the groundwork for doing so. It will probably be some time though before I get to it.
Let me know if this answered your question, or if any of the above does not make sense.
Matt

Hi Matthew,
I do not believe I will be at ICRA this year, sadly. Thank you for this detailed response on how TrajOpt works.  I ended up going with another library to solve my problem since it more directly solved the TOPP problem (Time-Optimal Path Parametrization) - I used the implementation from this paper:
 A General, Fast, and Robust Implementation of the Time-Optimal Path Parameterization Algorithm
Dave

Dave,
That paper seems like a much better choice for your application. I had not heard of the TOPP problem before. I skimmed through the paper, and I think that such a problem might be able to be solved in TrajOpt, but certainly not as efficiently as their methods would be. Thanks for sharing!
Matt


Enforce TLSv1 with curl breaks strong protocols (TLSv1.1 / TLSv1.2)

Since https://github.com/fguillot/picoFeed/commit/31a9e89d2acf0bc2d9910c3d949a625e505563b7 curl is forced to use TLSv1. If you try to load feeds from servers that only offer stronger protocols (TLSv1.1 / TLSv1.2) you get a "SSL-certificate invalid" error.
Preventing SSLv3 is a good thing. But preventing strong protocols at the same time is really bad. That's why I think this should be reverted. SSLv3 is disabled in cURL by default since 7.39.0 anyway.

Are you sure this is what is causing the invalid cert error? The value is CURL_SSLVERSION_TLSv1 which basically allows all TLS 1.x versions

See http://php.net/manual/en/function.curl-setopt.php

Also see http://curl.haxx.se/libcurl/c/CURLOPT_SSLVERSION.html

First of all, please provide a link to the failing feed. Otherwise it is impossible for others to reproduce and understand the issue.
It sounds more or less like a certificate validation error. Hard to say without any knowledge about your environment.

which php version do you use?
on which operating system?
is it you own server or is it provided by a hosting provider?
which hosting provider?

Maybe http://stackoverflow.com/a/16495053 solves your issue.

SSLv3 is disabled in cURL by default since 7.39.0 anyway.

At least the curl version bundled with php for windows is way older than 7.39.0. Furthermore you should take into account, that picofeed still supports php 5.3.

BTW: @mkresin @fguillot  at some point in time we need to think about dropping 5.3 support. http://www.zend.com/en/support-center/support/php-long-term-support shows that there is no security support for 5.3 and 5.4 security support will be dropped in 2 days.

Maybe branch it out into a separate branch for 5.3, create a release and then only backport security fixes (if you want to do that ofc)

As long as we do not rely on a feature of php 5.4+ I would prefer to keep 5.3 support. It's more or less the task of the server admins to update to a recent php version first, since their servers a vulnerable.
Especially if I have a look at the statistics from http://w3techs.com/technologies/details/pl-php/5/all and https://wordpress.org/about/stats/, I doubt that it's the right time to drop php 5.3 support.
Last but not least the used php version of the mayor linux distributions:

Debian 6 LTS (EOL February 2016): 5.3.3
Ubuntu 12.04 (EOL April 2017):  5.3.10
CentOS 6.6  (EOL Q2 2017/Nov. 2020): 5.3.3

Dropping the 5.3 support, because its not longer maintained, seams to be a no go for me.

which php version do you use?

5.4.36

on which operating system?

Debian Wheezy

is it you own server or is it provided by a hosting provider?

own server

which hosting provider?

First of all, please provide a link to the failing feed.

https://blog.fefe.de/rss.xml (results of ssllabs.com)
Testscript:
<?php

for($i=0;$i<=6;$i++) {
        if($i==2) continue;

        $ch = curl_init("https://blog.fefe.de");
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
        curl_setopt($ch, CURLOPT_USERAGENT, 'CURL_SSLVERSION '.$i);
        curl_setopt($ch, CURLOPT_SSLVERSION, $i);

        if(curl_exec($ch) === false)
                echo $i.': ' . curl_error($ch);
        else
                echo $i.': works';

        echo "\n";
        curl_close($ch);
}

Output:
0: works
1: error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure
3: error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure
4: works
5: works
6: works

It sounds more or less like a certificate validation error.

You will get SSL certificate problem: unable to get local issuer certificate instead of works if you don't trust the root certificate of cacert.org. I hope the problem is more clear now.

blog.fefe.de is a really special case. First of all, the used certificate is issued for ptrace.fefe.de. This should never be accepted as a valid certificate for blog.fefe.de.
According to ssllabs.com, fefe has disabled all protocols except TLSv1.2 for his blog.
If I run the script extended with curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); on ubuntu 14.04 (php 5.5.9), I get the following (expected) output:
0: works
1: works
3: error:1409442E:SSL routines:SSL3_READ_BYTES:tlsv1 alert protocol version
4: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version
5: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version
6: works

Debian Wheezy ships with curl/libcurl 7.26.0-1, which shouldn't be aware of the CURLOPT_SSLVERSION 4-6 according to http://curl.haxx.se/libcurl/c/CURLOPT_SSLVERSION.html. I've no clue why curl_init() doesn't return false for you. Maybe I'm completely wrong.
Have you confirmed that php/curl returns html content for you if you use 0, 4-6?

the used certificate is issued for ptrace.fefe.de. This should never be accepted as a valid certificate for blog.fefe.de

I think as blog.fefe.de is an Altnernative Subject Name in the certificate it should be accepted.

Debian Wheezy ships with curl/libcurl 7.26.0-1, which shouldn't be aware of the CURLOPT_SSLVERSION 4-6

Yeah, your right, I didn't notice that. I think it handles every invalid entry as 0. That explains my output for 4 and 5.

Have you confirmed that php/curl returns html content for you if you use 0, 4-6?

With these values I get the html content as expacted.

My bad! I didn't checked the subjectAltNames.

That explains my output for 4 and 5.

As mentioned, 6 should work as well.
It would be really helpful if you can do the packet capture and check the used TLS version with the different CURLOPT_SSLVERSION values.

Arghh typo. The CURLOPT_SSLVERSION 6 should not work.

It would be really helpful if you can do the packet capture and check the used TLS version

I don't know what you expect here. The server only supports TLSv1.2, so for every "working" connection it is TLSv1.2 of course.

6 should not work

As I said, every undefined entry is ignored and default (0) is chosen instead. That's why it is working for me when using 4-6. Have a look at the sources.

Ok, let's have a look at the sources:
cURL 7.33: https://github.com/bagder/curl/blob/curl-7_33_0/lib/ssluse.c#L1431
cURL 7.34: https://github.com/bagder/curl/blob/curl-7_34_0/lib/ssluse.c#L1431
The problem is obviously the different behaviour when choosing 1 (CURL_SSLVERSION_TLSv1). With versions up to 7.33 cURL calls TLSv1_client_method(), which enforces TLSv1. Since 7.34, choosing the same parameter 1 (CURL_SSLVERSION_TLSv1), SSLv23_client_method() is called instead, which allows SSLv3, TLSv1, TLSv1.1 and TLSv1.2.
So, there is another problem for people with cURL < 7.24 when choosing CURLOPT_SSLVERSION = 1: Even if a server supports TLSv1.1 or TLSv1.2, the less secure protocol TLSv1 will be enforced.
With the same parameter you get a completely different output resulting in less secure connections or even errors with older cURL versions. As Wheezy is still the stable Debian version and a very popular server OS, I still recommend to revert this at least until Jessie is out.
In my opinion it would be more important to prevent ciphers suites using things like RC4 or MD5 with the help of CURLOPT_SSL_CIPHER_LIST.

I don't know what you expect here.

I want to know which TLS Version is used if you set CURLOPT_SSLVERSION to 1, since I'm unable to reproduce your issue either on my development system or with the windows php binaries I had tested so far.
PHP 5.3.26 (curl 7.30.0/OpenSSL 0.9.8y):
0: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version [TLSv1.0]
1: error:1409442E:SSL routines:SSL3_READ_BYTES:tlsv1 alert protocol version [TLSv1.0]
3: error:1409442E:SSL routines:SSL3_READ_BYTES:tlsv1 alert protocol version [SSLv3]
4: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version [TLSv1.0]
5: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version [TLSv1.0]
6: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version [TLSv1.0]

PHP 5.4.38 (curl 7.40.0/OpenSSL 0.9.8zd):
0: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version [TLSv1.0]
1: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version [TLSv1.0]
3: error:1409442E:SSL routines:SSL3_READ_BYTES:tlsv1 alert protocol version [SSLv3]
4: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version [TLSv1.0]
5: Unsupported SSL protocol version
6: Unsupported SSL protocol version

PHP 5.5.22 (curl 7.40.0/OpenSSL 1.0.1k):
0: works [TLSv1.2]
1: works [TLSv1.2]
3: error:1409442E:SSL routines:SSL3_READ_BYTES:tlsv1 alert protocol version [SSLv3]
4: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version [TLSv1.0]
5: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version [TLSv1.1]
6: works [TLSv1.2]

PHP 5.6.6 (curl 7.40.0/OpenSSL 1.0.1k)
0: works [TLSv1.2]
1: works [TLSv1.2]
3: error:1409442E:SSL routines:SSL3_READ_BYTES:tlsv1 alert protocol version [SSLv3]
4: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version [TLSv1.0]
5: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version [TLSv1.1]
6: works [TLSv1.2]

The output of the "official" windows php binaries show why it's a really bad idea to enforce TLS 1.2 these days. Openssl > 0.9.8 is required for TLS 1.1 and TLS 1.2. But it isn't as widely deployed as I would expect it. For the sake of completeness, the up-to-date centos 6 of my hoster can not establish TLS 1.2 connections at all.

Ok, let's have a look at the sources:
cURL 7.33: https://github.com/bagder/curl/blob/curl-7_33_0/lib/ssluse.c#L1431
cURL 7.34: https://github.com/bagder/curl/blob/curl-7_34_0/lib/ssluse.c#L1431

Thanks for finding this! I've set up a Debian Wheezy and could reproduce the issue. Finally the links and your explanation is the answer to what happens here. Now we could think of a proper workaround.

So, there is another problem for people with cURL < 7.24

I assume you are talking about 7.34.

In my opinion it would be more important to prevent ciphers suites using things like RC4 or MD5 with the help of CURLOPT_SSL_CIPHER_LIST.

If enforcing TLS 1.x already shows compatibility issues like this one, I would bet that dropping ciphers suites is more or less a minefield.
@bastei @Raydiation @fguillot
I can imagine the following workarounds:

revert the commit and being prone to the poddle attack (IMHO the worst idea)

Set the CURLOPT_SSLVERSIONdependent on the curl version. This way we could limit the vulnerability to the poddle attack to cURL < 7.34.

Fallback to CURL_SSLVERSION_DEFAULT if curl_error($ch) returns something like error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure (what is the appropiate error to check for?) and the used curl version is < 7.34.

Check the used SSL/TLS version after getting a response and stop if it's SSLv3 (if anyone can figure out how this can be done!)

Very difficult and we need proper TLS for iframes, otherwise we'll get problems with mixed active content.
Adding proper ciphers or rather blacklisting insecure ones is very important I think. Either we are secure or we aren't.
IMHO we should be maximum secure by default, so basically require TLS 1.2 and secure ciphers. If the system does not support these we should throw a special kind of exception to allow devs to react to these things.
Sure there are feeds and servers that are too old to fullfill these requirements, so there should be config options to cater to these needs. IIRC we already offer to turn off SSL cert validation. We would probably also need an option to set the required tls protocols and ciphers to insecure versions.
As for the ownCloud News app: I'll probably require a secure curl version and secure ciphers, basically ignore old setups which is basically only Debian Wheezy. For Wheezy I'll add a guide to enable repositories with recent and secure versions, probably available in the backports repo. @bastei could help me with this.

Ah right, https iframes are handled by the browser :P
we should still enforce maximum security by default.

we should still enforce maximum security by default.

If we would have control over client and server I would agree. But it isn't the case. In other words, a feed library that only works with a minority of the feeds/server isn't worth the code. We have to find a balance between security and interoperability. I expect that known totally doomed stuff like SSLv3 and RC4 is blacklisted by default. Everything else should be optional.

IIRC we already offer to turn off SSL cert validation

Nope. Because people tend to disable validation instead of fixing their config.

I'll probably require a secure curl version and secure ciphers, basically ignore old setups which is basically only Debian Wheezy

As outlined before, it isn't only debian wheezy. I'm at least aware of debian squeeze and centos 6. I known, it is old stuff but it's still maintained and used.
What about shared hoster, where you don't have the possibility to update any binaries? At least Miniflux is ideal to be used with shared hoster. It simply works and the users don't have to be care about the server administration.
In the end it is a simple question: For whom you're writing software. Is the answer only for you, raise the requirements as you like. Is the answer for the users, you should carefully consider what kind of requirements do you introduce.
It's the same as with crypto stuff like gnupg. Great software, good crypto but obviously way to complex for even experienced users. The result: I can send encrypted stuff, but I've no one who could receive encrypted stuff.

This is fine, miniflux can still support old installations and hosting just fine. You'd just have to add something like this to your config:
$config->setTLSVersion(SSL::TLS1x)->whitelistCiphers([Cipher::RC4, Cipher::MD5]);

I'm not against compatibility, all I'm saying is: be secure out of the box and let devs tweak things to their likings.
It is always hard to follow updates and an update should not rip holes into your security because you forgot to turn on a more secure mode. It is better that something fails and the dev can easily fix it than security holes going unnoticed

You could even add that to the readme like: add these config parameters if you want to support debian wheezy etc.

It's more or less a question of the philosophy.
I agree that awfully broken stuff like RC4 should be irreversible blacklisted by default (similar to stripping style and script tags from content). But for anything else, it's up to the developers to blacklist stuff:
$config->setTLSVersion(SSL::TLS10)->disable();

Being maximum secure but literally non-functional without further interaction is a no go for me ;-).

Well, this is far from non-functional, this affects just one stable distro that will get a new stable release very soon ;)
It's less of a philosophy question and more a question of good API design. Also most PHP devs have no  clue about security and we need to keep that in mind.

Well, this is far from non-functional, this affects just one stable distro that will get a new stable release very soon ;)

It's not about stable releases, it's about supported releases. I don't know how familiar you are with server administration but at least I don't (and I don't know any sysadmin who) update the server os without any need, as long as the current version is still supported. You know, never touch a running system. Otherwise you have to check the used packages for new/changes features, updating configuration, do compatibility tests and updating documentation and so on.
Just to illustrate (my) the real world situation:

hosteurope.de, which is one of the biggest german hosting providers, uses debian squeeze LTS for their webpack product
uberspace.de (one of the coolest provider ever btw.) uses centos 6 on their servers

Now I count three widely used and still supported distribution that have their problems with TLSv1.2 for example. That's what I'm talking about.
Disclamer ;-): This isn't meant as offence. Hopefully it does not read as.

I'm late on this thread, but for me this library must work on all active LTS distros.
The solution 2 seems to be the most simple to implement?

Set the CURLOPT_SSLVERSION dependent on the curl version. This way we could limit the vulnerability to the poddle attack to cURL < 7.34.

Check the used SSL/TLS version after getting a response and stop if it's SSLv3

That would be the best solution. Unfortunately I'm not aware of any method how to do that... unless PHP 5.6 is used.

If enforcing TLS 1.x already shows compatibility issues like this one, I would bet that dropping ciphers suites is more or less a minefield.

I think you are wrong here. If I don't misinterpret the curl sources it passes the ciphers to openssl without modifying anything. So in contrast to the SSL protocols you have the full power over the parameters here.
Therefore, we could think about something like this as a potential workaround:
<?php
function curl($sslver, $ciphers) {
        $ch = curl_init("https://example.com");
        curl_setopt($ch, CURLOPT_NOBODY, true);

        curl_setopt($ch, CURLOPT_SSLVERSION, $sslver);
        curl_setopt($ch, CURLOPT_SSL_CIPHER_LIST, $ciphers);

        $result = curl_exec($ch);
        curl_close($ch);
        return ($result === false) ? false : true;
}

$goodcurl = (curl_version()['version_number'] >= 467456); // curl >= 7.34

if ( curl(0, "DEFAULT:!SSLv2:!SSLv3") ) {
        echo "connected via TLSv1.2\n";
        exit;
}
if( curl(1, "HIGH:!aNULL:!eNULL:!EXPORT:!DSS:!DES:!RC4:!3DES:!MD5:!PSK") ) {
        if($goodcurl)   echo "connected via TLSv1, TLSv1.1 or TLSv1.2\n";
        else            echo "connected via TLSv1\n";
        exit;
}
if( !$goodcurl && curl(0, "HIGH:!aNULL:!eNULL:!EXPORT:!DSS:!DES:!RC4:!3DES:!MD5:!PSK") ) {
        echo "certainly connected via SSLv3, maybe TLSv1.1. TLSv1.2 is very unlikely\n";
        exit;
}
if( curl(0, "DEFAULT") ) {
        echo "connected via SSLv3 or with weak ciphers\n";
        exit;
}
echo "Better don't contact that server...";

Explanation: To get the best encryption, we try to connect with gradually decreasing encryption strength:

Only TLSv1.2 ciphers. This enforces the TLSv1.2 protocol.

Strong ciphers and TLSv1, TLSv1.1 and TLSv1.2 for curl >= 7.34. Old versions will be forced to TLSv1. We can't test TLSv1.1 for them, but I think most of the servers with TLSv1.1 have support for TLSv1.2, too (and thus the first test would probably be successful).

Do an extra test for old curl versions, all protocols allowed, but only good ciphers. There is the possibility for combinations that don't fall under 1. and 2. but that work here, for example servers with TLSv1.2 protocol only but no TLSv1.2 ciphers or TLSv1.1 protocol only, even if I think these scenarios are very unlikely. The worst case here is SSLv3, but with good ciphers (which is absolutely not unlikely).

Default protocol, default ciphers. This is a fallback where only SSLv3 or bad ciphers are left. You can leave this out and throw an error.

Note that the main trick is using the TLSv1.2 ciphers to test TLSv1.2 support. It is also possible to use "DEFAULT" ciphers in 2. and then leave 3 if you don't want to mess around with ciphers too much.

I think you are wrong here. If I don't misinterpret the curl sources it passes the ciphers to openssl ...

What I mean with minefield is that libcurl isn't necessary compiled against openssl. On the mentioned contos 6 host, libcurl is compiled against NSS.
curl
cURL support 	enabled
cURL Information 	7.19.7
Age 	3
Features
AsynchDNS 	No
CharConv 	No
Debug 	No
GSS-Negotiate 	Yes
IDN 	Yes
IPv6 	Yes
krb4 	No
Largefile 	Yes
libz 	Yes
NTLM 	Yes
SPNEGO 	No
SSL 	Yes
SSPI 	No
Protocols 	tftp, ftp, telnet, dict, ldap, ldaps, http, file, https, ftps, scp, sftp
Host 	x86_64-redhat-linux-gnu
SSL Version 	NSS/<IP_ADDRESS> Basic ECC
ZLib Version 	1.2.3
libSSH Version 	libssh2/1.4.2

Does it work the same way here? How does the other supported ssl libraries (http://curl.haxx.se/docs/ssl-compared.html) behave.

Fallback to CURL_SSLVERSION_DEFAULT if curl_error($ch) returns something like error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure (what is the appropiate error to check for?) and the used curl version is < 7.34.

Answering my own question: Won't work. The errormessage depends on the used ssl library.

After reading your reply once again and under the premise that all by libcurl supported libraries behave the same (which is questionable according to http://stackoverflow.com/a/26764845), the following:
curl_setopt($ch, CURLOPT_SSLVERSION, 1)

should be equivalent to
curl_setopt($ch, CURLOPT_SSL_CIPHER_LIST, "DEFAULT:!SSLv2:!SSLv3")

Unfortunately I'm not aware of any method how to do that... unless PHP 5.6 is used.

And even this shouldn't work with curl. As far as I can see it's for the the Stream Context http client adapter and not for cURL.
But that leads me to another question: How to enforce TLS 1.x if the Stream Context http client adapter is used. As far as I know, openssl is used here always.
The ciphers options of http://php.net/manual/de/context.ssl.php looks suitable. Can anyone confirm that "DEFAULT:!SSLv2:!SSLv3" should do the trick?

What I mean with minefield is that libcurl isn't necessary compiled against openssl.

Ok, could be a problem... What is your output for print_r(curl_version()); ?

Can anyone confirm that "DEFAULT:!SSLv2:!SSLv3" should do the trick?

No, "DEFAULT:!SSLv2:!SSLv3" is the same as "TLSv1.2:!eNULL:!aNULL".
See the NOTES-section of https://www.openssl.org/docs/ssl/SSL_CIPHER_get_name.html#NOTES:

The TLSv1.0 ciphers are flagged with SSLv3. No new ciphers were added by TLSv1.1.

Basically, SSLv3 ciphers are all DEFAULT ciphers minus the TLSv1.2 ciphers. Nevertheless, you can connect via TLSv1.2 protocol but with SSLv3 ciphers.

How to enforce TLS 1.x if the Stream Context http client adapter is used.

I tried it and failed. It is the same problem as with curl for me: Using STREAM_CRYPTO_METHOD_TLS_CLIENT enfores TLSv1, STREAM_CRYPTO_METHOD_SSLv23_CLIENT allows SSLv3.

What is your output for print_r(curl_version());

Array
(
    [version_number] => 463623
    [age] => 3
    [features] => 1597
    [ssl_version_number] => 0
    [version] => 7.19.7
    [host] => x86_64-redhat-linux-gnu
    [ssl_version] => NSS/<IP_ADDRESS> Basic ECC
    [libz_version] => 1.2.3
    [protocols] => Array
        (
            [0] => tftp
            [1] => ftp
            [2] => telnet
            [3] => dict
            [4] => ldap
            [5] => ldaps
            [6] => http
            [7] => file
            [8] => https
            [9] => ftps
            [10] => scp
            [11] => sftp
        )

No, "DEFAULT:!SSLv2:!SSLv3" is the same as "TLSv1.2:!eNULL:!aNULL".

Thanks for the explanation. As you might already know, I've mixed up protocol and protocol cypher suits. My bad.

I tried it and failed. It is the same problem as with curl for me: Using STREAM_CRYPTO_METHOD_TLS_CLIENT enfores TLSv1, STREAM_CRYPTO_METHOD_SSLv23_CLIENT allows SSLv3.

Okay, no need for me to try it again.

I've opened https://github.com/fguillot/picoFeed/pull/104 which disables SSLv3 for known good libcurl version.
If anyone has better solution, feel free to send a superseding patch.

@bastei can you confirm that the issue is solved by the pull-request of @mkresin ?

@fguillot @mkresin
It works. Thanks for the fast fix and good discussion! :+1:


Implement byte padding for Elliptic Curve point encoding as fix for #173

Description
This change implements byte padding of Elliptic Curve point byte representations to match the required field size.
Checklist

[X ] All tests are passing
[ X] New tests were created to address changes in pr (and tests are passing)
[X ] Updated README and/or documentation, if necessary

Ok, I think this is clean enough to leave as-is, unless @fszlin has any suggestions!

I have added attributes to JsonWebKey and it's deriviatives to enforce the serialization order of properties. One user was getting an incorrect Account Key thumbprint during challenges as the key properties were serializing in a different (non-alphabetical) order resulting in a different json string being hashed.


fixes #1127 -- pipchecker can't find some packages

https://github.com/django-extensions/django-extensions/issues/1127

Coverage remained the same at 43.761% when pulling b9c3deaaaf21b208e343f0cc486d1e1fba276e49 on picturedots:patch-1 into 630a415612f411f1f007ec3e009a4d7161330113 on django-extensions:master.

Coverage remained the same at 43.761% when pulling b9c3deaaaf21b208e343f0cc486d1e1fba276e49 on picturedots:patch-1 into 630a415612f411f1f007ec3e009a4d7161330113 on django-extensions:master.

Could you please fix the style error ?
What are the odds that by replacing - with _ we match a different package ? (or does pypi protect against that ?)

Coverage remained the same at 43.761% when pulling 9b1292f5f3703301c9b67f5cf9b8183ed0de4c1a on picturedots:patch-1 into 630a415612f411f1f007ec3e009a4d7161330113 on django-extensions:master.

Coverage remained the same at 43.761% when pulling 9b1292f5f3703301c9b67f5cf9b8183ed0de4c1a on picturedots:patch-1 into 630a415612f411f1f007ec3e009a4d7161330113 on django-extensions:master.

Thanks !
Guess pipchecker in itself will not create a security thread. Thinking if a user might be tempted to install/upgrade to a typo-squatted named version if that's detected as a newer version.


[WIP]Add transaction fees for methods in contracts

fix #783

:x: Build AElf <IP_ADDRESS>21 failed (commit https://github.com/AElfProject/AElf/commit/0ac6bf7cb7 by @xiangxingxing)

Codecov Report

Merging #2052 into dev will decrease coverage by 0.18%.
The diff coverage is 30.32%.

@@            Coverage Diff            @@
##              dev   #2052      +/-   ##
=========================================
- Coverage   84.99%   84.8%   -0.19%
=========================================
  Files         599     612      +13
  Lines       21990   22145     +155
=========================================
+ Hits        18690   18781      +91
- Misses       3300    3364      +64

Impacted Files
Coverage Δ

...ntract/AElf.Contracts.Treasury/TreasuryContract.cs
98.63% <ø> (ø)
:arrow_up:

...Contracts.TokenConverter/TokenConverterContract.cs
100% <ø> (ø)
:arrow_up:

...ntract/AElf.Contracts.Economic/EconomicContract.cs
99.07% <ø> (ø)
:arrow_up:

...ntract/AElf.Contracts.Genesis/BasicContractZero.cs
88.11% <ø> (-7.93%)
:arrow_down:

...nfigurationContract_ACS1_TransactionFeeProvider.cs
0% <0%> (ø)

...t/AElf.Contracts.Economic/EconomicContractState.cs
100% <100%> (ø)
:arrow_up:

...AElf.Contracts.Configuration/ConfigurationState.cs
100% <100%> (ø)
:arrow_up:

...t/AElf.Contracts.Treasury/TreasuryContractState.cs
100% <100%> (ø)
:arrow_up:

...lf.Contracts.ParliamentAuth/ParliamentAuthState.cs
100% <100%> (ø)
:arrow_up:

...acts.TokenConverter/TokenConverterContractState.cs
100% <100%> (ø)
:arrow_up:

... and 37 more

:white_check_mark: Build AElf <IP_ADDRESS>43 completed (commit https://github.com/AElfProject/AElf/commit/308125bee4 by @xiangxingxing)

:white_check_mark: Build AElf <IP_ADDRESS>80 completed (commit https://github.com/AElfProject/AElf/commit/37d007aaa0 by @xiangxingxing)


[REEF-602] Treat .sln files as binary

JIRA:
REEF-602

Don't merge this yet, REEF-602 hasn't converged yet.

Replaced by #379

#379 did not conclude, reopening.

I've tested. I'll merge.


Update README to support failover

Purpose

Add changes to the README after supporting the siddhi operator to failover scenarios.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Buddhi Kothalawala seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Invesitgate and fix instances where an entire sentence ends up as the speaker name

sometimes an entire sentence ends up in the speaker column. Fix these so we are left with just the name

Should be fixed. @stephbuon check


Convert DuoTone Themes to Base16 Schemes

Hello,
There are a lot of very nice theme for the Atom IDE. I would love to try them somewhere else. Can we think about a way to use them as scheme for base16-buider?

You can discover the concept here (by @simurai).

Definitely something to consider in the future :smile:

I think @atelierbram used base16-builder to generate these: http://atelierbram.github.io/syntax-highlighting/duotones/

@simurai Thank you for the comment. @atelierbram made a great job. It will help a lot!

@simurai Just diff a bit and found this. PR is on the way!

@aloisdg Thanks for the compliments, it's flattering; but a few cautionary words on the PR which I hope will be helpful. The proof will be in trying to make these into light background themes, which can be an interesting academic exercise, but until now I only have been bothered with the dark background templates. It could be a tricky project trying to reconcile the differences in philosophy between DuoTone Themes and the Base16 themes, and to be honest, I question if it is doable without compromising one or the other, and in art there is no room for compromises.
So what I felt I had to do was to adept the (dark-background-)templates for Vim, Sublime Text, Prism and CodeMirror (the only "finished" themes) in order to get near the original, using the base16 template system and executable bash script as a convenient building tool. But which admittedly also leads me to explain my position in this issue here.
Here are some demos for Prism which can make the mapping of the color-values to base16-variables I used insightful. But, like I wrote, although bringing those schemes in here would probably get you more than halfway there (for most applications, but IMO not all the way there for all existing templates (for they where created with a different blueprint in mind).

@alexbooker @aloisdg :+1:  Love the new CLI; having played with it I have to say I am impressed by it's versatility and flexibility.
An update on my efforts of converting DuoTone Themes in light and dark variations. Bad news: wasn't really doable IMO with 16 color-variables (without it being a tangled mess), so no PR here. Good news: it still can be done with more color variables and generated with Base16 Builder. A proof of concept here for Prism with 32 colorvalues (not all used), but I hope it makes sense when seeing this demo. Am planning to take this further, for I think this has great potential, and wouldn't mind any feedback.

Thank you for your answer and the demo!
You Prism demo is, at least for me, completely usable. Great works you made their. If you can go further, I would love to see this :)
Just to be sure, you are not using the same color (or same colors with different shades) in both dark and light mode, aren't you?

@atelierbram Yes, thank you! I, too, would be very interested in seeing this. Good work so far :smile: :ok_hand:

Just to be sure, you are not using the same color (or same colors with different shades) in both dark and light mode, aren't you?

@aloisdg I guess I am, that is to say; the colors for the light mode map to different colorvalues but from the same scheme/colorscheme.yml file with 32 colorvalues declared (maybe could be 24, but this gives a bit more flexibility). But I hope hitting the "show colornames" button on the demo and switching between dark - and light mode will make this a bit more concrete and visual.
Maybe it's fair to say that my mental model of Simurai's great concept is (schematicaly) a bit like this: two colors with each two tonal scales ranging from dark to light in a saturated and de-saturated version. So objectively you have two hue values, but perceptively four colors in a dark to light scale. It's genius in it's simplicity for it almost guarantees harmonious colorschemes while at the same time giving just enough variation to make it visually interesting.
@alexbooker Have been thinking on calling it base32-duotones or duotones-base32 or something, but am undecided about it. On the one hand I want to avoid the confusion with the original project from @simurai , and confusion with base16 implementations, on the other hand, although this has great potential, I don't want to get overboard with this just yet, so yeah, need to think about that.

@atelierbram
I don't think I have problem with the idea of splitting schemes by brightness. We can apply the same model used for template. We would have db/schemes/{name}/{brightness}.yml. Most of the old schemes would be equals in both dark and light version (lot of duplicate here but it is the current choice made for template). What do you think of this alternative?

Will give it a day to wrap my mind around this idea. Right now, I think this will overcomplicate the base16 concept, and take away some clarity for Duotones, for it means two YML scheme files: db/schemes/{name}/dark.yml and db/schemes/{name}/light.yml, no? Somehow this doesn't sit quiet right with me yet ... what does @alexbooker think?

@atelierbram
You are right, It may not fit very well. Also It could be a full other project as you said before. (I like the name Base2 or BaseDuo ;).

@atelierbram, I value simplicity a lot. You are spot on when you say this "will overcomplicate the base16 concept, and take away some clarity for Duotones".
This is a tricky one :confounded:
All I can really say at this point is that I want to help you leverage the "core" as best I can. Maybe you can use the JavaScript API to build a separate tool. Another option would be to interface with command-line itself.

Thanks for the feedback, and yes, like with many things, the more you think about it, the more it seems obvious.
@aloisdg Your comment on naming was actually quite helpful in finding the right direction for picking something. So I settled on Base2Tone, which is less memorable then "Duotones", but the plural "s" to differentiate from "DuoTone Themes" was always more than a bit lame. And with "Base2" this loosely references Base16 and maybe also an association with "a build-tool", if this makes sense. So I'm happy enough with it, for it's also a fine excuse to create an animated SVG-logo with alternative spelling ("Base > Tone")  :smirk:
@alexbooker When I am far enough into the project to oversee the scope of it, maybe I will come back to you, that is, if - or when there will be a "Base2Tone-Builder" repo, if this is ever needed at all, I'm not sure. For now that will be overkill, and the command-line interface works fine with me. So I will just point to here for the dependencies, and let people install "Base16 Builder" globally on there machine from NPM, if they want to build there own scheme or template. Having those commands in executable bash-scripts, keeps it simple, but I 'm a low-fi creature :bug:, and don't know much about JavaScript.
Separating out into different repos for different apps, (like is being done with Base2Tone-prism) seems to be the right move at this time. Such a horizontal setup also allows linking to those repos from the main "Base2Tone" repo; with the advantage that this will not only have to be my own repos, but also potentially those from others, which might help keeping the project maintainable. So if you ever want to create a "Base2Tone-{app-name}", you are more then welcome and know where to find me to link this up from the main repo! :wink:

@atelierbram
Glad to be helpful for the name!
I think we can close this issue for now. What do you think @alexbooker?

@atelierbram I appreciate it a lot. I'm sure both myself and @aloisdg will be glad to help with any code relating to this project. Don't hesitate to get in touch, OK :smile:?
@aloisdg I don't think there's any harm in keeping it open for a while, do you?

@alexbooker of course we can. :smile:


Fix Duplicate Primary OCRs Bulk Job

Critical Changes
Changes
Issues Closed

Fixes #3063

New Metadata
Deleted Metadata

@smarovich, do we need to add more detail in the release note?


Not working on Andorid 11

Basic Information
Device type: Pixel 4
OS version: Android 11
EasyPermissions version: 3.0.0
The library works well until Android 11, where it doesn't show any runtime permission pop-up.

It's actually not a problem with the library. Check this link if you're in the same situation.


Added Dutch Translation

Btw I recommend you to use the intlib (if you are intersted and want help, just dm me on forum: 'Miniontoby' is the nick)

Thanks for adding the dutch translation! Regarding intlib - it says that it would be deprecated and should not be used by new mods, because Minetest has it's own Translation API now. The problem with it is, that it can only be used client side. The Documentation says there would be an option to do it serverside too, but I don't really understand it 😂 If you know how, I would really appreciate your help

intlib - Yes that's true that there is new system
Maybe we can have a chat at IRC, I am avaible at irc.ircforever.org #minetest (and #miniontoby is my own channel)
Nicknames: Miniontoby and MinionTest3

Miniontoby

Ok, then I'd rather use Discord. Heres my Discord: activivan#8135


Gh 60 remove dependencies for mapreduceclient

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.